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Preface 


Bienvenue  a Montreal  et  aED-MEDIA  2000! 

Whether  your  counting  system  indicates  that  we  have  begun  the  21st  century,  or  we  are  ending  the 
20th,  we  have  the  opportunity  at  this  historic  moment  to  look  backward  as  well  as  forward.  This 
12th  ED-MEDIA  conference  enables  us  to  view  the  growth  of  new  technologies  educadon  from 
the  perspective  of  a novel  idea  to  a major  element  in  our  educational  process.  This  ED-MEDIA 
conference  is  poised  to  join  the  ranks  of  the  successful  previous  11  conferences. 

The  Conference  is  organized  by  the  Association  for  the  Advancement  of  Computing  in  Education 
(AACE)  and  over  the  years  has  developed  a strong  international  following.  The  Conference  has 
grown  hand  in  hand  with  the  emergence  of  new  learning  technologies  and  now  represents  the 
premier  event  for  this  field  in  the  world.  This  year  s attendees  come  from  6 of  the  7 the  continents 
of  the  globe  and  represent  more  than  50  countries.  And  we  probably  would  have  participation  from 
Antarctica  had  there  not  been  prime  research  weather  there. 

The  interest  in  educational  media  around  the  globe  is  evidenced  by  the  number  of  submissions,  the 
extent  of  the  topics  covered  by  these  papers  both  long  and  short  as  well  as  proposals  for  demos 
and  panels.  We  received  over  600  proposals  of  full  and  short  papers,  workshops,  tutorials  and 
posters  from  nearly  50  different  countries.  Our  review  process,  which  itself  is  well  automated  with 
web-based  submissions,  reviews  and  responses,  usually  includes  3 referees  from  our  international 
panel  of  expert  reviewers  on  the  Program  Committee.  Those  submissions  selected  for  presentation 
represent  work  of  the  highest  quality.  In  addition,  we  are  extremely  proud  of  the  presentations 
scheduled  for  keynote  and  invited  talks  as  they  truly  represent  the  best  and  the  brightest  in  the 
history,  depth  and  breadth  of  our  discipline  as  well  as  the  international  aspect  of  our  conference.  We 
are  extremely  proud  to  have  a keynote  presenter,  for  the  first  time,  from  the  African  continent. 

You  may  wonder  what  Program  Chairs  do  during  those  dark  months  between  the  last  conference 
and  the  present  one.  We  have  had  the  opportunity  to  review  the  proposed  presentations,  work  with 
authors  to  hone  their  presentations,  help  develop  interesting  panels,  invite  provocative  keynoters 
and  of  course,  worry  about  details  so  that  your  experience  at  this  conference  and  with  these 
proceedings  wiU  provide  you  with  a worthwhile  overview  of  your  field. 

One  of  our  opportunities  is  to  review  the  entire  conference  as  it  emerges  and  to  do  a bit  of  research 
ourselves  into  the  nature  of  the  field  as  represented  by  your  submissions.  In  this  era  of  everything, 
we  expected  the  WWW  and  Internet  to  be  the  most  popular  terms.  A simple  keyword  search 
indicated  they  came  close  and  were  the  hands  down  winner  if  you  include  distance  education.  But 
other  trends  also  emerge.  The  concept  of  collaborative  learning  in  setting,  teamwork,  and  teaching 
style  is  a topic  of  interest.  While  prototype  and  application  development  endure  as  does 
programming,  new  areas  of  support  for  cognition  and  learning  styles  as  well  as  integration  of 
technological  support  across  the  curriculum  are  emerging  themes. 

Where  is  the  ED  in  EDMEDIA  now?  It  has  moved  from  being  the  target  for  ready-to-use  media  to 
becoming  a foundation  and  an  inspiration  in  designing  new  ones:  numbers  of  papers  base  their  work 
on  learning  and  instruction  theories  as  well  as  on  good  teaching  practices,  or  refer  to  them  for 
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evaluation.  We  see  that  interacting  has  replaced  using  technologies,  in  two  ways:  interacting  with 
other  people  over  computer  networks,  and  interacting  with  smart  software.  In  both  cases,  these 
smart  interactions  represent  meaningful  and  subtle  conversations  that  make  them  more 
interesting.  People  are  no  longer  users,  but  are  choosers  defined  by  the  roles  they  play  in  action: 
learners,  teachers,  managers,  etc.  Beyond  tools  and  infrastructures,  ED-MEDIA  2000  presents  all 
that  is  at  stake  with  technologies  in  education:  collaboration  or  commercialization,  equal  access  or 
disparities,  cultural  differences. 

The  best  papers  for  ED-MEDIA  2000  could  be  considered  those  that  received  a 4.5  (out  of  5.0)  or 
above  on  our  reviewer  rating  scheme.  From  among  those,  the  award  winning  papers  are  selected.  If 
we  look  to  them  as  a indication  of  where  our  field  is  situated  at  the  cusp  of  the  21st  century,  we  can 
note  that  educational  technology  researchers  can  be  found  around  the  world  and  well  represented  by 
both  men  and  women.  The  studies  documented  by  these  papers  are  well  founded  research 
methodologies  into  a variety  of  areas  including  an  adaptive  navigation  support  system  for  the  world 
wide  web,  techniques  for  annotation  of  learner  created  hyperlinks  or  trails,  the  use  of  agents  in 
teaching  and  learning,  and  an  investigation  into  the  learning  sequences  students  select  and  their 
relationship  to  learning  styles. 

It  is  incumbent  upon  researchers  in  educational  multimedia  to  know  where  we  have  been  and  to 
know  where  the  discipline  is  currently  focused,  but  it  is  also  necessary  to  plan  for  the  future.  One 
challenge  for  the  future  does  emerge.  Evaluations  of  systems  are  often  done  on  a small  scale,  relying 
on  local  review  and  evaluation.  We  acknowledge  that  research  into  large  scale  and  long  term 
evaluations  is  difficult  as  populations  move  and  times  change,  but  we  would  like  to  encourage  the 
attendees  to  ED-MEDIA  2000  and  the  readers  of  this  proceedings  to  consider  techniques  and 
strategies  and  programs  to  extend  the  breadth  and  depth  of  their  evaluation  techniques  and 
processes. 

As  Conference  Program  Co-chairs,  we  have  been  aided  considerably  by  a number  of  volunteers  and 
helpers  who  have  given  tirelessly  of  their  time.  In  particular  we  must  thank  the  Conference  Steering 
Committee  of  Gary  Marks,  Ivan  Tomek,  Tom  Reeves,  Betty  CoUis  and  Erik  Duval  who  led  and 
coordinated  the  conference  this  year.  We  would  like  to  thank  the  Chairs  of  the  various  Program 
Committees:  Niki  Davis,  Panels  Chair;  Sam  Rebelsky,  Workshops/Tutorials  Chair;  and  Martyn  Wild, 
Posters/Demos  Chair.  We  would  also  like  to  thank  the  crew  behind  the  scenes  from  AACE  for 
their  tireless  efforts  in  coordinating  and  managing  the  huge  number  of  administrative  functions  that 
a conference  like  this  necessarily  involves. 

We  look  forward  to  meeting  with  you  during  the  conference  and  trust  that  you  will  take  the  time  to 
renew  old  acquaintances,  meet  new  friends,  be  inspired  by  peers  and  take  home  some  new  ideas  and 
activities  to  report  back  at  ED-MEDIA  2001  in  Finland,  June  25-30. 

Program  Chairs: 

Jacqueline  Bourdeau 

Universite  du  Quebec  a Chicoutimi,  Quebec,  Canada 
(jbourdea@uqac.uquebec.ca) 

Rachelle  Heller 

The  George  Washington  University,  Washington  DC,  USA 
(sheller@seas.gwu.edu) 
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Changes  in  Canadian  Higher  Education  ICT  and  Support,  2000  to  2003. 

CariCuneo,  McMaster  University,  Canada;  Brian  Campbe/i,  Mount  Ai/ison  University,  Canada 
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Learning  Object  Containers:  a suggested  method  of  transporting  metadata  with  a learning  object. 

Duane  Currie,  Acadia  University,  Canada;  Craig  P/ace,  Acadia  University,  Canada 

Synchronous  and  asynchronous  discussion:  What  are  the  differences  in  student  participation? 

Gay/e  Davidson-Shivers,  Univ.  of  Souih  A/abama,  USA;  I^n  Mui/enberg  Univ.  of  Souib  Alabama,  USA;  Erica 
Tanner,  Univ.  of  Souib  A/abama,  USA 

The  Adaptation  and  Use  of  a WWW-Based  Course  Management  System  within  Two  Different  Types 
of  Faculties  at  the  University  of  Twente 

IP'im DeBoer,  University  of  Twenie,  Tbe Neiber/ands; Beity  Codis,  University  of  Tiven/e,  Tbe Netber/ands 
Designing  Adaptable  Learning  Environments  for  the  Web:  A Case  Study 

I/eana  De  /a  T^‘a,  LICEF  Research  Center,  Canada;  Andree  Longfrre,  LJCEF  Research  Center,  Canada;  Gi/bert 
Paquette,  LJCEF  Research  Center,  Canada 
Supporting  a community  of  practice:  the  role  of  workers  as  learners 

Maarten  de  Laat,  University  ofNi/megen,  Netber/ands;  Frank  de Jong  Education  and  Knowledge  Center  of  tbe  Dutch 
Po/ice,  Netber/ands; Ju/ian  terHuume,  Education  and  Know/edge  Center  of  tbe  Dutch  Po/ice,  Netber/ands 
A Web-based  Tutorial  on  Optical  Communications 

Ignacio  de  Migue/,  University  of  Va//ado/id,  Spain;  Car/os  Jesus  Fuertes,  University  of  Va//ado/id,  Spain;  Abe/  Prieto, 
University  of  Va//ado/id,  Spain;  Patricia  Femandei(,  University  of  Va//ado/id,  Spain;  Migue/ Lopei(,  University  of 
]Aa//ado/id,  Spain;  Fernando  Gontia/ei(^  University  of  \Ka//ado/id,  Spain;  Juan  Car/os  Aguado,  University  of  JAa//ado/id, 
Spain;  Ruben  M.  Lorenzo,  University  of  Va//ado/id,  Spain;  Evaris to  J.  Abri/,  University  of  Va//ado/id,  Spain 
E-Study:  The  design  of  an  international  online  kno\dedge  community 

SJoerdde  JAries,  University  of  Twente,  Tbe  Netber/ands;  Maurice  vanEgeraad,  University  of  Twente,  Tbe  Netber/ands; 
Dimiter  Bogdanov,  Bu/garian  Academy  of  Sciences,  Bu/garia;  Paid  B/oemen,  University  of  Twente,  Tbe  Netber/ands; 
Lonneke  Roossink,  University  of  Twente,  Tbe  Netber/ands 
Web  accessibility  Overview:  Who*s  Who?  What's  What?  How  to? 

French  Deanie,  Southwest  Texas  University,  USA 

Posting  Visual  Cues  on  Threaded-Message  Boards  in  Science  Curricula:  Enhancing  Transfer  of 
Theory  Knowledge  to  Clinical  Practice 

Gregory  A.  DeBourgb,  University  of  San  Francisco,  USA 

Prototype  Tools  for  Programming 

Fadi  Deek,  New  Jers^  Institute  ofTecbno/ogy,  USA;  James  McHugh,  New  Jers^  Institute  ofTecbno/ogy,  USA 

Preparing  our  teachers  for  distance  education 

Christina  Deb/er,  Concordia  University,  Canada;  Christina  De  Simone,  Concordia  University,  Canada 

Architectural  Aspects  of  An  Interactive  Multimedia  Environment  for  Training  Employees  in 
Internet/ Intranet  Technologies 

Gregory  Derekenan's,  University  of  Patras,  Greece; John  Garofa/akis,  University  of  Patras,  Greece; Jim  Prent:(as, 

University  of  Patras,  Greece;  Spiros  Sioutas,  University  of  Patras,  Greece;  Atbanassios  Tsaka/idis,  University  of  Patras, 
Greece 

The  «BacVert»  or  «Blue  Box»  - A Knovdedge  Recycling  Bin  for  Students 

Ce/ine  Dejardins,  Centre  de  recberc/ie  LICEF,  Canada;  Efoe' UAa//ace,  Centre  de  recherche  LICEF,  Canada;  C/aude 
RJcdardi-Rjgau/t,  Centre  de  recherche  LICEF,  Canada 

Building  Global  Information  Communities:  the  University  of  Iowa  Center  for  Electronic  Resources  in 
African  Studies  (CERAS) 

Barbara  Dew^,  University  of  Iowa  IJbraries,  USA 

Navigating  Scholarship:  A Study  of  Discipline-Based  Web  Gateways  in  Large  Research  Libraries 

Barbara  Dew^,  University  of  Iowa  IJbraries,  USA 

The  University  of  Iowa  Science  Information  Literacy  Initiative 

Barbara  Dew^,  University  of  Iowa  IJbraries,  USA 

Suffering  Remotely 

Ross  Dewstow,  UNTTEC,  Institute  of  Tecbno/ogy,  New  Zea/and;  Stuart  Young  UNTIE C,  Institute  ofTecbno/ogy,  New 
Zea/and;  Mae  McSporran,  UNTTEC,  Institute  ofTecbno/ogy,  New  Zea/and 

Introducing  a Distance  Learning  version  of  a Postgraduate  Program  on  Networking  in  Argentina. 

Dean  Javier  Diat(^  University  of  La  P/ata,  Argentine;  Maria  A/Jandra  Osorio,  University  of  La  P/ata,  Argentine;  Ana 
Pao/aAmadeo,  University  of  La  P/ata,  Argentine 

Increasing  Explorativity  by  Generation 

Stpban  Dieb/,  University  of  Saar/and,  Germany;  Andreas  Kerren,  University  of  Saar/and,  Germany 
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The  study  of  Newtonian  Mechanics  in  junior  high  school.  A new  techonology-based  learning 
environment. 

Pana^otis  DimitriMps,  Umversify  Of  Athens,  Greece;  Lamprinr  Pe^atsimpa,  Unwersrty  Of  Athens,  Greece;  Kostas 
Ka/rrponrts-Paparrrrha/ts,  Universtf  Of  Athens,  Greece;  Giannos  Karanikas,  Universif  Of  Athens,  Greece;  George 
KaJkanrs,  University  Of  Athens,  Greece 

The  Bones  Of  The  SkulL  Creating  Anatomical  Models  With  Quickdme  VR 

ManfnDipensa,  The  University  of  lom,  USA;  Jinr  Duncan,  The  University  of  lotva,  USA;  Jerr)>  Moon,  The 
University  of  Iowa,  USA 

Authoring  Educational  Hypermedia  Using  a Semantic  Network 

Duncan  Mu//ier,  Leeds  Metropo/itan  University,  England;  Mark  Dixon,  Leeds  Metropo/itan  University,  England 

The  impact  of  establishing  a virtual  university:  a case  study  at  NUST 

Nomusa  D/od/o,  Nation  a/  University  of  Science  and  Technology,  Zimbabwe 

A Distributed  Maritime  Simulation  Training  Environment  based  on  HLA 

Mirko  Dobermann,  Computer  Graphics  Center  ZGDIZ^  Germany;  Harw  Kuchar^wski,  MarineSoft  GmbH,  Germa^ 

Supporting  developers  of  inquiry  based  instruction:  Performance  support  on  the  Web 

Jackie  Dobrovo/nj,  Un.  of  Colorado  at  Denver,  USA;  Scott  Grabinger,  Un,  of  Colorado  at  Denver,  USA;Jenn  Ught, 

Un.  of  Colorado  at  Denver,  USA 

Accessing  Advocacy  Virtually  and  Physically 

TanisDoe,  Canadian  Assodation  of  Independent  Living  Centers,  Canada;  Scott  Wilson,  Computer  Master,  Canada 

Streaming  Video:  Pedagogy  and  Technology 

Trevor  Doerksen,  University  of  Ca/gay,  Canada;  Alike  Mattson,  University  of  Ca/gay,  Canada;  Jacques  Morin,  George 
Washington  University,  United  States 

Human  Anatomy  Predissection  Lecture-On-Demand  at  The  National  University  of  Singapore 

Gi/les  Doiron,  National  University  of  Singapore,  Singapore 

The  impact  of  visualisation  on  chemistty  teaching  and  learning 

Danica  Dolnicar,  University  of  If ubpana,  Slovenia;  Magareta  Vrfacnik,  University  of  If ub fan  a,  Slovenia;  VesnaFerk, 
University  of  If ub fan  a,  Slovenia;  Mat  fa  Sa/ovec,  Simon Jenko  Primay  School^  KranJ,  Slovenia 

Leading  Your  Faculty  Down  the  Information  Highway 

Thomas  Donahue,  University  of  St  Thomas,  USA 

Is  the  expression  "Instructional  Engineering"  justified? 

Sylvie  Done]  Ecole  de  technologie  s/perieure,  Canada;  Josianne  Basque,  Tele-universite]  Canada 

Student  Technology  Services:  A Case  Study  in  Experiential  Learning 

Joseph  Douglas,  University  of  Wisconsin-Milwaukee,  USA;  John  A.  Grvfk,  University  of  Wisconsin-Milwaukee,  USA 

Technology  Across  the  Curriculum 

Gerald  Drake,  Geoge  Mason  University,  USA 

ExploraGraph  and  CINEMA 

Aude  D/fresne,  Univ.  of  Montreal,  Canada;  Claire  Isabelle,  Univ.  of  Moncton,  Canada;  Roger  Nkambou,  Univ.of 
Montreal,  Canada;  Yan  Laporte,  Univ.  of  Montreal,  Canada;  Frank  Ferront,  Univ.  of  Montreal,  Canada 

ExploraGraph  : A flexible  and  adaptive  interface  to  support  distance  learning. 

Aude  Dufresne,  University  of  Montreal,  Canada;  Gilbert  Paquette,  LICEF  Reasearch  Center,  Tele-universite]  Canada 

The  Effect  of  Level  of  Technology  Training  on  Teachers'  Attitudes  Toward  Using  and  Integrating 
Technology 

Ronald  Dugan,  University  at  Albany/ SUNY,  United  States;  Jennifer  Richardson,  University  at  Albany /SUNY,  United 
States;  Dianna  L Newman,  University  at  Albany/ SUNY,  United  States 

Future  Special  Education  Teachers'  Abilities  to  Integrate  New  Technology  Into  Teaching  Reading 
Comprehension 

Catherine  Dumoulin,  Universite  dujfnebec  a Chicoutimi,  Canada; Jacqueline  Bourdeau,  Universite  du  Quebec  a 
Chicoutimi,  Canada 

School  Festival  on  the  Internet  - Project-based  and  cooperative  learning  - 

HirononEgi,  Keio  University,  Japan;  Yoshitomo  Tsutsui,  Kamogata  High  School,  Jcpan;  Yuuki  Nishimura,  Keio 
University,  Japan;  Keiichirou  Ishibashi,  Keio  University,  Japan 

ITBeankit:  An  Educational  Middleware  Framework  for  Bridging  Software  Technology  and  Education 

Abdulmotaleb  El  Saddik,  Darmstadt  University  of  Technology,  Germany;  Stephan  Fischer,  Darmstadt  U niversity  of 
Technology,  Germary;  Raf  Steinmety)  German  National  Research  Center  CMD-IPSI,  Germary 

The  ParlEuNet-project:  Problems  with  the  validation  of  socio-constructivist  design  principles  in 
ecological  settings 

Jan  Elen,  University  of  Leuven,  Belgium;  Geraldine  Clarebout,  University  of  Leuven,  Belgium 
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LeamScope  and  the  Development  of  Virtual  Learning  Communities 

A/lanEJ/is,  Soi^^hem  Cross  Umversity,  Australia;  Rol^n  li^eather/^,  NSW  Dept  of Educatior  arsi  Tratmn^  Austraiia 

Integrating  WWW  Technology  into  Classroom  Teaching:  Students*  Persepectives  of  the  Usefulness  of 
their  Course  Web  sites 

Mara/ Ei-Tigi  Sjracrse  Urmrsity^  USA 
Evaluation  of  a Systematic  Tele-Tutor  Training 

BruroEmars,  Uriverstty  of Amsteniam,  The  Netber/ards;  Noor  Christoph,  Urwersity  of  Amsterdam,  The  Nether/ards; 
Jacobi/r  Sardherg  Uriversity  of  Amsterdam,  The  Nether/ards 

Providing  Quality  Distance  Education  in  an  Outcomes-Oiiented  Program 
Virgiria  Ergmar,  Fort  Lewis  Co//^e,  USA;  Virgiria  Ergmar,  Fort  Lems  Co/iege,  USA 

A Model  of  Successful  Technology  Integration  in  a School  System:  Plano's  Curriculum  Integration 
Project 

Yoram  Eshet,  Te/ Hai  Co//ege,  Israe/;  Joe/  K/emes,  The  Oper  Urirersity  ofisrae/,  Israe/;  l^r  Herdersor,  James  Cook 
Urirersity,  Austra/ia;  Sara  Ja/a/i,  Irdperdert  Mu/timedia  Producer,  USA. 

Perceptions  of  a Fearless  Experimenter:  Student  Perceptions  of  Creating  Electronic  Portfolios 

Sue  Eskridge,  Urirersity  of  the  Paafc,  USA;  Ray  Pos^,  Uriversity  of  the  Paafc,  USA 

Helping  K-12  Teachers  Integrate  Internet  Technology  into  Their  Classrooms 

Chnstopher  Essex,  Irdiara  Uriversity,  USA 
A Course  Website  One  Year  Laten  Lessons  Learned 

Christopher  Essex,  Irdiara  Urirersity,  USA;  Briar  Beatty,  Irdiara  Uriuersity,  USA;  Pame/a  Sorg  Irdiara  Uriversi^, 
USA 

ALIS:  An  adaptable  learning  and  information  system 

Shahriar Fakher,  FerrUriHager,  Germar;y 

Examining  Medical  Students’  Attitudes  and  Learning  Experiences  in  BioWorld 

Soria  Faremo,  McCi//  Urirersity,  Carada;  Susarre  lufoie,  McCi//  Uriuersity,  Carada;  David  F/eist^er,  Facu/ty  of 
Medicire,  McCi//  Uriversity,  Carada 

CyberARTS:  A Integrated  Arts  Curriculum 

Tito  Faria,  Dor  Mi//s  Coi/egiate  Irsititute,  Carada;  Sho/om  Eiserstat,  Dor  Mi//s  Co//egiate  Irstitute,  Carada 

"Next  Steps" 

Josephire  Feraqy^o/i  Farre//,  State  Uriversity  of  New  York  at  Oswego,  USA 

Managing  Electronic  Resources  - Auto-registration  for  Distance  Learning 

Richard  Fasse,  Rochester  Irstitute  ofTechro/ogy,  US;  Damor  Bet/ow,  Rochester  Irstitute  qJTechro/ogy,  US;  Rardy 
Overbeck,  Rochester  Irstitute  of  Techro/ogy,  US 

The  Aristotle  Project 

Gordor  Fau/krer,  Rj/tgers  Uriversity,  USA,  Sumathi  Gopa/,  Rutgers  Uriversi^,  USA,  Abraham  Itiycheriah,  Rutgers 
Uriversity,  USA,  Richard  Mammore,  Rulers  Uriversity,  USA,  Aiti/a  Med/,  Rj/tgers  Uriversity,  USA,  Miros/av 
Novak,  Rj^tgers  Uriversity,  USA 

Conversation,  the  constructivist  persepective 

Gary  Fergusor-Smith,  Uriversity  of  Wo//orgorg  Austra/ia;  David A//ibor,  Apo//o  Parkways  Pnma^  Schoo/,  Austra/ia; 
Christire  Browr,  Uriversity  of  Wo//orgorg  Austra/ia;  Bar^  Hafper,  Uriversity  of  Wo/iorgorg  Austra/ia;  Johr  Hedberg 
Uriversity  of  WoZ/orgorg  Austra/ia 

Evaluating  motivational  aspects  of  a web-based  English  language  course  through  the  Website 
Motivational  Analysis  Checklist  (WebMAC) 

Arise  Ferreira,  Cathode  Uriversity  of  Sao  Pau/o,  Bratf/ 

User  Studies:  Evaluating  the  Use  of  EPSS  Tools  for  Self-Management  by  Children 

Gai/ Fitggera/d,  Uriversity  of  Missouri-Co/umbia,  USA;  Louis  P.  Semrau,  Arkarsas  State  Uriversity,  USA,  Hsiryt 
Perg  Uriversity  of  Missouri-Co/umbia,  USA 

Interactive  Training  Materials  for  Early  Childhood  Educators:  Cases,  Tools,  and  Reflection 

Gai/ Fiti^era/d,  Uriversity  of  Missouri-Co/umbia,  USA;  Louis  Semrau,  Arkarsas  State  Uriversity,  USA 

Integrating  computer  ethics  into  the  computer  science  and  computer  engineering  curricula 

Johr  Fodor,  Educatiora/ Media  Resources,  Ire.,  USA 

How  to  Prepare  Teachers  for  Integrating  Multimedia  and  Technology  into  K-12  Classrooms 

Barbara  Foster,  Spa/di rg  Uriversity,  USA 

A Catalyst  for  Collaboration:  Supporting  Technology  in  Teaching  through  Partnerships 

I^uisFox,  Uriversity  (fWashirgtor,  USA;  Tom  Lewis,  Uriversity  of  Washirgtor,  USA;  Scott  Mack/ir,  Uriversity  of 
Washirgtor,  USA 
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Development  and  Implementation  of  Computerized  Instructional  Materials:  The  Mismatch  Between 
Formal  and  Informal  Technology  Support  Structures  in  a Medical  College 

Q>n/hia  Frank,  The  Unwenify  of  Ari^na,  USA. 

Adapting  an  internet  based  learning  environment  to  a hybrid  approach 

Kafn'n  France,  TU  Dresden,  Germa^;  Hagen  IFahrig,  TU  Dresden,  Germany;  Ohf  Neumann,  TU  Dresden,  Germany 

Health  Issues  for  Human  Computer  Interaction  in  Electronic  Learning  Environments 

Joy  Fraser,  A/habasca  University,  Ganada;  Peie  HoU,  Aibabasca  University,  Ganada;  James  Mackintosh,  Simon  Fraser 
University,  Ganada 

Putting  Curriculum  in  Control:  Interactive  Courseware  Development  with  the  TechDisc  Instructional 
Design  Template 

TraigFriedncb,  S encore,  Inc.,  United  States;  Bi//  IFann,  Sencore,  Inc.,  United  States;  Karen  Kortb,  University  of  South 
Dakota,  United  States;  Mike  Hoad/ty,  University  of  South  Dakota,  United  States;  Dan  Eastmond,  University  of  South 
Dakota,  United  States 

Distance  Learning  Development  Teams 

Anthonj  Frislty,  Thomas  Jefferson  Univenity,  USA 

OCCA:  Development  of  an  Institutional  Strategy  for  ^ass  Customization’  of  Online  Learning 

Paul  Fritye,  University  of  Melbourne,  Australia;  Robert  Kemm,  University  of  Melbourne,  Australia 

Virtual  Reality  Simulation  of  Reovirus 

Denis  Gadbois,  University  of  Ca/gaty,  Canada;  David  Rittenhouse,  Advanced  Media for  Learning  Ganada;  Dave 
Gou/den,  Netera  Alliance,  Ca/gary,  Canada 

Orienting  Tasks  to  Enable  Hypertext  Learning 

James  Gad,  University  of  Northern  Colorado,  USA 

The  Reflection  Assistant:  Investigating  the  Effects  of  Reflection  Activities  in  Problem  Solving 
Environments 

C/audia  Gama,  University  of  Sussex,  UK 

Pedagogical  and  Instructional  Modeling  via  On-line  Assistants 

Penrty  Garcia,  University  of  Wisconsin  Oshkosh,  USA 

Designing  Synchronous  Interactive  Learning  Teleconferences:  Digital  Field  Experiences  At  The 
Columbus  Zoo,  An  Informal  Science  Contextual  Setting 

Tamara  J.  Garcia-Barbosa,  The  Ohio  State  University,  USA;  Andrea  K Ba/as,  The  Ohio  State  University,  USA;  Dr. 
Barbara  S.  Thomson,  The  Ohio  State  University,  USA 

A Split-Brain  Humand  Computer  Interface 

Gregoty  Garvty,  Quinnipiac/ Concordia  University,  USA/Canadr 

User  interface  design  in  a Learning  Environment  based  on  a Computer  Algebra  System 

Jean-Michei  Ge/is,  lUFM  de  Versai/ks,  France;  Dominique  LENNE,  Institut  National  de  Recherche  Pedagogique, 
France 

BIDACI:  Modeling  Agents  for  a Learning  Environment  based  on  a Digital  Library 

y^a/a  Gerardo,  Universidad  De  Las  Americas,  Mexico;  Cocon  Fe/ipe,  Universidad  De  Las  Americas,  Mexico;  Agosto 
Lay  da,  Universidad  De  Las  Americas,  Mexico 

Using  Virtual  Reality  to  Assist  Air  Travelers  With  Disabilities 

Clark  Germann,  Metropolitan  State  College  of  Denver,  USA;  Jane  Brvida,  Metropolitan  State  College  of  Denver,  USA 

Gender  and  Race:  The  Accuracy  of  Internet  Clip  Art 

Carol  Gillty,  University  of  Arkansas,  USA;  Elaine  Terrell,  University  of  Arkansas,  USA 

Implementing  Interactive  Multimedia  Courses  for  the  WWW  Using  Streaming  and  SMIL  Technology 

Thanasis  Giouvanakis,  University  of  Macedonia,  Greece;  KonstantinosTarabanis,  University  of  Macedonia,  Greece;  Haido 
Samaras,  University  of  Macedonia,  Greece 

Narrative,  Context  and  Cognition  in  Interactive  Media 

Lisa  GJedde,  Royal  Danish  University  of  Educational  Studies,  DENMARK 

Blazing  Trails  on  the  World  Wide  Web 

CorinneGtynn,  Grinned  College,  USA;  Rachel  Heck,  Grinned  College,  USA;  Sarah  Luebke,  Grinned  College,  USA; 
Five  other  Authors,  Grinned  College,  USA;  Samuel  Rebels^,  Grinned  College,  USA 

Establishing  and  Operating  an  Institutional  Centre  for  Instructional  Technology 

Diane  Goldstein,  University  of  Western  Ontario,  Canada;  Michael  Clarke,  University  of  Western  Ontarion,  Canada 

How  To  Be  Innovative  Designing  Educational  And  Interactive  Environments  For  Children 

Glona  Elena  Gome:^^  Escobar,  University  of  Los  Andes,  Colombia 
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A New  Education  Framework  Within  IMS-Specifications 

Antonio  Gomei^^Skameta,  University  of Mnrda,  Spain;  Ednardo  Marfine;^^  Gracia,  University  of  Murcia,  Spain;  Pedro 
Garda  Lope^  University  Rovira  i Virgil,  Spain 

A Multimedia  Matrix  Model  For  Designing  Instructional  Strategies  and  Learning  Experiences  for 
Students  of  Human  Movement  Studies 

Ha/ima  Goss,  Queensland  University  of  Technology  (QUT),  Australia;  Graham  Kerr,  Queensland  University  of 
Technology  QUT),  Australia 

Information  Literacy  in  a Virtual  Environment:  a Web-Based  Approach 

Ann  Gnfstein,  GUNY/  Co/iege  of  Staten  Is/and,  United  States 

Art  and  Technology  in  the  Classroom:  The  Activities  and  Impact  of  a National  Project. 

Neai  Grandgenett,  University  of  Nebraska  at  Omaha,  US.;  Sherwood  Dow/ing  National  Museum  of  American  Art, 
USA 

Being  Social  in  an  Anti-Social  Environment:  Is  Cooperative  Learning  Really  Possible  Using 
Computer-Mediated  Communication? 

Joseph  Grgg  Jones  Internationa/  University,  USA 
Virtual  Study  Groups  (VSG),  an  approach  to  networked  collaborative  learning 

Montse  Guitert,  Open  University  of  Cata/onia  fUOGJ,  SPAIN;  Ferran  Gimene:^  UOC,  Spain;  Thanasis  Daradoumis, 
UOC,  Spain; Joan  Mane/ Marques,  UOC,  Spain 
Visual  Navigation  for  the  Curriculum  Based  Learning  Process 

Rjdbiao  Guo,  University  of  Montrea/,  Canada;  C/aude  Frasson,  University  fMontrea/,  Canada;  Marc  Ka/ten  bach. 
University  of Montrea/,  Canada; Jan  Gecsei,  University  of  Montrea/,  Canada 
A Comparison  of  Static  and  Djmamic  Media  Types  for  Process  Oriented  Learning  Tasks 

Sisse/ Guttormsen  Schdr,  Swiss  Federa/ Instimte  of  Techno/ogy,  Zurich;  Hans  JorgZuberbiih/er,  Swiss  Federa/ Institute  of 
Technology,  Zurich;  He/mut  Krueger,  Swiss  Federal  Institute  of  Technology,  Zurich 
Teaching  Dynamics  of  Dairy  Herd  Health  and  Management 

C/audia  Haferkamp-lFise,  Come//  University,  USA;  Heather  G.  A/iore,  Come//  Univerdty,  USA;  Yfo  T Grohn, 
Come//  University,  USA;LonnD.  IFamick,  Come//  University,  USA 
Expert  Review  at  a Distance:  A Hybrid  Approach 

David  Ha/pin,  Virginia  Tech,  USA;  A/  Byers,  Virginia  Tech,  USA 
Scenarios  of  a New  Dimension  of  Learning  by  the  Co-operative  Structuring  of  Knowledge 
Thorsten  Hampe/,  University  of  Paderbom,  Germany 

Prototypical  Development  of  Multimedia  Explorations  with  Preventative  Medicine  as  an  Example 

Thorsten,  Harrpe/,  University  of  Paderbom,  Germany;  Thomas  von  der  Borch,  University  of  Paderbom,  Germany 

Keep  facilitating  pupils  and  English  teachers  appropriately 

Bo  Han,  Nationa/  TeacherdCo/iege  Inner  Mongo/ia,  China 

A Comprehensive  Model  for  Improving  Technology  in  Teacher  Education 

Robert  (Bob)  Hannafn,  Co/iege  of  Wi//iam  and  May,  US;  Robert  (Bob)  Hanny,  Co/iege  of  lFi//iam  and  May,  US. 

Interactive  Analogies  Prime  Learning  from  Visuali2ations 

Steven  Hansen,  Auburn  University,  USA;  N.  Had  Narayanan,  Auburn  University,  USA 

Designing  Interactive  Learning  Environments  to  include  construction  tools  for  personally  meaningful 
artefacts 

Bary  Haper,  University  of  Wo/iongong  Austra/ia;John  Hedberg  University  oflVo/iongong  Austra/ia;  Rob  IFdght, 
University  ofUZo/iongong  Austra/ia 

Time-Expanded  Audio  For  Learning 

Kevin  Hardgan,  Univerdty  of  Water/oo,  Canada 
Mariner-A  3-Dimensional  Navigation  Language 

Janet  Hartman,  I/iinois  State  University,  United  States;  Joaquin  Vi/a,  I/iinois  State  University,  United  States 
Course  Production  Applying  Object  Oriented  Software  Engineering  Techniques 

Rona/d  Hartwig  University  of  Luebeck,  Germary;  Huberta  Kddpnberger,  UniverdyofL/rebeck,  Germany;  Michae/ 
Herc^eg  University  of  Luebeck,  Germany 
Cognitive  Flexibility  Hypertext  and  the  Role  of  the  Learning  Task 

Doug/as  M.  Harvy,  Richard  S tockton  Co//ege,  United  States;  David  H.  Jonassen,  The  Penny /van ia  State  University, 
United  States;  Roy  C/ariana,  The  Penny /vania  State  University,  United  States 
A Framework  for  Evaluating  The  Usability  ofF  Political  Web  Sites  : Towards  Improving 
Cyberdemocracy 

Shahir^an  Hassan,  University  of  Strathcp/de,  Scot/and;  Feng  Li,  PhD,  University  f Strathc/yde,  Scot/and 
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Project  DEED:  Distance  Education  for  Educational  Diagnosticians.  Putting  available  technology  to 
work  via  internet  for  rural,  bilingual  sites  along  the  Texas/Mexico 

Rap/;  Haasman,  The  Unwersi/y  of  Texas  one/  Texas  Soathmosf  College,  USA 

Weaving  a Web  Without  a Net:  Browser-Abased  Educational  Applications 

ScouHea/h,  Bonf/s  B/ooe/  Center,  USA;  Dan/e/  R.  Amhraso,  University  of Coioraeio  School  of  Medicine,  USA 

An  Interactive  Course-Support  System  for  Greek 

Trade  Heift,  Simon  Fraser  University,  Canada;  Janine  Too/e,  Simon  Fraser  University,  Canada;  Pan/ McFetridge,  Simon 
Fraser  University,  Canada;  Fred Popowich,  Simon  Fraser  University,  Canada;  Stavrou/a  Tsip/akou,  Simon  Fraser 
University,  Canada 

Learning  Objects:  Communicating  the  Pedagogical  Potential 

CaQ!  Hepbam,  Acadia  University,  Canada;  Craig  Piace,  Acadia  University,  Canada 
Web-based  tools  for  courses  on  transport  phenomena 

Bernardo  Hemande:;;-Mora/es,  Universidad  Nadona/Aatonoma  de  Mexico,  Mexico;  Rafael  Femande:(;-F/ores, 
Universidad  Nadonai  Autonoma  de  Mexico,  Mexico;  Jorge  Te//e:(;-Martine;^  Universidad  Nadonai  Autonoma  de  Mexico, 
Mexico 

Shaknoma:  A collaborative  tool  for  shared  knowledge  management 

Oriel  Herrera,  Pontifda  Universidad  Cato/ica  de  Chile,  Chile;  Sergio  Ochoa,  Pontiftda  Universidad  Cato/ica  de  Chi/e, 
Chi/e;  V'idai  Rodrigue:;^  Pontfida  Univerddad  Cato/ica  de  Chi/e,  Chi/e;  David  Fuller,  Pontrfda  Universidad  Cato/ica  de 
Chi/e,  Chi/e 

Instructional  design  guidelines  for  authentic  activity  in  online  learning  units 

Jan  Herrington,  Edith  Cowan  University,  Arrstra/ia;  Heather  Sparrow,  Edith  Cowan  University,  Austra/ia;  Tor^ 
Herrington,  Edith  Cowan  University,  Austra/ia 

How  to  use  Lotus  Notes/  Learning  Space  and  The  Web  in  supervising  students  abroad. 

Wim  Hesse/s,  University  of  Higher  Prvfessiona/  Education,  The  Nether/ands 

Computer  Science  Instruction  in  a Virtual  World 

Curt  Hi//,  Ha//^  City  State  University,  USA;  Brian  S/ator,  North  Dakota  State  University,  USA 

Open  Educational  Hypermedia  Systems:  An  n-dimensional  framework  for  User  Profile  Interchange 

Gunter  Hi/mer,  University  of  G/amorgan,  Wa/es;  Geoff  E//iott,  University  qjC/amorgan,  Wa/es;  Danie/ Crrn/ffe, 
University  of  G/amorgan,  Wa/es;  Dorrg  Tudhope,  University  of  G/amorgan,  Wa/es 

Considering  Pedagogy  in  the  Design,  Development  and  Evaluation  of  Educational  Software 

J.  End^ue  Hinostro:;a,  Universidad  de  Frontera,  Chi/e;  Harv^  Me//ar,  Univerdty  of  London,  United  Kingdom 

New  Horizons  in  Distance  Education 

Jeannine  Hirt/e,  Sam  Houston  State  University,  USA;  Robin  McCrew-Zoubi,  Sam  Houston  State  University,  USA; 
Chdstopher Scqfie/d,  St  Martin's  University,  USA 

Using  Web-based  Templates  to  Support  Reflection  on  Learning  in  University  Classes 

Carry  Hoban,  Univerdty  ofWo/iongong  Arrstra/ia 

Biggs'  Constructive  Alignment:  Evaluation  of  a Pedagogical  Model  Applied  to  a Web  Course 

John  Hoddinott,  University  of  A/berta,  Canada 

Teachers  and  Computer  Technology  Training 

David  Hofmeister,  Centra/  Missouri  State  University,  USA;  Gerry  Peterson,  Centra/  Missorrri  State  University,  USA 
Results  and  conclusions  from  a Learning  Software  Design  Competition 

Brad  Hokanson,  Univerdty  Of  Minnesota,  USA;  Simon  Hooper,  University  Of  Minnesota,  USA;  Pard  Bernhardt, 
University  OfMinneosta,  USA 

Development  of  a Web-Based  Instructional  System  for  Emerging  Scientific  Technologies:  The 
Vanderbilt  Microarray  Shared  Resource,  a case  study. 

Wi//iam  T Ho/den,  Vanderhi/t  Univerdty,  USA;  Mark  K M^uain,  Vanderbi/t  Univerdty,  USA;  FK  Hase/ton, 
Vanderbi/t  University,  USA;  KJ.  Coff^,  Vanderbi/t  Univerdty,  USA;  D.W  Threadgi//,  Vanderbi/t  University,  USA 

Interactive  Computer  Assisted  Formulation  of  Retrieval  Requests  for  a Medical  Information  System 
using  an  Intelligent  Tutoring  System 

Andreas  Ho/rjnger,  Gra:^  University,  Austna;  Andreas  Kain;;^  University,  A/rstna;  G/renther  Ge//,  Gra^ 
University,  Austria;  Max  Bruno/d,  Gra^  University  OfTechno/ogy,  Arrstria;  Hermann  Maurer,  Gra:^  University  Of 
Techno/ogy,  Austria 

Graphical  Representations  of  Convergence  in  Web-based  Educational  Computer  Conferencing:  A 
Prototype 

Lawrence  Hopperton,  Simon  Fraser  University,  Canada;  Richard  Rjpp,  Simon  Fraaser  University,  Canada;  B.  Fisher, 
Simon  Fraaser  Univerdty,  Canada;  L Harasim,  Simon  Fraaser  University,  Canada;  M.  Luk,  Simon  Fraaser  University, 
Canada;  C Ost/er,  Simon  Fraaser  Univerdty,  Canada;  C Xin,  Simon  Fraaser  Univerdty,  Canada 
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Constructive  Hypertext  for  Learning 

H si-chi  Huang,  The  Ohio  State  University,  USA 

Traveling  Culture:  Digitizing  Special  Collections  Material  at  the  University  of  Iowa 

Carol  Ann  Hughes,  University  of  Iowa,  USA 
A Multimedia  Resources  Bank  for  Teaching  and  Learning 

Vincent  Hung  Hong  Kong  Institute  of  Education,  Hong  Kong’ Jacfy  Pow,  Hong  Kong  Institute  of Ech^cation,  Hong  Kong' 
IVinnie  So,  Hong  Kong  Institu^  of  Education 

Multimedia  Training  Materials  for  the  Community  University  of  the  Valleys 

Timotfy  Hutchings,  University  of  Glamorgan,  UK;  Mike  Watkins,  University  of  Glamorgan,  UK;  Geneen  Stubbs, 
University  of  Glamorgan,  UK 

Internet  Infrastructure  with  ATM  for  Small  Countries;  Case  Studies 

Necdet  Id/,  Eastern  Mediterranean  University,  Turkiye 

Inducting  on-campus  students  into  online  discussion 

Alistair  Ing/is,  RMIT  University,  Australia;  Jererrr}/  Keens,  RMIT  University,  Arrstra/ia 

NACSIS-ILL  Distance  Training  System  in  IDLE  Distance  Learning  Project 

Tomo'o  Inoue,  National  Institute  of  Informatics,  Japan;  Haruki  Ueno,  National Institrrte  of  Informatics,  Japan 

Teaching  and  Learning  with  STYLE  (Situated  Technology  Yielding  Learning  Enhancement 

Barbara  K Iverson,  Columbia  Co/iege  Chicago,  USA 

Designing  a media  management  system  to  match  your  needs:  a systematic  approach 

John  Jackson,  Univ  of  Virginia,  USA 

Let  your  mouse  do  the  talking:  encouraging  new  conversations  in  software  design. 

G/enda Jacobs,  UNI  TEC  Institute  of  Technology,  New  Zealand;  Duncan  M^er,  Long  Co/iege,  New  Zealand 

Excellent  Teaching  and  Early  Adopters  of  Instructional  Technology 

Miche/e  Jacobsen,  University  of  Ca/gary,  Canada 

Designing  a WWW-based  course  support  site  for  learners  with  different  cultural  backgrounds: 
Implications  for  practice 

Karen Jager,  University  of  British  Columbia,  Canada;  Betty  Co/iis,  University  qJTwente,  The  Nether/ands 

International  Group  Work:  New  Roles  for  Interaction? 

Diane  P.  Janes,  University  of  British  Co/rrmbia,  Canada 

IJnIdng  Active  Learning  to  Web-based  Instruction:  Students  Teaching  Students  through  Multimedia 
Productions 

James  S.  Javenkoski,  Department  of  Food  Sdence  and  Human  Nutrition,  Universidi  ofl/iinois  at  Urbana-Champaign, 
USA;  Elizabeth  F Reutter,  Department  of  Food  Sdence  and  Human  Nutrition,  University  ofl/iinois  at  Urbana- 
Champaign,  USA; James  E.  Painter,  Department  of  Food  Sdence  and  Human  Nrrtdtion,  University  ofl/iinois  at 
Urbana-Champaign,  USA 

Using  Tailored,  Interactive  Soap  Operas  for  Breast  Cancer  Education  of  High-Risk  Hispanic  Women 

Mada  Jiba/a,  Be^/or  Co/iege  of  Medidne,  USA;  Pard Kinge^,  H ami/ton  Fish  Institute,  USA;  Nang>  N^  Bay/or 
Co/iege  of Medidne,  USA; Quentin  Smith,  Be^/or  Co/iege  of  Medidne,  USA; Jennfer  Bowman,  USA; J.  David 
Ho/comb,  USA 

Learner’s  Cognitive  Construction  in  a Web-Courseware 

Hu^chingjanicejih,  Tamkang  University,  Taiwan 

webStract  - a Distributed  Tool  for  Collaborative,  Project-driven  Learning  on  the  Internet 

Werner  B.  Joerg  NetEssence  / U of  A/berta,  USA 

Multimedia  Case  Studies  for  Be^vior  Disorders:  A Comparison  of  Perceptions  of  Usability  and 
Patterns  of  Usage  Between  Majors  and  Nonmajors 

Lewis  R Johnson,  Arkansas  State  Univerdty,  USA;  Louis  P.  Semrau,  Arkansas  State  University,  USA;Gai/E. 

Fr  ’dgera/d,  University  of  Missoud  - Co/umbia,  USA 

Transforming  Active  Learning  Exercises  for  Distance  Education 

Nancy  Johnson,  Metropo/itan  State  University,  Minneapo/is,  USA 

visualization  of  symbolic  information:  empowering  student  introspection 

Pau/Jue/i,  North  Dakota  State  University,  USA 

Tips  and  Tricks  for  the  Development , Delivery  and  Management  of  Online  Courses 

Jonathan  Kadis,  Utah  State  University,  USA 

Tips  and  Tricks  for  the  Development , Delivery  and  Management  of  Online  Courses 

Jonathan  Kadis,  Utah  State  University,  USA 
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The  Evolution  of  Online  Learning  to  a Digital  Broadcast  Environment 

JobnKab/er,  WHYY,  USA;  Roger  MiuM,  WHYY,  USA;  Bi// Weber,  WHYY,  USA;  T0777  Murray,  Delaware 
County  Communi^  Co//ege,  USA;  Doug  McConatba,  WebStuefy,  Inc.,  USA 

Computer-Based  Interactive  Qualitative  and  Quantitative  Knowledge  Maps  Applications  Model  in 
College  Business  Courses 

A/ex  Ka/stura,  Drytona  Beach  Community  College,  USA 

Digital  Learning  System:  Web's  Cool 

Mjunghee  Kang  Ewha  Womans  University,  Korea 
The  Effects  of  PBL  Authenticity  on  Learning  and  Motivation 

/\e^unghee  Kang  Ewha  Womans  University,  Korea;  Nari  Kim,  Ewha  Womans  University,  Korea 

Field  Dependence/Independence  and  the  Needs  of  Students  in  a Web-Based  Instructional 
Environment 

Do/ores  Karv/ick,  US  Air  Force  Acalenry,  USA;  Doris  Cary,  University  of  Co/orae/o  at  Co/orae/o  Springs,  USA; 
Michae/  Wfyte,  A;^sa  Pacrfc  University,  USA 

An  Interactive  History  as  Reflection  Support  in  Hyperspace 

Akihiro Kashihara,  Osaka  University,  Japan; Shinobu Hasegawa,  Osaka  University,  Jepan;Jun/chi Tyoda,  Osaka 
University,  Japan 

OR-World  - Using  Learning  Objects  in  a Hypermedia  Learning  Environment 

Stephan  Kassanke,  University  o/Paderbom,  Germary;  Leena  Suh/,  University  of  Paderbom,  Germay 
Learning  Amidst  A Sea  Of  Information  In  The  New  Millennium 

Dan  Karewe/i,  University  of I/iinois,  Urbana-Champaign,  USA; Jim  Levin,  University  of  I //inois,  Urbana-Champaign, 
USA;  Daniel  Schiff,  University  of  I/iinois,  Urbana-Champaign,  USA;  Young-Jin  Lee,  University  of  I/iinois,  Urbana- 
Champaign,  USA 

Teaching  WAP  Technology  as  a part  of  a Multimedia  Course 

Harri  Keiho,  Tampere  University  ofTechno/ogy,  Fin/and;  Jari  Lahti,  Tampere  Univerrity  ofTechno/ogy,  Fin/and;  Jari 
Mu/tisi/ta,  Tampere  University  of  Techno/ogy,  Fin/and 

Collaborative  Learning:  an  Effective  and  Enjoyable  Experience!  A Successful  Computer-Facilitated 
Environment  for  Tertiary  Students 

Robert  E.  Kemm,  The  University  f Me/boume,  Austra/ia;  He/en  Kavnoudias,  The  Universiy  of  Me/boume,  Austra/ia; 
Debbi  Weaver,  The  University  of  Me/boume,  Austra/ia,  Pau/Frit;(e,  The  University  of  Me/boume,  Austra/ia;  Nic/ro/as 
Stone,  The  University  of  Me/boume,  Austra/ia;  Nei/  Wi/iiams,  The  University  of  Me/boume,  Austra/ia 
An@tomedia:  A New  Approach  TO  Medical  Education  Developments  In  Anatomy 

David  M.  Kennecfy,  Monash  University,  Austra/ia;  Norm  Eir^enberg  The  University  of  Me/boume,  Austra/ia;  Chris 
Briggs,  The  University  of  Me/boume,  Austra/ia;  Ivica  Grkovic,  The  University  of  Me/boume,  Austra/ia;  Prisci/h  Barker, 
The  University  of  Me/boume,  Austra/ia 

An  evaluation  of  interactive  multimedia  designed  to  support  problem-based  learning  in  medicine 

David M.  Kenneepi,  Monash  University,  Austra/ia;  Norm  Eiyynberg  The  University  of  Me/boume,  Austra/ia;  Gregor  E. 
Kennecpi,  The  University  of  Me/boume,  Austra/ia 

Web  Ravelen  An  Infrastructure  for  Transforming  Hypermedia 

Andrew  Kens/er,  Grinned  Co//ege,  USA;  Samue/  Rebe/s^,  Grinned  Codege,  USA 

Transforming  Traditional  Curricula:  Enhancing  Medical  Education  Through  Multimedia  and  Web- 
based  Resources 

MiJke  Kepped,  The  University  of  Me/boume,  Austra/ia;  Gregor  Kennec/y,  The  University  of  Me/boume,  Austra/ia;  Peter 
Harris,  The  University  of  Me/boume,  Austra/ia 

Enhancing  Total  Patient  Management  Skills  in  Dentistry  with  Interactive  Multimedia  Simulation 

Mike  Kepped,  The  University  of  Me/boume,  Austra/ia;  Karen  Kan,  The  University  of  Me/boume,  Austra/ia;  Louise 
Brear/y  Messer,  The  University  of  Me/boume,  Austra/ia 

Developing  Commtmication  Skills  Using  INSPIRE  Negotiations 

Margaret  Kersten,  Car/eton  Univesrity,  Canada;  Gregoy  Kersten,  Concordia  University,  Canada 

A Cenceptual  Framework  for  Web-Based  Authoring  System 

Badru/  Khan,  The  George  Washington  University,  USA 

Offering  Online  Degree  Programs:  A Case  Study  Issues,  Challenges,  Successes,  and  Lessons  Learned 

Gerard  Kicku/,  University  of  St  Francis,  USA;  Laure/Jeris,  University  of  St  Francis,  USA;  Michae/ LaRocco, 
University  (f  St  Fran  as,  USA 
Developing  a Distance  Education  Infrastructure 

Gerard  KicJku/,  University  of  St  Francis,  USA;  Mark  Snodgrass,  University  of  St  Francis,  USA 
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Effects  of  Cognitive  Style  on  Web  Search  and  Navigation 
I^ung-Suri  Kim,  Umversify  of  Missouri  ^ Co/umbia,  Urtiud  States 

The  Effects  of  a Hypermedia  CAI  on  the  Academic  Achievement  of  Elementary  School  Students  in 
Korea 

Su-Hee  KJm,  Hoseo  University,  South  Korea;  Won_Hjun  Lee,  Hoseo  University,  South  Korea 

Content  Management  for  Web  Based  Learning 

Thomas  K/einber^er,  TSECMATH  AC,  Germa^;  Pau/Mii/ier,  University  of  Kaiserslautern,  Germanj 

Computer-Mediated  Communication  for  Distance  Education:  Developing  and  Teaching  a Second 
Language  Course  in  Academic  Reading 

Esther  K/ein-li/ohi,  Open  University  of  Israel,  Israel 

The  Major  Promise  of  Distance  Education  Is  On  Campus 

W.  K (Bi/I)  K/emm,  Texas  A<SbM  University,  USA 

Interactive  Ophthalmic  Pathology  Tutor 

Gordon  K K/intivorth,  Duke  University  Medical  Center,  USA;  Anthony  N Benson,  Duke  University  Medical  Center, 
USA;  Ann  L Busfyhead,  Duke  University  Medical  Center,  USA 

Function  Design  of  Adventure  Game  for  Education  Considering  Development  Psychological  Feature 
of  Elementary  School  Students 

Hyeongjoo  Koh,  The  Korea  National  University  of  Education,  Republic  of  Korea;  Seong  Sik  Kim,  The  Korea  National 
University  of Edr^cation,  Republic  of  Korea 

The  often  missing  but  essential  component  for  online  learning:  a learning  resource  catalogue 

Tony  Koppi,  University  of Nejv  South  Wales,  Australia;  Lisa  Hodgson,  University  of  New  South  Wales,  Australia; Jason 
Bcyiy,  University  of  New  South  Wales,  Australia 

Background  for  designing  net-based  learning  situations 

Mika  Kopi,  University  of  Jyvdsky Id,  Finland 

Using  Electronic  Portfolios  to  Demonstrate  Beginning  Teacher  Competencies 

Karen  Kortecamp,  The  George  Washington  University,  USA;  Libly  Ha//,  The  George  Washington  University,  USA; 
Burcu  Tunca,  The  Geoge  Washington  University,  USA 
Cognitive  Idea  Processor  — Modified  Mandal-Art 

Ka^masa  Ko;pki,  Konami  Co,  Japan;  Atsushi  Tsubokura,  Osaka  E/ectro  Communiadon  Univ.,  Japan;  Shogo  Hanma, 
Osaka  E/ectro  Communiation  Univ.,  Japan;  Noborer  Ashida,  Osaka  E/ectro  Communiadon  Univ,  Japan;  Katsuhide 
Tsushima,  Osaka  E/ectro  Communiadon  Univ.,  Japan 

Dilemma  of  Inquiry  and  Reflection  Through  and  With  Technology:  New  Directions  for  Research  in 
Post-Secondary  Teaching 

0/ga  Kritskaya,  Michigan  State  University,  USA;  Tony  C/ay.  Michigan  State  University,  USA 

Completing  Design  Concepts  for  Lifelong  Learning 

Huberta  Krite^enbeger,  University  of  Luebeck,  Germany;  Michae/ Hercqyg  University  of  Luebeck,  Germany 

Prospects  and  Limits  of  Conceptual  Models  for  WBT  Course  Production 

Huberta  Kridynbeger,  University  of  Luebeck,  Germary; Juegen  de  Wa//,  Univerdty  of  Luebeck,  Germary;  Michae/ 
Herayg  University  of  Lteebeck,  Germany 

Assessing  and  Training  Admunct  Faculty  with  Technology:  Enhancing  their  Classroom  Learning 
Karen  Knpar,  Metropo/itan  State  Co/iege  of  Denver,  USA 
Letting  the  Academic  Mirgrants  in  from  the  Cold:  Assisting  Adjunct  Faculty  with  Technology 
Enhancements  in  their  Classrooms 

Karen  Krrpar,  Metropo/itan  State  Go/iege  of  Denver,  USA 

Cognitive  Issues  for  Teaming  and  Performance  From  Multimedia  Interfaces:  Implications  for  Design 

Maria  Loma  Kunnath,  University  Of  Centra/ F/orida,  USA; 

Students'  Acceptance  of  Videoconferencing  in  the  Lecture  Context 
Patrick  Kum^  Swiss  Federa/ Insdtute  ofTechno/ogy  (ETH),  Swidyr/and 
Design  an  Enhanced  Virtual  Experiment  Environment  Using  Science  Process  Skills  on  WWW 

Li-Ping  Kuo,  Chung  Yuan  Chrisdan  Univerdty,  Taiwan,  K O.  G;  Da-Xian  Dong  Chung  Yuan  Chrisdan  University, 
Taiwan,  R.  O.  G;  Chang-Kai  Hsu,  Chung  Yuan  Chrisdan  University,  Taiwan,  K O.  G;Jia-Sheng  Heh,  Chung  Yuan 
Chrisdan  University,  Taiwan,  KO.G 

From  Intelligent  Tutoring  Systems  to  Intelligent  Learning  Material 

Jaakko  Ksrrhi/a,  University  of  He/sinki,  Fin/and;  Erkki  Sudnen,  University  of  Joensrm,  Fin/and 
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Analysis  of  messages  on  the  Onty  One  Earth  Club;  TV-based  collaborative  learning  site  on  the 
Internet 

Hanw  Kurvkami,  Kana:^a)i^a  Universi/y,  Japan;  Tatsf^a  Horita,  Toyama  Umversify,  Japan;  YfM  Yamancbi,  IbaraAi 
University,  Japan 

Life  Long  learning  with  The  LeamingStation.com 

Mars Labrecqne,  Tbe  LeamingS/ation.com,  Canada 

Hypermedia  And  Cooperative  Learning 

Yasine Lajiji,  Institut Dinformatique  Universite  De  Annaba,  A/gerie;  Tabar Bensbaa,  Institut Dinformatique  Universite 
DeAnnaba,  A/gene;  Yasine  LaJiJi,  Institut  D'informatiqne  Universite  DeAnnaba,  A/gene 

Aged  and  Disability  Care  training:  A CD-ROM  based  project 

Mark  Laid/er,  RMIT  University,  Australia 

Civilization  in  the  21st  Century 

David  F.  I^ng>,  Utah  State  University,  USA;  David  DeB^,  Utah  State  University,  USA;  Megan  Andrejv-Hobbs, 
Utah  State  University,  USA 

Wired  for  Learning 

Donna  Landin,  West  Virginia  Deaprtment  of Edusation,  United  States;  Roberta  Toy /or,  IBM,  United  States;  Lynn 
B/anqy,  Wbee/ing  Park  High  Ssbooi,  United  States;  Susan  A/kire,  Romn^  Junior  HIgb  Ssbooi,  United  States 

Redesigning  an  individualized  paper-based  course 

Stepbane  Lavoie,  SOFAD,  Canada;  Jo-Ann  Stanton,  SOFAD,  Canada 

Team-paced  versus  Self-paced:  The  Effects  of  Educational  Game  Design  on  Collaboration,  Learning 
and  Attitude  towards  Information  Technology 

Editb  Lajv,  University  (J" British  Columbia,  Canada;  Mana  K/ajve,  University  (JBntisb  Columbia,  Canada;  Cristina 
Conad,  University  of  Bndsb  Columbia,  Canada;  John  Meecb,  National  Research  Coundi,  Canada 

Problems  Of  Student  Communication  In  A Cross-Cultural,  International  Internet  Course  Setting 

John  LeBaron,  University  of  Massachusetts  Lo)ve//,  USA; Jyrki  Pu/kkinen,  University  fOu/u,  Finland;  Patrick  Scoi/in, 
University  of  Massachusetts  Lojve/i,  USA 

A Paradigm  Shift  in  the  Technology  Era  in  Higher  Education 

Car/ Lee,  Centra/  Michigan  University,  USA 

Delivering  Distance  Education  Using  Synchronous  Web  Audio  Technology 

Danie/  Y.  Lee,  Shippensbug  University,  USA 

Learner  Perceptions,  Learning  Styles,  and  Learning  Strategies:  In  an  Asynchronous,  Open,  Text- 
based,  Task-Oriented,  and  Gender-mixed  Web  Environment 

InsookLee,  SJong  University,  Repub/ic  f Korea 

Assured  Quality  Strategies  in  Web-based  Accredited  Programs 

Juefy  Lee,  University  f Centra/ F/orids,  USA 

Web-based  Foreign  Language  Learning  in  a Cooperative  Environment:  The  Effects  of  Ability  and 
Group  Composition  on  Reading,  Writing,  and  Listening  Comprehension 

Mijvha  Lee,  Pusan  Nationa/  University  of  Education,  Korea 

Professional  Development  through  Flexible  Delivery 

Gerak/ine  Lefoe,  University  f Wo/iongong  Austra/ia 

Training  Students  to  Be  Self-Learners  in  Educational  Hypermedia  and  Technology  Courses 

Any  S.  C Leh,  Ca/rfomia  State  University  San  Bernardino,  USA; 

Effective  Use  of  Client-Server  Software  and  Teaching  Strategies  in  Online  Courses 

Any  S.  C Leh,  Ca/rfomia  State  University  San  Bernardino,  USA;  Laura  Hoarse//  Young  Ca/rfomia  State  University 
San  Bernardino,  USA 

Integration  of  varied  cognitive  activities  in  a multimedia  environment : the  PHYLOGENE  example. 

Dominique  Lenne,  I nsdtut  Nationa/  De  Recherche  Pedagogique,  France;  Monique  Drprds,  Institut  Nationa/  De  Recherche 
Pe'c/agogique,  France; Jean-Franfois  Rodes,  lufm  De  Versai//es,  France;  Naorrm  Sa/ame,  Institrrt  Nationa/  De  Recherche 
Pedagogique,  France 

How  does  one  evaluate  the  effectiveness  of  electronic  learning  in  a South  African  context? 

Cedi  Leonard,  University  of  the  Western  Cape,  South  Africa 

Stepnet:  quality  newspapers  and  web-based  learning  in  the  Netherlands 

Jan  Lepe/tak,  Merdenhoff  Edrrcatief  The  Nether/ands 

Mathematical  Statistics  in  the  Net  - The  Pythagoras  Learning  Environment 

Jarkko  Levasma,  Tampere  University  of  Techno/ogy,  Fin /and 

Towards  a Design  Methodology  for  Distributed  Virtual  Laboratories 

Char/es  Levert,  Eco/e  Pofytechniqrre,  Canac/a;  Samue/  Pierre,  Eco/e  Pofytechnique,  Canada 
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Using  Technology  to  Teach  Linear  Algebra 

Lawrefjce  Levme,  S Sevens  of  Techno/qgy,  USA 

Towards  an  Intelligent  Authoring  Shell  on  Reuse 

Kam  Wing  U,  Unwersif  of Sonth  Anstra/ia^  Ansfra/ia;  Wamn  JameSj  University  of  South  Austraiiaj  Austra/ia 

Understanding  collaborative  learning  in  CMC:  A research  in  an  elementary  classroom. 

QingU,  University  of  TorvntOj  CanaSa 

A meta-analysis  of  computer  programming  on  cognitive  outcomes:  An  updated  synthesis 
Yuen-kmng  LiaOj  National  Hsinchu  Teachers  Co/kge,  Taiwan^  R.  O.  C. 

A meta-analysis  of  gender  differences  on  attitudes  toward  computers  for  studies  using  Loyd  and 
Gressard's  CAS 

Yuen-kuangLiaOj  National Hsinchu  Teachers  Coi/ege,  Taiivan,  K O.  C. 

The  Art  of  Technology:The  Technology  of  Art 

L(^  Lichtman,  Victoria  University,  Austra/ia 

T2  - Planning  and  Managing  Online  Courses  and  Learning  Resources 

Andreiv  Litchfe/d,  /[/acquarie  University,  Austra/ia 

The  status  of  online  professional  development  available  to  support  higher  education  staff. 

Andrew  Litchfe/d,  /Macquarie  University,  Austra/ia;  Peta  Spear,  ASacquarie  University,  Austra/ia 

e.leaming@mq:  Staff  Development  to  Integrate  Online  Media  into  Learning  and  Teaching  at 
Macquarie  University 

Andrew  Litchfe/d,  /Macquarie  University,  Austra/ia 

Supporting  the  millennium  teacher:  faculty  development  in  the  use  of  information  and 
communications  technologies 

A//ison  IJtt/^ohn,  University  of  Strathctyde,  Scot/and 

Exploring  the  Use  of  Virtual  Multimedia  Examinations  in  Teaching  and  Learning:  Results  from  the 
Fielding  Testing 

/\4in  Liu,  University  of  Texas-Austin,  USA;  Erini  Papathanasiou,  Ctyprus;  Winnie  Hao,  Univ.  of  Texas- Austin,  USA; 
John  Kappe/man,  University  of  Texas-Austin,  USA 

Custom-Authored  Hypermedia  Exercises  for  Use  in  College  Classrooms 

AmyLobben,  Centra/ /[dichigan  University,  USA 

Groupware  Tools:  Web  File  System  Allows  Students  to  Access  Information  Anywhere,  Anytime 

Tyrone  I^bo,  SiteScape,  Inc.,  Canada 

Learner  as  designer-producer:  Physical  and  health  education  students  experience  Web-based  learning 
resource  development 

Lori  Loc/^er,  University  of  Woi/ongong  Austra/ia;  Yvonne  Kerr,  University  ofWo/iongong  Austra/ia 

The  development  of  an  on-line  learning  community  of  physical  and  health  education  professionals. 

Lon  Locker,  University  of  Woi/ongong  Austra/ia;  Gregg  Row/and,  University  of  Woi/ongong  Austra/ia;  John  Patterson, 
University  of  Woi/ongong  Austra/ia 
Intensive  Ecommerce  Business  Training  Course 

Joe  Luca,  Edith  Cowan  University,  Austra/ia;  Larry  Quick,  Dow  Digita/,  Arrstra/ia;  Renee  E//is,  DowDigita/, 
Austra/ia 

Teaching  Information  Literacy  to  Undergraduates:  Stand-alone  Courses  or  Across  the  Curriculum? 

Barbara  lyman.  Southwest  Texas  State  University,  USA 
Using  Virtual  Reality  in  Teaching  Secondary  School  Physics 

Bruce  i^a.  City  University  of  Hong  Kong  Hong  Kong' Peter  ]\dak,  STFA  Tam  Pak  Yu  Co//^e,  Hong  Kong 
Distance  Learning  via  Interactive  Video  Conferencing  for  the  Small  College:  Pitfalls  and  Possibilities 

Kate  /\dacDona/d,  I\4assachr4setts  Co/iege  of  Pharmacy  <&Hea/th  Sciences,  USA;  J\diche//e  Chapman,  /\dassachr4setts 
Co/iege  of Pharmay  (tirHea/th  Sciences,  USA;  Francis  Rogers,  Massachusetts  Co/iege  of  Pharma y cir  Hea/th  Sciences, 
USA;  Lynne  Sy/via,  Massachusetts  Co/iege  ofPharmay  Hea/th  Sciences,  USA;  James  Mqy,  Massachrrsetts  Co/iege  of 

Pharmay  itir  Hea/th  Sciences,  USA 

A Game-play  Intranet-based  Multimedia  Teaching  Support  System  for  Information  Systems  Training 

Hing  Yin  Mak,  Hong  Kong  Baptist  University,  Hong  Kong  SAR,- Andrew  Ma/iard,  Brunei  University,  United  Kingdom 

A Web-based  Hypermedia  Pedagogical  Course 

A //a  N Makarova,  St-Petersburg  State  Pedagogica/  Univeristy,  Rs/ssia 

Multimedia  Performance-Based  Language  Assessment:  The  Computerized  Oral  Proficiency 
Instrument  (COPI) 

Va/erie  ida/abonga.  Center  for  Appded  Lingrdstics,  USA;  Dory  Kenyon,  Center  for  App/ied  Linguistics,  USA 
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Combining  Instructional  Models  and  Enabling  Technologies  to  Embed  Best  Practices  in  Course 
Instructional  Design 

PauJ M.  Ma/onej  Unwenit)/  (^UPater/oOj  Canaih;  Catherine  F.  Sch^/er,  Unit^rsity  (^IFater/oOj  Canada;  IC/n/an  Rossner- 
Merri/l  University  of  WaterJoOj  Canada 

Phase  Theoiy:  A Teleological  Taxonomy  Of  Web  Course  Management  For  Distance  Education 

Bruce  Mannj  Memoria/ University  of  Neufound/and,  Canada 

Divided  Attention  in  Multimedia  Learning 

Bruce  Mann^  Memorial  University^  Canada;  Paul  Nejvhorrsej  Edith  Cojvan  Universityj  Australia;  Jererr^  Pagram^  Edith 
Cojvan  University^  AustraJia;  A/istair  Campbell  Edith  Cojvan  University^  Australia 

Info  Pursuits 

Kerrie  Manning  Leichhardt  High  School  Australia;  Rosemary  Ward,  UTS,  Australia;  Tor;y  Ward,  Aufard  Consulting 
Enterprises,  Australia 

Reconfiguring  the  Academic  Library  for  the  New  World  of  Learning 

James  Marcum,  Co/Zege  of  Staten  Is/and,  CUNY,  USA 

To  teach  remotely  by  controlling  the  didactic  interaction  of  a student  with  a multiapplication 
environment 

Nathalie  Masserrx,  Auvergne  University  Institute  of  Teachers  Training  France 
Content  Area  Integration:  A Step  Toward  Emiratisation 

Reo  H.  McBride,  Dubai  Women's  College,  United  Arab  Emirates;  Michael  Ford,  Dubai  Women's  College,  United  Arab 
Emirates;  David  Thomson,  Dubai  Women's  College,  United  Arab  Emirates;  Ritymond  Yars/ty,  Drrbar  Women's  College, 
United  Arab  Emirates;  Lada  Hajvker,  Dubat  Women's  College,  United  Arab  Emirates 
TRIAD:  Tactical  Readiness  Instruction,  Authoring,  and  Deliveiy 

James  McCarthj,  Sona/ysts,  Inc,  USA;  Michael  Hannafn,  University  of  Georgia,  USA;Terty  Stanard,  Klein 
Associates,  USA;  Jack  Wajne,  Sona/ysts,  Inc,  USA;  Paul  Radtke,  Nava/ Air  Wa  fare  Center  Training  Systems 
Division,  USA 

The  Inadvertent  Branding  of  a University  Web-based 

Kevin  McC/ain,  University  f Virginia,  USA 

Evaluating  the  Influence  of  Web  Based  Seminar  Preparation  for  Undergraduate  History  Students 

Pauline,  McCormack,  University  of Nemast/e  upon  Ijne,  UK;  Dr  Dona/d  A,  Spaeth,  University  of G/asgow,  UK 

Using  Instructional  Technology:  What  do  the  students  think? 

Cynthia  McCourt,  GeorgeMason  University,  US;  Sharon  A/ayne  Widmcyer,  George  Mason  University,  US 

Agents  For  Education:  the  Problem  Of  Synthetic  Personality 

Peter  McKenna,  Livepoo/ Hope  University  Co//ege,  Eng/and;  Atrf  Waraich,  Uvepoo/ Hope  University  Co/iege,  Eng/and 

The  Total  Student  Experience 

PardMckty,  NextEd,  Austra/ia 

Preparing  Learners  for  Self-Regulation  and  Professional  Practice:  Design  Recommendations 

Catherine  McLough/in,  University  of  Nerv  Eng/and,  Austra/ia;  Barbara  McCarthy,  University  of  New  Eng/and, 
Austra/ia 

Developing  Professional  Skills  And  Competencies  In  Tertiary  Learners  Through  On-Line  Assessment 
And  Peer  Support 

Catherine  McLorrgh/in,  University  of  New  Eng/and,  Arrstra/ia; Joe  Luca,  Edith  Cowan  University,  Ar/stra/ia 

Fostering  teacher  inquiry  and  reflective  learning  processes  through  technology  enhanced  scaffolding 
in  a multimedia  environment 

Catherine  McLough/in,  University  of  New  Eng/and,  Arrstra/ia;John  Baird,  The  University  ofMe/boume,  Austra/ia;  Keith 
Pigdon,  The  University  of Me/boume,  Austra/ia;  Mara/yn  Woo/y,  The  University  of  Me/bo rrme,  Austra/ia 
Supporting  constructivist  learning  through  learner  support  on-line 

Catherine  McLorrgh/in,  University  of  New  Eng/and,  Austra/ia;  Koos  Winnips,  University  ofTwente,  Nether/ands;  Ron 
0/iver,  Edith  Cowan  University,  Austra/ia 

Developing  Web-based  learning  Strategies:  a comparison  between  the  Web  and  traditional  learning 
environments 

Mark  McMahon,  Edith  Cowan  University,  Arrstra/ia 

Courseware  Management  Tools  and  Customised  Web  Pages:  Rationale,  Comparisons  and  Evaluation 

Mark  McMahon,  Edith  Cowan  University,  Austra/ia;  Joe  Luca,  Edith  Cowan  University,  Austra/ia 

The  distance  educator's  tool  box:  An  overview  of  technologies  for  on-line  learning 

Thomas  Fox  McManus,  Saginaw  lYaZ/ty  State  University,  USA 

High  Tech  Classrooms-Going  The  Distance  with  Distance  Education 

Patricia  McNames,  Indiana  University  Southeast,  USA 
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Developing  new  roles  for  university  teachers:  Grass-roots  local  staff  development  serviced  by  top- 
down  investment  and  policy 

Came/ McNaughf,  RMIT  Umven//y,  Amtra/ia;  Pat  J Ken aeefyj  EMIT  U nwers/fy,  Australia 

Collaborative  links  between  distributed  databases.  Developing  a framework  for  an  inventory  of 
computer-facilitated  learning  (CFL)  materials  across  the  higher  education  sector  in  Australia 

Came/ McNang/)/  EMIT  Unwersi/yj  Aus(ra//a;  Rob  Pb///^s,  Mnn/ocb  Unmrs/'/yj  Ans/ra//a;  Darien  Rossi/er, 
^neens/ant/  Unirersi/y  o/Tecbno/o^,  Anstra/ia;  Jennj  l/K/nn,  Queens/ane/  Universi/y  ofTeebno/ogy,  Austra/ia 

Dynamic  Databases  as  Knowledge  Construction  and  Communication  Tools  in  Online  Courses 

Sara  McNei/,  Univen/iy  of  Honsion^  USA;  Bemart/ Robin,  U niveni/y  of  Hom/on,  USA 

Students'  Perceptions  of  Self-Change  During  an  Online  Course 

Sara  McNei/,  UnirersiiyofHonsion,  USA;  Bemani  Robin,  University  of  Hous/on,  USA 

An  on-line  introduction  to  quantitative  methods 

Moira  McPherson,  LakeheaS  University,  Canat/a;  Wi//iam  Montepare,  Lakehead  University,  Canada 

The  third  time  is  the  charm:  the  evolving  nature  of  web  design 

Martha  Meacha/n,  St.  Edjvards  University,  USA 

Supporting  Collaborative  Teleleaming  Research  using  Server  Logs 

^vind  Meistad,  University  of  Bergen,  Norway;  Barbara  Wasson,  University  of  Bergen,  Norway 

Evaluating  Hypermedia  Systems,  Models  and  Methodologies  in  Education 

Emi/ia  Mendes,  The  University  ofArrck/and,  New  Zea/and 

A Study  of  the  Use  of  Online  Supported  Learning  Facilities  at  Charles  Sturt  University 

John  Messing  Char/es  Strrrt  University,  Arrstra/ia 

The  price  a distance  education  student  pays  when  using  electronic  learning  resources 

John  Messing  Char/es  Sturt  University,  Arrstra/ia 
Principles  of  cognitive  evaluation  for  an  educational  CD-ROM  for  history 

Christina  Metaxaki-Kossionides,  University  of  Thrace,  Greece;  E/evtheria  Gonida,  University  of  Thrace,  Greece;  Stavrorda 
Lia/iorr,  University  of  Athens,  Greece;  Georgios  Korrroupetrvg/orr,  University  of  Athens,  Greece 

Authoring  Multimedia,  Designing  Animations  for  Physics  Education 

Dona/d J.  Met!(f  University  of  Winn  J>eg  Canada 

Dynamic  Communication  Layer  Between  Virtual  Laboratory  and  Intelligent  Agents 

Kaufnann  Meud/a,  UniversitT  de  Sherbrooke,  Canada;  Roger  Nkambou,  UniversiTT  de  Sherbrooke,  Canada 

One  teacher,  one  text,  four  lessons:  a case  study  of  collaboration  and  innovation  in  a multilingual 
New  Zealand  classroom. 

Drrncan  M^er,  Long  Bay  Co/iege,  New  Zea/and;  G/enda  Jacobs,  UNI  TEC,  New  Zea/and 

Description  of  a web -based  program  suitable  for  easy  adaptation  for  on-line  delivery  of  many 
educational  resources. 

Geoffr^  M^er,  University  of  Western  Arrstra/ia,  Austra/ia 
Telematic  Education  in  Nursing  Education 

Sa/o/ne  Meyer,  University  of  Pretoria,  South  ^rica;  Sa/o/ne  M^er,  University  of  Pretoria,  South  Afica 
A Rule-based  Method  to  Shift  between  Learning  Protocols 

YongwuMiao,  GMD  - Ceman  Nationa/ Research  Center  for  Infomation  Techno/ogy,  Gemanj;JdrgHaake,  GMD - 
Geman  Nationa/  Research  Center  for  Infomation  Techno/ogy,  Cemany;  Raf Stein/net:^  GMD  - Ceman  Nationa/ 
Research  Center  for  Infomation  Techno/ogy,  Gema^ 

Design,  Delivery,  and  Evaluation  of  Online  Courses:  A Primer 
Jero/d  Mi//er,  United  States  Internationa/  University,  USA 
Discovering  xyAlgebra:  Intelligent  Interactive  Internet  Instruction 
John  Mi//er,  The  City  Co//ege  of  CUNY,  USA 

Technology  Acculturation  Among  Adolescents:  The  School  And  Home  Environments 

Les/ie  /ili//er.  Rice  University,  USA;  Heidi  Schweingruber,  Rice  University,  USA;  Christine  Brandenburg  Rice 
University,  USA 

Problem-based  Learning  and  Flash  4.0:  An  Experiment  in  Science  Education 

Les/ie  Mi//er,  Rice  University,  USA;  Janice  Mayes,  Rice  University,  USA;  Donna  Smith,  Rice  University,  USA 

On-Line  Education  Using  Video  Broadcasting  Delivered  from  Perth  Campus,  Algonquin  College 

Maike  Euiken  Mi//er,  A/gonqrdn  Co//ege,  Canada;  Dave  Osborne,  A/gonqrdn  Co/iege,  Canada;  Ian  McComick, 
A/gonqrdn  Co/iege,  Canada 

Unlocking  the  Gates  of  the  Kingdom:  Designing  Web  Sites  for  Accessibilityity 

Steven  C.  Mi//s,  University  of  Kansas,  USA 
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Gender  and  instructional  use  of  computers 

Anancia  Mitra,  U^ake  Forest  Unwemfy,  USA 

From  Brawn  to  Brain:  Towards  an  Emerging  MindsK>n  Approach  in  Integrated  Distributed  Learning 
Environments  (IDLE 

Marya  Mononen-Aa/toner,  Unmrsity  of  He/sinki,  Fm/anc^;  Seppo  Te//a,  Unmrsity  of  He/smkij  Fm/ard 

Using  Scenarios  to  create  a CAL  Design  Method 

MtM/e  Mont^omeiy  Masters,  Umrersity  o/G/asgow,  UK 

TLC:  Teachers’  And  Learners’  CoUaborascope  — A Platform  For  Anatysing  And  Evaluating  Online 
Education 

Micbe/k  Mon tgomejy  Masters,  Un/rers/ty  qpG/asgojr,  UK;  Stewart  Macnei%  Untuersi^  of  Glasgow,  UK;  Prof Ma/co/m 
Atkinson,  University  of  Glasgow,  UK 
Educational  Multimedia  Systems  for  Students  with  Autism 

Daniel  Moore,  Leeeis  Metropolitan  Universi^,  UK;  Panl McGrath,  Leeds  Metropolitan  Uniuersi^,  UK;fobn  Thorpe, 
Leeds  Metropolitan  Universi^,  UK 

Meaningful  Design  for  Meaningful  Learning:  Applying  Cognitive  Theory  to  Multimedia 
Explanations 

Roxana  Moreno,  University  of  California,  Santa  Barbara,  USA;  Richard  Mayer,  University  of  California,  Santa 
Barbara,  USA 

Life-Like  Pedagogical  Agents  in  Constructivist  Multimedia  Environments:  Cognitive  Consequences 
of  their  Interaction 

Roxana  Moreno,  University  of  Cal fomia,  Santa  Barbara,  USA;  Richard  Mayer,  Universi^  of  Calzfomia,  Santa 
Barbara,  USA;  fames  Lester,  North  Carolina  State  University,  USA 

Why  don't  we  weight  them? 

Jack  E.  Morin,  The  University  of  Calgary,  Canada;  Janice  Bakal,  The  Universi^  of  Calgary,  Canada 

The  third  dimension  for  world  of  knowledge:  an  acceptable  way  to  get  in. 

Mikhail Moroe^v,  Mari  State  Thechnical  Universi^,  Rjzssia;  Aleksander  Markov,  Mari  State  Thechnical  Universi^, 
Rjzssia 

Using  Mobile  Computer  Technology  and  Internet  Tools  to  Promote  Constructivist  Teaching 

Chrystalla  Mouz;a,  Columbia  University,  USA 

Distributed  Repository  of  Programming  Examples 

Tomas t;Mrddner,  Acadia  University,  Canada;  F^ic^  Shiv,  Acadia  University,  Canada 

Finding  out  the  intention  of  a user  of  Educational  Hypermedia 

Duncan  Midlier,  Leeds  Metropolitan  University,  England;  David  Hobbs,  Leeds  Metropolitan  University,  England;  David 
Moore,  Leeds  Metropolitan  University,  England 

Learning  Legal  Case  Solving  with  the  Computer  Program  PROSA 
Antoinette f Munfewerff,  University  of  Amsterdam,  The  Netherlands 
Role  Play,  Discussion,  and  Analysis:  Project-based  Learning  in  a Web-based  Course 
Karen Murpfy,  Texas A<2FM  University,  USA;  Yakut  Ca^,  Texas AeFM  University,  USA 

Representing  Oral  Traditions  through  Interactive  Multimedia  - an  example  from  Hindustani  Classical 
Music 

John  Myers,  Simonas  Rock  College,  USA 

Difficulties  Students  are  Having  with  Cognitive  Processes  in  Problem  Based  Learning  Environments. 
SomNaidzr,  The  University  of  Melbourne,  Australia;  Iain  McApine,  The  University  of  Melbourne,  Australia 

Scafifolding  Student  Learning  in  Information-Dense  Technology-Enhanced  Teaching  and  Learning 
Environments 

SomNaidu,  The  University  of  Melbourne,  Australia;  Mike  Keppell,  The  University  ofMelbozzme,  Aizstralia;  Gregor 
Kenneety,  The  University  of  Melbourne,  Australia 

Ability  grouping  for  teaching  computer  literacy  classes 

Michio  Nakanishi,  Osaka  University,  JAPAN;  Akira  Harada,  Osaka  University,  JAPAN 

Scenario  Building  to  Design  a Distance  Learning  Program 

Usa  Neal,  EDS,  USA 

Using  a Web  Site  to  Increase  Student  Motivation  to  Write  and  Enhance  the  Reading/ Writing 
Relationship 

Susan  Nelson,  Elm  Street  Elementary  School,  USA;  Harrison  Yang  State  University  of  New  York  at  Oswego,  USA 

Learner  Engagement  in  Mixed-Mode  Delivery  Models 

John  C.  Nesbit,  Technical  University  of  British  Colzzmbia,  Canada 
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Increasing  interaction  through  computer-mediated  communication  in  distance  education 

Kjpok-chi  NG,  The  Open  University  of  Hong  Kong,  Hong  Kong  SAR,  China 

Initiating  and  Sustaining  Collaboration  in  Technology:  Critical  Elements 

Pau/a Nicho/s,  Lansar  University,  U.S A.;  StephenieYearwooei,  Lamar  University,  USA 

SMILe:  Giving  Internet  Instruction  a Human  Face 

Matthew  Nickerson,  Southern  Utah  University,  USA 
TELSI,  Web-based  software  for  collaborative  learning 

Gracie/a  Nie/sen,  Norw^ian  State  CenterforASu/t  Education,  Norway;  Boo  Hever,  Ma/omas  Vuxen^mnasium, 
Sweden;  Aase  Steinmet^  Undervisningsministerium,  Denmark 
Development  of  Speech  Input  System  for  Web-based  Courseware 

Ryuichi  Nisimura,  Nara  Institute  of  Science  and  Techno/ogy , Japan;  Shfi  Krfita,  Nagoya  University,  Japan;  Kai^rrya 
Takeda,  Nagoya  University,  Japan;  Fumitada  Itakura,  Nagoya  University,  Japan 
Integrating  Learning  agents  in  a virtual  laboratory 

Roger  Nkamhou,  UniversitT  de  Sherbrooke,  Canada;  Yan  Laporte,  UniversitT  de  Sherbrooke,  Canada 
SGML  Based  Course  Development:  Balancing  Instructional  Design  and  Technological  Requirements 
So/vrgNorman,Open  Learning  Agency,  Canada 

A Framework  for  Generating  Non-trivial  Interactive  Mathematical  Exercises  in  the  Web:  Dynamic 
Exercises 

OssiNykdnen,  Tampere  University  ofTechno/ogy,  Fin/and 

CoCoA2:  Computer  Supported  Collaborative  Language  Learning  Environment  Based  on  Online 
Proofreading 

Hiroaki  Ogata,  Tokushima  University,  Japan;  Yoshiaki  Hada,  Tokushima  University,  Japan;Yoneo  Yano,  Tokushima 
University,  Japan 

A Case  Study  of  Student  Tool  Use  During  Hypermedia-Based  Problem  Solving 

Kevin  O liver,  Virginia  Tech,  USA 

The  Instructional  Portal  Project 

Kevin  Oliver,  Virginia  Tech,  USA 

Exploring  the  Development  of  Students’  Generic  Skills  Development  in  Higher  Education  Using  A 
Web-based  Learning  Environment 

Ron  0/iver,  Edith  Cowan  University,  Australia;  Jan  Herrington,  Edith  Cowan  University,  Australia;  Catherine 
McLough/in,  Unibersity  of  New  England,  Australia 

The  ANTA  Flexible  Toolbox  Project:  Developing  Sustainable  and  Scalable  On-Line  Learning 
Materials  for  Vocational  Education  and  Training 

Ron  0/iver,  Edith  Cowan  University,  Australia;  Stephen  Towers,  Queensland  Open  Learning  Network,  Australia;  Nick 
Pear/,  Austra/ian  Nation  a/  Training  Authority,  Austra/ia 
Information  and  Communications  Technology  Literacy  — Getting  serious  about  IT 

Ron  0/iver,  Edith  Cowan  University,  Austra/ia;  Stephen  Towers,  Queens/and  Open  Learning  Network,  Austra/ia;  He/en 
0/iver,  Edith  Cowan  University,  Austra/ia 

Assessment  of  online  interaction:  helping  or  hindering  the  goals  of  educators  and  learners? 

Meg  O'Rei/^,  Southern  Cross  University,  Austra/ia 
Commercialization  of  Higher  Education 
Dan  OShea,  YouthStream  Media  Networks 
Integrating  and  Adapting  Multimedia  Resources 

Mary  Pan ko,  UNTTEC  Institute  of  Techno/ogy,  New  Zea/and 
DISCETECH-BIMBOTECH  an  experimental  project  for  introducing  innovative  technology  within 
the  Italian  school  system 

Pao/o  Pao/ini,  Po/itecnico  di  Mi/ano,  Itafy;  Pier/uigi  De//a  Vigna,  Po/itecnico  di  Mi/ano,  Ita/y;  Francesca  Co/iina, 

Po/itecnico  di  Mi/ano,  Itafy;  Pao/o  Magatti,  Po/itecnico  di  Mi/ano,  Itafy;  Si/via  Moio/a,  Po/itecnico  di  Mi/ano,  Ita/y 
Added  value  services  on  the  Greek  Web  for  schools 

Giorgos  Papadopou/os,  He//enic  Pedagogica/  Institute,  Greece;  Micha/is  Karamanis,  He/ienic  Pedagogica/  Institute,  Greece; 
Petros  Roussos,  He/ienic  Pedagogica/  Institicte,  Greece 
Explora:  An  Open  Virtual  Campus 

Gi/bert  Paquette,  Te/e-universite,  Canada;  I/eana  De /a  TJa,  Te/e'-universite]  Canada;  Aude  DuJresne,  Universitede 
Montrea/,  Canada 

Multiple  Representations  in  Interactive  Learning  Environments:  two  Case  Studies 

A/an  Parkes,  Lancaster  University,  UK;  Constantinos  Drakakis,  Lancaster  University,  UK;  George  Marse/os,  Lancaster 
University,  UK 
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A Multi-Method  Strategy  for  the  Assessment  of  Adult  Literacy  Software 

Graham  Passmore,  University  of  Toronto,  Canaria;  Wayne  Leivchuk,  McMaster  Universi^,  Canaria;  Ceoffi^  Rockmii, 
McMaster  University,  Canaria;  Sanefy  Shaw,  McMaster  University,  Canaria 

World  Wide  Web  Course  Delivery  in  Developing  Countries:  A South  African  Perspective 

Shaun  Father,  University  of  Durban  Westvii/e,  South  Africa;  Geoff  Erwin,  University  of  Durban  Westvi/ie,  South  Africa 

Students'  engagement  behaviours  in  computer-assisted  mass  lectures 

Nopphoi  Pausawasdi,  James  Cook  University,  Australia;  Lunette  Hemierson,  James  Cook  University,  Australia 

The  Essen  Learning  Model  - a Multi-Level  Development  Model 

Jan  M.  Pawlowski,  University  of  Essen,  Germany 

Learning  Experiences  in  New  Learning  Environments 

Heli  Pekkala,  University  of  Jyvas^ la,  Finland 
Online  Conferencing:  Variations  on  Structure  and  Participation 

Hsinyi Peng,  University  of  Missouri-Columbia,  USA;  Gail Fit^erald,  University  of  Missouri-Columbia,  USA;  Louis 
Semrau,  Arkansas  State  University,  USA 

A Constructivist  Framework  for  WWW-based  learning  Environments  that  Combine  Streaming  Media 
and  WWW  functionalities 

Oscar  Peters,  University  of  Twente,  The  Netherlands;  Bety  Collis,  UniversiyofTwente,  The  Netherhnds 
Choosing  the  Tools  Wisely:  Learning  the  Lessons  the  Hard  Way 

Denise  Peterson,  South  Dakota  State  Universiy,  US;  Loye  Romereim-Holmes,  South  Dakota  State  Universiy,  US 
Microsoft  Office  as  a Learning  and  Teaching  Tool 

Craig  Place,  Acadia  Universiy,  Canada;  Michael  Shaw,  Acadia  Universiy,  Canada 
An  XML-based  Approach  for  Web-based  Self  Assessment 

Nathalie  Poerwantoro,  FemUniversitaet Hagen,  Germa^;  Bemd Kraemer,  FemUniversitaet Hrgen,  Germany 
A Learning  Experiment  Using  A&O  Open  Learning  Environment 

Seppo PoJfolainen,  Tampere  Univ.Technology,  Finland; Heli R/rokamo,  Tampere  Univ.Technology,  Finland;  Ossi 
Nykdnen,  Tampere  Univ.  Technology,  Finland;  Kaja  Pesonen,  Tampere  Univ.  Technology,  Finland;  Hesa-Matti 
Hartikainen,  Tampere  Univ.  Technology,  Finland 

Students'  expectations  and  perceptions  of  efficacy  for  self-regulated  learning  in  mediated  learning 
environments:  Partial  results  of  an  evaluation 

Laura  Helena  Porras,  U niversickd  de  las  Americas,  Puebla,  Mexico 

Getting  Published  in  the  Field  of  Instructional  Technology 

Robert  V.  Price,  Texas  Tech  Universiy,  USA;  Nany  Maushak,  Texas  Tech  Universiy,  USA 

An  Online  Model  For  Precalculus 

Hari  Pulapaka,  Valdosta  State  Universiy,  USA;  Denise  Reid,  Valdosta  State  Universiy,  USA 

The  TOMUS  model  of  multimedia:  an  empirical  investigation 

Helen  C.  Purchase,  The  Universiy  of  J2ueensland,  Australia;  Daniel  Naumann,  The  Universiy  of  Q^^eensland,  Australia 

IMAGUS:  An  Environment  To  Design  Video  Applications  For  Digital  Libraries 

Andre  Luis  Alice  Raabe,  Ponti/icia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil;  Luaa  Maria  Martins  Giraffa, 
Ponti/icia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil 

Narrative  and  Visual  Design  of  Web-based  Learning  Materials 

Andy  Reiiy,  The  Open  Universiy,  United  Kingdom;  Alison  George,  The  Open  Universiy,  United  Kingdom 

Art,  Poetry,  and  Instructional  Media 

Thomas  Reinart(f  Rosemorrnt  High  School,  USA;  Brad  Hokanson,  Universiy  of  Minnesota,  USA 

Didactical  Activities  and  Strategies  in  the  Use  of  WWW-based  Course-Support  Environments:  Design 
Guidelines  for  Instructors 

Elka  Remmers,  Universiy  of  Twente,  The  Netherlands;  Bety  Collis,  Universiy  of  Twente,  The  Netherlands 

From  the  "Invisible  Web"  to  Google:  Techniques  for  Improving  Web  Searching 

Jrrdi  Rjpman,  Georgia  Southern  Universiy,  USA;  Randal  Carlson,  Georgia  Southern  Universiy,  USA;  Elizabeth 
Downs,  Georff a Southern  Universiy,  USA;  Kenneth  Clark,  Georgia  Southern  Universiy,  USA 

Moving  an  Australian  Dual  Mode  University  to  the  Online  Environment:  A Case  Study 

Shirly  Reushle,  The  Universiy  of  Southern  Queensland,  Australia; Jacquelin  McDonald,  The  Universiy  of  Southern 
Queensland,  Australia 

Learning  Code 

Sur^anne  Rhodes,  Human  Code,  USA 

An  Empirical  Test  On  The  Use  Of  Directed  Wodd  Wide  Web  Navigation  Guidance 

David  Ribeiro  Lamas,  Universidade  Fernando  Pessoa,  Portugal; Jennifer Jerrams-Smith,  Universiy  of  Portsmouth,  UK; 
David Heathcote,  Bournemouth  Universiy,  UK;  Feli^  Ribeiro  Gouveia,  Universidade  Fernando  Pessoa,  Portrrgal 
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A Java- Based  Hypermedia  Framework  for  Delivering  Fluid  Power  Training  via  the  Internet 

Thomas  Rkbarcb,  Unwersit^ro/Bath,  UrtifedKingiiom;  Ecimuncn-laghes,  Unwersityo/Bath,  United  Kingdom;  Derek 
Tii/^f  Unwersiiyo/Baih,  United  Kingdom 

The  Affordances  and  Constraints  of  Asynchronous  Learning  Networks:  Looking  at  Interaction  in  an 
Online  Learning  Environment 

Jennifer  Richardson,  Empire  State  Co/iege,  USA;  Craig  Tnnwa//,  Empire  State  Co/iege,  USA;  Caro/  Cameva/e,  Empire 
State  Co/iege,  USA 

Evaluating  Software  for  the  New  Milleniiun:  An  Example  from  Project  Links 

Jennfer  Richardson,  University  at  A/bany,  USA;  Karen  Swan,  University  at  A/ba^,  USA;  Dianna  Newman, 
University  at  A/bany,  USA 

How  to  Make  the  Most  of  Online  Interaction 

Jennfer  Richardson,  University  at  A/bany/ SUNY,  USA;  Karen  Swan,  University  at  A/bany/ SUNY,  USA 

CAMPUS:  An  innovative  Web-based  approach  for  simulative,  flexible,  case-oriented  education  in 
Medicine 

Jens  Riede/,  University  of  Heick/berg,  Germany;  Reiner  Singer,  University  of  Heide/berg,  Germany;  Fran:(^JosfLeven, 
University  of Heide/berg  Germary;  Heiko  Geiss,  University  of  Heide/berg  Germary;  Bnrkhard  Toenshoff,  University  of 
Heide/berg  Germary 

Students  as  Producers:  Changing  the  Way  we  Teach  and  Learn 

Gad  Ring  University  of  F/orida,  USA;  Sebastian  Foti,  University  of  F/orida,  USA;Me/issa  /\4cCa//ister,  University  of 
F/orida,  USA;  Tamara  Pearson,  University  of  F/orida,  USA;  Ebraheem  A/ka:(eemi,  University  of  F/orida,  USA 

Meaningful  Learning  in  a Virtual  University  by  Using  High  Speed  Internet  Coimections 

Riitta  Rinta-Fi/pprda,  CERN,  Swit!(er/and;  Anne-/\4aria  Korhonen,  CERN,  Swit!(er/and 

The  GPB  project;  A successful  PHD  seminar  using  Videoconference  in  real  time  and  a list  to 
communicatebetween  the  classes 

C/aire  Roberge,  University  of  Quebec  in  /[dontrea/,  Canada 

Teaching  Image  Ethics  in  Multimedia  Programs 

Paida  Roberts,  University  f South  Austra/ia,  Austra/ia; Jenry  Webber,  University  of  South  Austra/ia,  Austra/ia 

Report  on  Learning  Technology  Standards 

Rably  Robson,  Oregon  State  University,  USA 

Evaluating  the  effects  of  web-based  collaboration  in  a distance  education  program 

Paid Rodes,  Indiana  University,  B/oomington,  USA;  Hajin  Chung  Indiana  University,  B/oomington,  USA;  Carrie 
Chapman,  Indiana  University,  B/oomington,  USA;  Dennis  Knapcryk,  Indiana  University,  B/oomington,  USA 
Instructional  Design  Agents  - An  Integration  of  Artificial  Intelligenceand  Educational  Technology 
Erika  Rogers,  Ca/ifomia  Po/ytechnic  State  University,  USA;  Caro/  Scheftic,  Ca/fomia  Po/ytechnic  State  University, 

USA;  Emi/io  Pass/  Ca/fomia  Pofytechnic  State  University,  USA;  Sharon  Lanaghan,  Ca/fomia  Po/ytechnic  State 
University,  USA 

Automating  Online  Course  Evaluations  Over  the  WWW 

Guido  Rdf /ing  University  f Siegen,  Germary;  Bemd Freis/eben,  University  of  Siegen,  Germary 

Mathematical  Abilities  and  Development  of  Mathematical  Word  Problem-Solving  Skills  in  a 
Technology-Based  Learning  Environment:  Methods  and  Main  Results 

He/i  Ruokamo,  Tampere  University  of  Techno/ogy,  FINLAND 

The  design,  development  and  implementation  of  education  in  Edubox  (r) 

Eden  Rusman,  Open  University  of  the  Nether/ands,  The  Nether/ands;  Peter  B.  S/oep,  Open  university  of  the  Nether/ands, 
The  Nether/ands 

Is  computer-mediated  communication  (CMC)  really  open  and  flexible? 

Ma/co/m  Ryan,  University  of  Gr^nwich,  UK 

You  CAN  Get  There  From  Here  - Developing  Assessments  from  Skill  Standards 

Peter  Safund,  Bedevue  Community  Co/iege,  USA;  Lauren  Hebert,  The  Chauncy  Group  Internationa/,  Inc,  USA 

A distributed  notebook  system  for  vocabulary  learning 

Tomoko  Sagisaka,  University  of  E/ectro-Communications,  Jrpan;  Tsutomu  I\dunakata,  Tamagawa  University,  Japan; 
Katsrya  Hako:(aki,  University  of  E/ectro-Communications,  Japan 

Collaborative  Hybrid  CD-ROM/Intemet  in  a "Learning  by  Doing  and  Creating"  environment 
Nico/etta  Sa/a,  Acaderry  of  Arch.  University  of Ita/ian  Switi^er/and,  Swit;(er/and 

Multimedia  Training  , Virtual  Instrumentation  and  Remote  Laboratory  a New  Approach  to  the 
Electronic  Courseware 

Nico/etta  Sa/a,  Acaderry  of arch.  University  of  Ita/ian  Switi;er/and,  Switi^er/and 
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A Case  Study  on  Building  an  Organizational  Learning  and  Information  Support  System  (OLISS) 

Mark  SaJisbf^^j  Umversity  of  New  Mexico,  USA; Jan  P/ass,  University  of  New  Mexico,  USA 

Developing  and  Evaluating  an  On-line  Cross  Cultural  Collaborative  Course 

Aeirian  Sal/is,  Auck/and  University  of  Technology,  New  Zealand;  Jennj  ^grave,  Anck/and  University  fTechno/ogy,  New 
Zealand;  Grant  Carran,  Anck/and  University  of  Technology,  New  Zealand 

A Study  of  Usability  Evaluation  for  Interactive  Computer  Mediated  Tele  training  Systems 
Francoise  Sandot^-Gnermond,  INSA,  France;  Gerard  Benchot,  INSA,  France 
Methodological  Resources  AssistantzHow  To  Apply  Instructional  Design  To  Our  Web-Based 
Materials 

A/bert  Sangra,  Universitat  Oberta  De  Cata/nnya,  Spain;  Andrea  Be//of,  Universi/a/  Oberta  De  Cata/anya,  Spain;  Josep 
Dnart,  Universi/a/  Ober/a  De  Ca/a/ar/ya,  Spain 

Teaching  and  learning  in  a virtual  campus:  The  model  of  the  Universitat  Oberta  de  Catalunya 

A/ber/  Sangra,  Universi/a/  Ober/a  De  Ca/a/arrya,  Spain;  Uoardes  Guardia,  Universi/a/  Ober/a  De  Ca/a/urrya,  Spain 
Rethink  Education:  How  we  make  our  learners  instructors 

Sicco  San/ema,  Def/  University  of  Technology,  The  Ne/her/ands;  Raph  Genang  Def/  University  of  Technology,  The 
Ne/her/ands 

Cooperation  Model  for  Learning:  A System  of  Patterns 

F/dvia  San/oro,  Universidade  Federal  do  RJo  de  Janeiro,  Bra:^/;  Marcos  Borges,  Universidade  Federa/  do  Rio  de  Janeiro, 
BrayJ/;  Neide  San/os,  Universidade  do  Es/ado  do  Rio  de  Janeiro,  Bra:^/ 

Initiating  Constructivist  Learning  Stratigies  THrough  Computer  Generated  Concept  Mapping 

F.  Thomas  ScappaPcci,  King's  Co/Iege,  USA 

Austrian  Literature  Moving  to  Cyberspace  — A Framework  for  Building  an  Open  Distance  Learning 
Website  using  Platform  Independent  Standards  Like  XML 

Alexander  Scha//en,  Vienna  Universi/y  of  Technology,  Aus/ria;  K/aus  Zelewi/t(f  University  Sa/t(burg  Aus/na;  A Min 
T/oa,  Vienna  University  of  Technology,  Aus/na; Johann  S/ockinger,  Ministry  for  Science,  Austria 

Technological  tests  of  educational  theories:  A research  role  for  the  Generative  Virtual  Classroom 

Lyne/te  Schaverien,  University  of  Technology,  Sjdn^,  Aus/ra/ia 

Certifying  Technological  Proficiencies  of  K-12  Teachers 

Caro/ Scheftic,  Ca/rfomia  Po/y/echnic  State  University,  San  Lids  Obispo,  USA 
Support  Environment  for  Distributed  Case  Studies 

Manfred  Schert/er,  University  of  Er/angen-Nuremberg  Germanj;  Freimut  Bodendof,  University  of  Er/angen-Nuremberg 
Germanj 

Organisation,  collaboration  and  learning:  - Shaping  education  and  learning  in  virtual  environments 

Friedrich  Scheuermann,  University  oflnnsbrrrck,  Austria;  Ken  Larsson,  Stockholm  University/ KTH,  Sweden;  Roxanne 
To/o,  The  Pennyy/vania  State  University,  USA 

Flaming,  Banter,  and  Bonding:  Understanding  the  Rhetoric  of  Electronic  Groups 

Rae  Schipke,  Centra/  Connectiart  State  Universi/y,  USA 

Strategies  for  Teaching  and  Learning:  Lessons  Gained  from  an  Ethnography  of  Graduate  Classes  in 
an  Interactive  Distance  Learning  Studio 

Eorraine  G.  Schmert^ng  University  of  West  F/orida,  USA;  Richard  W.  Schmertyjng  Va/dosta  State  University,  USA 

From  Student  Needs  to  Instructor  Roles:  An  Ethnographic  Viewpoint 

Richard  Schmertjng  Va/dosta  State  University,  USA;  Lorraine  Schmert^ing  University  of  West  F/orida,  USA 

Creating  an  On-line  MBA  Program 

Robert  Schramm,  University  fWisconsin- Whitewater,  USA;  Roger  Yin,  University  fWisconsin-Whitewater,  USA 

Introduction  of  a New  Media  Concept  for  a Teaching  Profession  Degree  in  Computer  Science:  A 
Formative  Evaluation 

Axe/ Hunger,  Gerhard  Mercator  Universi/y  Duisburg  Germanj;  Stefan  Werner,  Gerhard  Mercator  Universi/y  Duisburg 
Germarj,\Andre  Bresges,  Gerhard  Mercator  Universi/y  Duisburg  Germarj;  Frank  Schwan^  Gerhard  Mercator 
University  Duisburg  Germanj 

Use  of  Telecommunications  Technology  in  Radiation  Accident  Simulation 

Domenic  Screnci,  Boston  University  Schoo/  of  Medicine,  USA;  Kirsten  Lej,  Boston  University  Schoo/  of  Medicine,  USA; 
Erwin  Hirsch,  Boston  University  Schoo/ of  Medicine,  USA;  Richard Aghababian,  University  f Massachusetts  Medica/ 
Schoo/,  USA;  Tracj  Russo,  Boston  Universi/y  Schoo/  of  Medicine,  USA 

Building  and  Transforming  Educational  Web  Sites  with  SATIRIC  and  Contextual  Patterns 

Jared  Seaman,  Grinne//  Co/iege,  USA;  Vivek  Venugopa/,  Grinne//  Co/iege,  USA;  Samue/  Rebe/s^,  Grinne//  Co//ege, 
USA 
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Teaching  data  anatysis  using  thin  client  technology  for  remote  access 

Robert  Seicimarif  San  Diego  State  University^  USA;  K Micbaei  PeddecorS,  San  Diego  State  University^  USA 

Developing  an  Educational  Multimedia  Digital  Library:  Content  Preparation,  Indexing,  and  Usage 

P.  P Semp/e,  University  of  Mag/and,  USA;K  B.Aiien,  University  of Maryiand,  USA;  A.  Rose,  University  f 
Maryland,  USA 

NSA  Partnership  With  Cal  State  LA.:  Courseware  Conversion  And  Evaluation  Project 

Pene/ope  Semran,  Ca/fomia  State  University^  LosAnge/es,  USA;  Barbara  B^er,  Cahfomia  State  University,  Los 
Ange/es,  USA 

Computing  Environments  For  Metaleaming:  Interconnecting  Hypermedia  Concept  Maps 

Per/a  Senas  Senas,  Universidad  Nacionai  dei  Snr,  Argentina;  Norrrra  Moroni,  Universidad  Nadonai  dei  Sur,  Argentina 

The  Internet  Chemistry  Set:  Web-based  remote  laboratories  for  distance  education  in  chemistry 

Fred  Senese,  Frostburg  State  University,  USA;  Cbris  Bender,  Frvstburg  State  University,  USA 

Educational  Technology  And  Its  Terminology:  New  Developments  In  The  End  Of  The  20th  Century 

Peter  Serdiukov,  University  of  Utah,  USA 

Multifunctionality  Of  Educational  Technology  Applications 

Peter  Serdiukov,  University  of  Utah,  USA 

Student  Satisfaction  and  Perceived  Learning  in  Internet-Based  Higher  Education 

Peter  Sbea,  State  University  of  Nejv  York,  USA;  Eric  Fredericksen,  State  University  of  Neiv  York,  USA;  Alexandra 
Pickett,  State  University  ofNejv  York,  USA 
A Distance-Learning  Model  Based  on  Web  Mining 

Rjdmin  Sben,  Jiaotong  University  of  Shanghai  of  PRC,  China;  ^'un  Wang  Jiaotong  University  of  Shanghai  of  PRC, 
China 

Deploying  Web-Based  Assessments 

Eric  Shepherd,  Question  Mark  Corporation,  USA 

Scenario-based  prototyping:  a user-centered  method  for  the  design  of  CALL  systems 

JaeEun  Shin,  University  of  Manchester,  UK;  David  G.  Wasted,  University  of  Manchester,  UK 
Flexibility  and  Facilities  in  Children's  Electronic  Textbooks 

Norshrrhada  Shiratuddin,  Uniof  Strathcfyde,  Scot/and/ Uni  Utara  Malaysia,  Malaysia;  Monica  I^ndoni,  PhD,  Urn,  of 
Strathclyde,  Scotland 

Multimedia  Guide  to  Fractal  Geometry 

Vladimir  Shlyk,  Belanrsian  State  Pedagogical  University,  Belarrrs 

Project-Based  Learning  + Multimedia:  Adding  Value  to  Students'  Education 

Michael  Simkins,  Silicon  Vall^  Network,  USA;  William  R Penuel,  SRI  International,  USA 

The  Search  for  the  Skunk  Ape:  An  Information  Literacy  WebQuest 

Charlotte  Slater,  The  Walden  Institute,  USA;  Pamela  Sawallis,  Florida  Guf  Coast  University,  USA 
What  Is  So  Revolutionary  About  The  Internet  In  Education? 

Peter  B.  Sloep,  Open  University  of  The  Netherlands,  The  Netherlands;  Ellen  Rusman,  Open  University  of  The 
Netherlands,  The  Netherlands 

Online  Learning  in  a 3-D  Robotics  Workshop:  The  NASA  ROVer  Ranch 

Stephanie  Smith,  NASA/LinCom,  USA 

OATS:  Operational  Amplifier  Tutorial  Simulation;  a dynamic  presentation 

Sarah  Sniderman,  Concordia  University,  Canada;  Ga;y  Boyd,  Concordia  University,  Canada;  Get^ajoos,  Concordia 
University,  Canada 

Online  Mentoring  - A Case  Study  Involving  Cognitive  Apprenticeship  and  a Technology  Enabled 
Learning  Environment 

Kathleen  Srryckr,  IBM  TJ.  Watson  Research  Center,  USA;  Robert  Farrell,  IBM  T J.  Watson  Research  Center,  USA; 
Norma  Baker,  Prfect  Executive  Programs  and  Support,  IBM  Coporadon,  USA 
The  NURAXI  Web-based  learning  environment  architecture 

Loremp  Sommaruga,  Mediatech  Sri,  Itafy; Nadia  Catena^,  Mediatech  S.r.l,  Italy;  Sjlvain  Giroux,  CRS4,  Italy; 
Claude Motdin,  Mediatech  S.r.l,  Itafy 
The  Munchhausen-Trick:  Learning  Internet  by  Internet 

Rafh  Sontag  TU  Chemnit^  Germany;  Dietrich  Thie,  TU  Chemnit^  Germany 
An  Online  Environment  that  Scaffolds  Moving  from  Novice  to  Expert  Collaborative  Learners 

Josph  B.  South,  Brigham  Young  University,  USA;  Laurie  Miller  Nelson,  Brigham  Young  University,  USA;  The  Digital 
Learning  Environment  Research  and  Development  Group , Brigham  Young  University,  USA 
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How  to  show  Web  pages  for  learners:  Teaching  and  Learning  with  Web  Recorder 

Toshio  IBM  Research,  Japaa;  Chk  Oh/am,  Tamagawa  Gakuen,  Japan;  Yoshinori  Aokh,  IBM  Research,  Japan; 

Yoshie  Masnla,  IBMJcpan,  Ltd.,  Japan;  Asnane  Nakijima,  IBM  Research,  Japan 

In  defence  of  ^shovehvare’:  A powerful  tool  in  staff  development  for  online  teaching  and  learning 

Heather  Sparrow,  Edith  Cowen  University,  Austra/ia;  Jan  Herrington,  Edith  Cowen  University,  Anstra/ia;  Tony 
Herrington,  Edith  Cowen  University,  Australia 

Evaluating  the  Impact  of  System  Dynamics  Based  Learning  Environments 

Michael  Spector,  Syracuse  University-IDD(cS’E,  USA:  Dean  Christensen,  University  of  BergenEIST,  Norwey;  Da/ton 
McCormack,  University  of  Bergen-  IFI,  Norway 

An  Evaluation  of  Video  Compression  Schemes  for  Teleteaching  Applications 

Ognjen  Stanovcic,  Open  Systems  AG  Zurich,  Swit^er/and;  Thomas  Waiter,  Swiss  Federal  Institute  of  Technology  Zurich, 
Swrlyr/and;  Bernhard  P/attner,  Swiss  Federai  Institute  of  Techno/ogy  Zurich,  Swit:yriand 

Web-Based  Courses  Improve  English-Language  Skills  of  Foreign  University  Science  Students 

Yvonne  Stapp,  University  ofTsukuba,  Japan 

Educational  Technology  and  Context:  an  Exploration  of  Values,  Roles,  and  Concerns  from  Lecturers, 
Tutors  and  Instructors. 

John  Steel,  Sheffie/d  Ha/iam  University,  Eng/and,  UK;  Aiison  Hudson,  Sh^e/d  Ha/iam  University,  England,  UK 

Educational  Technology  and  Learning  & Teaching:  Pedagogy,  Process  and  Culture 

John  Steel,  Sheffie/d  Ha/iam  University,  England,  UK;  Aiison  Hudson,  Sheffieid  Haiiam  University,  Engiand,  UK 

Facilitating  Learning  via  Computer  Conferencing:  Aspirations,  Requirements  and  Hard  Facts 

Stuart  Stein,  University  of  the  West  of  Engiand,  United  Kingdom 
Multibook:  making  web  based  learning  resources  personal 

Achim  Steinacker,  TU  Darmstadt,  Germary;  Comeiia  Seebeg  GMD  IPSi,  Cermary;  Raf  Steinmet^  GMD  IPS I, 
Germany 

Tracking  Human  Expression  Actions  in  a Lecture 

Tiimann  Steinberg  Dartmouth  Coiiege,  USA;  Li  S hen,  Dartmouth  Coiiege,  USA;  Ling  Cheng  Dartmouth  Coiiege, 
USA;  Fiiiia  Makedon,  Dartmouth  Coiiege,  USA 

Technology  Learning  Communities:  Collaborations  to  increase  technology  integration  in  education 

May  Stphen,  Harris-Stowe  State  Coiiege,  USA;  Diane  Smoot,  Harris-Stowe  State  Coiiege,  USA 

Four  Challenges  for  TeleTeachers  in  Rural  Schools 

Ken  Stevens,  Memoriai  Universiy  ofNenfoundiand,  Canada 

Math  and  Science  Curriculum  Revision:  A Collaborative  Approach  to  Improving  Preservice  Teachers* 
Use  of  Technology  Knowledge  and  Instructional  Skills. 

David  Stokes,  Westminster  Coiiege,  USA;  Carofyn  Jenkins,  Westminster  Coiiege,  USA;  Lord  Huhnke,  Westminster 
Coiiege,  USA;  Gothard  Cry,  Westminster  Coiiege,  USA;  Cheyi Manning  Bryant  Intermediate  Schooi,  USA 

Integrating  pedagogy,  content  and  technology  into  your  curriculum 

David  Stokes,  Westminster  Coiiege,  USA;  Wanda  Carras^rdiio-Gome^  Westminster  Coiiege,  USA 

oz-TeacherNet:  Supporting  teacher  use  of  Information  and  Communications  Technologies  in  the 
Elementary  Curriculum 

Jackie  Stokes,  Queenskind  Universiy  ofTechnoiogy,  Austraiia;  Jennrfer  Masters,  Queensiand  Universiy  ofTechnoiogy, 
Austraiia 

SMILE  Maker  - An  Intelligent  Learning  Environment  for  Problem  Solving 

Svetoshv  Stqyanov,  Universiy  of  Twente,  The  Netheriands 

Group  interaction  as  a predictor  of  learning  effectiveness  in  a computer  supported  collaborative 
problem  solving 

Neii  Stqyanova,  Universiy  of  Twente,  The  Netheriands 

A Web-based  instruction  model  for  open  learning  -A  theorical  and  practical  framework 
DrArmandSt-Pierre,  Rqyai Miiitay  Coiiege,  Canada 
Re-usability  problems  in  large-scale  content  management  of  database  driven  Web  based 
environments 

Aiiard  Stryker,  Universiy  of  Twente,  Netheriands 
Constructivist  Environments  versus  Behaviourist  Demands 

Geneen  Stubbs,  Universiy  of  Giamorgan,  UK;  Mike  Watkins,  Universiy  of  Giamogan,  UK 
An  Adaptive  Web-based  Course  in  Financial  Engineering  with  Dynamic  Assessment 

Aiberto  Sudre:^  Universidad  Autonoma  de  Madrid,  Spain;  Estreiia  Prdido,  Universidad  Arddnoma  de  Madrid,  Spain; 
Rosa  Carro,  Universidad  Autonoma  de  Madrid,  Spain 
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Building  Learner  and  Teacher  Autonomy  for  New  Learning  Environments 

Leena  Subraj  Umvemfyof]yvas^/ajFm/ancI;LJisaKa//Wf  Unwersifyof]yvas^/ajFm/and;U//aLautjamnj  Umverst^ 
of  Jyvasfy/aj  Fw/anii 

Development  of  a Computer  Aided  Cooperative  Classroom  Environment 

Akira  S/^^anuma,  I^ushu  Unwersiiy,  Japan;  I^uaosuke  Ffi/im^Oj  ^usbu  Umversiiy,  Japan;  Yu/aka  Tsn/snn/i, 
Kumamoto  Gakuen  University,  J^p^^ 

A Teachware  Management  Framework  for  Multiple  Teaching  Strategies 

Christian  Siif  Universitat  Passau,  Germa^;  Rueiof  Kammeri,  Universitat  Passass,  Germany;  Burkharei  Freita^, 
Universitat  Passau,  Germar^ 

Redesign  a Biology  Laboratory  Experiment  as  a Multimedia  Simulation 

Jonathan  Sjva/iow,  University  of  Waterloo,  Canada;  Norm  Scott,  University  of  Waterloo,  Canada;  Tom  Car^,  University 
of  Waterloo,  Canada 

Electronic  Literacy  Standards  for  the  21st  Century 

Karen  Sjuan,  University  at  Albar^,  USA 

Technology  Integration  & Professional  Development:  A Mentoring  Model 

Karen  Sjuan,  U niversity  at  Albar^,  USA;  Aliya  Holmes,  University  at  Albany,  USA;Jssan  JYargas,  University  at 
Albany,  USA;  Sjbilfyn  Jennings,  Sage  College,  USA 
Computer  Support  For  Unobtrusive  Assessment  Of  Conceptual  Knowledge 

Steven  Tanimoto,  University  of  Washington,  USA;  Adam  Carlson,  University  of  Washington,  USA;  Earl  Hunt, 
University  of  Washington,  USA;  David Madigan,  A.T<dsrT  Shannon  Laboratoy,  USA; Jim  Minstrell,  Talana,  Inc., 
USA 

Using  Knowledge  Models  in  Intelligent  Tutoring  Systems 

Stphen  G.  Taylor,  Champlain  Regional  College,  Canada 

Situated  learning 

Sarah  Teo,  Singapore  Computer  Systems  Ltd,  Singapore 

Preliminary  Investigation  of  IMM  Evaluation  Instrument 

Jennifer  D.E.  Thomas,  Pace  University,  USA 

The  Effects  of  Activity  Structure  and  Learner  Characteristics  on  Acceptance  of  Control  Opportunities 
in  Hypermedia  Environments:  A Report  of  Quantitative  Findings 

Denise  Tolhurst,  University  of  Nejv  South  Wales,  Australia;  Esmond  L Debus,  The  University  of  Sydny,  Australia 

The  Emergence  of 'School'  as  a Factor  Influencing  Patterns  of  Navigational  Choices  in  Hypermedia 

Denise  Tolhurst,  University  of  Nejv  South  jvales,  Australia;  Raymond  L Debus,  The  University  of  Sydny,  Australia 

An  Evaluation  of  Interactive  Studio  Physics  I 

Holfy  Traver,  Rensselaer  Polytechnic  Institute,  USA;  Michael KaMer,  Rensselaer  Po^technic  Institute,  USA;  Karen 
Cummings,  Rensselaer  Pofytechnic  Institute,  USA;  Keith  Hmieleski,  Rensselaer  Po^technic  Institute,  USA 

Grasp  of  Few  body  element's  correlative  operation  using  Multi  points  measure  data-Compair  with  2 
kind  selection- 

Atsushi  Tsubokura,  Osaka  Electro-Communiation  Univ,  Japan;  Klyoshi Masui,  Osaka  Electro-Communication  Univ., 
Japan;  Noboru  Ashida,  Osaka  Electro-Communication  Univ.,  Japan;  Kagemasa  Ko:^4ki,  Konami  Co.,  Japan;  Katsuhide 
Tsushima,  Osaka  Electro-Communication  Univ.,  Japan 

A Linguistic  Approach  to  Use  Cultural  Web  Pages  in  a ClassroomA 

Yutaka  Tsutsumi,  Tei^o  Univ.  Fukuoka Jr.  Collie,  JAPAN;  Ryq/i  Matsuno,  Prefectural  University  of  Kumamoto, 
JAPAN;  Ka:^w  Ushi/ima,  ^ushu  University,  JAPAN 

Offline  projects  for  cross-cultural  competence  improvement 

lYictoria  Tu:fukova,  Rostov  State  Pedagogical  University,  Rjissia;  Irina  Rotjna,  Rostov  State  Pedagogical  University, 
Russia;  Cyndy  Woods,  Aritypna  state  University,  USA 

Analyzing  Artists'  Interaction  with  an  Artificial  Reality 

Elpida  T^afestas,  National  Technical  University  of  Athens,  Greece 

Electronics  Education  System 

Maria  D.  Valdes,  University  of  Vigo,  Spain;  Jose  A.  Tarrio,  University  of  Vigo,  Spain;  Maria  J.  Moure,  University  of 
Vigo,  Spain;  Enrique  Mandado,  University  of  Vigo,  Spain; Julio  Gon^ale:;^  University  of  New  York  at  New  Palty^ 
USA. 

Cultural  and  gender  differences  in  computing  among  secondary  school  children 

Johan  van  Braak,  Vri/e  Universiteit  Brussels,  Belgium 
A Patient  Simulation  System  for  Nursing  Education 

Johan  C.  van  Roqyen,  University  of  the  Orange  Free  State,  South  Africa;  C.  Janse  Tolmie,  University  of  the  Orange  Free 
State,  South  Africa 
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Evaluadon  of  ICT  course  material  in  the  context  of  an  acoustic  design  project 

Pieter  V'andae/ef  Ghent  Universi^f  Belgium;  Dick  BotteMoorenj  Ghent  University^  Be/gium;  E/s  l^an  Ze/ej  Ghent 
University,  Belgium; Josephina  Lenaerts,  Ghent  University,  Be/gium;  RaouJ  MeuJdermans,  KaHo  Sint  Ueven,  Be/gium; 
/[dare  Van  Overmeire,  Vnje  Universiteit  Brtmei,  Be/gium;  Gerrit  Vermeir,  KU  I^uven,  Be/gium 
Supporting  Virtual  Classrooms  through  Extranet  technology:  the  Eurydices  system 

Kostas  Bovi/as,  University  of  Patras,  Greece;  Aspasia  Kanta,  University  of  Patras,  Greece;  Nikos  Piskopos,  University  of 
Patras,  Greece;  Bi//  Vassi/iadis,  University  of  Patras,  Greece;  At hanassios  Tsaka/idis,  University  of  Patras,  Greece; John 
Tsaknakis,  University  of  Patras,  Greece;  Evange/os  Sakkopou/os,  University  of  Patras,  Greece;  Christos  Adakris, 

University  of  Patras,  Greece 
Malaysian  Smart  School  - Vision  Vs.  Reality 
Nair  Vi/ay,  Adu/timedia  University,  Ada/aysia 

A Synchroni2ed  Multimedia  Integration  Language  (SMIL)  Presentation  Generator 

Joaquin  Vi/a,  I//inois  State  University,  USA;  Barbara  Became,  I/iinois  State  University,  USA 

Structural  Communication  and  Web  Based  Instruction 

Gar/os  Vi//a/ba,  Syracuse  University,  USA;  A/exander  Romis^wski,  Sjracuse  University,  USA 

A Domain-Independent  Reasoning  Mechanism  For  An  Its  Authoring  Tool 

Ad  aria  Virvou,  University  of Piraeus,  Greece 

An  Empirical  Study  Concerning  Graphical  User  Interfaces  that  Manipulate  Files 

Adaria  Virvou,  University  of  Piraeus,  Greece;  Katerina  Kabassi,  University  of  Piraeus,  Greece 
A Web-based  educational  tool  for  solving  equations 

Adaria  Virvou,  University  of  Piraeus,  Greece;  Adaria  Adoundridou,  U niversipi  of  Piraeus,  Greece;  Adichae/ I^iqps, 

University  of  Piraeus,  Greece;  Athanasios  Papachristou,  University  of  Piraeus,  Greece;  Niko/aos  Po/ye^tis,  University  of 
Piraeus,  Greece 

Using  A Speech-Driven,  Anthropomorphic  Agent  In  The  Interface  Of  A WWW  Educational 
Application 

Adaria  Virvou,  U niversity  of  Piraeus,  Greece;  Nikitas  Sgouros,  University  of  Piraeus,  Greece;  Adaria  Adoundridou, 
University  of  Piraeus,  Greece;  Dimitrios  Adanargias,  University  of  Piraeus,  Greece 

Evaluation  throughout  the  Life  Cycle  of  a Multimedia  Tutoring  System 

Adaria  Virvou,  University  of  Piraeus,  Greece;  Victoria  Tsiriga,  University  of  Piraeus,  Greece 

Evaluation  of  an  ITS  for  the  Passive  Voice  of  the  English  language  Using  the  CIAO!  Framework 

Adaria  Virvou,  University  of  Piraeus,  Greece;  Victoria  Tsiriga,  University  (f Piraeus,  Greece;  Dimitris /daras.  University 
of  Piraeus,  Greece 

EDIT  - A Distributed  Web-based  Learning  Network  for  Distance  Education 

Ljraan  Voinea,  University  *Po/itehnica^^ (f Bucharest , Romania;  Gabrie/ Dima,  University”Po/itehnica” (f Bucharest , 
Romania;  Giprian  Guda/bu,  University^Po/itehnica^  of  Bucharest , Romania;  Ion  Adiu,  IBAd  G/oba/  Services, 
EducationZVTraining  Romania;  Adarce/  Prof rescu.  University  ^Po/itehnica^^  of  Bucharest,  Romania 

Design  Processes  Involved  in  the  Creation  of  CBLEs:  Preliminary  Results 

Steve  Wakeham,  JdcGi//,  Ganada;  Robert  Bracewe//,  AdcGi//,  Ganada 

Portable  FM  Radio  Stations  - Broadcasting  with  the  Sun 

David  Wa/ker,  The  Gommonivea/th  of  Learning  Ganada 
Neuromance,  Or  Is  The  Old  Flame  Still  Best? 

Simon  Wa/ker,  University  of  Greenudeh,  UK 
Visual  Lab  - A Multimedia  Virtual  Experiment  Environment  on  WWW 

Ghao-Ghiu  Wang  Ghung  Yuan  Ghristian  University,  Taiwan,  K O.C.;Adaiga  Ghang  Ghung  Yuan  Ghristian  University, 
Taiwan,  R.O.C.;  Ghang-Kai Hsu,  Ghung  Yuan  Ghristian  University,  Taiwan,  KO.C.;Jia-ShengHeh,  Ghung  Yuan 
Ghristian  University,  Taiwan,  KO.G. 

Constructing  a Learning  Environment  for  Knowledge  Advancement 

Feng-Kwei  Wang  University  of  Adissouri  - Go/umbia,  USA 

Training  Faculty  to  Use  Active  and  Collaborative  Learning  and  Web-Based  Courses  in  an  Integrated 
Curriculum  for  the  First  Two  Years  of  an  Engineering  Program  of  Study: 

John  Watret,  Embry-Ridd/e  Aeronautica/  Univeristy,  USA;  David  Pedersen,  Emb^-Ridd/e  Aeronautica/ Univeristy, 
USA;  Ghar/es  Adartin,  Emb^-Ridd/e  Aeronautica/  Univeristy,  USA 

The  Use  of  Cooperative  and  Collaborative  learning  in  a Web-based  Integrated  Curriculum  for  the 
First  Two  Years  of  an  Engineering  Program  of  Study 

John  Watret,  Embry-Ridd/e  Aeronarrtica/  University,  USA;  Char/es  Idartin,  Em  b^-Ridd/e  Aeronautica/  University, 
USA 
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TeamCMU:  An  Electronic  Toolset  For  Collaborative  Development  Projects 

Weinberg,  Carnegie  Me//on  University,  USA;  Diana  Bcy  \ek,  Carnegie  Me//on  University,  USA 

Professional  Development  Institute  vs.  On-line  Course:  Comparing  Two  Online  Collaborations  for 
Designing  Technology  Integrated  Instruction 

E/i;(abe/h  We//man,  UCLA,  USA;  Maya  CreeSman,  UCLA,  USA;  Jana  F/ores,  UCLA,  USA 

Taking  Our  Own  Medicine:  Learning 

Martin  Wessner,  GMDJPSI,  Germarty;  Come/ia  Seeberg,  DarmstaSt  University  of  Techno/og;,  Germany;  Raf  S/einmet^ 
DarmstaSt  University  o/Tecbno/ogy,  German 

Teachers,  students  and  the  university  walking  together  in  one  direction:  analyzing  the  potential  for 
ICT  in  higher  education  from  different  perspectives 

Joacbim  Wetter/ing  University  of  Tjvente,  Tbe  Netber/anSs 

Faculty  reaction  to  WebCT:  one  university's  experience 

Sbaron  A/ayne  Widmayer,  George  Mason  University,  USA;  Cyntbia  L McCorrrt,  George  Mason  University, 

Designing  Virtual  Learning  Spaces  for  Students  With  Special  Needs 

Wi//iam  K Wiencke,  State  University  of  West  Georgia,  USA;M.  D.  Robber,  State  University  of  West  Georgia, 

Using  a Wrench  to  Pound  in  a Screw:  The  Misapplication  of  Communication  Technologies  in 
Education 

Eric  Wigna//,  Purdue  University  Ca/urrret,  USA 

Distance  Learning  For  Government  Agencies  / Online  Academies 

Josjrb  Wi/tgak,  American  West  Enterprise,  USA 

The  Management  of  the  Telecommunications  Function:  The  Impact  on  Organizational  Support, 
Planning  and  Training  Quality 

John  Wi/iems,  Eastern  Idinois  University,  Karen  Ket/er,  Eastern  I/iinois  University, 

Implementing  a 'scaffolding  by  design'  model  in  a WWW-based  course  considering  cost  and  benefits 

Koos  Winnips,  University  of  Twente,  Metber/ands;  Betty  Co//is,  University  of  Twente,  Metber/ands;JfMoonen,  University 
ofTwente,  Netber/ands 

Applications  and  categorization  of  software-based  scaffolding 

Koos  Winnips,  University  ofTrvente,  Netber/ands;  Catherine  McL/rrrgb/in,  University  of  New  England,  Australia 

Developing  a Storyboarding  Process  for  Online  Content:  From  Microsoft  Powerpoint  to  Macromedia 
Flash 

Jennfer  Wo/verton,  Brigbam  Yorrng  University,  Waddoups,  Brigbam  Young  University,  USA 

Implementing  a Professional  Development  Program  in  Instructional  Design  & Technology 

C James  Wong  Sorrtbwestem  I//inois  Co/Zege , USA 

Creating  Multimedia  Instructional  Projects:  Fun  or  Frustration? 

C.  James  Wong  Sorrtbwestem  Idinois  Codege,  USA 

Formative  Evaluation  of  Leamer^Centered  Web  Course  Design:  A Strategic  Analysis 

C.  James  Wong  Southwestern  Idinois  Codege,  USA 

WALTS:  Web-based  Adaptive  Programming  Language  Tutoring  System 

Chong  Woo,  Kookmin  University,  Corrprrter  Sdence,  Korea;  Jinwoo  Choi,  Kookmin  University,  Computer 

Science,  Korea 

Online  Learning  Communities:  Vehicles  for  Collaboration  in 

Stpbanie  Wood^,  University  of  Co/orado  at  Denver,  United  States;  Stac^  L/rdwig-Hardman,  University  rf  Colorado  at 
Denver,  United  States 

Online  Learning  Communities:  Vehicles  for  Collaboration  in 

Stephanie  Wood^,  University  Colorado  at  Denver,  United  States;  Stac^  Lrrdwig-Hardman,  University  Colorado  at 
Denver,  United  States 

The  Construction  of  World-Wide-Web  Resource  for  Chinese  Medicine  and  Acupuncture 

Fan  Wu,  China  Medical  Codege,  Taiwan;  Sbao-Fu  Hrrang  China  Medical  Codege,  Taiwan 
Developing  Collaborative  Technology-Enhanced  Programs 

Nancy  Wyatt,  Penn  State  University  Delaware  County,  USA;  George  Fran:^^  Penn  State  University  Delaware  County , 
USA;  Karen  Hi//,  Penn  State  University  De/aware  County,  USA;  Matthew  Bodek,  Penn  State  University  De/aware 
County,  USA;  Susan  Ware,  Penn  State  University  De/aware  County , USA 
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Automatic  Generation  of  the  Optimal  Tutorial-Plan  in  Adaptive  Educational  Hypermedia  System 

Haiyan  Xuj  Unwerst(}i  of  Science  Techno/qg}/  of  Chinaj  P.  K China;  XueHai  Zbon,  Univeryi^y  of  Science  eS*  Technology 

of  Chinaj  P.KChina; Zhenxi ZhaOj  Univeni/y  of  Science  eS*  Technoiog}/  of  China^  PK China 

Synchronized  Notes  for  Digital  Class  Video 

Michiro yabuki,  Meisei  Uninersi/yj  JAPAN;  Akira  Wafanabe,  Meisei  Univeniiyj  JAPAN 

A Cognitive-Affective  Framewodc  for  Instructional  Interactivity 
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Abstract:  Scholars  have  been  predicting  a revolution  in  the  academic  publishing  world  since 
the  early  1 990's,  including  university  centered  distribution,  extensive  online  peer  review,  and 
innovative  multimedia  enhancements.  Online  journals  have  indeed  proliferated,  yet 
interactive  multimedia  electronic  journals  (iwey  journals)  have  not  yet  become  widespread, 
nor  have  they  approached  their  potential  for  scholarly  communication.  In  this  article,  we 
discuss  the  obstacles  in  the  way  of  imey  journals  for  academic  publication,  predict  some 
technological  changes  that  should  hasten  their  adoption,  and  argue  for  their  promotion  as  an 
excellent  forum  for  disseminating,  verifying,  and  refining  scholarly  research. 


1.  The  Predicted  Revolution  in  Scholarly  Publication 

For  over  a decade,  academics  have  been  predicting  a revolution  in  scholarly  publication.  In  1991  Ann 
Okerson,  then  Director  of  the  Office  of  Scientific  and  Academic  Publishing  of  the  Association  of  Research 
Libraries,  concurred  with  the  growing  opinion  that  paper  journals  were  becoming  obsolete,  advocating  a 
"university-based  system  into  which  scholars  from  all  disciplines  would  place  their  findings"  (Okerson  1991), 
Andrew  Odlyzko  of  AT&T  Research  Labs  predicted  the  "impending  demise  of  traditional  scholarly  journals" 
(Odlyzko  1 995)  while  Princeton’s  Stevan  Hamad  advocated  a system  of  "scholarly  skywriting  in  the  post- 
Gutenberg  galaxy,"  where  academics  would  share,  critique,  and  add  to  each  other’s  findings  in  an 
electronically-accelerated  process  of  research  and  publication  (Hamad  1990,  1997).  A number  of  models  for 
network-based  electronic  publishing  were  proposed,  generally  envisioning  university-centered  distribution, 
lower  publication  costs,  24-hour  computer  access  to  documents,  faster  review  processes,  a hierarchy  of 
publication  stages  according  to  the  level  of  peer  review,  extensive  online  discussion  and  updates,  powerful 
search  techniques,  and  multimedia  enhancements  (Bailey  1994).  Though  no  single  publication  model  has 
dominated  the  scene,  many  of  these  changes  have  been  evolving.  Certainly,  the  number  of  online  journals  has 
exploded  in  the  past  ten  years,  with  initiatives  like  Project  Muse,  JSTOR,  Project  Gutenberg,  ACM’s  Digital 
Library,  OCLC,  Springer  Link,  Ginsparg’s  preprint  library,  and  on  and  on. 

Nevertheless,  a closer  look  at  existing  online  journals  shows  that  they  fall  short  of  the  predictions. 
Online  academic  journals  (that  is,  those  that  have  no  traditional  print  counterparts)  do  not  enjoy  the  acceptance 
and  respect  that  would  make  them  competitive  with  hard  copy  publications.  Nor  have  electronic  journals  taken 
advantage  of  the  added  value  and  marketable  difference  that  multimedia  enhancements  could  offer.  Were  the 
predictions  overblown,  or  must  we  simply  be  patient  for  changes  that  will  inevitably  come?  What  is  the  state  of 
the  art  in  interactive  multimedia  journals  for  academia,  and  what  can  we  hope  for  in  the  next  ten  years? 


2.  Today's  Multimedia  Journals 

Let  us  first  put  our  interest  in  online  multimedia  publication  in  context.  In  1998,  The  IMEJ  of 
Computer- Enhanced  Learning  — an  interactive  multimedia  electronic  journal  — was  founded  at  Wake  Forest 
University,  spurred  by  the  university’s  campus-wide  distribution  of  laptop  computers  and  a growing  interest  in 
educational  technology.  Our  belief  was  that  imej  journals  (pronounced  "image")  have  much  to  offer  to 
education  and  scholarship,  and  we  wanted  to  see  by  our  own  experience  if  this  publication  model  was  viable. 
We  envisioned  a journal  that  seemed  ideal  for  the  subject  of  computer-enhanced  learning,  where  readers  with 
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practical  questions  about  the  value  of  technology  in  education  could  see  and  experiment  with  what  others  have 
done  - trying  out  simulations,  taking  sample  quizzes,  manipulating  3-D  images,  and  hearing  teachers  explain 
their  pedagogical  innovations  in  their  own  words.  In  the  two  years  since  the  journal’s  inception,  we  have  been 
monitoring  the  growth  of  interactive  multimedia  publications  in  academia  and  evaluating  our  own  attempts  to 
produce  a Journal  that  lives  up  to  our  vision  for  online  scholarly  publication. 

Our  most  recent  informal  search  of  existing  electronic  journals  shows  that,  although  the  word 
multimedia  is  used  generously  to  describe  online  publications,  there  is  still  relatively  little  real  multimedia 
development,  particularly  of  the  interactive  sort.  Nearly  all  journals  that  describe  themselves  as  ’’multimedia” 
have  only  tables,  graphs,  and  pictures,  and  the  term  interactive  generally  implies  the  ability  to  post  comments  to 
authors  on  a bulletin  board,  the  availability  of  email  popup  windows,  the  use  of  hyperlinks  to  related  subjects, 
and  occasionally  the  linking  of  footnote  numbers  to  the  footnotes  at  the  end  of  the  paper. 

We  expected  to  find  a good  amount  of  multimedia  material  in  scientific  and  medical  publications, 
where  the  usefulness  of  3-D  images  and  simulations  is  easy  to  imagine  (Eason  et  al.  1997)  (Wilkinson  1998), 
but  our  findings  were  disappointing.  The  Springer  Link  electronic  library  (www.link.springer.de)  has  an 
extensive  collection  of  scientific  journals  and  marks  each  with  a symbol  indicating  where  supplementary 
electronic  material  is  included.  However,  a sampling  of  these  revealed  mostly  ,pdf  and  image  files,  a number  of 
videos,  and  only  one  rotating  molecule  as  an  example  of  interactivity.  The  Royal  Chemical  Society's  Web  site 
(www.rsc.org/is/ioumals/i  1 .htm)  similarly  marks  its  journals  containing  supplementary  electronic  material,  but 
once  again  we  found  only  image  and  document  files.  McGill  University  has  a site  for  The  Online  Journal  of 
Cardiolo^  (www.mmip.mcgill.ca)  with  links  to  teaching  videos,  a Virtual  Stethoscope,  online  quizzes,  and 
moving  Xray  images,  but  all  this  material  appears  to  be  their  own  production  rather  than  submitted  research 
publications.  Among  the  journals  we  found,  two  with  a significant  multimedia  interactivity  were  the  Internet 
Journal  of  Chemistry  (IJC,  www.iic.com)  and  the  Radiology  Society  of  North  America  Electronic  Journal 
(RSNA  EJ,  http://ei.rsna.org  ).  IJC  lists  its  multimedia  element  under  a separate  link,  categorizing  them  as 
images,  interactive  chemical  structures,  interactive  graphs,  movies,  etc.  From  this  list  of  categories  there  are 
links  to  the  articles  containing  the  multimedia  enhancements.  RSNA  EJ  contains  a number  of  animated 
radiology  images  using  video,  VRML,  or  Java-based  simulations. 

Multimedia  has  possibilities  in  the  humanities  as  well.  A few  music  journals  such  as  EOL 
{Ethnomusicology  Online,  http://research.umbc.edu/efhm/eol.html)  take  advantage  of  sound  files.  In  the 
creative  vein,  some  journals  exploring  interactive  narrative  and  art  have  sprung  up,  such  as  Cultronix 
(http://english-www.hss.cmu.edu/cultronix/)  at  Carnegie  Mellon  or  Labyrinth  at  the  University  of  Southern 
California  (www.annenberg.edu/labvrinth).  But  overall,  it  is  difficult  to  find  examples  of  the  type  of  journal 
predicted  since  the  early  1990's.  Even  The  Journal  of  Interactive  Media  in  Education  (http://www- 
iime.open.ac.uk).  an  early  and  well-produced  entry  into  the  field  of  multimedia  publication,  appears  to  be 
backing  away  from  multimedia  interactivity  in  recent  issues.  Its  1 999  volume  has  only  one  article  marked  as 
containing  an  interactive  element,  as  opposed  to  seven  in  1998.  Admittedly,  our  survey  was  informal  and 
necessarily  incomplete.  We  invite  our  readers  to  let  us  know  of  examples  of  interactive  multimedia  that  we 
may  have  missed.  Our  intent  is  not  to  indict  the  quality  of  the  journals  mentioned  here,  but  instead  to  raise  our 
central  questions:  Have  we  expected  too  much  too  soon  in  imej  journals?  What  are  the  obstacles  to  their 
development,  and  will  they  eventually  be  overcome? 


3.  Obstacles  in  the  Way  of  imej  Journals 

The  obstacles  in  the  way  of  the  success  of  /wq/ journals  are  both  technical  and  human.  On  the 
technical  side,  the  problems  include  bandwidth  limitations,  the  speed  of  technological  change,  and 
compatibility  issues.  To  be  widely  accepted,  /wq/ journals  must  be  more  user-friendly  than  they  currently  are. 
Videos  are  still  too  small  (in  display  size),  too  jerky,  and  too  long  to  download,  especially  for  readers  accessing 
electronic  journals  through  modems.  The  adoption  of  cable  modems,  DSL,  and  faster  network  connections  in 
our  homes,  along  with  improved  compression,  streaming  media  delivery,  dedicated  media  servers,  and 
bandwidth  load  balancing  will  eventually  bring  the  technology  up  to  user  expectations. 

Standardization  issues  may  take  longer  to  resolve,  and  the  phenomenom  of  rapidly-evolving 
technology  will  leave  vestiges  of  obsolete  platforms  in  archived  online  journals.  In  our  first  issue  of  IMEJ,  for 
example,  we  used  VRML  requiring  Cosmo  Player  for  viewing,  an  already  obsolete  medium.  The  Java  language 
continues  to  evolve  month-by-month,  requiring  constant  upgrades  to  the  plug-in.  Despite  the  best  efforts  of  an 
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electronic  journal's  producers  to  choose  the  longest-lived  development  tools,  it  is  inevitable  that  they  will  use 
applications  that  will  become  outdated. 

Some  of  the  most  attractive  and  fastest- evolving  multimedia  features  are  the  very  ones  where 
standardization  has  not  settled  in,  including  audio/visual  media  delivery  and  dynamic  html.  In  the  media  player 
world,  we  have  RealPlayer  G2,  Microsoft  Windows  MediaPlayer,  and  Quicktime  battling  it  out.  In  the  browser 
wars,  Internet  Explorer  and  Netscape  have  still  not  reconciled  to  a common  way  of  handling  Java  security  and 
dhtml.  The  adoption  of  the  W3C  standard  and  the  next  version  of  both  browsers  will  bring  them  closer  to  this 
goal  (www.webstandards.org/dhtml.html).  In  the  meantime,  it  is  feasible,  though  time  consuming,  to  offer 
multimedia  supplements  in  multiple  platforms  to  accommodate  as  many  readers  as  possible. 

Clearly,  the  technical  obstacles  are  challenging,  but  we  believe  that  they  can  and  will  be  overcome. 
The  human  problems,  as  one  might  expect,  may  be  more  difficult  to  solve.  The  user-compatibility  issue  is 
complicated  by  the  fact  that  production  of  an  article  in  an  imej  journal,  in  the  final  stages,  is  a collaborative 
work  between  the  authors  and  the  journal's  multimedia  developers.  Even  if  a journal  adopts  multimedia  and 
browser  standards  to  serve  the  greatest  number  of  readers  — or  better  yet,  if  multimedia  elements  are  offered  in 
a choice  of  all  commonly-used  platforms  - the  journal's  multimedia  developers  must  get  the  raw  material  from 
the  authors.  From  our  experience  at  The  IMEJ  of  Computer-Enhanced  Learning,  we  have  seen  that  authors 
generally  work  on  their  accustomed  platforms  with  their  favorite  tools,  and  they  prefer  to  give  us  multimedia 
material  they  have  already  prepared  or  that  they  can  prepare  easily  in  their  usual  manner.  One  author  gives  us  a 
Lotus  Screencam  file,  a second  provides  a video,  and  a third  author  sends  a PowerPoint  presentation.  In  this 
setting,  it  is  all  the  more  difficult  to  set  production  standards  or  offer  multiple  versions.  Authors  are  also 
understandably  particular  about  how  their  work  is  transformed  by  multimedia  additions.  The  author  and 
editor/multimedia  developer  may  have  a different  view  of  the  article,  and  what  one  might  consider  a multimedia 
enhancement,  the  other  might  consider  superfluous. 

The  last  obstacle  is  the  Catch  22.  Generally,  the  scholars  who  have  the  best  material  to  offer  to  imej 
journals  are  reluctant  to  publish  in  them  because  they  fear  electronic  publications  will  be  considered  minor 
league  by  their  colleagues.  But  at  the  same  time,  /mey  journals  cannot  enter  the  major  leagues  until  they  attract 
better  quality  publications.  How  do  we  find  our  way  out  of  this  dilemma?  We  see  some  possible  remedies,  but 
they  can  only  work  over  time. 

As  computer  use  becomes  the  norm  in  the  academic  world,  and  as  programs  for  media  production 
become  more  user  friendly  and  better  standardized,  some  of  the  production  problems  of  //wey  journals  will  be 
alleviated.  Just  as  scholars  now  typically  do  their  own  word  processing  and  document  production,  it  is  likely 
that  they  will  also  do  a large  part  of  their  own  multimedia  production  in  the  future.  Templates  for  journal 
publications  will  make  it  possible  for  authors  to  submit  their  material  in  prescribed  formats.  Work  has  already 
been  done  in  this  direction.  We  at  IMEJ  have  templates  for  article  production,  though  they  are  used  in-house 
rather  than  as  a submission  mechanism  for  authors. 

New  Web  authoring  tools,  markup  languages,  and  network  protocols  are  now  under  development,  and 
their  standardization  will  be  important  for  the  viability  of  imej  journals.  The  W3C  consortium  has  been 
working  on  its  recommendation  for  the  next  version  of  HTML,  to  integrate  XML  — a language  facilitating 
collaborative  Web  page  development  — and  cascading  style  sheets.  A companion  network  protocol  called 
WebDAV  is  being  worked  on  as  a collaboration  of  researchers  at  IBM,  Merant  (formerly  Intersolv),  INSO, 
Microsoft,  Novell,  Rational,  and  the  University  of  California  at  Irvine 

(http://msdn.microsoft.com/workshop/standards/webdav.asp).  WebDAV,  which  stands  for  Web  Distributed 
Authoring  and  Versioning,  will  allow  multiple  developers  to  work  on  Web  documents,  including  multimedia 
components,  from  distributed  locations  with  the  necessary  mechanisms  for  locking  and  advanced  features  for 
searching  based  on  XML  metadata  properties.  A standard  for  WebDAV  is  being  developed  by  an  Internet 
Engineering  Task  Force,  promising  a tool  well-suited  for  the  collaborative  production  inherent  in  /mey  journals. 

To  gain  scholarly  and  professional  acceptance  of  imej  journals,  we  must  be  careful  about  being 
identified  with  commercial  or  non-academic  sites.  Technically,  we  can  learn  from  these  sites  and  the  polish  of 
their  presentation.  Microsoft's  MSDN  Show,  for  example,  blends  entertainment  and  education  by  means  of 
online  source  code,  videos,  and  synchronized  video  transcripts.  The  Discovery  Channel  Web  site  offers 
excerpts  from  and  supplements  to  its  TV  programs.  CSPAN's  Booknotes  presents  interviews  of  the  latest 
authors.  These  sites  are  useful  and  educational,  but  combine  information  with  entertainment  and  advertising, 
which  contributes  to  the  notion  that  Web  publications  are  meant  to  sell  us  something  or  amuse  us. 

Scholarly  publications  must  establish  a different  presence  among  the  electronic  media,  using  the  power 
of  multimedia  to  elucidate  rather  than  entertain,  imej  editors  can  earn  their  credibility  by  careful  peer  review, 
substantive  content,  purposeful  use  of  multimedia,  and  polished  technical  production.  Peer  review  is  in  a sense 
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the  easy  part,  facilitated  by  ease  and  speed  of  communication  in  the  electronic  environment.  But  first,  there 
must  be  something  to  review,  /wey  journals  must  attract  high  quality  submissions  in  spite  of  their  unestablished 
reputations,  and  this  we  believe  they  can  do  by  exciting  the  interest  of  those  who  enjoy  taking  a chance  on 
innovation  and  creativity.  By  gradually  attracting  the  work  of  energetic,  risk-taking  scholars  and  accumulating 
a body  of  credible  work,  the  first  imej  journals  can  slowly  but  steadily  show  the  potential  that  multimedia 
interactivity  holds  to  promote  scholarship  and  learning,  and  fulfill  the  promise  of  this  new  forum  for  academic 
publication. 
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Figure  1:  Home  page  of  The  IMEJ  of  Computer-Enhanced  Learning 


4.  How  Are  We  Doing  So  Far? 

A biannual  publication.  The  IMEJ  of  Computer-Enhanced  Learning  released  its  third  issue  in  April  of 
2000.  The  first  issue  had  eleven  articles,  the  second  had  nine,  and  the  third  had  seven.  The  smaller  number  of 
articles  over  time  has  resulted  from  our  realization  of  the  amount  of  time  required  for  multimedia  development. 
Aside  from  the  peer  review  of  articles,  all  the  editing  and  production  for  the  journal  is  done  by  Wake  Forest 
professors.  Academic  Computing  Specialists,  and  student  assistants  as  a sideline  from  their  regular  work, 
motivated  by  an  interest  in  multimedia  development  and  computer-enhanced  learning.  No  subscription  fee  is 
charged  for  journal  access.  Wake  Forest  University  supports  the  journal's  costs  as  an  investment  in  educational 
technology. 

We  regularly  receive  email  from  readers  interested  in  joining  our  project  in  an  editorial  capacity,  and 
so  far  sixteen  volunteers  have  been  added  to  our  pool  of  reviewers,  twelve  from  the  United  States  and  the 
remainder  from  Greece,  New  Zealand,  Germany,  and  Canada.  Our  readership  is  steadily  growing,  as  evidenced 
by  a trace  of  accesses  to  our  server.  Hits  have  grown  by  about  50%  since  the  launch  of  the  journal,  with  a 
steady  increase  in  unique  visitors. 


BEST  COPY  AVAILABLE 


The  number  of  yearly  submissions  to  IMEJ  is  small,  and  our  experience  in  this  regard  is  not  unique 
among  electronic  journals.  In  a May  1999  issue  of  The  Chronicle  of  Higher  Education,  Vincent  Kieman 
reports  that  electronic  publications,  such  as  The  Chicago  Journal  of  Theoretical  Computer  Science,  Earth 
Interactions  (both  from  MIT  Press)  and  Open  University's  Journal  of  Interactive  Media  in  Education,  receive 
only  about  a dozen  submissions  per  year.  Kieman  suggests  that  a perception  of  impermanence  and  a fear  that 
such  publications  will  not  count  toward  tenure  and  promotion  are  the  main  drawbacks.  Electronic  journals  that 
are  also  published  in  hard  copy  versions  seem  to  have  the  greatest  success.  Connections  with  online  archives, 
where  preprints  of  articles  can  be  posted  directly  by  authors,  are  a boost  to  an  emerging  electronic  journal  as 
well.  The  Journal  of  High  Energy  Physics  (http://ihep.sissa.it/),  for  example,  selects  papers  from  the  physics 
electronic  archive  at  Los  Alamos  National  Laboratory,  where  authors  can  post  their  preprints  by  filling  out  a 
form.  By  this  means  they  receive  a large  number  of  articles. 

The  majority  of  the  articles  for  IMEJ  thus  far  have  been  gleaned  from  presentations  at  EDMEDIA 
conferences  where,  with  the  endorsement  of  AACE,  we  have  selected  representative  "best  works"  to  be  updated 
and  augmented  for  publication  in  interactive  multimedia  form.  This  has  helped  us  to  overcome  the  startup 
problems  any  new  journal  faces,  particularly  one  in  a relatively  new  and  academically  non-traditional  medium. 
We  hope  that  we  have  begun  to  show  proof  of  the  production  quality  and  editorial  standards  possible  in  an  imej 
journal,  and  we  are  approaching  the  critical  mass  of  articles  needed  to  convince  contributors  that  our  journal  is  a 
credible  --  even  a desirable  — place  to  submit  their  best  work. 


5.  What  Lies  Ahead? 

Where  do  we  hope  to  go  with  /me/' journals  in  the  next  ten  years?  What  is  the  best  that  /me/' journals 
might  have  to  offer? 

Initially,  prognosticators  were  most  excited  about  the  increased  level  of  scholarly  discourse  that  online 
discussions  and  peer  review  offered  in  electronic  publication.  Computers  have  undoubtedly  increased  the  ease 
and  amount  of  communication  among  researchers.  But  in  our  opinion,  bulletin  board  discussions  and  electronic 
peer  review  are  not  where  the  greatest  potential  lies  in  /me/' journals  within  academia. 

It  is  in  the  presentation  of  research  results  themselves  where  we  stand  to  gain  the  most  in  interactive 
multimedia  publications.  We  imagine  scholarly  journals  that  offer  more  direct  access  to  and  verification  of  our 
colleagues’  research.  Experiments  can  be  replicated  with  the  reader's  own  data.  Computer  programs  can  be  run 
and  their  reported  efficiency  checked  on  a variety  of  input.  Graphs  can  be  redisplayed  with  new  data. 
Simulations  can  be  tested.  Medical  procedures  can  be  demonstrated.  Molecules  can  be  rotated.  Architectural 
spaces  can  be  explored.  Art  work  can  be  studied.  Atomic  and  subatomic  particles  can  be  visualized.  Sounds 
can  be  played  and  examined  graphically.  Eventually,  even  surfaces  can  be  explored  tactilely.  This  mutual 
testing,  replication,  experimentation,  and  verification  of  research  results  among  scholars  inspires  further 
research  and  creative  collaborations  across  disciplines  and  across  the  globe.  We  believe  that  such  advances  in 
scholarship  and  learning  are  truly  worth  our  efforts  to  overcome  the  obstacles  that  lie  in  the  way. 
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Abstract:  We  have  developed  a collaborative  learning  environment  (CLE)  as  a student- 
centred  approach  to  lecture  replacement,  with  a special  focus  on  assisting  students'  learning  of 
difficult  concepts.  The  majority  of  the  program  is  structured  around  cost-efficient  web- 
delivered  tutorials  incorporating  re-usable  interactive  components.  These  are  supported  by 
several  stand-alone  computer-based  learning  tutorials  including  ones  that  we  developed  to 
allow  students  to  construct  their  own  models  of  physiological  mechanisms,  together  with 
computer-facilitated  semester-long  investigative  projects  to  enhance  their  communication  and 
critical  reasoning  skills.  Each  week  for  two  hours  during  two  semesters,  students  work  in 
groups  of  three  with  an  iMac  computer.  The  computer-facilitated  tasks  are  designed  to  support 
and  extend  their  three  weekly  lectures  by  encouraging  peer-learning  and  peer-teaching.  In  this 
article,  the  successful  attributes  of  this  collaborative  learning  environment  are  described  and 
evaluated.  In  addition,  relationships  between  the  use  of  CLE  and  the  students'  approaches  to 
learning  are  being  investigated. 
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Introduction 


Tertiary  institutions  are  moving  increasingly  towards  the  delivery  of  courses  using  computers  to 
provide  students  with  the  opportunities  to  learn  at  their  own  pace,  together  with  a reduction  in  traditional 
lectures.  There  is  also  a trend  to  provide  asynchronous  access  to  courses  via  the  Internet.  There  is  broad 
recognition  that  traditional  forms  of  university  course  delivery  are  inappropriate  in  preparing  students  for  a 
dynamic  workforce  in  a post-industrialist,  knowledge  society  (Drucker  1995,  Lohrey  1995).  Where  it  has  been 
successfully  implemented,  collaborative  learning  is  recognised  as  a potent  transition  factor  in  supporting  the 
development  of  higher  order  cognitive  abilities  (Johnson  & Johnson  1992).  Despite  this  recognition,  evidence 
of  direct  positive  effects  on  student  learning  remains  largely  anecdotal  (Meloth  1999).  Face-to-face 
collaborative  learning,  with  academics  as  facilitators,  is  an  area  where  campus-based  universities  have  a major 
advantage  in  tertiary  education.  We  contend  that  to  provide  such  collaborative  learning  opportunities  has 
advantages  over  relying  only  on  electronic  communication  in  virtual  study  groups.  However,  amidst 
contracting  resources  and  increasingly  crowded  curricula,  there  is  an  understandable  reluctance  to  establish  the 
substantially  different  and  potentially  risky  conditions  of  learning  necessary  for  successful  collaborative 
learning  (Cooper  & Sweet  1999). 

We  have  now  evolved  a collaborative  learning  environment  (CLE)  in  which  we  develop  and  present 
computer-facilitated  learning  as  a means  of  effectively  broadening  the  learning  opportunities  of  science  students 
studying  physiology.  Since  1993  our  efforts  in  computer-assisted  learning  have  aimed  to  extend,  enhance  and 
replace  some  of  the  students'  lecture  experiences  with  multimedia-based  tutorials  in  an  environment  that 
encourages  peer-learning  and  peer-teaching.  A global  aim  in  our  science  teaching  in  physiology  is  to  use  the 
curriculum  to  develop  in  our  graduates  some  understanding  of  the  abilities  required  of  them  as  practicing 
scientists.  Emphasis  is  given  to  understanding  the  experimental,  research,  theoretical,  communication  and 
critical  reasoning  base  of  the  discipline. 

An  attribute  of  key  importance  to  development  of  our  CLE  was  to  bring  different  experiences  to 
students  learning.  This  diversity  was  essential  to  address  the  wide  range  of  backgrounds  of  the  students  entering 
our  courses,  such  as  their  achievement  levels,  approaches  to  learning  and  cultural  backgrounds.  Our  approaches 
included  introducing  scientific  principles  and  methods  for  investigating  physiology  (eg  simulated  experiments), 
and  using  a diverse  range  of  computer-aided  interactive  teaching  formats.  Importantly  these  approaches  were 
not  designed  to  stand  alone,  rather  to  integrate  with  other  learning  resources  such  as  lectures  and  printed  texts. 
Students  were  also  encouraged  to  engage  in  continuous  reflection  and  evaluation. 

Our  concept  of  a collaborative  learning  environment  consists  of:  a friendly  and  informal  physical 
workspace  that  is  conducive  for  group  interactions,  with  an  optimum  group  size  of  three  students  per  computer; 
an  economical  production  model  that  provides  a coherent  set  of  weekly  web-based  problems;  supplementary 
standalone  highly-interactive  multimedia  tutorials  dealing  with  essential  concepts;  a tutor  to  guide  and  assist 
(not  teach);  and  extension  after  hours  into  the  virtual  classroom  of  the  internet  using  electronic  communications. 

Results  so  far  suggest  that  the  introduction  of  our  collaboratively  based  approaches  to  course  delivery 
has  had  a significant  positive  effect  on  student  learning  outcomes.  These  outcomes  include  not  only  traditional 
examination  results,  but  also  more  ‘authentic’  (Wiggins  1992)  methods  of  assessment  such  as  observation  of 
group  role  fulfillment  and  student  audit  trails  of  problem  solving  processes. 


Background 

The  Department  of  Physiology  at  the  University  of  Melbourne  has  played  a leading  role  in  computer- 
assisted  learning  over  many  years.  In  1985  we  first  tried  to  improve  our  practical  class  teaching  of  physiology 
with  pilot  studies  using  the  Apple  II  and  a computer-oscilloscope  for  recording  from  human  nerves, 
accompanied  by  a computer-aided  tutorial  relating  the  theory  and  practical  experience  (Delbridge  & Kemm, 
1985).  However,  it  was  not  until  1992  when  we  purchased  64  Mac  Ilci  computers,  each  with  a MacLab 
physiological  recording  system,  that  we  really  had  the  computer  performance  to  significantly  improve  our 
students'  practical  classes  and  then  consider  multimedia  tutorials  as  an  additional  option. 

Since  1 993  we  have  been  continually  successful  with  our  applications  for  competitive  government  and 
University  funds  to  produce  standalone  multimedia  tutorials  that  aim  to  help  students  understand  difficult 
physiological  concepts.  The  Physiology  Department  has  always  been  a leader  in  multimedia  development  at  the 
University  of  Melbourne  and  produced  the  first  site-licensed  multimedia  tutorial  sold  internationally  in  1993. 
Since  then  we  have  been  producing  and  evaluating  a series  of  tutorials  that  require  students  to  build  and  test 
their  own  models  of  physiological  processes,  and  the  first  of  these  is  now  being  distributed  internationally 
(Weaver  et  al,  1999).  However,  although  these  initiatives  have  proven  to  be  very  effective,  production  of  these 
highly  interactive  tutorials  was  slow  and  expensive,  so  they  could  not  be  used  to  provide  two  semesters  of 
coursework  for  our  science  students  within  the  time  frame  and  budget  available. 
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In  1997,  the  Department  introduced  fortnightly  scheduled  3-hour  sessions  of  computer  assisted 
learning.  Initially  these  CAL  sessions  were  used  to  revise  lecture  material.  Tutorials  were  based  on  HyperCard 
programs,  produced  in-house  by  one  of  the  authors  (using  Chemistry’s  ’Tutorial  Tools’  by  Paul  Fritze),  and  a 
variety  of  other  standalone  tutorials  obtained  nationally  and  internationally.  However,  although  the  in-house 
tutorials  were  well  received,  the  standard  of  external  tutorials  received  a very  mixed  response 

In  1998,  we  produced  a coherent  set  of  in-house  tutorials  specifically  designed  for  our  students, 
supplemented  by  a select  few  of  the  best  available  standalone  tutorials.  Content  was  delivered  economically  on 
an  Intranet  for  a Web  browser,  using  templates  and  tools  developed  in  association  with  Paul  Fritze  of  the 
University's  Multimedia  Education  Unit.  The  model  for  tutorials  was  that  students  work  on  a problem  in  groups 
of  three  with  the  computer  providing  questions,  feedback  and  hints  as  they  proceeded  with  calculations  and 
exploring  issues.  These  1.5  hr/week  CAL  sessions  were  not  compulsory  and  were  held  in  a Science  Faculty 
multimedia  computer  laboratory  remote  from  the  Department.  There  was  no  formal  assessment  of  this  course 
component.  Attendance  fell  over  each  semester,  largely  because  of  the  many  technical  problems  experienced 
with  the  unsatisfactory  maintenance  of  the  software  and  hardware  and  the  remoteness  of  the  laboratory  from  the 
support  systems  within  the  Department.  There  was  no  significant  change  in  student  performance. 

For  the  1 999  teaching  year  the  Department  of  Physiology  equipped  a refurbished  laboratory  with  1 5 
iMac  computers  and  a G3  server.  Additionally,  we  introduced  group  learning  of  a topic  covered  in  the 
curriculum,  and  assessed  student  participation  in  the  collaborative  learning  process  to  encourage  attendance. 
The  duration  of  these  computer  assisted  learning  sessions  was  increased  to  2 hours.  All  this  helped  staff  to 
provide  better  support  and  to  emphasise  that  the  CLE  was  as  important  as  lectures  for  learning  physiology. 

In  2000,  we  are  reflecting  on  all  aspects  of  the  Web  delivered  course  and  upgrading  it  where 
appropriate  with  new  interactive  Web  components  that  will  allow  us  to  provide  more  directed  feedback  to 
students  with  different  backgrounds  and  achievement  levels  (Fritze  & Kemm,  2000).  We  are  continuing  to 
develop  a few  highly  interactive  multimedia  modules  dealing  with  difficult  concepts  for  which  students  develop 
their  own  mechanistic  models. 


The  Current  Status  of  the  Evolved  Collaborative  Learning  Environment 

The  evolution  of  our  collaborative  learning  environment  is  the  result  of  adjustments  made  in  response 
to  many  surveys  and  educational  experts’  observations  of  how  students  go  about  their  learning  when  facilitated 
by  computer  programs.  The  CLE  sessions  can  be  regarded  as  a medium  between  students  who  actively 
construct  meaning  and  skills,  using  resources  of  the  environment  that  include  discipline  content,  methods  and 
experiences,  such  as  laboratory  work,  demonstrations,  real-life  experience  and  particularly  social  interactions. 

CLE  Sessions  and  the  Physical  Environment: 

Students  attended  sessions  of  2 hours  weekly,  with  12  sessions  in  each  of  two  semesters.  This  part  of 
the  course  was  first  assessed  in  1999  and  counted  for  5%  of  the  course  marks  (see  section  on  Assessment). 
Most  of  the  360  students  enrolled  for  CLE.  Attendance  was  remained  high  throughout  semester  in  the 
Department's  own  facility. 

Each  collaborative  learning  environment  (CLE)  session  accommodated  42  students  normally  working 
in  groups  of  three  on  a computer  assisted  learning  (CAL)  package  presented  on  an  iMac  computer.  Groups  of 
three  were  chosen  as  is  claimed  to  be  the  optimum  group  size  for  interactive  computer  programs.  A G3  Server 
used  AppleShare  6.1  IP  to  network  the  14  students'  iMacs  and  the  tutor’s  iMac.  The  tables  and  benches  were 
arranged  to  encourage  groups  to  talk  and  argue  in  a relatively  noisy  working  environment.  The  tutor's  approach 
further  emphasised  the  friendly  and  relatively  informal  learning  environment.  A wide  range  of  textbooks  was 
provided  as  student  resources.  The  head  CAL  tutor  and  developers  were  located  next  door  to  the  laboratory. 

Computer  Assisted  Learning  (CAL)  Components: 

Our  overall  approach  to  our  CAL  development  used  a constructivist  approach  and  addressed  both 
pedagogical  and  instructional  design  issues  from  the  outset,  such  as  using  guidelines  from  the  work  of  Reeves 
and  Harmon  (1994)  that  was  designed  for  the  evaluation  of  completed  multimedia.  We  used  a group 
development  model  with  assistance  from  education  experts,  instructional  designers,  as  well  as  academics  and 
multimedia  authors  who  developed  and  delivered  the  material  on  the  Web.  We  used  an  iterative  process,  to 
adjust  tutorials  in  line  with  results  of  formative  assessment  with  students  throughout  the  development. 

A true  multi -media  approach  was  adopted  where  students  used  a combination  of  paper  based  work 
textbook  referencing  and  interactive  computer  delivered  tasks,  side  by  side.  The  idea  was  to  develop  activities 
that  would  allow  students  to  build  on  their  own  knowledge  within  their  own  learning  framework,  with  the 
computer  available  as  a virtual  tutor  to  help  them  check  their  answers,  provide  a level  of  tutorial  feedback  and  to 
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pose  further  questions  to  stimulate  their  interpretations.  The  problems  were  initially  based  on  the  areas  of 
difficulties  in  physiology  that  staff  had  previously  identified  in  conventional  tutorials. 

Students  were  presented  with  problems  to  investigate  that  required  them  to  examine  experimental  data, 
perform  calculations,  answer  questions  and  keep  written  records  of  their  work.  A wide  variety  of  tasks  were 
presented  on  the  Web,  occupying  about  1.5  hours  of  each  CLE  session.  Production  of  sufficient  material  for  24 
weeks  required  an  efficient  production  method  that  could  be  easily  modified  on  reflection  after  evaluation. 

These  weekly  problems  were  supplemented  by  a variety  of  highly  interactive  standalone  tutorials  that 
could  occupy  up  to  one  hour  of  CAL  activities.  These  included  two  of  our  set  of  highly  interactive  cell-model 
building  multimedia  tutorials  (Weaver  et  al,  1996;  Kemm  et  al,  1997;  Weaver  et  al,  1999a  & 1999b)  as  well  as 
externally  produced  modules.  Students  were  given  the  opportunity  at  the  end  of  the  session  and  during  the  last 
session  of  each  semester  to  test  their  knowledge  with  a variety  of  multiple  choice  and  open-ended  questions. 


Semester-long  Tasks: 

In  Semester  I:  Students  formed  extended  groups  (of  six)  to  investigate  one  of  four  problems,  such  as  the 
consequences  of  blood  doping  on  physiological  systems.  This  semester  long  project  was  introduced  with  the 
aim  of  students  learning  to  be  perceptive  in  their  reading  of  short  excerpts  of  scientific  writing.  They  were 
required  to  determine  the  key  phrases  in  the  passage,  their  relative  importance  and  then  to  write  a concise  clear 
report  on  their  interpretation  of  that  piece.  This  project  was  designed  to  assist  students  to  develop  generic  skills 
in  determining  the  level  of  knowledge  required  and  writing  abilities  to  communicate  their  knowledge  more 
effectively. 

Each  group  was  given  a weekly  series  of  tasks,  researching  the  material  and  progressively  developing 
their  interpretation.  30-minute  periods  at  the  end  of  each  CLE  session  were  designated  for  these  assignments. 
Students  submitted  work  on  the  various  stages  to  their  own  private  computer-based  discussion  forum.  They 
were  provided  with  weekly  feedback  on  their  progressive  submissions  as  a computer  checklist,  and  at  the  end  of 
six  weeks  submitted  their  report  to  another  group  for  peer  review.  The  reviewing  group  was  within  the  same 
class  and  had  worked  on  the  same  question,  therefore  were  'expert  reviewers'. 

We  used  the  learning  framework  'TopClass'  to  allow  students  to  communicate  within  their  group,  and 
to  keep  track  of  their  progressive  work  submissions  on  a semester  long  project.  Science  students  study  such  a 
wide  range  of  subjects  that  it  is  difficult  for  them  to  form  cohesive  groups  to  assist  their  learning  (contrasted 
with  professional  courses,  eg.  dentistry),  so  that  electronic  communication  is  encouraged  to  reinforce  the 
exchanges  that  occur  in  the  weekly  CLE  sessions. 

The  key  learning  issues  were  to  work  cohesively  as  a group  and  make  consensus  decisions  about 
identifying  and  ranking  key  concepts,  being  able  to  express  themselves  concisely,  unambiguously,  accurately 
and  be  able  to  read  another  text  with  some  insight  as  to  what  is  expected...  In  essence,  we  aimed  to  improve 
their  collective  skills  in  negotiation,  communication,  reading,  and  writing  and  reviewing  their  science. 

In  Semester  2:  We  chose  to  build  on  the  collective  skills  of  collation  and  consensus  writing  undertaken  in  the 
semester  with  a course  aimed  at  providing  students  with  some  insight  into  critically  reading  their  Physiology. 
We  formally  applied  reasoning  skills  to  a graded  series  of  reading  tasks  using  the  interactive  program  'Reason!'. 
Although  we  have  reported  on  this  process  applied  to  third  year  students  (van  Gelder  et  al,  1999),  the 
introduction  at  this  time  was  only  done  as  a trial.  The  second  year  students  were  not  as  successful  in  applying 
'Reason!',  and  some  modifications  will  be  needed  to  allow  this  less  sophisticated  group  to  gain  more  benefit 
from  the  process. 


Course  Assessment: 

CLE  Assessment:  The  CLE  assessment  (5%  of  total)  was  aimed  at  rewarding  students  who  effectively 

participated  in  these  sessions.  The  mark  was  based  on  attendance  and  tutors'  records  of  student  participation  (by 
observation  of  a group's  activities  and  their  submissions,  together  with  individual  student's  roles  in  discussion 
and  record  keeping).  A student  must  have  attended  the  majority  of  sessions  be  deemed  to  have  participated 
effectively  in  group  learning  (tutor's  observations  and  computer  submissions)  to  have  gained  a score  of  greater 
than  3/5  (rated  as  good  in  Table  1). 

Examinations:  These  are  described  in  a following  Section,  which  discusses  attempts  to  relate  the  participation 
in  CLE  with  examination  performance. 
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Evaluation  Strategies 


We  have  a number  of  evaluation  strategies  in  place  and  collected  data  in  1998  and  1999,  as  part  of  our 
overall  action  research  strategy  dealing  with  global  learning  outcomes  from  the  CLE  as  well  as  in  students' 
extensipn  of  cell  model  building  skills  to  new  or  hypothetical  situations.  We  used  a comprehensive  set  of 
methods  to  collate  and  assess  information,  in  order  to  gain  a broad  understanding  of  student  reactions  and  to 
obtain  a more  detailed  examination  of  student  learning  processes  and  actions.  We  obtained  human  ethics 
approval  for  our  surveys  and  to  log  important  student  activities  in  the  computer  tutorials 

The  tutor-facilitators  play  a key  role  in  the  implementation  of  the  program  so  their  impressions  of  the 
course  are  most  relevant  to  understanding  student  reactions.  They  made  observations  and  kept  records  of 
students'  work  and  participation  in  the  CLE  sessions.  Formative  evaluation  continued  throughout  with  regular 
formal  meetings  between  the  main  developer  and  tutors,  as  well  as  many  informal  interactions  amongst  the 
students,  tutors  and  the  academic  developers  whose  nearby  location  enabled  and  encouraged  this  latter  process. 

We  have  found  that  students  cooperate  well  with  questionnaires  and  interviews  if  they  are  fully 
informed  and  can  see  that  they  are  developmental  and  that  timely  feedback  will  assist  their  own  learning.  This 
is  unlike  their  less  enthusiastic  response  to  repeated  general  University  circulated  questionnaires  on  teaching 
quality  in  each  course. 

Student  Questionnaires 

Questionnaires  specific  to  the  CLE  were  used  to  survey  students'  attitudes  to  various  aspects  of  the 
CAL  tutorials  and  the  CLE  sessions,  in  consultation  with  our  educational  advisors.  These  were  supplemented 
by  focus  group  interviews  with  students. 

In  addition  we  investigated  students'  self-assessment  of  their  approaches  to  learning.  We  used  a 
modified  study  process  questionnaire  to  extend  the  investigation  of  deep,  achieving  and  surface  learning- 
approaches  (Biggs,  1 987)  so  it  included  additional  learner  characteristics.  This  is  discussed  in  application  to 
one  of  standalone  interactive  tutorials  (Kemm  et  al,  1997),  Such  additional  information  will  be  used  to  further 
analyse  the  results  reported  in  this  article  at  a later  time. 

The  student  questionnaires  were  modified  each  semester  from  initially  determining  general  reactions  to 
the  style  and  depth  of  coverage  of  the  tutorials,  to  focus  on  particular  issues  that  were  relevant  to  the  efficacy  of 
the  collaborative  learning  environment.  The  study  process  questionnaire  was  administered  once  each  year.  The 
last  questionnaires  had  approximately  70  questions  designed  to  reveal  students'  attitudes  and  use  of  the  CLE, 
covering  aspects  such  their  pattern  of  work  with  the  CLE,  development  of  independent  learning  skills,  relevance 
of  the  CLE  to  their  learning  compared  with  lectures,  and  their  attitudes  to  group  work.  Additionally  we  had 
some  specific  questions  about  their  use  of  the  feedback  screens  in  their  learning  and  about  the  nature  of  some 
tutorials,  as  part  of  our  interest  in  learning  which  tutorial  styles  worked  best  and  for  which  students.  Most 
questions  required  students  to  rank  their  responses  on  a 5-point  scale,  supplemented  by  several  open-ended 
questions. 


Results 

Influences  on  Assessment: 

The  aim  was  to  determine  whether  any  part  of  the  CLE  assisted  students  to  have  a better  examination 
outcome.  In  all  courses,  examinations  consist  of  multiple  choice  questions  (MCQ),  short  answers  questions 
(about  10  min)  and  essays  (about  30  minutes).  The  written  examination  questions  were  set  so  that  one  question 
related  to  material  only  covered  in  lectures,  one  specifically  designed  around  material  obtained  in  CLE  sessions 
and  one  question  specifically  aligned  to  the  semester  long  CLE  project  (choice  of  the  four  projects  undertaken). 
Multiple  choice  questions  (MCQ),  covering  the  whole  syllabus,  were  no  different  in  their  coverage  from 
previous  years.  Faculty  Scores  for  student's  year  performance  (average  of  best  75%  of  subjects)  were  used  to 
give  an  idea  of  the  students'  previous  achievement  levels. 

Table  1 shows  the  influence  of  effective  participation  in  the  CLE  sessions  on  the  student  performance. 
Student  groups  were  divided  into  “levels  of  achievement”  based  on  their  previous  1st  year  academic 
performance.  Subgroups  based  on  CLE  participation  ratings  used  scores  of  3.5  and  3,  a score  that  should 
distinguish  combined  attendance  and  participation  from  non-attendance  or  attendance  alone  (see  course 
assessment).  The  data  in  Table  1 shows  a marked  polarisation  in  the  mean  scores  between  these  subgroups. 
Thus  we  are  confident  that  the  subgroups  did  reflect  those  students  who  were  involved  in  the  material  and  the 
learning  process,  and  those  who  were  not.  The  data  suggests  that  there  is  an  improvement  in  exam  outcome  for 
all  groups  with  good  CLE  participation,  with  a greater  proportional  change  for  the  “lower  achieving”  students. 
The  examination  marks  have  the  CLE  component  removed  and  are  normalised,  so  a perfect  score  is  100%. 
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Prior  Faculty 
Score  Grouping 

CLE  Participation 
Rating  (Subgrouping) 

Number 
in  CLE 

Mean  Prior 
Faculty  Score  % 

Total  Exam 
Mark  % 

CLE  Score 
(/5) 

15 

good 

27 

79  ± 0.5 

71  ± 2 

4.7  ± 0.1  » 

poor/none 

11 

78  ± 1.0 

67  ± 3 

1.9  ± 0.4  ♦ 

61-74 

good 

76 

68  ± 0.3 

63±  1 ♦ 

4.5  ± 0.1  ♦ 

poor/none 

39 

67  ± 0.5 

54±  2* 

1.3  ± 0.2* 

60 

good 

63 

57  ± 0.5 

51  ± 2* 

4.3  ± 0.1  * 

poor/none 

39 

56  ± 0.8 

45±  2* 

1.4  ± 0.2* 

* probability  < 0.05  for  the  good  and  poor/none  pairing  for  each  subgroup 

Table  1:  Influence  of  collaborative  group  interactive  CLE  sessions  on  the  results  with  each  subgroup 

Table  2 shows  the  distribution  of  examination  marks  for  science  students  dependent  on  their 
participation  in  the  CLE  sessions.  The  data  shows  a shift  to  higher  grades  for  those  students  participating  in 
CLE,  again  most  significantly  for  the  lower  achieving  students.  Of  interest  is  that  55%  of  the  students  in  this 
group  who  attended  and  participated  in  CLE  passed.  By  contrast  only  35%  passed  of  the  group  who  chose  not 
to  participate/attend. 


First  Year 
Faculty  Score 
Grouping 

CLE 

Participation 

Subgrouping 

>80 

Range  of  Physiology  Examination  Marks 
70-79  60-69  50-59 

<50 

75 

good 

30 

22 

41 

7 

0 

poor/none 

18 

27 

27 

18 

9 

61-74 

good 

7 

24 

33 

24 

11 

poor/none 

0 

6 

34 

34 

26 

60 

good 

0 

7 

32 

22 

39 

poor/none 

0 

0 

12 

23 

65 

Analyses  are  expressed  as  percentages  of  subgroups 

Table  2:  Effects  of  CLE  participation  on  distribution  of  examination  results  of  science  students  taking  2""^  year  Physiology. 


Table  3 shows  the  influence  of  the  CLE  participation  on  the  major  components  of  the  examination,  and 
the  improvement  is  apparent  across  both  the  written  and  multiple  choice  components.  We  had  hoped  that  the 
semester  long  written  assignment  in  particular  may  have  enhanced  the  Science  students’  performance  in  the 
written  part  of  the  examination.  If  this  were  true  then  the  written  part  of  the  exam  should  have  been  done  better 
by  the  group  taking  CLE,  but  the  MCQ  mark  might  be  expected  to  be  independent  of  CLE  participation.  The 
data  however  show  improvement  in  both  the  written  and  multiple  choice  exam  components  for  those  in  the  high 
participation  group.  Accordingly,  from  the  performance  data  alone,  there  is  no  such  clear-cut  distinction  that 
might  indicate  the  influence  of  the  different  components  in  the  CLE.  It  is  suggested  that  the  apparent 
improvement  in  learning  skills  was  influenced  by  the  different  components  of  the  CLE  (e.g.  the  CAL  tutorials). 


Prior  Faculty 
Score 
Grouping 

CLE 

Participation 

Subgrouping 

n 

Total  Exam 
Mark 
(%) 

Written  Exam 
Component 
(/50) 

Multiple  Choice 
Exam  Component 
(/50) 

CLE 

Participation 
Mark  (/5) 

75 

good 

27 

71  ± 2 

33  ± 1 

38  ± 1 

4.7  ± 0.1  * 

poor/none 

11 

67  ± 3 

30±  2 

51  ±1 

1.9  ± 0.4* 

61-74 

good 

76 

63  ± 1 * 

30  ± 1 * 

33  ± 1 * 

4.5  ± 0.1  * 

poor/none 

39 

54  ± 2* 

25  ± 1 * 

30  ± 1 * 

1.3  ± 0.2* 

60 

good 

63 

51  ± 2 * 

23  ± 1 

28  ± 1 

4.3  ± 0.1  * 

poor/none 

39 

45  ± 2 * 

20  ± 1 

25  ± 1 

1.4  ± 0.2* 

* probability  < 0.05  for  the  good  and  poor/none  pairing  for  each  subgroup 

Table  3:  influence  of  CLE  Participation  on  Components  of  the  Examination 

The  final  observation  from  the  assessment  was  a marked  reduction  in  the  failure-rate  to  <15% 
compared  with  25%  over  many  previous  years.  It  should  be  noted  that  we  have  not  been  able  to  reduce  this 
failure  rate  in  previous  modifications  of  our  teaching  to  assist  the  students  most  at  risk  who,  as  might  be 
expected,  were  those  students  with  a poor  L*  year  performance. 
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Successful  Attributes  of  the  CLE 


As  will  be  seen  from  the  data  analysis  above,  it  was  not  possible  to  attribute  changes  in  learning 
outcomes  to  any  particular  aspect  of  our  CLE.  We  know  from  previous  experience  that  many  factors  can  be 
detrimental,  such  as  the  students  not  realising  that  concepts  and  material  covered  in  the  CLE  are  as  important  as 
lectures,  and  that  students  are  intolerant  of  poor  quality  material  or  unreliable  hardware  or  software. 

We  believe  that  the  successful  learning  outcomes  are  due  to  a combination  of  successful  attributes  of  the  CLE, 
identified  by  the  student  surveys  and  observations,  and  these  are  presented  in  the  following  sections. 

The  Physical  Environment  for  the  CLE: 

The  ease  of  installation  and  reliability  of  the  iMac  computers  and  the  G3  Server  has  been  crucial  to  the 
success  of  the  project.  They  required  minimal  maintenance  since  the  beginning  of  1999.  Also,  attention  to 
software  protection  and  automatic  overnight  reinstatement  of  any  student  modifications  of  the  desktop 
eliminated  significant  problems  that  we  had  in  teaching  in  an  open-access  laboratory  in  previous  years. 

Our  surveys  showed  that  74%  of  students  preferred  to  work  in  groups,  showing  a minority  of  students 
still  liked  to  work  alone.  At  the  beginning  of  the  semester,  more  high  achieving  students  preferred  to  work 
alone  but  many  changed  their  minds,  having  experienced  the  benefits  of  group  work.  There  may  also  be 
cultural  differences  in  these  preferences.  Focus  group  interviews  confirmed  these  results  and  the  major 
advantage  seen  from  group  work  were  the  interactions  and  discussions,  which  were  confirmed  by  the  noise 
levels  in  the  classes. 

The  scheduling  of  sessions  for  group- work  was  crucial  for  collaboration,  since  our  experience  of  other 
courses  with  open-access  computer  laboratory  sessions  shows  that  most  students  tend  to  work  alone,  a few  in 
pairs  and  rarely  in  larger  groups.  We  believe  this  is  a critical  issue  to  encourage  collaborative  learning, 
especially  in  Science  where  the  diversity  of  courses  that  students  undertake  make  it  difficult  for  them  to  find 
collaborators  with  similar  timetables. 

The  CLE  sessions  induced  a 'sense  of  belonging'  to  a community  within  a course  where  the  students 
come  from  quite  disparate  backgrounds,  generating  more  of  the  interactions  that  exist  in  our  more  selective 
professional  courses.  There  was  identification  within  each  session  of  40,  rather  than  just  being  an  anonymous 
student  amongst  the  class  of  350.  The  open  discussions  and  studying  together  also  reinforced  a common 
approach  to  scientific  investigation  and  enhanced  their  appreciation  of  the  philosophical  approaches  to  learning. 

The  modem,  pleasant  and  comfortable  surroundings  were  found  to  encourage  lively  interaction,  the 
main  features  being  the  open  layout,  lighting,  carpets  and  comfortable  seating.  Furthermore,  its  location  within 
the  Department  and  easy  access  to  support  staff  and  academics  next  door  showed  the  commitment  of  the 
Department  to  assisting  the  students.  This  sense  of  personalised  support  was  reinforced  by  having  a tutor  look 
after  each  class  and  by  providing  easy  email  access  by  students  to  staff  specialists.  Lastly  reference  should  be 
given  to  hugely  successful  paisley  60s  style  couch  in  the  adjoining  corridor,  where  many  successful  physiology 
and  non-physiology  related  communication  skills  were  reinforced. 


Web  Delivered  CAL  Components: 

It  is  not  possible  to  relate  the  learning  outcomes  to  any  specific  CAL  components.  There  is  insufficient 
space  to  describe  all  the  characteristics  of  the  computer-assisted  web  components  that  worked  well  with  our 
students  in  the  CLE  sessions,  but  the  highlights  are  as  follows,  including  some  representative  screen  shots: 

Weekly  tasks  not  only  had  a Web-delivered  component,  but  also  were  accompanied  by  a paper  task 
sheet  indicating  the  scope  of  the  task  for  the  session.  This  gave  the  students  some  take-away  material  that  could 
be  used  in  later  study/reflection.  In  addition  students  were  required  to  keep  a record  book  and  make  notes,  and 
do  calculations  as  they  progressed,  so  further  reinforcing  their  learning  and  providing  their  own  records. 

The  developed  Web-authoring  tools  were  essential  for  producing  the  core  of  the  material  for  the  24- 
weeks  of  CLE  sessions  in  a cost-effective  manner.  A single  important  feature  over  stand-alone  programs  is  that 
both  material  and  tasks  could  be  updated  relatively  easily  to  incorporate  new  knowledge,  or  change  emphasis  in 
tasks  with  changes  in  the  staff  presenting  lectures.  The  Interactive  Web  Objects  were  controlled  by  scripts  that 
were  readily  modified  by  standard  text  editors  and  we  used  Dreamweaver  to  manage  the  website.  This  meant 
that  'authoring'  of  the  web  pages  did  not  require  a programmer,  but  an  academic  with  some  skills  in  word 
processing  and  HTML  could  be  employed  instead.  This  allowed  the  focus  to  be  on  appropriate  pedagogy  and 
relevant  physiological  content.  The  collaboration  between  the  Multimedia  Education  Unit  and  the  Physiology 
Department  was  essential  to  both  bodies,  providing  Physiology  with  a suite  of  Web  based  tools  while  allowing 
the  authors  the  feedback  to  develop  and  test  them  using  pedagogical  requirements  firmly  grounded  in  the 
context  of  the  real  course. 
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• Students  indicated  that  the  diversity  of  tasks  and  web-presentations  were  stimulating  and  very  helpful 
for  their  learning.  These  included  calculations  with  relevant  human  data,  interactive  graphing  of 
relationships,  open-ended  answers  to  questions,  sorting  sequences  of  processes,  building  models  of 
cellular  functions.  Fig.  1 shows  a screen  showing  one  of  a sequence  of  tasks  requiring  investigation 
about  cell  communication.  This  requires  association  of  modes  of  cell  communication  with  a diagram  as 
a simple  revision  task. 


Figure  1:  Set  of  Tasks  listed,  one  of  which  is  shown  requiring  matching  of  mechanisms  to  diagrams. 

• A popular  format  (Fig.  2)  was  the  sorting  of  sequences  of  events  in  a physiological  process,  which 
students  found  challenging  and  rewarding  and  generated  much  discussion  as  they  explored  many  of  the 
possible  solutions.  As  developers,  we  did  not  predict  this  preference.  This  emphasised  that  continuous 
evaluation  of  what  is  working  is  essential,  since  academics  often  misjudge  what  students  will  find 
useful  and  challenging. 

• Our  students  have  a poor  understanding  of  the  significance  of  the  shape  of  graphs,  so  we  use  a free 
form  drawing  tool  (Fig.  3)  to  allow  them  to  represent  changes  that  they  might  expect  in  important 
relationships.  Here  they  have  drawn  the  straight  line  as  the  change  they  expect  in  the  supplied  curve. 
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Figure  2:  Sorting  a sequence  of  physiological  events.  Figure  3:  Showing  use  of  free-form  graphics  tool. 


• Students  used  the  textual  feedback  on  what  was  appropriate  or  inappropriate  about  their  responses  and 
found  this  to  be  very  helpful  in  their  learning.  The  feedback  was  designed  to  progressively  reinforce 
students'  development  of  an  appropriate  framework  for  their  solutions  to  problems  - such  as  their  own 
summaries  with  the  key  features  of  negative  feedback  control  by  hormones  (see  insets  in  all  Figures). 

• While  the  students  found  the  content  challenging,  the  computer-aided  tutorials  gave  them  a better  idea 
than  lectures  of  the  level  of  understanding  required  to  understand  physiological  mechanisms. 

• Students  developed  appropriate  learning  strategies:  such  as  exploring  mechanisms  using  interactive 
programs,  reinforcing  lecture  material  by  more  study,  and  by  reflection  of  the  material  and  by  redoing 
components  of  the  CAL. 
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standalone  Interactive  Multimedia  Components: 

Students  found  the  interactive  model-building  standalone  tutorials  to  be  challenging  and  very  helpful 
for  understanding  some  difficult  concepts.  However,  their  approaches  to  using  these  tutorials  is  being 
researched  separately  and  is  too  extensive  to  fully  report  here.  Previous  investigations,  (Kemm  et  al,  1997), 
showed  that  students  were  unable  to  construct  hypothetical  cells  from  their  experience  of  constructing  a cell  to 
secrete  stomach  acid.  A revision  tutorial  that  shows  the  principles  in  the  operation  of  membrane  transporters 
was  then  developed  to  allow  students  to  think  out  their  solutions  (Fig.  4)  and  apply  this  knowledge  to  new  cell 
models,  such  as  for  the  operation  of  cells  in  the  proximal  tubule  of  kidney  (Fig.  5).  We  are  currently 
investigating  this  transfer  of  model  building  skills  in  a controlled  study  to  see  if  we  can  increase  the  number  of 
students  who  think  out  solutions  rather  than  simply  use  trial  and  error.  We  use  audit  trails  to  investigate  the 
actual  steps  students  take  in  arriving  at  their  model  solutions.  In  later  modules,  more  emphasis  has  been  given 
to  placing  the  cell  modelling  tasks  in  context  with  overall  requirements  of  the  tissues  in  which  they  are  found. 
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Figure  4:  Animated  tool  to  show  transporter  operations.  Figure  5:  Student's  partially  constructed  model  of  a cell. 
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Figure  6:  Using  Neural  Elements  to  build  a Neural  Circuit 
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Figure  7:  Introduction  to  a Semester  Long  Project 


Another  approach  to  model  building  is  to  develop  neural  circuits  underlying  the  operation  of 
physiological  control  systems.  Students  found  our  first  version  of  a tutorial  on  the  control  of  blood  pressure 
very  difficult.  Our  analysis  of  audit  trails  and  questionnaires  showed  that  we  needed  to  separate  their  learning 
of  how  the  elements  of  the  model  worked,  using  a simple  'playground'  (Fig.  6)  that  also  allowed  them  to  develop 
a negative  feedback  neural  circuit.  They  were  then  better  able  to  deal  with  the  complex  neural  feedback 
pathway  involved  in  the  regulation  of  blood  pressure  when  moving  from  a lying  to  standing  position. 


Semester-long  Tasks: 

Students  undertook  the  semester  long  task  of  writing  about  a topical  issue  with  enthusiasm.  There  was 
lively  discussion,  although  they  were  quite  frustrated  by  some  features  of  the  TopClass  Framework  which  meant 
it  proved  less  than  optimal  for  their  purposes.  There  was  considerable  variation  in  the  standards  of  both  the 
final  submissions  and  the  quality  of  peer  reviews,  but  students  generally  produced  good  work.  Not  many  were 
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able  to  identify  all  of  the  requirements,  such  as:  the  key  issues  in  their  problem,  the  relative  importance  of 
various  factors,  and  to  produce  concise  and  unambiguous  writing  about  the  problem.  In  2000  we  have 
introduced  a new  means  of  for  students  to  develop  their  submissions,  receive  feedback  and  to  track  their  efforts 
using  more  effective  interactive  Web  tools  (OCCA  described  in  Fritze  & Kemm,  2000). 


Discussion: 

Learning  Outcomes: 

It  is  extraordinarily  difficult  to  use  traditional  quantitative  methods  to  show  differences  in  learning 
outcomes  attributable  to  a particular  curriculum  intervention  by  using  traditional  statistical  methods  (Reeves, 
1993).  Even  if  there  are  reasonable  control  groups,  large  numbers  are  often  necessary  to  overcome  the  many 
confounding  factors  influencing  the  students  in  studying  in  a course,  although  there  have  been  occasional 
successes  such  as  (Reeves  et  al,  1997). 

In  our  study,  we  were  fortunate  to  have  a self-selected  control  group  who  did  not  participate  effectively 
in  the  CLE.  Although  less  than  ideal,  we  were  not  permitted  to  design  an  experiment  in  which  some  students 
are  not  offered  this  learning  opportunity.  Nevertheless,  the  self-selected  group  did  represent  a broad  coverage  of 
student  achievement  and  there  was  no  difference  in  sub-groupings  based  on  prior  achievements  (first  year 
Faculty  Score).  The  significant  results  show  that  effective  CLE  participation  was  associated  with  an 
improvement  in  learning  outcomes  of  some  groups  of  students  when  compared  with  performance  in  another 
subject  in  the  same  year. 

These  improvements  were  observed  in  students  across  the  achievement  scale  with  students  always 
doing  better  if  they  attended  and  took  CLE.  A significant  outcome  was  a marked  reduction  in  the  failure-rate  to 
<15%  compared  with  an  average  of  25%  over  many  previous  years.  It  is  of  consequence  that  while  CLE 
enhanced  the  outcome  of  the  Physiology  students  most  likely  to  fail  (45%  ± 2 to  52%  ± 2)  the  same 
improvement  was  not  evident  in  their  Biochemistry  mark,  a closely  related  subject  (51%  ± 2 and  51%  ± 2). 
This  preliminary  information  gives  further  support  to  the  notion  that  the  CLE  has  enhanced  learning  in 
Physiology  in  this  low  achieving  group.  It  is  of  note  that  we  have  not  been  able  to  reduce  this  failure  rate  in 
previous  modifications  of  our  teaching  to  assist  the  students  most  at  risk.  Perhaps  the  more  diverse  learning 
strategies  provided  these  students  with  new  and  additional  ways  to  understand  and  communicate  physiology. 

It  was  not  possible  to  identify  the  improved  learning  with  any  particular  aspect  of  the  CLE.  We  believe 
that  it  is  the  combination  of  computer  and  human  facilitated  learning  environment  within  which  students  work 
together  and  discuss  the  subject  on  a regular  basis  that  has  contributed  to  a general  improvement  in  learning 
skills  and  attitudes 

Important  Characteristics  of  our  Collaborative  Learning  Environment: 

The  important  characteristics  that  we  believed  supported  our  successful  implementation  were: 

• The  choice  of  an  appropriate  and  flexible  set  of  delivery  tools  that  allowed  development  of  an 
appropriate  pedagogy  for  students  to  learn  effectively  from  their  tasks.  It  was  important  not  to  allow 
development  to  be  held  up  by  awaiting  the  'latest'  version  of  any  delivery  tool,  but  to  work  our  way 
around  it  to  ensure  that  one  could  deliver  an  appropriate  task  on  time,  even  if  resorting  to  a greater 
dependence  on  written  tasks.  Our  productivity  exceeded  that  in  our  prior  multimedia  developments. 

• Using  the  computer's  interactivity  to  focus  on  concepts  not  easily  covered  in  lectures,  with  the  CAL  not 
designed  to  replace  all  lectures  or  students'  use  of  textbooks  and  other  resources. 

• A generation  of  a pleasant  and  friendly  learning  space  was  cmcial  to  encouraging  students  to  openly 
discuss  issues  and  to  discover  the  power  of  peer-learning  and  peer-teaching,  with  supportive  staff 

• Choice  of  reliable  software  and  hardware  (eg  our  experience  with  iMacs,  compared  with  previous 
years,  meant  staff  could  concentrate  on  learning  issues  and  not  solving  technical  problems). 

• Students  needed  to  recognise  that  learning  in  CLE  sessions  was  as  important  as  attending  and  learning 
from  lectures.  Assessment  was  an  important  signal  that  encouraged  students  to  attend,  but  it  must  be 
make  it  so  competitive  that  students  no  longer  wish  to  collaborate.  We  found  that  allocating  5%  for 
participation  in  CLE  sessions  did  encourage  the  CAL  to  be  taken  more  seriously  than  previous  years, 
but  increasing  this  will  bring  on  more  issues  of  equity  and  accuracy  of  assessment  methods. 

• The  proximity  of  the  developers  to  the  CLE  laboratory  provided  opportunities  for  them  to  obtain  a 
deeper  understanding  to  the  cultural  and  individual  characteristics  of  the  learning  experiences. 

• Students  openly  showed  their  enjoyment  of  the  learning  environment  and  there  was  always  lively 
discussion  of  physiology  (and  not  extraneous  social  matters).  This  also  encouraged  both  the  tutors'  and 
the  developers'  interest  in  further  improving  the  collaborative  learning  experience  in  line  with  the 
students'  survey  results  and  informal  feedback.  The  use  of  scheduled  classes  was  imperative  to  allow 
students  to  establish  and  maintain  working  relationships  with  their  group. 
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Conclusions  and  Future  Directions 


We  have  found  that  there  is  significant  value  in  using  scheduled  face-to-face  on-campus  collaborative 
learning,  to  complement  student’s  individual  self-paced  asynchronous  learning,  that  is  facilitated  by  computer 
tutorials  and  resources.  Continuing  investigation  of  our  collaborative  learning  environment  has  the  potential  to 
build  on  existing  initial  successes  and  to  provide  some  credible  evidence  and  advice  to  support  the  management 
of  such  positive  educational  change.  Additional  evaluation  of  what  works  and  why  with  each  of  our  computer 
tutorials,  and  with  which  students,  will  allow  us  to  make  more  use  of  adaptive  feedback  with  new  embedded 
Web  objects  (Fritze  8c  Kemm,  2000).  Thus  we  could  provide  more  appropriate  learning  challenges  to  students 
who  have  different  approaches  to  learning  and  have  different  achievement  levels. 

While  our  results  appear  promising,  in  order  to  maximise  the  benefit  of  this  innovation  there  is  a need 
to  investigate  empirically  which  factors  provide  the  most  positive  influences.  This  could  allow  the  development 
of  a transferable  framework  for  establishing  even  more  successful  collaborative  learning  environments.  In 
particular,  we  need  to  learn  more  about  where  to  allocate  human  and  other  resources  in  order  to  have  optimal 
effect  on  student  achievement  (Thomas,  Sammons  & Mortimore  1995).  In  addition  to  existing  teaching 
processes,  this  would  entail  a carefully  planned  action  research  project  that  will  provide  some  more  precise  and 
actionable  indicators  of  key  factors.  More  specifically,  it  would  help  to  create  more  educative  links  between 
instruction,  learning  and  assessment  essential  for  the  development  of  ‘deeper’  and  more  durable  learning 
abilities  by  students  (Boud  1994,  Loacker,  Cromwell.  & O’Brien  1986).  Such  data  would  be  a valuable 
contribution  to  other  academic  staff  taking  up  the  challenge  of  introducing  more  progressive  teaching 
approaches  that  foster  greater  levels  of  self-directed  and  higher  order  learning  such  as  collaborative  problem 
solving  and  active  reflection  and  metacognitive  analysis  (Slavin  1990,  Mezirow  1994).  It  would  also  provide  a 
local  infrastructure  and  an  adaptable  generic  model  for  generating  useful  ongoing  measures  to  inform  quality 
assurance  and  continuous  improvement  processes. 

Overall,  we  will  continue  to  use  an  iterative  approach  to  the  development  of  collaborative  learning 
tasks  that  address  effective  pedagogy  in  a campus-based  computer-facilitated  learning  environment.  We  will 
use  a pragmatic  mix  of  available  tools  (such  as  commercial  products  that  support  emerging  standards  for 
delivery  of  interactive  multimedia)  and  locally  developed  solutions  that  address  our  own  needs.  In  particular, 
our  aim  is  to  provide  a more  inclusive  curriculum  with  learning  opportunities  for  the  wide  range  of  abilities  and 
learning  styles  of  our  students. 
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Abstract:  The  purpose  of  this  paper  is  to  discuss  the  applicability  and  utility  of  Fault  Tree 
Analysis  in  the  diffusion  of  a technological  innovation  in  a higher  education  setting.  Diffusion 
of  innovation  literature  suggests  that  the  involvement  of  the  intended  users  in  the  diffusion 
process  will  increase  the  subsequent  use  of  the  innovation.  The  educational  literature  has 
suggested  there  are  six  factors  which  inhibit  or  influence  the  use  of  technology  in  schools: 
time;  expertise;  access;  resources;  support  (Leggett  & Persichitte,  1998)  and  perceived  need 
(Sherwood,  1999).  By  modifying  the  Fault  Tree  Analysis  process  to  be  more  inclusive,  it  is 
theorized  that  the  process  will  be  both  applicable  and  utile  and  that  intended  users  of  the 
technology  in  question  will  be  more  likely  to  successfully  integrate  the  innovation  into  their 
educational  milieu. 


Introduction 


I believe  that  the  motion  picture  is  destined  to  revolutionize  our  educational  system  and 
that  in  a few  years  it  will  supplant  largely,  if  not  entirely,  the  use  of  textbooks. 

The  education  of  the  future,  as  I see  it,  will  be  conducted  through  the  medium  of  the 
motion  picture  ...  where  it  should  be  possible  to  obtain  one  hundred  percent  efficiency. 

Thomas  Edison,  1 922 

The  history  of  modem  education  is  littered  with  the  trash  of  technology  left  behind 
by  unrealistic  purchases,  naive  users,  and  vendor  representatives  working  on  a quota  system. 

Polley,  1977  (in  Albright  & Graf,  1992) 

...  the  very  efficiency  of  the  machine  was  drastically  curtailed  by  the  failure  to  achieve  in 
society  a set  of  harmonious  and  integrated  purposes. 

Lewis  Mumford,  1934 

I begin  with  these  quotes  in  order  to  establish  the  arc  of  thought  that  my  research  will  follow.  The  first  quote  highlights 
the  promise  of  a panacea  that  many  people  see  in  any  new  technology  destined  for  use  in  educational  settings.  Substitute 
"computer  technology",  or  any  other  previous  new  technology  touted  as  the  greatest  thing  to  hit  education,  for  "motion 
picture"  and  a statement  made  in  1 922  becomes  as  relevant,  repeated  and  potentially  unrealistic  today.  People  seem  all 
too  ready  to  accept  technology  as  something  which  is  value-free  and  necessary  (Cornell,  1999;  Kerr,  1996).  Perhaps 
even  more  disturbing,  is  the  idea  that  if  a technology  can  do  something  it  therefore  follows  that  it  should  be  done.  If 
medical  technology  was  used  with  as  little  thought  for  consequences  there  would  be  a huge  public  outcry  yet  very  little 
such  outcry  occurs  in  education.  As  Postman  writes  in  his  book  "The  End  of  Education"  (1996,  p.  41),  the  role  that  new 
technology  should  play  in  schools  or  anywhere  else  is  something  that  needs  to  be  discussed  without  the  "hyperactive 
fantasies  of  cheerleaders".  He  goes  on  to  call  computers  and  their  accompanying  technology  "Faustian  bargains".  We 
must  be  more  mindful  regarding  the  choices  and  decisions  we  make  concerning  our  use  of  technology  in  education  lest 
we  end  up  with  something  cheered  about  by  many  but  wholly  inadequate  and  inappropriate  for  the  majority  of  our  needs. 
I have  attempted  to  apply  this  mindfulness  in  my  examination  of  the  applicability  and  utility  of  Fault  Tree  Analysis. 

Like  Postman’s  "Faustian  bargains",  Polley  (in  Albright  & Graf,  1992)  also  felt  that  perhaps  far  too  much  emphasis 
was  placed  on  the  acquisition  of  technology  without  first  fully  understanding  what  those  acquisitions  would  imply. 
Policy’s  words  illustrate  that  the  unchecked  and  poorly  thought  out  use  of  information  technology  in  the  schools  has 
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resulted  in  somewhat  of  a backlash.  Yet  this  backlash  has  occurred  throughout  history  with  very  little  consequence. 
Polley's  statement  is  but  one  of  many  such  examples  of  disappointment  people  feel  in  the  reality  new  technology  often 
provides.  There  is  a large  contingent  of  technological  cautionaries  attempting  to  temper  the  frequently  unchecked 
enthusiasm  the  technological  advocates  put  forth.  Critics  (see,  for  example,  Winner,  1998,  1999;  Postman,  1988,  1996; 
Oppenheimer,  1997;  McKibben,  1992;  Mander,  1992)  of  the  heavy  and  costly  emphasis  of  placing  the  latest  information 
technologies  into  the  schools  argue  that  with  so  little  evidence  for  their  effectiveness,  this  effort  would  be  much  better 
placed  on  other  aspects  of  schooling. 

Experience  with  technology  seems  to  indicate  that  the  crux  of  the  problem  lies  between  acquisition  and  use.  It  is 
clear  that  the  diffusion  of  innovation  is  not  merely  a matter  of  purchase.  What  happens  between  the  promise  and  the 
reality?  Why  do  some  technologies  take  hold  and  others  collect  dust  in  a comer?  Why  is  it  that  some  schools,  such  as 
Appleby  Elementary  School  in  Marathon,  New  York  (Sherwood,  1999)  and  even  entire  school  districts,  such  as 
Chittenden,  Vermont,  for  example,  which  is  widely  renowned  for  its  use  of  technology  (Brauer,  1995)  are  able  to 
seamlessly  integrate  technology  into  their  curricula  while  others  can  only  manage  the  most  rudimentary  use  of 
technology?  This  situation  regarding  the  use  of  technology  in  education  does  not  only  apply  to  the  elementary  grades.  It 
seems  particularly  acute  in  the  higher  education  setting.  Dalton  (1989)  notes  that  in  spite  of  the  impact  computer 
technology  has  had  on  society  in  general,  the  impact  on  the  practice  of  education  has  been  relatively  minor.  According  to 
Spotts  and  Bowman  (1995),  the  influence  of  technology  on  higher  education  pedagogy  has  been  weak  to  almost  non- 
existent. The  results  of  a survey  measuring  instructional  technology  awareness  and  use  amongst  faculty  members 
indicated  that,  while  faculty  recognize  the  importance  of  instmctional  technology,  they  do  not  currently  use,  nor  plan  to 
use,  instructional  technology  in  their  teaching  (Spotts  & Bowman,  1995).  The  results  also  indicated  that,  while  the 
majority  of  faculty  have  a high  degree  of  knowledge  and  experience  concerning  older  technology  such  as  video  and  film, 
fewer  than  20%  of  them  actually  use  any  technology  on  a regular  basis. 

Recent  findings  are  somewhat  more  encouraging  but  use  still  seems  to  be  concentrated  on  the  communication 
aspects  of  computers  (e-mail,  presentations,  etc.)  rather  than  on  integration  with  the  curriculum  (Mitra,  Steffensmeier, 
Lenzmeier  & Massoni,  1999).  Faculty  has  long  been  resistant  to  advanced  technology-based  education  and  while,  as 
Rickard  (1999)  argues,  faculty  seem  more  willing  to  use  new  technology  and  are  even  engaging  in  the  development  of 
such  courses,  there  remains  a well-entrenched  skepticism  not  in  the  technology  necessarily  but  in  the  ability  of  the 
educational  institution  in  which  they  work  to  support  its  use.  Support  for  the  technology,  then,  seems  to  be  one  area  to 
which  attention  should  be  paid.  Fault  Tree  Analysis,  by  virtue  of  its  inclusivity  (at  least  my  variation  of  it  to  promote 
inclusivity)  and  involvement  of  members  from  the  entire  focal  system,  engenders  this  notion  of  support.  Others  believe 
that  too  much  is  being  invested,  at  too  great  a speed,  in  technology  that  has  yet  to  be  proved  to  increase  educational 
quality  and  productivity  (Larson,  in  Rickard,  1999).  Larson  also  argues  that  where  evidence  does  exist  that  technology 
"can  actually  help  improve  educational  quality  [it]  is  really  just  the  tendency  of  technology  to  make  course  material  more 
engaging"  (para.  12).  In  other  words,  paying  more  attention  to  engaging  our  students  in  the  learning  process  may,  even 
without  technology,  be  all  that  is  needed  to  make  education  more  effective.  If  one  applies  this  rationale  to  Fault  Tree 
Analysis,  which  engages  participants  in  the  process,  it  may  add  value  to  the  process  of  diffusion. 

Mumford’s  understanding,  gained  sixty-five  years  ago,  that  the  society  in  which  a technology  is  to  be  implemented 
needs  to  be  in  agreement  over  the  implementation  process  and  desired  outcomes,  may  hold  the  key  to  successful 
technological  diffusion  in  educational  settings.  The  process  of  Fault  Tree  Analysis  and  the  modifications  which  have 
been  proposed  clearly  attempts  to  achieve  this  agreement.  The  supposition  that  the  manner  in  which  technology  is 
introduced,  and  the  elements  at  play  in  the  context  in  which  this  introduction  occurs,  will,  by  virtue  of  its  systemic 
nature,  come  to  the  fore  while  conducting  the  Fault  Tree  Analysis.  Fault  Tree  Analysis  is  “a  logic  diagramming 
technique  relating  combinations  of  possible  events  (or  subsystems  within  a system)  which  interact  to  produce  a 
predefined  undesired  event  [and  can  be]  generally  thought  of  as  a technique  for  increasing  the  likelihood  of  success  in 
any  system  by  analyzing  the  most  likely  modes  of  failure,  in  order  to  prevent  future  failures”.  (Stephens,  1972,  p.  2).  The 
main  purpose  of  this  technique  is  to  improve  the  likelihood  of  success  in  any  system  by  understanding  potential  areas 
and  causes  for  failure,  with  the  goal  being  to  prevent  future  failures.  The  first  step  in  Fault  Tree  Analysis  is  system 
definition  - a definition  which  includes  a discussion  of  the  factors,  both  positive  and  negative,  which  serve  to  influence 
the  system.  I believe,  and  there  is  literature  (c.f.,  Akrich,  1993;  Burt,  1973;  Van  Den  Akker,  Keursten,  & Plomp,  1991; 
Walsh,  1991;  Walsh  & Bayma,  1996)  to  substantiate  and  support  this  belief,  that  Mumford's  words  concerning  the 
importance  of  social  context  highlight  what  is  frequently  an  absent  element  in  the  diffusion  of  new  information 
technologies  into  educational  settings  - the  direct  involvement  of  the  intended  users  of  the  innovation.  The  modifications 
I have  proposed  to  Fault  Tree  Analysis  clearly  attempts  to  address  this  void.  It  seems  apparent  that,  in  spite  of  the  critics 
and  the  lack  of  concrete  and  unconfounded  proof  of  effectiveness,  information  technologies  are  going  to  be  bought  and 
attempts  are  going  to  be  made  to  integrate  them  into  educational  settings  (an  effect  that  has  been  described  as  the 
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"bandwagon  effect”  (Abrahmson  & Rosenkopf,  1993))  - it  therefore  behooves  us  to  research  and  understand  the  most 
effective  ways  in  which  this  can  be  accomplished. 

The  above  is  not  to  say  that  technological  determinism  is  inevitable,  but  past  patterns  of  use  of  technology 
seemingly  indicate  that  educators  use  what  is  offered,  perhaps  looking  for  a solution  to  the  "problem"  of  education. 
Given  this  initial  willingness  to  use  technology  (though  history  also  indicates  that  continued  use  rarely  occurs),  finding 
ways  to  help  ensure  responsible  and  productive  use  is  crucial.  An  understanding  of  why  one  wants  to  use  a particular 
innovation,  how  and  even  if  it  should  be  used  and  whether,  given  ideal  circumstances,  it  will  succeed  when  used  is 
essential.  As  others  have  indicated,  I too  believe  that  there  is  far  too  much  emphasis  on  technology  for  technology's  sake 
without  nearly  enough  emphasis  on  the  surroundings  in  which  the  technology  will  be  utilized  or  even  if  it  should  be 
utilized  at  all.  Technology  may  not  be  entirely  pernicious  but  it  is  also  most  certainly  not  a panacea.  In  any  case,  it  is 
rarely  innocuous.  Some  circumstances  warrant  the  use  of  technology  but  some  situations  would  be  better  off  without 
technological  intervention.  Fault  Tree  Analysis  and  the  subsequent  validation  of  this  analysis  by  the  members  of  the 
focal  system  appears  to  be  an  excellent  tool  by  which  we  can  gain  insight  into  these  very  crucial  elements,  thereby 
significantly  adding  to  our  understanding  of  how  best  to  implement  technology  into  educational  settings. 

The  first  teacher  ever,  the  priest  in  preliterate  Mesopotamia  who  sat  down  outside  the  temple 
with  the  kids  and  began  to  draw  figures  with  a twig  in  the  sand,  would  be  perfectly  at  home  in 
most  classrooms  in  the  world  today.  Of  course,  there  is  a blackboard,  but  otherwise  there  has 
been  little  change  in  tools  and  none  in  respect  to  methods.  The  one  new  teaching  tool  in  the 
intervening  8,000  years  has  been  the  printed  book.  And  that  few  teachers  really  know  how  to 
use— or  else  they  would  not  continue  to  lecture  on  what  is  already  in  the  book. 

The  priest  in  ancient  Mesopotamia  was  also  the  first  doctor.  If  he  returned  today  to  a modem 
operating  room  in  the  hospital,  he  would  not  conclude  that  he  could  do  as  well.  Yet  today's 
doctors  are  no  better  men  than  the  first  doctors  were.  They  certainly  are  no  better  than  the 
"father  of  medicine,"  Hippocrates.  They  stand  on  his  shoulders.  They  know  more,  and  above 
all,  they  know  better.  They  have  a different  methodology.  They  have  different  tools.  As  a 
result,  they  do  entirely  different  things,  and  do  them  differently. 

Drucker,  1969,  p.  347 

Whether  or  not  one  fully  agrees  with  Drucker,  the  contrast  between  medicine  and  education  is  striking  and  it  is  also 
obvious  to  any  educator  or  student  that  there  is  truth  to  his  idea  that  very  little  has  changed  in  today's  classrooms.  One 
need  only  sit  in  on  a majority  of  university  classes  to  see  that  generally,  as  was  the  case  hundreds  of  years  ago,  the 
teacher  talks  while  the  pupils  listen.  That  is  not  to  say,  of  course,  that  new  technologies  have  not  been  developed  and 
attempts  at  use  have  not  been  made.  In  fact,  introducing,  implementing  and  ultimately  integrating  technology  into 
classrooms,  as  well  as  businesses,  government  facilities,  etc.,  is  not  a new  phenomenon.  From  slate  to  radio  to  the 
overhead  projector  to  the  multi-media  computer  platform,  a variety  of  technologies  have  been  introduced  into  various 
environments  with  varying  degrees  of  success  and  failure  (Cuban,  1986;  1993;  Pellegrino  & Altman,  1997;  Cornell, 
1999;  Rickard,  1999).  It  is  not  sufficient  to  have  functional  equipment  - the  equipment  must  be  accepted  and  integrated 
in  order  for  it  to  be  used.  The  best  of  intentions  are  one  thing  - actual  use  another.  Recently,  Drucker  (1999)  reiterated 
the  same  conclusions  he  made  in  1969  regarding  the  use  of  technology  in  education,  though  on  this  occasion  he  turns 
primarily  to  the  influence  the  information  revolution  has  had  on  the  business  community.  According  to  Drucker,  while 
routinization  of  traditional  processes  has  occurred,  “almost  none  of  the  effects  of  information  envisaged  forty  years  ago 
have  actually  happened  (1999,  par.  16).  He  argues,  though,  that  the  psychological  impact  of  technology  has  been  far 
greater,  especially  on  the  manner  in  which  children  learn.  Drucker  maintains,  as  he  did  in  1969,  that  the  manner  in  which 
schools  teach  is  inconsistent  with  the  way  children  learn.  In  other  words,  schools  are  still  behind  where  they  should  be. 

Five  critical  factors,  culled  from  decades  of  literature,  inhibiting  the  use  of  technology  in  the  schools  have  been 
identified  by  Leggett  and  Persichitte  (1998).  These  factors  are:  time;  expertise;  access;  resources  and  support.  Perceived 
need  has  also  been  identified  as  an  essential  factor  when  it  comes  to  the  successful  integration  of  technology  (Sherwood, 
1999).  Technology  programmes  therefore  need  to  be  developed  which  are  systemic  and  intentional  or  purposive 
(Coughlin,  1999).  Based  on  the  findings  in  the  literature  regarding  examples  of  the  failure  of  diffusion  of  information 
technology  in  different  settings,  and  using  the  above  six  characteristics  as  a means  of  categorizing  the  findings,  it  may  be 
possible  to  develop  a model  of  successful  diffusion  of  innovation  easily  applicable  and  suited  to  the  needs  of  educational 
institutions  implementing  a new  technology.  It  will  be  interesting  to  see  if  the  Fault  Tree  Analysis  uncovers  these  factors 
as  those  contributing  to  the  failure  (or  potential  failure)  of  the  innovation  in  question. 
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Large  sums  of  money  and  huge  investments  of  time  are  being  expended  on  technology  in  the  schools  with, 
seemingly,  very  little  success  (Coughlin,  1999;  Sherwood,  1999).  In  light  of  the  technological  failures  in  the  past  and  the 
continued  amount  of  money  that  is  being  spent  on  technology,  Fault  Tree  Analysis  may  prove  to  be  a viable  and 
practicable  method  schools  can  use  when  dealing  with  the  influx  of  a new  technological  innovation,  thereby  saving  huge 
amounts  of  time,  money  and  frustration.  While  practitioners  and  researchers  alike  currently  agree  that  there  are  problems 
with  the  use  of  technology  in  the  schools  and  efforts  have  been  made  to  look  into  these  problems,  there  has  been  no 
systemic  and  systematic  study  of  failure  of  the  sort  Fault  Tree  Analysis  offers.  Education  is  clearly  a success-oriented 
endeavour.  Indeed,  it  has  oft  been  said  that  if  one  focuses  on  failure,  one  probably  will.  Fault  Tree  Analysis  may  focus 
on  failure  but  it  does  so  with  the  goal  of  success  in  mind.  Furthermore,  the  very  process  of  Fault  Tree  Analysis  improves 
the  individual  sense  of  ownership  and  control  over  the  technological  diffusion,  loss  of  which,  as  previously  discussed, 
teachers  fear.  The  process,  which  includes  direct  involvement  and  system-wide  representation,  also  serves  to  build  a 
cohesive  team  which  improves  and  strengthens  the  social  environment  into  which  the  technology  is  being  diffused 
(Jonassen,  Tessmer  & Hannum,  1999;  Stephens,  1972;  Wood,  Stephens  & Barker,  1979)  - factors  which  have  been 
previously  identified  as  essential  in  the  diffusion  of  innovation.  The  current  study  clearly  involves  the  individuals  to 
whom  the  innovation  is  being  targeted.  Fault  Tree  Analysis  appears  to  be  the  ideal  method  to  address  some  of  the 
problems  of  technological  diffusion  in  educational  settings.  I believe  that  this  technique  will  prove  to  be  a useful  tool  in 
helping  schools  face  the  daunting  challenge  of  integrating  technology  into  their  educational  environments. 


Fault  Tree  Analysis  Site 

The  site  chosen  for  this  study  is  a college  in  Montreal.  The  college  is  a multicultural  institution  and  reflects  the 
international  and  cosmopolitan  nature  of  the  city  in  which  it  is  situated.  In  the  last  five  years  courses  have  been  offered 
on  how  to  use  the  internal,  e-mail,  web  page  design,  FirstClass,  etc.  Most  have  met  with  an  enthusiastic  response  but 
little  follow  through.  Loss  of  interest  seems  to  be  a particularly  common  problem.  The  technological  innovation 
currently  challenging  the  administrators  is  the  use  of  the  Internet  as  an  integral  element  of  the  teaching  process.  Their 
goal  is  to  increase  the  use  of  the  Internet  in  the  current  classroom  model  as  well  as  develop  on-line  courses  with  an  eye 
towards  increasing  the  schools  potential  for  distance  education  courses.  Fault  Tree  Analysis  seems  to  be  a practical  tool 
that  may  help  the  college  face  their  goal  of  technological  integration.  Testing  its  applicability  and  utility  in  this 
environment  will  provide  a rich  basis  upon  which  applicability  can  be  assessed. 

Using  Fault  Tree  Analysis,  I will  attempt  to  gain  an  understanding  of  the  college's  outlook  on  the  diffusion  of 
technology  into  their  system.  I believe  that  this  technique  will  prove  to  be  a useful  tool  in  helping  schools  face  the 
daunting  challenge  of  integrating  technology  into  their  educational  environments.  Examining  and  understanding  how 
technological  innovations  fail  will,  according  to  advocates  of  Fault  Tree  Analysis,  increase  future  probability  of  success. 
Participating  in  the  fault  tree  process,  will  provide  faculty  members  at  the  college  with  a unique  opportunity  to  have  a 
voice  in  the  manner  in  which  a new  technology  is  introduced  in  their  environment.  They  will  be  given  an  opportunity  to 
have  a direct  say  in  how,  and  even  whether,  a new  technology  will  be  implemented.  They  will  be  involved  in  a direct  and 
concrete  manner.  They  will  not  only  be  asked  their  opinion  but  will  be  asked  to  shape  the  final  result  as  well.  The 
participants  (the  majority  of  whom  are  faculty  members)  will  also  see  that  administrators  are  being  willing  and 
cooperative  partners  in  the  decision-making  process  of  technological  implementation,  Lawler,  Rossett  and  Hoffman 
(1998)  have  clearly  indicated  the  importance  of  a supportive  environment  in  the  integration  of  technology  in  an 
educational  environment. 


Significance  of  the  Research 

The  potential  significance  of  this  research  is  fourfold.  Firstly,  this  is  the  first  study  that  will  examine  in  a rigorous 
fashion  the  applicability  and  utility  of  Fault  Tree  Analysis.  It  is  the  first  study  to  apply  this  method  specifically  to  the 
diffusion  of  a technological  innovation  in  an  educational  setting.  In  previous  documented  uses  of  Fault  Tree  Analysis, 
the  purpose  was  to  analyze  a problem  using  Fault  Tree  Analysis,  not  to  study  Fault  Tree  Analysis  itself  as  a tool.  In 
addition  to  being  the  first  “process”  study  of  Fault  Tree  Analysis,  this  study  also  investigates  an  innovation  in  the 
process  for  the  first  time  - specifically,  it  addresses  the  utility  of  contributions  from  the  focal  system,  beyond  the 
appointed  FTA  team,  and  the  match  or  mismatch  between  the  contributions  of  the  two  groups. 
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Secondly,  the  research  will  also  have  practical  significance  to  the  technology  planners,  implementers  and  faculty  at 
the  college.  The  results  of  this  study  will  be  directly  applicable  to  a reality-based  situation.  The  members  of  the  focal 
system  will  have  developed  a plan  of  how  best  to  proceed  with  the  diffusion  of  the  innovation  in  question  - a plan 
shaped  by  the  direct  input  and  validation  of  the  intended  users  of  the  technology  - elements  which  the  diffusion  literature 
indicates  as  crucial  to  the  process  of  implementation. 

Thirdly,  by  providing  a detailed  description  of  the  Fault  Tree  Analysis  process  (as  experienced  by  the  members  of 
the  focal  system  and  experienced  and  observed  by  the  researcher),  this  study  will  attempt  to  fill  a gap  in  the  literature 
concerning  the  diffusion  of  technology  into  educational  settings.  The  research  clearly  indicates  that  the  best  way  to 
proceed  to  ensure  use  when  implementing  technology  is  to  directly  involve  the  intended  users  in  the  decision-making 
and  implementation  stages  of  the  plan.  Stephens’  original  model  of  Fault  Tree  Analysis,  while  including  members  of  the 
focal  system  on  the  FTA  team,  does  not  allow  for  the  direct  contribution  of  the  members  of  the  entire  system  - the 
modification  proposed  herein  clearly  does.  This  study  will  be  examining  whether  what  has  been  identified  as  important 
in  theory,  works  in  practice. 

Finally,  based  on  the  results  of  this  study,  a model  of  Fault  Tree  Analysis  in  the  process  of  diffusion  of  innovation  in 
a higher  education  setting  will  be  proposed  which  can  then  serve  to  inform  the  diffusion  of  innovation  literature  and  have 
direct  utility  in  other  educational  systems. 


Conclusion 

The  purpose  of  this  research  is  to  determine  the  applicability  and  utility  of  Fault  Tree  Analysis  in  the  diffusion  of 
technological  innovations  in  educational  settings.  Once  the  applicability  and  utility  is  established  it  is  believed  that  the 
development  of  a fault  tree  may,  by  feeding  into  the  systemic  process  of  diffusion  of  innovation  and  as  feedback  and 
feedforward  loops,  help  shape  the  way  technology  is  integrated,  implemented  and  perhaps  even  developed  in  the  future. 
Developing  a fault  tree  of  a school  currently  dealing  with  the  influx  of  technology  may  help  to  provide  insight  into  how 
and  why  people  adapt  to  and  adopt  a new  technology.  It  may  also  serve  to  highlight  the  roadblocks  people  face  and 
whether  or  not  these  impediments  are  different  than  those  identified  in  the  diffusion  of  innovation  literature.  Insight 
gained  from  such  an  analysis  may,  in  the  medium  and  long  term,  allow  for  more  effective  implementation,  training  and 
use  expectation  policies.  Underlying  my  belief  in  the  importance  of  Fault  Tree  Analysis  is  that  by  merely  illuminating  to 
the  participants  the  reasons  for  failure  and  the  areas  in  which  they  are  most  likely  to  occur  is  sufficient  to  cause  a shift  in 
behaviour  and  attitude. 

The  important  contribution  this  research  in  general,  and  in  specific  relation  to  the  educational  milieu,  is  clear.  If  we 
are  to  discern  the  impact  technology  has  had  and  will  have  in  higher  education  processes,  understanding  the  elements 
that  influence  that  impact  and  providing  a systemic  approach  to  address  the  needs  of  the  technology’s  intended  users  will 
have  strong  sociological,  economical,  educational  and  policy-making  benefits.  If  we  know  how  to  uncover,  and  then 
correct,  the  failure  events  that  influence  the  acceptance  and  integration  of  technology,  savings  of  time,  effort  and  money 
can  be  had  by  overcoming  the  impediments  to  the  diffusion  of  technology  in  educational  settings  before  they  have  a 
chance  to  occur. 

Diffusion  of  innovation  occurs  for  a multitude  of  reasons  and  within  a variety  of  contexts.  Understanding  loci  of 
failure  and  the  contexts  in  which  they  occur  will  help  allow  for  a much  more  efficient  process  of  technological 
integration.  Understanding  these  failure  elements  may  also  help  us  recognize  when  technology  is  not  warranted  and  feel 
confident  in  the  decision  not  to  use  it.  This  Fault  Tree  Analysis  will  serve  to  help  gain  insight  into  the  implementation 
process  of  technological  innovation.  This  insight  and  the  subsequent  development  of  a systemic  diffusion  of  innovation 
model  using  Fault  Tree  Analysis  may  help  to  inform  policy  and  decision  makers  in  their  implementation  strategies  - 
necessary  and  desirable  goals  in  light  of  the  speed  at  which  new  technologies  are  being  developed  and  introduced  to 
schools  in  North  America  and  the  frequent  failure  these  implementations  meet. 

References 

Abrahamson,  E.,  & Rosenkoph,  L.  (1993).  Institutional  and  competitive  bandwagons:  Using  mathematical  modeling  as  a tool  to 
explore  innovation  diffusion.  Academy  of  Management  Review,  /5(3),  487-517. 

Akrich,  M.  (1993).  A gagozene  in  Costa  Rica:  An  experiment  in  technosociology.  In  P.  Lemonnier  (Ed.),  Technological  choices: 
Transformation  in  material  cultures  since  the  Neolithic,  (pp.  289-337).  London:  Routledge. 

Albright,  M.  J.,  & Graf,  D.  L.  (Eds.).  (1992b).  Teaching  in  the  information  age:  The  role  of  educational  technology.  San  Francisco: 
Jossey-Bass. 

Brauer,  R.  (1995).  The  internet  as  school,  or  welcome  to  our  MUD  room.  Technos,  4(3),  14-21 . 


65 


Burt,  R.  S.  (1973).  The  differential  impact  of  social  integration  on  participation  in  the  diffusion  of  innovations.  Social  Science 
Research,  2,  125-144. 

Cornell,  R.  (1999).  Paradigms  for  the  new  millennium:  How  professors  will  certainly  change!  Educational  Media  International, 
36{2\  89-96. 

Coughlin,  E.  (1999).  Professional  competencies.  Learning  and  Leading  with  Technology,  2 7{3),  22-21. 

Cuban,  L.  (1986).  Teachers  and  machines:  The  classroom  use  of  technology  since  1920.  New  York:  Teachers  College  Press. 

Cuban,  L.  (1993).  How  teachers  taught:  Constancy  and  change  in  American  classrooms,  1890-1990  (2nd  ed.).  New  York:  Teachers 
College  Press. 

Dalton,  D.  W.  (1989).  Computers  in  the  schools:  A diffusion/adoption  perspective.  Educational  Technology,  29(1 1),  20-27. 

Davis,  F.  D.  (1993).  User  acceptance  of  information  technology:  Systems  characteristics,  user  perceptions  and  behavioral  impacts. 
International  Journal  of  Man-Machine  Studies,  38(3),  475-487. 

Drucker,  P.  F.  (1969).  The  age  of  discontinuity:  Guidelines  to  our  changing  society.  New  York:  Harper  Row. 

Drucker,  P.  F.  (1999).  Beyond  the  information  revolution.  [48  paragraphs].  The  Atlantic  Monthly  [On-line  serial],  254(4). 
Available:  http://www.theatlantic.com/issues/99oct/9910dmcker.htm 

Gould,  J.  D.,  Boies,  S.  J.,  & Lewis,  C.  (1991).  Making  usable,  useful,  productivity-enhancing  computer  applications. 
Communications  of  the  ACM,  34(1),  74-85. 

Jonassen,  D.  H.,  Tessmer,  M.,  & Hannum,  W.  H.  (1999).  Task  analysis  for  instructional  design.  Mahwah,  NJ:  Lawrence  Erlbaum 
Associates,  Publishers. 

Kerr,  S.  T.  (1996).  Visions  of  sugarplums:  Technology,  education,  and  schools.  In  S.  T.  Kerr  (Ed.),  Technology  and  the  future  of 
schooling:  Ninety-fifth  yearbook  of  the  National  Society  for  the  Study  of  Education.  Part  If  (pp.  1 27).  Chicago:  NSSE. 

Lawler,  C.,  Rossett,  A.,  & Hoffman,  R.  (1998).  Using  supportive  planning  software  to  help  teachers  integrate  technology  into 
teaching.  Educational  Technology,  55(5),  29-34. 

Leggett,  W.  P.,  & Persichitte,  K.  A.  (1998).  Blood,  sweat,  and  TEARS:  50  years  of  technology  implementation  obstacles. 

Techtrends,  45(3),  33-39. 

McKibben,  B.  (1992).  The  age  of  missing  information.  New  York:  Random  House. 

Mander,  J.  (1992).  In  the  absence  of  the  sacred:  The  failure  of  technology  and  the  survival  of  the  Indian  nations.  San  Francisco: 
Sierra  Club  Books. 

Mitra,  A.,  Steffensmeier,  T.,  Lenzmeier,  S.,  and  Massoni,  A.  (1999).  Changes  in  attitude  toward  computers  and  use  of  computers  by 
university  faculty.  .Journal  of  Research  on  Computing  in  Education,  32,  (1). 

Mumford,  L.  (1934).  Technics  and  civilization.  New  York:  Harcourt,  Brace  and  Company. 

Nickerson,  R.  S.  (1981).  Why  interactive  computer  systems  are  sometimes  not  used  by  people  who  might  benefit  from  them. 
International  Journal  of  Man-Machine  Studies,  /5(4),  469-483. 

Oppenheimer,  T.  (1997,  July).  The  computer  delusion.  The  Atlantic  Monthly,  280  (1),  45-62. 

Pellegrino,  J.  W.,  & Altman,  J.  E.  (1997).  Information  technology  and  teacher  preparation:  Some  critical  issues  and  illustrative 
solutions.  Peabody  Journal  of  Education,  72(\  x),  89-121. 

Postman,  N.  (1988).  Conscientious  objections.  New  York:  Elisabeth  Sifton  Books. 

Postman,  N.  (1996).  The  end  of  education:  Redefining  the  value  of  school.  New  York:  Knopf 

Rickard,  W.  (1999).  Technology,  education,  and  the  changing  nature  of  resistance.  [22  paragraphs].  Educom  Review  [On-line  serial], 

54(1).  Available:  http://www.educause.edu/ir/librarv/litml/emi99 1 5.html 

Rogers,  E.  M.  (1995).  Diffusion  of  innovations  (4th  ed.).  New  York:  The  Free  Press. 

Sherwood,  S.  (1999).  1 hear,  and  1 forget.  1 see,  and  I remember.  1 do,  and  1 understand.  Learning  & Leading  with  Technology, 
27{3\  28-31. 

Spitzer,  D.  (1980).  Try  fault  tree  analysis,  a step-by-step  way  to  improve  organization  development.  Training,  17(2),  58,  64,  68-69. 
Sports,  T.  H.,  & Bowman,  M.  A.  (March-April,  1995).  Faculty  use  of  instructional  technologies  in  higher  education.  Educational 
Technology,  55(2),  56-64. 

Stephens,  K.  G.  (1972).  A fault  tree  approach  to  analysis  of  educational  systems  as  demonstrated  in  vocational  education. 
Unpublished  doctoral  dissertation.  University  of  Washington. 

Van  Den  Akker,  J.,  Keursen,  P.,  & Plomp,  Tj.  (1991).  The  integration  of  computer  use  in  education.  International  Journal  of 
Educational  Research,  17,  65-76. 

Walsh,  J.  P.  (1993).  The  social  context  of  technological  change:  The  case  of  the  retail  food  industry.  The  Sociological  Quarterly, 
52(3),  447-468. 

Walsh,  J.  P.,  & Bayma,  T.  (1996).  Computer  networks  and  scientific  work.  Social  Studies  of  Science,  26(3),  661-703. 

Winner,  L.  (1998).  Technology  as  "big  magic"  [Special  feature:  T & S interview  with  Langdon  Winner].  IEEE  Technology  and 
Society  Magazine,  77(3),  4-16. 

Winner,  L.  (1999).  Building  the  basics  [Special  feature:  Interview  with  Langdon  Winner].  January/February  1999,  [44  paragraphs]. 
Educom  Review  [On-line  serial],  M(0*  Available:  http://www.educause.edu/ir/librarv/litml/erni99 1 0.html 

Wood,  R.  K.,  Stephens,  K.  G.,  & Barker,  B.  O.  (1979).  Fault  Tree  Analysis:  An  emerging  methodology  for  instructional  science. 
Instructional  Science,  8,  1-22. 

Zemke,  R.,  & Kremlinger,  T.  (1982).  Figuring  things  out:  A trainer's  guide  to  needs  and  task  analysis.  Reading,  MA:  Addison- 
Wesley  Publishing. 


BEST  COPY  AVAILABLE 

o 

ERIC 


66 


Changing  Interaction  Paradigms  in  Annotation  Environments 


Christian  M.  Adriano,  Alberto  B.  Raposo,  Ivan  L.  M.  Ricarte,  Leo  P.  Magalhaes 
Department  of  Computer  Engineering  and  Industrial  Automation  (DCA) 
School  of  Electrical  and  Computer  Engineering  (FEEC) 

State  University  of  Campinas  (Unicamp)  - Brazil 
{medeiros,  alberto,  ricarte,  leopini}@dca.fee. unicamp.br 


Abstract:  Implementation  of  novel  learning  scenarios  frequently  implies  the  adoption  of  new 
interaction  paradigms  provided  by  new  media.  Changes  on  media  bring  several  issues  to 
educational  metaphors,  such  as  deciding  which  characteristics  should  be  maintained,  removed 
and  improved.  This  paper  discusses  these  issues  by  means  of  a case  study  comparing  evolving 
scenarios  for  the  annotation  metaphor:  from  paper-based  to  hypertext  and  virtual  reality. 


Introduction 

The  design  of  innovative  educational  environments  involves  providing  novel  learning  scenarios,  which 
most  of  the  time  requires  the  support  of  new  interaction  paradigms.  We  start  from  the  aspect  stated  by  the 
following  hypothesis:  changes  on  the  medium  enable  the  introduction  of  new  interaction  paradigms  and  further 
opportunities  to  improve  educational  metaphors. 

By  using  "metaphor"  we  mean  the  transposition  of  the  meaning  of  one  thing  to  a different  one.  For 
instance,  the  "text  metaphor"  supposes  a place  with  visual  information  (words,  graphs,  etc.).  "Paradigm"  is 
understood  as  a model  for  relationships  among  metaphors  and  rules  responsible  to  turn  them  into  concrete 
entities.  In  other  words,  a metaphor  implies  a model  (paradigm)  and  a paradigm  is  not  easily  grasped  without 
metaphors  (Winograd  98).  Metaphors  and  paradigms  exist  only  over  a medium  that  provides  perceivable 
features.  In  parallel,  media  inexorably  affect  the  corresponding  interaction  paradigm,  because  each  medium  is 
identified  by  a proper  set  of  rules,  which  defines  possible  paradigms.  In  the  case  of  hypertext  paradigm  the 
decisive  importance  of  media  in  defining  the  corresponding  paradigms  is  shown.  This  paradigm  was  described 
by  Vannevar  Bush  (Bush  45)  to  be  implemented  on  microfilm  medium,  but  it  really  succeeded  when  a more 
adequate  medium  was  adopted  (the  digital  text). 

Therefore,  the  adoption  of  new  media  requires  a careful  study  of  how  they  affect  the  current  paradigm. 
A second  problem  is  then  stated:  how  to  deal  with  metaphors  in  consequence  of  changes  on  paradigms  and 
media?  Which  characteristics  should  be  maintained,  eliminated  and  improved  in  the  new  implementation  of  the 
metaphor? 

The  first  difficulty  is  the  different  features  of  source  and  target  media,  causing  trouble  to  see  beyond 
well-known  metaphors  and  use  all  the  functionality  of  the  target  medium  (Halasz  & Moran  82).  For  example, 
considering  a word  processor  as  a typewriter  would  not  make  it  clear  that  there  could  be  facilities  such  as 
"copy"  and  "paste".  This  problem  is  particularly  serious  in  the  shift  of  computer  applications  towards  a 3D 
paradigm,  because  we  tend  to  use  2D  desktop-like  metaphors  of  current  systems  to  guide  the  development  of  3D 
interactive  environments.  In  order  to  decide  on  features  that  must  be  supported  by  new  metaphors,  a careful 
study  of  how  media  affect  new  interaction  paradigms  is  necessary. 

The  next  section  presents  a discussion  about  different  interaction  paradigms  within  daily  practices. 
These  ideas  are  then  analyzed  by  means  of  a case  study  on  different  scenarios  for  the  annotation  metaphor.  This 
metaphor  usually  appears  in  text-based  educational  environments  (Adriano  et  al.  99,  Davis  & Huttenlocher  98, 
Harger  96),  but  it  lacks  appreciation  for  some  new  media,  like  3D  virtual  environments. 
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Interaction  Paradigm  Ruptures 


The  rupture  of  interaction  paradigms  has  been  a recurrent  fact  throughout  the  history  of  many  artifacts 
for  knowledge  transmission.  A good  example  was  the  advent  of  the  medieval  codex  (sewn  paper  pages,  not 
much  different  from  the  current  books),  substituting  the  parchment  of  antiquity.  Among  the  consequences  of 
this  change,  there  was  the  possibility  for  readers  to  work  simultaneously  with  several  books  and  make 
annotations  on  them.  Such  facility  was  not  feasible  with  the  parchment  because  readers  had  to  use  both  hands 
to  keep  it  opened  (Chartier  97).  Even  though  soundly  benefic,  some  historical  changes  were  not  immediate  and 
a lot  of  years  are  needed  to  overcome  the  natural  resistance  to  the  new  interaction  paradigm.  Thus,  in  order  to 
minimize  resistance  and  provide  adequate  paradigms  for  new  media,  it  is  necessary  to  understand  how  current 
metaphors  are  affected. 

In  search  for  paradigm  ruptures  within  daily  practices,  Michel  de  Certeau,  in  an  anthropological  study 
(Certeau  90),  identified  the  following  ruptures  in  modes  of  describing  places:  place  vs,  space,  map  vs,  trail, 
frontier  v.y.  bridge.  These  ruptures  obey  the  following  patterns:  from  personal  to  unpersonal,  from  biased  to 
unbiased,  from  transient  to  permanent  and  from  static  to  dynamic.  These  patterns  distinguish  different  uses  for 
the  narrative  metaphor.  The  word  metaphor  (from  the  greek  metaphorai)  denote  "public  means  of 
transportation",  which  is  the  sense  applied  to  the  narrative  and  annotation  metaphors.  The  former  conveys  the 
speaker  to  the  described  places,  while  the  latter  helps  the  reader  move  among  cores  of  ideas  in  a text. 

According  to  Certeau,  a place  is  a referential,  static,  and  permanent  organization  of  things.  On  the 
other  hand,  a space  is  perceived  when  vectors  of  direction,  velocity  and  time  are  used.  The  "city  place'\  for 
example,  is  composed  of  streets,  parks  and  buildings,  while  the  "city  space'"  involves  the  circulation  of  people 
and  a sum  of  every  citizen’s  instantaneous  impressions.  Similarly,  the  "text  place"  is  the  stylistic  design  of 
paragraphs,  phrases  and  figures.  The  "text  space"  is  the  interaction  with  the  written  word,  like  annotating, 
discussing  and  summarizing.  A similar  analysis,  but  with  interchanged  terminology,  was  made  by  (Harrison  & 
Dourish  96),  According  to  them,  a place  is  a space  with  some  additional  features,  like  social  meaning  and 
cultural  knowledge  about  individual  roles. 

The  distinction  between  trail  and  map  stems  from  narratives  of  space  and  place.  Trail  was  the 
prevailing  paradigm  in  medieval  descriptions  of  geographic  extensions,  which  involved  cities  where  to  visit, 
stop,  rest  and  pray.  Nowadays,  the  map  paradigm  prevails,  describing  regions  by  global  landmarks  like 
meridians,  names  of  cities,  roads,  etc.  Annotations  on  paper  compose  a total  and  reduced  vision  of  a 
commented  text,  which  corresponds  to  the  map  paradigm.  The  trail  paradigm  imposes  the  traverse  of  links  in 
order  to  obtain  an  annotation. 

Narratives  of  region  demarcations  inhabit  the  paradigms  frontier  and  bridge,  A region  is  a meeting 
of  actions  and  there  are  as  many  regions  as  the  number  of  possible  inter-actions.  The  frontier  delimits  the  static 
division  among  parts,  like  geographic  references  (rivers  and  mountains).  The  bridge  separates  by  the 
opportunity  of  interacting  or  crossing,  like  the  narrative  of  Columbus'  navigation  to  the  Antilles.  As  with  the 
case  of  hyperlinks  in  a hypertext,  an  annotation-place  is  a bridge  between  two  regions,  the  annotation  and  the 
text.  On  the  other  hand,  annotations  on  paper  are  ruled  by  the  frontier  paradigm. 

The  ideas  of  Certeau  seem  very  appropriate  to  guide  our  case  study  about  changing  interaction 
paradigms  for  the  annotation  metaphor,  which  is  the  topic  of  the  following  section. 


Case  Study:  Annotation  Metaphor 

Annotations  are  contents  appended  to  a text;  they  have  well-defined  boundaries  and  are  resultant  of 
some  cognitive  effort.  Removing  an  annotation  inflicts  no  damage  to  the  corresponding  text.  Annotation-places 
are  responsible  to  structure  groupings  of  annotations,  provide  contexts  and  define  interaction  paradigms. 
Annotation-places  are  also  superstructures  holding  relevant  metadata  about  annotations. 

The  learning  scenarios  for  this  case  study  were  built  in  CALM  (Computer  Aided  Learning  Material) 
(Adriano  et  al.  99),  a recommender-like  system  developed  at  the  University  of  Campinas. 
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Text-Based  Annotations 


A learning  scenario  in  CALM  starts  with  a student  selecting  learning  goals.  A learning  goal  is  a set  of 
subjects  of  interest,  which  are  mapped  to  topics  recommended  by  the  system.  The  student  sets  a specific  goal  by 
choosing  one  topic  to  accomplish.  The  learning  goal  is  accomplished  by  doing  delivered  study  units.  A study 
unit  is  composed  of  a topic,  a list  of  related  topics,  some  tests  and  exercises,  which  are  defined  according  to  the 
student's  learning  goal  and  a history  of  previous  interactions. 

Annotations  are  made  during  the  study  unit.  In  the  context  of  a study  unit,  three  interactive  learning 
scenarios  based  on  annotations  were  investigated:  commenting,  authoring  and  discussing.  The  first  one 
corresponds  to  annotating  to  raise  a doubt  or  include  a link  to  another  point  in  the  material.  In  this  scenario,  the 
underlying  text  being  annotated  is  self-contained  with  respect  to  the  specific  objective  it  intends  to  accomplish. 
Therefore,  the  original  text  is  not  modified.  Two  annotation-places  support  this  scenario:  bridge  and  in-line 
(Fig.  1 ),  a and  c,  respectively. 
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Figure  1;  Three  annotation  scenarios  in  CALM. 

Annotations  on  paper  use  the  frontier  paradigm  that  distinguishes  annotations  by  a colored  pen  or  a 
handwriting  style.  The  bridge  paradigm  cannot  be  established  over  paper  medium.  The  frontier  paradigm  is 
adopted  in  the  digital  text  by  means  of  the  in-line  annotation-place  (Fig.  1),  c.  Shifts  between  these  paradigms 
depend  on  the  necessity  to  reduce  the  stylistic  pollution  (excess  of  annotations)  or  to  undermine  the  difficulty  of 
navigation  (excess  of  links).  This  scenario  demonstrates  the  necessity  to  modify  features  of  a metaphor 
according  to  media  changes. 

The  second  scenario  is  the  collaborative  authoring  by  means  of  in-line  annotations.  This  scenario 
requires  a total  vision  of  the  work  (the  vision  of  author).  Insofar,  it  is  paramount  not  to  miss  the  distinction 
between  what  is  annotation  and  what  is  text.  The  association  of  two  paradigms,  yrowrier  and  map,  obtains  the 
desired  vision.  These  paradigms  support  an  appropriate  annotation-place  that  not  only  provides  version  and 
date  information,  but  also  displays  the  annotation  contents  within  the  commented  text.  The  frontier  paradigm 
displays  the  annotation  distinguished  from  the  text  contents  and  the  map  paradigm  organizes  the  annotations 
under  an  authoring  vision.  Since  text  on  paper  already  supports  both  paradigms,  this  scenario  demonstrates  the 
necessity  to  maintain  features  of  a metaphor  when  changing  the  medium. 
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The  third  scenario  comprehends  the  discussion  by  means  of  annotations  (Fig.  1),  b.  A newsgroup-like 
annotation-place  was  implemented  to  organize  the  annotations  hierarchically.  The  text  is  the  motivation  of 
discussion  and  it  must  not  be  modified.  Therefore,  this  new  accomplishment  of  the  annotation  metaphor  could 
not  be  done  over  the  hypertext  paradigm  that  would  mean  inserting  hyperlinks  in  the  text.  The  annotation-place 
for  discussion  permits  the  insertion  of  annotations  without  modifying  the  text.  This  annotation-place  shares  the 
paradigm  of  trail  with  the  hypertext,  but  they  diverge  with  respect  to  the  paradigms  of  place  and  space,  A 
^^space  of  discussion*’  is  necessarily  dynamic,  while  the  non-modifiable  hypertext  defines  only  a place  (static). 
This  is  an  example  of  an  innovation  in  consequence  of  changing  the  medium,  since  the  paper  medium  cannot 
support  such  scenario. 


Virtual  Reality  Annotations 

A motivation  for  the  shift  towards  a 3D  paradigm  would  be  the  need  to  introduce  more  appropriate 
and  novel  interaction  among  metaphors  in  a learning  situation.  For  learning  situations  in  which  interaction  is 
greatly  based  on  real  objects,  a text  based-scenario  becomes  too  cumbersome,  due  to  highly  descriptive 
narratives  of  positions,  appearance  and  movement.  Virtual  reality  based  learning  can  bring  convenient 
interaction  and  visualization  features  (McLellan  96). 

Moving  the  annotation  to  a 3D  environment  would  impose  at  first  a creation  of  a 3D  metaphor  for  the 
annotation-place.  Which  are  then  the  implications  of  having  the  annotation-place  as  a 3D  metaphor? 

As  in  the  case  of  text-based  annotations,  it  is  necessary  to  investigate  how  the  3D  spatial  notion  affects 
interaction  paradigms  in  order  to  decide  about  features  of  annotation  metaphors.  The  ideas  of  Certeau  were  also 
shown  to  be  an  appropriate  strategy  to  define  interaction  paradigms  in  3D  environments  (Jensen  99). 

The  implemented  learning  scenario  is  a virtual  reality  model  based  on  VRML  (VRML  97),  combining 
a 3D  paradigm  with  the  world  metaphor.  The  consequences  to  the  annotation  metaphor  and  the  annotation- 
place  are  numerous.  First,  annotation-places  are  connected  to  objects  that  provide  richer  context  to  the 
annotations.  For  example,  a grocery  shopping  list  will  probably  be  placed  on  the  refrigerator  door  or  on  some 
other  agreed  place  in  the  kitchen.  Second,  relations  between  annotation-places  can  be  drawn  more  easily,  based 
on  the  objects  they  are  attached.  Third,  the  expected  readers  of  the  annotations  can  be  selected  by  these  different 
relationships.  For  example,  a virtual  classroom  may  have  an  annotation  place  on  the  entrance  door,  available  to 
every  one  at  school,  a group-available  annotation-place  on  the  blackboard,  and  also  a private  one  on  each 
student  desk. 

We  implemented  an  illustrative  prototype  of  an  annotation-place  located  on  a refrigerator  door.  Such 
place  enables  addition,  removal  and  edition  of  annotations  that  are  located  all  over  the  door.  When  moving  the 
mouse  over  an  annotation,  its  author  name,  date  and  keywords  are  displayed.  A click  on  annotation  displays  its 
content  in  an  editable  dialog  (Fig.  2). 


Figure  2:  VR-based  annotations. 
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The  implemented  3D-annotation  environment  corresponds  to  the  commenting  scenario  discussed  in 
the  previous  section.  Regarding  the  paradigm  ruptures  described  by  Certeau,  the  3D  annotation  is  dynamic  in 
every  aspect.  It  is  considered  a space  due  to  the  interaction  opportunities  for  editing,  removing  or  adding 
annotations.  The  paradigm  of  bridge  is  present  in  the  links  relating  the  post-it  like  object  and  the  annotation 
contents.  The  trail  paradigm  was  used  because  it  is  difficult  to  offer  a total  vision  of  textual  information  {map) 
in  3D  environments. 


Conclusions 

Thinking  of  annotations  as  metaphors  subjected  to  paradigms,  and  these  latter  as  direct  outcomes  of 
adopting  and  combining  media,  is  a complementary  approach  to  investigate  the  effects  of  changing  paradigms 
when  designing  interactive  learning  scenarios.  Annotations  on  paper  and  the  classic  3M  post-its  are  well- 
proofed  successful  implementations  of  the  annotation  metaphor.  Moving  them  to  digital  media  is  still  a 
challenge  that  needs  further  understanding  on  how  actors  in  an  educational  environment  perceive  spaces  and 
places  for  annotation.  Corroborating  these  ideas,  there  is  some  work  investigating  effects  of  interfaces  for 
annotation  (Wohjan  et  al.  98),  student  learning  improvement  by  use  of  annotations  (Armel  & Shrock  96),  and 
architectures  for  annotate-capable  documents  (Phelps  & Wilensky  97). 

The  3D  prototype  was  an  interesting  experience  and  opened  the  possibility  for  innovative  uses  of 
annotation-places  such  as;  audio  and  video  annotations,  3D  annotation-places  for  authoring  and  discussion,  and 
annotation-places  defining  a specific  text  format  (e.g.,  via  XML  - Extensible  Markup  Language  with  an 
appropriate  style  definition). 

The  study  of  paradigm  ruptures  can  also  be  applied  to  other  educational  metaphors,  such  as  interface 
agents,  user  avatars,  3D  spatial  notion,  etc.  When  interaction  paradigms  for  certain  metaphors  become  clear  to 
designers  of  educational  environments,  new  media  will  be  better  used  and  the  risk  of  rejection  to  changes  will 
be  reduced.  Therefore,  it  will  be  given  a further  step  towards  the  definition  of  a more  general  paradigm  to  be 
used  in  the  transmission  and  manipulation  of  knowledge  in  the  digital  era. 
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Abstract:  The  transition  from  text-based  computer  systems  with  a narrow  band  of 
interactivity,  to  multimedia  systems  with  much  broader  potential  for  interaction,  has  urged 
designers  to  consider  human-factor  issues  to  increase  usability.  In  terms  of  machine 
intelligence,  multimedia  technology  is  no  different  from  earlier  text-based  systems.  But  in 
terms  of  human  use,  there  is  a strong  need  for  the  development  of  guidelines  in  designing 
those  systems  and  more  research  is  required  in  this  field.  This  section  starts  by  highlighting 
some  multimedia  design  issues  such  as  design  layers,  navigation,  interactivity,  and  user 
control,  then  the  researcher  describes  three  categories  of  multimedia  design,  which  are: 
learning  design,  interface  design,  and  screen  design.  The  summary  highlights  important 
issues  for  consideration  by  multimedia  designers,  pitfalls  to  be  avoided,  and  areas  where 
more  research  is  needed. 


1.  Introduction 

The  advent  of  interactive  multimedia  has  resulted  in  a greater  consideration  of  presentation  and  interaction 
styles  and  methods  (Ring  and  Ring,  1997),  The  design  of  multimedia  learning  environments  may  be  considered 
in  two  broad  design  layers:  the  conceptual  design  layer  and  the  presentation  design  layer.  The  conceptual 
design  layer  deals  with  the  structural  organisation  of  the  learning  environment  and  involves  creating  the  deep 
learning  architecture  of  the  system,  the  presentation  design  layer  however,  deals  with  the  realisation  of  the 
system  as  a computer  based  artefact.  In  presentation  design,  the  abstractions  of  conceptual  design  are  mapped 
onto  a real  multimedia  environment  (Boyle,  1997). 

Since  interactive  multimedia  uses  a variety  of  media  and  sophisticated  interactions  via  several  sensory 
channels,  and  taking  into  account  the  fact  that  learner’s  brain  has  limited  capabilities,  it  can  easily  be 
overloaded  with  information.  Ring  and  Ring  emphasise  that  designers  of  interactive  multimedia  must 
understand  the  memory  limitations  of  the  learners  and  should  think  about  how  to  create  meaningful 
environments  that  require  minimal  effort  to  learn  and  remember  (Ring  and  Ring,  1997).  As  users  have  different 
types  of  language,  background,  values,  and  traditions,  each  needs  to  work  at  different  paces  and  in  different 
styles.  Vossen  states  that  effective  multimedia  design  requires  the  involvement  of  a systematic  and 
comprehensive  approach  to  analyse  the  context  of  use  which  includes  modelling  all  the  components  and  aspects 
such  as  the  user,  task,  and  setting.  He  identifies  four  elements  of  multimedia  design:  content,  structure,  access, 
and  style  (Vossen  et  al,  1997). 
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2.  Design  for  Interaction 


The  major  difference  between  traditional  instruction  and  instruction  delivered  by  multimedia  is  interaction 
(Schwier  and  Misanchuk,  1993).  Good  design  of  interactive  multimedia  is  measured  by  interactivity  which 
makes  the  application  more  useful.  The  powerful  feature  of  multimedia  is  that  it  allows  individual  learners  to 
be  active  participants  in  the  multimedia  application,  by  enabling  them  to  interact  with,  and  control  the  flow  of 
information  within  the  application,  Hannafin  emphasises  that  it  is  important  to  design  as  much  meaningful 
interactivity  as  possible  into  a multimedia  application  (Hannafin,  1989).  McAteer  and  Shaw  see  that  the  well 
designed  human-computer  interface  prompts  the  user  to  explore  and  learn  (McAteer  and  Shaw,  1995). 

Gillham  and  Buckner  identify  interactivity  as  a term  that  implies  an  existing  involvement  in  the  way  the 
information  is  presented  and  retrieved  (Gillham  and  Buckner,  1997),  Deegan  states  that  multimedia  systems 
are  generally  designed  to  be  interactive  in  that  they  allow  users  to  interrogate  the  programme  in  a linear  and 
non-linear  way  (Deegan  et  al,  1996).  Some  useful  guidelines  which  are  believed  to  promote  learner’s 
interactivity  are  provided  by  (Poncelet  and  Proctor,  1993). 


3.  Design  for  User  Control 

Multimedia  designers  are  concerned  with  user  control  which  is  believed  to  be  a central  topic  to  the  design  of 
interactive  instruction,  it  includes  the  aspects  of  interface  and  instructional  design  (Stoney  and  Wild,  1997). 
McKerlie  and  Preece  state  that  learner  control  takes  three  forms:  control  over  the  structure,  control  over  the 
media,  and  control  over  the  working  environment  (McKerlie  and  Preece,  1993).  It  is  suggested  to  put  control  in 
the  hand  of  the  user  to  enhance  learning,  Laurillard  believes  that  learners  should  be  given  more  control  over 
the  content  by  providing  the  ability  to  navigate  through  the  multimedia  application  at  their  own  pace 
(Laurillard,  1987).  In  addition,  McAteer  and  Shaw  advise  designers  to  put  as  much  control  of  the  package  as 
possible  in  the  hands  of  the  user,  so  they  can  proceed  at  their  own  pace,  but  they  point  out  that  this  control 
should  be  at  a certain  point  (McAteer  and  Shaw,  1995).  Merideth  and  Richards  indicate  as  a result  of  their 
study  that  multimedia  positively  impact  students’  learning  by  giving  the  learners  control  over  the  learning 
materials  and  by  supporting  their  higher  order  thinking  skills  (Merideth  and  Richards,  1997). 


4.  Design  of  Educational  Multimedia 

The  following  is  a description  of  the  three  categories  of  the  design  of  educational  multimedia.  Theses  are: 
learning  design,  interface  design,  and  screen  design. 


4.1  Learning  Design 

As  multimedia  resources  add  pedagogical  value  to  the  learning  materials,  the  instructional  design  has  a 
significant  impact  on  the  quality  of  learning.  Hedberg  suggests  that  consideration  must  be  given  to  the 
cognitive  load  placed  on  the  user  if  designers  want  the  user  to  access  the  information  effectively  and  efficiently. 
He  believes  that  improvement  in  learning  outcomes  can  be  supported  by  allowing  learners  to  focus  on 
metacognitive  processes  as  a component  support  incorporated  within  the  interface  (Hedberg  et  al,  1994). 
Harasim  argues  that  the  use  of  new  technology  does  not  improve  educational  outcomes  by  itself  She  sees  the 
need  to  focus  on  new  designs  for  learning  as  well  as  new  ways  to  design  the  technological  environments  that 
can  support  learners’  cognitive  process  (Harasim,  1995). 

Edward  states  that  computer  based  learning  (CBL)  has  a long  history  from  the  early  text  based  to  today’s 
virtual  reality  multimedia  offering.  He  believes  that  the  majority  of  multimedia  learning  packages  are 
concerned  with  teaching  and  testing  the  acquisition  of  large  scale  factual  information.  He  suggests  the  need  for 
designing  multimedia  learning  packages  that  facilitate  conceptual  understanding  approaches  (Edwards,  1997). 
Kennedy  claims  that  many  educators  think  that  interactive  multimedia  learning  programmes  have  failed  to 
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deliver  their  promised  potential,  he  refers  to  the  use  of  an  inappropriate  design  model  for  computer  assisted 
learning  (CAL)  and  hypermedia.  He  sees  that  these  instructional  design  models,  are  developed  from  the 
assumptions  that  underlie  the  behavioural/objectivist  pedagogy  of  teaching  and  learning  (Kennedy,  1995). 

Suni  and  Ross  argue  that  instructional  design  is  a systematic  process  for  creating  learning  and  instructional 
principles,  and  combine  them  into  instructional  materials  (Suni  and  Ross,  1997).  This  argument  is  supported  by 
Boyle  who  sees  three  stages  of  instructional  development:  needs  analysis,  selection  of  instructional  methods 
and  materials,  and  evaluation  (Boyle,  1997). 

Fortunately  hypertext  and  hypermedia  authoring  packages  allow  designers  of  multimedia  courseware  to  design 
the  learning  materials  in  a structured  and  unstructured  way,  and  allow  them  to  design  the  programme  giving 
greater  flexibility  for  the  learners  to  structure  the  learning  environment  to  suit  their  own  learning  needs, 
benefited  from  the  non-linearity  of  the  programme  (Stanton  and  Stammers,  1989).  In  this  regard,  Jonassen 
believes  that  there  is  a benefit  of  hypertext  in  which  it  permits  text  presentation  to  be  locally  modified  by  the 
learners  in  order  to  meet  their  individual  needs  (Jonassen,  1986). 

A classifications  of  instructional  design  models  can  help  in  deciding  on  an  appropriate  multimedia  design  to 
achieve  the  educational  objectives  for  any  specified  course.  Developers  must  understand  and  distinguish 
between  various  learning  and  instructional  strategies.  Boyle  discusses  two  approaches  to  the  design  of 
computer  based  learning  environments:  traditional  instructional  design  and  constructivism.  He  believes  that  the 
traditional  instructional  design  has  been  criticised.  He  sees  constructivism  as  a dominant  paradigm  in 
educational  multimedia  design,  it  has  a much  stronger  base  in  the  psychology  of  learning  and  cognitive 
development  (Boyle,  1997). 

Midoro  warns  educators  against  creating  multimedia  systems  for  doing  old  tasks.  He  suggests  the  use  such 
systems  to  open  up  new  possibilities.  He  urges  researchers  to  fully  exploit  the  multimedia  features  to  enhance 
the  teaching  and  the  learning  practices  (Midoro,  1993).  Compagnoni  and  Ehrlich  conclude  in  their  study  two 
significant  issues.  The  first  one  is  that  individuals  with  better  visualisation  skills  were  faster  in  accessing 
information  in  hypertext  learning  systems  compared  to  those  with  poorer  visualisation  skills.  The  second  one  is 
that,  hypertext  seems  to  cause  learning  by  browsing,  which  is  a better  strategy  compared  to  the  use  of 
predetermined  menus  (Compagnoni  and  Ehrlich,  1989). 

Montgomery  identifies  four  characteristics  which  seem  to  contribute  to  the  success  of  any  multimedia 
courseware  package.  These  are  engaging  the  learner,  the  ability  to  provide  immediate  access  to  information,  the 
ability  to  provide  alternative  explanations  or  examples,  finally  a practice  to  provide  practical  opportunities  for 
skill  development  (Montgomery  et  al,  1993).  Authoring  packages  allow  the  use  of  multimedia  components  such 
as  sound,  graphics,  and  moving  images  which  provide  a powerful  opportunity.  McAteer  and  Shaw  however, 
warn  that  authors  might  be  seduced  into  focusing  on  the  impressive  technology  and  losing  sight  of  engaging, 
informing  and  educating  the  students  (McAteer  and  Shaw,  1995  ). 


4.2  Interface  Design 

A multimedia  learning  package  is  meaningless  if  there  is  no  means  to  navigate  through  it  or  experience  it.  User 
interface  design  as  described  by  Lisa  Lopuck  “is  the  process  of  ergonomically  and  strategically  presenting 
media  in  order  to  communicate  a message”  (Lopuck,  1996).  The  emergence  of  interactive  multimedia  has 
raised  several  new  issues  in  interface  design.  Ring  and  Ring  emphasise  that  interfaces  of  interactive  multimedia 
products  should  be  aesthetically  pleasing,  and  should  offer  the  learner  sophisticated  interactions  insuring  the 
ease  of  use.  They  state  that  good  interface  design  makes  use  of  a variety  of  functions  and  visual  entities  to 
enhance  both  interactivity  and  appearance.  They  argue  that  the  goal  of  the  interface  is  to  facilitate  interaction 
and  to  make  the  organisational  structure  and  content  visible  (Ring  and  Ring,  1997). 

Davies  stresses  that  designers  should  create  a conceptual  model  “which  is  the  overall  view  of  the  system,  or  part 
of  the  system,  as  conceptualised  by  the  designer”.  Also  they  should  create  a mental  model  “which  is  how  users 
actually  view  the  system  which  will  vary  with  the  level  and  experience  of  the  user”.  The  reason  for  the  creation 
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of  conceptual  and  mental  models  is  to  make  the  interaction  as  comfortable  and  efficient  as  possible  (Davies  et 
al,  1994).  The  goal  of  the  mental  model  is  “to  create  an  environment  that  facilitates  the  mapping  of  the 
designers  model  onto  the  user  model”  (Ring  and  Ring,  1997).  If  users  can  easily  build  mental  models  of 
multimedia  computer  assisted  learning  environments  they  will  perceive  the  systems  as  user  friendly  and  easy  to 
learn.  If  the  models  are  too  complex,  the  system  will  be  perceived  as  confusing  and  unfriendly.  The  mental 
model  should  not  focus  on  what  the  computer  is  doing,  but  rather  on  what  is  going  on  in  the  context,  objects, 
and  other  representations  (Calvi,  1997). 

Designers  of  multimedia  learning  programmes  should  help  users  to  develop  an  appropriate  conceptual  model 
by  providing  them  with  a familiar,  clear,  and  consistent  metaphor  that  represents  the  way  the  system  actually 
works  and  behaves.  There  is  now  general  agreement  that  consistency  enhances  usability  (Waterworth,  1992). 
Some  common  metaphors  are  “office”,  “book”,  “encyclopaedia”,  “tutorial”,  “discovery”,  “exploration”,  etc.  The 
desktop  metaphor  is  used  at  the  top  level  of  the  application,  and  it  should  be  enhanced  with  application-specific 
actions  or  replaced  with  other  appropriate  real-world  metaphors.  After  an  application  is  opened,  it  should  use  a 
real-world  metaphor  appropriate  to  its  functions.  For  example,  in  an  inventory  application,  displays  might 
reflect  the  actual  equipment  they  represent,  and  equipment  could  be  placed  or  removed  from  locations  which 
look  like  shelves  and  slots. 

The  issue  of  users  being  disoriented  getting  lost  in  the  hypermedia/multimedia  application  " has  been  a 
concern.  Conklin  warns  interface  designers  from  the  disorientation  problems  which  may  get  the  user  lost  in  the 
program,  or  may  make  the  user  not  sure  where  he/she  is  in  relation  to  other  parts  of  the  program  network 
(Conklin,  1987).  However,  Mayes  sees  that  the  disadvantage  of  disorientation  may  be  an  advantage  for  the 
learner  in  some  type  of  exploratory  learning.  The  problem  may  not  be  whether  a map  can  be  found  but  how  a 
map  can  be  created  by  the  learner  (Mayes  et  al,  1 990).  Calvi  believes  that  getting  lost  in  the  hyperspace  is  not 
only  about  multimedia  or  hypermedia  systems,  she  sees  that  printed  text  books  do  entail  a certain  degree  of 
disorientation  (Calvi,  1997). 

In  addition,  Riley  believes  that  there  is  a risk  that  learners  will  lose  their  bearings  and  be  “lost  in  the 
hyperspace”.  He  suggests  that  designers  should  provide  sufficient  navigational  aids  for  users  to  navigate  a 
program  with  ease  (Riley,  1 995).  However,  some  research  indicates  that  designing  multimedia  and  hypermedia 
with  complete  user  control  leads  to  a deeper  level  of  understanding  (Suni  and  Ross,  1997).  Elliott  sees  a great 
weakness  in  the  use  of  hypermedia  applications  which  lies  in  the  degree  of  learners’  control,  he  suggests  that  to 
give  direction  when  a user  engages  with  the  application  can  overcome  this  weakness  (Elliott  et  al,  1995). 
Further  guidelines  on  learner  control  are  given  by  (Orr  et  al.,  1 994). 

One  of  the  most  common  ways  of  identifying  or  accessing  information  in  a multimedia  programme  is  by 
navigating  its  interface.  By  creating  a motivating  style  for  interaction  control  and  access,  learners  may  quickly 
learn  where  to  look  in  the  programme  to  interact  with  the  interface.  Calvin  believes  that  an  easy  navigation 
depends  upon  learners’  ability  to  abstract  the  application  and  its  display  in  order  to  build  a conceptual 
representation  of  its  architecture  (Calvin,  1997).  Search  suggests  the  need  to  develop  interfaces  with 
orientation  cues  that  help  learners  navigate  through  the  multimedia  applications  (Search,  1993). 

4.3  Screen  Design 

Good  screen  design  attempts  to  impose  a consistency  on  the  layout  of  the  screens  of  a multimedia  program. 
Visual  organisation  of  media  in  the  application  will  undoubtedly  influences  the  ease  and  quality  of  learning, 
and  has  an  important  impact  on  cognitive  load  (Stoney  and  Wild,  1997).  McAteer  and  Shaw  believe  that 
consistency  in  screen  design  and  visual  organisation  of  media  in  the  application  influences  the  ease  and  quality 
of  learning  (McAteer  and  Shaw,  1995).  Preece  suggests  that  organising  information  in  an  easily  accessible 
form  can  reduce  the  time  taken  to  find  a specific  piece  of  information  (Preece,  1990). 

Screens  include  some  visual  parts  and  areas  that  should  be  associated  with  specific  tasks  such  as  titles  or 
headings,  information  area,  instruction  area,  navigation  area,  and  feedback  area.  Hannafin  and  Hooper 
emphasise  that  designers  should  consider  the  visual  aspects  including  the  design  and  placement  of  text, 
graphics,  and  various  navigational  elements  and  keep  the  screen  as  simple  and  uncluttered  as  possible 
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(Hannafin  and  Hooper,  1989).  Milheim  and  Lavix  define  some  aspects  of  screen  design  which  include  the 
location  of  various  components  on  each  screen,  and  the  consistency  of  those  locations  throughout  the 
multimedia  programme  (Milheim  and  Lavix,  1992). 

The  importance  of  screen  design  has  been  emphasised  by  many  researchers.  Mukherjee  and  Edmonds  stress 
that  screen  design  is  becoming  a very  important  issue  when  developing  multimedia  learning  materials  in  many 
different  content  areas.  They  introduce  a broadened  definition  of  screen  design  which  includes  the  co- 
ordination of  textual  and  graphical  elements  to  easily  present  the  content  of  the  learning  materials  to  learners 
(Mukherjee  and  Edmonds,  1993).  Milheim  and  Lavix  believe  that  if  an  instructional  screen  in  a multimedia 
application  provides  effective  instruction,  visual  aesthetics,  and  appropriate  navigational  tools,  it  will 
undoubtedly  allow  for  maximum  learning  from  the  materials  whilst  providing  the  learner  with  adequate  control 
of  the  learning  process  (Milheim  and  Lavix,  1992).  Hannafin  and  Hooper  stress  that  screen  design  should 
assist  the  user  to  easily  facilitate  the  process  of  accessing  the  information  providing  the  user  with  cognitive 
benefits  to  perceive,  organise,  and  integrate  information  (Hannafin  and  Hooper,  1989). 

The  proper  use  of  colours  is  a very  important  issue  for  multimedia  design,  and  they  should  be  considered 
carefully.  Boyle  states  that  the  aim  of  the  use  of  appropriate  colours  is  to  insure  an  effective  and  pleasing  visual 
display  (Boyle,  1997).  Durrett  and  Stimmel  indicate  that  the  excessive  use  of  colours  may  have  a negative  effect 
on  learning  (Durrett,  Stimmel,  1987).  Whilst  the  use  of  certain  colour  combinations  can  lead  to  a complete  lack 
of  perception  (McAteer,  Shaw,  1995).  Some  authors  (Milheim  and  Lavix,  1992;  Orr  et  al.,  1994)  provided 
some  guidelines  for  using  colours. 

In  summary,  good  screen  design  can  fulfil  the  following  requirements:  (1)  focus  learners’  attention,  (2)  develop 
and  maintain  interest,  (3)  promote  processing,  (4)  promote  engagement  between  the  learner  and  the  lesson 
content,  (5)  help  learners  find  and  organise  information,  and  (6)  facilitate  lesson  navigation  (Hannafin  and 
Hooper,  1989;  Mukherjee  and  Edmonds,  1993). 


5.  Summary 

5.1  Designer  Considerations 

♦ Create  a meaningful  environment  that  requires  minimal  effort,  by  modelling  all  the  components  and  aspects 
(user,  task,  and  educational  environment). 

♦ Constructivism  is  better  than  the  traditional  design  in  which  it  considers  the  student  as  a main  participant  in 
the  knowledge  system,  through  interacting  with  a given  situation. 

♦ Designers  should  create  conceptual  and  mental  models  by  providing  learners  with  familiar  metaphors  which 
represent  the  way  the  system  actually  works  and  behaves. 

♦ Screens  should  be  designed  with  consistency  and  with  visual  organisation  of  text,  graphics,  and  various 
navigational  elements. 

♦ Simplicity  of  the  design  influences  the  ease  and  quality  of  learning  and  has  an  important  impact  on 
learners’  cognitive  load. 

5.1  Pitfalls  for  Designing  Educational  Multimedia 

♦ Design  for  complete  user  control  can  lead  to  disorientation. 

♦ The  use  of  an  inappropriate  design  models  may  mean  that  the  program  will  fail  to  deliver  the  promised 
potential  of  multimedia. 

♦ The  impressive  technology  of  multimedia  may  lead  designers  to  lose  sight  of  engaging,  informing,  and 
educating  the  students. 

♦ Current  authoring  packages  provide  templates  which  help  designers  in  laying  out  the  screen  in  a matter  of 
minutes,  but  they  do  not  necessarily  guard  against  bad  design. 

5.3  Further  Studies 


o 
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There  is  a need  to  carry  out  controlled  studies  to  establish  the  effectiveness  of  user  control  paradigms  (total 
control  V,  guided  navigation)  and  instructional  design  models.  To  date  little  work  has  been  done  to  analyse  the 
effects  of  educational  setting,  environment,  and  learner  culture  on  multimedia  design.  The  authors’  studies 
highlight  the  importance  of  cultural  issues  in  the  effectiveness  of  the  design  of  multimedia  learning  materials. 
In  addition,  a new  focus  is  needed  on  design  for  learning  in  the  new  technological  environment,  for  example 
using  virtual  reality  in  learning.  More  research  is  needed  on  how  to  upgrade  design  guidelines  into  authoring 
packages. 
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Abstract:  This  paper  features  a study  on  the  use  of  electronic  catalogs  and  retail  storefronts  as  a 
concrete  collaborative  class  project  for  the  Electronic  Commerce  Course  in  the  Business  curriculum 
and  a survey  of  the  MBA  students  that  undertook  this  project  in  response  to  the  course 
requirements.  The  survey  results  provide  positive  indications  for  the  achievement  of  the  goals  of 
collaborative  inquiry,  team-based  and  active  learning,  constructivist  pedagogical  approach,  and 
learning  by  doing.  The  technical  details  of  the  project  are  also  described  in  the  study. 


Introduction 

The  Internet  is  providing  an  extraordinary  environment  for  designing  teaching  and  learning  experiences. 
This  paper  will  focus  on  how  the  World  Wide  Web  (WWW)  could  be  utilized  for  teaching  Electronic  Commerce  (E- 
Commerce)  in  Management  Information  Systems  (MIS)  program  of  the  Business  curriculum.  The  opening  of  the 
Internet  to  businesses  in  the  early  nineties  initiated  the  E-Commerce  phenomena  that  has  not  only  revolutionized  the 
conduct  of  business  online  but  also  overhauled  the  way  E-Commerce-related  business  courses  need  to  be  taught. 
The  unusual  aspect  of  these  courses  is  that  the  WWW  is  primarily  the  environment  in  which  online  establishments 
also  conduct  their  business.  This  means  students  can  actually  simulate  the  conduct  of  business  via  the  same 
environment  where  their  learning  takes  place.  Not  only  that — the  instructional  setting  can  also  take  advantage  of 
web  hosting  services  used  to  support  online  businesses  to  supplement  if  not  entirely  replace  inadequate  university 
information  technology  infrastructure  that  needs  to  undergird  the  activities  of  electronic  merchants. 

This  paper  reports  the  experiences  of  the  MBA  students  at  the  School  of  Business,  Montclair  State 
University,  in  building  electronic  storefronts  using  Merchandizer  software,  which  is  a requirement  in  their 
Management  Information  Systems  course  that  has  an  E-Commerce  theme.  Using  the  survey  technique,  student 
feedback  was  gathered  from  two  MBA  classes  that  took  the  course  during  the  summer  and  fall  1999.  Data  was 
conducted  to  measure  students’  perceptions  of  the  quality  of  their  overall  learning  experience  and  their  ability  to 
gain  collaborative  team  work  skills,  discipline-specific  skills  in  building  an  electronic  storefront  and  catalog,  skills 
in  using  the  Merchandizer  software,  and  various  forms  of  thinking  skills. 

The  E-Commerce  course  could  be  taught  by  combining  the  pedagogical  principles  that  surround  a number 
of  appropriate  teaching  strategies:  cooperative  learning,  constructivist  approach,  active  learning,  and  learning  by 
doing.  These  strategies  can  be  used  to  structure  and  coordinate  electronic  learning  spaces.  In  a cooperative  learning 
environment,  the  instructor  organizes  content  and  topical  activities  around  tasks,  problems,  and  projects  that  student 
teams  with  a balanced  mix  of  abilities  and  backgrounds  are  designed  to  undertake  (Adams  and  Hamm,  1996). 
Cooperative  learning  also  provides  valuable  opportunities  for  students  to  work  together  across  gender  and  ethnic 
lines  as  long  as  communication  skills,  prior  achievement,  expectations,  friendships,  and  group  status  are  factored  in 
(Chizhik,  1999).  A meta-analysis  of  university-level  and  adult-learning  courses  found  that  use  of  collaborative 
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learning  concepts  promoted  higher  achievement,  higher-level  reasoning,  more  frequent  generation  of  ideas  and 
solutions,  and  greater  transfer  of  learning  than  did  individualistic  or  competitive  learning  strategies  (Johnson  et  al., 
1991).  Team  members’  social  skills,  also  called  "interpersonal  intelligence"  according  to  Howard  Gardner, 
encompassing  interpersonal  communication,  task  management,  group  interaction,  and  conflict  resolution  are 
constantly  honed  throughout  the  process. 

The  constructivist  pedagogical  approach  invites  students  to  build  personal  meanings  out  of  the  material 
taught  and  encourages  them  to  reflect  on  how  new  knowledge  builds  on  their  previous  base  of  understanding 
(Pankratius  and  Young,  1995).  In  a group  setting,  the  constructivist  approach  could  lead  to  the  social  creation  of 
knowledge  as  a basis  of  learning. 

"The  yeast  of  knowledge,  openness,  and  enterprise  raises  the  need  for  a multiplicity  of  learning  media  and 
technological  tools  "(Adams  and  Hamm,  1996).  The  very  wealth  of  Web  site  building  tools  that  is  now  available 
allows  students  to  experiment  and  "play"  through  "learning  by  doing"  activities  and  significantly  changes  the  whole 
instructional  experience.  Students  have  more  control  over  the  end  product  via  the  software  tools  and  they  assume 
greater  responsibility  over  their  learning  process  (Adams  and  Hamm,  1996).  MIS  instructors  are  strongly 
encouraged  to  move  towards  the  "learning  by  doing"  mode  which  is,  by  far,  the  most  effective  way  of  teaching 
students  "how  to"  skills.  While  this  is  clear  to  most  teachers,  it  is  not  easy  to  do  unless  certain  prerequisites  are  in 
place:  professional  development,  teacher  preparation,  technological  infrastructure  support  (hardware,  software,  and 
helpline  service),  an  appropriate  incentive  system  that  will  reward  the  countless  hours  that  will  go  into  a more 
technologically-oriented  class  (as  opposed  to  a traditional  lecture-style  one),  and  above  all,  an  overarching  plan  that 
clearly  defines  the  role  of  technology  in  the  pedagogical  process  (Adams  and  Hamm,  1995). 

Merchandizer  Software  Features 

Students  are  asked  to  build  an  electronic  retail  storefront  for  either  consumer-to-business  or  business-to- 
business  commerce.  What  this  means  is  that  the  store  can  sell  a product  that  individual  consumers  may  be  interested 
in  buying  such  as  an  Amazon.com  for  books  or  a CdNow.com  for  musical  CD-ROMs.  Or,  the  storefront  could  be 
an  electronic  catalog  for  firms  interested  in  buying  materials  they  need  in  running  their  business  such  as  office 
supplies,  for  instance — staplers,  bond  paper,  paper  clips,  etc. — in  other  words,  these  will  be  businesses  buying  from 
another  business.  The  product  used  in  the  project  is  Merchandizer  Software,  which  is  an  online  solution  that  allows 
users  to  build  and  manage  an  electronic  storefront  with  Just  a browser  software  such  as  Netscape  Communicator  or 
Microsoft  Internet  Explorer  and  an  Internet  connection  which  an  Internet  Service  Provider  (ISP)  or  web  hosting 
service  can  provide  for  a fee.  In  designing  the  site,  students  could  either  enter  all  product  information  using  a step- 
by-step  wizard  or  they  could  work  directly  with  hypertext  markup  language  (HTML)  forms  to  both  make  and 
modify  their  sites.  In  building  the  electronic  catalog  itself,  students  either  enter  information  manually  using  the 
wizard  templates  or  they  may  import  extensive  information  using  Microsoft  Access  database  tables  which  are  loaded 
onto  the  servers. 

Merchandizer  software  makes  use  of  frames  or  windows  appearing  within  the  browser’s  display  area,  each 
of  which  displays  the  contents  of  a different  HTML  file  (Carey,  1998).  The  software  also  allows  storebuilders  to 
divide  their  product  lines  in  a hierarchical  fashion,  first  into  groups,  then,  into  subgroups,  and  finally,  into  catalog 
items.  The  group  and  subgroup  headers  become  navigational  links  in  one  frame,  displaying  product  pages  or  other 
HTML  pages  or  entire  web  sites  altogether  created  by  the  students.  The  frame  concept  facilitates  the  navigation  of 
the  customer  through  the  storefront. 

Once  groups,  subgroups,  and  catalog  items  have  been  defined,  students  enter  product  information  either  of 
two  ways.  They  can  enter  the  information  using  simple  electronic  forms  that  allow  them  to  submit  information  like 
product  name,  code,  description,  and  price.  Alternatively,  they  could  use  preformatted  Microsoft  Access  tables  that 
essentially  mimic  the  wizard  online  templates.  These  tables  are  more  convenient  to  fill  out  when  merchants  have 
hundreds  or  even  thousands  of  product  items  to  enter.  Since  students  are  building  only  prototype  stores  with  a 
limited  product  line,  they  end  of  using  the  forms,  although  the  use  of  the  Access  table  is  demonstrated  to  them 
should  they  be  interested  in  trying  it. 

Merchandizer  has  features  like  a search  function  that  will  allow  hypothetical  customers  to  search  the 
storefront  using  any  word  or  phrase  related  to  the  products  on  sale.  Merchandizer  also  incorporates  a shipping  table 
for  UPS  directly  into  the  software  so  that  the  students  do  not  have  to  be  concerned  with  this.  They  need  to  specify 
the  appropriate  sales  tax,  though.  Merchandizer  keeps  track  of  all  sales  transactions,  customers,  and  prospects 
(those  who  visit  but  do  not  buy).  A report  generating  facility  gives  students  information  on  the  most  saleable 
products,  customers  who  buy  most  from  the  store,  aggregate  sales  figures  for  the  week  or  the  month,  and  areas  of 
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interest  of  prospects,  among  other  things.  The  software  can  also  autogenerate  email  messages  to  prospects  about 
special  promotional  activities  the  store  is  conducting. 

Version  3.0  of  Merchandizer  offers  students  a wide  range  of  features  to  play  with  to  simulate  a real  online 
store  even  more  realistically:  easy  translation  of  the  web  site  into  any  language;  ability  to  insert  forms  with  special 
buying  instructions  before  the  shopper  checks  out;  offer  global  coupon  numbers  to  shoppers  so  that  they  can 
accumulate  their  purchases  and  eventually  received  discounts  on  items;  ability  to  submit  the  store  site  to  top  search 
engines;  more  managerial  report  forms  that  can  be  generated  from  the  sales  transaction  data;  a personalization 
field  for  each  catalog  item  for  shopper  input  that  will  be  carried  over  to  the  order  tables;  ease  of  uploading  and 
downloading  Access  tables,  among  other  more  advanced  features. 

HipHip  software,  the  company  behind  the  Merchandizer  storefront  building  tool,  is  also  a web  hosting 
service.  A merchant  that  does  not  have  networking  facilities  to  support  the  store  could  subscribe  to  this  service  and 
pay  a one-time  setup  fee  and  monthly  maintenance  fees  that  vary  depending  on  the  number  of  catalog  items 
supported  in  the  store.  HipHip  software  offers  educational  pricing  for  universities  interested  in  using  their  hosting 
services  for  the  purpose  of  instruction.  This  arrangement  is  a boon  for  universities  and  colleges  that  do  not  have 
adequate  information  technology  infrastructure  to  allow  E-Commerce  instructors  to  mount  interesting  student 
projects. 

The  web  hosting  service  is  important  because  of  the  backend  office  component  of  the  online  store.  Sales 
transaction  information  submitted  by  customers  and  personal  information  keyed  in  by  interested  prospects  who  do 
not  purchase  during  their  visit  to  the  store  need  to  be  stored  in  web  servers  so  that  the  order  could  be  processed  and 
sales  prospects  could  be  reached  through  promotional  email  messages  in  the  future.  Backend  office  processing 
requires  the  availability  of  web  servers  that  can  store  order  transaction  data  in  databases  that  can  later  be  accessed 
using  some  form  of  structured  query  language  so  that  reports  and  statistics  could  be  generated  for  managerial 
decision  making.  The  extensive  requirements  of  the  storefront  make  the  storefront  project  too  challenging  for  the 
normal  capabilities  of  a typical  university’s  academic  technology  support  group.  It  is  far  better  to  sign  up  with  the 
web  hosting  service — if  reasonable  fees  could  be  negotiated  with  the  company — and  have  their  technical  support 
staff  provide  the  services  necessary  for  optimizing  all  functionalities  and  features  of  the  storefront. 

Methodology 

This  study  used  the  survey  technique  using  a written  questionnaire,  which  was  administered  to  two  classes 
of  the  MIS  503,  Management  Information  Systems  course  at  the  MBA  level.  Data  was  gathered  during  the  Summer 
and  Fall  of  1999.  Both  classes  had  the  theme  of  “Electronic  Commerce”  and  the  project  assigned  to  the  students 
was  the  construction  of  an  online  retail  storefront  for  either  consumer-to-business  or  business-to-business 
commerce.  Students  were  assigned  in  teams  of  four  to  five  students.  Each  team  was  in  charge  of  a storefront  web 
site,  which  was  maintained  and  supported  by  HipHip  Software.  The  technical  support  staff  that  advises  the  students 
and  oversees  the  web  servers  that  host  the  software  and  product  information  entered  by  the  students  is  located  in 
Miami,  Florida.  Survey  data  was  analyzed  using  descriptive  statistics,  correlations,  and  simple  regressions  among 
key  variables. 

Findings 

The  sample  size  consists  of  43  students  in  the  MBA  program  of  the  School  of  Business,  Montclair  State 
University,  who  all  took  the  INFO  503  class  in  Management  Information  Systems,  with  special  emphasis  on 
Electronic  Commerce.  The  students  were  at  various  stages  in  the  MBA  program  and  majority  of  them  hold  full-time 
jobs  in  industry. 

Students  were  asked  to  respond  to  a five-point  Likert  scale  for  different  items  where  l=Strongly  disagree, 
2=Disagree  to  some  extent,  3=Uncertain,  4=Agree  to  some  extent,  and  5=Strongly  agree.  Most  of  the  students  felt 
that  the  retail  storefront  project  was  an  important  learning  experience  overall  (mean=4.44).  Students  found  the 
project  instructive  in  terms  of  teaching  them  new  ways  of  doing  business  over  the  World  Wide  Web  (mean=4.35). 
They  also  thought  that  the  project  taught  them  skills  they  could  use  after  the  completion  of  the  course,  particularly 
the  setting  up  of  an  online  Web  site  for  a product  or  service  (mean=4.30).  They  also  felt  the  project  gave  them  a 
chance  to  create  Web  site  content  that  was  meaningful  to  the  entire  team  (mean=4.30).  In  the  construction  of  the 
storefront  site,  they  thought  that  they  learned  the  following  specific  skill  sets  in  decreasing  order  of  importance: 
design  issues  for  creating  an  online  catalog  (mean=4.30);  overall  planning  issues  for  preparing  an  online  retail  web 


O 

ERIC 


82 


site  (mean=4.14);  web  site  product  information  and  content  issues  (mean=4.07);  advertising  (mean=3.84);  and 
product/service  promotional  activities  (mean=3.74). 

The  next  important  set  of  findings  relate  to  the  students’  perceptions  of  how  this  project  enhanced  their 
ability  to  collaborate  with  peers  in  a team  setting.  Most  students  found  the  project  a good  venue  for  encouraging 
collaborative  teamwork  (mean=4.40).  Communication  skills  were  paramount  in  getting  the  work  done 
(mean=4.33).  While  standards  for  personal  accountability  were  set  within  the  team  setting  (mean=4.30),  members 
made  sure  they  also  helped  each  other  out  (mean=4.28).  The  project  clearly  taught  the  students  a variety  of  group 
process  skills:  (a)  looking  out  for  the  best  interest  of  all  concerned  over  and  above  one’s  own  personal  interests 
(mean=4.26);  (b)  arriving  at  mutually  acceptable  solutions  (mean=4.26);  (c)  dividing  the  work  equitably  among 
members  (mean=4.23);  (d)  practising  interpersonal  social  skills  (mean=4.21);  (e)  learning  from  members’  different 
ways  of  thinking  (mean=4.21);  (f)  learning  positive  interdependence  (mean=4.21);  (g)  respecting  a diversity  of 
opinions  (mean=4.16);  and  (h)  striving  for  goals  as  a team  rather  than  as  individuals  (mean=4.12). 

They  gained  a better  understanding  E-Commerce  as  a subarea  of  Management  Information  Systems  (MIS) 
in  the  School  of  Business  curriculum  (mean=4.07).  The  storefront  project,  which  is  a natural  cross-over  between 
marketing  and  MIS,  made  them  more  aware  of  the  interdisciplinary  connections  within  the  business  curriculum 
(mean=3.95).  They  perceived  the  project  as  a venue  for  learning  about  E-Commerce  vendors  of  hardware  and 
software  (mean=4.28),  consulting  services  (mean=4.12),  professional  associations  (mean=3.79),  and  job  openings 
(mean=3.63). 

It  is  very  likely  that  the  positive  experiences  reported,  thus,  far  would  not  have  been  possible  if  the 
Merchandizer  storefront  software  itself  was  difficult  to  use.  Most  students  found  learning  the  software  a fairly  easy 
task  (mean=4.19)  and  that  they  found  it  was  a reachable  goal  to  be  skillful  in  the  use  of  it  (mean=4.14).  The 
Merchandizer  software  templates  were  easy  to  understand  (mean=4.07)  so  that  students  were  able  to  learn  them  on 
their  own  without  extensive  instruction  (mean=4.05). 

Two  important  attributes  of  students  in  the  sample  were  measured:  frequency  of  computer  use  and  length 
of  time  spent  working  with  the  computer.  In  a simple  regression  procedure,  it  was  found  that  frequency  of  computer 
use  significantly  predicts  students’  perception  of  the  ease  of  learning  the  Merchandizer  software  at  the  significance 
level  of  p<.  10  (F=3. 24284;  p=.0791).  In  two  other  simple  regression  runs,  it  was  found  that  the  students’  perception 
of  the  project  as  having  enhanced  collaborative  team  work  predicts  their  perception  of  the  storefront  as  being  an 
important  learning  experience  overall  (F==l  1 .42868;  p=.0016)  and  their  ability  to  create  a web  site  with  content  that 
is  meaningful  to  the  entire  team  (F=3. 59981;  p=.0648).  Then,  students’  perception  of  Merchandizer  as  an 
application  software  that  was  easy  to  use  predicted  their  organizing  skills  in  arranging  web  site  information  so  that 
the  site  could  be  presented  more  effectively  (F=l  2.275 18;  p=.001 1). 

When  taken  together  as  two  independent  variables  in  a regression  equation,  both  the  students’  perception 
of  the  project  as  having  enhanced  collaborative  team  work  and  their  perception  of  Merchandizer  as  an  application 
software  that  was  easy  to  use  predict  a number  of  dependent  variables.  Significance  levels  were  tightest  for  their 
organizing  skills  in  arranging  web  site  information  so  that  the  site  could  be  presented  more  effectively  (F=7.28378; 
p=.0020)  and  their  perception  of  the  storefront  as  being  an  important  learning  experience  overall  (F=7. 14604; 
p=.0022).  Both  independent  variables  in  a regression  equation  also  significantly  predicted  student’s  perception  that 
the  storefront  project  encouraged  creative  thinking  (F=4. 12591;  p=.0235),  their  perception  that  the  project  enhanced 
their  integrating  skills  or  their  ability  to  put  information  or  content  together  (F=3. 40767;  p=  0430),  and  their 
perception  that  the  project  developed  useful  skills  in  web  site  development  they  could  use  even  after  the  course  was 
over(F=2.62482;p=.0849). 

The  association  among  selected  variables  using  Pearson’s  correlation  values  was  also  investigated.  Only 
those  variables  that  bonded  very  strongly  at  the  significance  level  of  p<.0001  are  shown  here. 

Learning  important  lessons  in  determining  web  site  product  information  and  content  issues  correlated 
strongly  with  the  following  seven  variables:  learning  positive  interdependence  in  the  team  (.4999);  learning 

different  ways  of  thinking  in  the  team  (.5271);  achieving  the  goals  as  a team  (.5511);  considering  the  storefront 
project  an  important  learning  experience  overall  (.5193);  creating  web  site  content  meaningful  to  the  team  (.4843); 
developing  skills  usable  even  after  the  course  such  as  building  a web  site  (.6225);  learning  overall  planning  issues  in 
a storefront  (.6334);  and  learning  design  issues  for  an  electronic  catalog  (.6854). 

Learning  how  to  achieve  goals  as  a team  correlated  strongly  with  five  other  variables;  perceiving  the 
project  as  encouraging  collaborative  team  work  (.7380);  learning  to  communicate  with  team  members  (.7264); 
learning  positive  interdependence  (.8872);  learning  interpersonal  social  skills  (.8041);  and  learning  how  to  achieve 
goals  as  a team  (.9168). 

Learning  design  issues  for  creating  an  online  catalog  correlated  strongly  with  the  following  four  variables: 
perceiving  the  project  as  encouraging  collaborative  team  work  (.5269);  learning  to  communicate  with  team 
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members  (.5038);  learning  different  ways  of  thinking  in  the  team  (.5784);  and  learning  how  to  achieve  goals  as  a 
team  (.5380). 

Learning  different  ways  of  thinking  in  the  team  correlated  strongly  with  four  other  variables:  perceiving 
the  project  as  encouraging  collaborative  team  work  (.7333);  learning  to  communicate  with  team  members  (.7026); 
learning  positive  interdependence  in  the  team  (.8792);  and  learning  interpersonal  social  skills  (.8245). 

Conclusion 

This  paper  has  shown  concrete  data  supporting  the  promise  of  the  use  of  the  WWW  for  instructional 
purposes  in  the  area  of  E-Commerce  in  the  business  curriculum.  Not  only  does  the  WWW  provide  an  environment 
that  perfectly  simulates  the  subject  of  the  course  itself,  but  the  web  hosting  service  arrangement  also  provides 
information  technology  infrastructure  support  to  universities  that  are  resource  poor  or  unable  to  afford  the  hardware, 
software,  and  manpower  requirements  of  mounting  E-Commerce  projects.  The  survey  results  provide  positive 
indications  for  the  achievement  of  the  goals  of  collaborative  inquiry,  team-based  and  active  learning,  constructivist 
pedagogical  approach,  and  learning  by  doing. 
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Teacher  Education  by  Hypermedia  learning  Environments  - An 
Example  from  Germany 


Stefan  Aufenanger,  University  of  Hamburg/Germany,  Stefan@aufenanger,de 


In  Germany  the  problem  of  teacher  education  for  school  in  the  information  society  has  been  recognized  in 
the  last  years.  But  there  are  many  obstacles  to  change  concepts  of  traditional  curricula  for  teacher 
education,  to  train  the  teacher  educators,  to  implement  new  media  at  the  university,  and  so  on.  The 
necessarity  of  educating  teachers  and  teacher  students  for  using  computers  and  the  internet  for 
educational  purposes  became  also  aware  by  the  government.  There  are  many  interested  teachers  but  there 
are  not  enough  teacher  educators  to  face  that  challenge.  Media  in  general  and  television  especially  are 
rarely  used  in  German  schools.  German  teachers  mostly  think  of  electronic  media  as  a virus  which  is 
infecting  the  children  with  not  good  ideas  so  they  try  to  keep  them  out  of  their  classroom.  Only  the 
computer  and  the  internet  had  changed  some  teachers  mind.  The  pressure  of  using  computers  in  every  day 
life  and  in  all  areas  of  economy  is  now  reaching  the  schools.  Now  schools  are  equipped  with  computers 
and  internet  access  but  teachers  are  rarely  prepared  to  pedagogical  settings  for  the  use  of  its. 

Supported  by  a grant  from  the  Bertelsmann  Stiftung  in  Germany  seven  universities  in  Germany  have  built 
a network  to  cooperate  in  teacher  education  or  media  education  for  teachers,  respectively.  Teacher 
educators  of  he  Department  of  Education  of  each  of  the  seven  universities  came  together  to  develop  a 
concept  for  educating  teachers  for  media  education  and  computer  technology  at  schools.  The  challenge  of 
that  network  is  to  develop  and  to  implement  a curricula  that  could  be  accepted  by  each  of  the  seven 
universities.  All  participating  universities  have  a center  for  media  education  and  some  experiences  for 
teaching  with  computers  and  internet  but  all  have  not  enough  capacities  and  resources  to  implement  a 
whole  curricula  for  teaching  the  teachers  in  media  education  and  computer  technologies.  For  that  reason 
the  network  could  offer  each  university  the  chance  to  develop  a curriculum  on  the  one  hand,  and  to 
implement  that  curriculum  on  the  other  hand,  if  new  media  would  be  used.  The  aim  of  the  network  is  in 
general  to  exchange  experiences  in  media  education  for  teachers,  and  especially  to  develop  together 
hypermedia  learning  environments  for  that  topic.  If  each  part  of  the  networking  would  develop  one  part  of 
the  curriculum  and  one  part  of  a hypermedia  learning  module  system  resources  could  be  better  shared  and 
different  views  to  the  same  topic  could  be  offered. 

In  a first  step  the  general  task  of  the  network  was  break  down  to  the  theme  “Teaching  and  Learning  in  the 
Informational  Age”.  In  a second  step  the  main  issues  of  that  theme  has  been  outlined.  So  it  is  important  to 
understand  what  are  the  main  topics  of  the  informational  age.  Or  it  had  to  be  clarified  what  are  the  main 
concepts  of  teaching  with  computers  and  concepts  of  learning  with  computers.  In  a further  step  different 
concepts  for  the  educational  approach  has  been  developed. 

The  aims  of  the  curricula  has  been  formulated  as  follows: 

• To  help  teachers  to  get  media  competencies 

• To  have  an  understanding  of  media  education  and  the  pedagogical  aspects  of  computers  use 

• To  create  learning  environments  for  better  learning  with  computers  and  the  internet 

• To  manage  the  hardware  and  the  software  problems  and  to  be  aware  of  school  development  for  the 
new  media 
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One  decision  was  to  use  hypermedia  learning  environments  for  teacher  education  at  the  universities. 
That’s  the  aspect  of  distant  learning  which 

Three  working  groups  has  been  put  on  the  way:  one  for  the  educational  aspects,  one  for  the  didactic 
aspects,  and  one  for  the  information  technology  aspects.  The  paper  will  concentrate  on  the  work  of  first 
working  group. 

The  members  of  that  working  group  decided  to  develop  a modular  system  in  which  each  aspect  of  the 
theme  can  be  produced  if  the  resources  need  are  available.  Two  of  the  possible  modules  will  be  presented. 
The  first  one  is  called  ‘media  competencies’  and  offers  a concept  of  the  competencies  which  are 
necessary  to  use  the  new  media  - f e.  computers  and  the  internet  - in  a probate  way.  The  learning 
environment  offers  the  user  the  different  dimensions  of  media  competencies  - a cognitive  dimension,  a 
social  dimension,  a ethical  dimension,  an  esthetical  dimension,  an  affective  dimension,  and  the  dimension 
of  handling  media  - by  a hypermedia  text  with  different  examples  for  each  dimension. 

The  second  example  is  a case-based  learning  environment  called  ‘media  ethics’.  The  user  will  be 
confronted  with  a critical  case  of  media  content  which  could  harm  children.  The  underlying  learning 
theory  of  that  hypermedia  learning  environment  is  taken  from  Roger  Schank’s  case-based  multimedia 
learning  approach.  It  is  the  aim  of  that  hypermedia  text  to  challenge  the  user  to  find  a solution  which  is 
not  given  to  her  or  him  but  which  has  to  be  constructed  by  her  or  him. 

These  two  examples  will  show  how  that  modular  system  will  work.  The  aim  of  the  workgroups  is  to 
develop  a pool  of  modules  which  could  be  used  by  each  department  of  education.  The  process  of 
implementation  in  teacher  education  will  be  evaluated.  First  results  will  be  presented. 
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Abstract:  Most  of  these  cooperative  learning  systems  have  a static  type  that  the  learning  policy 
decided  the  beginning  of  system  development.  In  this  paper,  we  proposed  the  model  of 
coordination  policy  that  can  change  policy  dynamically  through  separating  of  coordination  and 
implementation.  TTie  proposed  coordination  policy  model  provided  interaction  structure  which 
import  two  concept  of  virtual  bus  to  cooperative  learning  system  developers  and  defined 
CPDL(Coordination  Policy  Definition  Language)  to  model  coordination  policy. 

1.  Introduction 

Through  the  improvement  of  computer  system  capabilities  and  extension  of  information  network 
communication  system,  it  is  increased  to  request  of  Computer  Supported  Cooperative  Work  (CSCW)  applications 
which  support  that  a number  of  people  globally  distributed  make  a decision  and  do  their  cooperative  work  through 
the  sharing  their  information(ACM,  1998).  Especially,  the  research  about  cooperative  learning  to  get  educative 
effectiveness  is  being  actively  performed  (Makrakis,  1998). 

Most  of  these  cooperative  learning  systems  have  a static  type  that  the  learning  policy  decided  the  beginning 
of  system  development(Sistemas  & Borges  & Santos,  1999).  There  is  a problem  that  these  static  policy  models  can’t 
change  dynamically  when  people  perform  learning  system.  Because  it  is  possible  to  static  modification,  it  includes 
problem  that  every  policy  for  cooperative  learning  should  be  decided  before  the  execution  of  learning  system.  This 
prevents  performing  of  the  original  facility  of  cooperative  learning. 

In  this  paper,  therefore,  we  propose  the  model  of  coordination  policy  that  can  change  policy  dynamically 
through  separating  of  coordination  and  implementation.  The  proposed  coordination  policy  model  provides 
interaction  structure  which  import  two  concept  of  virtual  bus  to  cooperative  learning  system  developers  and  defines 
CPDL(Coordination  Policy  Definition  Language)  to  model  coordination  policy.  Also,  using  rule  base  method  not 
procedural  method  to  control  and  process  defined  policy  with  CPDL,  it  is  possible  to  process  flexibly  about 
changing  status  dynamically. 

2.  Coordination  Policy 

It  is  the  aim  that  cooperative  learning  system  easily  provides  works  for  learning  happens  to  between  many 
learners  using  computer.  Cooperative  learning  is  coordinated  by  special  policy.  In  other  words,  cooperative  learning 
can  restrict  role  and  interaction  of  participating  learners.  So,  in  order  that  cooperative  learning  system  supports 
group  learning  effectively,  it  should  include  these  rules  in  software.  These  rules  are  coordination  policies. 
Coordination  policy  include  access  control,  concurrency  control,  floor  control,  role/session  constraints,  exception 
handling(Edwards,  1995,  Gustafsson  & Shahmehi,  1996,  Sikkel,  1997). 
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Coordination  policy  can  be  made  by  working  type  and  organization  structure  of  learning  groups,  and  that 
can  be  coordinated  by  other  policy  according  to  learning  stage.  The  development  of  cooperative  learning  system  is 
made  by  communication  of  experts,  developers,  or  users  for  defining  of  coordination  policy  generally.  These 
method  are  defined  when  system  is  developed,  this  policy  is  called  once-and-for-all  policy  which  is  applied  every 
case.  A lot  of  research  is  being  progressed  about  very  various  cooperative  learning  system  in  CSCW  during  past  1 0 
years.  Most  of  these  systems  developed  by  once-and-for-all  policy.  These  approaches  don’t  provide  flexible 
cooperative  learning  because  the  decided  policy  on  system  development  applies  to  every  case.  So,  it  is  required 
development  of  cooperative  learning  system  which  import  method  separates  system  and  policy. 


3.  Dynamic  Coordination  Policy 

In  this  chapter,  we  propose  coordination  policy  model.  A necessary  coordination  policy  on  overall 
cooperative  learning  is  separated  learning  group  policy  and  learner  policy  in  proposed  model.  And  it  is  possible  that 
dynamic  policy  changing  through  separating  system  implementation 

3.1  CPDL(Coodination  Policy  Description  Language) 

CPDL  is  a language  to  define  learning  group  policy  and  learner  policy.  It  is  used  to  define  policy  about 
overall  cooperative  learning.  As  developer  of  cooperative  learning  system  defines  policy  with  CPDL  besides  general 
application  such  as  white  board  or  tele-conferencing,  he  can  provide  cooperative  learning  applications  with  new 
policy  to  learner  without  modification  of  application  in  spite  of  dynamic  change.  Policy  with  CPDL  consist  of  part 
about  type  of  learning  group,  definition  of  sharing  application  to  use  in  learning  group,  definition  of  necessary 
events  in  learning  group  and  each  rules. 

Policy  about  learning  group  type 

The  type  of  learning  group  consists  of  title  of  learning  group,  its  role,  stage  of  learning  group,  list  of 
application,  number  of  learners,  openness  of  learning  group,  control  method  of  right  to  speak,  sorts  of  events,  and 
rules  of  learning  group.  Role  of  learning  group  and  stage  of  learning  group  operation  is  defined  List  1 . Objects  and 
events  used  group  can  composed  of  object  definition  list  and  event  definition  list.  And  numbers  of  learner  which  can 
participate  groups  are  able  to  be  defined  overall  number  of  learners  and  each  role  base  number  of  learners.  Group 
property  has  only  one  property  either  PUBLIC  or  PROTECTED.  If  group  property  is  PROTECTED,  group  can 
restrict  learners  participate  groups  using  secret  numbers.  The  control  of  right  to  speak  among  learners  groups  are 
TOKEN_CONTROL,  ROUND-ROBIN,  FREE  method. 


List  I : Group  Grammar 
Group_dcJinition  ::  = Group  identifier : 

Title  rolejist  stagejist  objectjisi 

participantNum  group _attribute  fioor_control 
eventjlist  rule_definition 
rolejist  = ROLES : idjist 
stagejist  STAGES : idjist 

object Jist  .•.  = OBJECTS : object_definlionlist  ( nothing 
event  Jist  EVENTS : event  jJefinition  f nothing 

idjist ::  — identifier,  idjist  / identifier 

obJeci_definilionlist  . ;=  object _defmition  / object_definition  object _defintionlist 

participantNum  ::  = PJi:  total_number  number _per_roles 

number _per_roles  .■.  = (identifier  =>  total _number ) + 

group _attribute  PUBLIC ( PROTECTED  : password 

floor j:ontrol  ::=  TOKEN JCONTROL  j ROUND_ROBlN / FREE 

Objects  and  event  definition 

Group  can  share  object  and  it  can  different  from  sharing  application  according  to  role.  Also  it  can  use 
object  to  different  right  though  it  shares  same  object.  Object  definition  consists  of  object  name,  class  name,  role,  and 
right  used  object. 


List  2 : Object  Grammar 

object jdefinition  ::  = OBJECT : objectname  classname  objectactivity  right 

objectname  ::  = NAME  : identifier 

classname  ::=  CLASS : identifier 

objectactivity  .;=  USING_BY : identifier 

right  w'f  R 
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Events  happen  to  interaction  process  of  learners  and  participated  groups.  And  appropriate  processes  of 
events  need  to  support  smooth  learning  activity.  The  event  type  is  consisted  of  events  source,  events  destination,  and 
field  list  consisting  of  events. 


List  3 : Event  Grammar 

event jJefinition  EVENT  identifier : simplejevent 

simple_event event_src  eventjdes  event Jieldlist 

event _src  = SRC  : contextjist 

eventjdes  ::=  DST : contextjist 

event  Jieldlist  ::=  FIELDS  .fieldlist  f nothing 

fieldlist  ::=  FIELD  . fieldlist  f nothing 


Rule  definition  for  group  activity 

It  defines  that  how  interaction  of  learners  happen  to  learning  group  and  pass  event  of  learners  process 
through  rules.  The  rules  consist  of  condition  and  behavior  list.  If  the  condition  is  satisfied,  the  descriptive  behavior  is 
performed  to  appropriate  learner.  In  the  case  of  learner,  the  target  can  be  restricted  to  itself,  and  can  be  the  learner 
performing  the  specific  roles  and  all  learners.  The  condition  is  appropriated  to  the  case  of  input  or  output  of  specific 
data. 

List  4 : Rules  using  group 
rulejdefinition  ::=  RULES : ndejist 
rulejist  ::=  rule_declare  / ndcjJeclarc  rulejist 
rulejdeclare  ::  = RULE  : rule 

rule  target  list : condition  =>  actionlist  WHEN  stage  BY  actor 

target  list  ::=  LOCAL  f ALL  f rolelist 
condition  ::  = mode  :•  datatype 
mode  ::=  IN ( OUT 

datatype  ::=  session jcommand  j mediajdata  { event jdata 

session _command  JOIN  j LEA  VE  / LIST_M EMBER  / GETjiCTJNFOf  SET_STAGE  f GET_MEMBERJNFO 

/ LlST_POSSlBLE_ROLES ! IS_MEMBER  / TOKEN JOWNER  / END_ACTIVITY 
mediajlata  ::=  connector_name 
connector_name  string 
event  jlata  ::=  identifier 
stage  ::=  ALL  j idjist 
actor  ALL  j idjist 

actionlist  .■;=  actionjdeclare  / action  jdeclare  actionlist 
actionjdeclare  ::=  string 


3.2  Two  way  coordination  policy 

Learners  participated  in  cooperative  learning  has  common  topic.  So,  coordination  policy  about  group 
affects  on  interaction  among  all  participants.  Also,  learners  interact  with  other  people  according  to  purpose  and 
policy  of  itself  However,  because  the  legacy  study  defined  only  group  coordination  policy,  individual  policies  of 
each  learner  don’t  be  considered. 

In  this  paper,  it  supports  both  group  coordination  policy  and  individual  policies  of  each  learner, 
coordination  policy  of  overall  cooperative  learning  perform  through  harmony  of  two  coordination  policy.  To  this, 
the  bus  concept  is  imported.  The  bus  consists  of  collaboration  bus  that  affects  on  learning  group  coordination  policy 
and  coordination  bus  which  affect  on  learner’s  policy.  Because  learners  participating  group  is  connected  with 
collaboration  bus,  it  is  controlled  by  collaboration  bus  learning  group  coordination  policy.  Coordination  bus  is 
connected  between  cooperative  learning  applications  like  whiteboard  and  tele-conferencing  and  private  coordinator 
for  controlling  learner  policy.  So,  because  each  application  is  connected  Coordination  bus,  it  is  controlled  by 
Coordination  bus  learner  coordination  policy. 

learning  group  coordination  policy 

Learning  group  policy  is  defined  with  CPDL  by  group  creator.  Group  policy  defined  by  group  creator  apply 
to  every  learner  participated  in  group  and  use  the  basic  policy  of  group  activity.  So,  it  has  higher  priority  than 
learner  policy  and  is  valid  as  long  as  a group  exists.  The  following  is  item  group  coordination  policy. 

• Group  Name 

• used  Role  in  group 

• used  shared  object  in  group 

• the  number  of  all  learner  and  learner  per  role  participated  in  group 

• property  of  group 

• group  title 

• type  of  right  to  speak 

• a way  of  processing  when  the  defined  events  are  generated 
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Learner  coordination  policy 

Learner  policy  defines  rule  about  transferred  data  by  system.  So,  most  of  learner  coordination  policy  is  rule 
definition. 

3,3  Coordinator  . . 

Cooperative  learning  policy  is  described  by  CPDL  and  it  is  managed  and  is  executed  by  Coordinator. 

Composition  of  Coordinator  is  illustrated  in  Figure  I.  Coordinator  consists  of  text  editor,  translator,  coordination 
manager,  working  memory,  reasoning  engine,  knowledge  base  and  actor.  Text  editor  is  a tool  that  learner  can 
describe  with  CPDL.  Translator  is  a parser  that  converts  CPDL  into  coordination  grammar  type.  Coordination 
manager  controls  policies  of  overall  cooperative  learning. 


4.  Experiments 

We  implement  cooperative  learning  system  to  show  propriety  of  the  proposed  dynamic  coordination  policy. 
And  we  show  that  possible  interaction  of  users  groups  according  to  learning  group  policy  and  learner  policy  through 
processing  of  implemented  system. 

4.1  Development  environment  and  cooperative  learning  system 

Experiment  environment  consists  of  one  teacher,  two  learners,  and  one  attendance,  and  in  this  class,  teacher 

decides  the  type  of  cooperative  learning  group  and  group  policy. 

The  overall  cooperative  learning  system  consists  of  white  board  and  video-conferencing  using  real  time  stream. 
And,  audio/video  data  of  system  and  image  data  of  whiteboard  is  controlled  flowing  according  to  group  policy. 

To  do  cooperative  learning,  first  teacher  decides  cooperative  learning  group  policy.  After  that,  if  group  is 
created,  learners  and  attendants  participates  group,  and  decide  their  policy.  The  necessary  information  which  makes 
group  is  consisted  of  group  name,  roles  to  be  used  by  group,  working  steps  of  group,  number  of  learners  and  role  to 
participate  group,  information  of  objects  to  be  used  by  learners  of  group,  control  type  of  right  to  speak,  group 
property,  definition  of  event  to  use  group,  and  rules  of  group  activity.  These  group  types  and  group  policy  are 
described  in  List  5 using  designed  CPDL  in  this  paper. 


List  5 : Croup  policy  of  education  system 
Classroom:  ‘Overview of CSCW" 

ROLES . Lecturer  Student  Attendant 

STAGES : Teaching  Question  Discussion  Reporting 

EVENTS: 

EVENT  CommandTeaching 
SRC : Lecturer 
DST : Student  Attendant 
FILEDS: 

FIELD : Content 
EVENT  Report  Result 
SRC : Student 
DST : Student 
FILEDS : 

FIELD : Content 
RULES  : 


Figure  1 : The  composition  of  Coordinator 


RULE  : 

LOCAL  : IN:-  REQUESTJOKEN  WHEN  Teaching  BY  Student 
->  "Deny  Request " 


BEST  COPY  AVAILABLE 
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RULE: 

Lecture  : IN Report  Result  WHEN  Discussion  BY  Student 
=*>  "Report  Result" 

RULE : 

ALL  : IN ComniandTeaching  WHEN  ALL  BY  Lecturer 
=*>  "Command  by  Lecture  " 

RULE : 

LOCAL  : IN sound  WHEN  Teaching  B Y Student  Attendant 
=>  "Prohibit  Speech  " 


In  the  case  of  lecture  step,  group  rules  ignore  request  of  right  to  speak  of  learners,  and  in  the  case  of 
discussion  step,  if  learners  try  to  pass  to  teacher  discussion  result,  it  describe  CPDL  which  can  receive  discussion 
result,  Also,  during  lecture,  teacher  can  pass  image  data  from  learner  or  attendance,  but  data  transmission  through 
audio  data  and  whiteboard  restrict  for  smooth  lecture, 

4.2  Group  creation 

CPDL  described  in  List  5 change  to  necessary  information  for  group  creation  through  parser.  The  necessary 
information  for  group  creation  consists  of  rules  to  be  used  group  policy  management  as  saving  knowledge  base.  The 
Parser  uses  template  such  as  List  6 to  change  CPDL  type  to  CLIPS(C  Language  Integrated  Production  System) 
(Friedman-Hill,  1998)  type. 


List  6 : Template  for  changing  CPDL  to  CLIPS 
(defiemplate  ROLELIST  (multislot  rolelist)) 
(deftemplate  STAGELIST  (multislot  stagelist)) 
(dcftemplate  OBJECTLIST  (multislot  objectlist)) 
(deftemplate  EVENTLIST  (multislot  eventlist)) 
(deftemplate  STAGES  (slot  (stage))) 
(deftemplate  RULE  (slot  tagetlist) 

(slot  mode) 

(slot  datatype) 

(slot  WHEN) 

(multislot  by)) 


The  contents  of  CPDL  is  changed  by  parser  is  transmitted  manager  handling  a portion  and  then,  it  used. 
List  7 are  rules  to  connect  definition  rules  with  CPDL  with  java  method.  This  rule  becomes  of  knowledge  base  of 
Jess,  it  is  applied  appropriate  rule  about  several  accident  happen  during  group  activity. 


List  7 : The  changed  rule  of  CLIPS  type 
(defrule  Group_RULEI 

(RULE  (targetlist  LOCAL)(mode  IN)(datatype  REQUEST_TOKEN) 

(WHEN  Teaching)(By  Student)) 

(STAGES  (stage  Teaching)) 

= > (printout  t "Deny  Request " crlf ) 

(call  RequestHandling  LOCAL  IN  REQUESTJTOKEN  Teaching  Student 
Deny_Request) ) 

(defrule  Group_RULE2 

(RULE  (targetlist  A LL)(mode  IN)(datatype  CommandTeaching) 

(WHEN  A LL)  (By  Lecturer)) 

(STAGES  (stage  ALL) 

= > (printout  t "Command  by  Lecture" crlf) 

(call  RequestHandling  ALL  IN  CommandTeaching  ALL  Lecturer  Command  by  Lecture)) 
(defrule  Group_RULE3 

(RULE  (targetlist  LOCAL)(mode  IN) (datatype  sound) 

(WHEN  Teaching)(By  Student  Attendant)) 

(STAGES  (stage  Teaching) 

- > (printout  t "Prohibit  sound"  crlf) 

(call  RequestHandling  LOCAL  IN  sound  Teaching  Student  Attendant  Prohibit  sound)) 


4.3  Participation  of  learner 

Learner  can  owns  policy  on  group  activity  and  defines  owns  group  activity  in  addition  to  group  policy 
defined  by  group  chairman.  When  the  change  of  event  is  receiving  to  a teacher,  Leamer-A’s  System  makes  the 
overall  application  step  change  by  generating  “Change  Mode”  message.  Also,  if  received  data  from  learner  or 
attendant  in  question  time.  System  should  not  show  the  received  data  and  if  don’t  show  the  received  data  on 
whiteboard  in  reporting  the  result  of  conversation,  leamer-A  describes  like  the  following  list, 

RULES: 

RULE: 

LOCAL  : IN :-  CommandTeaching  WHEN  ALL  BY  Lecturer 
=*>  "Change  Mode” 
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RULE : 

LOCAL  : IN whiteboard  WHEN  Question  BY  Student  Attendant 
=>  “Prohibit  whiteboard” 

RULE  : 

LOCAL  : IN . - whiteboard  WHEN  Report  BY  Student  Attendant 
=>  “Prohibit  whiteboard” 


Leamer-B  like  leamer-A  changes  the  application  step  on  receiving  step  change  event  in  overall  step.  Also,  if 
received  the  voice  data  from  learner  or  attendant  in  question  time,  it  describes  with  CPDL  like  the  following  list  for 
no  receiving  the  receiving  data, 

RULES: 

RULE  : 

LOCAL  : IN CommandTeaching  WHEN  ALL  BY  Lecturer 
->  “Change Mode” 

RULE  : 

LOCAL  : IN:-  sound  WHEN  QUESTION  BY  Student  Attendant 
= > “Prohibit  sound  " 

CPDL  of  learner  A and  B convert  CLIPS  using  template  like  group  CPDL  and  it  saves  in  Knowledge  Base.  If  the 
condition  is  satisfied,  Reasoning  Engine  transmits  message,  and  makes  it  launch  java  object  connected  message. 


4.4  Policy  change  of  teacher 

In  the  case  of  adding  new  policy  after  it  decided  policy,  System  can  execute  new  policy  Just  compiling  new 
mles  and  adding  to  knowledge  base  without  recompiling  overall  policy.  For  example,  if  group  stage  is  not 
“Teaching”  group  stage,  and  try  to  prohibit  access  of  other  users,  it  add  the  following  CPDL 


RULE  ; 

LOCAL  : IN :- JOIN  WHEN  Question  BY  Student  Attendant 
= >“ Prohibit  JOIN” 

RULE : 

LOCAL  : IN  .-JOIN  WHEN  Discussion  BY  Student  Attendant 
“Prohibit  JOIN” 

RULE  : 

LOCAL  : IN:- JOIN  WHEN  Reporting  BY  Student  Attendant 
= >“ Prohibit  JOIN” 

5.  Conclusion 

In  this  paper,  we  proposed  the  model  of  coordination  policy  that  can  change  policy  dynamically  through 
separating  of  coordination  and  implementation.  The  proposed  coordination  policy  model  provided  interaction 
structure  which  import  two  concept  of  virtual  bus  to  cooperative  learning  system  developers  and  defined 
CPDL(Coordination  Policy  Definition  Language)  to  model  coordination  policy.  Also,  using  rule  base  method  not 
procedural  method  to  control  and  process  defined  policy  with  CPDL,  it  was  possible  to  process  flexibly  about 
changing  status  dynamically.  The  proposed  coordination  policy  model  solves  problem  of  static  policy  model,  which 
it  can’t  change  policies  dynamically  on  the  execution  of  learning  system,  and  can  provide  various  learning  step  and 
learning  way  through  changing  coordination  policy  dynamically. 
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Abstract:  This  paper  discusses  the  growing  importance  of  teaching  webs  - both  as  an  augmentation 
aid  to  conventional  teaching  and  as  a replacement  for  'stand  and  deliver  courses.  The 
development  of  teaching  webs  is  discussed  within  the  framework  of  a case  study  relating  to  the 
development  of  an  intranet/lntemet  facility  to  support  the  teaching  of  a two-semester,  final-year, 
BSc  degree  module  in  human-computer  interaction.  Some  of  the  advantages,  problems  and  pitfalls 
of  this  approach  to  educational  delivery  are  briefly  discussed. 


1.  Introduction 

Increasingly,  the  Internet  and  other  related  technologies  such  as  intranets  and  extranets  are  being  used  as  support 
aids  to  facilitate  the  delivery  of  teaching  and  learning  materials  (Hopkins,  1998;  French  et  al,  1999).  Indeed,  web- 
based  training  (WBT)  and  Internet-based  training  (IBT)  are  now  rapidly  becoming  two  of  the  most  popular 
mechanisms  for  making  training  materials  available  within  both  academic  and  commercial  organisations  (Steed, 
1999;  Horton,  2000).  Tliere  are  two  main  ways  in  which  WBT  and  IBT  are  currently  being  used.  First,  as  an 
augmentation  aid  in  order  to  facilitate  the  delivery  of  conventional  courses  (Gillham  et  al,  1999;  Hall  and  Dalgleish, 
1 999).  Second,  as  a complete  and  integrated  solution  to  the  presentation  of  courses  that  are  to  be  used  by  students  in 
a self-study  environment  or  in  an  open  learning  fashion  (Barker,  1999a;  1999b;  Duggleby,  2000).  This  latter 
technique  forms  the  basis  of  many  of  the  new  Web-based  approaches  to  distance  learning  using  the  principles  of 
‘electronic  classrooms’  and  ‘virtual  university’  environments  (Whittington  and  Sclater,  1998;  Barker,  1998). 
Although  these  are  important,  this  paper  is  primarily  concerned  with  the  first  of  these  two  approaches.  It  considers 
the  various  ways  in  which  a teaching  web  - running  on  an  Internet  and/or  an  intranet  server  - might  be  used  in  order 
to  improve  the  quality  of  the  educational  experiences  available  to  students  studying  within  a conventional  university 
environment  and  who  are  exposed  to  conventional  'stand  and  deliver'  lecture  courses  as  their  main  mechanism  of 
information  delivery. 

Internet-based  technologies  can  be  used  for  a variety  of  purposes  within  an  educational  setting.  Typical 
uses  include:  making  reference  material  available,  providing  access  to  course  documentation  and  the  collection  of 
student  demographic  data.  In  contrast  to  these  basic  information  provision  and  data  acquisition  roles,  the  purpose  of 
a ‘teaching  web’  is  to  facilitate  the  development  of  knowledge  and  skills  within  the  user  population  for  which  it  is 
designed.  As  well  as  achieving  knowledge  and  skill  transfer,  teaching  webs  are  intended  to  monitor,  assess  and 
record  students’  progress  and  performance  - giving  appropriate  feedback  both  to  students  and  course  administrators. 
As  well  as  providing  access  to  teaching  and  learning  materials,  teaching  webs  must  therefore  be  closely  integrated 
with  assessment  tools  and  databases  that  can  keep  records  of  performance  data. 

Creating  teaching  webs  is  a much  more  difficult  task  than  just  authoring  some  informational  pages  and 
mounting  them  on  an  Internet  or  intranet  server.  Because  of  the  more  complex  tasks  that  are  involved,  this  paper 
describes  and  discusses  some  of  the  principles  involved  in  creating  a teaching  web.  It  illustrates  these  principles  by 
means  of  a case  study  that  describes  a dynamic  teaching  web  facility  that  we  have  recently  built  in  order  to  support 
the  teaching  of  the  subject  of  human-computer  interaction  (HCI).  As  well  as  describing  the  operational  system,  the 
paper  outlines  some  of  the  advantages  of  using  this  approach  and  identifies  a number  of  problems  and  pitfalls  that 
we  have  encountered. 
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2.  Creating  Teaching  Webs 


Before  a teaching  web  is  constructed  it  is  necessary  to  make  a number  of  important  decisions  relating  to  how,  if  at 
all,  the  development  process  should  proceed.  It  is  also  necessary  to  decide  upon  the  form  the  web  should  take,  its 
content,  structure  and  the  way  in  which  it  should  be  produced.  Some  of  the  more  important  initial  questions  that 
need  to  be  answered  therefore  include: 

Can  the  development  of  a web  be  justified? 

What  institutional  infrastructures  currently  exist  to  support  a web? 

What  parts  of  an  existing  course,  if  any,  should  be  made  available  on  the  web? 

What  type  of  platform  is  to  be  used? 

What  financial,  human  and  technical  resources  will  be  available? 

What  materials  currently  exist? 

Should  these  be  imported  into  the  web  facility? 

Can  they  be  imported  into  the  new  system? 

What  relevant  ‘off-the-shelf  web-based  teaching  materials  currently  exist? 

Can  these  be  easily  imported  into  the  new  system? 

What  new  materials  need  to  be  developed? 

What  types  of  facility  should  they  include? 

What  development  tools  are  available? 

What  will  be  the  roles  of  teaching  staff? 

How  will  students  access  the  course  materials? 

The  questions  listed  above  are  presented  in  order  to  stimulate  thoughts  and  ideas  about  what,  in  any  given  situation, 
has  to  be  done  in  order  to  establish  a successful  operational  teaching  web  facility.  These  questions  will  not  be 
discussed  and  debated  in  any  depth  here.  Instead,  the  intention  of  this  paper  is  to  concentrate  on  just  three  basic 
areas:  the  hardware  support  that  is  needed;  the  range  of  software  development  tools  that  are  required;  and  the 
organisational  infrastructures  that  need  to  be  in  place  in  order  to  facilitate  the  use  of  a teaching  web.  Each  of  these 
issues  is  briefly  discussed  below. 


Hardware  Resources 

The  important  types  of  hardware  resource  that  need  to  be  decided  upon  include:  the  nature  of  the  servers  that  are  to 
be  employed;  the  network  infrastructure  that  has  to  be  in  place;  the  types  of  computer  terminal  that  will  be  used  to 
gain  access  to  the  web;  and,  whether  or  not  remote  access  via  modems  will  be  supported.  In  the  HCl  case  study  that 
is  presented  in  the  following  section,  an  in-house  server  farm  was  used  to  host  the  main  part  of  the  teaching  web. 
However,  some  use  was  also  made  of  external  Information  Service  Providers  - as  and  when  this  became  necessary 
for  certain  parts  of  the  course.  A campus-wide  Novell  network  was  employed  as  a means  of  providing  access  to  the 
servers.  Both  desktop  computers  (local  and  remote)  and  laptop  computers  were  used  (by  students  and  staff)  to 
access  the  network  and  the  teaching  webs  held  on  the  servers.  For  remote  access,  the  public  switched  network  was 
employed  - this  required  modems  at  both  ends  of  the  connection. 


Software  Resources 

Prior  to  the  advent  of  sophisticated  authoring  packages  (such  as  HotMetal  Pro,  FrontPage  and  DreamWeaver),  the 
creation  of  learning  materials  in  HTML  format  for  use  on  a web  was  quite  a labour-intensive  process.  Although  it 
was  not  difficult,  it  was  both  tedious  and  prone  to  errors.  Nowadays,  however,  it  is  a relatively  easy  matter  to  create 
learning  materials  as  Word  Documents  and  then  import  them  into  an  appropriate  HTML  development  system  for 
integration  and  linking  with  other  relevant  resources.  In  addition,  the  wide  range  of  browser  plug-ins  and  add-ons 
that  is  now  available  means  that  interactive  learning  materials  can  still  be  developed  in  particular  development 
environments  (such  as  Delphi,  ToolBook,  Director,  and  so  on)  and  then  ‘run’  by  the  browser.  Of  course,  the 
availability  of  JavaScript  and  Java  programming  tools  means  that  enhanced  interactivity  can  be  built  into  web-based 
materials  without  placing  too  great  an  overhead  on  the  server  that  is  used  to  deliver  the  course  materials.  In 
situations  where  special  types  of  interactivity  and  data  storage  mechanisms  have  to  be  implemented,  server-side 
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software  may  need  to  be  developed  using  languages  such  as  Perl  or  C.  For  the  HCI  case  study  that  is  discussed  in 
the  next  section,  all  of  these  techniques  were  used.  However,  apart  from  the  browser  software  (both  Internet 
Explorer  and  Netscape  Navigator  were  used),  the  three  packages  that  were  most  heavily  used  were  SoftQuad’s 
HotMetal  Pro,  Frontpage  and  Microsoft’s  Word  for  Windows. 

Organisational  Infrastructures 

In  addition  to  the  technical  infrastructures  mentioned  above,  a number  of  organisational  infrastructures  have  to  exist 
in  order  to  facilitate  the  effective  use  of  teaching  webs.  Undoubtedly  the  most  important  of  these  relate  to  the 
administrative  infrastructure  and  the  organisation  of  the  teaching  and  learning  environment.  One  of  the  main 
administrative  issues  that  we  had  to  consider  in  the  HCI  teaching  web  was  the  way  in  which  students’  "attendance^ 
and  assessment  data  were  collected  and  managed  - and  subsequently  integrated  into  overall  organisation- wide 
student  performance  measures.  As  is  discussed  later,  this  was  achieved  through  the  use  of  electronic  registers  and 
assessment  mechanisms;  the  results  being  stored  and  processed  centrally  on  one  of  the  servers.  An  important 
consideration  from  the  perspective  of  the  teaching  and  learning  environment  was  the  flexible  way  in  which  the 
teaching  web  allowed  the  integration  of  new  and  old  mechanisms  of  course  delivery.  As  is  discussed  later,  although 
lectures  were  given,  attendance  at  them  was  not  compulsory  since  all  of  the  HCI  course  was  available  on  and  could 
be  accessed  via  the  teaching  web. 


3.  Case  Study  - The  HCI  Teaching  Web 

This  section  contains  a short  description  of  a teaching  web  that  we  have  developed  to  support  the  teaching  and  self- 
supported  study  of  material  relating  to  the  subject  of  human-computer  interaction.  The  web  has  been  operational  for 
about  three  years.  Despite  this,  we  do  not  feel  that  it  is  yet  complete!  We  continue  to  modify  and  fine-tune  its 
content  and  structure  in  order  to  improve  its  performance  and  its  appeal  to  students.  In  the  remainder  of  this  section, 
the  following  major  perspectives  are  briefly  discussed:  web  structure;  resources  available;  notice  boards  and 
schedules;  monitoring  and  assessment;  technical  and  academic  support;  data  files;  providing  access  to  additional 
support  materials;  future  developments;  and  potential  application  areas. 


Web  Structure 

The  HCI  teaching  web  is  organised  primarily  as  a hierarchical  structure  embedded  within  which  are  numerous  cross- 
references  that  are  implemented  as  hyperlinks.  From  the  options  available  on  the  home  page  (see  table  1),  users  can 
navigate  to  those  parts  of  the  web  that  are  of  particular  interest  to  them  at  any  given  time.  This  may  involve 
browsing  the  online  material  associated  with  any  of  the  40  lectures  or  participating  (in  self-study  mode)  in  one  of  the 
20  practical  classes.  The  exercises  contained  in  these  are  based  on  the  principle  of  active  learning.  This  involves 
students  having  to  do  things  (relating  to  HCI)  for  themselves  - sometimes  individually  and  sometimes  in  groups. 
Each  practical  class  usually  has  associated  with  it  appropriate  support  data,  a hints  file  and  a worked  solutions  file. 
As  is  discussed  later,  students  are  required  to  submit  an  electronic  register  for  each  of  the  practicals  that  they 
attempt;  they  must  also  fill  in  an  electronic  checklist  for  every  practical  that  they  complete.  The  home  page  for  the 
HCI  Teaching  Web  is  available  for  inspection  at  the  following  Internet  address: 

http://www-scm.  tees.  ac.  uk/users/philip.  barker/hcihome.  htm 

Because  the  bulk  of  the  HCI  material  is  made  available  using  an  in-house  intranet  facility,  the  links  from  the  above 
Web  address  are  all  password  protected  so  that  only  authorised  users  can  gain  access  to  the  underlying  resources. 


Resources  Available 

The  nature  of  the  resources  available  on  the  teaching  web  is  reflected  in  the  "contents'  list  that  appears  on  the  home 
page  when  students  access  the  web  using  their  browsers.  The  options  currently  available  are  listed  in  table  1 . 
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A Riddle  - Why  Study  (HCl)? 

Aims  and  Objectives  of  the  Course 

Course  Description 

Course  Content 

Computer  Accounts 

Lecture  Material 

Practical  Classes 

The  Electronic  Register,  Checklists  and  Schedules 

Books  on  HCl 

Other  Information  Sources 

The  HCl  Notice  Board 

Past  Examination  Papers 

Research  Opportunities  in  HCl 


Table  1:  The  Main  Options  on  the  Home  Page  of  the  HCl  Teaching  Web 


The  two  most  heavily  used  options  in  this  list  are  the  ‘Practical  Classes ' and  the  ‘Lecture  Material Each  of  these 
options  leads  to  a screen-based,  interactive,  rectilinear  matrix  of  options  ( ‘Practical  I ’ through  ‘Practical  20 ' and 
‘Lecture  1 ' through  ‘Lecture  40  \ respectively)  that  give  students  access  to  the  underlying  course  material.  This  can 
be  browsed,  printed  or  copied  to  students’  individual  machines  for  later  use.  In  principle,  all  of  the  material  that 
students  need  to  study  the  HCl  course  is  available  in  electronic  form  from  the  teaching  web. 


Notice  Boards  and  Schedules 

One  of  the  reasons  for  introducing  the  HCl  teaching  web  was  to  encourage  greater  degrees  of  independent,  self- 
managed  learning.  Students  are  therefore  encouraged  to  be  flexible  with  respect  to  how  and  when  they  study. 
Naturally,  although  flexibility  is  encouraged,  some  mechanisms  need  to  be  available  in  order  to  ensure  that  students 
‘move  forward’  with  respect  to  progressing  through  the  course.  Two  embedded  web-based  bulletin  boards  are 
therefore  used  to  provide  sources  of  ‘motivation’  for  students.  The  first  of  these  is  the  'HCl  Notice  Board',  this  is 
used  to  post  both  general  messages  and  news  items  that  are  relevant  to  the  course.  The  second  online  bulletin  board 
is  the  'Schedules  Notice  Board' . This  is  used  as  a means  of  giving  students  details  of  the  ‘recommended’  start  dates 
and  completion  dates  for  Practical  Classes  and  Lecture  Units. 


Monitoring  and  Assessment 

As  was  mentioned  previously,  the  HCl  teaching  web  provides  two  basic  automated  mechanisms  for  monitoring  what 
students  are  doing  as  they  progress  through  the  course.  First,  there  is  the  electronic  register  that  keeps  details  of 
students’  ‘attendance’  at  the  ‘virtual’  laboratory  classes.  Second,  there  are  the  online  checklists  that  students 
complete  at  the  end  of  each  laboratory  session.  Each  of  these  monitoring  tools  is  implemented  as  a HTML  form. 
Form  handlers  on  the  server  are  then  used  to  process  the  data  that  is  submitted  by  students.  The  form  handlers  are 
fairly  standard  CGI  scripts  that  are  written  in  Perl.  The  data  that  is  obtained  from  these  monitoring  and  assessment 
processes  is  extracted  and  imported  into  a data  base  that  can  be  used  (in  conjunction  with  dynamic  HTML 
programming  techniques)  to  generate  feedback  both  for  course  administrators  and  for  students. 


Technical  and  Academic  Support 

Students  using  the  HCl  teaching  web  are  provided  with  a range  of  academic  and  technical  support  facilities  that  they 
can  call  upon  as  and  when  they  need  to.  Two  basic  types  of  support  mechanism  are  used:  face-to-face  and  online 
(Duggleby,  2000).  Face-to-face  support  is  provided  both  by  means  of  a help-desk  facility  (that  is  manned  during 
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normal  working  hours)  and  through  a series  of  regular  time-tabled  'co??7e  if  you  need  to\  open-access  laboratory 
sessions.  Online  support  is  provided  both  through  email  and,  at  particular  times  of  day,  via  online  chat  facilities. 


Data  Files 

An  important  aspect  of  the  HCI  work  that  students  undertake  during  the  course  is  the  development  of  practical 
competencies  with  respect  to  various  types  of  data  manipulation  and  data  analysis.  In  order  to  facilitate  this,  the 
teaching  web  facility  makes  available  a large  data  bank  of  material  of  various  sorts  that  students  can  use  as  part  of 
their  practical  exercises.  Typical  examples  of  the  material  that  is  held  on  the  underlying  intranet  include:  raw 
questionnaire  data,  spreadsheet  files  and  experimental  data  from  HCI  experiments  (such  as  reading  rates  and 
reaction  times  for  various  visual  and  audible  stimuli). 


Additional  Support  Material 

The  intranet  facility  that  is  associated  with  the  HCI  teaching  web  provides  a powerful  mechanism  for  making 
available  a wide  range  of  ancillary  course  support  materials.  Typical  examples  of  the  types  of  resource  we  provide 
to  students  include:  electronic  documents  (relating  to  particular  course  topics)  that  are  hyperlinked  to  other  Internet 
resources;  image  sequences  that  are  chosen  to  illustrate  particular  end-user  interfaces  to  various  consumer  products 
(such  as  MP3  audio  players,  digital  radios  and  mobile  telephones);  and  hyperlinked  reference  lists  for  students  to 
follow  up  as  part  of  their  self-managed  study  activities. 


Future  Developments 

Although  the  HCI  teaching  web  has  now  been  operational  for  over  three  years,  there  are  still  a number  of  features 
and  improvements  that  we  feel  we  would  like  to  make  to  the  system.  These  fall  into  short-term  and  long-term 
development  goals.  The  two  most  important  short-term  future  developments  that  we  would  like  to  make  are:  (1)  the 
addition  of  a ‘user-friendly’  help  system  that  would  provide  new  users  with  general  help,  guided  walk-throughs  and 
advice  relating  to  using  the  system;  and  (2)  the  provision  of  a topic  index  and  global  web  search  facility  to  enable 
students  to  search  the  teaching  web  in  order  to  locate  particular  items  of  interest  to  them.  Some  prototype  software 
to  test  out  each  of  these  requirements  has  already  been  written  and  evaluated.  From  the  perspective  of  longer-term 
developments,  the  most  important  major  change  that  we  would  like  to  make  would  be  to  incorporate  XML  as  the 
basic  mark-up  language  (Eckstein,  1999).  We  believe  this  is  important  because  it  will  facilitate  the  preparation  of 
the  catalogues  and  indexes  that  we  need  in  order  to  implement  efficient  and  effective  searching  of  the  teaching  web. 
In  our  view,  the  use  of  XML  will  also  simplify  data  exchange  with  respect  to  the  administrative  tasks  associated 
with  monitoring  and  assessment  - and  subsequent  reporting  of  feedback  to  both  students  and  teaching  staff. 


Potential  Application  Areas 

We  believe  that  during  the  next  few  years  there  will  be  tremendous  opportunities  for  using  teaching  webs  in  a wide 
variety  of  different  contexts  both  within  conventional  educational  environments  and  for  the  support  of  lifelong 
learning  and  continuing  professional  development  (Barker,  1999b;  Barker  et  al,  1998).  One  important  application 
area  that  we  are  currently  exploring  is  the  utility  of  such  webs  within  the  context  of  patient  education  within  hospital 
and  health-care  environments  (Kaur,  1999;  Barker  and  van  Schaik,  2000).  Within  this  project  we  have  obtained 
some  interesting  results  and  we  are  very  optimistic  about  future  developments  in  this  and  other  related  areas. 


4.  Conclusions 

There  have  been  a number  of  advantages  associated  with  the  introduction  of  a teaching  web  into  the  HCI  course 
described  in  this  paper  - there  have  also  been  a number  of  ‘problems  and  pitfalls’ ! One  of  the  major  advantages  is 
the  fact  that  because  all  the  course  materials  are  available  online  in  electronic  form,  students  can  gain  access  to  them 
at  any  time  they  wish  to.  Furthermore,  because  the  intranet  facility  can  be  accessed  over  the  Internet,  both  local 
(campus-based)  and  remote  (from  home)  access  is  available.  Another  important  feature  of  the  teaching  web  is  the 
support  it  offers  for  self-managed  study  and  the  realisation  of  the  constructivist  approach  to  learning.  Students  can 
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download  copies  of  the  HCI  teaching  web  and  store  it  on  their  own  computers  - usually  on  a 100-megabyte  Iomega 
ZIP  disk.  They  can  then  modify  and  extend  their  personal  version  of  the  web  in  ways  that  are  appropriate  to  their 
own  individual  needs. 

In  our  view,  the  advantages  associated  with  using  teaching  webs  are  an  ongoing  motivation  for  their  further 
future  development.  However,  there  are  a number  of  drawbacks  associated  with  their  use  and  implementation. 
From  the  perspective  of  a staff  member,  developing  ‘quality’  web  pages  for  use  in  teaching  webs  is  undoubtedly  a 
very  time  consuming  process.  Naturally,  this  can  substantially  detract  from  an  academic’s  primary  role  as  a teacher 
and  researcher.  It  is  therefore  unfair  to  expect  the  ‘average’  academic  to  develop  the  specialist  skills  needed  to 
produce  such  pages.  Institutional  support  with  respect  to  HTML  and  multimedia  resource  development  is  therefore 
a necessary  pre-requisite  if  this  approach  to  teaching  and  learning  is  to  succeed.  Admittedly,  there  are  automation 
tools  available  (such  as  Microsoft’s  FrontPage  and  MacroMedia’s  DreamWeaver)  that  can  help  the  development 
process.  Unfortunately,  however,  there  are  also  many  pitfalls  associated  with  the  use  of  these  - if  the  intent  of  a 
development  project  is  to  produce  highly  portable  code  that  is  independent  of  any  particular  manufacturer’s 
development  environment.  Of  course,  there  are  also  numerous  security  issues  that  need  to  be  addressed  - for 
example,  the  use  of  CGI  scripts  may  be  allowed  on  an  in-house  intranet  server  but  is  a facility  that  is  often  not 
available  on  many  Internet  servers.  This  again  can  create  problems  with  respect  to  having  a system  that  is  totally 
portable  and  transparent  to  its  users. 

We  still  have  many  technical  problems  to  solve  in  our  future  work.  Despite  this,  we  strongly  believe  that 
the  benefits  associated  with  using  an  interactive  teaching  web  (to  support  the  ideas  underlying  constructivist  learning 
principles)  far  outweigh  the  problems  and  pitfalls  that  we  have  encountered  with  respect  to  the  realisation  of  this 
approach  to  teaching  and  learning.  We  are  therefore  very  optimistic  about  the  future  development  plans  that  we 
have  with  respect  to  enhancing  the  quality  of  the  learning  environments  that  we  offer  to  our  students  through  the  use 
of  teaching  webs. 
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Abstract:  This  paper  describes  the  development  of  an  internet-based  3-D  service  system  for  students.  Starting  as  an 
information  service  for  just  one  major  field  of  study  (business  education),  the  anarchic  implementation  process 
permanently  enlarged  the  system  boundaries.  From  the  viewpoint  of  technical  rationality,  the  system  design  of  the 
reported  project  was  badly  planned  and  designed.  It  is  argued,  that  following  the  directions  of  a technical  engineer 
approach  would  have  resulted  in  a stop  of  the  project.  In  contrast  to  the  traditional  technical  solution,  we  will,  based 
on  lessons  learned  from  organzition  theories,  show  that  the  logic  of  bricolage  (a  mental  model  of  handcraft)  is  an 
adequate  metaphor  for  describing  and  maybe  even  for  planning  design  processes. 


The  Project 

The  project  started  with  the  idea  to  offer  all  relevant  information  services  for  Business  Education  students  via  the 
internet  with  a 3-D  interface  (Baumgartner  & Wydra  1999).  This  idea  was  motivated  by  four  basic  cognitive 
principles: 

• Linking  and  remembering  relevant  study  information:  Empirical  studies  in  Cognitive  Science  demonstrated  that 
information  embedded  in  space  or  linked  with  spacial  structures  is  easier  to  remember  and  learn  (eg  Gunther 
1998).  The  prototype  of  this  idea  can  be  explored  at:  http://iol  1 .uibk.ac.at/wipaed/. 

• Building  up  mental  models  of  the  study  organization  procedures:  Moving  around  in  a 3-D  virtual  representation 
helps  to  build  up  mental  models  of  the  correlated  and  interconnected  information  which  is  relevant  for 
mastering  the  situations  in  question.  From  the  constructionist  point  of  view,  it  is  important  to  interact  with  the 
(virtual)  environment  in  order  to  reach  viable  solutions  for  problems.  Instead  of  passively  looking  up  study 
problems  like  “what  do  I have  to  do  to  register  my  thesis”  in  a database  of  Frequently  Asked  Questions  (FAQs) 
we  could  implement  a robot  avatar  which  guides  the  students  around  and  shows  them  where  to  pick  up  the 
relevant  information,  forms  etc. 

• Building  up  social  virtual  communities:  In  our  faculty  we  have  a lot  of  students  who  spend  many  hours 
travelling  to  and  from  Innsbruck.  For  this  reason  we  want  to  provide  our  students  with  an  online  service  where 
they  can  get  relevant  information  without  having  to  come  to  the  university  in  person.  Clearly  for  faster  access 
we  have  implemented  this  information  in  normal  2-D  HTML.  Due  to  a 3-D  multi-user  world  our  students 
cannot  only  communicate  in  real-time  to  exchange  their  views,  but  this  also  leads  to  closer  contacts  among  the 
students.  Tools  like  the  avatar  studio  by  blaxxun  (http://www.blaxxun.de)  can  provide  the  necessary  personal 
information  to  facilitate  the  process  of  building  a community  of  students. 
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In  contrast  to  the  original  idea  our  system  developed  from  a student  information  system  to  an  overall  system  not 
only  for  students  but  also  for  lecturers  and  secretaries.  What  went  wrong?  To  understand  this  organizational 
trajectory  we  firstly  have  to  examine  in  detail  the  different  changes  in  the  design  process  and  secondly  to  generalize 
this  process  in  the  light  of  organizational  theory. 

History  of  the  project 

February  1998:  The  new  professor  (Peter),  who  is  scheduled  to  get  his  professorship  in  Business  Education  in  the 
winter  term  98/99  is  launching  the  idea  of  a 3-D  student  information  system.  An  external  graphic  designer  (Detlef, 
http://www.wydra.de)  has  been  contracted.  Additionally  another  lecturer  brought  in  the  context  of  the  Business 
Education  program. 

During  this  early  stage  it  was  realized  that  for  the  planned  project  different  departments  had  to  work  together. 
Impossible!  - There  is  simply  nobody  who  has  got  the  time,  who  is  willing  and/or  has  enough  power  to  bring  the 
different  departments  together  (former  efforts  already  failed).  So  the  project  was  changed  from  a Business  Education 
program  to  a department  project. 

March  1998:  The  department  planned  to  merge  with  another  department.  Lecturers  requested  to  discuss  the  quite 
different  business  processes  at  department  level.  But  at  this  level  the  secretaries  opposed  to  it.  Therefore:  nothing 
happened,  but  the  usage  of  a database,  developed  from  a lecturer  from  the  other  department,  to  provide  the 
administration  of  the  department,  was  at  this  point  envisioned. 

April  1998:  The  development  of  the  homepage  was  assigned  to  a lecturer  (Stefan),  an  external  designer 
(http://www.ampuls.at)  was  contracted.  A coordinating  meeting  (3  involved  lecturers  and  the  homepage  designer) 
should  clarify  the  relation  between  the  3-D  student  information  system,  the  homepage,  the  database  and  an  existing 
central  www-database  of  course  information  implemented  by  and  fitted  to  the  requirements  of  the  central  EDP- 
department. 

The  following  was  decided  withing  two  hours: 

• to  connect  the  administration  database  with  the  central  course  database  and  to  use  the  database  with  small 
changes  for  the  whole  department. 

• to  use  the  central  course  database  to  provide  the  homepage  with  data  of  the  courses.  Because  of  financial 
reasons,  we  decided  to  connect  these  two  systems  with  cgi-scripts.  It  was  a genera!  agreement  that  a database 
solution  would  be  more  efficient  for  the  whole  system,  but  there  was  no  budget  available. 

• that  the  3-D  information  system  should  be  based  on  the  existing  administration  database,  on  the  homepage  and 
course  database.  Only  additional  data  should  be  captured  in  a separately  linked  database. 

Finally,  in  a short  discussion  all  existing,  prospective  and  isolated  planned  systems  were  canned.  Only  small 
corrections  were  discussed,  interfaces  were  defined  on  an  abstract  level.  The  3-D  information  system  should  build 
on  a various  mixed  systems. 

September  99:  The  homepage  is  finished,  the  graphic  designer  and  a lecturer  meet  to  make  a detailed  plan  for  the 
info  system.  3 points  were  discussed:  1.  A lot  of  data  from  different  systems  must  be  imported  in  the  database, 
because  we  need  them  for  different  inquiries  from  the  database.  2.  Secretaries  and  lecturers  would  not  be  able  to  see 
where  to  put  in  the  data,  due  to  a lack  of  a logic  system.  3.  Through  a simple  database  important  data  for 
administration  (eg  marks)  have  to  be  brought  in  the  system  via  different  sets  of  www- forms  (from  the  homepage,  eg 
consultation  hours,  from  the  central  database,  eg  schedule  for  the  courses  and  from  the  3-D  info  system,  eg. 
information  for  a special  course). 

At  the  same  time,  the  secretaries  announced  that  they  would  not  support  the  much  less  complex  database  for 
administration  purposes. 

Crises  hits:  two  days  later  it  was  clear  financial  support  (more  than  the  homepage  budget!)  was  needed  to  develop  an 
integrated  system  with  one  database,  and  the  system  had  to  meet  the  needs  of  secretaries  and  lecturers  too. 

November  (plan!):  7 months  after  the  first  coordinating  meeting  a second  one  will  take  place.  The  secretaries  and 
one  professor  will  be  in  the  meeting  too.  Business  processes  will  be  discussed  and  fixed  in  detail.  In  addition  to  the 
central  course  one  integrated  database  for  the  department  will  be  modelled.  This  database  will  be  handled  only  via  a 
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www-surface.  A new  collaboration  with  another  department  and  new  budgeting  rules  make  it  possible  to  get  a 
budget  for  this. 

The  result  will  be  an  integrated  system  with  two  databases  (central  and  department)  with  two  sets  of  www-forms.  It 
is  a system  for  a department  and  its  students  (some  of  them  study  Business  Education)  and  not  a system  for  the 
Business  Education  program,  which  would  need  a lot  of  information  from  other  departments.  Attention  is  not  only 
drawn  on  students,  but  also  on  the  lecturers  and  secretaries.  The  difference  between  'outside*  (infos  for  the  students) 
and  ’inside’  (the  administration  at  department  and  lecturer  level)  vanished. 


the  necessity  of  systematically  analysing  the  business  processes  and  the  need  of  disobey  the 
systemic  approach 

So  what?  In  every  text  book  (eg.  Scheer  1998)  we  will  find  the  recommendation  that  first  of  all  we  have  to  analyse 
systemic  (on  a gestalt  level)  and  systemtically  all  the  different  flows  of  the  business  process.  This  means  you  have  to 
identify  different  entities  Gobs,  departments,  ...)  and  their  relations  and  describe  in  a very  detailled  and  formal  way 
the  task-,  performance  and  information  flows.  The  description  of  the  organization  and  the  different  flows  should 
result  in  integrated,  generalized  business  process  models  that  do  not  depend  on  the  actual  human  and  incidental 
characteristics  of  the  processes.  When  these  tasks  are  finished  the  system  engineer  can  generate  a respository,  a data- 
, function-,  process-,  organization-,  performance  and  control  system.  For  the  purpose  of  analysis  we  will  call  this 
approach  of  system  engineers  the  mental  model  of  technical  rationality. 

No  question  - our  strategy  was  stuporous.  We  developed  isolated  systems  which  became  obsolete  before  they 
started.  Specifically  under  the  perspective  of  an  engineer  model  it  is,  and  really  was,  inefficient  and  expensive.  The 
question  is  why  didn’t  we  follow  this  seemingly  logical,  obvious  and  coercive  strategy?  Already  in  our  first  meeting 
the  question  of  the  integration  on  gestalt  level  for  the  whole  system  raised,  we  failed  to  deal  with  it  over  7 months. 

Our  anticipated  conclusion:  Until  the  day  we  are  confident,  that  the  project  would  have  broken  down,  if  we  had  had 
analysed  the  business  processes  and  started  to  develop  an  integrated  system  at  the  first  meeting.  Why? 

• At  this  time,  the  three  departments  were  still  two  different  institutions  with  quite  different  business  processes. 
Especially  the  secretaries  felt  uncertain  about  the  results  of  the  merging  process.  At  that  time  efficient  and 
systematical  clarification  of  the  business  processes  would  have  resulted  in  a highly  explosive  atmosphere  in  the 
departments. 

• Due  to  its  time  consuming  aspects  none  of  the  staff  members  wanted  to  chair  the  project. 

• It  would  have  been  difficult  to  raise  additional  financing. 

• Designing  one  integrated  system  would  have  resulted  in  difficulties  in  sharing  work  responsibilities  on  the  part 
of  the  agency  and  the  graphic  designer. 

• Planning  the  entire  project  from  scratch  would  have  caused  a major  delay  in  starting  the  homepage. 

The  arguments  mentioned  above  do  not  defeat  the  potential  usefulness  of  a system  design  process  based  on  a 
systemic  and  systemically  analyses  of  the  business  processes.  Strictly  speaking  they  even  do  not  refute  the  standard 
textbook  procedure.  But  a common  discursive  reflection  in  retrospect  showed  convincingly  that  the  financial 
shortcoming  as  well  as  the  motivation  of  all  the  parti cpants  opposed  a “cleaner”  and  more  general  approach.  This 
assessment  accepted,  we  were  lucky  that  persons  involved  had  not  been  practiced  the  engineer  model:  According  to 
this  model  the  design  process  would  have  to  be  interrupted  when  the  pre-established  goals  could  not  be  reached. 

So  what?  - as  Weik  (1985,  242)  noted,  a reasonable  question  in  organizations.  We  continued  and  succeeded.  In  order 
to  explore  the  logic  (!)  behind  the  success  of  our  case,  we  now  refer  to  some  concepts  from  the  organization  theory 
which  give  us  valuable  insights  in  the  process  of  (the)  organisation  (of  System  Design). 
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Lessons  learned  from  organization  theory 


Lesson  1:  In  the  mental  model  of  technical  rationality  you  need  to  find  solutions  for  existing  clearly  defined 
problems.  In  the  garbage  can  model  (Cohen/March/Olsen  1972;  March/Olson  1976)  problems,  solutions, 
participants  and  choice  opportunities  are  independent  streams  of  events  that  flow  into  and  through  organizations, 
much  like  a random  selection  of  waste  gets  mixed  together  in  a garbage  can. 

As  in  our  case  study  demonstrated  problems  followed  envisioned  solutions.  The  starting  point  was  the  vision  of  a 
solution  (Schon  1983  and  1987):  The  perspective  of  an  interactive  3-D  virtual  multiuser  world.  A fascinating  idea 
and,  as  you  can  see,  a good  research  object.  Of  course  much  more  fascinating  than  dull  problems  on  both 
administration  and  technical  level.  A budget  was  there  and  a producer  for  the  system  was  found.  Participants,  who 
were  willing  to  invest  a considerable  part  of  her  working  time  were  difficult  to  find  and  at  the  beginning  the  choice 
opportunities  simply  didn’t  exist.  Therefore  a lot  of  interlocked  smaller  (homepage,  administration  database)  and 
larger  (central  course  database)  solutions  exist,  which  are  partly  claimed  to  solve  the  information  problems  of  the 
students  and  partly  to  solve  other  problems,  eg  from  lecturers  and  secretaries.  At  the  beginning  nobody  saw  the 
ability  to  actualize  a more  systemic  and  systematical  process  and/or  nobody  was  willing  to  invest  more  time  than 
developing  her/his  small  solution.  As  a consequence  (eg  with  the  garbage  can  model)  we  ignored  emerging 
problems  without  reflecting  on  them. 

Lession  2 and  3:  Ortmann  (1990,  39 Iff)  introduced  the  concept  of  a ’mico politic  bricolage’  in  the  organization 
theory.  The  concept  based  on  the  bricolage-engineer  concept  from  Levi-Strauss  lesson  2),  enriched  with 
micropolitics  aspects  of  power  and  control  (lesson  3) 

The  mental  model  of  the  engineer  is  described  above.  The  mental  model  of  the  participants  in  this  case  was  that  of  a 
'Bricoleur'  (a  handicraft  man).  A Bricoleur  looks  a the  actual  situation  and  tries  to  find  a solution  with  the  tool  box 
available.  For  her/him  the  result  is  not  fixed  established,  it  is  accepted  that  the  planned  system  is  undergoing  a 
continuous  change  depending  on  available  resources.  Even  goals  can  be  adapted  and  compromises  are  allowed.  For 
a Bricoleur  it  is  okay  to  accept  limited  budgets,  to  patch  consisting  systems  together  etc.  And  when  he  plans 
substantial  changes  in  the  system  - like  the  two  described  movements  - s/he  will  not  ask  wether  the  original  goal  is 
achieved,  s/he  will  ask,  whether  the  new  solution  is  useful,  even  for  another  persons. 

The  bricoleur  metaphor  has  got  limited  in  scope  when  more  Bricoleurs  with  different  interests  and  world  views  have 
to  work  together  or  maybe  work  against  each  other  (see  Ortmann  et  al  1990,  395).  Therefore  you  have  to  bring  up 
the  social  process  in  mind.  In  the  micropolitic  approach  of  organization  theory  the  developing  and  implementation 
of  a service  system  can  be  interpreted  as  'games  of  innovation'  (Ortmann  1992,  465),  games  which  challenge  the 
consisting  ’routine  games,'  the  existing  regime  of  division  of  tasks,  the  negotiated  arrangements  etc.  It  is  a meta- 
game, which  defines  interests,  rules,  structures,  profits  and  losses  and  possible  strategies  of  future  games  of  routines. 
Hence,  these  games  were  played  with  high  interests,  in  particular  from  those,  who  see  the  existing  chances 
endangered. 

In  this  view  the  implementation  of  a system  is  not  only  a technical  process,  but  has  also  his  social  characteristics.  In 
the  model  of  technical  rationality  this  is  only  another  problem  (“to  raise  the  level  of  acceptance”,  “to  reduce  the 
resistance”)  to  be  solved  by  participation  and  information.  In  the  model  of  a ’micropolitic  bricolage’  the  technical 
and  social  process  can't  be  divided.  The  social  process  will  influence  the  technical  solution  and  vice  versa. 

Lesson  4:  In  the  mental  model  of  technical  rationality  one  has  to  follow  a certain  procedure  to  get  to  the  solution. 
The  goal  is  optimization  of  different  variables  for  the  purpose  of  efficient  functionality.  The  “know  how”  of  these 
procedures  provides  the  professional  knowledge  of  the  specialist.  In  the  mental  model  of  bricolage  knowing  how  is 
replaced  by  “to  be  able  to”  the  usable  skill  in  an  actual  situation.  To  know  how  to  change  a spare  tire  is  not  the  same 
as  to  be  able  to  do  it.  If  someone  confuses  “knowing  how”  with  “to  be  able  to  do”  then  there  is  the  intrinsic 
fundamental  rise  of  failure  (Baumgartner  1993,  76). 


Supplements  of  two  logics:  dream  and  reality 

As  you  can  see  in  the  time-table,  we  are  in  an  ongoing  process  and  we  are  actually  dreaming  to  change  from  the 
bricolage  to  the  engineer  model.  The  possibilities  to  change  the  project  from  bricolage  to  technical  rationality  (eg  to 
the  engineer  model)  are  quite  good.  It  appears,  that  there  is  now  one  of  these  seldom  lucky  chances,  when  solutions. 
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problems,  participants  and  choice  opportunities  come  together  and  results  in  a problem  solution  on  a system  level. 
Our  minds  are  full  of  rational  concepts,  nonetheless  we  are  sure,  that  the  ongoing  social  process  and  (not)  available 
resources  will  jumble  our  concepts  and  will  influence  the  database,  respectively  the  information  system. 

In  this  sense,  the  two  different  logics  as  summarized  in  the  table  below  are  not  only  excluding  each  other,  but  are 
supplements  (Derrida)  in  the  process  of  organization. 


Dimensions Technical  Rationality  (“engineer”) micropolitic  bricolage 


focus 

the  whole  system  and  the  purpose  of  the 
system 

the  tool  box  available,  the  system  and  the 
purpose 

orientation 

the  “objective”  reality,  the  world  out 
there 

the  tool  box,  the  actors,  the  situation 

result 

ge ne rated  by  the  an alysi s of  the 
problem;  should  not  be  changed 

is  not  fixed  established,  is  undergoing  a 
continuous  change  depending  of  actors,  their 
influence,  power  and  strategies 

means-end 

relation 

the  goal  is  the  priority;  the  solution  is 
subordinated  and  has  to  fulfill  the 
purpose 

solution  and  problems  are  co-existent;  solutions 
generate  new  problems 

organization  of 
the  design 

process 

paradigm  of  technical  rationality;  the 
social  process  is  just  one  of  the  marginal 
conditions 

social  and  technical  process  can't  be  divided, 
the  actual  actors  and  decision  makers  are 
important 

social  process 

...  is  only  a question  of  resistance  and 
acceptance,  actual  social  practices  are 
not  important 

...  are  social  practices,  the  situation  of  power, 
hierarchy  and  negotiation  between  participiants 
are  important 

failure 

if  pre-established  ends  can't  be  reached 

goals  can  be  adapted,  compromises  are  allowed 

organisation  is 

...  a set  of  fixed  relation  and  their  formal 
connections 

...  a set  of  social  practices  which  have  to  be 
negotiated  all  the  time 

direction 

from  the  formal  function  to  the  real 
application  (deduction) 

from  the  (possible)  solution  to  the  formal 
functions  (induction) 

evaluation 

if  the  goal  is  achieved 

one  has  also  to  reexamine  the  goal  itself 

Table  1:  Two  logics  of  system  design 
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Learning  in  the  Labyrinth:  Hypertext  and  the  Changing  Roles  of  Instructor 

and  Learner  in  Higher  Education 


Abstract:  The  media  through  which  academic  discourses  are  conducted  are  rapidly  shifting  from 
the  printed  text  and  face  to  face  dialogue,  to  the  new  electronic  forms  enabled  by  web-based 
hypertext,  hypermedia  and  computer  mediated  conferencing.  These  new  forms  can  be  seen  to 
embody  and  make  explicit  many  of  the  key  principles  emerging  from  recent  critical  theory.  Such 
theory,  by  radically  challenging  traditional  notions  of  textual  stability  and  the  nature  of  language, 
also  draw  into  question  the  possibility  of  stable  meaning  and  systems  of  knowledge,  the  pursuit  of 
which  lie  at  the  basis  of  traditional,  positivist  academic  thought.  This  paper  draws  on  such  theory  to 
consider  not  only  how,  by  ‘de-stabilising’  the  academic  text,  the  new  educational  media  are 
enabling  new  forms  of  academic  discourse  to  emerge,  but  also  how  such  changes  in  discourse  have 
the  potential  fundamentally  to  alter  the  roles  of  instructor  and  learner,  and  the  nature  of  the 
academic  institution  itself 


The  Printed  Text  and  the  University 

As  learners  and  teachers  in  higher  education,  we  work  from  within  a scholarly  paradigm  predicated  on  print 
technology.  Where  the  early  manifestation  of  scholarly  activity  in  the  copying  and  annotating  of  manuscripts 
necessarily  involved  textual  drift  and  idiosyncratic  intervention  - making  each  manuscript  a unique  artefact  - print 
allowed  us  to  apparently  ‘fix’  text,  to  create  for  ourselves  the  possibility  of  a static,  unitary,  reproducible  and  ‘true’ 
version  in  which  academic  knowledge  could  be  inscribed  and  disseminated.  Commentators  on  the  impact  of  print 
technology  on  our  culture  have  shown  how  it  has  informed  contemporary  notions  of  scholarship  (McLuhan  1962), 
while  historians  draw  our  attention  to  the  involvement  of  the  university  with  the  very  origins  of  the  book  trade 
(Haskins  1957).  The  crests  and  logos  of  countless  academic  institutions  which  are,  or  wish  to  appear,  ancient  and 
eminent  repeatedly  show  the  image  of  the  printed  and  bound  book,  sitting  at  the  very  heart  of  the  university’s 
symbolic  representation  of  itself. 

This  central  position  of  the  printed  text  occurs  because  the  notion  of  textual  fixity,  or  stability,  which  print 
technology  both  symbolises  and  perpetuates  is  intrinsically  bound  up  with  what  has  traditionally  been  seen  as  the 
fundamental  basis  of  scholarly  activity  - the  pursuit  of  truth,  authority,  objectivity  and  stable  systems  of  knowledge 
which  characterise  ‘traditional’  academic  thought.  During  the  latter  half  of  this  century,  such  ideas  have  been  under 
sustained  attack  from  literary,  cultural,  political  and  scientific  theorists  and  philosophers,  but  it  is  only  since  the 
advent  of  hypertext,  hypermedia  and  other  manifestations  of  the  new  communication  technologies  that  a material 
and  explicit  alternative  to  print  has  been  available  to  us  to  question  the  formulations  of  meaning  and  knowledge 
inherited  from  print  technology. 

As  Landow  reminds  us: 


Sian  Bayne 
University  of  Bristol 
England,  UK 
sian.bayne@bristol.ac.uk 


Ray  Land 

University  of  Edinburgh 
Scotland,  UK 
ray . land@ed  .ac.uk 


We  so  tend  to  take  print  and  print-based  culture  for  granted  that. . .we  have  ‘naturalized’  the  book 
by  assuming  that  habits  of  mind  and  manners  of  working  associated  with  it  have  naturally  and 
inevitably  always  existed.  (Landow  1 997) 


Electronic  texts  - in  the  form  of  hypertext  and  hypermedia-based  learning  materials,  web-based  and  hyperlinked 
computer-mediated  conferencing  exchanges,  and  even  simple  email  - are  rapidly  gaining  ground  in  universities  as 
means  of  generating  and  sharing  academic  knowledge.  While  we  must  take  care  not  to  subscribe  blindly  to  what 
Duguid  (1996)  describes  as  a preoccupation  with  technological  ‘supersession’  (the  idea  that  each  new  technological 
type  vanquishes  or  subsumes  its  predecessors),  or  to  other  forms  of  technological  determinism,  we  might  reasonably 
predict  that  such  technologies  will  in  time  displace  print  as  the  primary,  ‘naturalized’  medium  through  which 
academic  discourses  are  conducted.  As  this  occurs,  we  face  fundamental  changes  in  every  area  of  the  academy,  from 
the  ways  in  which  the  relations  between  instructor  and  learner  are  formulated,  through  the  discourses  and  structures 
of  our  institutions  and  ultimately  to  our  very  perceptions  of  the  nature  of  the  ‘scholarly’  and  what  it  means  to  pursue 
academic  knowledge.  The  university,  an  essentially  modernist  institution  (Bloland  1995),  is  being  driven  towards 
postmodemity  by  the  postmodern  artefacts  of  hypermedia. 


The  Electronic  Text  and  the  Transformed  Role  of  the  Reader 

Such  fundamental  changes  are  predicated  on  the  fact  that  hypertext  and  hypermedia  force  us  to  change  both  the  way 
we  read  and  our  ideas  about  what  constitutes  ‘text’.  Hypertext  theory  teaches  us  that  texts  created  in  the  electronic 
writing  space  do  not  conform  to  the  Aristotelian  notion  that  texts  should  have  a beginning,  middle  and  end,  and  be 
structured  and  read  sequentially.  Readers  of  hypertext  enter  the  textual  world  at  any  point  (in  the  case  of  web-based 
hypertext  this  world  is  almost  infinite),  determine  for  themselves  a path  through  that  world  by  following  links  in  an 
order  not  determined  by  an  author  or  instructor  but  by  their  own  inclination,  and  emerge  not  when  a text  is 
‘finished’,  but  when  they  feel  ready  to  abandon  it.  Hypertext  challenges  traditional  models  of  sequential  progression, 
based  on  centuries  of  print-based  technology,  and  presents  a multi-linear,  reader-defined  model  of  interaction  with 
text.  The  text  does  not  exist  in  singular,  static  form  but  varies  with  each  reading. 

Hypertext  theory  emerges  from  critical  theories  of  the  last  three  or  four  decades,  in  particular  those  of  post- 
structuralism. The  latter  have  fundamentally  altered  our  conception  of  the  nature  of  text  (regardless  of  what  media  it 
is  presented  in)  and  strikingly  anticipate  the  emergence  of  hypertext  and  hypermedia,  and  the  new  ways  of  reading 
necessitated  by  these  forms.  It  will  be  useful  at  this  point  briefly  to  outline  such  theories,  and  the  ways  in  which  they 
alter  the  relationships  - forged  in  the  context  of  the  dominance  of  print  technology  - between  author,  reader  and 
text.  We  will  then  consider  how  these  new  relationships  are  extended  into  the  learning  context  and  what  the 
implications  might  be  for  institutions  of  higher  education. 

‘Traditional’  theory,  or  that  which  has  its  roots  in  the  Romantic  movement  of  the  nineteenth  century,  assumes  a 
fundamentally  hierarchical  relationship  between  the  writer  of  a text  and  its  reader.  The  writer  is  seen  as  the 
originator  of  the  text,  as  the  owner  or  inventor  of  the  text’s  meaning  (the  word  ‘author’  itself  clearly  implies  this 
authoritative  position).  The  author  is,  in  the  Romantic  view,  the  possessor  of  a superior  creative  soul,  one  capable  of 
communion  with  higher  truths  with  which  they  enlighten  the  rest  of  the  world  through  the  medium  of  the  written 
word.  The  reader  in  this  scheme  is  the  lowly  recipient  of  the  writer’s  vision  (it  is  his  or  her  duty  to  attempt  to  extract 
from  the  text  its  true  meaning,  i.e.  that  which  the  author  has  put  into  it). 

This  assumption  was  much  questioned  by  twentieth  century  theorists.  In  particular,  those  of  the  reader-response 
school  articulated  theories  which  stressed  the  primary  role  of  the  reader  in  the  construction  of  meaning.  Iser  claims 
that  the  literary  work  is  not  simply  the  text  itself,  but  a synthesis  of  the  text  and  its  ‘realization’  in  the  mind  of  the 
reader  (Iser  1972),  while  Fish  goes  further,  suggesting  that  there  is  no  meaning  inherent  in  the  text  itself.  A work  is 
only  ever  a product  of  the  reader’s  interpretive  processes,  and  has  no  objective  existence  independent  of  the  reading 
experience  (the  reader  is  the  writer)  (Fish  1980).  Barthes  distinguishes  between  the  lisible  (‘readerly’)  classic  text 
which  places  the  reader  in  the  position  of  passive  consumer  of  narrative  and  meaning  controlled  by  the  author,  and 
the  schptible  (‘writerly’)  modem  work  which  forces  its  readers  to  be  active  in  the  construction  of  meanings  which 
are  multiple  and  inexhaustible  (Barthes  1970). 

Though  none  of  these  theorists  refers  explicitly  to  hypertext  or  other  forms  of  electronic  writing,  the  connection  is 
clear.  Hypertext  forces  the  reader  to  take  control  of  his  or  her  own  experience  of  the  text  by  requiring  choices  to  be 
made  about  how  and  in  what  order  connections  between  blocks  of  text,  or  units  of  meaning,  are  made.  With  certain 
forms  of  electronic  text,  including  that  generated  in  hyperlinked  computer-mediated  conferencing  environments,  the 
vision  of  the  author  as  the  single  originator  of  the  text  and  possessor  of  its  meaning  is  submerged  completely  (the 
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reader  quite  literally  is  the  writer).  The  reader,  figuratively  of  central  importance  to  the  reader  response  theorists,  is 
made  materially  central  in  the  electronic  medium.  As  Bolter  puts  it: 

The  computer  makes  concrete  the  act  of  reading  (or  misreading)  as  interpretation  and  challenges 
the  reader  to  engage  the  author  for  control  of  the  writing  space...  The  computer  makes  visible  the 
contest  between  author  and  reader  that  in  previous  technologies  has  gone  on  out  of  sight,  ‘behind’ 
the  page.  (Bolter  1991) 

Alongside  such  theories  of  reader  response,  we  must  also  consider  the  ideas  of  the  post-structuralist  theorists  (the 
philosopher  Derrida,  the  historian  Foucault  and  the  psychoanalyst  Lacan),  and  their  focus  on  the  indeterminacy  or 
instability  of  meaning,  the  fragmentation  of  individual  identity,  and  the  centrality  of  discourse  (Land  & Bayne 
1998).  Post-structuralist  thought  emphasises  that  language  does  not  provide  us  with  a set  of  terms  or  labels  which  we 
use  to  name  entities  which  already  exist  independently  in  the  world.  Rather  language  precedes  the  existence  of  these 
entities,  enabling  us  to  make  the  world  intelligible  by  differentiating  between  concepts  in  a way  which  is  particular 
to  our  own  cultural  context.  The  way  in  which  we  formulate  meaning  cannot  be  fixed  and  pinned  down  - language 
operates  as  a system  of  signs  in  which  the  relationship  between  signifiers  (words,  written  or  spoken)  and  signifieds 
(the  concept  or  meaning  referred  to)  is  constantly  shifting,  always  in  a state  of  flux.  Meaning  cannot  be  nailed  down, 
it  is  inherently  unstable,  ‘never  fully  present  in  any  one  sign  alone,  but  rather  a kind  of  constant  flickering  of 
presence  and  absence  together’  (Eagleton  1 983). 

Ideas  about  the  nature  of  text  itself  are  radically  affected  by  the  ideas  of  the  post-structuralists.  Texts  are  no  longer 
seen  as  closed,  stable  entities,  holding  within  them  definite  meanings  which  the  reader  is  required  to  decipher. 
Rather,  they  become  a temporary  arena  for  the  endless  play  of  signifiers  - they  cannot  be  tied  down  to  any  one 
meaning,  but  remain  open  and  pluralistic,  subject  to  multiple  interpretation.  All  texts  are  woven  out  of  other  texts  in 
that  every  element  of  them  refers  to  other  writings  which  surround,  precede  or  follow,  much  as  the  sign  itself  is 
composed  of  traces  of  every  other  sign.  This  is  broadly  referred  to  as  ‘ inter-textual ity’. 

What  is  more,  the  traditional  hierarchical  relationship  between  texts  is  undermined.  Where  traditionally  a so-called 
central  or  ‘primary’  text  takes  a prime  position,  providing  a focus  for  discussion  and  interpretation  in  the  form  of 
marginal  or  ‘secondary’  texts,  Derridean  deconstructionists  would  claim  not  only  that  the  central  can  become 
marginal  (and  vice  versa)  but  that  the  very  difference  between  the  two  can  be  brought  into  question.  Since  both  are 
subject  to  infinite  play  of  meaning  and  interpretation,  we  cannot  say  where  one  ends  and  the  other  begins.  The  text  is 
‘a  differential  network,  a fabric  of  traces  referring  endlessly  to  something  other  than  itself,  to  other  differential 
traces.  Thus  the  text  overruns  all  the  limits  assigned  to  it  so  far’  (Derrida  1991). 

The  relevance  of  all  this  to  the  study  of  electronic  text  is  implied  in  a passage  from  Eagleton ’s  Literary  Theory 
which  describes  the  post- structuralist  vision  of  language; 

Instead  of  being  a well-defined,  clearly  demarcated  structure  containing  symmetrical  units  of 
signifiers  and  signifieds,  [language]  now  begins  to  look  much  more  like  a sprawling  limitless  web 
where  there  is  a constant  interchange  and  circulation  of  elements,  where  none  of  the  elements  is 
absolutely  definable  and  where  everything  is  caught  up  and  traced  through  by  everything  else. 

(Eagleton  1983) 

Web-based  hypertext  and  hypermedia  can  be  seen  as  materialising  the  concepts  of  post-structuralist  theory.  The 
world  wide  web,  sprawling,  limitless,  consisting  of  endless,  unstable,  changing  elements,  each  one  potentially  linked 
to  each  other  and  therefore  bearing  within  it  infinite  traces  or  connective  possibilities,  embodies  the  characteristics 
the  post-structuralists  see  in  language  systems.  Hypertext  is  endlessly  recenterable.  As  a reader  moves  through  a 
hypertext  construct,  or  as  a student  moves  through  a virtual  learning  environment,  he  or  she  continually  shifts  centre 
or  focus,  is  continually  active  in  creating  new  possibilities  for  meaning.  Such  environments  are  also  explicitly 
intertextual,  stressing  connectivity  rather  than  what  Bolter  calls  ‘textual  independence’  - ‘the  electronic  space 
permits  us  to  visualize  intertextuality  as  no  previous  medium  has  done’  (Bolter  1991). 

This  intertextuality  enables  hypertext,  in  both  art  and  pedagogy,  to  break  through  traditional  generic  and  disciplinary 
boundaries,  in  the  process  violating  the  hierarchy  of  centre/margin  (primary  text/secondary  text)  identified  by 
deconstructionist  theory.  Just  as  a hypertext  poem  might  contain  within  it  links  to  prose  sites,  to  critical  analyses, 
biographies,  video  and  sound,  so  a section  of  online  learning  material  might  link  to  news  clips,  cartoon  animation 
and  discussion  fora  as  well  as  to  pedagogical  material  drawn  from  other  related  subject  areas  and  disciplines.  Just  as 
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the  work  itself  is  not  constrained  to  a single  medium,  neither  does  it  contain  a fixed  centre.  The  centre  is  where  the 
reader  or  learner  is  at  any  given  moment;  the  marginal  becomes  central  at  the  click  of  a mouse  button. 


Teaching  and  Learning  in  the  Hypertextual  University 

How  will  academic  study  be  affected  as  electronic  text  (which  as  we  have  seen  embodies  and  makes  ‘obvious’  some 
radical  theoretical  principles)  begins  to  replace  print-based  text  as  the  ‘norm’? 

One  chief  effect  of  electronic  hypertext  has  been  the  way  it  challenges  now-conventional 
assumptions  about  teachers,  learners,  and  the  institutions  they  inhabit.  It  changes  the  roles  of 
teacher  and  student  in  much  the  same  way  it  changes  those  of  writer  and  reader.  Its  emphasis  upon 
the  active,  empowered  reader,  which  fundamentally  calls  into  question  general  assumptions  about 
reading,  writing  and  texts,  similarly  calls  into  question  our  assumptions  about  the  nature  and 
institutions  of  education  that  so  depend  upon  these  texts.  (Landow  1 997) 

The  transformational  implications  of  hypertext  for  teaching  and  learning,  and  for  the  academy  more  generally,  can 
be  identified  at  a number  of  levels.  We  have  already  seen  how  at  the  textual  level  it  implies  a shift  to  openness, 
inconclusivity,  infinite  recenterability  and  the  empowerment  of  the  reader;  the  final  section  of  this  paper  will  show 
how  such  potential  for  transformation  is  extendable  to  the  roles  of  instructor  and  learner,  to  the  formulations  and 
boundaries  of  disciplines,  to  the  possibility  for  new  forms  of  academic  discourse,  and  finally  to  the  nature  of  the 
higher  education  institution  and  community. 

The  Role  of  the  Learner 

Landow  describes  ways  in  which  working  with  hypertext  transforms  the  student  learning  experience.  Because, 
within  hypertextual  space,  students  gain  both  information  and  habits  of  thinking  critically  in  terms  of  ‘multiple 
approaches  or  causes’,  hypertext  can  be  seen  to  orient  students  towards  developing  a habit  of  ‘making  connections’. 
This  fosters  ‘a  participatory  reading-and- writing  environment’  which  empowers  students  ‘by  placing  them  within  - 
rather  than  outside  - the  world  of  research  and  scholarly  debate’.  It  thus  enables  students  to  explore  and  create  new 
modes  of  discourse  suited  to  modes  of  reading  and  writing  increasingly  used  in  e-space.  (Landow  1 997) 

A clear  implication  for  the  role  of  the  learner  in  such  environments  is  the  requirement  that  they  adopt  an  active 
approach.  Hypertextual  systems,  argue  Jonassen  and  Grabinger  ‘place  more  responsibility  on  the  learner  for 
accessing,  sequencing  and  deriving  meaning  from  the  information’  (Jonassen  & Grabinger  1990).  In  this  regard 
hypertext  is  probably  best  considered  as  a learning  rather  than  teaching  environment  in  which  an  exploratory, 
‘research’  approach  to  learning  may  flourish.  By  disposing  students  towards  multilinear  and  multisequential  patterns 
of  thinking,  and  immersing  them  in  environments  of  multi  vocal  ity,  hypertextual  systems  may  also  stimulate 
processes  of  ‘integration  and  contextualization’  - high  order  cognitive  skills  - in  ways  not  achievable  by  linear 
presentation  techniques  (Mayes  et  al  1990).  This  accords  with  Landow’s  interpretation  of  conceptual  skill  as  ‘chiefly 
making  connections’,  a capacity  eminently  endorsed  by  hypertext’s  defining  characteristic  of  connectivity. 

The  Death  of  the  Tutor? 

Barthes  once  famously  proclaimed  the  ‘death  of  the  Author’  (Barthes  1977)  - announcements  of  the  death  of  the 
Tutor  may  be  premature  but  the  pedagogical  application  of  hypertext  does  alter  the  role  of  instructors  by  shifting 
some  of  their  authority  and  function  to  the  learner.  As  we  have  seen,  engagement  with  hypertext  necessarily 
empowers  the  reader  or  learner  so  that  a transmissive  theory  of  knowledge  which  characterises  teachers  as  ‘the  ones 
who  know’  and  their  task  as  ‘to  channel  what  they  know  into  the  students’  is  increasingly  difficult  to  sustain 
(Spender  1995). 

Such  shifts  in  authority  imply  a transference  of  power  to  learners,  which  imposes  upon  them  a duty  of  responsibility 
for  autonomy,  critical  judgement,  and  a higher  degree  of  tolerance  of  complexity  and  ambiguity.  This  in  turn  might 
be  seen  to  have  an  emancipatory  effect  upon  the  role  of  the  teacher,  who  is  freed  from  the  traditional  status-oriented 
informative,  didactive  and  confrontational  interventions  of  traditional  practice  in  favour  of  more  supportive  and 
guidance-oriented  cathartic,  catalytic  and  facilitative  interventions. 
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Entwistle,  Thomson  and  Tait  (Entwistle  et  al  1992)  argue  that  encouraging  a ‘deep  approach’  as  opposed  to  a 
‘surface  approach’  in  learners  requires  both  an  active  approach  and  interaction  with  peers  and  tutors.  Hypertext,  as 
Landow  shows  (Landow  1992)  affords  excellent  opportunity  for  collaborative  learning,  whereas  such  activity,  as 
Ede  and  Lunsford  (Ede  & Lunsford  1 990)  demonstrate,  tends  to  be  mistrusted  in  closed  text  environments,  where 
‘the  pervasive  commonsense  assumption’  is  ‘that  writing  is  inherently  and  necessarily  a solitary,  individual  act.’ 
Entwistle  and  colleagues  also  show  the  need  both  for  a ‘well-structured  learning  environment’  - a tall  order  in  the 
hypertextual  labyrinth  but  nonetheless  crucial  to  pre-empt  learner  disorientation  and  intimidation  - and  for  ‘a 
motivational  base’,  the  support  and  encouragement  needed  by  all  learners.  Assessment,  too,  Landow  reminds  us, 
must  seek  evidence  of  the  connectivity  and  contextualisation  hypertext  affords,  if  it  is  to  have  validity.  So  a rich  but 
challenging  role  remains  for  the  instructor  - one  that  is  hypertextually  transformed  into  that  of  facilitator-moderator. 

Interdisciplinarity 

It  is  no  longer  possible.  Spender  argues,  for  teachers  to  appear  as  experts  in  a computer-based  world.  ‘There  is  just 
too  much  information  for  any  one  individual  to  master’  (Spender  1995).  Within  the  print-based  university  - the 
society  of  the  closed,  static  text  - knowledge  was  in  short  supply,  a special  commodity.  This  empowered  the 
gatekeepers,  the  initiators  into  disciplines,  to  determine  intellectual  boundaries,  to  regulate  entry  to  the  next  level  of 
information,  and,  as  the  ambivalence  of  the  Latin  root  of  ‘discipline’  suggests,  to  control  and  to  teach 
simultaneously.  Lyman  talks  of  an  ‘ecology  of  knowledge’  in  which,  when  knowledge  is  scarce,  centralisation  and 
conservation  take  precedence.  In  an  information-saturated  environment,  however,  ‘our  distinctive  information 
problem  is  one  of  selection,  as  much  as  conservation’  (Lyman  1 994).  There  is  no  entry  test.  Spender  reminds  us,  for 
TV  viewing.  The  logic  of  hypertext,  a postmodern  phenomenon,  is  to  erode  the  controlling  boundaries  of  the 
discipline,  which,  in  its  unifying  and  totalizing  tendency,  is  a modernist  concept.  The  impetus  of  hypertext  is 
inexorably  towards  interdisciplinary  connection.  The  learner  is  no  longer  dominated  by  the  dominus. 

Democratic  Publishing  and  Contextualised  Debate 

New  hypertextual  environments  have  the  potential  to  transform  the  way  in  which  scholarly  publishing  and  debate 
operates.  Buckingham  Shum  and  Sumner  (Buckingham  Shum  & Sumner  1998),  for  example,  discuss  how 
hypermedia  might  enable  academic  authors  to  break  free  of  some  of  the  constraints  of  traditional,  paper-based 
scholarly  publishing  (its  expense,  long  publication  timescales,  restriction  to  static  text  and  image  and  generally  non- 
existent facility  for  subsequent  debate  and  discussion).  Even  the  majority  of  web-based  e-joumals,  they  argue, 
remain  ‘papyrocentric’,  using  technology  to  replicate  traditional  publishing  approaches  rather  than  to  make 
meaningful  advances  on  current  forms  of  academic  discourse. 

Their  toolkit  for  the  publishing  and  discussion  of  web  documents  - the  ‘Digital  Document  Discourse  Environment’ 
(D3E),  operates  from  the  premise  that  ‘the  intellectual  cut  and  thrust  of  discussion  and  debate  between  peers  as  they 
contest  the  ideas  proposed  in  a document’  is  a key  academic  activity.  It  automates  the  procedures  around  publishing 
a document  to  the  web  and  integrates  the  document  (which  may  be  in  a variety  of  media)  with  a discussion 
environment  which  opens  it  up  to  clearly  contextualised  debate.  Readers’  comments  link  back  to  the  text  and  vice 
versa,  and  authors  are  able  easily  to  revise  and  update  documents  as  their  research  develops  or  ideas  change.  The 
traditional,  static,  closed,  paper-based  academic  text  becomes  an  accessible  dynamic  entity,  potentially  open  to 
immediate  input  from  a worldwide  academic  community.  The  toolkit  also  assists  the  creation  of  innovatory  student 
assignments,  with  modes  of  student  writing  mirroring  those  of  their  instructor-publishers.  This  represents  another 
manifestation  of  hypertext’s  functioning,  in  Galegher  and  Kraut’s  phrase,  as  a ‘permissive  technology’,  fostering 
new  forms  of  academic  discourse,  and  allowing  ‘current  practices  to  be  extended  into  new  realms  in  which  they  had 
previously  been  impracticable’. (Galegher  & Kraut  1990) 

Boundariless  Anxiety,  Widened  Community 

Hypertextual  environments,  in  their  protean  boundarilessness,  pose  daunting  dilemmas  for  the  university  as 
modernist  institution.  As  Bergquist  argues,  in  modem  organisations  ‘boundaries  (and  identities  defined  by  roles  and 
rules)  serve  as  “containers”  of  anxiety’  (Bergquist  1995).  We  can  see  the  function  of  disciplines,  modularisation, 
academic  management  and  reward  stmctures  in  this  regard.  However  in  the  postmodern  condition  these  boundaries 
are  eroded  and  there  is  a ‘spilling  out  of  anxiety’,  a ‘sense  of  living  on  the  edge’.  Another  container  of  anxiety  has 
traditionally  been  admission  criteria  and  the  division  of  individuals  into  those  allowed  in  and  those  kept  out. 
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But  in  the  fully  ‘technology-normalized’  institution,  access  becomes  virtual  and  community  easily  widened, 
undermining  the  difference  between  centre  and  margin,  what  is  inside  the  academy  and  what  (or  who)  must  remain 
outside,  Deleuze  has  used  the  term  ‘enclosed  environments’  to  describe  territories  which  are  circumscribed  within 
organisations  ‘to  concentrate;  to  distribute  space;  to  order  time;  to  compose  a productive  force  within  the  dimension 
of  space-time  whose  effect  will  be  greater  than  the  sum  of  its  component  forces’  (Deleuze  1992),  while  Taylor 
suggests  that  terms  such  as  ‘boundaries’  and  ‘sites  of  enclosure’  are  used  ‘to  describe  attempts  by  institutions  to 
create  contexts  - enclosed  territories  - in  which  only  the  occupants  define  the  particular  rules  and  practices  which 
govern  internal  operations’  (Taylor  1999).  But  the  educational  institution  now  has  the  potential  to  develop  as  an 
unbounded  environment,  with  all  its  anxieties  and  its  emancipatory  promise  - an  environment  into  which  hypertext, 
in  the  guise  of  some  neo-Promethean  tool,  beckons  us. 
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Abstract:  As  instructional  technology  becomes  ubiquitous  in  classrooms,  faculty  will  be 
asked  to  utilize  new  technologies  in  their  pedagogy.  Some  will  accept  new  ways  to  teach 
with  technology  while  others  resist.  What  are  factors  that  influence  faculty  to  adopt  this 
technology  and  find  effective  pedagogy  to  utilize  it?  The  Department  of  Learning 
Resources  at  the  State  University  of  West  Georgia  asked  its  faculty  about  influences  to 
technology  adoption  while  creating  a plan  to  tackle  technology  resistance.  The  resulting 
data  showed  the  faculty  recognized  common  barriers  to  instructional  technology  as  well 
as  influences  that  helped  them  overcome  the  barriers. 


As  higher  education  faculty  adopt  new  instructional  technologies,  it  will  be  important  for  influencing 
factors  and  barriers  to  their  adoption  of  instructional  technology  to  be  addressed.  The  success  of  our  faculty  in 
utilizing  technology  will  impact  our  students’  education  and  ultimately  success  in  their  careers.  In  this 
technology  age,  higher  education  must  prepare  students  to  integrate  technology  in  their  learning  if  our 
institutions  expect  to  continue  to  be  viable  avenues  for  students  to  prepare  for  life  in  the  workforce. 

Literature 

The  definition  of  instructional  technology  used  in  this  study  is  one  that  refers  to  the  use  of  technology  to 
achieve  an  instructional  objective  (Spotts  & Bowman,  1995).  This  technology  includes  computer  hardware  and 
software,  networks,  email,  multimedia,  and  computer  peripherals  such  as  CD  players.  Traditional  media  such  as 
video  and  audio  are  also  defined  as  instructional  technology. 

Albright  (1996)  said  in  a keynote  address  to  the  Southern  Regional  Faculty  and  Instructional  Development 
Consortium  that  so  many  barriers  have  existed  in  higher  education  to  the  adoption  of  instructional  technology 
that  faculty  use  of  technology  has  been  accomplished  in  spite  of  the  campus  environment.  He  notes  that 
education  may  be  the  only  business  that  actually  discusses  that  there  are  barriers  to  its  employees'  adoption  of 
technology.  Albright  also  points  out  that  higher  education  faculty  have  a conservative  approach  to  its  institution 
and  a commitment  to  the  traditional  means  of  teaching  - a barrier  to  innovation  and  technology  in  instruction. 
Approximately  80%  of  higher  education  instruction  is  delivered  by  lecture,  a form  in  which  instructional 
technology  can  be  utilized,  but  only  in  rudimentary  ways.  Research  suggests  that  lecture  is  not  the  best  method 
for  developing  problem  solving  and  critical  thinking,  often  desired  skills  in  the  workplace  (Oblinger  & Rush, 
1997). 

Change  in  higher  education  has  traditionally  been  slow.  In  the  literature  there  is  a reference  to  an 
anonymous  quote  apparently  uttered  by  a frustrated  instructional  technologist: 

The  pace  of  academe  is  perhaps  best  measured  by  the  25  years  it  took  to  get  the 
overhead  projectors  out  of  the  bowling  alley  and  into  the  classroom  (in  Gilbert  &. 

Green,  1997,  p.  25). 

The  time  span  does  not  matter,  but  the  heart  of  the  issue  remains.  When  faculty  began  adopting  the  new 
instructional  technology  there  were  issues  that  affected  its  acceptance  and  slowed  its  adoption. 

The  fear  of  failure  in  using  the  technology  is  a factor  that  has  been  cited  as  an  initial  barrier  (Hannafin  & 
Savenye,  1993).  While  faculty  members  are  experts  in  their  content  area,  they  are  not  experts  in  the  use  of 
technology.  They  are  also  concerned  about  instructional  technology  operating  consistently  in  the  classroom 
(Byron,  1995).  When  the  technology  is  difficult  to  use,  the  faculty  would  often  have  problems  and  refuse  to  use 
the  technology  again  (Albright,  1996). 
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The  availability  of  technical  support  staff  to  assist  when  in-class  problems  arise  is  important  for  faculty  if 
they  are  going  to  tackle  instructional  technology  (Nantz  & Lundgren,  1998).  Improved  classroom  designs  will 
also  alleviate  concerns  about  difficult  to  use  technology.  Another  anxiety  that  faculty  face  in  adopting 
instructional  technology  is  the  devaluation  of  their  profession  and,  consequently,  the  elimination  of  their  jobs 
(Novek,  1996).  As  technologies  are  promoted  as  the  next  best  way  to  reach  more  students,  such  as  through 
distance  learning,  faculty  worry  about  the  role  of  the  traditional  campus  and  their  possibly  diminished  place  in 
the  education  system  (Oblinger  & Rush,  1997). 

Novek  (1996)  also  said  faculty  were  concerned  about  the  dehumanizing  potential  of  technology.  These 
instructors  were  concerned  that  students  and  teachers  would  become  alienated  by  technologies  such  as  distance 
learning.  Oblinger  and  Rush  (1997)  suggest  that  some  professorial  functions  such  as  simple  transmission  of 
facts  may  be  replaced,  but  more  important  functions  will  continue  to  be  essential.  Leading  discussions,  helping 
students  make  sense  of  the  whole,  and  suggesting  resources  are  among  the  aspects  that  will  create  an  improved 
learning  environment.  Research  in  asynchronous  learning  environments  reported  students  have  more  access  to 
their  professors  than  in  traditional  lectures  (Oblinger  & Rush,  1997). 

Whether  the  use  of  instructional  technology  in  the  classroom  is  effective  for  teaching  is  also  an  issue 
which  faculty  consider  important  (Byron,  1995).  Faculty  question  whether  students’  performance  will  be 
improved  with  the  use  of  technology.  Research  has  shown  that  it  is  not  the  technology  itself  but  how  the 
technology  is  used  that  improves  learning  and  increases  student  interest  (Albright,  1996;  Charp,  1998). 

Though  students  have  said  that  faculty  are  not  ready  to  make  the  paradigm  shift  to  use  multimedia  and 
instructional  technology  in  the  classroom  (DeLoughry,  1995),  instructional  technologists  have  evangelized  for 
years  that  there  are  proven  ways  to  encourage  faculty  to  adopt  and  develop  instructional  technology  (Albright, 
1996).  For  example,  the  availability  of  instructional  technology  is  an  influence  to  its  adoption  by  faculty. 
Faculty  also  welcome  an  effective  support  system.  In  a series  of  faculty  focus  groups  at  the  University  of  North 
Texas,  Byron  (1995)  found  that  knowledgeable,  friendly  technical  support  staff  who  respond  in  a timely  manner 
are  among  the  top  factors  for  technology  adoption. 

Faculty  members  say  it  takes  time  to  learn  the  technology  and  it  takes  even  more  time  to  develop 
instructional  technology  materials  (Hirschbuhl  & Faseyitan,  1994;  Nantz  & Lundgren,  1998).  Green  (1998)  says 
that  institutions  have  not  been  prepared  for  the  sustained  investment  in  time  and  support  and  financial  resources 
needed  to  develop  quality  instructional  resources.  One  of  the  most  important  support  issues  influencing  faculty 
to  adopt  technology  is  training  (Armstrong,  1996).  In  the  Campus  Computing  Project  1998  National  Survey  of 
Information  Technology  in  Higher  Education,  Green  (1998)  stated  that  “assisting  faculty  integrate  technology 
into  instruction”  as  the  single  most  important  issue  reported  and  providing  adequate  user  support  as  the  second 
(p.l). 

As  faculty  adapt  technology  to  their  pedagogy  and  grow  in  its  use,  they  deserve  recognition  for  their 
efforts.  There  must  be  a reward  system  that  recognizes  the  adoption  of  technology  through  promotion  and 
tenure  (Nantz  & Lundgren,  1998).  It  has  become  even  more  important  as  the  adoption  of  technology  has 
increased,  but  Green  finds  the  majority  of  universities  and  colleges  have  not  addressed  this  issue  (Green,  1 997). 
Technology  support  centers  and  faculty  mini-grants  are  needed  as  well,  but  failing  to  recognize  and  promote 
faculty  who  are  integrating  technology  "sends  a chilling  message  about  institution  commitment,"  Green  says  (p. 
2).  Growth  in  the  adoption  of  instructional  technology  requires  a continuing  investment  of  time,  personnel,  and 
funding  (Green,  1 996).  Administrative  commitment  is  required  for  this  long-term  development  because  only  the 
institution  can  provide  these  resources. 

Innovative  teachers  can  be  adopters  or  non-adopters  of  technology,  but  innovative  adopters  found 
significant  advantages  in  the  use  of  technology  (Reimers,  Rathbun  & Goodrum,  1 999).  Innovative  technology 
adopters  discovered  instructional  technology  gave  them  flexibility  in  designing  course  materials  (Reimers, 
Rathbun  & Goodrum,  1 999). 

One  of  the  keys  to  the  success  of  instructional  technology  may  be  the  change  in  pedagogy  from  students 
being  passive  recipients  of  knowledge  to  becoming  active  participants  in  the  instructional  process.  If  students 
are  to  become  more  active  learners,  faculty  must  establish  more  communication  and  student-centered  learning. 
When  faculty  are  encouraged  to  utilize  instructional  technology  their  pedagogy  evolves.  In  the  project  at 
Hofstra  University,  as  faculty  adopted  multimedia  technology  they  changed  teaching  styles  to  focus  on  guiding 
and  facilitating  instead  of  just  lecturing  about  the  material  (Haile,  1998). 

Interaction  between  faculty  and  student  has  been  enhanced  with  the  increased  usage  of  email  to  transmit 
class  assignments  and  make  faculty  available  outside  of  class  hours  to  answer  questions  (Reimers,  Rathbun  & 
Goodrum,  1 999).  Students  can  access  courses  anytime  and  anywhere  by  the  use  of  the  web  and  Internet. 
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During  the  1990s,  Kenneth  Green  has  tracked  the  changes  in  the  adoption  of  technology  by  surveying 
college  and  university  computing  officials.  He  reports  that  faculty  technology  adoption  had  reached  a self- 
sustaining  point  based  on  Everett  Rogers*  diffusion  of  innovation  (Green,  1996).  Rogers  described  this  point  as 
when  15-20%  of  individuals  have  adopted  an  innovation  (Rogers,  1995). 

Data  from  the  1998  National  Survey  of  Information  Technology  shows  there  has  been  a jump  from  8.4% 
in  1997  to  22.5%  in  1998  in  the  use  of  web  pages  for  class  materials  and  resources  (Green,  1998).  Email  use  in 
support  of  courses  has  Jumped  from  32.8  percent  in  1997  to  44  percent  in  1998. 

Until  as  late  as  the  mid-1990s,  there  had  not  been  significant  research  on  the  use  of  instructional 
technology  by  faculty  in  higher  education.  Green  has  annually  reported  since  1990  the  results  of  Campus 
Computing  Project  annual  National  Survey  of  Information  Technology  in  higher  education.  However,  the 
survey  information  is  provided  by  officials  at  some  500-600  two-  and  four-year  colleges  and  universities  and 
not  directly  from  faculty  members. 

The  purpose  of  this  study  is  to  examine  the  aforementioned  barriers  and  influences  to  faculty  adoption  of 
instructional  technology  as  well  as  to  consider  the  technology  that  faculty  are  using  in  the  classroom  and  the 
faculty's  perceived  knowledge  of  technology. 

Methodology 

This  research  studies  the  influences  on  faculty  adoption  of  instructional  technology  and  the  integration 
technology  into  instruction  at  a state  university  with  348  full-time  faculty  and  about  8,800  students.  A survey 
with  61  questions  was  adapted  from  an  instrument  previously  used  by  Groves  & Zemel  (1999).  Questions  were 
designed  to  determine  the  faculty’s  self-reported  knowledge  and  use  of  technology,  factors  influencing  their  use 
of  technology,  and  perceived  barriers  to  the  use  of  technology  in  the  classroom.  Faculty  were  also  asked  about 
the  importance  of  instructional  technology  to  their  teaching  and  if  they  would  continue  to  adopt  new  technology 
(Groves  & Zemel,  1999).  They  were  also  asked  about  department,  home  computer  use,  rank  and  years  in 
education. 

The  study  was  performed  in  April,  1999.  There  were  348  surveys  sent  by  campus  mail  to  all  full-time 
faculty  members  along  with  a cover  letter  explaining  its  purpose  and  informing  respondents  they  would  remain 
anonymous.  There  were  156  surveys  returned  by  campus  mail  for  a return  of  just  over  44%. 

Results  and  Conclusions 

This  study  considers  questions  that  technology  support  personnel  have  debated  for  some  time.  What  are 
the  factors  that  influence  faculty  in  their  adoption  of  instructional  technology  and  what  are  barriers  that  restrict 
faculty  from  utilizing  technology?  Closely  tied  to  these  factors  are  the  technologies  the  faculty  have  adopted  in 
their  instruction  and  their  perceived  level  of  knowledge. 

The  factors  that  influence  faculty  to  adopt  technology  in  this  were  rated  and  ranked.  The  top  two 
influences  to  the  adoption  of  instructional  technology  in  this  survey  are  instructional  issues  - improved  student 
learning  (96.8%  important  to  critically  important),  and  clear  advantages  over  traditional  methods  (96.1% 
important  to  critically  important).  Equipment  availability  (94.9%  important  to  critically  important)  ranked  third 
in  this  study.  Increased  student  interest  (93.6%  important  to  critically  important)  was  ranked  fourth.  This 
suggests  that  the  faculty  perceive  that  technology  must  have  a significant  impact  on  instruction  for  faculty  to 
adopt  its  use  even  though  the  technology  must  be  available. 

Equipment  factors  that  ranked  high  were  ease  of  use  (93%  important  to  critically  important),  time  to  learn 
technology  (91%  important  to  critically  important),  and  training  (86.6%  important  to  critically  important).  The 
high  rankings  also  reflect  critical  factors  for  faculty,  which  perceive  that  time  to  learn  technology  and  training  in 
technology  are  essential.  Personal  comfort  level  (78.2%  important  to  critically  important),  even  though  above  a 
majority,  does  not  seem  to  have  a major  impact  compared  to  pedagogy  and  hardware  issues. 

The  second  lowest  rated  influence  with  positive  ranking  was  administrative  support  with  80.1%  rating  it 
important  to  critically  important.  As  an  influence,  faculty  perceive  administrative  support  technology  as 
necessary  to  instructional  technology  but  not  necessarily  critical.  Least  important  of  the  influences  to  adoption 
in  this  study  was  technology  use  by  colleagues.  It  seems  the  technology  that  a colleague  is  utilizing  in  class  does 
not  sway  the  respondents  to  adopt  similar  technologies.  This  was  the  only  influence  that  faculty  rated  negatively 
with  36.5%  reporting  that  it  was  not  important  and  another  26.9%  saying  it  was  only  sorhewhat  important 
(Table  1). 
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Table  1.  Factors  influencing  use  of  technology 


FACTORS  INFLUENCING  USE  OF  TECHNOLOGY  * 

UWG 

UWG  rank 

Groves  & Zemel 

G & Z rank 

Improved  student  learning 

97 

1 

97 

2 

Advantage  over  traditional  teaching 

96 

2 

92 

4 

Equipment  availability 

95 

3 

97 

1 

Increased  student  interest 

94 

4 

96 

3 

Ease  of  use 

93 

5 

91 

5 

Compatibility  with  discipline 

93 

6 

86 

8 

Time  needed  to  learn 

91 

7 

91 

6 

Materials  in  discipline 

90 

8 

83 

9 

Compatibility  with  materials 

90 

9 

77 

12 

Training 

87 

10 

87 

7 

Administrative  support 

80 

11 

80 

11 

Personal  comfort 

78 

12 

80 

10 

Colleague  use 

13 

13 

35 

13 

* very  importantly  to  critically  important 

Issues  of  influences  and  barriers  are  not  only  linked  but  in  many  cases  are  opposite  sides  to  the  certain 
factors.  The  top  three  barriers  were  equipment-oriented  issues  that  also  ranked  high  as  influences  - lack  of  time 
(87.4%  important  to  critically  important),  lack  of  equipment  (76%  important  to  critically  important),  and  lack  of 
training  (75.8%  important  to  critically  important).  These  factors  reflected  their  comparable  influences  in  their 
importance  to  faculty.  Even  if  technology  can  have  a positive  impact  on  student  learning,  the  faculty  member 
must  overcome  these  personal  barriers  in  order  to  integrate  instructional  technology  into  their  instructional 
processes. 

Three  factors  that  received  strong  negative  ratings  as  barriers  were  lack  of  interest  in  technology  (70.4% 
not  important  to  somewhat  important),  lack  of  relevance  to  the  discipline  (65%  not  important  to  somewhat 
important),  and,  surprisingly,  lack  of  contribution  to  professional  development  (61.4%  not  important  to 
somewhat  important).  With  pedagogical  issues  ranking  as  such  strong  influences,  faculty  seem  to  be  saying  that 
the  student  is  the  focus  and  not  the  teacher. 

Influences  and  barriers  are  also  linked  to  knowledge  of  technology.  Not  surprisingly,  this  study  showed 
word  processing  is  known  almost  universally.  Ninty-three  percent  of  the  respondents  ranked  their  knowledge 
good  to  expert.  Email  ranked  second  in  this  study  (88.4%). 

Internet  use  for  research  and  content  support  also  was  strong  in  third  rank  with  70.1%  good  to  expert 
knowledge.  Presentation  software  and  multimedia  use  ranked  near  the  bottom  in  this  survey.  In  this  survey 
presentation  software  rated  37.5%  good  to  expert  and  multimedia  rated  35.1%.  Since  multimedia/presentation 
equipment  is  being  installed  rapidly,  the  lower  ranking  of  this  knowledge  creates  a challenge  for  technology 
trainers. 

A disappointing  lack  of  knowledge  in  distance  learning  and  computer  conferencing  is  not  unexpected  but 
causes  concern  in  the  current  environment  that  has  state-wide  projects  focused  on  the  development  of  distance 
learning.  While  23.5%  in  this  study  said  their  knowledge  is  good  to  expert,  another  58.8%  said  they  know  little 
to  nothing  about  distance  learning.  Computer  conferencing,  which  could  be  defined  as  a part  of  distance 
learning,  was  ranked  last  in  this  study  with  21 .6%  good  to  expert  and  58.6%  none  to  a little.  (Table  2). 
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The  faculty’s  knowledge  of  technology  is  reflected  by  its  use  of  technology  in  the  classroom.  Just  as  word 
processing  and  email  were  ranked  at  the  top  in  knowledge,  they  were  also  ranked  at  the  top  in  usage.  Word 
processing  was  frequently  used  by  93%,  Email  communication  with  colleagues  ranked  second  at  86,6% 
followed  by  email  communication  with  students  at  61,8%,  Presentation  software  and  multimedia  usage  also 
mirrored  its  knowledge  ratings.  Thirty-eight  percent  said  they  use  presentation  software  weekly  to  almost  every 
class.  Multimedia  usage  was  30,7%, 

Sixty-one  percent  of  respondents  said  they  used  the  Internet  weekly  to  almost  every  class.  For  this 
institution  and  the  state  university  system,  this  reflects  the  importance  that  has  been  placed  on  the  development 
of  virtual  libraries  and  a continued  development  of  on-line  resources  by  the  state  of  Georgia,  The  National 
Survey  of  Information  Technology  in  Higher  Education  showed  33%  of  classes  used  the  Internet  as  an  aspect  of 
the  syllabus  (Green,  1998), 

Just  as  in  a similar  survey  by  Groves  and  Zemel,  distance  learning  use  is  ranked  low.  Just  over  10%  used  it 
almost  every  class,  but  68,8%  said  they  never  use  distance  learning.  Because  email  with  students  can  be  an 
aspect  of  distance  learning,  it  may  be  that  faculty  have  a limited  definition  of  distance  learning.  Questions  on 
future  surveys  should  be  designed  differently  to  reflect  use  of  the  components  of  distance  learning  and  if  those 
components  are  for  distance  learning  or  on-campus  classes. 

When  asked  what  technologies  they  use  in  class,  faculty  included  word  processing,  email  and  Internet  as 
the  top  ranked  technologies.  When  asked  what  technology  they  would  like  to  use,  faculty  in  this  study  said 
multimedia  (27%)  and  presentation  software  (13%),  When  asked  if  they  would  adopt  a new  technology  in  the 
coming  year,  61%  (n  = 92)  of  the  faculty  respondents  said  it  was  very  likely  to  highly  likely.  Only  6%  of  the 
respondents  said  it  was  not  at  all  likely  that  they  would  adopt  a new  technology,  Spotts  and  Bowman  (1995) 
reported  22%  were  very  likely  to  highly  likely  to  adopt  new  technology  and  26%  said  it  was  not  likely  that  they 
would  adopt  a new  technology.  This  seems  to  reflect  the  growth  of  technology  in  society  as  a whole. 

How  significant  is  instructional  technology  perceived  to  be?  More  than  half  of  faculty  in  this  survey 
(53.2%)  said  technology  is  essential  to  instruction  with  83  respondents  noting  it  is  very  important  or  critically 
important.  Another  24.4%  (n  = 38)  said  it  is,  at  the  least,  important.  These  results  were  higher  than  reported  by 
Grove  and  Zemel  who  found  that  46%  said  it  was  very  important  or  critically  important,  Spotts  and  Bowman 
(1995)  reported  38%  rated  instructional  technology  very  important  or  critically  important,  A majority  of  the 
faculty  also  said  there  were  incentives  to  the  adoption  of  technology  (65,9%),  but  it  seems  they  have  a variety  of 
ideas  about  the  definition  of  incentives. 

In  conclusion,  the  higher  knowledge  and  use  results  in  this  survey  as  compared  to  previous  surveys  could 
be  explained  in  various  ways.  The  environment  at  this  state  university  has  become  more  conducive  to  the 
adoption  of  technology.  An  Associate  Professor  in  the  Library  stated  in  the  survey,  "It  is  obvious  that  the 
climate  on  this  campus  rewards  those  who  appear,  at  least,  to  be  innovative,"  There  is  strong  administrative 
support  for  technology.  The  President  and  Vice  President  of  Academic  Affairs  are  active  in  the  use  of  new 
instructional  technology,  supportive  of  technology  through  funding  and  have  encouraged  the  adoption  of  a 
technology  plan  which  has  been  created  by  a faculty/technology  staff  committee.  Faculty  have  stated  that 
equipment  availability  is  important  and  there  has  been  an  on-going  development  of  campus  networks  as  well  as 
video  and  multimedia  development  in  the  classrooms.  Two  deans  have  committed  significant  funds  to  the 
development  of  multimedia.  One  college’s  classrooms  are  all  multimedia  and  the  other  has  almost  all 
multimedia  classrooms.  Faculty  have  said  training  is  important  and  there  have  been  extensive  training 
programs.  There  is  a strong  distance  learning  department  and  recent  development  of  fully  on-line  programs. 

When  this  survey  is  next  administered,  there  are  a number  of  elements  that  need  to  be  redesigned  to  fit  this 
institution,  its  technology,  and  its  programs.  Technologies  will  be  defined  to  reflect  more  accurately  what  is 
being  utilized.  The  extensive  use  of  WebCT  is  not  mentioned  in  this  survey.  Distance  learning  needs  to  be  more 
appropriately  defined  for  the  university.  Several  additional  factors  also  need  to  be  explored  such  as  the 
technology  environment,  availability  of  technology  and  training,  and  administrative  support. 

The  influences  and  barriers  to  technology  adoption  define  its  use  and  possible  incentives.  As  the 
administration  and  technology  support  staff  continue  to  improve  the  environment  by  funding  new  technology, 
and  training  teachers,  faculty  will  adopt,  adapt,  and  integrate  new  technology.  The  faculty  at  this  institution  is 
young  and  most  have  technology  in  their  homes.  Technology  will  continue  to  evolve  and  become  easier  to  use. 
Faculty  and  students  will  accept  and  adapt  new  technology  to  instruction  and  learning  as  they  have  the  overhead 
projector.  But  this  adoption  won’t  take  the  25  years  it  took  to  bring  the  overhead  projector  from  the  bowling 
alley  to  the  classroom  (Gilbert  & Green,  1997)  because  the  technology  wave  has  already  swept  across  the 
private  sector  and  our  society.  Students  will  expect  instructional  technology  when  they  attend  post  secondary 
education  and  it  must  delivered  or  else  they  will  go  to  an  institution  that  will  provide  the  technology. 
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Multimedia/Multiliteracy:  The  multimedia  arts  education  program 


J.  David  Betts,  University  of  Arizona,  USA 


Introduction 

At  the  heart  of  the  Multimedia  Arts  Education  Program  (MAEC)  is  a strong  mix 
of  literacy  activities  from  the  several  arts  disciplines  and  techniques.  This  is  a descriptive 
study  of  this  after  school  arts  technology  program  for  middle  school  youth.  This  paper 
describes  the  first  year  of  an  on-going  longitudinal  research  project  that  will  examine  the 
effects  of  on  high  school  success,  graduation,  and  career  aspirations.  Data  gathered  in 
this  phase  will  serve  as  part  of  the  base-line  data  for  that  extended  study. 

MAEC  is  an  intensive  computer-mediated  arts  technology  program  begun  in  1996 
by  the  Tucson  Pima  Arts  Council  (TPAC).  Middle  school  youth,  targeted  as  being  at  risk 
from  disadvantaged  families,  participate  in  a series  of  semester-long  after  school  classes. 
The  program  is  based  on  several  literacies.  Central  to  all  the  activities  are  written  output 
from  students  consisting  of  journals,  proposals,  and  descriptive  abstracts,  as  well  as 
poems  and  stories  that  are  realized  in  video,  computer-mediated  animation  or  graphics. 
Students  work  in  labs  with  professionals  in  computer  graphics  and  publishing,  language 
arts  and  word  processing,  computer  animation,  and  video  production.  A final,  fifth 
semester  portfolio  class  capitalizes  on  cumulative  skills  and  yields  a home  page  a 
newsletter  and  a multimedia  presentation.  This  study  examines  student  development  of 
literacies  and  computer  skills  as  they  effect  school  performance  and  prepare  them  for  the 
electronic  arts  workplace. 

Population 

Approximately  8-10  middle  school  students  were  enrolled  in  each  of  the  five  labs 
each  semester.  Most  were  Hispanic  and  bilingual  in  Spanish  to  some  extent.  Most  started 
the  program  during  their  6*  or  7*  grade.  A balance  of  male  and  female  students  was 
maintained  by  selection.  Families  must  qualify  for  the  free  or  reduced  lunch  program  at 
their  school  and  not  already  have  a computer  in  the  home.  Students  are  not  auditioned  for 
this  program. 

MAEC  youth  who  were  selected  based  on  factors  contributing  to  high-drop  out 
rates.  Of  the  44  students  in  the  Spring  1998  program,  all  but  two  were  on  free  or  reduced 
lunch  programs  at  their  school.  54%  were  bilingual  Spanish,  79%  Hispanic,  7%  Native 
American,  7%  African  American  and  7%  Anglo.  There  is  now  a waiting  list  of  interested 
students  and  families.  Most  live  in  neighborhoods  near  downtown  Tucson  with  low 
graduation  rates.  Students  are  required  to  maintain  a “c”  average  in  school  in  order  to 
continue  in  MAEP. 

Theory 

The  integration  of  art  into  education  facilitates  the  construction  of  new 
knowledge  by  young  people  based  on  this  mediation.  New  facts  are  engaged  with  new 
skills  in  new  media.  These  skills  call  forth  new  ideas  and  the  transmediation  of  those 
ideas  in  a reciprocating  spiral  of  learning  (Salomon,  1990).  The  artistic  content  of  the 
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actions  and  operations  of  the  MAEC  provides  a parallel  means  of  connection  or 
engagement  for  the  learner. 

Leland  and  Harste  (1994)  define  literacy  in  terms  of  transmediation,  or 
"movement  between  and  among  communication  systems"  (p.  340).  MAEC  provides 
many  opportunities  for  students  to  transmediate  between  sign  systems,  such  as  placing 
the  words  of  a poem  onto  an  illustration,  or  animating  drawn  characters  from  a 
storyboard.  The  computer  mediates  the  artistic  expression  of  the  student’s  new  ideas. 

Those  new  ideas  can  come  from  any  area  of  a child's  life,  including,  as  Moll 
(1992)  writes,  the  "funds  of  knowledge"  that  are  in  family  and  neighborhood.  The 
infusion  of  technology  into  the  students’  lives  at  the  MAEC,  and  ultimately  into  their 
families’  lives  has  an  effect  that  ricochets  among  the  several  domains  including  school. 
Eisner  (1994)  acknowledges  the  importance  of  connections  between  school  knowledge 
and  students'  lives  outside  of  the  academic  environment.  Transfer  of  learning  will  occur, 
Eisner  asserts,  when  students  encounter  tasks  and  relationships  similar  to  those  outside  of 
school. 

We  can  see  that  the  MAEC  activities  are  complex,  and  as  they  are  in  transition 
into  adolescence,  their  influence  on  their  home  and  community  is  changing.  In  many 
cases  we  see  evidence  of  their  new  learning  having  an  effect  on  their  family. 

The  Multimedia  Arts  Education  Center 

The  MAEC  views  the  arts  as  a critical  factor  in  the  mastery  of  computer  and 
media  technology.  The  MAEC  teachers  are  professionals  in  their  area  of  art  technology, 
each  with  several  years  teaching  experience.  The  large  TPAC  building,  with  several 
computer  labs,  also  includes  an  art  gallery.  Many  local  artists  pass  through  and  influence 
the  culture  of  the  facility. 

The  curricula  used  in  the  labs  each  include  art  and  design,  technology,  and  literacy 
components.  Students  learn  several  professional  level  application  programs  and  tools. 
They  learn  to  make  assthetic  considerations  in  design,  to  develop  a critical  eye  and  to 
revise.  They  keep  journals  and  write  proposals  for  their  projects  and  create  storyboards, 
video  logs  and  scripts,  as  well  as  poems  and  stories. 

The  Language  Arts  lab  students  do  word  processing  and  basic  desktop  publishing. 
There  is  an  emphasis  on  basic  English  grammar  and  composition.  The  Computer 
Graphics  lab  students  learn  to  do  computer  photo-manipulation,  drawing  and  printing. 
Various  projects  such  as  letterhead,  calendars  and  logos  involve  integration  of  many 
electronic  arts  tools.  The  Animation  students  develop  narrative  storyboards  and  two- 
dimensional  computer  animations.  The  Video  lab  has  several  camera  kits  with  lights  and 
microphones,  editors,  and  a special  effects  generator.  Students  learn  to  use  this 
equipment  and  basic  production  techniques  as  they  work  through  group  and  individual 
projects.  The  Portfolio  lab  emphasis  is  on  developing  multimedia  presentations  based  on 
their  work  from  previous  semesters. 

The  students  receive  a small  stipend  ($25)  twice  each  semester  upon  attaining  the 
required  skill  level  benchmarks  and  completion  of  their  individual  projects.  When  they 
complete  of  the  program,  each  student  receives  a desktop  computer  of  his  or  her  own. 
This  has  been  a very  motivating  aspect  of  the  program,  as  most  of  the  families  involved 
are  not  able  to  have  a computer  at  home. 
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Methodology 

This  longitudinal  study  of  the  MAEC  activity  utilizes  several  strands  of  data  for 
analysis  of  the  program  in  its  context,  taking  into  account  its  history  and  the  history  of  its 
stakeholders,  the  students,  faculty  and  staff.  A coordination  of  qualitative  and 
quantitative  data  gathering  techniques  will  yield  a picture  of  the  nature  and  dynamics  of 
this  program.  When  we  know  how  the  students  did  finally  in  their  high  school  careers, 
hopefully  we  will  also  know  something  of  what  they  brought  to  the  program  and  what 
they  took  from  it,  what  their  family  lives  were  like  and  how  it  was  related  to  the 
individual  and  group  story.  We  know  now  that  all  the  program  graduates  are  still  in 
school,  the  first  will  graduate  next  year  (2000).  This  study  should  contribute  to  the 
program’s  continuance  because  it  represents  an  effort  to  monitor  and  be  accountable  for 
its  continued  improvement  and  ability  to  adapt  to  new  needs  and  affordances. 

Research  questions 

This  longitudinal  study  is  designed  to  track:  a)  students’  success  in  high  school;  b) 
the  impact  of  the  program  on  extended  families;  and  c)  the  impact  of  the  program  on 
students’  college  and/or  career  goals. 

In  addition,  the  study  will  look  at  a)  the  development  of  the  MAEC,  its  curricula 
and  management;  b)  the  development  of  students’  perceived  self-efficacy  and  attitudes 
about  art  technology;  c)  literacy  skills  development;  d)  evidence  of  aesthetic  response; 
and  e)  school-to-work  skills  acquisition. 

Exit  interviews 

At  the  end  of  each  semester,  this  investigator  interviewed  the  students  who  were 
about  to  graduate.  They  were  each  asked  the  following  questions,  among  others:  “Are 
you  feeling  like  you  accomplished  something  here  in  five  semesters?”  “What  project  do 
you  remember  the  best,  the  most,  what  stands  out?”  “How  is  your  family  with  this  whole 
activity?  Do  you  get  a lot  of  support?” 

It  seemed  that  the  students  we  have  interviewed  so  far,  who  have  all  completed 
five  semesters  of  the  MAEC,  were  pleased  to  have  made  it,  to  be  graduating.  They  were 
proud  of  what  they  had  accomplished  and  each  had  some  aspect  of  their  multimedia  work 
that  they  were  pleased  to  recall  and  describe.  They  saw  it  as  having  been  substantially 
different  from  their  school  experience.  Most  felt  that  the  language  arts  lab  was  the  most 
helpful  in  their  schoolwork.  There  was  a wide  range  of  plans  for  after  high  school.  All 
saw  high  school  graduation  as  a clear  and  attainable  goal.  They  had  few  complaints  or 
suggestions,  other  than  the  snacks.  A few  of  the  more  technologically  advanced  wished 
there  had  been  more  computer  hardware  and  software  for  their  use.  They  often  reported 
that  their  family  was  proud  of  their  accomplishment.  The  new  computer  that  they  were 
anticipating  would  have  a place  in  their  home  and  they  were  already  fielding  questions  as 
to  access  by  family  members.  The  goals  they  had  for  themselves  and  the  program  were 
generally  similar  and  related  to  new  computer  abilities  and  arts  experiences.  They  were 
somewhat  motivated  by  the  educational  incentive  payments,  several  put  these  checks 
away  for  software  or  a printer  to  use  with  their  new  computer. 

Parent  interviews 
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These  interviews  were  conducted  shortly  after  the  students  graduated.  Parents 
will  be  interviewed  again  before  their  child  graduates  from  high  school.  Some  of  the 
questions  asked  in  these  interviews  were:  “What  do  you  think  was  the  most  important 
thing  in  your  child's  participation  in  MAEC?”  “What  did  the  program  mean  to  you,  as  a 
parent?” 

At  the  end  of  each  interview,  parents  were  asked  to  respond  to  some  scaled 
questions  that  helped  them  assign  values  to  various  aspects  of  the  program.  Such  as, 
“How  much  did  the  multimedia  program  help  your  child  with  his/her  self-esteem?”  And  “ 
How  much  did  the  multimedia  program  help  your  child  with  his/her  writing  skill? 

Results  from  these  interviews  showed  a high  degree  of  satisfaction  and  parental 
involvement  the  program.  Parents  played  an  important  role  in  helping  their  children  meet 
the  long-term  commitment  to  MAEP.  Parents  said  that  the  fact  that  their  child  completed 
the  program  was  the  most  important  aspect  for  them.  Many  said  that  it  helped  them  grow 
up.  Several  parents  remarked  about  the  job-related  skills  that  were  required. 

Most  were  unfamiliar  with  the  art  technology  that  lies  at  the  heart  of  the  program,  but 
appreciated  the  computer  skills  the  students  acquired. 

Participant  observation 

Over  first  school  year  of  the  study,  university  students  worked  as  teaching 
assistants  in  the  labs.  They  were  asked  to  record  and  submit  their  observations  of  the 
activities  randomly  twice  a week.  Their  duties  were  to  assist  the  artist/teachers  with  their 
teaching  and  help  the  students  with  their  projects.  They  also  worked  with  the  teachers  in 
formalizing  the  learning  objectives  and  curriculum  for  each  lab. 

Skill  tests 

These  brief  tests,  given  at  the  beginning  and  end  of  each  semester  in  each  lab, 
were  instituted  to  help  the  artist-teachers  develop  a scope  and  sequence  for  their  arts 
technology  lab.  Basically,  each  teacher  developed  a series  of  questions  based  on  what  the 
students  will  need  to  learn  to  meet  the  requirements  of  the  lab. 

Questionnaires 

Perceived  self-efficacy  and  attitude  questionnaires  were  administered  several 
times  over  the  course  of  the  first  few  years  of  the  program.  These  were  designed  to  elicit 
student  attitudes  and  perceived  self-efficacy.  Five  of  the  items  were  language  arts  skills 
designed  to  determine  if  any  individuals  were  coming  into  the  program  with  serious 
deficits  in  language.  The  questionnaires  were  given  several  times  during  the  course  of 
each  individual’s  program. 

Perceived  self-efficacy 

The  perceived  self-efficacy  (Bandura,  1993)  items  covered  a variety  of  activities  in 
literacy,  art,  technology,  and  design,  as  well  as  social  abilities  such  as  collaboration  and 
communication.  Items  addressed: 

1 . Technology-based  applications 

2.  Creativity  and  design 

3.  Communication 

4.  Writing 
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5.  Arts  awareness 

6.  Reading 

Attitudes 

Another  group  of  items  on  the  questionnaire  dealt  with  student  attitudes  regarding 
work,  school,  art,  community,  and  the  program.  Items  addressed: 

1 . Sense  of  community 

2.  Personal  goals 

3 . Real  world  perceptions 

Two  of  the  items  were  more  or  less  open-ended  responses  about  their 
expectations  for  the  program  and  their  plans  for  after  high  school.  Preliminary  results 
indicate  that  the  majority  had  goals  of  college  and  a successful  career  in  mind  and  they 
felt  that  the  program  would  help  them  attain  those  goals.  These  responses  will  be 
tabulated  and  further  analyzed  for  language  use. 
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Abstract;  Several  standards  like  the  Learning  Technology  Systems  Architecture  (LTSA) 
or  the  Instructional  Management  Systems  Project  (IMS)  provide  a format  to  interchange, 
reuse,  and  combine  learning  contents.  They  do  not  cover  didactical  methods  in  detail. 
Therefore  we  developed  an  approach  to  specify  methods  as  a part  of  the  description  of 
learning  sequences.  This  representation  and  selection  of  learning  and  teaching  methods  is 
an  integral  part  of  the  Essen  Learning  Model  (ELM),  a generic  development  model 
supporting  developers,  educators,  and  users  on  different  levels  of  educational  activities. 
We  show  how  to  use  structured  teaching  processes  as  a guideline  to  implement  learning 
environments. 


Introduction 

Due  to  the  enormous  amount  of  developments  in  the  field  of  educational  technologies,  a variety  of 
approaches,  architectures,  and  systems  have  emerged  in  the  last  decades.  Recent  educational  concepts  like 
Virtual  Universities  changed  the  research  focus  from  single  user,  local  architectures,  to  a multi-user, 
globally  distributed  architecture.  The  need  to  combine  and  integrate  different  systems  has  led  to  several 
standardization  projects,  determining  standard  architectures  and  formats  for  learning  environments,  such  as 
Learning  Technology  Systems  Architecture  (Farance,  Tonkel,  99)  or  Instructional  Management  Systems 
Project  (EDUCAUSE,  99).  One  of  the  main  issues  is  to  standardize  of  learning  contents  in  order  to  make 
those  contents  available  for  integration,  e.g.,  in  a Virtual  University.  Less  attention  has  been  devoted  to  the 
standardization  of  learning  and  teaching  methods.  When  choosing  a course  for  the  integration  into  a 
curriculum,  the  decision  depends  on  the  one  hand  on  the  content,  on  the  other  hand,  the  educator  has  to 
consider  the  teaching  method  used  for  a certain  content.  Therefore  we  focused  on  the  representation  of 
learning  and  teaching  methods  in  order  to  develop  a generic  description.  First  of  all,  this  description  acts  as 
a decision  tool  when  integrating  courses,  secondly  it  guides  the  author  implementing  a course  or  a unit. 

The  representation  and  selection  of  learning  and  teaching  methods  is  an  integral  part  of  the  Essen  Learning 
Model,  a generic  development  model  supporting  developers,  educator,  and  users  on  different  levels  of 
educational  activities  (Pawlowski,  2000). 

In  the  following  we  point  out  the  need  of  a standard  representation  of  methods.  The  representation  of 
learning  processes  was  modeled  in  ARIS  (Scheer,  98a),  a standard  modeling  tool.  Our  implementation  used 
this  representation,  but  other  process  modeling  tools  could  be  used  as  well.  We  briefly  describe  ARIS,  the 
implemented  methods,  the  method  selection  process  in  the  Essen  Learning  Model,  and  present  a sample 
implementation  from  our  university. 

Essen  Learning  Model  (ELM) 

The  Essen  Learning  Model  was  developed  at  the  University  of  Essen.  It  is  currently  being  evaluated  to  be 
used  in  different  departments  of  our  university.  We  will  describe  the  design  principles  and  the  general  flow 
of  the  approach. 

The  Essen  Learning  Model  is  a modular  system  (Fig.  1),  supporting  development  processes  as  well  as  the 
system’s  use  system  on  different  levels:  The  support  of  curriculum  design  (C-level),  the  development  of 
learning  sequences  (D-level),  and  the  development  of  learning  units  (E-level).  We  distinguish  between 
three  abstraction  levels:  the  generic  development  model  provides  knowledge  for  a variety  of  contexts.  This 
generic  model  is  customized  depending  on  the  user’s  needs  and  preferences,  and  transformed  into  a specific 
process  model  for  each  project.  The  process  model  provides  a framework  for  educational  technology 
projects.  The  third  level  is  the  result  of  the  development  process  in  form  of  certain  implementations  for 
each  module.  The  result  of  ELM-C  is  a detailed  network  of  learning  objectives  and  goals.  The  structure  and 
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Fig.  1:  The  Essen  Learning  Model 


each  module.  The  result  of  ELM*C  is  a detailed  network  of  learning  objectives  and  goals.  The  structure  and 
relations  of  learning  sequences  (courses)  are  determined.  Based  on  these  results,  learning  sequences  are 
developed  in  ELM-D.  The  focus  of  this  phase  is  to  find  an  adequate  didactical  method  including  the  right 
technology  depending  on  learning  objectives  and  user  group.  The  selection  of  teaching  methods  which  is 
supported  in  ELM-D  is  modeled  in  ARIS  in  this  approach.  Finally,  single  learning  units  are  designed  and 
implemented  in  ELM-E. 


Modeling  Techniques:  ARIS 

Modeling  of  processes  is  very  complex.  One  approach  is  to  use  the  Architecture  of  Integrated  Information 
Systems  (ARIS).  This  architecture  is  applicable  for  every  type  of  (business)  process:  in  manufacturing,  in 
the  service  industry,  and  in  the  public  services  (Scheer,  98a).  We  will  show  that  teaching  methods, 
respectively  teaching/leaming  processes,  can  be  described  as  process  models  as  well. 

ARIS  is  a frame  concept  for  a global  description  (modeling)  of  computer  supported  information  systems 
covering  the  whole  life-cycle  range:  from  business  process  design  to  information  technology  deployment 
(Scheer,  98b).  In  order  to  reduce  the  complexity,  the  process  model  is  divided  into  individual  views  that 
represent  discrete  design  aspects  and  can  be  handled  (largely)  independently  which  simplifies  the  task.  The 
ARIS  concept  distinguishes  four  views:  organization  view,  data  view,  function  view,  and  control  view,  the 
latter  is  used  to  establish  the  relationships  between  the  components  (Scheer,  98a). 

The  processes  are  modeled  using  the  ARIS-Toolset,  supporting  the  user  in  modeling,  analyzing,  and 
navigating  through  the  business  processes.  It  utilizes  the  event-driven  process  chain  method  (EPC).  The 
model  of  event-driven  process  chains  (EPCM)  can  be  described  as  a model  based  on  graphs  which  consist 
of  vertices  and  edges.  Furthermore  it  is  possible  to  add  several  attributes  to  an  EPC,  like,  e.g.,  the  duration 
of  a process,  etc.  (Keller,  Teufel,  97).  A detailed  presentation  of  the  elements  of  an  event-driven  process 
chain  is  described  in  (Keller,  Teufel,  97). 


Standardized  Representation  and  Selection  of  Methods 

Learning  sequences  are  typically  described  by  certain  attributes,  characterizing  the  contents,  learning 
objectives,  and  sometimes  learning  materials.  These  definitions  are  being  standardized  in  the  Learning 
Object  Metadata  Model  by  the  Learning  Technology  Standards  Committee  of  IEEE  (LTSC)  (Farance, 
Tonkel,  99)  and  the  Instructional  Management  Systems  Project  (EDUCAUSE,  99).  The  goal  of  this 
standardizationis  to  develop  a format  for  interchange,  reuse,  and  combination  of  learning  contents.  An 
important  criteria  for  choosing  an  alternative  from  a variety  of  learning  sequences  is  the  didactical  method 
used  for  specific  learning  contents.  The  above-mentioned  standard  approaches  do  not  cover  a detailed 
description  of  didactical  methods.  Usually  only  informal  information  is  added,  which  does  not  cover  all 
aspects  of  a method  and  its  usability  for  certain  contents. 

Therefore,  we  developed  an  approach  to  specify  methods  as  a part  of  the  description  of  learning  sequences. 
Learning  sequences  are  represented  as  process  models  using  ARIS.  These  models  cover  the  main  activities, 
interactions,  information  systems,  and  persons  involved.  Besides  the  specification  aspect,  it  helps  the 
author,  teacher,  and  learner  to  navigate  through  a learning  sequence.  It  also  provides  the  opportunity  to 
easily  combine  and  reuse  learning  sequences  or  parts  of  a sequence.  The  approach  is  not  limited  to  ARIS  as 
a modeling  tool.  Other  specifications  like  IDEF  (Adelsberger,  Komer,  95)  or  UML  (Booch,  Jacobson, 
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Rumbaugh,  98)  can  be  integrated  easily,  A further  benefit  is  the  option  to  integrate  business  processes  and 
learning  processes  in  order  to  close  the  traditional  gap  between  operations  and  education  (Adelsberger, 
Komer,  Pawlowski,  98a,  Adelsberger,  Komer,  Pawlowski,  98b). 

In  the  following,  we  will  present  the  representation  and  selection  of  didactical  methods.  Furthermore,  we 
will  show  a sample  implementation.  A learning  sequence  on  “Simulation  in  SIMAN”  was  developed  for 
students  of  Business  Computing,  following  the  Essen  Learning  Model.  The  Learning  processes  were 
modeled  using  ARJS,  the  learning  environment  was  implemented  in  XML  (W3C,  98). 

Representation 

Within  the  scope  of  our  research  activities  (Adelsberger,  Bick,  Kraus,  Pawlowski,  99)  we  developed  and 
implemented  a computer  supported  simulation  game  at  the  University  of  Essen  for  a learning  sequence  on 
“Simulation  in  SIMAN''.  Using  the  phase  model  of  Kolb,  the  modeling  of  this  teaching  method  was 
implemented  as  an  EPC.  Thus  the  EPC  can  serve  as  a guideline  for  creating  simulation  games. 

A simulation  game  is  an  active  teaching  method  for  processing  and  solving  practical  problems  by  one  or 
more  teams.  It  allows  experimental,  competitive  learning  (action  learning).  The  essential  pedagogical  goal 
of  the  simulation  game  method  is  to  increase  the  students’  thinking  flexibility  (Geuting,  89). 


Fig.  2:  Kolb’s  learning  cycle  and  the  phases  of  simulation  games  (Ruohomaki,  95) 

The  learning  cycle,  based  on  the  theory  of  experiential  learning,  is  commonly  accepted  for  simulation 
games  (Graf,  89,  Ruohomaki,  95,  Adelsberger,  Bick,  Kraus,  Pawlowski,  99)  and  has  four  related  stages 
(Fig.  2):  Concrete  Experiences,  Reflective  Observation,  Abstract  Conceptualization,  and  Active 
Experimentation . 

Learning  is  understood  as  a continuous  process  whereby  knowledge  is  created  from  experience.  The 
cyclical  model  is  best  to  be  used  in  simulation  games  for  education  and  training  since  they  are  a type  of 
“controlled  experience”  which  introduces  the  process  mentioned  above  (Ruohomaki,  95). 

It  is  common  to  view  playing  simulation  games  as  a three-phase  process,  involving  Briefing,  Activity,  and 
Debriefing,  These  phases  of  the  simulation  game  and  the  experiential  learning  cycle  are  closely  linked.  The 
learning  cycle  starts  with  a Briefing,  The  participants  are  generally  introduced  in  the  game  and  its 
surroundings,  followed  by  activities  and  Concrete  Experiences,  The  next  phase  of  the  learning  cycle  offers 
an  opportunity  to  reflect  and  observe  experiences  from  many  perspectives,  e.g.,  by  sharing  reactions  and 
observations  in  discussion  with  other  participants.  Abstract  Conceptualization  is  the  process  in  which  the 
participants  create  concepts  that  transform  observations  into  logical  theories.  The  following  Active 
Experimentation  could  be  used  to  develop  hypotheses  of  reality  and  to  test  them  during  the  game.  Different 
kinds  of  solutions  can  be  tested  and  probable  consequences  can  be  seen.  These  solutions  can  then  be 
corrected,  modified  and  retested.  Debriefing  is  the  process  which  examines  and  discusses  the  experience  of 
the  simulation  game  and  thus  turns  it  into  learning. 


Selection 
The  selection 
Course,  ends 
Evaluation  in 


of  a teaching  method  is  supported  by  a generic  process  (Fig.  3).  It  starts  with  the  event  New 
with  the  event  Well  Trained  Student,  and  has  the  processes  Select  Method,  Activity  and 
between. 
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Fig.  3:  Generic  learning  process 
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The  process  Select  Method  (Fig.  4)  describes  the  sequence  of  selecting  a method  focus,  as  well  as  the 
selection  of  a teaching  method.  The  selection  of  the  teaching  method  is  being  controlled  by  attributes  of  the 
function  Select  Teaching  Method, 

After  choosing  the  teaching  method,  the  process  Activity  is  initiated  (Fig.  4).  In  this  approach  an  exemplary 
description  of  the  process  Activity  for  a simulation  game  is  shown. 

After  that  an  evaluation  of  the  learning  process  follows,  both  the  moderator  and  the  students  are 
participating.  The  evaluation  method’s  selection  is  supported  by  the  Knowledge  Base  and  the  User  Model. 

The  development  process,  supported  by  ELM,  is  based  on  an  Architecture  of  Computer  Supported 
Learning  Environments  (Adelsberger,  Komer,  Pawlowski,  98a,  Adelsberger,  Komer,  Pawlowski,  98b). 
This  architecture  consists  of  the  components:  Learning  Engine,  Knowledge  Base,  Presentation  Component, 
Communication  Component,  User  Model,  Evaluation  Component,  and  Methods  Base. 

The  Methods  Base,  used  for  a method’s  selection,  contains  different  didactical  methods  and  concepts  which 
have  been  modeled  with  ARJS.  The  moderator/author  can  use  these  structured  EPC  concepts  to  teach 
different  kinds  of  learning  contents.  Traditional  static  learning  environments  are  usually  based  on  a single 
learning  concept.  Recent  developments  adapt  methods  depending  on  user  and  content  characteristics. 

Furthermore,  the  above-mentioned  process  of  selection  is  supported  by  the  components  Knowlegde  Base 
and  User  Model  (Fig.  5).  The  Knowledge  Base  contains  learning  objectives  and  contents,  conceptual 
modeled  in  ARIS.  The  knowledge  representation  is  implemented  in  XML.  The  User  Model  contains 
attributes,  characteristics,  and  knowledge  of  a user.  Ideally,  the  knowledge  is  represented  adequately,  based 
on  the  User  Model,  regarding  individual  learning  pace,  preferred  learning  method,  and  preferred 
presentation  format. 


Fig.  5:  Methods  Base 

The  attributes  contained  in  the  Knowledge  Base  and  the  User  Model  offer  the  author  a selection  of  teaching 
methods  by  means  of  an  inference  mechanism  enabling  to  choose  a method  according  to  his  preferences. 
The  author  is  supported  by  the  structured  EPCs  of  the  specific  methods,  the  given  teaching  materials,  and 
the  User  Model. 

The  developed  sample  implementation  of  a learning  sequence  on  “Simulation  in  SIMAN”  follows  the 
Essen  Learning  Model  (Fig.  6).  We  have  decided  to  use  the  simulation  game  teaching  method,  because  of 
the  learning  goals  in  simulation  are  so  complex,  the  students  have  to  be  trained  to  be  able  to  work  in  teams, 
and  our  department  focuses  on  application-oriented  education. 
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Fig.  6:  Learning  goals  example 

The  Activity  process  is  used  as  a guideline  for  the  implementation  of  the  simulation  game.  During  the 
development  process,  the  Presentation  Component,  the  Communication  Component,  and  the  Evaluation 
Component  were  specified.  The  complete  implementation  of  this  example  is  modeled  in  XML. 
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Abstract:  This  paper  discusses  some  issues  and  research  findings  of  the  project  we  call 
UNIBASE.  In  order  to  improve  the  learning  processes  through  hypermedia  technology, 
we  investigated  the  role  of  cognitive  mapping  and  the  application  of  certain  cognitive 
ambiguities  and  breakdowns.  The  addition  of  clear  learning  goals,  explicitly  expressed 
through  cognitive  mapping  tools,  allowed  students  to  acquire  the  right  knowledge  and 
helped  them  adopt  a productive  strategy.  Complementary  to  this  elicitation  process, 
video  information  embedded  in  hypermedia  learning  environments  was  used  to  increase 
the  levels  of  attention  and  motivation,  using  devices  such  as  movement,  novelty  and 
appeal.  Our  aims  were  to  explore  and  develop  the  application  of  innovative  hypermedia 
artefacts. 


1.  Introduction 

The  challenges  ahead  are  usually  identified  with  changes  in  how  the  learning  experience  is  delivered, 
changes  in  who  is  delivering  the  learning  experience,  and  changes  in  the  role  of  the  learner  as  part  of  a 
target  population.  Radical  changes  have  already  taken  place  in  the  latter  and  it  seems  reasonable  to 
acknowledge  that  a new  pedagogical  approach  is  needed  to  cope  with  the  arrival  of  a new  generation  of 
potential  students.  We  can  describe  this  change  as  a shift  from  a “drill  generation”  to  a “play  generation”. 
Somebody  already  called  this  new  target-group  the  “playstation”  generation. 

To  cope  with  these  radical  changes,  digital  technology  is  being  increasingly  used  to  create  new  learning 
artefacts.  At  the  heart  of  this  technology  is  the  "system”  perspective,  as  proposed  by  Davenport  (1997). 
Input  and  output  devices  serve  as  surrogate  extensions  of  our  senses,  providing  connectivity  to  the  world. 
Computers  and  networks  bear  some  correspondence  to  our  own  brains  and  nervous  systems.  When 
connected  together  into  a system,  these  devices  enable  the  digital  creation  of  systemic  worlds  and 
scenarios.  The  learner’s  involvement  becomes  active  and  conversational,  rather  than  passively  receptive, 
and  leads  to  the  co-construction  of  meaning. 

The  UNI  BASE  project*  aims  were  to  explore  and  develop  the  application  of  innovative  hypermedia 
artefacts.  This  paper  discusses  some  issues  and  research  findings  of  the  project.  The  research  activities 
and  design  experiments  have  been  carried  out  with  a group  of  16  students,  whose  task  was  to  prepare 
conceptual  maps  and  hyperspaces,  using  a particular  type  of  mapping  software^  with  editing  and 


* Designed  and  carried  out  by  Universidade  Aberta  (Open  University  in  Portugal)  and  the  Informatics 
Department  of  the  University  of  Lisbon,  Portugal. 

^ MindManager^^  (www.mindman.com) 
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communication  tools.  A group  of  instructional  designers  also  participated  in  the  design  of  an  experimental 
prototype  for  the  Web,  using  knowledge  mapping  in  the  same  way  as  the  students  did.  These  activities 
were  embedded  in  realistic  and  relevant  contexts  with  rich  audio-visual  representations  and  real-world 
data. 


2.  Facing  the  Problem 

What  learning  activities  can  be  supported  by  hypermedia  artefacts?  What  are  the  actual  cognitive  and 
affective  strengths  of  hypermedia?  How  do  we  design  for  this  medium?  Much  has  been  said  about  the 
potential  for  interactive  learning  sustained  by  the  “right”  design  of  hypermedia  in  systems  such  as  the 
World  Wide  Web.  But  when  designers  attempt  to  give  choices  to  the  user,  as  to  how  to  proceed,  options 
seem  often  to  be  unengaging  and  ineffective.  A clear  sense  of  the  experiential  and  reflective  modes 
(Norman,  1993)  that  should  support  such  design  efforts  is  not  attained  by  the  users/leamers.  However,  the 
human  mind  seems  to  work  like  the  World  Wide  Web,  in  a dynamic,  creative,  fractal  and  unpredictable 
way. 

The  problem  we  want  to  address  can  be  summarised  as  follows:  which  combination(s)  of  interaction 
processes,  authoring  functionality  and  synergy  between  complementary  elements  of  multimedia 
information  can  improve  learning  in  a learner-centred  educational  environment?  Once  an  author  or 
designer  abandons  the  notion  of  total  control  over  the  learning  material  and,  instead,  engages  the  audience 
in  a process  of  co-construction,  he  discovers  that  a more  fundamental  question  surfaces:  can  the  learning 
material  become  organic,  adaptive,  and  generative? 

Our  approach  has  favoured  the  mapping  of  knowledge  and  the  application  of  certain  cognitive 
ambiguities  and  breakdowns,  in  order  to  improve  the  learning  processes.  The  addition  of  clear  learning 
goals,  explicitly  expressed  through  cognitive  mapping  tools,  allowed  students  to  acquire  the  right 
knowledge  and  helped  them  adopt  a productive  strategy.  Complementary  to  this  elicitation  process,  video 
information  embedded  in  hypermedia  learning  environments  was  used  to  increase  the  levels  of  attention 
and  motivation,  using  devices  such  as  movement,  novelty  and  appeal. 


3.  The  use  of  Reference  Models 

An  adequate  usage  of  reference  or  mental  models,  for  instance  as  concept  maps  or  mind  maps,  stimulates 
and  facilitates  the  recognition  and  transfer  of  information.  Basically,  we  strive  to  develop  mental 
reference  models  that  are  meaningful  organisations  of  information  in  our  brains.  Adequate  usage  of  these 
reference  models  suggests  that  we  use,  for  example,  meaningful  sounds,  pictures  and  graphics  to  express 
ideas. 

Schade,  cited  by  Hoogeven  (1997),  argues  that  if  reference  images  are  added  to  text,  the  understanding  of 
a story  is  about  75%  faster  than  if  we  are  confronted  with  a text  only  story.  He  also  hypothesises  that  by 
using  reference  models,  innate  or  acquired  in  our  early  childhood,  text  and  picture  stories  are  stored  faster 
and  more  efficiently  in  our  long-term  memory  than  text-only  stories.  He  states  that  people  tend  to 
remember  25-35%  more  of  a text  and  pictures  story  than  of  a text  only  story. 

The  use  of  reference  models  is  not  exclusive  to  educational  practice;  it  is  a well-accepted  marketing 
technique  to  use  basic  reference  pictures  and  sounds  to  improve  advertisements  and  commercials.  Often 
family  scenes,  romantic  environments,  status  symbols,  attractive  women  and  responsible  men  are  weaned 
into  the  marketing  material.  We  assume  that  an  adequate  use  of  reference  models  is  an  important  variable 
with  regard  to  an  effective  learner-centred  design  process. 


4.  The  Role  of  Audio-Video 

The  integration  of  “raw”  audio-visual  information  with  the  learner’s  knowledge  structure  may  facilitate 
retention  and  retrieval.  Training  learners  "to  transform  complex  visual  images  (i.e.,  to  analyse,  describe, 
categorise  them)  can  provide  an  important  link  between  visual  and  verbal  storage  and  retrieval."  (Kozma, 
1986,  p.l6).  This  has  been  implemented  in  UNIBASE  by  means  of  cognitive  mapping  devices  that 
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support  the  integration  of  audio- video  segments.  The  final  result  could  be  easily  published  as  a 
hypermedia  structure. 

Hyperlinks  could  be  mouse-clicked  by  the  learner  to  play  short  narrative  explanations  or  video  clips.  It 
was  recognised  by  students  and  designers  participating  in  our  experiments  that  one  of  video’s  greatest 
strengths  is  the  power  to  generate  attitudes  and  emotions  as  no  other  medium  can.  According  to  Anderson 
(1983,  p.72):  “It  is  an  excellent  tool  for  displaying  affective  information.”  The  affective  domain  is 
concerned  with  the  learner’s  reception  and  responses  to  the  stimuli  presented.  Furthering  this  idea 
Azarmsa  (1996,  p.  39)  identifies  five  categories  that  describe  the  levels  of  this  domain;  (a)  receiving,  (b) 
responding,  (c)  valuing,  (d)  organisation  of  values,  and  (e)  internalisation,  which  is  characterised  by  the 
learner  holding  a particular  value  or  set  of  values. 

To  enhance  the  learner  involvement  in  the  initial  stages,  we  had  to  find  means  to  engage  the  learner  in 
novel  and  interesting  ways.  Rather  than  just  trying  to  simulate  existing  reality  on  the  computer  screen,  or 
trying  to  create  a “virtual  reality”,  we  found  that  it  is  far  more  sustainable  to  create  a new  experience  for 
the  viewer  with  simple  means.  In  the  past,  micro  worlds  have  been  created  to  provide  an  entirely  new 
framework  for  the  learner  to  explore,  but  today  the  Web  can  already  provide  plenty  of  worlds  to  interact 
with.  According  to  cognitive  flexibility  theory  (Spiro  and  Jengh,  1990),  learners  are  able  to  spontaneously 
restructure  their  knowledge  in  many  ways  as  they  chart  their  courses  through  the  hypermedia  material. 
So,  we  constructed  artefacts  - designated  knowledge  hyperscapes  (Bidarra  and  Guimaraes,  1 999)  initiated 
by  the  instructor  and  further  developed  by  the  students  in  an  organic,  adaptive  and  generative  manner  (see 
figure  1). 


Figure  L Map  representing  the  hypermedia  structure  of  the  site  produced. 
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5.  Hypervideo  Mechanisms  as  Authoring  and  Learning  Tools 


Constructivists  have  argued  strongly  for  the  need  of  authentic  learning  experiences  and  video  is  an 
appropriate  tool  to  support  this.  Video  clips  can  greatly  enhance  the  authenticity  of  a computer  based 
learning  environment  (Boyle,  1997).  However,  for  a system  to  allow  learner  reflection,  it  must  have  a 
compositional  and  representational  medium  that  affords  adding  new  representations,  modifying  and 
manipulating  old  ones,  and  perform  comparisons  (Norman,  1993).  Reflection  also  requires  the  time  and 
ability  to  elaborate  upon  and  compare  ideas.  The  medium  must  afford  the  time  for  reflection.  In  this 
sense,  broadcast  television  cannot  augment  human  reflection,  it  does  not  afford  composition  or  the  time  to 
reflect.  However,  television  and  video,  when  properly  constructed,  can  be  a powerful  tool  for  reflection. 
If  the  user  can  select  what  is  to  be  seen  and  control  the  pace  of  the  material,  and  it  is  easy  to  go  back  and 
forth,  to  stop,  to  make  annotations,  to  compare  and  to  relate  to  other  materials. 

Effective  reflection  requires  some  structure  and  organisation  (Norman,  1993).  Structuring  and 
organisation  of  information  is  the  main  issue  in  hypermedia.  However,  being  multimedia  is  not  enough 
for  a system  to  be  truly  hypermedia.  Different  media  can  be  used  purely  as  illustration  in  a system  where 
links  are  restricted  to  text.  Hypervideo  refers  to  the  integration  of  video  in  truly  hypermedia  documents. 
True  integration  of  video  requires  a more  powerful  hypermedia  model,  taking  into  account  its  spatial  and 
temporal  dimensions,  defining  the  semantic  structures  and  mechanisms  for  linking  video,  finding  new 
concepts  of  navigational  support,  and  taking  into  account  the  aesthetic  and  rhetoric  aspects  of  integrating 
several  media  in  hypermedia  (Chambel,  1999a). 

The  design  rationale  we  adopted  for  the  educational  artefacts  being  developed,  in  the  context  of  the 
UNIBASE  project,  is  based  on  the  notion  that  the  integration  of  basic  elements  of  information  must 
explore  the  cognitive  bias  of  the  different  materials  and  create  added  value  through  adequate  bridges 
between  those  elements.  Effective  integration  must  be  sought  by  allowing  the  learner  to  exercise  the 
"naturar  cognitive  attitude,  while  inducing  proactive  "breakdowns"  (Winograd,  1986)  that  trigger 
reflection  processes  (Norman,  1993).  On  a video  based  material,  such  as  the  one  we  are  proposing,  this 
means  letting  the  user  enjoy  and  absorb  the  video  information  in  very  much  the  same  way  as  it  is 
currently  "consumed"  in  the  television  type  of  interaction.  On  the  other  hand,  as  this  type  of  information 
processing  is  essentially  experiential,  the  hypervideo  environment  must  be  designed  to  involve  the  user  in 
such  a way  that  he  is  led  to  "stop,  think,  and  correlate"  different  types  of  information. 

Hypervideo  mechanisms  are  therefore  the  basic  tools  for  the  purposeful  and  directed  integration  of  video 
information  in  learning  environments  (Chambel,  1999b).  In  this  context,  we  developed  a model  and  some 
tools  for  hypervideo  support  on  the  Web  (Chambel,  1999a),  as  extensions  to  HTML  and  existing  Web 
tools,  that  are  being  used  to  integrate  video  in  the  UNIBASE  hypermedia  learning  environment. 


6.  Experience  with  Learners 

It  is  assumed  that  technologies  like  interactive  multimedia  help  students  to  form  rich  representations  of 
learning  events  and  promote  deeper  understandings.  However,  this  does  not  happen  by  simply  adding 
more  information  and  more  (rich)  media.  We  also  assume  that  students  learn  in  very  different  ways  and 
we  expect  them  to  be  able  to  recognise  relevant  domain  information  and  be  able  to  process  it  at  some 
depth. 

In  our  experiments,  learners  used  their  natural  language  to  argue,  question,  and  make  connections  to  other 
subjects,  other  units  of  study,  and  their  own  experience.  Many  explained  the  course  of  the  learned  events 
and  worked  out  a knowledge  map.  Some  learners  also  displayed  their  understandings  by  means  other  than 
words.  They  also  created  new  situations,  produced  images  and  short  videos  and  represented  the  theme  as 
a hypermedia  production.  Finally,  they  published  their  work  on  the  World  Wide  Web  (see 
http://www.univ-ab.pt/~bidarra/hyperscapes). 

The  experimental  set-up  was  based  on  a group  of  16  students  following  a course  on  video  pedagogy  and 
technology  in  a master's  programme  on  multimedia  communication.  Their  previous  experience  included 
teaching  for  at  least  one  year  and  Internet  common  usage  skills.  They  were  given  a basic  conceptual  map, 
with  relevant  knowledge  for  the  course,  which  they  had  to  explore  and  develop  further,  both  in  class  and 
online.  They  were  encouraged  to  proceed  from  (non-linear)  thinking  to  (non-linear)  authoring  of 
knowledge  hyperscapes. 


Our  experimental  goals  were: 

• to  examine  the  use  of  hypermedia  and  communication  tools  in  their  work; 

• to  investigate  the  effectiveness  of  the  hypermediaAVeb  environment  that  we  set  up; 

• to  find  ways  of  improving  the  usability  of  hypermedia  learning  interfaces. 

Data  on  the  interactions  was  gathered  by  means  of  direct  observation,  video  recordings,  screen  capture. 
These  were  later  interpreted  and  checked  against  the  evaluation  of  individual  portfolios.  Final  assessment 
of  the  students  was  accomplished  via  work  portfolios  and  a final  (individual  or  group)  project.  Overall, 
the  results  were  very  encouraging  when  compared  with  previous  years. 


7.  Conclusions 

The  approach  to  build  an  effective  learner-centered  system  was  to  collect  descriptive,  qualitative 
information  on  complex  real  work  settings.  This  required  research  tools  that  were  capable  of  examining 
interaction  with  the  hypermedia  environment  and  also  interaction  within  the  surrounding  social  system. 
The  gathered  data  was  to  be  later  interpreted  and  integrated  into  the  design  process. 

Our  findings  suggest  that: 

• User  control  and  interactivity  are  very  important.  Students  appreciated  the  freedom  to  access  and 
integrate  different  resources,  to  build  new  materials  as  hypermedia  productions,  and  to  respond 
to  challenges,  creating  new  situations  and  working  out  their  own  knowledge  maps. 

• The  ability  to  integrate  video  in  hyperdocuments  was  also  an  important  feature,  especially  in  the 
context  of  the  video  pedagogy  and  technology  course,  where  it  could  be  used  to  illustrate  and 
demonstrate  the  concepts  being  taught  and  learned,  in  an  integrated  manner. 

• Constructivist  activities  were  very  motivating  to  the  majority  of  students  and  MindManager^*^ 
software  was  easily  mastered.  Precise  focus  on  the  objectives  was  attained  by  workgroups 
without  wasting  time  “playing  around”  with  the  software. 

• We  found  that  user  acceptance  increases,  if  tools  for  interaction,  problem  formulation,  design 
and  system  building  are  supported  by  reliable  information  about  the  needs  and  desires  of 
potential  users  and  the  setting  in  which  systems  are  to  be  used. 

• However,  students  are  not  prepared  to  contribute  themselves  and  reflect  on  the  main  issues,  if 
there  is  no  previous  experience  with  the  software  tools  that  enable  it.  Introductory  information, 
preparation  classes  or  online  help  are  necessary. 

• Guidance  on  the  thematic  criss-crossing  of  the  subject  domain  is  a must.  Students  get  easily  lost 
if  there  is  no  tutor  or  expert  support,  no  matter  the  access  they  have  to  the  Internet  and  external 
knowledge  bases. 

The  formal  features  of  our  instructional  program,  especially  the  video  component,  could  be  seen  to  exert  a 
great  influence  on  the  affective  domain  of  the  learners.  However,  we  assumed  that  it  was  the 
responsibility  of  the  instructional  developer  to  stimulate  the  learner  to  such  a degree  that  higher  levels  of 
involvement  were  reached.  We  think  that  the  powerful  affective  attribute  of  video  needs  to  be  explored 
further. 
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Abstract:  This  proposal  outlines  research  on  the  comprehension  of  narrative  dynamic  multimedia 
such  as  video.  I describe  an  approach  to  investigating  comprehension  based  on  a synthesis  of 
methods  and  models  developed  in  classical  cognitive  studies  (ICintsch,  1998;  Frederiksen,  Donin, 
and  Bracewell,  1987;  Bracewell  and  Breuleux,  1994)  with  theoretical  work  in  cognitive  film 
analysis  (Bordwell,  1989)  and  film  semiotics  (Metz,  1968).  This  approach  provides  a principled 
argument  for  constructing  a filmic  text-base  (Kintsch,  1998)  that  serves  as  the  basis  for  studying  the 
situation  and  frame  models  used  by  viewers  to  understand  narrative  content  in  dynamic  multimedia. 
The  paper  applies  this  approach  in  a child’s  recall  task.  Discrepancies  between  the  child’s  protocol 
and  a text  derived  from  a shot  script  of  the  viewed  video  are  discussed  in  light  of  the  situation 
knowledge  needed  to  adequately  construct  a “problem/plot  frame”.  This  work  has  implication  for 
assessing  instructional  design  modules  that  include  video. 


Introduction 

Video,  whether  stand  alone  or  embedded  in  interactive  multimedia  has  become  a powerful  resource  in 
many  classroom  settings.  There  is  considerable  research  assessing  the  learning  occurring  with  video  delivered 
course  content,  such  as  Vanderbilt’s  Jasper  series.  However,  research  on  how  students  comprehend  dynamic 
multimedia  narratives  needs  more  attention.  Movies  and  video  are  such  a pervasive  part  of  K-12  culture  that  their 
comprehension  seems  almost  a naturally  acquired  skill.  The  mimetic,  iconic  illusions  in  most  films  and  videos 
children  view  lead  to  a perspective  of  movie  viewing  as  being  a simple  mapping  from  world  reality  to  screen 
diegeis.  But,  on  closer  look,  understanding  a “moving  narrative”  is  a cognitively  demanding  task. 

The  purpose  of  this  paper  is  2-fold:  to  gain  insight  into  the  cognitive  processes  of  film  comprehension  in 
recall;  to  provide  a principled  basis  for  applying  discourse  and  propositional  analysis  to  the  inherently  non-natural 
language  of  dynamic  multimedia.  My  research  is  based  on  a recall  task  in  which  a 1 0 year-  old  is  asked  to  “tell  me 
what  the  movie  was  about”.  The  movie  was  a 20-  minute  stop-action  animation,  “Wallace  and  Gromit:  a Grand  Day 
Out”.  Based  on  theories  of  comprehension  and  film  semiotics,  I show  how  a text  base  of  film  may  be  constructed 
and  accommodate  the  kind  of  scrutiny  for  comprehension  strategies  successfully  undertaken  in  classical  cognitive 
accounts  in  text  and  discourse.  Using  propositional  and  frame  analysis,  I show  how  the  child’s  construction  of 
meaning  may  be  represented  as  structured  around  narrative  and  plot  frames.  Two  text  bases  are  provided:  the  child’s 
recall  and  a shot  script  derived  from  repeated  viewing  of  the  video.  I compare  the  propositions  and  plot  frames 
derived  from  an  episode  in  the  two  text  bases  and  discuss  the  results  in  light  of  summarizing  strategies  and  situation 
knowledge. 

Narrative  film  comprehension  is  generally  under  researched.  Chafe’s  (1980)  discourse  analysis  of  film 
recall  production  in  children  and  adults  yielded  interesting  insights  into  comprehension.  However,  his  research 
failed  to  account  for  the  filmic  units  of  analysis;  his  work  failed  to  articulate  on  what  basis  imagery  and  time  were 
conflated  into  declarative  knowledge.  In  the  field  of  cognitive  film  studies,  Bordwell  (1989)  has  used  script-based 
knowledge  representation  to  develop  his  own  model  of  film  interpretation.  Bordwell  theorizes  that  critics  adopt 
particular  scripts  or  schemas  and  apply  them  to  understand  the  connotations  in  film.  As  Bordwell  states,  however, 
there  may  be  different  mechanisms  involved  in  the  expert  critic’s  interpretation  of  a film  than  in  the  casual  viewer’s 
more  immediate  comprehension  process. 

This  study  is  based  on  Kintsch’s  model  (1998)  of  the  comprehension  process  as  constructed  in  a loosely 
structured,  bottom  up  process  which  only  gradually  integrates  knowledge  into  a more  stable  set  or  mental 
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representation.  Kintsch  differentiates  comprehension  knowledge  into  a text  base  and  situation  model  with  the  text 
base  representing  the  knowledge  constructed  from  the  text  itself  and  the  situation  model  representing  the  frames, 
scripts,  or  models  derived  from  the  readers/listener  knowledge  of  similar  texts,  situations,  and  world  knowledge. 
Kintsch  has  applied  his  model  to  many  comprehension  tasks  yet  admits  that  most  of  his  work  in  narrative  discourse 
focused  on  single  situation  problems.  Frederiksen,  based  on  a similar  comprehension  model,  has  explicitly  probed 
texts  and  discourse  for  multiple  situations  or,  in  his  model,  multiple  frame  construction.  Frederiksen’s  model  focuses 
on  the  representation  of  comprehension  in  both  text  (1987  ) and  discourse  (1998),  as  well  as  being  applied  to  text 
production  situations  (Bracewell,  and  Breuleux,  1994).  The  model  identifies  three  types  of  representational  units: 
natural  language  units  (e.g.  clauses,  words);  units  of  propositional  meaning  (e.g.,  events,  states,  processes);  units  of 
conceptual  meaning  (related  structures  in  which  concepts  are  associated  to  form  conceptual  knowledge  as  in  frames 
and  situation  models.  The  basic  unit  in  these  studies  has  been  the  proposition. 

Current  cognitive  inquiry  into  comprehension  is  based  on  a fundamental,  if  not  entirely  uncontroversial 
assumption  that  comprehension  knowledge  may  be  represented  at  the  semantic  level  as  propositions.  Kintsch  points 
out  the  alternative  views  that  point  to  the  non-linguistic  representation  of  mental  imagery  ( Kosslyn,  1994).  But  even 
Kosslyn  used  spatial  predicates  to  represent  the  deep  knowledge  level  of  surface  pictorial  forms.  Without  attempting 
to  finalize  the  debate  between  linguistic,  spatial,  or  symbolic  representation  of  visual  media  such  as  movies,  I 
proceed  cautiously  with  Kintsch ’s  basic  assumption.  In  order  to  find  an  analogous  representational  structure  to 
language,  I looked  to  film  semiotics,  specifically  as  developed  by  Metz  (1968). 

Metz  is  adamant  that  while  film  has  expressive  and  communicative  features,  it  cannot  be  considered  as 
having  a syntactical  system.  As  in  art,  there  is  a tight  coupling  between  sign  and  signified.  Film  comprehension 
proceeds  from  “blocks  of  reality”  which  are  actualized  with  their  total  meaning  in  discourse  or  in  the  filmic 
presentation.  According  to  Metz,  these  blocks  are  shots,  either  individual  (“autonomous”)  or  in  sequence,  which 
semantically  relate  to  one  another.  Metz’s  ambitious  “Grand  Syntegma”  constructs  a hierarchical  film  structure 
organized  around  shots  and  sequences. 

Agreeing  with  Kintch  that  propositions  may  adequately  represent  comprehension  knowledge  even  in  non- 
linguistic  phenomenon,  the  syntegcmatic  framework  suggests  that  comprehension  may  proceed  from  the 
autonomous  or  simple  sequence.  A main  idea  or  event  is  presented  in  these  syntegemes,  and  associated  with 
subsequent  syntegemes  into  a hierarchical  structure  of  sequences  and  episodes.  In  essence,  the  shot  or  simple 
sequence  may  function  as  a clause  in  comprehension.  Individual  syntegemes  cohere  similarly  to  Kintch’s 
microrules,  whereas  macrorules  relate  events  to  sequences  to  episodes.  With  this  assumption,  a propositional 
representation  may  be  derived  from  a filmic  Text’. 

Method 

This  study  is  based  on  the  recall  protocol  of  a 10  year-  old  girl  who  was  asked  to  “tell  me  what  the  movie 
was  about”.  The  purpose  of  the  research  was  to  see  what  summarizing  strategies  she  used,  and  to  compare  her 
understanding  to  a text-base  derived  from  a shot  script  constructed  from  the  researcher’s  viewing  of  the  film. 

The  movie  was  a 20-  minute,  stop  animation  video  by  Nick  Park,  “Wallace  and  Gromit,  A Grand  Day 
Out”.  The  child  had  seen  the  film  previously  as  well  as  his  other  animations  in  the  series.  She  was  therefore  exposed 
to  his  filmic  style,  the  characters,  and  the  narrative  genre.  The  researcher’s  protocol  was  derived  from  constructing  a 
“shooting  script”  while  viewing  and  reviewing  the  movie.  Both  texts  were  segmented  into  clauses  analyzed  using 
propositional  analysis  as  similarly  developed  and  applied  in  Frederiksen,  Donin-Frederiksen,  and  Bracewell  (1987). 
A narrative  frame  and  plot  frame  is  mapped  on  to  the  respective  propositions  to  reveal  the  conceptual  structures 
present  in  the  child’s  and  researcher’s  knowledge.  The  study  focuses  on  a particular  episode  in  the  final  act  of  the 
movie,  in  which  the  main  characters  are  discovered  by  a robot  on  the  moon  and  rush  to  return  home. 

Findings 

The  frames  were  compared  for  frame  completeness  and  accuracy,  particularly  on  the  marking  of  events, 
themes,  temporal  ordering,  and  causal  and  conditional  links.  The  child  correctly  and  completely  recalled  the  plot 
structure  but  had  difficulty  in  identifying  and  ordering  the  plot  themes  and  events. 
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Discussion 


Deviations  between  the  child’s  and  researcher’s  comprehension  as  compared  in  their  respective 
construction  of  the  narrative  and  plot  frames  are  discussed  in  light  of  summarizing  strategies  and  situation 
knowledge.  According  to  Kintsch,  summarizing  is  a form  of  recall  which,  like  other  types  of  comprehension,  is 
governed  by  macrorules.  These  rules  stipulate  the  selection  of  key  events  or  objects,  the  generalization  to  a more 
abstract  version,  and  the  construction  of  a new  ‘text’  (vanDiJk  and  Kintsch,  1 983).  Thus,  while  the  child  had  little 
problem  in  identifying  this  sub-plot  as  being  significant  to  the  main  characters  - and  correctly  selecting  this  in  her 
recall  - and  subsequently  generalizing  to  a global  problem-goal-plan-action-reponse  structure,  her  accounting  of  the 
events  reveal  gaps  which  would  have  required  more  knowledge  of  the  technology  depicted  in  the  film.  As  an 
example  of  multi-situation  comprehension,  this  study  suggests  that  developers  and  users  of  narrative  dynamic 
multimedia  must  identify  frame  knowledge  needed  to  comprehend  the  content  and  relate  it  to  the  knowledge  of  their 
viewers. 

Conclusion 

Understanding  how  students  comprehend  narrative  dynamic  visual  media  was  the  direct  objective  of  this 
paper.  Providing  a rationale  for  inferring  a ‘text’  from  dynamic  visual  media  became  necessary  in  order  to  apply 
comprehension  theories  and  methods  successfully  used  in  text  and  verbal  discourse.  A synthesis  between 
comprehension  models  and  methods  and  film  semiotics  furnished  a basis  in  which  to  approach  the  topic.  This 
appraoch  may  be  useful  in  assessing  and  developing  multimedia  projects  for  children. 

References 

Bordwell,  D.  (1989).  Making  meaning:  inference  and  rhetoric  in  the  interpretation  of  cinema.  Cambridge,  MA: 
Harvard  University  Press 

Bracewell,  R.J.,  Breuleux,  A.  (1994).  Substance  and  romance  in  the  analysis  of  think-aloud  protocols.  In  P. 
Smagorinsky  (Ed),  Speaking  about  writing:  reflections  on  research  methodology,  (pp.  55-88).  Newbury  Park,  CA: 
Sage 

Chafe,  W.  (Ed).  (1980).  The  Pear  stories:  cognitive,  cultural,  and  linguistic  aspects  of  narrative  production. 
Norwood,  NJ:  Ablex 

Frederiksen,  C.  and  Donin,  J.  (in  press)  Conceptual  Processing  in  text  comprehension  and  learning  through  situated 
discourse 

Frederiksen,  C.  Donin-Frederiksen,  J.  and  Bracewell,  R.  (1987).  Discourse  analysis  of  children’s  text  production.  In 
Matsuhashi,  A.  (Ed.)  Writing  in  real  time:  modeling  production  processes,  (pp. 254-290).  Norwood,  NJ:  Ablex 

Kintsch,  W.  (1998).  Comprehension:  a paradigm  for  cognition.  NY;  Cambridge  University  Press 

Kosslyn,  S.  (1994).  Image  and  brain  : the  resolution  of  the  imagery  debate.  Cambridge,  MA:  MIT  Press 

Metz,  C.  (1968)  Film  language:  some  points  in  the  semiotics  of  the  cinema.  In  Braudy,  L.  and  Cohen,  M.  (Eds.). 
1999.  Film  theory  and  criticism:  introductory  readings  (Fifth  edition).  NY:  Oxford  University  Press 

vanDiJk,  T.  and  Kintsch,  W.  (1983).  Strategies  of  discourse  comprehension.  NY:  Academic  Press 


136 


Integrating  ARCS  Motivational  Strategies  into  CALL  Courseware  for  Holistic  Practice:  A Study 


Isabel  Borras 

San  Diego  State  University 
United  States 
borras@mai  1 . sdsu . edu 

Maria  Coronel 
San  Diego  State  University 
United  States 
mcoronel  @mai  1 . sdsu.  edu 


Abstract:  This  study  investigated  the  effects  on  writing  performance  of  three  levels  of  integration  (basic,  medium, 
full)  of  ARCS  motivational  conditions  (Attention,  Relevance,  Confidence,  and  Satisfaction)  into  a CALL 
(Computer- Assisted  Language  Learning)  program  for  holistic  foreign  language  practice.  It  also  addressed  the  effects 
of  two  degrees  of  relevance  (low,  high  of  the  learning  tasks.  Forty-eight  undergraduate  students,  randomly  assigned 
to  six  treatments,  accomplished  two  writing  activities  using  one  of  the  six  versions  of  the  program.  The  ARCS  full 
integration  groups  performed  significantly  better  than  did  the  medium  and  basic  groups.  The  degree  of  task 
relevance  did  not  affect  significantly  performance.  Students  felt  that  the  program  gave  them,  in  this  order, 
confidence,  attention,  relevance,  and  satisfaction.  Motivational  strategy  and  task  relevance  impacted  significantly 
the  intensity  and  persistence  of  students’  learning  effort,  as  revealed  by  five  sets  of  data  traced  by  the  program's 
tracking  tools:  Intro  Time,  Activity  Time,  Help  Time,  Help  Frequency,  and  Overall  Frequency  of  Access.  Writing 
draft  analysis  uncovered  the  effects  of  three  writing- fostering  features  of  the  program  on  subjects'  use  of  "practice," 
"resourcing"  and  "output  structuring"  strategies. 


Introduction 

Motivational  factors  arc  an  important  mediator  of  performance  and  are  considered  instrumental  in  the  expenditure  of  cognitive 
effort  (Bandura,  1977,  Salomon,  1983).  An  instructional  design  model  that  addresses  motivation  is  the  ARCS  model  (Keller, 
1987).  The  model,  which  has  been  applied  to  courseware  design  (Keller  and  Suzuki,  1987),  includes  subcategories  of 
motivational  characteristics  and  examples  of  strategies  that  stimulate  or  sustain  each  motivational  condition  "Attention," 
"Relevance,"  "Confidence"  and  "Satisfaction." 

Instructional  technology  (IT)  research  has  addressed,  with  mixed  results,  the  ways  different  ARCS  strategies 
embedded  in  instructional  materials  affect  learner’s  motivation  and  performance.  Some  studies  have  found  that  confidence- 
enhanced  materials  augment  learner’s  assuredness  (Naime-Diefenbanch,  1991)  or  that  relevance-enhancing  strategies  result  in 
enhanced  motivation  and  performance  (Means,  Jonassen,  and  Dwyer,  1997).  Other  studies,  however,  have  indicated  no  effects 
of  confidence-enhanced  materials  on  learners'  achievement  or  confidence  (Moller,  1993).  Means  et  al.  (1997)  contend  that  this 
imparity  of  effects  is  due  to  the  fact  that  the  ARCS  conditions  are  interdependent  and  that  it  is  not  clear  how  motivational 
enhancement  affect  cognitive  activity. 

Motivation  is  also  an  issue  in  Second  Language  Acquisition  (SLA).  Ushioda  (1993)  sustains  that  motivational 
outcomes  such  as  learner’s  participation,  attentiveness,  and  enthusiasm  should  be  viewed  as  significant  by  themselves,  apart 
from  their  particular  relevance  to  second  language  development.  Spolsky  (1989),  in  his  theory  of  second  language  learning, 
asserts  that  language  knowledge  and  skills  in  the  future  are  the  result  of  a number  of  factors  including  knowledge  in  the 
present,  abilities,  motivation/affect,  and  opportunity.  Spolsky  sustains  that  learner’s  acquisition  depends  on  the  various 
strengths  of  these  conditions  and  that  the  greater  the  amount  of  each,  the  greater  the  learning.  Research  is  needed  to 
understand  how  to  build  up  and  combine  the  Spolsky’s  factors  that  allow  intervention,  motivation  and  opportunity,  within 
language  learning  environments  and,  particularly,  within  those  supported  by  technology. 

On  the  other  hand,  IT  and  SLA  have  been  equally  interested  in  the  learning  and  attitudinal  effects  of  tasks  that  are 
inherently  relevant  to  the  learner.  Research  from  both  fields  suggest  that  perceptions  of  learning  task  meaningfulness  influence 
learner’s  "cognitive  capacity  usage"  (Salomon,  1993;  Pokay  & Blumenfeld,  1990),  and  that  perceptions  of  enjoyableness  of 
language  learning  tasks  correlate  with  learning  effectiveness  (Green,  1993).  However,  intentional  cognition  is  not  always  based 
on  the  task’s  interest.  Means,  et  al.'s  (1997)  assertion  that  "many  learners  in  undergraduate  courses  are  not  intrinsically 
motivated  to  learn  course  content"  (p.  15)  is  all  the  more  true  for  learners  in  foreign  language  courses  since  cultural  context  and 
background  may  affect  their  perceived  relevance  of  second  language  tasks  (Green,  1993).  Thus,  research  is  also  needed  on  how 
to  design  Computer- Assisted  Language  Learning  (CALL)  programs  whose  manipulated  motivational  strengths  affect 
performance  and  attitude  beyond,  or  regardless  of,  the  inherent  motivational  value  of  the  programs'  instructional  tasks. 


Purpose  of  the  Study 
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Within  the  above  framework,  this  study  investigated  if  the  comprehensive  integration  of  ARCS  motivational  strategies  in  CALL 
courseware  for  holistic  practice  (Wyatt,  1987),  affected  the  writing  performance  in  a foreign  language  of  undergraduate  college 
students. 

The  study  equally  explored  the  effects  on  writing  performance  of  language  tasks  relevant  to  students'  interest,  the 
potential  interactions  between  motivational  strategy  integration  and  task  relevance,  the  effects  of  motivational  strategy 
integration  on  attitudes,  and  the  correlation  between  attitudes  and  performance. 

Additionally,  in  accordance  with  "contextualized"  research  tenets  (Salomon,  1991),  the  study  investigated  a number 
of  issues  with  a potential  impact  on  the  main  outcomes.  These  included  the  impact  of  courseware  motivational  strategy 
integration  and  task  relevance  on  learner's  intensity  and  persistency  of  effort  (Keller,  1987)  and  the  effects  of  courseware 
writing-fostering  features  on  subjects'  use  of  "practice,"  "resourcing"  and  "output  structuring"  cognitive  strategies. 


Method 

Subjects 

Forty-eight  lower-division  undergraduate  students,  28  women  and  20  men,  enrolled  in  the  various  sections  of  an  intermediate 
Spanish  course  during  the  spring  semester  at  San  Diego  State  University  constituted  the  experimental  sample.  Students  were 
randomly  assigned  to  the  six  experimental  treatments  of  the  study,  which  involved  completing  two  writing  tasks  in  Spanish 
using  a CALL  program.  The  subjects  had  quite  heterogeneous  academic  backgrounds  and  their  academic  skills  were  mostly 
limited  to  word-processing. 

Variables 


The  main  independent  variables  of  this  study  were  motivational  strategy  integration  and  task  relevance.  The  motivational 
strategy  variable  consisted  of  the  levels  of  integration,  basic,  medium  and  full,  of  the  ARCS  motivational  strategies  found  in  the 
six  versions  of  an  original  courseware  package IfW/mg  in  Spanish:  Two  Practice  Sessions  (WS)  developed  for  the  study 
(Borrds,  1999). 


[1]  From  the  "leader  of  activity"  viewpoint,  CALL  software  is  routinely  classified  as  tutorial,  holistic  practice,  and  games. 
Basically,  the  holistic  practice  variety  provides  "higher-level,  contextualized,  practice  activities,"  "idea  processor"  and  "on-line 
thesaurus"  (Wyatt,  1987:  87). 

The  WS  versions  featured  different  number  of  screens.  To  the  five  screens  of  the  basic  versions  (Title,  Menu, 
Introduction,  Activity,  and  Credits),  the  medium  versions  added  one  (Help),  and  the  full  versions  two  (Help  and  Archives). 
Common  to  all  versions,  the  Activity  screen  included  a four-page  notepad  for  writing  the  drafting  and  final  renditions  of  the 
tasks,  a toggling  writing  prompt,  on-demand  access  to  contextualized  multimedia  resources,  and  users'  input  locking 
capabilities.  Tracking  tools  available  in  all  versions  allowed  the  self-saving  of  user's  writing  input  and  navigational  paths  data 
on  to  an  automatically  created  folder  located  on  the  hard  disks  of  the  computers  used  in  the  study. 

As  shown  in  Figure  1,  the  basic  level  of  strategy  integration  consisted  of  the  message  found  in  the  Introduction 
screens  of  the  six  versions  of  WS.  Elements  of  the  message  included  a statement  of  the  program's  goal,  three  comments  on  the 
task's  nature,  benefits  and  resources,  and  a warming  up  expression.  The  elements,  which  fell  into  the  motivational  strategies 
categories,  "Experience,"  "Present  Worth,"  "Learning  Requirements,"  and  "Positive  Outcomes"  (Keller's,  1987),  were  aimed  at 
reaching  three  ARCS'  motivational  conditions,  relevance,  confidence,  and  satisfaction. 


138 


3ClBaWS/n.KMBNT5/SU?FOET:  mATBCrKS/CCMOmONS 

OTTlODUCTDir-  j 

hifodmiaymnmgt: 

• pxo$rixnfs  god 

• Ktivityinitun 
btndits.  tndnaeiutti  j 

• «nceurts«ttuxU  - 

Experience  'i  Fete»«nce 

Present  worth  f 

Leinung Requstmenti  Contaderre 

Poiitive  Outcome  » Sitisliction 

ACTIVirf 

"oelt:  dgmtiimSinmtttl 

• Wrtii^pwmptl . Output  JtrurtiiiiEti 
. Mottpid  >. 

- I . 

\t 

^ 

• 

r 

1 T 

KBU 

Pog^/lddr. 

• Ktivltptunction 

■ itmptai  torumtttiDn  I 

■ IritiatoRnBT^tdta  1 

■ erg  titsiiis  taints  f 

■ coatectuiUsel  vet  tbulix^t 

■ y^inununliniiee  J 

1 Certtaet  '| 

1 Cexxretinisi^  •>  Attention 

! Vuistnkty  j 

NeedMitchng  »>  Fotavtne# 

Ixpec  titaons  ^ Corfldeit  e 

ABCMIVSS 

■ eoR«ctiM  Itedbaekea  1 
pnvton  ■cUTtty  J 

Unexpected  Fewiniri  ^ Settalection 

Positive  Outcome  i* 

Figure  1.  WS  "Basic,"  "Medium,"  and  "Full"  Integration  Levels  of  ARCS  Motivational  Strategies 

The  medium  level  of  strategy  integration  included  the  introductory  component  and  the  contents  of  a Help  screen 
common  to  four  versions  of  WS.  The  screen  included  information  on  the  function  of  the  assigned  task,  and  a number  of  macro- 
aides  (samples  for  imitating,  brainstorming,  and/or  organizing  the  task)  and  micro-aides  (audio-visual  monolingual  vocabulary 
lists,  and  grammar  reference).  These  contents  embodied  such  motivational  strategies  as  "Conflict,"  Concreteness, 
"Variability,"  "Need  Matching,"  and  "Expectations"  and  targeted  the  motivational  conditions  attention,  relevance,  and 
confidence. 

The  high  level  of  strategy  integration  comprised  the  introductory  and  help  components  together  with  the  Archives 
screen  common  to  two  versions  of  WS.  The  screen,  which  provided  access,  through  a metaphorical  button,  to  the  instructor's 
corrective  feedback  on  the  writing  activities  subjects  accomplished  during  the  first  experimental  session,  addressed 
motivational  strategies  like  "Unexpected  Rewards"  and  "Positive  Outcomes"  in  order  to  attain  the  motivational  condition 
satisfaction. 

The  second  independent  variable,  task  relevance  degree  (high,  little),  was  based  on  the  subjects’  higher  or  lower 
interest  in  the  writing  tasks  of  the  WS  program.  Taking  into  account  Crookes  and  Schmidts  (1991)  assertion  that  a program 
which  appears  to  meet  the  students'  own  expressed  needs ...  will  be  more  motivating,  more  efficient,  and  thus  more  successful." 
(p.  492),  the  selection  of  the  tasks  drew  on  the  results  of  a survey  conducted  one  month  before  the  experiment  with  1 19 
students  of  Intermediate  Spanish.  Surveyees  were  asked  to  rank  from  1 (least  liked)  to  10  (most  liked)  the  ten  authentic  writing 
activities  of  a Task  Relevance  questionnaire.  Based  on  the  results  of  the  survey,  four  activities,  two  of  them  "highly  relevant" 
and  two  others  "little  relevant,"  were  selected  to  support  subjects'  wnting  practice  with  WS  during  two  experimental  sessions. 

The  primary  dependent  variable  of  the  study,  writing  communicative  performance,  was  defined  as  achievement  in 
conveying  information  in  writing  (narration,  description,  persuasion,  and  argumentation).  The  second  dependent  variable, 
attitudes,  was  aimed  at  measuring  the  reactions  of  subjects  toward  the  computerized  writing  experience,  and  to  find  out  which 
were  the  features  of  the  WS  package  that  according  to  subjects'  perceptions  contributed  the  most  to  the  activation  of  the  ARCS' 
motivational  conditions. 

Other  variables  including  learner's  intensity  and  persistency  of  effort,  and  learner's  use  of  particular  strategies 
during  writing  practice  were  also  considered  to  better  understand  where  and  how  WS  was  strong  or  weak  in  producing  the 
desired  motivational  and  cognitive  outcomes. 

Treatments 


Using  a two-way  3x2  factorial  design,  subjects  were  randomly  assigned  to  one  of  the  six  treatments  resulting  from  the 
combination  of  the  levels  of  motivational  strategy  integration  (basic  vs  medium  vs  full)  with  the  degree  of  relevance  of  the 
writing  task  (little  v5  high).  The  treatments  included  T1  (basic  motivational  strategy/little  relevant  task);  T2  (basic  motivational 
strategy/highly  relevant  task);  T3  (medium  motivational  strategy/little  relevant  task);  V4  (medium  motivational  strategy/highly 
relevant  task);  T5  (full  motivational  strategy/little  relevant  task);  and  T6  (full  motivational  strategy/highly  relevant  task). 
Depending  on  their  assignment  to  a given  treatment,  subjects  used  one  of  the  six  versions  of  iheiVnting  in  Spanish  package. 
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Instruments 


The  main  dependent  variable,  writing  performance,  was  computed  by  applying  Gaudiani's  Composition  Analytical  Scales 
(1981)  to  the  writing  samples  produced  by  the  subjects  using  theWS  package.  The  Scales  allow  the  separate  scoring  of  four 
composition  features  including  grammar/vocabulary,  style,  organization,  and  content. 

To  measure  the  secondary  dependant  variable,  attitudes,  three  slightly  modified  versions  of  Keller’s  (1993) 
Instructional  Materials  Motivational  Survey  (IMMS)  were  used  to  collect  attitudinal  data  from  the  subjects  working  with  WS 
under  the  basic  (T1-T2),  medium  (T3-T4)  or  full  (T5-T6)  treatments.  The  IMMS  (Cronbach’s  alpha  reliability  coefficient  = 
.96)  asks  the  survey-takers  to  assess  the  degree  of  truthfulness  of  its  36  statements  using  a 5-point  Likert  scale  (1=  not  true;  5 = 
very  true).  In  the  IMMS  adaptations,  Al,  A2,  and  A3,  statements  27  and  28  were  replaced  by  two  others  targeting  subjects’ 
attitudes  toward  the  motivational  strategies  embedded  in  the  Introduction,  Help,  and  Archives  screens  ofWS.  Also,  statement 
29  was  removed  because  of  its  tangential  pertinence. 

Procedure 

The  WS  program  was  aimed  at  providing,  in  varying  degrees,  Spolsky's  (1989)  “motivation”  and  "opportunity"  conditions  for 
language  learning  by  exposing  learners  to  a diversity  of  tasks  and  sources  that  promoted  their  use  of  receptive  and  productive 
language  skills  (Krashen  & Terrell,  1983).  Thus,  using  the  six  versions  of  WS,  subjects  in  the  different  treatments  performed 
the  variety  of  tasks  involved  in  the  writing  activities  they  completed  during  the  two  experimental  sessions  (thereafter,  SI  and 
S2). 

In  SI,  subjects  working  with  the  basic  versions  of  WS  accomplished  a number  of  tasks  including  reading  the 
message  of  the  Introduction  screen,  reading  the  writing  prompt  and  exploring  the  multimedia  resources  of  the  Activity  screen, 
and  writing  the  draft  and  final  versions  of  the  experimental  activity.  Furthermore,  subjects  in  the  medium  and  full  motivational 
strategy  integration  treatments  explored  the  macro-aides  and  micro-aides  of  the  Help  screen  available  in  the  WS  versions  they 
were  using. 

In  S2,  subjects  accomplished  a number  of  activities  similar  to  the  ones  they  did  in  the  first  session.  In  addition, 
subjects  in  the  basic  strategy  integration  treatments  could  revisit  the  Activity  screen  of  the  first  session  to  explore  its  resources 
and  what  they  had  written  in  its  notepad.  Subjects  in  the  medium  strategy  integration  treatments  could  access  both  the  Activity 
and  the  Help  screens  of  the  WS  version  they  worked  with  in  the  first  session.  And  subjects  in  the  full  strategy  integration 
treatments  were  able  to  access  not  only  the  Activity  and  the  Help  screens  of  the  version  used  in  the  previous  session  but  also 
the  Archives  screen  with  the  instructor's  feedback  on  their  first  writing  sample.  At  the  end  of  S2,  subjects  filled  in  one  of  the 
three  adaptations  of  the  IMMS  (Al,  A2,  or  A3),  depending  on  the  treatment  basic,  medium,  or  full  to  which  they  were 
assigned. 

Subjects  were  encouraged  to  accomplish  the  activities  as  well  as  possible.  In  order  to  keep  performance's  standards 
equally  high  in  both  sessions,  it  was  not  until  that  the  experiment  was  concluded  that  students  were  told  that  the  first  of  the 
sessions  had  just  a training  purpose.  Implementation  of  the  experiment  extended  over  fourteen  days  and  took  about  one 
hundred  and  forty  hours. 

Throughout  the  sessions,  seven  sets  of  data  were  unobtrusively  traced  by  the  WS'  tracking  tools  for  weighing  the 
intensity  and  persistency  of  students'  effort  using  the  package.  These  included  the  overall  time  subjects  spent  on  the  entire 
program,  the  time  they  spent  on  the  program's  Intro,  Activity,  and  Help  sections,  and  the  frequency  with  which  they  accessed 
the  entire  program  and  the  Help  and  Intro  sections. 


Results 


Pcrfonuance  Outcomes 


The  effects  and  interactions  of  the  independent  variables  motivational  strategy  integration  and  task  relevance  were  analyzed 
for  the  composite  writing  performance  scores  using  analysis  of  variance  (ANOVA).  The  means  and  standard  deviations  for 
performance  scores  for  each  treatment  group  in  S2  showed  that  the  highest  mean  score  across  treatments  was  obtained  by  the 
T5  subjects  (3. 19)  —the  maximum  possible  score  being  4.00. 

Statistically  significant  effects  were  found  for  the  independent  variable  of  primary  interest,  motivational  strategy 
integration  (F  (2,  42)  = 3.844,  p<  .05).  Results  of  Tukey-Honestly  Significant  Difference  (HSD)  post-hoc  tests  demonstrated 
that  in  S2  the  highly  motivational  strategy  integration/highly  and  little  relevant  task  combinations  were  statistically  different 
from  both  the  medium  motivational  strategy  integration/highly  and  little  relevant  task  conditions  {HSD  (3,  42)  = 4.53,/?  < .05) 
and  the  low  motivational  strategy  integration/highly  and  little  relevant  task  conditions  {HSD  (3, 42)  = 4.90,/?  <.05). 

The  full  motivational  strategy  integration  treatments  seemed  to  be  the  most  beneficial  across  time  as  indicated  by  the 
significant  effects  yielded  by  the  ANOVA  performed  on  the  combined  writing  performance  scores  of  SI  and  S2  (F  (2,  42)  = 3. 
327,  p<  .05). 
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No  significant  effects  were  found  for  task  relevance  independent  variable  on  the  ANOVA  performed  on  the  writing 
scores  of  S2.  The  T5  group,  which  accomplished  the  little  relevant  tasks,  obtained  the  highest  mean  score.  Finally,  analysis  of 
S2  data  also  revealed  that  a variable  interaction  (motivational  strategy  integration  x task  relevance)  occurred  in  the  basic  and 
medium  strategy  integration  treatments.  No  interaction  was  detected  for  the  full  strategy  integration  treatment. 

Attitudinal  Outcomes 

The  one-way  ANOVA  performed  on  the  data  yielded  by  the  three  IMMS  versions  (Al,  A2,  and  A3)  indicated  that  the 
motivational  strategy  integration  variable  did  not  have  a significant  effect  on  attitudes.  The  three  strategy  integration  groups, 
however,  demonstrated  a quite  positive  attitude  toward  the  writing  practice  with  the  courseware. 

The  most  motivated  subjects  appeared  to  be  those  in  the  basic  strategy  integration  treatments  (Tl,  T2)  (mean  = 3.63 
out  of  a possible  1-5  range)  followed  by  those  in  the  full  (T5,  T6)  (mean  = 3.43)  and  medium  features  treatments  (T3,  T4) 
(mean  = 3.29). 

Pearson  r of  overall  attitude  ratings  (Al,  A2,  A3)  with  overall  writing  performance  mean  scores  (SI  and  S2) 
indicated  a correlation  of  . 1 704  (the  Pcv  for  two-tailed  test  a=  .05  and  46  df  r's  for  the 

basic  (T1-T2),  medium  (T3-T4)  and  full  (T5-T6)  strategy  groups  respectively  showed  correlations  of  .1778,  .4631,  and  -.2778 
(the  Pcv  for  two-tailed  test  a=  .05  and  df 

perfomiance  for  the  basic  and  full  integration  treatments,  and  low  positive  correlation,  but  almost  significant,  for  the  medium 
features  treatments. 

Additional  Findings 


Subject's  Perception  of  IVS’  Motivational  Value.  Responses  to  the  attitudinal  surveys'  statements  grouped  by  the 
motivational  condition  they  addressed  indicated  that  subjects  across  the  "high",  medium  and  "low"  strategy  integration 
treatments  found  that  the  highest  motivational  value  conveyed  by  WS  was  Confidence  (mean  = 3.56)  followed  by  Attention 
(mean  = 3.49),  Relevance  (mean  = 3.22),  and  Satisfaction  (mean  = 3.12). 

Four  WS'  attributes  including  appropriate  level  of  difficulty  of  the  activities,  spare  number  of  screen  components, 
suitability  of  the  audiovisual  resources,  and  comments  encouraging  previous  writing  efforts,  were  viewed  by  the  subjects  as 
the  main  motivating  components  of  the  program. 

The  effect  on  student's  confidence  of  the  appropriateness  of  tasks  in  the  WS  package  ratifies  tenets  of  expectancy- 
value  theory  (Lewin,  1938)  which  assume  that  people  are  motivated  to  engage  in  an  activity  if  they  have  a positive  expectation 
for  success.  The  impact  on  subjects'  attention  of  the  frugal  use  of  media  attributes  by  WS  proves  the  validity  of  the  interface 
design  principles  that  were  applied  to  the  package  to  achieve  visual  clarity  (Mayhew,  1992)  or  avoid  cognitive  overload 
(Miller,  1953).  The  effects  of  WS  praising  comments  about  previous  writing  efforts  on  subjects'  satisfaction  supports  research 
findings  which  indicate  that  feedback  on  past  successes  can  be  a powerful  mediator  in  improving  self-efficacy  (Schunk,  1989). 

Subject's  Intensity  and  Persistency  of  Working  Effort  with  WS.  The  M ANOVA  performed  on  four  sets  of  data 
gathered  by  the  WS  tracking  tools  in  S2,  Intro  Time,  Activity  Time,  Help  Time,  and  Overall  Working  Time  revealed  that  task 
relevance  affected  significantly  the  time  subjects  in  the  T3-T6  treatments  spent  on  the  Help  screen  (F  (I,  28)  = 8.961,/?  <.05), 
and  the  time  subjects  in  the  different  treatments  invested  on  the  Activity  screen  (F  (1, 42)  = 4.098,/?  < .05). 

On  the  other  hand,  the  MANOVA  performed  on  three  other  sets  of  data.  Intro  Frequency,  Help  Frequency,  and 
Overall  Frequency  of  Access  on  S2,  showed  that  task  relevance  impacted  significantly  (F  (1,  28)  = 6.153,/?  <.05)  the  frequency 
with  which  subjects  in  the  T3-T6  treatments  accessed  the  Help  screen.  Both  motivational  strategy  integration  (F  (2,  42)  = 
7.482,  p < .05)  and  task  relevance  (F  (2,  42):  4.250,  p < .05)  affected  significantly  the  overall  frequency  of  subjects'  access  to 
the  program. 

These  findings  indicate  that  the  strategy  integration  and,  particularly,  the  task  relevance  variables  affected 
significantly  the  persistency  and  intensity  of  the  effort  subjects  invested  on  the  WS  program.  The  finding  are  consistent  with 
those  of  previous  SLA  (Crookes  & Schmidt,  1991)  and  IT  (Ames  & Ames,  1989)  research  showing  that  personal  relevance  of 
instructional  content  is  a good  determinant  of  selective  learner  attention  and  results  in  increased  time-on  task. 

Subject's  Use  of  WS  Writiri2  Stratesv-Fosterin2  Features.  A descriptive  source  of  data  was  used  to  explore  the  impact 
of  the  WS  motivational  enhancements  on  subjects'  cognitive  activity.  Forty-eight  draft  versions  of  the  writing  activities,  twenty- 
four  per  session  and  four  per  treatment,  were  analyzed  according  to  four  sets  of  criteria  including  length  (number  of  words), 
nature  (outline,  vocabulary,  or  rough  work),  language  (native  or  target),  and  quality  (inclusiveness  of  activity's  requested  ideas). 
Drafts'  analyses  indicated  the  ways  three  WS  writing-fostering  features  including  dedicated  drafting  space,  comprehensive  yet 
discretely  available  help,  and  systematic  activity  directions,  affected  subjects'  use  of  strategies  such  as  "practice,"  "resourcing" 
and  "output  structuring,"  which  are  relevant  to  language  learning  in  general  (Oxford,  1990)  and  CALL  (Liou,  1997)  in 
particular. 
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Conclusion  and  Implication 

This  study  investigated  the  effects  of  ARCS  motivational  strategies  integration  into  the  WS  courseware  package  on  the  writing 
performance  and  attitudes  of  college  students  of  Intermediate  Spanish.  Results  indicated  that  subjects  who  worked  with  the  full 
strategy  integration  versions  of  WS,  which  provided  the  highest  number  and  recurrence  of  ARCS  strategies,  significantly 
outperformed  subjects  who  worked  with  the  basic  and  medium  integration  modalities.  Strategy  integration  level  resulted  in 
greater  performance  gains  than  did  the  inherent  interest  of  the  learning  task. 

The  study  provides  a working  model  on  how  to  manipulate  in  combination  two  language  learning  conditions, 
motivation  and  opportunity,  through  the  comprehensive  application  of  ARCS  motivational  strategies  to  a CALL  courseware 
holistic  paradigm.  More  research  is  needed  to  consistently  validate  the  effectiveness  of  such  model. 

Results  of  the  study  imply  that  with  high  levels  of  motivational  strategy  integration  it  is  possible  to  boost  the 
performance  of  learners  of  a foreign  language  not  necessarily  interested  in  the  activities  of  a CALL  package  for  holistic 
practice.  Future  tests  of  CALL  packages  of  similar  nature  featuring  longer  treatment  periods  and  greater  number  of  participants 
may  provide  wider  evidence  of  this  possibility. 
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Abstract:  The  work  presented  in  this  paper  focuses  on  software  tools  that  act  as  video  explorers 
and  annotation  platforms  and  more  specifically  as  diary  composers  for  younger  children.  Through 
such  tools,  children  can  have  as  their  raw  material  video  recordings  from  everyday  life  and  they  are 
given  the  opportunity  to  review  this  material,  reflect  upon  it  and  annotate  it,  thus  creating  the 
stories  of  everyday  life  from  their  own  perspective.  The  specific  tool  proposed,  focuses  on  multi - 
perspective  capture  of  events  by  one  or  more  children  and  their  cooperation  during  the  annotation 
procedure. 


Introduction 

Current  educational  software  rarely  offers  either  the  child  or  the  adult  facilitator  options  for  ‘deeper  interaction’.  By 
‘deeper  interaction’  we  refer  to  a procedure  where  the  child  takes  active  control  over  the  material  presented  to  it, 
reflects  upon  this  material,  annotates  it  alone  or  in  teams  and  creates  his  own  version  of  reality.  This  work  focuses 
on  developing  both  at  interface  levels  of  interaction  and  within  the  software,  features  that  could  explore  a number  of 
dimensions  of  learning  (Umaschi  1997).  The  software  paradigm  being  proposed  will  aim  at  supporting  stages  of 
reflection,  levels  of  abstraction  and  opportunities  for  cooperation  in  grounded  context  that  are  accessible  to  young 
children.  The  ultimate  aim  is  to  enhance  children’s  development  both  at  cognitive,  physical  and  social 
communication  level  (Baecker  & Posner  1998),  (Umaschi  1996). 

More  specifically,  interaction  with  the  video  material  recorded  previously  is  performed  through  a video  explorer. 
This  video  explorer  is  essentially  a platform  for  the  first  stages  of  reflection.  These  will  include  the  software  and 
interface’s  functionalities  of  location,  navigation  and  individual  or  collaborative  viewing/annotation.  The  video 
explorer  must  therefore  provide  tools  for  both  navigating  inside  videos  and  annotating  them,  thus  consisting  a diary 
composer.  Navigating  features  are  suggested  to  be  inherent  to  the  diary  composer  through  a control  panel,  while 
annotation  features  are  organized  in  distinguished  toolbox  palettes  accompanying  the  composer.  Functionality  of  the 
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palettes  could  include  traditional  editing  functions  such  as  cut,  paste,  drag,  drop,  coupled  with  other  levels  of 
annotation  such  as  sequence  by  time,  event,  emotion,  people,  object,  movement. 

The  research  and  implementation  leading  to  the  tool  proposed  here,  have  taken  place  within  the  framework  of 
project  “Today’s  Stories”,  part  of  the  Long  Term  Research  Task  4.4,  (i3  -ESE,  Project  Nr.  29312,  URL: 
http://stories.starlab.org/).  The  Today’s  Stories  project  is  evolving  a wearable  technology  facilitated  approach  to 
learning  for  young  children  (4  to  8 years  old)  that  is  aimed  at  supporting  the  development  of  social,  communicative 
and  emotional  skills  of  children  in  the  context  of  the  everyday  activities.  In  this  paper,  a thorough  reference  is  being 
made  to  those  requirements  that  must  be  met  by  an  innovative  diary  composing  tool,  such  as  the  one  already 
outlined  (see  Druin  1998),  (see  Norman  1990).  Consecutively,  a list  of  the  functional  specifications  of  the  proposed 
implementation  is  given.  Based  on  the  above,  the  diary  composer  prototype  environment  is  described,  escorted  by 
implementation  and  usage  details. 


Related  Work 

Within  the  framework  of  the  work  presented  in  this  paper,  a thorough  evaluation  of  the  respectful  number  of  current 
commercial  and  research  activities  in  the  field  of  multimedia  editing  and  storyboarding  for  younger  children  has 
been  performed  (see  Buckleitner  et  al.  1999).  Among  the  commercial  products  taken  under  consideration,  the  most 
outstanding  ones  were: 

• Kid  Pix  Studio  Deluxe  from  Broderbund  Software,  California 

• Magic  Theatre  from  Instinct  Corporation,  Silicon  Valley 

• Stanley’s  Sticker  Stories  from  Edmark  Corporation,  an  IBM  company 

• The  Multimedia  Workshop  from  Davidson 

• Kid  Works(TM)  Deluxe  from  Davidson 

• ClarisWorks  for  Kids  from  Apple  Computer,  Inc 

These  products  were  evaluated  with  respect  to  the  proposed  tool  and  a summary  of  their  features  is  given  in  (Tab.  1). 


Feature 

Kid  Pix  Studio 
Deluxe 

Magic 

Theatre 

Stanley’s 
Sticker  Stories 

The  Multimedia 
Workshop 

Kid  Works 
Deluxe 

ClarisWorks 
for  Kids 

Authoring  platform 

X 

X 

X 

X 

X 

X 

Playback  platform 

X 

X 

X 

X 

X 

Age  range 

3-12 

pre- 

readers 

3-7 

8 and  up 

4-9 

5-11 

Create  video 

X 

X 

X 

X 

X 

X 

Insert  video 

X 

X 

X 

Create  sound 

X 

X 

X 

X 

X 

Insert  sound 

X 

X 

X 

X 

X 

Text-to-speech 

X 

X 

X 

X 

Verbal  help 

X 

X 

X 

X 

Icon-based  UI 

X 

X 

X 

X 

X 

Automatic  saving 
and  naming  of  files 

X 

Spelling 

X 

X 

X 

Wizard 

X 

Table  1 : A collection  of  features  from  commercial  diary  composing  tools  for  younger  children 


An  interesting  research  activity  in  the  field  of  storyboarding  consists  The  Fabula  project:  Interactive  children’s 
storybooks  to  promote  bilingual  literacy’  (Tabula’  Project  1998).  Fabula  was  a two-year  project  in  which  partners  in 
Ireland,  Spain,  the  Netherlands  and  the  United  Kingdom  worked  together  to  develop  an  easy  to  use  multimedia 
software  kit  for  schools.  An  authoring  tool  was  developed,  to  enable  teachers  to  create  illustrated  bilingual 
multimedia  story  books  to  help  primary  school  children  learn  a second  language  (and  gain  a deeper  understanding  of 
their  first)  through  the  written  word,  supported  by  pictures,  sound,  animation  and  interactive  graphics.  As  children 
read  the  'pages’  of  those  storybooks,  using  the  specially  developed  Fabula  viewer  software,  they  were  able  to  exploit 
the  unique  opportunities  provided  by  a multimedia  environment. 
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Requirements 


In  this  section  a more  detailed  set  of  requirements  for  stimulating  reflection  through  the  confrontation  of  viewpoints 
and  perspectives  is  proposed.  The  design  approach  of  the  video  explorer  tool,  given  in  the  following  sections,  is 
entirely  based  on  this  set  of  requirements.  The  diary  composer  environment  should  be  essentially  shared  (as  opposed 
to  single  user).  Although  it  must  be  possible  for  children  to  reflect  on  their  own  material  alone,  we  expect  the  most 
productive  sessions  to  be  based  on  collaborative  confrontation  of  children’s  interpretations.  The  diary  composer 
should  be  as  much  as  possible  language  neutral.  We  do  not  assume  literacy  so  we  do  not  propose  the  use  of  text.  We 
also  envisage  that  the  spoken  word  is  a powerful  means  of  annotation,  allowing  the  children  for  example  to  narrate, 
in  their  own  language  and  words,  an  interpretation  of  an  episode. 

The  tool  should  be  open  to  include  arbitrary  libraries  of  annotations,  including  those  that  have  been  created  by 
children  (symbols  from  scanned  drawings  for  instance).  It  should  allow  making  explicit  interpretations.  The  core  of 
interpretation  is  by  annotation  with  symbols  that  children  can  select  from  a given  repository.  The  collaborative 
aspect  is  essential  here.  The  design  choice  followed  is  to  use  a single  repository  of  annotations,  also  when  different 
video  frames  are  rolling.  This  does  not  mean  that  there  is  only  one  consistent  interpretation  of  the  events  but  will 
foster  mutual  understanding  of  them.  The  tool  should  finally  act  as  a self-organizing  memory  support,  avoiding  file 
naming  and  saving  operations.  The  different  recorded  episodes  will  be  organized  on  a time  line  reflecting  the  times 
of  their  occurrence  e.g.  throughout  the  day. 


Functional  specifications 

Storing  and  Indexing 

The  structure  of  the  database  of  video  recordings  is  proposed  to  be  based  on  a time  line  indexing,  with  a time  span  of 
e.g.  one  day.  Functions  to  be  performed  on  this  database  are: 

• Adding  a new  recording 

• Erasing  a recording:  Children  should  be  allowed  to  delete  recordings  and  preserve  for  editing  and  annotation 
only  those  considered  to  be  more  interesting 

Annotating 

Functions  to  be  provided  here  include: 

• Interaction  with  the  video  player:  Either  by  using  pause,  forward,  backward  buttons  or  by  a mechanism  that 
translates  movement  of  the  mouse  over  the  video  screen 

• Preview  of  the  meaning  of  an  annotation  icon  by  clicking  on  it 

• Insertion  of  new  annotations,  either  verbal  or  graphical  ones 

• Implementation  of  a wizard  acting  as  a guide  throughout  the  annotation  procedure 

Archiving 

This  set  of  functions  aims  at  maintaining  a database  of  the  annotated  versions  in  addition  or  as  a replacement  of  the 
original  video  recordings.  The  two  alternatives  are  either  to  delete  the  previous  annotated  version  of  a recording  if 
re-editing  is  desirable  or  to  perform  re-editing  over  an  existent  annotated  version  by  removing  old  and  adding  new 
annotations.  In  both  cases  saving  an  annotated  video  is  done  automatically.  In  cases  where  the  original  video  needs 
to  be  destroyed  for  privacy  reasons,  the  better  approach  is  to  automatically  delete  a video  after  an  expiry  date. 

Worthwhile  Features 

Based  on  our  research  in  the  field  of  commercial  diary  composing  environments,  we  have  ended  up  with  a set  of 
features  that  such  a tool  as  the  one  proposed  should  try  to  inherit.  Some  of  the  most  important  features  are: 

• Possibility  to  send  a “story”  (annotated  video)  to  a friend,  parent,  etc. 

• Easy-to-use  icon  based  User  Interface. 
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• Option  to  insert  sounds  (e.g.  verbal  annotation). 

• Option  to  insert  pictures  and  sounds  either  by  the  children  or  imported  from  an  external  source. 

• Providing  both  an  authoring  and  a playback  platform  combined  in  a transparent  way  to  the  child. 

• Providing  help  on  the  meaning  of  annotation  buttons  by  visual  animation  which  could  be  accompanied  or  not  of 
sounds  suggestive  of  its  action. 


A general  description  of  the  diary  composer  prototype  environment 

The  prototype  environment  proposed  for  development  consists  of  two  modules:  the  ‘Timeline’  display  and  the 
‘Annotation’  display.  In  (Fig.  1),  those  two  displays  are  presented  simultaneously.  However,  this  is  not  obligatory.  It 
is  proposed  that  the  ‘Timeline’  display  is  permanent  on  screen,  while  the  ‘Annotation’  display  appears  only  when  it 
is  required:  during  the  procedure  of  video  annotation. 


Figure  1:  The  ‘Timeline’  and  ‘Annotation’  displays 


The  ‘Timeline’  display  expands  to  the  whole  screen  displaying  one  curved  line  for  each  child  using  the  diary 
composer.  Each  one  of  the  curved  lines  in  (Fig.  1)  represents  a day  for  one  child.  One  thumbnail  for  each  video 
recording  is  placed  on  a child’s  timeline.  In  (Fig.  1),  three  such  thumbnails  are  created  in  the  first  timeline  and  two 
thumbnails  are  placed  on  the  second  timeline.  Thumbnails  are  placed  on  the  timelines  in  such  a way  that  the  time  of 
recording  is  reflected,  thus  the  distance  of  two  thumbnails  on  a certain  timeline  is  proportional  to  the  real  time 
interval  between  the  two  recordings.  The  ‘Annotation’  display  consists  of  two  groups  of  annotation  icons,  one  group 
of  icons  symbolizing  sounds  and  another  group  of  icons  symbolizing  emotions  and  any  other  kind  of  annotation. 
Each  group  consists  a tool  bar  placed  at  the  screen  edges  (the  placement  of  the  ‘Sounds’  icon  bar  on  the  top  left  part 
of  the  screen  and  the  ‘Annotations’  icon  bar  on  the  bottom  right  of  the  screen  in  (Fig.  1 ) is  indicative). 

The  ‘Annotation’  display  provides  children  with  the  tools  for  annotating  the  videos.  The  icons  representing  sounds 
can  be  either  buttons,  which  when  pressed  allow  for  the  playback  of  the  correspondent  sound,  or  drag  and  drop 
items,  which  the  children  will  be  able  to  drop  on  a video  frame  in  order  to  annotate  the  instance  displayed  with  the 
correspondent  sound.  The  icons  for  the  rest  of  the  annotations  (showing  expressions,  feelings  etc.)  can  only  be  drag 
and  drop  icons,  dragged  by  the  children  on  certain  video  frames,  during  the  annotation  procedure. 

Both  the  icon  bars  of  the  ‘Annotation’  display  will  be  global  in  the  diary  composer  environment.  In  other  words,  the 
annotation  tool  bars  will  always  be  visible  during  the  annotation  procedure  of  different  videos,  instead  of  appearing 
each  time  an  annotation  procedure  is  initiated  and  being  imminently  related  to  the  current  video  under  annotation. 
This  will  allow  children  to  treat  the  diary  composer  as  a homogeneous  tool  that  provides  the  same  layout  and  options 
regardless  of  the  current  thumbnails  and  video  processing  procedures. 

The  annotation  procedure 
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When  one  or  more  children  decide  to  annotate  a video  of  an  incident  they  have  already  recorded,  they  have  to 
simply  ‘click’  on  this  video’s  thumbnail,  for  the  annotation  procedure  to  begin.  Immediately,  the  thumbnail  is 
transformed  to  an  enlarged  video  display,  extending  to  a large  part  of  the  screen  (see  Fig.  2).  The  dimensions  of  the 
video  display  will  be  customizable  but  it  is  definite  that  they  will  not  extend  to  the  whole  of  the  screen.  Space  on  the 
screen  must  be  preserved  for  the  ‘Annotation’  display  and  the  timeline(s)  will  remain  at  the  background  for  the 
children  to  return  to  them  and  select  another  thumbnail,  if  needed. 

Navigation  in  a video  recording  for  the  purposes  of  annotation  will  be  made  either  by  the  buttons  appearing  on  the 
bottom  of  the  video  (right  arrow  for  ‘Forward’,  left  arrow  for  ‘Backward’  and  the  hand  icon  for  ‘Stop’  or  ‘Slow 
Motion’  playback)  or  by  imminently  clicking  on  the  video  itself  The  latest  feature  will  allow  children  to  move 
forwards  in  the  video  by  placing  their  mouse  at  the  right  hand-side  of  the  video  display,  move  backwards  in  the 
video  by  placing  their  mouse  at  the  left  hand-side  of  the  video  display  and  pausing/resuming  playback  of  the  video 
by  clicking  on  the  video  display. 
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Figure  2:  Annotations  are  added  to  the  video 


Once  the  child  has  successfully  acquired  the  navigation  techniques,  it  can  then  playback  the  video  and  pause/stop 
playback  at  any  moment  in  order  to  annotate  it.  In  the  case  where  both  sound  and  expression/emotion  annotations 
are  of  a drag  and  drop  nature,  the  diary  composer  screen  will  look  like  that  of  (Fig.  2).  Two  annotations,  one  sound 
and  one  expression/emotion  annotation  are  added  to  the  particular  video  frame  and  can  appear  as  small  icons, 
dropped  on  the  top  right  comer  of  it.  Of  course,  the  placement  of  annotations  will  not  be  restricted. 

During  the  annotation  procedure,  any  annotations  and  changes  on  the  current  video  will  be  automatically  saved. 
However,  when  a child  wishes  to  make  a modification,  it  will  be  able  to  locate  the  annotated  video  frame  it  wishes 
to  edit  and  remove  or  add  annotations  with  the  drag  and  drop  fashion.  Removing  an  annotation  from  a video  frame 
will  be  achieved  by  dragging  and  dropping  it  out  of  the  space  where  the  video  is  displayed.  Removing  all 
annotations  added  to  a video  recording  and  restart  annotating  the  pure  video  will  be  achieved  by  clicking  on  the 
thumbnail  of  the  video  recording,  which  remains  on  the  timeline  in  the  background.  Clicking  on  the  ‘Timeline’ 
display,  which  during  the  annotation  procedure  comprises  the  diary  composer’s  background,  will  terminate  an 
annotation  procedure,  saving  automatically  all  annotations  in  the  current  video. 

Speciflc  features  of  the  Diary  Composer 

An  interesting  feature  that  is  proposed  for  the  ‘Timeline’  display  is  that  of  dynamic  behavior.  Although  the  diary 
composer  will  allow  a child  to  reflect  only  on  his  own  material,  by  watching  only  his  own  timeline  and  thumbnails 
on  the  ‘Timeline’  display,  it  will  also  allow  collaborative  confrontation  of  children’s  interpretations.  This  feature  will 
be  supported  by  the  dynamic  the  timelines.  It  is  proposed  that  in  cases  where  two  children  have  recorded  the  same 
incident  (from  different  points  of  view)  and  wish  to  annotate  their  recordings  cooperatively,  the  two  different 
thumbnails  of  this  same  incident  will  be  associated.  This  association  will  be  visualized  by  the  two  children’s 
timelines  ‘approaching’,  as  shown  for  the  first  thumbnails  of  the  two  timelines  in  (Fig.  3).  This  metaphor  will  initiate 
the  procedure  on  collaborative  annotation  of  the  videos. 
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Conclusions  and  Future  Work 

It  is  evident  by  the  nature  and  design  of  the  tool  proposed  here  that  a lot  of  its  implementation  choices  consist  open 
issues  with  regard  to  their  pedagogical  value.  The  tool  seems  to  succeed  in  acquiring  collaboration  of  its  young 
users,  by  presenting  simultaneously  video  recordings  on  the  same  event  and  provoking  small  groups  of  children  to 
comment  upon  them  and  annotate  them.  It,  therefore,  distinguishes  itself  among  relevant  existent  tools. 

The  issue  is  here,  how  to  stimulate  the  children’s  involvement  in  the  editing  and  annotating  procedure  as  much  as 
possible.  On  this  goal,  should  future  work  on  the  diary  composer  focus.  The  issues  to  be  dealt  with  are  the 
improvement  of  the  controls  for  loading,  editing  and  annotating  the  videos  as  well  as  the  library  of  annotations,  so 
that  they  will  approach  as  much  as  possible  the  children’s  perception. 
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Abstract:  Design  Principles  for  Collaborative  Teleleaming  have  been  extracted  from  design 
experiments  and  validated  by  instructional  designers.  The  design  approach  integrates  individual 
learning,  teamwork  and  group/class  instruction,  and  links  on-site  and  on-line  learning  situations  in 
a coherent  instructional  continuum.  The  challenge  was  to  build  scenarios  that  integrate  a three- 
level  set  of  learning  activities:  individual,  team  and  group  activities  and  to  support  collaboration  in 
action. 


Introduction 

Teleleaming  represents  a new  educational  opportunity  that  offers  many  of  the  same  features  as  distance  learning,  but 
is  different  in  some  ways  as  well.  Teleleaming  provides  opportunities  not  available  in  traditional  distance  learning, 
such  as  highly  interactive  and  media-enhanced  peer  collaboration.  The  potential  for  mediated  social  interactions 
raises  a number  of  issues.  Our  research  focuses  on  two  of  these  issues:  1)  What  is  the  nature  of  collaboration  in 
collaborative  teleleaming?;  2)  What  principles  guide  the  design  of  collaborative  teleleaming  scenarios  and  software 
environments? 

Our  definition  of  teleleaming  is  as  follows:  “The  term  teleleaming  is  used  to  designate  new  forms  of  distance  or  of 
computer  mediated  learning,  where  the  distance  is  not  only  distance  in  terms  of  space  or  time  as  in  traditional 
distance  learning,  but  the  mediation  in  learning  activities  served  by  media,  such  as  multimedia-based  shared 
workspaces,  multimedia  communication,  or  multimedia  servers”  (Bourdeau  & Wasson,  1997).  The  definition  of 
collaborative  learning  by  Salomon  (1991)  contains  three  major  components:  sharing,  interdependencies  and 
involvement,  Ohlson  & Ohlson  (1993)  see  collaboration  in  the  context  of  computer-supported  collaborative  work  as 
a combination  of  communication  and  coordination.  Coordination  science  defines  coordination  as  managing 
interdependencies,  which  can  apply  to  interdependencies  between  actors  (Bourdeau  & Wasson,  1 997).  In  order  to 
develop  design  principles  for  collaborative  teleleaming,  existing  learning  situations  needed  to  be  analyzed  and  new 
ones  created.  TTie  purpose  was  to  extract  and  elicit  principles,  then  determine  whether  these  principles  could  be 
implemented  in  the  scenario  or  whether  some  roles  could  be  played  by  a software  agent.  The  general  type  of 
learning  situation  selected  is  called  a distributed  simulation  lab,  which  consists  of  a scenario  plus  a software 
environment. 

Starting  from  general  instmctional  design  principles  for  distance  learning,  (Bourdeau  & Bates,  1996),  the  challenge 
was  to  extend  or  specify  existing  principles,  add  new  ones,  and  validate  the  new  set.  The  starting  set  of  principles 
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was  as  follows:  1)  Select  media  based  on  access  for  students;  2)  Develop  fully  readable,  understandable  and  feasible 
material  and  guidance;  3)  Plan  both  synchronous  and  asynchronous  two-way  communication;  4)  Design  a variety  of 
learning  activities;  5)  Design  a structured  interactive  student  support  system;  6)  Plan  opportunities  for  peer 
collaboration;  7)  Provide  appropriate  feedback  to  students;  8)  Anticipate  errors  by  students,  both  in  terms  of  content 
and  process;  9)  Set  clear  evaluation  standards  for  learning  outcomes  to  compensate  for  the  high  distribution  of 
students  and  tutors  and  to  ensure  equity;  10)  Plan  for  the  evaluation  of  instructional  materials  and  student  support. 

Project  6.2.3  of  the  Teleleaming  Network  of  Centres  of  Excellence  (www.teleleam.ca)  focused  on  the  development 
of  Instructional  Design  principles  for  collaborative  teleleaming.  The  design  approach  is  far  more  inclusive  than 
existing  ones  in  that  it  integrates  individual  learning  (self-directed  learning  and  self-tutoring),  teamwork  and 
group/class  instruction.  It  also  links  on-site  and  on-line  learning  situations  in  a coherent  instructional  continuum. 
The  challenge  was  to  build  scenarios  that  integrate  a three-level  set  of  learning  activities:  individual,  team  and  group 
activities  based  on  the  belief  that  learning  is  maximized  when  the  three  situations  are  used  appropriately  (in 
alternation  or  combined).  An  additional  challenge  arose  from  the  desire  to  support  collaboration  in  action,  not  just 
in  discourse,  in  the  context  of  multimodal  scenarios  that  integrate  synchronous  multimedia  interactions  with  textual 
asynchronous  communication. 


Method 

Design  principles  are  the  prescriptive  knowledge  produced  in  the  field  of  instructional  technology,  based  on 
instmctional  and  learning  theory  (Gagne,  1992;  Merrill,  1994;  Reigeluth,  1983,  1997,  1998;  Lebow,  1995)  that  serve 
a community  of  practice  (Tessmer  & Wedman,  1995).  Their  use  and  validity  are  strongly  related  to  the  field  of 
practice.  The  methodological  approach  adopted  here  respected  requirements  of  rigor  and  validity,  while  working 
within  the  constraints  of  technological  innovation.  The  following  principles  will  be  revised  as  technological 
environments  and  our  understanding  of  how  people  learn  within  them  evolve. 

A series  of  different  methods  was  employed  to  develop  and  validate  principles  : a)  Field  testing  to  extract 
knowledge  about  collaborative  teleleaming  in  a distributed  simulation  lab  for  management  students;  b)  Design 
experimentation  to  test  a new  design  for  a distributed  simulation  lab  in  physics;  c)  Modelling  to  analyze 
interdependencies  and  the  potential  roles  of  a coordination  agent;  d)  Focus  group  to  validate  principles. 

The  field  test  for  extracting  knowledge  about  collaborative  teleleaming  in  a distributed  simulation  lab  in  the  strategic 
management  field  consisted  in  a series  of  structured  observations  in  various  on-campus  learning  sessions.  The 
central  element  of  the  learning  environment  is  a business  simulation  (www.cetai.ca/netstrat)  that  is  run  as  a game. 
Several  teams  can  participate  at  a distance  within  a predetermined  timeframe  set  by  the  gamemaster  (Bourdeau  & 
Winer,  1997).  Participants  were  students  in  various  business  administration  programs  at  the  Ecole  des  HEC  (School 
of  Commerce)  in  Montreal. 

The  scenarios  involved  students  in  a mix  of  face-to-face,  distributed  group,  as  well  as  inter-  and  intra-team  activities. 
Researchers  observed  and  analyzed  interaction  within  and  among  the  teams,  between  the  students  and  the  professor, 
and  between  the  students  and  the  computer  simulation  environment.  Observations  covered  the  briefing,  action 
rounds,  report  writing  and  debriefing.  Particular  attention  was  paid  to  the  following  questions:  a)  What  means  of 
communication  did  the  students  use  and  to  what  end?  (Communication);  b)  Which  activities  or  methods  encouraged 
the  creation  of  interdependencies?  (Interdependencies);  c)  Did  all  team  members  participate  in  the  decision-making 
process?  (Involvement);  d)  Which  coordination  strategies  did  the  various  teams  develop?  (Coordination) 

Data  sources  included  a structured  journal  and  semi-structured  comments  from  each  participant,  structured 
observations  during  group  sessions,  as  well  as  copies  of  messages,  decisions  and  reports.  Results  from  the  analysis 
of  the  field  test  showed  that  the  processes  involved  in  working  and  learning  in  a collaborative  way  need  to  be 
improved.  The  processes  of  information-sharing,  fostering  relationships  among  participants,  participating  in  the 
decision-making  process,  and  building  specific  coordination  strategies  among  team  members  all  need  to  be 
supported  by  implicit  and  explicit  tools. 
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A design  experiment  was  conducted  to  test  a new  distributed  science  lab  in  physics  (Winer,  Chomienne  & Vazquez- 
Abad,  1999).  The  scenario  consisted  of  three  experiments:  Experiment  1 was  a qualitative  exploration  of  the 
behaviour  of  simple  electric  circuits.  It  involved  handling  common  electrical  materials,  such  as  bulbs,  batteries, 
connecting  wires,  etc...  Experimentation  and  data  measurements  were  conducted  using  real  instruments.  The  data 
analysis  was  carried  out  on  an  individual  basis.  Results  were  interpreted  and  conclusions  were  drawn  as  a team. 
Experiment  2 was  on  Ohm’s  Law.  The  experiment  consisted  of  working  with  a computer  simulation. 
Measurements  were  automated,  data  analysis  was  carried  out  on  an  individual  basis,  and  interpretations/conclusions 
were  drawn  as  a team.  Experiment  3 was  on  KirchhofPs  Law  on  the  distribution  of  current  in  a circuit.  The 
experiment  was  again  conducted  with  a simulator  and  measurements  were  automated.  Data  analysis,  interpretations 
and  conclusions  were  carried  out  as  a team. 

A software  environment  was  developed  and  implemented  at  a distance  learning  college  to  test  the  learning  scenario 
under  distance  learning  conditions  (i.e.,  students  in  different  locations  interacting  in  a software  environment  with 
their  peers).  See  (http://crosemont.qc.ca/ccfd/soci/contenu/203-FPF-03/Section2/laboratoires/lab3/electrik  html). 

Data  sources  included  observations,  student  productions,  and  team  interviews.  Results  enabled  researchers  to 
establish  the  following  guidelines:  a)  Provide  a workspace  for  the  individual,  not  just  a shared  space  for  the  team;  b) 
Conduct  a group-based  formative  evaluation  of  learning  materials  with  different  learners  to  ensure  that  the 
instructions  and  vocabulary  are  not  only  internally  consistent,  but  also  subject  to  the  same  interpretation;  c)  Offer  a 
variety  of  learning  activities;  d)  Error  analysis  must  distinguish  between  productive  and  non-productive  errors  so  as 
to  permit  the  former  and  block  the  latter;  e)  The  technology  should  enhance  the  learning  experience;  f)  Each 
learner’s  contribution  must  be  necessary  but  not  sufficient  for  both  the  individual  and  the  group  to  succeed  in  the 
learning  activity. 

The  modelling  conducted  to  analyze  interdependencies  and  the  potential  roles  of  a coordination  agent  was  based  on 
both  Salomon’s  framework  for  CL  (Salomon,  1991)  and  the  Crowston  & Malone  framework  for  coordination 
(Crowston  & Malone,  1 994)  using  a modelling  tool  called  MOT  (www.licefteluq.uquebec.ca).  Several  learning 
scenarios  in  the  strategic  management  course  were  modelled  and  analyzed  to  identify  interdependencies,  and  extract 
coordination  principles  (Bourdeau  & Wasson,  1997;  Wasson,  1998a).  The  following  observations  were  made: 
“Actors  in  a collaborative  teleleaming  situation  must  or  need  to:  1)  share  goals  to  complete  activities;  2)  share 
activities  to  achieve  goals;  3)  share  resources  to  complete  activities;  4)  share  activities  to  produce  resources 
(Bourdeau  & Wasson,  1997).  Interdependencies  can  be  either  collaborative  or  competitive.  They  differ  in  the 
coordination  processes  required  to  manage  them.  Coordination  software  agents  can  be  designed  to  manage 
interdependencies.  Preliminary  work  on  identifying  roles  to  be  played  by  agents  indicates  that  agents  can  either 
serve  as  coordination  managers  or  coordination  facilitators.  Other  roles  they  could  play  include:  team-building 
agent,  group  decision-making  assistant,  scheduling  manager,  synchronization  manager,  competition  manager,  and 
competition  monitor  (Wasson,  1999). 

The  Focus  group  was  conducted  to  validate  principles.  Preliminary  principles  derived  from  the  steps  above  were 
developed  as  follows: 

1 . Select  media  based  on  their  ability  to  support  collaboration  in  learning  activities  (i.e.,  by  being  multidirectional, 
interactive  and  sharing). 

2.  Ensure  shared  understanding  of  learning  materials-text,  graphics,  audio,  video-by  conducting  formative 
evaluations  in  a team  setting. 

3.  Create  meaningful  and  balanced  opportunities  for  synchronous— face-to-face  or  mediated— and  asynchronous 
interactions  among  students,  between  distributed  teams  and  the  system,  as  well  as  between  distributed  teams 
and  the  instructor. 

4.  Provide  specific  support  tools  and  structures  for  the  team-building  phase  of  the  activity,  whether  teams  are 
formed  by  participants  or  course  leaders. 

5.  Introduce  collaboration  gradually.  Ensure  that  students  are  familiar  with  the  learning  environment  before 
requiring  intensive  collaboration. 

6.  Implement  coordination  mechanisms  when  designing  each  learning  activity. 

7.  Design  a student  support  system  capable  of  supporting  distributed  teamwork. 

8.  Create  authentic  and  feasible  collaborative  learning  activities  with  built-in  interdependencies. 

9.  Provide  multidirectional  feedback  channels  from  student  to  student,  team  to  team,  and  system  to  team. 
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10.  Anticipate  possible  technical  and  conceptual  errors.  Eliminate  non-productive  errors  by  stronger  guidance 
and/or  interface  design.  Use  productive  errors  as  input  for  team  discussions,  to  provoke  cognitive  conflict 
among  team  members,  and  to  stimulate  individual  reflection. 

1 1 . Design  an  assessment  method  for  both  individual  and  team  achievements  by  tracking  individual  contributions, 
and  by  asking  individuals  to  reflect  on  teamwork  processes  and/or  performance. 

12.  Plan  for  obsolescence.  Foresee  frequent  updates  of  collaboration  mechanisms. 

13.  Provide  strong  guidance  for  learning  how  to  use  the  tools,  as  well  as  support  for  teamwork  and  a design  for 
active  learning  to  develop  skills. 

A group  of  eight  (8)  instructional  designers  were  selected  based  on  their  professional  experience  and  interest  in  new 
ideas.  Participants  were  asked  to  attend  a one-day  seminar  with  a course  design  of  their  own.  Each  participant 
completed  a profile  to  answer  the  following  questions:  How  many  years’  experience  do  you  have  in  instructional 
design?  Have  you  received  formal  training  in  instructional  design?  Have  you  designed  or  participated  in  the  design 
or  evaluation  of  "traditional"  distance  education  or  training  involving  computer-based  learning?  How  often  do  you 
refer  to  ID  references  when  designing  instructional  methods?  If  and  when  you  consult  references,  what  are  you 
looking  for? 

Each  principle  was  presented  to  the  group  with  illustrative  examples.  The  group  was  then  divided  into  teams  and 
asked  the  following  questions:  Are  the  principles  easy  to  understand?  Is  there  any  terminology  that  is  confusing?  Do 
the  principles  apply  to  your  design  situation?  Were  the  principles  useful  in  the  design  process?  Do  you  have 
instructional  design  questions  specific  to  Web-based  learning  that  were  not  addressed  by  the  principles?  Is  there  an 
order  of  presentation  that  would  be  useful?  Are  the  principles  independent,  or  does  it  make  sense  to  look  at  them  in  a 
specific  order?  Do  you  have  any  other  comments  or  questions? 

After  the  teams  had  answered  the  questions,  a group  session  was  organized  to  discuss  teamwork  results,  summarize 
the  most  important  comments  and  suggestions,  both  on  the  wording  and  order  of  the  principles. 


Results 

The  preliminary  principles  were  revised  and  reordered  by  design  phase  (analysis,  1-5;  design/development,  6-11; 

evaluation,  12;  delivery  and  management,  13)  as  presented  below: 

1 . Identify  opportunities  for  authentic  and  feasible  collaborative  learning  activities. 

2.  Select  media,  channels  and  tools  based  on  their  ability  to  support  collaboration  in  learning  activities  (i.e.,  by 
being  multidirectional,  interactive  and  sharing). 

3.  Provide  feedback  channels  from  student  to  student,  team  to  team,  and  student  support  system  to  team. 

4.  Anticipate  possible  technical  and  conceptual  learner  errors.  Use  productive  errors  as  input  for  team  discussions, 
to  provoke  cognitive  conflict  among  team  members,  and  stimulate  individual  reflection.  Eliminate  non- 
productive errors  by  providing  stronger  guidance  and/or  interface  design. 

5.  Use  a structured  learning  approach  for  learning  how  to  use  tools  and  an  active  learning  approach  for  developing 
skills. 

6.  Design  meaningful  and  balanced  opportunities  for  synchronous— face-to-face  or  mediated— and  asynchronous 
interactions  among  team  members,  between  distributed  teams  and  the  system,  as  well  as  between  distributed 
teams  and  the  instructor. 

7.  Design  an  assessment  method  for  both  individual  and  team  achievements  by  tracking  individual  contributions, 
and  by  asking  individuals  to  reflect  on  teamwork  processes  and/or  performance. 

8.  Provide  specific  support  tools  and  structures  for  the  life  span  of  the  team.  For  example,  during  the  team- 
building phase  of  the  activity,  whether  teams  are  formed  by  participants  or  course  leaders. 

9.  Introduce  collaboration  gradually.  Ensure  that  students  are  familiar  with  the  learning  environment  before 
requiring  intensive  collaboration. 

10.  Implement  coordination  mechanisms  when  designing  each  learning  activity. 

1 1 . Design  a student  and  tutor  support  system  capable  of  supporting  distributed  teamwork. 

12.  Conduct  formative  evaluations  in  a team  setting  to  ensure  shared  understanding  of  learning  materials— text, 
graphics,  audio,  video — and  coordination  mechanisms. 

13.  Plan  for  upgrades.  Foresee  frequent  updates  of  collaboration  tools  and  mechanisms. 
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As  a by-product  of  the  Teleleaming  project  (6,2,3),  a learning  product  and  the  profile  for  a software  coordination 
agent  were  also  developed.  The  design  experiment  in  Step  2 was  used  to  build  and  implement  a learning  scenario 
and  environment.  The  product  is  called  a distributed  collaborative  science  learning  lab:  DCSLL  (Winer  et  al., 
1999),  The  modelling  exercise  in  Step  3 served  to  complete  the  analysis  of  the  roles  of  a coordination  agent 
(Wasson,  1998b). 


Conclusions  and  Outlook  for  the  Future 

The  results  of  this  project  provide  an  in-depth  understanding  of  collaborative  teleleaming  scenarios  distributed  over 
both  time  and  space.  The  groundwork  has  been  laid  to  coordinate  actors  in  collaborative  teleleaming  scenarios.  As 
a result,  the  design  of  pedagogical  agents  for  collaborative  teleleaming  has  been  further  advanced. 

The  foundations  for  collaborative  science  teleleaming  laboratories  have  also  been  established  and  tested.  More 
specifically,  a scenario  in  a Web-based  environment  for  a physics  lab  has  been  developed,  implemented  and  tested. 
Collaborative  teleleaming  scenarios  were  modelled  using  the  MOT  knowledge  editor  in  order  to  extract  instructional 
design  principles. 

Instructional  designers  and  consultants  have  validated  preliminary  principles.  Design  principles,  based  both  on 
theoretical  foundations  and  training  practices,  should  ideally  provide  instructional  designers  with  the  knowledge 
they  need  for  innovative  teletraining,  while  maintaining  high-quality  standards. 

Until  now,  work  has  been  carried  out  using  “bridges’’  to  develop  and  test  training  solutions:  Management  training 
was  tested  in  a higher  education  environment  at  the  Ecole  des  Hautes  Etudes  Commerciales  (HEC),  and  technical 
training  was  tested  at  the  collegiate  continuing  education  level  - Centre  Collegial  de  Formation  d Distance  (CCFD), 
This  made  it  possible  to  test  preliminary  solutions  in  situations  that  were  realistic  enough  for  a formative  evaluation 
of  scenarios  and  technological  environments. 

Future  work  in  the  Teleleaming  Network  of  Centres  of  Excellence  should  include:  1)  Creation  of  a Workplace 
Training  portfolio,  consisting  of  a collection  of  basic  learning  scenarios,  as  well  as  a methodology  for  adapting 
scenarios  to  specific  learning  situations  or  needs  (DelaTeja  et  al.,  submitted  to  EDM  EDI  A ’2000);  2)  Design  of  a 
coordination  agent  able  to  play  the  roles  identified  above  (Wasson,  1998b);  3)  Implementation  of  the  coordination 
agent  in  a virtual  campus  environment,  such  as  LlCEF’s  Explora  (Paquette  et  al,,  submitted  to  EDMEDIA’2000), 
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Abstract:  The  discipline  of  learning  environment  design  has  expanded  greatly  in  the  past 
decade.  The  need  for  effective  methods  to  capture,  represent  and  communicate  design 
knowledge  remains  pressing.  High  level  principles,  such  as  those  provided  by  constructivism, 
are  not  sufficient  in  themselves.  This  paper  provides  an  exposition  of  the  concept  of  PALs 
(Patterns  for  Learning).  These  are  re-usable  design  patterns  that  can  be  applied  in  the 
development  of  computer  supported  learning  environments.  The  paper  discusses  how  PALs 
may  be  extracted  to  capture  and  represent  design  knowledge.  It  is  argued  that  these  re-usable 
design  patterns  provide  a productive  currency  for  metadesign,  the  theoretical  discourse  about 
design. 


Introduction 

There  has  been  a rapid  expansion  of  learning  technology  in  the  past  decade.  This  expansion  has  involved  not 
just  an  increase  in  the  size  of  the  community  but  also  a burgeoning  of  design  innovations.  Many  of  these 
innovations  are  linked  with  the  increasing  influence  of  constructivist  conceptions  of  the  learning  process 
(Cunningham  et  al.  1993,  Jonassen  et  al.  1993).  There  remains,  however,  a considerable  fragmentation  in  the 
representation  of  design  knowledge,  and  often  a considerable  gap  between  exhortation  and  implementation. 
There  is  a need  to  develop  a systematic  conceptual  representation  of  design  knowledge. 

There  are  different  ways  of  capturing  and  representing  design  knowledge.  Constructivism  has  presented  design 
guidance  at  two  main  levels:  principles  and  tactics.  Constructivist  principles  include  an  emphasis  on  authentic 
learning  tasks  which  evoke  a high  degree  of  interaction  and  collaboration  to  engage  the  learner  in  the 
knowledge  construction  process.  These  principles,  emphasizing  authentic  tasks,  interaction,  engagement, 
collaboration  and  ownership  of  the  learning  process,  have  a high  heuristic  value  for  the  designer.  However, 
there  is  considerable  freedom  of  interpretation  in  how  they  should  be  implemented.  These  principles  may  be 
realized  through  various  methods.  These  methods  range  from  constructionisn  with  a very  strong  emphasis  on 
learner  freedom  in  constructing  artifacts  (Papert  1993)  through  to  structured  guided-discovery  methods  such  as 
CORE  (Boyle  et  al  1994).  These  methods  may  include  refinement  and  expansion  of  constructivist  principles 
and/or  some  degree  of  structured  method  for  doing  design. 

These  representations  of  design  knowledge  are  both  valuable.  However,  there  remain  a number  of  problems. 
This  becomes  particularly  manifest  when  teaching  learning  environment  design.  There  is  often,  for  example,  a 
marked  gap  between  exhortation  and  implementation.  For  novice  designers  the  principles  are  quite  abstract. 
How  do  they  relate  these  principles  to  the  specific  problem  situation?  In  software  engineering  the  re-use  and 
tailorability  of  software  components  has  been  a very  important  issue  in  design.  However,  support  for 
tailorability  and  re-use  in  learning  environment  design  is  weak.  Constructivism  provides  reusable  principles  but 
at  a very  high  level.  The  designer  may  struggle  to  extract  re-usable  components  from  examples  of  constructivist 
systems.  Constructivist  oriented  methods  for  learning  environment  design,  such  as  Cognitive  Apprenticeship, 
does  not  fully  solve  this  problem.  Some  features  still  remain  at  a high  level  of  representation.  How  do  modeling 
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and  scaffolding  apply  at  the  level  of  detail,  for  example,  of  developing  a learning  environment  for  program 
design?  How  do  you  apply  the  concepts  when  dealing  with  issues  such  as  the  complexity  and  abstraction  of  the 
material  to  be  learned?  The  concepts  provide  good  starting  principles.  But  how  you  progress  from  these  starting 
principles  to  develop  successful  systems  is  often  not  clear. 

There  is  thus  often  a marked  gap  between  principles  and  design  practice.  A central  issue  is:  how  do  you  capture 
design  insights  in  a form  that  provides  re-usable  techniques  that  can  be  applied  to  create  a bridge  from 
principles  into  practice? 


Design  patterns 

There  is  considerable  research  work  that  indicates  that  much  expert  knowledge  is  represented  at  the  level  of 
'patterns*.  These  are  abstract  representations  relating  problem  space  characteristics  to  solution  frames  that  can  be 
applied  to  a range  of  concrete  problem  situations.  Patterns  have  been  identified  as  a key  presentation  of  expert 
knowledge  in  domains  such  as  chess,  programming  (Hohmann  et  al.  1992,  Clancy  and  Linn  1999)  and 
hypermedia  design  (Lyardet,  Rossi  and  Swabe  1998).  In  modem  object  oriented  design  'design  patterns'  has 
become  recognized  as  a key  way  of  capturing  and  disseminating  high  level  design  expertise  (Gamma  et  al. 
1995).  These  patterns  capture  expert  knowledge.  They  are  the  'chunks'  that  enable  experts  to  deal  at  a much 
higher  level  of  representation  in  solving  problems.  They  fit  in  with  the  classical  psychological  theory  of  how 
people  overcome  the  limitations  of  working  memory  - 7 pus  or  minus  2 bits  of  information  (Miller  1956). 
Experts  deal  with  chunks  or  patterns  of  information  (sometimes  at  a very  high  level)  not  isolated  elements.  The 
development  of  expertise  involves  building  up  these  patterns  of  information  together  with  the  skills  to  apply 
them  effectively. 

Work  has  started  in  the  area  of  object  oriented  design  on  the  concept  of  pedagogical  patterns  as  a way  of 
capturing  and  representing  expert  knowledge  (see  especially  http://www-lifia.info.unlp.edu.ar/ppp/index.html). 
The  work  in  this  area  has  begun  to  tackle  some  important  issues,  such  as  developing  formats  for  the 
representation  of  pedagogical  patterns  (see  http://www-lifia.info.unlp.edu.ar/ppp/format.htm).  However,  this 
work  has  tended  to  be  focused  mainly  on  the  domain  of  object-oriented  programming  and  design.  The 
importance  of  the  concept  of  re-usable  design  patterns  as  a way  of  capturing  re-usable  'domain  free*  pedagogical 
techniques  has  made  little  impact  on  the  wider  learning  environment  community.  The  concept  of  design  patterns 
should  provide  a powerful  tool  for  analysis  and  synthesis.  It  should  provide  a tool  to  analyze  existing 
approaches,  extract  re-usable  design  patterns  and  support  the  use  of  these  design  patterns  in  new  creative 
systems  or  learning  architectures.  These  are  the  objectives  of  the  approach  advocated  in  this  paper. 


PALs  for  designers 

The  aim  is  to  develop  an  approach  that  will  help  extract,  represent  and  apply  the  high  level  knowledge  that  is 
possessed  by  expert  designers  of  computer  based  learning  environments.  The  term  used  in  the  paper  for  these 
re-usable  deign  patterns  is  PALs  (Patterns  for  Learning).  This  explicated  knowledge,  in  turn,  should  provide  a 
currency  for  productive  dialogue  among  designers.  It  should  facilitate  a real  debate,  a dialectic  of  development 
in  furthering  the  explication  and  adaptation  of  design  knowledge. 

There  are  two  main  sources  for  extracting  re-usable  design  patterns: 

* Abstraction  from  examples  of  good  practice 

* Analysis  of  existing  methods 

This  discussion  will  focus  on  the  second  source.  This  provides  a very  direct  and  productive  way  of  extracting 
re-usable  design  patterns.  It  supports  the  creation  of  a level  of  design  knowledge  at  a level  of  granularity  below 
that  of  methods  or  tactical  trameworks.  It  complements  and  extends  traditional  ways  of  representing  design 
knowledge. 
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Analysis  of  existing  methods 


There  have  been  a number  of  methods  proposed  for  the  design  and  development  of  learning  environments,  such 
as  Cognitive  Apprenticeship  (Collins  et  al,  1989),  CORE  (Boyle  et  al.  1994)  and  CaBLE  (Feifer  and  Allender 
1994).  However,  there  is  a limit  to  the  theoretical  utility  of  having  a number  of  design  methods  presented  as 
alternative  choices.  There  ought  to  be  common  techniques  for  the  analysis  of  these  methods  into  smaller,  re- 
usable design  components  that  can  be  adaptively  used  by  creative  instructional  designers.  Where  can  we  find 
PALs  in  these  methods  that  can  be  used  as  design  entities  independently  of  the  'macro'  methods  in  which  they 
were  originally  embedded?  Applying  this  approach  should  free  up  knowledge  embedded  in  these  larger 
methods  and  provide  a more  flexible  currency  for  the  exchange  and  re-use  of  creative  design  insights. 

The  extraction  of  PALs  from  these  larger  methods  may  be  illustrated  using  the  CORE  Approach.  This  is  a 
constructivist  method  for  designing  learning  environments  for  complex  skills  or  highly  structured  knowledge.  It 
was  first  applied  to  the  domain  of  programming  (Boyle  and  Margetts  1992,  Boyle  and  Davies  1996).  It  has 
subsequently  been  applied  to  a range  of  domains  ranging  from  the  formal  recording  of  animal  behaviour  (Boyle 
et  al  1996)  to  nuclear  medicine  (Hogg  et  al.  1999).  The  CORE  method  involves  structuring  learning  blocks 
based  on  a sequence  of  Context,  Objects,  Refinement  and  Expression.  Each  learning  block  commences  by 
setting  the  context.  This  presents  the  new  skill  embedded  in  a larger  working  environment.  This  is  followed  by 
the  provision  of  examples  of  the  learning  target  disembedded  from  the  working  context.  A refinement  section 
follows.  This  is  based  on  a series  of  carefully  chosen  questions  with  feedback.  This  dialogue  of  questions  and 
answers  enables  the  learners  to  construct  their  knowledge  of  the  target  domain  rather  than  this  knowledge  being 
transmitted  didactically.  Each  learning  block  terminates  in  an  'expression'  phase  where  the  student  applies  the 
new  skill  acquired.  At  the  curriculum  level,  each  of  these  learning  blocks  is  organized  around  a target 
competence,  with  later  learning  blocks  building  on  the  accomplishments  of  earlier  blocks. 

A designer  might  wish  to  use  certain  features  of  this  approach  combined  with  features  of  other  methods  such  as 
Cognitive  Apprenticeship.  How  can  this  selective  use  of  specific  pedagogical  features  be  achieved?  Here  we  use 
the  CORE  approach  as  an  example.  A similar  analysis  could  be  applied  to  other  methods  such  as  CaBLE  or 
Cognitive  Apprenticeship  to  yield  re-usable  PALs.  The  'Refinement'  aspect  of  CORE,  for  example,  provides 
pedagogical  techniques  that  might  be  extracted  and  used  independently  of  the  full  CORE  approach.  An 
examination  of  the  'Refinement'  reveals  that  it  has  a link  and  chain  structure  phase.  Two  main  design  patterns 
can  be  extracted  - the  link  PAL,  and  the  chain  level  PAL  which  provides  a higher  level  organization  of  the  link 
components.  In  this  discussion  we  may  focus  on  the  'link'  pattern. 

The  CORE  'link'  pattern  is  based  on  a microproblem-response-feedback  structure.  In  its  simplest  realization  this 
is  a structured  variation  on  a 'question-answer- feedback'  exchange.  CORE  uses  example-based  learning.  Each 
link  involves  the  presentation  of  an  example.  This  example  is  an  instantiation  of  some  underlying  structure,  e.g. 
a construct  in  a programming  language.  The  learner  has  to  form  a hypothesis  or  guess  about  the  underlying 
pattern  based  on  this  example.  This  underlying  hypothesis,  or  guess,  informs  the  learner's  response.  When 
students  answer  the  questions  they  receive  feedback.  This  feedback  provides  the  mature  user's  response  with  a 
short  explication  of  this  response.  For  example,  if  the  learner  is  presented  with  a yes/no  question  the  feedback 
will  always  give  the  reason  for  the  expert's  yes/no  answer.  This  is  the  most  important  part  of  the  feedback.  It 
allows  the  learner  to  compare  their  response  with  the  experts  and  use  the  feedback  to  elaborate  or  'refine'  their 
hypothesis,  or  idea,  about  the  underlying  structure.  It  is  important  to  note  that  the  feedback  provided  by  the 
software  does  not  require  any  Al  type  parsing.  The  information  is  provided  as  a knowledgeable  response.  The 
method  encourages  the  learner  to  compare  their  response  with  the  expert's  response  and  actively  refine  their 
understanding  as  a result.  The  feedback  also  adds  new  formative  knowledge  that  the  user  may  apply  to  solve  the 
next  example-based  question.  In  this  way  a didactic  exposition  of  knowledge  can  be  replaced  by  a highly 
constructivist  'example-infer-feedback'  chain.  The  structure  of  the  CORE  link  PAL  is  set  out  in  Figure  1 . 

The  'link  pattern  in  CORE  may  be  extracted  as  a PAL  entity  in  its  own  right.  In  the  CORE  method  these  links 
are  organized  in  a dialogue  'chain'  that  has  its  own  structural  properties.  However,  systematically  applying  the 
full  method  is  very  time  consuming.  Pure  example-based  learning,  as  in  CORE,  is  very  demanding  to  develop. 
A learning  environment  designer  might  wish  to  employ  smaller  sets  of  'CORE  links'  combined  with  direct 
exposition  - as  in  'modeling'  espoused  by  Cognitive  Apprenticeship.  The  separation  of  the  'CORE  link'  PAL 
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supports  this  creative  and  adaptive  re-use.  The  designer  may  wish  to  break  down  a complex  area  into  episodes 
of  exposition  followed  by  short  sets  of  CORE  links  ( e.g.  three  to  five  links).  In  this  way  the  designer  may  wish 
to  attain  a balance  between  economy  in  development  time  with  the  use  of  example-based  constructive  learning. 
The  analysis  of  CORE  and  Cognitive  Apprenticeship  into  re-usable  PALs  would  actively  supports  this  creative 
're-purposing*  by  the  designer. 


Initial  entry 

i 


Example  based 
microproblem 


i 


Figure  1:  Diagrammatic  representation  of  CORE  link  PAL 


Towards  metadesign  - discourse  about  design 

One  of  the  learning  objectives  of  constructivist  theory  is  to  encourage  metacognition  - thinking  about  thinking. 
Learners  should  not  only  learn  about  the  domain  and  skills  in  problem  solving,  they  should  also  be  able  to 
reflect  on  how  they  solve  problems  and  how  they  think  about  the  domain.  This  is  a high  level  skill.  If  this  is  a 
skill  that  should  encouraged  in  learners  it  is  certainly  a target  that  we  as  designers  should  doubly  strive  for.  It  is 
not  enough  to  do  good  design  - we  need  to  reflect,  analyze,  explicate  and  communicate  about  design  expertise. 
We  need  a disciple  of  'metadesign'  - a way  of  representing  and  communicating  about  the  process  of  design.  This 
is  not  'doing'  design;  it  is  reflecting,  explicating  and  developing  dialogues  about  the  doing  of  design.  A dialogue 
about  design  requires  a currency  in  which  communication  and  exchange  can  be  expressed.  The  idea  of  PALs 
and  design  patterns  should  make  a significant  contribution  to  this  currency  of  exchange.  It  can  provide  a 
significant  contribution  to  the  language  in  which  we  can  communicate,  think  and  solve  problems  in  the  domain 
of  learning  environment  design. 
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Conclusion 


As  a community  we  need  to  pay  greater  attention  to  the  nature  and  development  of  design  knowledge.  The  idea 
of  PALs  or  re-usable  design  patterns  provides  a productive  contribution  to  this  debate.  The  concept  of  re-usable 
PALs  points  to  a number  of  key  activities  for  the  design  community.  These  activities  include: 

* the  analysis  of  existing  'macro'  methods  into  flexible,  re-usable  design  patterns; 

* the  synthesis  of  new  knowledge  based  on  the  combination  and  adaptation  of  existing  PALs; 

* developing  a knowledge  base  to  capture  and  provide  access  to  these  design  patterns 

More  broadly  we  need  to  expand  our  theoretical  discourse  about  design.  The  concept  of  PALs  and  re-usable 
design  patterns  provides  a useful  'currency'  for  the  exchange  of  ideas  about  design  in  a form  that  is  directly 
related  to  the  needs  of  designers.  It  is  non-prescriptive.  Design  patterns  from  different  and  even  ideologically 
opposed  methods  can  be  extracted  and  re-used.  The  assessment  of  a PAL  is  its  utility  in  solving  problems  in  the 
design  of  pedagogically  sound  learning  environments. 
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Abstract 

As  educational  media  becomes  more  interactive,  there  has  been  a proliferation  of,  and  renewed  interest  in,  courses 
relating  to  multi-media  construction.  On-line  course  and  web  page  construction  now  occupy  countless  development 
hours  within  educational  institutions.  But  in  the  ‘real’  world  of  media  and  software  development,  computer 
programmers  and  graphic  designers  now  work  in  partnership  to  produce  the  final  product.  The  question  is  posed:  when 
educating  and  training  teachers  to  develop  new  media  for  the  classroom  are  we  providing  them  with  the  visual,  graphic 
and  design  skills  necessary  to  produce  quality  products? 

This  paper  will  argue  the  case  for  greater  inclusion  of  the  basic  visual,  graphic  and  design  principles  to  be  included  in 
educational  multi -media  courses.  A number  of  core  skills  and  principles  will  be  outlined  which  should  be  developed 
and  included  for  successful  course  implementation. 


Design  shouldn't  be  mere  decoration  - Roger  Black,  1997. 

Advocacy.  This  paper  unashamedly  advocates  a greater  role  for  visual,  graphic  and  design  principles  to  be 
included  in  all  educational  multi-media  courses.  In  1996  the  Australian  Good  Universities  Guide  listed  three 
specialisations  in  information  technology  (IT).  By  1999  there  were  over  fourteen  specialisations  with  dozens  of 
connected  courses.  This  is  just  one  example  of  the  dramatic  increase  or  mini  explosion  of  course  development  in 
the  area  of  information  technology  and  multi-media  in  Australia.  The  presumption  that  learning  can  be  enhanced 
through  technology,  including  on-line  courses  has  been  debated  and  supported  through  recent  studies. new 
technologies,  such  as  Web  based  delivery,  promises  superior  classroom  instruction  in  relation  to  traditional 
instructional  delivery.  (Turoff,  1995).  As  more  and  more  courses  go  on-line  and  instruction  moves  from  the 
traditional  method  educators  are  expected  to  develop  skills  to  cope  with  this  paradigm  shift.  Universities  have 
been  charged  with  the  responsibility  to  play  a major  role  in  this  shift. 

Questions  arise  to  what  is  appropriate  for  inclusion  in  a multi-media  course  instruction.  What  content  is  needed? 
What  skills  should  educators  develop?  What  course  components  should  be  categorised  mandatory  or  elective? 
The  problem  of  content  selection  could  be  (and  should  be)  as  problematic  as  the  delivery  of  the  course. 

This  paper  has  been  motivated  by  ‘apologies’.  The  researcher  has  observed  a number  of  educational  multi-media 
conference  presentations  where  authors  have  prefaced  their  talks  by  apologising  for  the  ‘look’  of  their  products, 
be  they  packages,  on-line  courses  or  web  page  design.  It  was  obvious,  through  their  own  admission,  that 
considerable  time  and  effort  had  been  given  to  such  areas  as  research,  problem  design,  navigation,  design 
solutions,  metaphor  choice,  prototype  construction,  usabiltiy  testing,  choosing  the  appropriate  cognitive 
processing  tools  but  little  effort  had  been  given  to  the  ‘look’  of  their  product. 

Informal  investigations  eluded  to  the  possibility  that  multi-media  course  instruction  minimises  the  importance  of 
visual  design  in  their  instructional  courses.  This  paper  proposes  two  notions: 

• Visual  design  is  an  integral  mandatory  component  of  all  educational  multi-media  courses 

• Connections  between  learning  cognitive  theory  and  visualisation  should  be  made  more  explicit 
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Visual  Design  is  an  Integral  Mandatory  Component  of  all  Educational  Multi-media 
Courses 

According  to  Lynch  (1994),  highly  interactive  multimedia  electronic  documents  pose  unique  graphic  design 
problems.  Visual  design  skills  affect  virtually  every  aspect  of  software  interface  design.  Students  need  to  develop 
skills  in  the  following  areas:  spatial  relations  or  layout,  perceptual  or  aesthetic  principles  (such  as,  proximity, 
similarity,  continuity,  closure  etc),  designing  for  computer  screens,  graphic  interface  design,  visual  design 
conventions  (such  as,  typography,  or  colour ),  animation,  sound.  There  are  many  principles  relating  to  interface 
design  that  should  also  be  developed  and  included  in  course  content  but  this  paper  is  mainly  dealing  with  visual 
design  skills. 

Traditional  graphic  design  principles  must  be  adapted  to  work  successfully  with  new  technologies.  There  needs 
to  be  a paradigm  shift  for  working  specifically  with  digital  media.  According  to  Heller  and  Drennan  (1997),  that 
although  a high  level  of  traditional  design  expertise  is  imperative  students  must  develop  a new  way  of  ‘reading* 
and  ‘seeing*. 

Students  need  to  be  aware  of  the  role  of  visual  communication  and  the  impact  on  the  learner.  A successful  on* 
line  course,  web  page  or  educational  multi-media  package  needs  to  bring  together  all  the  elements  of  successful 
interface  including  the  visual  design  component. 

Connections  Between  Learning  Cognitive  Theory  and  Visualisation  Should  be  Made 
More  Explicit 

The  recent  move  to  constructivist  theory  allows  educators  to  reinforce  the  ways  in  which  the  leamer*s  derive 
meaning  and  review  the  learning  process.  Traditional  theories,  such  as  Gestalt,  combine  with  recent  work  by 
Herle  (1996),  concluding  that  the  brain  works  by  making  patterns,  making  connections  and  deriving  meaning. 
Metacognition  has  allowed  educators  to  see  how  individuals  can  problem  solve  and  reflect  on  their  own  thinking 
processes.  Within  multi-media,  visualisation  allows  the  user  to  participate  in  the  learning  process.  In  reality,  the 
majority  of  educational  multi-media  produced  today  is  highly  relevant  on  visual  design  to  help  the  learner  make 
important  connections  for  meaning.  Tools  such  as  visual  mapping  tools  are  contributing  to  teaching  and  learning 
in  a constructivist-cognitive  paradigm. 

The  role  of  multi-media  education  is  to  encourage  interconnectedness  of  knowledge  while  allowing  accessibility 
to  visually  supported  information  (Hyerle,  1996).  Educators  using  new  technologies  are  now  confronted  with  a 
new  set  of  problems.  How  will  problems  be  represented?  How  do  students  control  and  access  information?  And 
for  developers  of  educational  mult-media  specifically;  how  do  you  visually  present  and  design  the  information? 

The  structure,  choice  and  visual  design  is  imperative  for  learning.  (Tufte,  1990).  According  to  Tufte  (1990), 
effective  and  efficient  visual  displays  are  borne  of  the  necessity  to  escape  the  endless  flatlands  of  the  bland 
tabulated  mass  of  predigested  data.  Designers  of  educational  multi-media  need  skill  in  visual  design  and  an 
understanding  of  how  visuals  can  provide  meaning  and  context  for  the  learner. 

Often  educational  institutions  spend  energy  and  resources  trying  to  improve  interface  and  navigation  for  multi- 
media  packages,  on-line  subjects,  web  page  construction,  while  neglecting  the  visual  impact  of  the  product. 
Visually  they  look  uninviting,  inappropriate  and  often  gives  the  user  the  impression  that  visual  quality  has  little 
importance.  Although  products  are  often  well  organised,  clear  and  easy  to  navigate,  they  can  at  times,  be  visually 
unappealing. 

Course  designers  are  charged  with  the  task  of  developing  a course  for  students  which  tries  to  pull  all  the 
elements  together:  content,  organisation,  navigation,  interaction  and  visual  design.  On  examination  of  many 
educational  multi-media  courses,  the  final  component  is  either  minimally  represented  or  totally  lacking.  If  this  is 
the  case,  it  raises  a number  of  questions.  What  design  principles  should  be  included  in  a multi-media 
instructional  course  to  allow  students  to  develop  a design  aesthetic?  What  design  rules  and  techniques  need  to  be 
employed?  What  skills  should  be  developed  by  students  for  greater  success?  How  much  time  should  be  devoted 
to  the  components  of  visual,  graphic  and  design  principles  within  a course? 
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What  is  important  is  that  visual  design  should  not  be  just  an  added  component,  but  instead  an  integrative, 
integral  part  of  the  total  design  process.  Designers  have  acknowledged  cognitive  recognition  is  central  to  the 
process  of  interface  design...  then  an  added  basic  understanding  of  visual  elements  and  their  employment  in  the 
design  projects  would  only  enhance  the  process.  For  successful  interface  designers  must  have  a well  rounded 
understanding  of  perceptual  elements  and  the  affect  on  the  user.  Often  we  find  that  design  fundamental  rhetoric 
outlined  for  effective  interface,  focuses  merely  on  the  functional  characteristics  of  the  design.  (Hobart  1997, 
Mullet  1994,  Mountford  1990,  Laurel  1990,  Apple  1988).  That  is,  visual  structures  are  identified,  such  as, 
buttons,  icons,  metaphors,  text  fields,  windows  without  recognising  the  importance  of  visual  elements  in  the 
design  (colour,  shape,  size,  direction,  texture).  (Salomon  1990,  Black  1997) 

Visual  design  skills  affect  virtually  every  aspect  of  the  multi -media  construction.  According  to  Lynch,  visual 
design  skills  affect  every  component  from  simple  generic  objects  like  windows  to  the  animated  illustration  of 
interactive  three-dimensional  structures  within  electronic  documents.  (Lynch  1994;  Marcus  1992;  Schneiderman 
1992;  Tognazzini  1992,). 

Students  enrolled  in  educational  multi-media  construction  courses  do  not  need  to  be  qualified  graphic  designers 
or  artists,  but  they  do  need  fundamental  skills  and  exposure  to  principles  of  graphical  user  interface,  including 
visual  design  principles. 

The  following  principles,  skills  or  knowledge,  (in  relation  to  the  visual,  graphic  and  design  area),  should  be 
included  in  all  multi-media  construction  courses.  This  is  not  an  exhaustive  list  but  instead  provides  a basis  for 
initial  scholarly  debate  and  a review  of  existing  course  content. 

Students  need  to 

• be  exposed  to  visual  design  conventions; 

• be  able  to  develop  a consistent  visual  structure; 

• understand  and  employ  visual  design  elements; 

• understand  and  apply  robust  graphics; 

• employ  discretion  concerning  typography; 

• understand  the  need  for  clear  graphic  layout; 

• be  able  to  identify  and  apply  visual  layering; 

• be  aware  of  spatial  relationships; 

• see  the  relationship  between  visual  and  functional  goals; 

• be  aware  of  the  positive  use  of  the  visual  elements. 

As  Lynch  (1994)  contends,  elements  within  the  design  must  appear  in  a stable,  intuitive  and  predictable 
relationship.  This  paper  argues  the  notion  that  this  will  not  occur  by  chance...  it  must  be  developed,  nurtured  and 
taught.  The  author  challenges  course  designers  to  review  course  content,  re-evaluate  time  allocation  and  include 
a visual,  graphic  and  design  component  in  your  courses. 
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Abstract:  The  purpose  of  this  paper  is  to  discuss  the  impediments  to  distance  education  (DE)  programs  and  the  critical  value 
of  interaction  and  dialog  in  DE  learning  environments.  Many  speciflc  types  of  interaction  like  student-student  and  student- 
content  will  be  presented.  Examples  showcasing  distance  delivered  k-12  support  curriculum  projects  from  the  National 
Aeronautics  and  Space  Administration  (NASA)  will  then  be  discussed  with  respect  to  their  interaction  and  collaboration.  The 
NASA  examples  provided  will  be  tied  back  to  the  pedagogy  surrounding  distance  education  interaction. 


Impediments  to  Distance  Education 

Distance  education  (DE)  has  usually  been  defined  as  the  separation  between  instructor  and  student,  whether 
temporally  or  by  spatial  distance.  While  this  physical  and  temporal  distance  is  a fact,  it  erroneously  limits 
the  factors  affecting  learning  at  a distance.  Transactional  distance  more  aptly  describes  the  relational 
separation  between  learner  and  instructor  and  is  based  on  the  interplay  between  the  environment,  patterns 
of  behavior  or  dialog  between  the  learner  and  teacher  and  the  autonomy  of  the  learner  (Moore,  1996). 

With  the  physical  separation  of  the  leaner  and  instructor  there  are  psychological  and  sociological 
implications  that  limit  communication  (Wolcott,  1996).  In  effect  the  psychological  distance  may  generate  a 
feeling  of  isolation  for  the  student  and  hinder  the  development  of  rapport  between  the  learner  and  teacher. 
Wolcott  (1 996)  states  that  it  is  this  rapport  that  allows  the  learning  discourse  between  teacher-student  and 
student-student  relationships  to  occur.  Social  presence  or  acceptance  as  perceived  by  the  distance  learner  is 
a significant  factor  for  consideration  (Mclsaac  & Blocher,  1 998)  and  should  be  included  in  the  design 
process. 

Distance  education  programs  malfunction  due  to  their  lack  of  consideration  for  the  human  dimension 
(Spitzer,  1998).  While  tremendous  strides  have  been  made  in  telecommunication  hardware,  Spitzer  (1998) 
states  that  it  is  the  social  interaction  level  between  humans  that  will  determine  if  a DE  program  fails  or 
succeeds. 


Necessity  of  Interaction 

Moore  (1997)  describes  the  necessity  of  a purposeful  instructional  dialog,  which  facilitates  and  increases 
the  interaction  between  student-student  and  teacher-student  communication.  In  a purposeful  dialog 
environment  all  learners  are  active  participants,  listening  and  building  upon  the  contributions  of  other 
individuals.  Moore  continues  on  that  dialog  is  also  influenced  by  teacher  and  learner  personality  and 
content.  Thus,  intentionally  designed  collaborative  opportunities  should  be  incorporated  into  the  instruction 
for  distance  learning  environments  to  be  most  effective.  As  dialog  and  learner  control  increase  transactional 
distance  decreases  (Saba  & shearer,  1994).  For  learning  is  not  an  objective  search  for  prescribed 
knowledge,  but  one  that  is  experienced  and  formulated  based  on  how  we  interact  with  our  environment  and 
others  (Duffy  & Jonassen,  1991). 

Equally  effecting  the  exchange  of  dialog  or  interaction  between  participants  is  the  way  a distance  learning 
program  is  structured.  Selecting  the  right  instructional  strategy,  such  as  that  of  interaction  facilitates 
learning  of  the  higher  levels  of  cognition  such  as:  analysis,  synthesis,  and  evaluation  (Weston  & Cranton, 


1 986).  Programs  that  are  extremely  rigid  and  that  do  not  allow  flexibility  for  student  contributions, 
interactions  or  discussions  are  detrimental  to  the  learning  outcomes  desired  (Moore,  1996). 

Several  forms  of  interaction  are  possible  in  a DE  environment,  all  of  which  should  be  considered  during  the 
instructional  design  phase  of  a course.  Interaction  occurs  between  the  following  categories:  learner- 
instructor,  learner-learner,  and  learner-content  (Wagner,  1997).  A fourth  dimension  of  interaction  was 
added  in  1994,  that  between  the  learner  and  the  interface(Hillman,  Willis,  & Gunawardena,  1994).  When 
designing  instruction  one  should  first  should  look  at  the  intended  instructional  outcomes  desired  for  a 
learning  experience,  analyzing  the  type  of  learning  that  is  desired  (verbal  or  intellectual  skills,  psychomotor 
skills  or  attitude  changes),  then  select  the  appropriate  type  of  interaction  that  will  facilitate  this  type  of 
learning  (Wagner,  1997).  A brief  list  of  the  types  of  interaction  possible  will  finish  this  section. 


Types  of  Interaction: 

Interaction  of  the  following  types  should  be  considered  when  designing  a DE  program  (Wagner,  1997): 

1.  Interaction  to  increase  Learning 

2.  Interaction  to  increase  participation 

3.  Interaction  to  develop  communication 

4.  Interaction  to  receive  feedback 

5.  Interaction  to  enhance  elaboration  and  retention 

6.  Interaction  to  support  learner  control/self-regulation 

7.  Interaction  to  increase  motivation 

8.  Interaction  for  negotiation  and  understanding 

9.  Interaction  for  team  building 

10.  Interaction  for  discovery 

1 1.  Interaction  for  exploration 

12.  Interaction  for  clarification  of  understanding 

13.  Interaction  for  closure 

With  the  impediments  to  distance  education  and  the  necessity  of  interaction  in  DE  environments  discussed, 
a look  at  several  instructional  examples  from  NASA's  k-12  education  program  will  next  be  presented.  The 
first  NASA  program  discussed  below  takes  advantage  of  the  interaction  between  teacher-student  and 
student-student  to  increase  learning,  participation,  team  building,  motivation,  negotiation  and 
understanding. 


NASA  Online  Interactive  Projects:  NASA  Quest/Sharing  NASA  (bttp://quest.arc.nasa.gov) 

Senator  A1  Gore  originally  funded  this  project  in  1991  as  part  of  the  NASA  High  Performance  Computing 
and  Communications  (HPCC)  program.  Capitalizing  on  the  motivational  aspects  of 
collaborating/communicating  with  NASA  individuals,  teachers  have  expressed  the  value  of  the  "Sharing 
NASA"  supplementary  support  curricula  aligned  with  the  National  Science  Education  Standards  (NSES). 
Sharing  NASA  provides  "real-world"  authentic  contexts  and  situated  learning  opportunities  that  connect  the 
science  content  and  process  skills  currently  taught  in  the  classroom,  transcending  the  physical  and  temporal 
classroom  limitations. 

At  the  link  above  you'll  find  a listing  of  online  collaborative  NASA  projects  where  one  may  participate  in  a 
variety  of  conferencing  opportunities  with  NASA  scientists,  engineers,  researchers  and  support  personnel 
via  email,  synchronous  chats,  asynchronous  threaded  discussions,  and  streaming  video.  In  addition  to  the 
interaction  opportunities,  online  activities,  project  background  information,  mission  updates,  and 
biographies  are  available. 

A combination  of  software  solutions  and  "Smartfilter"  designates  (volunteer  teachers,  paid  employees) 
manage  the  enormous  influx  of  inquiries,  aggregating  common  questions  and  facilitating  the  asynchronous 
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interaction  between  the  limited  time  of  the  NASA  employee  participants  and  the  classrooms  across  the 
nation. 

Past  examples  of  "Sharing  NASA"  projects  are  archived  and  available  online.  One  example  will  now  be 
discussed. 


Live  From  Mars:  July  1996-December  1997  rhttD://quest.arc. nasa.gov/lfm/index.html) 

Created  in  conjunction  with  the  "Passport  to  Knowledge-Live  From"  series,  the  "Live  From  Mars"  Sharing 
NASA  project  had  a host  support  curriculum  with  the  following  programmatic  categories:  featured  events, 
live  video,  photo  galleries,  questions,  chat,  kids  comer,  teacher’s  lounge,  background.  Mars  team 
(biographies,  images),  and  whaf s new. 

One  featured  event  Weather  Worlds  challenged  student  classrooms  from  around  the  country  to  debate  what 
key  weather  measurements  and  instruments  they  thought  were  most  important  to  gather  here  on  Earth 
(similar  to  decisions  the  NASA  Mars  Pathfinder  scientists  had  to  decide  for  the  surface  of  Mars). 
Classroom  submitted  their  proposals  online  with  justifications  for  debate.  For  example,  if  a classroom  felt 
temperature  was  a worthy  measurement,  they  then  had  to  figure  out  protocols  and  procedures  on  how, 
when  and  within  what  range  and  frequency  they  should  collect  temperature  data. 

Electronic  individual  class  results  were  collected  by  the  "Sharing  NASA”  personnel,  placed  in  a 
spreadsheet,  and  responses  aggregated  and  debated  online.  From  these  collective  results,  final  instruments 
and  protocols  were  tallied  and  presented. 

Ironically,  this  type  of  interaction  doesn’t  necessitate  high  bandwidth  connectivity  or  a computer  lab.  The 
learning  is  done  in  the  classroom  facilitated  by  the  teacher,  then  submitted  via  email  for  presentation  and 
debate.  Current  "Sharing  NASA"  projects  include; 

1 . Space  Scientists  Online 

2.  Aero  Design  Team  Online 

3.  Women  of  NASA 

The  next  NASA  educational  curriculum  support  area  that  will  be  discussed  focuses  on  the  interaction 
between  the  learner  and  the  content,  and  learner-interface  while  also  integrating  teacher-student  and 
student-student  interactions  to  facilitate  learning,  exploration,  feedback,  and  elaboration  and  retention. 


NASA’s  Learning  Technologies  Project:  (Tittp://leam.ivv.nasa.gov) 

Here  you’ll  find  the  entry  page  to  approximately  50  NASA  funded  Internet  projects  focusing  on  interaction 
between  the  learner  and  content.  NASA  is  supporting  the  development  of  projects  in  disciplines  like 
Agriculture,  Aeronautics,  Aquatics,  Earth  Science,  Tourism,  and  Space  Science.  At  this  site  one  may  search 
for  projects  by  discipline  or  state.  One  project  from  NASA's  LTP  will  be  referenced  below. 

NASA  Glen  Research  Center: 

FoilSim  (Basic  Aerodynamics  Software) 
http://www.lerc.nasa.gov/WWW/K-12/aerosim/index.html 

FoilSim  is  simulation  software  that  allows  students  to  interact  and  control  the  parameters  of  airflow  around 
a wing  or  baseball  by  varying  the  airspeed,  altitude,  thickness,  camber,  and  area.  Students  also  have  learner 
control  over  the  placement  of  the  probe  gathering  aerodynamic  data,  and  receive  immediate  feedback  via  a 
Plotter  View  Panel  that  depicts  graphical  changes  in  surface  pressure,  speed  at  surface  and  lift.  The  student 
can  also  visually  see  how  far  a curve  ball  would  "break"  across  homeplate  for  various  elevations 
(Cleveland,  Denver,  Mt.  Everest).  This  Java  applet  (Mac  or  PC)  when  used  in  conjunction  with  guided 
questions  and  the  necessary  content  background  facilitates  the  learning  of  the  factors  that  influence  lift. 
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The  final  NASA  educational  project  that  will  be  discussed  concerns  the  interactions  between  learner- 
interface,  learner  content,  and  learner-learner  while  enhancing  learning,  communication,  feedback,  team 
building,  exploration  and  discovery. 


EarthKAM  (littp://www. earthkam.ucsd.edu/) 

EarthKAM  is  an  opportunity  for  students  to  investigate  Earth  from  the  unique  perspective  of  space  via 
digital  photographs  they  request  during  live  space  flights  and  from  previous  mission  image  archives. 

Piloted  on  4 previous  space  shuttle  missions,  EarthKAM  will  be  the  first  operational  payload  on  the 
International  Space  Station. 

In  essence,  students  plan  investigations  that  can  in  part  be  answered  from  examination  and  manipulation  of 
remotely  sensed  images.  For  example  EarthKAM  can  be  used  to  answer  questions  involving  real  world 
problems  like  deforestation,  beach  erosion,  urban  crawl,  etc. 

Uniquely  designed  web  interfaces  allow  students  to  search  from  images  either  visually  by  geographic 
location,  by  image  index  number,  by  mission,  by  school,  or  by  geographic  name  (China,  Nile  River,  etc.). 
Once  a particular  image  is  selected,  students  can  then  examine  the  image  by  zooming  in/out  or  panning  left, 
right,  up,  and  down.  To  place  the  image  in  the  "big  picture"  a student  can  overlay  the  digital  photograph 
onto  a composite  map  showing  the  elevations  and  contours  of  the  surrounding  areas  or  a Jet  Navigation 
Map  (JNM)  displaying  the  location  and  names  of  major  geographic  reference  points  (rivers,  cities,  etc.). 

Also  available  is  the  ability  to  view  the  map  against  the  animated  cloud  cover  that  was  present  (within  3 
hours)  on  the  day  the  image  was  taken  using  weather  satellite  remote  sensing  image  overlays.  Finally,  with 
the  use  of  a VRML  browser  plug-in,  the  student  can  take  a composite  map  (image  overlaid  against 
surrounding  colored  contour  map  or  JNM)  and  visualize  the  image  in  a 3-D  environment.  This  environment 
allows  the  student  to  fly  by,  fly  over,  fly  under,  or  fly  around  the  image  with  the  elevations  literally 
portrayed  with  vertical  depth  as  well  as  2-D  color.  Finally,  using  free  image  editing  software  (NIH  image), 
student  can  label  their  photographs  and  make  measurements  like  scale  distances,  area,  and  perimeter. 

This  environment  provides  complete  learner  control  and  higher  learner-interface,  learner-content 
interaction.  It  should  be  said  that  while  this  program  is  extremely  technologically  savvy,  without  proper 
instruction  and  ancillary  reference  documents  to  facilitate  worthy  investigative  questions,  image 
manipulation  alone  merely  becomes  a novelty  toy. 


Summation: 

For  distance  education  to  be  effective  design  efforts  must  consider  the  impediments  that  hinder  effective 
learning  such  as:  transactional,  psychological  and  sociological  distance  (Moore,  1996;  Wolcott,  1996). 
Intentionally  designed  programs  that  foster  interaction  and  dialog  between  the  student-teacher  and  student- 
student  enhance  learning  and  emphasize  specific  learning  skills  if  properly  facilitated  (Wagner,  1997). 
Several  NASA  education  support  curriculum  projects  seem  to  emphasize  interaction  and  were  described 
herein. 
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Abstract 

The  building  of  collaborative  online  communities  raises  questions  about  the  nature  of 
discourse  in  these  environments:  Underlying  these  questions  are  fundamental  notions  about 
the  social  construction  of  knowledge.  CMC  has  the  potential  to  even  out  social/technology- 
related  gender  issues.  However,  a body  of  literature  has  emerged  from  various  domains  that 
describes  the  presence  of  gendered  discourse  patterns  in  online  communities.  Would  these 
gendered  patterns  extend  to  the  discourse  patterns  among  facilitators  and 

conference  participants'?  What  effect  would  these  patterns  have  on  the  conversation, 
especially  for  women  participants?  This  exploratory  study  analyzes  the  CMC  transcripts  of 
two  junior  undergraduate  courses  employing  a conference  forum  as  part  of  the  learning 
design.  A framework  integrating  several  theories  of  online  discourse  contextualizes  findings. 


The  1980’s  saw  a large-scale  post-secondary  technology  infrastructure  project  supporting  the  use  of 
productivity  tools  such  as  word  processors  and  spreadsheets  in  numerous  computer  labs  across  campus.  The  IT 
field  focused  on  the  creation  of  optimal  blueprint  models  of  computer-based  instruction  in  which  learners 
worked  alone,  often  on  tasks  that  were  procedural  and  criterion-based,  with  little  interaction  among  themselves, 
with  their  instructors,  or  even  with  the  authentic  content  of  the  domain  (Ewing,  Dowling,  & Coutts,  1998).  The 
nature  of  the  technology  itself  maintained  a hierarchical  relationship  between  instructors  and  students  in  which 
the  instructor  was  regarded  as  the  Intellectual  Authority;  the  source  of  all  knowledge  in  the  classroom. 

However,  with  increasing  access  to  other  sources  of  knowledge,  and  new  constructivist  approaches, 
learning  designs  now  focus  on  social  interaction  among  communities  of  learners.  The  building  and  maintenance 
of  online  communities  as  forums  in  which  to  construct  and  share  knowledge  naturally  raises  questions  about  the 
nature  of  discourse  in  these  communities:  What  counts  as  knowledge?  Whose  knowledge  counts?  And  how  is 
it  valued  in  computer-mediated  conversation  (CMC)?  Underlying  these  questions  are  fundamental  notions 
about  the  social  construction  of  knowledge  through  language,  and  especially  through  computer-mediated 
language  in  virtual,  asynchronous,  online  communities  in  which  the  traditional  and  taken-for-granted  cues  of 
social  discourse  are  mostly  absent.  Sherry  Turkle  describes  a new  social  construction  of  computers,  in  which  a 
“new  set  of  intellectual  and  emotional  values  more  like  those  applied  to  harpsichords  than  hammers"  prevails 
(p.280,  in  Kirkup,  1992).  Here,  groups  that  have  been  traditionally  excluded  by  an  epistemological  value  set, 
such  as  women,  will  find  a learning  environment  that  distributes  “authority  to  bring  everyone  into  an 
educational  process  extending  beyond  time  and  physical  space  and  sociopolitical  boundaries”  (Slatin,  1992,  p 
50).  CMC  goes  beyond  a technological  tool  to  provide  a pedagogical  framework  that  can  encourage  active 
learning  in  a context  of  "constant  collaborative  conversation”  (Hollingsworth,  in  Elliot  & Woloshyn,  1997). 

Much  has  been  written  about  the  potential  of  CMC  to  even  out  the  gender  inequities  described  in 
educational  technology  studies  over  the  past  30  years.  Equally,  however,  a body  of  literature  has  emerged  from 
sociolinguistics,  discourse  analysis,  feminist  theory,  and  educational  computing  research  that  describes  the 
presence  of  gendered  discourse  patterns  in  online  communities.  We  were  curious  to  know  whether  that 
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gendered  pattern  extended  to  the  discourse  patterns  of  online  facilitators.  As  a first  step  towards  understanding 
the  problem,  we  have  analyzed  two  junior  undergraduate  courses  in  which  a conference  forum  was  an  integral 
part  of  the  learning  design.  One  facilitator  was  male;  the  other  female.  In  the  first  phase,  reported  here,  we 
intended  to  develop  a rubric  from  which  to  examine  and  describe  any  gender-related  differences.  The  next  step 
will  explore  their  effects  on  the  conversation  of  the  learning  community. 


Theoretical  Framework 

This  study  is  embedded  in  four  intersecting  research  strands.  Taken  together  from  a feminist 
perspective,  they  form  an  epistemological  framework  for  investigating  the  design  and  implementation  of 
technology-based  learning  environments  in  the  post-secondary  setting.  These  strands  are:  the  exploration  of 
technology  as  a social  construct;  the  seminal,  yet  controversial,  work  characterized  by  the  phrase  women 's  ways 
of  knowing  (W  WK);  and  the  emerging  research  base  in  gender  characteristics  of  online  discourse.  The  promise 
of  CMC  for  constructivist  learning  is  the  fourth  strand  that  weaves  them  all  together. 


Technology  as  Social  Construct 

Women’s  relationships  with  technology  have  been  aptly  summed  up  by  the  admonishment,  ‘Don’t 
touch  it,  you’ll  get  a shock’  (Turkic  in  Spender,  1995,  p.  173).  Computer  culture  may  reproduce  and  reinforce 
sexist  gender  ideologies  (deCastell  in  Luba,  1997,  p.23)  represented  by  technological  innovations  such  as 
individualized  computer-based  learning  software,  used  in  contexts  (such  as  computer  labs)  that  are  isolated  and 
unappealing  to  women.  An  established  body  of  work  in  the  social  sciences  has  emphasized  that  innovations 
reflect  social  constructs  (Cockbum  & Ormond,  1993,  p.  7).  Therefore,  technology  is  a culture  out  of  which 
artifacts  are  created. 

If  men  have  been  the  creators  and  designers  of  technological  artifacts  (the  thinking  goes),  men’s  social 
domination  of  the  technological  culture  is  extended.  Technology  relations,  then,  ultimately  become  gender 
relations  (Cockbum  & Ormond,  1993,  p.  155).  On  the  political  face  of  it,  technological  solutions  to  power  over 
women  and  the  contexts  in  which  they  live,  work,  and  learn  are  global  issues  of  knowledge  creation  and 
representation,  language  use,  and  access,  and  could  be  condemned.  However,  if  culture  is  socially  negotiated 
rather  than  imposed,  CMC  can  be  used  as  a critical  pedagogy;  an  agent  of  change  in  the  real  world. 

Beardon  and  Worden  (1995)  describe  a collaboratively  constmcted  learning  environment  in  which  we 
see,  "the  value  of  a broad  experience  of  possible  applications;  (and)  the  advantages  of  students  setting  their  own 
agendas  "(p.  65).  Implementing  electronic  communications  in  the  classroom  is  a decision  to  build  an  active 
community  for  learning  that  is  grounded  in  conversation. 

In  its  role  in  sustaining  relationships  with  others,  Bmffee  (1993)  characterizes  conversation  as 
connected  knowing  and  the  site  for  constructing  knowledge.  This  aspect  of  conversation  reflects  its  centrality  in 
the  constmctivist  environment,  because  dialogue  is  a cognitive  process  in  which  students  work  together  to 
decenter,  moving  beyond  personally  held  views  to  construct  new  and  expanded  representations  (Fosnot,  1996), 
and  transforming  classroom  relationships  (Slatin,  1992,  p.30).  However,  teachers  may  have  trouble  re- 
conceptualizing their  roles  in  these  active,  participatory,  and  democratic  environments  (Ewing,  Dowling  & 
Courts,  1998,  p 6).  We  wondered  if  the  difficulty  in  this  process  is  inherently  gender-related. 

In  the  virtual  classroom  CMC  provides  both  a pedagogical  framework  and  a strategy  for  collaborative 
knowing.  Pedagogically,  CMC  emphasizes  extensive  and  sustained  interaction  that  is  more  student-centered, 
than  teacher-controlled  (Kearsley,  Lynch,  & Wizer,  1994).  Strategically,  the  asynchronous  nature  of  CMC 
expands  user  control  because  learners  can  control  the  timing,  amount,  and  pace  of  their  contributions  since 
opportunities  to  speak  are  not  limited  or  constrained  by  time  or  turn-taking  concerns,  or  to  social  position  and 
language  ability. 


Women’s  ways  of  knowing 

Women  may  be  socialized  to  process  information  in  concrete,  relational,  and  associative  ways  (Elliot 
& Woloshyn,  1997),  as  opposed  to  the  abstract,  systematic  ways  required  by  androcentric  designs  (c.f  Shade, 
1993;  Winkelman,  1997).  This  view  has  informed  feminist  criticism  of  traditional  learning  design  for  two 
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decades.  Articulated  by  Carol  Gilligan  in  1982,  and  elaborated  in  the  work  of  Belenky,  Clinchy,  Goldberger, 
and  Tarule  (1986,  1996),  women's  ways  of  knowing  (WWK)  describes  an  alternate  view  of  cognitive 
development  in  which  the  received  knowing  of  Western  intellectualism  is  contrasted  with  the  relational  position 
oi connected  knowing.  The  criticism  of  malestream  technology  design  is  exemplified  by  Margaret  Wylie's 
(1995)  observation  that  "the  Internet  is  male  territory.  Considering  its  roots  are  sunk  deep  in  academia  and  the 
military-industrial  complex,  that’s  hardly  surprising"  (p.  3).  The  WWK  framework  suggests  that  knowledge  can 
be  socially  constructed  in  ways  that  include  intuition,  emotion,  and  experience;  and  shared  through  relationship 
and  conversation.  Therefore,  much  of  the  work  of  WWK  continues  to  have  a critical,  activist  stance. 

The  adult  learning  literature  suggests  different  learning  styles  that  are  somewhat  gender-related. 
Theorists  in  this  field  suggest  two  developmental  paths  (Table  1):  the  autonomous,  separate,  or  independent 
path,  typifying  most  men  (and  some  women);  and  the  relational,  connected,  or  interdependent  path,  typifying 
most  women  (and  some  men).  Virtual  learning  designs  based  on  relational  concepts  may  accommodate  learners 
who  are  autonomous  or  separate  as  well  as  connected  learners,  but  not  vice  versa  (MacKeracher,  1996). 


Separate  Knowers 

Connected  Knowers 

• are  concerned  with  controlling  their  environment 

• are  concerned  about  remaining  connected  to  others 

• adopt  a highly  critical  and  adversarial  stance  when 
disagreeing  with  another 

• adopt  a narrative  or  descriptive  stance  when 
disagreeing 

• force  a choice  if  there  appear  to  be  opposing  views 

• search  for  a way  to  incorporate  the  ideas  of  all 

• are  independent 

• are  interdependent 

• are  wary  of  collaborative  learning 

• prefer  collaborative  learning 

• use  analytical  thinking  styles 

• use  holistic  thinking  styles 

• look  for  logic  and  consistency  as  signposts  of  truth 

• test  for  truth  by  using  believability  measures 

• prefer  to  keep  their  thoughts  and  feelings  separate 

• are  better  able  to  integrate  thoughts  and  feelings 

Table  1:  Paths  of  Knowing 


Gender  and  online  discourse 

Much  has  been  written,  and  expected  of,  the  virtual  community  of  CMC  as  social  barriers  of  sex,  age, 
race,  physical  appearance,  and  facility  in  face-to-face,  real-time  discourse  are  apparently  eliminated  (or  at  least 
mediated).  CMC  provides  an  opportunity  to  solve  the  problem  of  inequitable  distribution  of  dialogue  space, 
reflected  in  gender  inequalities  in  language  use  (Spender,  1995).  However,  Kirkup  (1995)  suggests  that  the  style 
of  talk  that  CMC  participants  are  required  to  engage  in  is  very  dialogue-oriented,  which  privileges  the 
expository  style  of  men.  These  communicative  styles,  or  cultures,  are  described  by  Deborah  Tannen  (1990),  a 
sociolinguist,  whose  work  has  been  extended  to  include  data  from  a study  of  men  and  women  communicating  in 
the  workplace  (1994a)  and  online  (1994b).  Herring  (1996),  for  example,  described  women’s  tendency  to 
attenuate  (Tannen’s  rapport  talk)  online,  whereas  men  tended  to  adopt  an  aggressive  and  adversarial  stance, 
challenging  the  ideas  of  others  (Tannen’s  report  talk).  Tarule  (1996)  points  out  a gendered  discourse  style  that 
emphasizes  a game  and  privileges  a separate  knowing  approach  to  understanding  and  analyzing  dialogues  (p. 
279).  Dialogue  as  debate  defeats  a constructivist  learning  design,  and  does  not  reflect  women's  ways  of 
knowing.  Do  we  see  this  gendered  style  in  facilitator  talkl  Table  2 identifies  several  gender-related  styles  of 
discourse  noted  in  online  forums  (Herring,  1996;  Hiltz  & Turoff,  1993;  Taylor  and  others,  1993): 


Women 

Men 

• Attenuated  assertions,  consensual 

• Apologies,  approbation 

• Explicit  justifications 

• Concrete  examples  (often  from  experience) 

• Questions  seeking  answers 

• Personableness 

• Supports  others,  cooperative 

• May  employ  some  features  of  male  language  in 
order  to  be  taken  seriously 

• Strong,  forceful  assertions 

• Self-promotion 

• Presuppositions 

• Rhetorical  questions,  abstract  examples 

• Authoritative  orientation,  ownership  of  ideas 

• Challenges  others,  competitive,  adversarial 

• Humour/sarcasm/mild  profanity/sexual  innuendo 

• Directness,  task-orientation 

• Control  of  cross-gender  conversations 
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Table  2:  Styles  of  Discourse 


Accepting  the  view  that  knowledge  is  a product  of  social  interchange,  the  nature  of  that  exchange 
through  electronic  conversation  is  pivotal  in  the  experience  of  meaning-making  (Bohan,  1993).  And,  if  the 
nature  of  conversation  in  these  environments  is  gendered,  the  social  construction  of  knowledge  is  more  likely  to 
reflect  a gendered  worldview.  Learners  (especially  younger  and/or  novice  learners)  may  expect  an  online 
experience  resembling  traditional  classroom  discourse  in  which  the  instructor  is  the  Intellectual  Authority, 
knowledge  is  received,  and  peer  perspectives  are  not  welcomed  or  valued  (Magolda,  1 992).  The  expectations  of 
both  learners  and  instructors  are  quite  likely  gendered  in  this  regard.  We  believe  it  evident  that  facilitator  talk, 
which  represents  and  supports  a particular  view  of  knowledge  representation,  is  pivotal  in  developing  learning 
environments  that  reflect  either  received  or  connected  knowing. 

The  Study 

To  help  make  sense  of  the  many  relevant  theories  and  models  related  to  the  four  strands  elucidated 
above,  we  constructed  a research  rubric  that  attempts  to  weave  the  main  ideas  together  along  three  (of  many 
possible)  dimensions  of  the  facilitator’s  online  discourse  patterns,  and  their  potential  underlying  epistemologies, 
shown  in  Table  3,  below. 


Issue 

Hypothesis 

Examples  of  associated  literature 

Sex  1 Tendency 

....  , ^ . : , . - ^ 

Dimension  1:  Task  Role 

Focus  of  facilitation 

F 

Socio-emotional  group  process 

Berge,  1995;  Kirkup,  1992; 
McConnell,  1994;  Savicki,  1995 

M 

Task  maintenance,  procedural 

Personal  orientation 

F 

Learner-centered 

Kearsley  et.al.,  1994 

M 

Teacher-centered 

CMC  as  pedagogical 
strategy 

F 

Activist,  collaborative,  relational 

Mason,  1990;  Rogers,  1996: 
Winkelman,  1997 

M 

Task  maintenance,  intellectual 

Dimension  2:  Discourse  Style 

Initiating  a new  thread 

F 

Shared  responsibility 

Slatin,  1992 

M 

Self 

Answering  technical  or 
management  questions 

F 

Will  re-direct  to  group 

Berge,  1995;  Mason  1990; 

M 

Answers  himself;  management  focus 

Ratio  of  facilitator  talk 
to  participant  talk 

F 

Gives  online  wait  time,  encourages 
participants  to  talk  to  each  other 

Kirkup,  1992;  Shade,  1993; 
Spender,  1995;  Taylor  et.al,  1993; 
Wylie,  1995 

M 

Will  dominate 

Length  of  message 

F 

Shorter 

Blanchette,  1999,  Herring  (1996) 

M 

Longer 

Nature  of  facilitator 
talk 

F 

Exploratory,  believing  game 

Herring,  1996;  Tannen,  1990; 
Winkelman,  1997;Tarule,  1996 

M 

Expository,  doubting  game,  rhetorical 

Purpose  of  language 
use 

F 

Attenuating,  collaborative 

Kirkup,  1992;  Tannen,  1990; 
Tarule,  1996 

M 

Assertive,  argumentative 

Dimension  3:  Source  of  Authority 

Direction  of 
interaction 

F 

Each  other  as  sources  of  knowledge 

WWK;  Mackeracher,  1996; 
Magolda,  1992 

M 

To  content,  to  self 

Messages  referenced 
most  often  as  evidence 

F 

Student  messages,  or  experts 

Slatin,  1992 

M 

Own  messages  as  expert,  other  experts 

Knowledge 

representation 

F 

Others,  intuition,  self,  narrative.  Truth 

Beardon  & Worden,  1995; 
Magolda,  1995;  Tarule,  1996 

M 

Truth,  self 

Table  3:  Research  Rubric 
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Preliminary  Results:  Table  4 


Both  first-year  undergraduate  courses,  from  two  different  Faculties,  each  had  a face-to-face  (F2F) 
component  supported  by  an  online  conference.  The  male  facilitator’s  (MF)  class  size  was  1/3  that  of  the  female 
facilitator's  (6  to  18,  respectively);  he  also  had  a student  participating  by  videoconference.  The  MF’s  students 
were  less  active  which  may  be  reflected  in  his  level  of  direct  participation.  The  FF  replaced  some  F2F  sessions 
with  CMC,  while  the  MF  did  not.  The  MF  teaches  in  a bilingual  Faculty,  i.e.  he  is  a language  teacher  as  well  as 
a content  expert.  The  FF  teaches  in  the  Humanities;  the  MF  in  a science-based  domain.  These  contexts  are 
important  as  facilitator  talk  is  embedded  in  social  constructs  related  to  academic  disciplines  as  well  as  gender. 


Dimension 

MF 

FF 

Task  Role 

• Interactions  emphasize  task  maintenance; 
management  and/or  procedurally  oriented 

• Initially  suggests  ways  to  build  a community. 

• Reminds  students  to  keep  talking  to  each  other 

• Informative,  very  active 

• Withdraws  as  tutorials  progress 

• Elaborates  on  student  responses 

• Stresses  evidence-based  answers 

• Directs  attention  to  class  resources 

• Experience/beliefs  acceptable  evidence 

Discourse 

Style 

• Always  initiates  new  thread  or  topic,  based 
on  module  topics 

• Sets  tutorial  boundaries  but  students  initiate 
threads,  sometimes  based  on  student  interests 

• Very  helpful  personally 

• Invites  help  on  technical  questions 

• '42%  of  total  postings 

• Usually  responds  to  individual  student 
postings 

• Averages  1 1%  of  total  postings 

• Waits  average  of  14  s.  postings  before 
responding 

• Length  often  50%  of  students'  posts 

• Varies  from  10-50%  of  students'  posts 

• Thanks  students  for  contribution;  evaluates 
responses 

• Compliments  students  by  name 

• Often  refers  to  literature  and  research 

• Thanks  students  by  name  for  contribution; 
rarely  offers  evaluative  opinion 

• Sometimes  synthesizes  and  weaves  comments 
into  course  content,  notes  literature 

• Provides  evaluative  feedback 

• Attenuates:  ‘it  seems  to  me. . . If  I can. . ." 

Source  of 
Authority 

• Does  not  usually  redirect  conversation 

• Student  messages  contain  evidence;  ideas 

• Expert,  evidence-based 

• Importance  of  how  others  interpret  learning 

• Science  key,  vs.  emotion  or  personal  beliefs 

• Asks,  “whose  account  counts?” 

Table  4:  Preliminary  Results 
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Abstract:  This  paper  shows  how  reasoning  upon  questions  and  hypotheses  built  collaboratively 
can  lead  to  knowledge-building  and  conceptual  change  in  hypermedia  conferencing  systems.  It  is  a 
qualitative  study  of  the  conditional  reasoning  process  carried  on  in  a biology  mixed-mode 
undergraduate  course  on  mammals  delivered  in  the  fall  semester  of  1998.  The  professor  applied 
Socratic  maieutic  by  posing  online  questions  to  the  students,  and  then  reasoning  upon  the  answers 
and  raising  new  questions  in  face-to-face  encounters.  We  applied  the  meaning  implication  analysis 
to  study  the  role  of  production  systems  in  the  conditional  reasoning  activity,  taken  in  their  logical 
(procedural)  and  referential  (declarative)  dimensions  altogether.  Results  show  qualitative 
circumscribed  evidence  of  high  order  reasoning  and  conceptual  change,  and  indication  that  high 
order  learning  had  been  achieved  through  collaborative  knowledge  building.  The  study  discusses 
implications  for  course  and  conferencing  systems  design,  and  gives  directions  for  new  forms  of 
representation  in  asynchronous  conferencing  systems. 


Introduction 

This  paper  shows  how  reasoning  upon  questions  and  hypotheses  built  collaboratively  can  lead  to 
knowledge  building  and  conceptual  change  in  hypermedia  conferencing  systems.  We  studied  one  sub-conference  of 
a biology  mixed-mode  undergraduate  course  on  mammals  delivered  in  the  fall  semester  of  1998  in  a French 
speaking  Canadian  university.  The  professor  applied  Socratic  maieutic  by  posing  online  questions  to  the  students, 
and  then  reasoning  upon  the  answers  and  raising  new  questions  in  face-to-face  encounters. 

Results  show  the  knowledge  building  process  that  led  some  of  the  participating  students  to  achieve  high 
order  reasoning  and  conceptual  change  in  the  part  of  the  conference  chosen  for  study.  We  understand  high  order 
reasoning  as  the  ability  to  logically  operate  the  modules  of  content,  and  conceptual  change  as  the  ability  to  rebuild  a 
conceptual  (logical)  based  on  the  assumption  that  the  previous  construction  had  premises  that  were  inconsistent  with 
the  conclusion  of  the  argument.  Conceptual  change  suggests  that  high  order  learning  was  achieved,  i.  e.,  that  the 
content  was  retrieved  and  stored  in  the  memory.  The  circumscribed  collaborative  knowledge  building  found  in  the 
students’  learning  processes,  reified  by  the  transcripts  as  objects  of  knowledge  (Popper,  1994;  Bereiter,  1994; 
Scardamalia,  & Bereiter,  1996),  was  assessed  by  transcript  analysis.  The  meaning  implication  transcript  analysis 
technique  was  applied  to  identify  the  chains  of  interconnected  meaning  implications,  and  how  collaborative 
conditional  reasoning  and  hypotheses  formulation  evolved  in  the  hypermedia  conversation  (Campos,  1998). 


Theory  and  Method 

Learning  is  the  process  of  acquiring  expertise  (Bereiter,  & Scardamalia,  1993;  Bruer,  1994)  by  storing 
mental  objects  as  memory  traces  (Changeux,  1997).  Assessing  expertise  acquisition  in  conceptual  specific  domains 
is  usually  done  by  applying  the  traditional  comparison  of  the  way  novices  and  experts  solve  problems,  like  in 
physics  (Larkin,  1981;  Larkin,  McDermott,  Simon,  & Simon,  1980),  and  how  the  schemas  of  the  procedures  are 
accommodated  (Piaget,  1976a).  Methods  of  assessing  how  novices  and  experts  solve  well-defined  problems  in 
conceptual  domains,  in  which  axiomatization  allows  necessity  and  sufficiency  (Grize,  1990),  cannot  be  simply 
transposed  to  ill-defined  domains,  in  which  the  degree  of  the  subjective  referencing  of  natural  language  impeaches 
the  occurrence  of  logical  necessity  and  sufficiency  (Grize,  1990).  Indeed,  the  natural,  social  and  human  sciences,  in 
different  degrees,  all  suffer  the  lack  of  relative  and  provisional  certainty  provided  by  the  hard  sciences. 
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To  allow  assessing  reasoning  in  ill-defined  domains,  in  which  classic  problem  solving  is  not  possible 
due  to  the  constraints  of  the  referential  function  of  natural  language,  we  developed  a method  in  which  both  the 
logical  and  referential  dimensions  were  taken  into  account  (Campos,  1998).  This  method  looks  at  the  way  we  reason 
through  natural  language  and  the  conditional  operations  subjacent  to  the  inferences  we  make.  The  inferential 
capacity,  expressed  by  conscious  or  unconscious  conditional  reasoning,  is  at  the  core  of  the  cognitive  process  and 
can  be  considered  its  gist  (Piaget,  1976c). 

To  apply  our  method  to  hypermedia  learning  environments  we  developed  a technique  of  analyzing 
transcripts  of  online  conversation.  The  analysis  allows  the  study  of  production  systems  in  the  conditional  reasoning 
activity,  taken  in  their  logical  (procedural)  and  referential  (declarative)  dimensions  altogether.  This  technique  allows 
the  identification  of  implications  among  meanings  in  online  discourse  (Campos,  1998). 

The  classical  definition  of  meaning  implication  is  that  it  is  the  source  of  necessary  relations  (Piaget, 
1976b,  1977),  and  that  if  part  of  a meaning  C is  embedded  in  B and  a part  of  a meaning  B is  embedded  in  A,  then  A 
implies  C in  terms  of  meaning  (Piaget,  1991):  a backtracking  process,  though.  This  definition  applies  strictly  to  the 
level  of  actions  and  the  abstract  logical  concepts  (mainly  procedural,  logical  propositions,  knowledge  construction) 
and  loosely  to  that  of  statements  (mainly  declarative,  referred  enunciates,  knowledge  building).  Statements  are 
referred  notions  whose  referencing  depends  on  the  subjective  social-cultural-psychological  symbolic  universe  of  the 
individual  (Campos,  1998;  Ramozzi-Chiarottino,  1998).  The  notion  of  meaning  implication,  a late  development  in 
genetic  epistemology,  allows  the  inclusion  of  pragmatics  in  Piaget’s  syntactic-semantic  model  of  knowledge.  It 
enhances  the  classic  Piagetian  theory  to  the  domain  of  psychosocial  linguistic  processes  allowing  the  establishment 
of  consistent  links  with  Vygotsky’s  pragmatic  psychological  model  (Z.  Ramozzi-Chiarottino,  personal 
communication,  August  1999)  and  the  classic  model  of  cognitive  architecture. 

Applied  to  statements,  meaning  implications  are  notional  conditional  enunciates  in  which  subjacent  If- 
Then  operations  do  not  have  the  logical  consistency  of  propositional  argumentation.  They  are  encountered  in  ill- 
defined  knowledge  domains  in  which  the  validity  of  the  notions  is  given  by  the  act  of  signifying  through  natural 
language.  These  symbolic  operations  are  analogous  and  complementary  to  those  found  in  action  implication.  Action 
implications  are  If-Then  operations  encountered  in  physical  actions  independently  of  the  use  of  natural  language  to 
express  them  (Ex:  if  I run  then  I am  able  to  cross  the  field  before  that  mad  cow  reaches  me,  given  that  I really  run  to 
cross  the  field  before  that  mad  cow  reaches  me). 

Meaning  implications  are  as  varied  as  the  possibilities  of  natural  language.  Generally,  they  are 
tautological  because  from  the  psychological  point  of  view  of  the  individual  who  writes  or  talks,  the  argumentation  is 
always  correct.  Nonetheless,  in  a hypermedia  conversation,  the  participants  can  check  the  tautological  character 
given  by  individuals  through  refutation,  making  knowledge  building  possible.  When  the  content  coincides  with 
logical  rules,  inductive  validity  results  from  the  agreement  between  logic  and  reality  (Ex:  if  I step  on  a banana  it  is 
likely  that  I will  fall),  although  necessity  and  sufficiency  might  not  be  achieved.  When  the  content  does  not  coincide 
with  logical  rules,  inductive  validity  results  from  strict  psychological  subjectivity  (Ex:  if  all  mammals  develop  two 
breasts  as  human  beings  then  they  will  all  also  look  sexy).  There  is,  though,  a scale:  the  closer  the  content  comes  to 
the  agreement  between  logic  and  objective  reality,  the  more  plausible  is  the  reasoning  and  the  further  it  is  from 
subjective  Judgment.  This  scale  is  also  applicable  to  the  sciences:  from  the  closest  to  pure  form  (propositional  logic, 
mathematics)  to  the  closest  to  pure  content  (history,  the  social  sciences)  (Granger,  1994,  1992). 


The  Study 

Our  goal  was  to  show  circumscribed  qualitative  evidence  of  collaborative  learning  in  a hypermedia 
asynchronous  conversation  triggered  by  the  process  of  interrogation  and  reasoning  about  scientific  dilemmas  as  a 
teaching  method. 

The  following  instruments  were  used  in  the  research;  Transcript  analysis,  in-depth  interview  with  the 
professor  and  the  teaching  assistant,  and  the  analysis  of  the  comments  written  and  the  marks  given  to  the  messages 
published  by  the  students  in  the  online  learning  environment  (comments  and  marks  were  provided  by  the  professor). 

We  studied  a biology  mixed-mode  undergraduate  course  on  mammals  delivered  in  the  fall  semester  of  1998 
in  a French  speaking  Canadian  university.  Seventy-three  persons  took  part  of  the  course,  including  students,  the 
professor  and  the  teaching  assistant.  A root  conference  was  opened  from  which  eight  sub-conferences  were  created. 
The  root  conference  aimed  to  provide  all  needed  information,  including  hypermedia  navigation  manuals  to  help  the 
students,  and  also  hyperlinks  to  a number  of  mammalogy  web  sites.  Each  sub-conference  had  a different  topic: 
Introduction  ( in  which  general  notions  applied  to  the  field  were  discussed),  Characters,  Constructive  criticism  (in 
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which  students  could  discuss  the  use  of  hypermedia  conferencing  for  learning  mammalogy),  Origins  and  evolution, 
Reproduction,  Domestication,  Dentition,  and  Locomotion. 

The  online  teaching  strategy  applied  by  the  professor  was  that  of  triggering  discussion  through 
questions  related  to  unsolved  scientific  dilemmas  in  mammalogy.  Different  hypotheses  explaining  those  dilemmas 
formulated  by  biology  theorists  were  presented  to  the  students  by  the  professor  through  traditional  face-to-face 
lecturing.  The  students  were  then  encouraged  to  propose  alternative  solutions  for  the  dilemmas  in  the  hypermedia 
conferences  by  applying  the  knowledge  about  mammals  and  biology  they  had  already  acquired.  No  readings  or 
literature  research  were  required  for  the  course,  although  the  students  were  encouraged  to  do  so.  To  guide  their 
reasoning,  the  students  had  primarily  access  to  black  and  white  drawings  and  graphics  provided  by  the  professor,  in 
addition  to  the  notes  they  were  able  to  take  during  the  lectures. 

The  instructor  is  a full  professor  who  teaches  this  course  without  significant  changes  for  more  than  20 
years.  The  professor  was  not  knowledgeable  about  technology,  did  not  want  to  get  involved  in  learning  how  to  use 
computers,  and  did  not  participate  directly  in  the  hypermedia  part  of  the  course.  According  to  the  professor,  the 
hypermedia  component  was  chosen  as  a tool  that  would,  eventually,  enhance  the  learning  process  and  allow 
discussions  that  would  never  take  place  in  the  classroom  due  to  the  way  the  course  is  taught. 

A graduate  student,  expert  in  educational  technology,  worked  as  an  exclusive  online  teaching  assistant, 
responsible  for  posting  the  questions  and  providing  instructions  to  the  students  on  how  to  participate.  No  facilitator 
or  moderator  was  assigned  to  mediate  the  conferences.  The  students  were  left  free  to  take  charge  of  the  discussions. 
Participation  was  not  mandatory. 

After  each  of  the  sequential  hypermedia  conferences  was  over,  the  professor  marked  the  messages 
according  to  the  level  of  plausibility  of  the  hypotheses  formulated  and  the  quality  of  argumentation.  Together  with 
the  marks,  there  were  sometimes,  written  comments.  The  marks  did  not  count  for  evaluation  purposes.  Marks  given 
were: 

Zero  - no  meaningful  contribution, 

!4  - minor  contribution, 

1/2  - reasonable  contribution,  and 

1 - outstanding  contribution  (plausible  hypothesis  and  argumentation). 

Later,  the  comments  (marks  were  not  made  available  to  the  students)  and  the  messages  were  discussed  in  class.  The 
professor  commented  on  the  most  plausible  solutions  presented,  raised  further  questions,  and  discussed  the  state  of 
the  dilemmas. 

We  hypothesized  that  if  implications  among  meanings  were  found  in  arguments  present  in  different 
messages,  then  the  quality  of  the  arguments  marked  “1”  could  be  explained  by  the  conditional  contribution  given  by 
others.  In  addition,  it  would  indicate  that  high  order  learning  (attributed  by  the  professor  through  the  marks)  in  the 
knowledge  building  process  was  achieved  through  collaboration.  Furthermore,  such  predicted  result  would  provide 
qualitative  evidence  that  the  process  of  interrogation  and  reasoning  about  scientific  dilemmas  would  be  a consistent 
method  for  teaching  evolutionary  biology. 


Results 


After  reading  the  texts  of  all  conferences  and  identifying  meaning  implications  sustaining  the 
arguments,  we  have  chosen  the  conference  on  characters  of  mammals  because  it  was  the  most  interactive  both  in 
terms  of  implications  among  meanings  and  in  terms  of  a reasonable  (although  poor)  use  of  the  threading  feature  of 
the  conferencing  system  used  (Virtual-U).  This  feature  allows  users  to  sort  messages  by  threads,  as  shown  in  Figure 
1. 

Five  questions  were  posed  to  the  students  in  the  sub-conference  about  characters.  We  chose  the  chain 
of  thought  of  question  2 for  this  study  by  following  the  meaning  implications  built.  The  reason  for  the  choice  of 
question  2 was  that  two  messages  related  to  this  question  were  marked  “T’  by  the  professor  (outstanding 
contributions).  In  three  of  the  four  remaining  questions  only  one  single  message  was  marked  “1”  by  the  professor. 

To  study  the  meaning  implication  chain  of  question  2,  we  focused  on  backtracking  the  meaning 
implications  between  the  arguments  and  hypotheses  marked  “1”  in  order  to  understand  how  were  they  built  (identify 
knowledge  building),  and  to  examine  whether  they  had  been  gestated  in  other  arguments  written  by  other  students  in 
previous  messages  (identify  collaboration).  Messages  that  were  published  after  the  last  contribution  marked  “1”,  all 
isolated  from  threads,  were  ignored.  We  studied  six  messages  in  which  we  identified  1 2 meaning  implications.  Four 
students  participated  in  this  part  of  the  discussion. 
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It  is  remarkable  that  only  one  of  the  six  messages  was  a reply,  although  all  meaning  implications  found 
were  linked  to  each  other,  be  in  a message  or  among  different  messages.  Meaning  linking  was  not  related  to  the 
built-in  threads. 
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Figure  1 - In  the  studied  hypermedia  conference,  knowledge  was  built  through  collaboration  but  meaning 
linking  was  not  necessarily  related  to  the  built-in  threads. 

In  addition,  qualitative  analysis  shows  that  in  both  messages  marked  “1”,  hypotheses  were  built  upon 
non-plausible  hypotheses  formulated  in  previous  messages.  Two  different  students  wrote  these  messages.  The 
plausibility  of  the  formulated  hypotheses,  assessed  by  the  professor’s  expert  marking,  indicates  that  high  order 
learning  was  achieved.  In  addition,  one  of  the  students  whose  message  was  marked  1 passed  through  a process  of 
conceptual  change  when  he/she  realized  that  his/her  previous  analysis  was  a misinterpretation  of  the  evidence 
(highlighted  by  the  professor  in  a comment)  and  decided  to  re-build  his/her  own  argument.  This  student,  thus, 
accommodated  his/her  own  learning  and  changed  the  previous  conceptualization  of  the  problem,  adjusting  it  to  the 
evidence.  In  addition,  the  analysis  indicates  that  all  participating  students  built  knowledge  through  collaboration.  In 
other  words,  built  their  arguments  by  discussing  the  premises  formulated  by  the  colleagues. 


Implications 

For  course  design 

The  study  suggests  that  the  teaching  strategy  might  be  strongly  related  to  the  results.  The  Socratic  maieutic 
was  applied  by  posing  questions  to  the  students,  reflecting  upon  the  answers  provided,  building  and  communicating 
arguments  both  in  the  hypermedia  environment  and  in  face-to-face  encounters.  Questions  were  challenging,  and 
they  are  still  to  be  answered  by  evolutionary  biology.  This  aspect  seems  to  be  specifically  attractive  for  the  students 
because,  as  novices,  they  were  given  the  opportunity  to  exercise  their  own  sense  of  how  to  apply  background 
knowledge  to  scientific  dilemmas,  given  the  chance  to  solve  ill-defined  problems  that  scientists  had  still  not  done, 
and  had  the  opportunity  to  provide  meaningful  contribution  to  the  discipline.  The  importance  of  questioning  in 
educational  processes  is  common  sense.  In  Socrates,  it  is  a teaching  method  (Mondolfo,  1972)  through  which 
problems  about  the  empirical  world  could  be  responded  inductively  (Jaeger,  1987).  Questioning  seems  to  be  an 
appropriate  teaching  method  for  exploring  scientific  dilemmas  in  online  written  conferencing.  The  depth  of 
argumentation  triggered  by  questioning  would  be  difficult  to  achieve  in  a face-to-face  conversation  because  writing 
demands  deeper  reflection  and  allows  thought  refining. 
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For  conferencing  systems  design 


Data  indicate  that  the  agreement  of  threads  and  meanings  in  hypermedia  conversation  does  not  emerge 
by  itself.  The  analysis  of  the  way  the  students  linked  their  chain  of  thoughts  shows  that  the  threading  feature  is 
neither  necessary  nor  sufficient  for  knowledge  building  through  collaboration.  Most  messages  were  not  threaded 
although  they  were  linked  by  meaning.  This  finding  is  consistent  with  that  of  a previous  study  and  is  a strong 
indicator  that  hypermedia  conferencing  systems  still  need  further  development  and  increased  flexibility.  Their 
underlying  formal  structure  should  adapt  to  the  way  we  use  natural  language,  mold  our  symbol  systems  to  assimilate 
what  the  others  have  to  say,  and  accommodate  knowledge.  Advance  of  conferencing  systems  depends  on 
representations  closer  to  the  functioning  of  our  neural  systems.  A reply-to-many  feature  would  be  a first  step  to 
allow  more  interaction  (Campos,  1 998).  A pop  up  help  tool  to  guide  conference  participants  in  the  use  of  threads 
would  be  another  important  feature  to  be  added. 


Conclusion 

We  discussed  in  this  paper  how  we  assessed  qualitative  circumscribed  evidence  of  high  order 
reasoning  and  conceptual  change,  achieved  through  collaborative  knowledge  building.  The  meaning  implication 
transcript  analysis  allowed  us  to  assess  this  circumscribed  evidence  in  the  studied  online  written  conversation.  The 
analysis  shows  how  conditional  reasoning  governed  the  inferences  students  made  upon  the  contribution  of  their 
colleagues.  In  addition,  it  presents  hypothesis  formulation  and  argument  building  as  a likely  result  of  memory 
retrieval  of  both  information  presented  in  face-to-face  lecturing  and  the  study  of  drawings  provided  by  the  professor 
(Changeux,  1997),  of  acquisition  of  scientific  reasoning  expertise  (Bruer,  1994;  Bereiter,  & Scardamalia,  1993)  and 
of  in-depth  reflection  upon  contributions  provided  by  others  (Scardamalia,  & Bereiter,  1994). 

The  analysis  of  the  way  the  students  built  their  chain  of  thoughts  shows  that  the  threading  feature,  as  it 
is  today,  is  neither  necessary  nor  sufficient  for  meaning  linking.  This  finding  is  consistent  with  a previous  study 
(Campos,  1998)  and  is  a strong  indicator  that  hypermedia  conferencing  systems  still  need  further  development  to 
make  their  underlying  formal  structure  more  flexible  in  order  to  be  adapted  to  the  way  we  use  natural  language,  to 
the  way  we  converse  and  mold  our  symbol  systems  to  assimilate  what  the  others  have  to  say,  and  to  accommodate 
this  knowledge  into  our  mental  semiotic  configurations.  Furthermore,  the  study  suggests  that  the  Socratic  maieutic, 
as  a teaching  strategy,  might  be  strongly  related  to  the  results. 

These  reasons,  altogether,  might  explain  why  such  a level  of  collaborative  knowledge  building  was 
achieved  in  a situation  in  which  the  students  were  left  free  to  take  charge  of  their  own  learning  processes  without 
any  of  the  award  systems  found  in  pedagogical  behaviorism.  The  fact  that  the  professor  did  not  participate  at  all  is 
additional  indication  of  the  inner  learning  possibilities  of  hypermedia  conferencing.  Another  fact,  not  unimportant,  is 
that  the  studied  course  was  mixed-mode.  The  use  of  this  teaching  hypermedia  modality  suggests  that  mixing  face-to- 
face  encounters  with  the  use  of  the  conferencing  component  provided  specific  opportunities  that  enriched 
experiences  leading  to  social  interaction  and  learning  (Laferriere,  Breuleux,  & Campos,  1 999).  The  teaching  and  the 
learning  processes  of  this  mixed-mode  course  seem  to  show  that  when  proper  teaching  strategies  are  present, 
knowledge  building  and  collaboration  are  achieved  (Campos,  & Harasim,  1999). 
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Abstract:  In  1998  The  University  of  Iowa  Libraries  was  the  recipient  of  a grant  from 
the  Library  of  Congress/Ameritech  National  Digital  Library  Competition  in  support  of 
a project  to  digitize  a portion  of  its  unique  and  extensive  Redpath  Chautauqua 
Collection.  Although  the  staff  of  the  Information  Arcade  at  the  University  of  Iowa 
Libraries  has  extensive  experience  in  instructional  technology,  this  was  the  first 
opportunity  for  them  to  engage  in  the  digitization  of  a large  text  collection  of  rare 
materials.  This  article  describes  the  processes  undertaken  in  the  project  as  well  as  the 
challenges  when  the  theory  of  how  to  approach  a digitization  project  of  significant 
scope  clashed  with  the  realities  faced  by  the  project  team. 


Introduction 

In  1998  The  University  of  Iowa  Libraries  was  the  recipient  of  a grant  from  the  Library  of 
Congress/Ameritech  National  Digital  Library  Competition  in  support  of  a project  to  digitize  a portion  of  its 
unique  Redpath  Chautauqua  Collection.  The  competition,  which  was  funded  for  three  years  by  a $2-million 
partnership  between  the  Library  of  Congress  and  the  Ameritech  Corporation,  has  made  awards  to  a total  of  29 
institutions  with  the  aim  of  supporting  the  digitization  of  a range  of  materials  of  significance  to  the  social  and 
cultural  history  of  the  United  States. 

One  of  the  goals  of  the  competition  was  to  encourage  U.S.  libraries,  archives,  museums,  and  historical 
societies  to  assist  in  the  development  of  the  American  Memory  electronic  collections  of  the  Library  of  Congress, 
But  in  making  the  awards,  the  LC/Ameritech  program  also  offered  many  institutions  that  would  not  otherwise 
have  had  sufficient  institutional  resources  to  learn  by  "getting  their  hands  dirty"  about  the  sometimes 
confounding  issues  involved  in  planning  and  carrying  out  a complex  digitization  project.  Although  the  staff  of 
the  Information  Arcade  at  The  University  of  Iowa  Libraries  has  extensive  experience  in  instructional  technology, 
this  was  the  first  opportunity  for  them  to  engage  in  the  digitization  of  a large  text  collection.  This  article  will 
describe  the  processes  undertaken  in  the  project  as  well  as  the  challenges  faced  by  the  project  team  when  the 
theory  of  how  to  approach  a digitization  project  of  significant  scope  clashed  with  the  realities  faced  by  the 
project  team. 


The  Redpath  Chautauqua  Collection 

The  University  Libraries  collection  chosen  for  the  award  is  a series  of  talent  flyers  in  its  distinguished 
Redpath  Chautauqua  Collection  and  the  project  was  named:  "Traveling  Culture:  Circuit  Chautauqua  in  the 
Twentieth  Century."  The  circuit  Chautauqua  Movement  represents  an  early  embodiment  of  the  American  drive 
for  self-realization  and  self-improvement.  Its  importance  as  a manifestation  of  the  "American  spirit"  and  as  a 
dominant  influence  on  American  culture  has  been  well  documented  in  both  scholarly  and  general  works.  By  the 
mid- 1920s  there  were  twenty-one  circuits  of  traveling  performers  providing  seven-day  programs  in  more  than 
10,000  rural  communities  in  45  states  to  an  estimated  40  million  people.  Programs  included  discussions  on 
political,  literary,  scientific,  and  moral  topics,  music  (from  accordionists  to  yodelers),  and  entertainment  for  all 
ages  of  both  high  and  popular  culture. 
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Comprising  some  648  linear  feet  of  materials  dating  between  1 890  and  1 940,  the  Redpath  collection  is 
considered  to  be  the  most  extensive  holding  of  circuit  Chautauqua  materials  in  existence.  Its  particular  strength 
lies  in  the  breadth  of  coverage  and  length  of  time  over  which  the  Redpath  Bureau  conducted  business  as  the 
premier  booking  agency  in  the  U.S.A. 

The  talent  portion  is  the  largest  series  in  the  collection  consisting  of  nearly  8,000  flyers  promoting  the 
talents  of  nearly  4,600  performers.  At  the  beginning  of  the  project  the  flyers  were  housed  in  folders  arranged 
alphabetically  by  performer’s  name  in  279  manuscript  containers  in  which  flyers,  correspondence,  and  contracts 
were  intermingled.  The  series  comprised  139  linear  feet  of  primary  research  materials  especially  rich  in 
contemporary  information  by  and  about  notable  people  of  the  period.  Selection  of  this  collection  for  digitization 
was  not  the  difficult  part  of  this  project.  The  talent  flyers  were  a natural  resource  to  select  for  presentation  to  the 
widest  possible  audience  via  the  web. 

The  spirited  language  and  vivid  art  work  used  in  the  promotional  flyers  reflect  the  emotions  and  ideals 
of  the  Chautauqua  movement.  Presenting  the  flyers  in  digital  form  makes  the  possibly  dry  topic  of  U.S.  history 
come  alive  to  the  casual  and  serious  researcher  alike.  For  instance,  young  students  are  able  to  discover  that  the 
public's  fascination  with  pet  tricks  did  not  begin  with  David  Letterman.  Rather,  they  would  be  able  to  see  that 
Pamahasika's  50  Highly  Educated  Pets  were  an  amazing  and  popular  source  of  entertainment  in  the  early  part  of 
the  century.  Secondary  school  students  could  learn  that  outrageous  expenditure  of  federal  funds  is  not  a new 
concern.  Judson  King's  lecture  on  how  a "certain  state  government  recently  paid  $10  each  wholesale  for  some 
jack-knives  which  were  worth  $2.50  each  retail"  would  point  out  that  government  reformers  have  long  had  their 
work  cut  out  for  them.  The  general  public  of  today  would  be  just  as  curious  as  their  great-grandparents  about 
tales  of  hardship  and  misfortune,  such  as  the  experiences  of  Mrs.  Florence  E.  Maybrick.  Unjustly  convicted  of 
poisoning  her  husband,  she  was  sentenced  by  a judge  who  went  insane  shortly  thereafter.  Yet  she  served  14  1/2 
years  of  her  punishment  and  then  went  on  the  circuit  to  lecture  on  prison  life  and  the  need  for  judicial  and  prison 
reform. 


Initial  Plan  for  Workflow 

Both  the  text  and  the  graphic  layout  of  the  flyers  are  highly  valuable  for  the  study  of  American  popular 
culture  so  the  digitization  strategy  had  to  include  adequate  access  to  both  the  textual  information  on  the  flyers 
and  the  graphic  representation  of  the  flyers.  As  with  most  advertising  material,  there  was  no  standard 
presentation  or  design  to  any  of  the  flyers,  although  it  was  very  rare  that  a flyer  would  be  more  than  four  pages  in 
length.  Moreover  the  physical  state  of  the  flyers  and  correspondence  was  such  that  a preservation  effort 
simultaneous  to  the  digitization  effort  was  essential.  Therefore  the  project  had  four  distinct  workflows  that  had 
to  be  managed  separately  within  a timeframe  of  1 8 months  (a  stipulation  of  the  award)  with  an  eye  toward  re- 
assembling all  the  pieces  in  both  physical  and  digital  form  in  the  final  phase.  The  workflows  include: 
preservation,  keying/encoding  the  textual  information  on  the  flyers  for  full  text  searching,  providing  intellectual 
access  to  the  individual  flyers,  and  imaging  of  the  flyers. 

Because  the  flyers  are  rare,  no  consideration  was  given  to  outsourcing  the  imaging  of  the  flyers.  Iowa 
City  is  not  near  enough  to  a major  urban  area  to  allow  a vendor  to  come  on-site  to  perform  outsourced  tasks. 
Therefore  the  budget  and  workflow  plan  had  to  accommodate  student  staff  for  the  imaging  workflow.  The 
planning  team  estimated  that  full-time  staff  would  be  needed  to  train,  supervise,  and  provide  quality  control  for 
the  student  workers  doing  the  imaging.  There  would  be  no  possibility  of  including  in  the  grant  budget 
additional  full-time  staff  for  the  training,  supervision,  and  quality  control  for  students  who  might  be  doing 
OCR/re-keying  and  SGML  encoding  - especially  considering  the  timeframe  in  which  the  project  was  to  be 
completed.  Therefore  the  task  of  keying  and  "light"  SGML  encoding  (based  on  a TEl-Lite  DTD)  was 
outsourced  to  a vendor  through  a bid-process  managed  by  the  University  of  Iowa's  Purchasing  Department. 

Over  the  years  flyers  have  been  used  by  researchers  and  staff  who  were  not  uniformly  attentive  to 
replacing  materials  in  strict  order.  It  was  anticipated  that  some  reorganization  would  be  needed  to  ensure  that  all 
flyers  were  in  the  correct  folders.  The  number  of  flyers  in  the  collection  and  the  number  of  unique  performers 
was  estimated  by  a random  sampling  one  folder  in  each  of  the  256  boxes.  This  process  indicated  that  there  were 
9,600  flyers  representing  7,600  performers  and  those  figures  were  the  basis  for  estimating  the  number  of  page 
images  to  be  scanned  for  purposes  of  writing  the  grant  proposal.  However  early  in  the  project  it  was  discovered 
that  there  were  fewer  performers  (about  4,600)  represented  by  about  8,400  flyers.  Lesson:  Know  the 


er|c 


184 


dimensions  of  the  project  you  are  undertaking  and  then  add  in  sufficient  time  to  accommodate  the  unexpected  - 
especially  if  one  is  working  in  a special  collection  of  substantial  size  and  variety. 


Preservation 

Although  the  flyers  were  in  satisfactory  condition  to  withstand  the  rigors  of  digitization,  the 
accompanying  materials  had  badly  deteriorated.  Those  materials  are  letters  and  news  clippings  on  acidic  paper, 
or  even  onionskin,  or  carbon  copies  of  original  documents  in  other  files. 

To  ensure  that  the  folders  containing  the  flyers  and  correspondence  would  be  handled  only  once,  a plan 
was  developed  comprised  of  the  following  steps:  a folder  is  removed  from  its  container,  the  brochures  and 
correspondence  are  separated,  a preservation  photocopy  of  the  correspondence  is  made  on  archival  bond  paper, 
and  returned  to  the  collection.  These  preservation  activities  are  not  funded  as  part  of  the  LC/Ameritech  grant  so 
the  tracking  of  staff  time  and  the  costs  involved  in  handling  the  preservation  of  these  materials  has  to  be 
accounted  for  separately. 

A photocopy  surrogate  of  the  entire  flyer  is  made  for  use  by  the  vendor  in  keying/encoding  the  text  and 
a second  photocopy  of  the  flyer's  first  page  is  made  for  use  by  the  catalogers.  At  this  juncture  a unique 
identifier,  which  is  the  directory/file  structure  for  the  electronic  file,  is  written  on  both  copies  of  the  first  page. 
This  identifier  is  the  common  bond  that  links  the  cataloging  record  to  the  digitized  image.  After  the  flyer  is 
cataloged,  the  title  page  photocopy  and  the  original  flyer  in  a new  acid-free  folder  are  sent  to  the  in-house 
imaging  process.  After  imaging  is  completed,  talent  flyers  are  re-united  with  the  correspondence  related  to  that 
performer,  each  in  separate  acid-free  folders,  and  replaced  in  alphabetical  order  in  archival  boxes. 


Keying/Encoding 

As  mentioned  above,  the  initial  estimate  of  the  number  of  flyers  and  the  number  of  performers  involved 
was  lower  than  expected  based  on  initial  samples.  Unfortunately,  this  was  not  the  case  with  the  keying/SGML 
encoding  estimates. 

The  Library  of  Congress  encourages  SGML  encoding  of  textual  materials  in  the  American  Memory 
project  in  support  of  full-text  searching  and  to  serve  as  the  basis  for  more  sophisticated  search  and  discovery 
strategies  that  might  be  available  in  the  future.  Although  the  talent  flyers  are  highly  graphical  and  the  nature  of  a 
promotional  flyer  might  not  seem  to  warrant  full  text  searching,  the  quality  of  the  text  and  the  richness  it 
provides  as  context  to  the  images  led  the  project  team  to  concur  with  a plan  for  encoded  full  text.  However  the 
team  also  decided  that  deep  encoding  was  not  necessary  and  only  nineteen  elements  were  identified  for  the 
vendor's  tagging  work.  The  elements  selected  were  primarily  structural,  like  <figure>  and  <cell>.  Other 
elements  focused  on  the  types  of  text  that  might  be  likely  to  provide  special  insight  into  contemporary  culture 
and  style,  like  citations,  quotes  from  verse,  and  quotations.  It  was  reasoned  that  if  use  of  the  collection 
warranted  more  encoding  it  could  be  inserted  at  a later  date  when  more  money  was  available  for  such  activities. 

Initial  estimates  of  a character  count  of  28  million  characters  for  purposes  of  the  grant  proposal  were 
based  on  counting  the  number  of  characters  (including  spaces)  in  a relatively  small  sample  of  flyers  during  the 
grant-writing  process.  Unfortunately  the  sample  proved  to  be  skewed  and  the  number  of  characters  required  for 
keying  and  encoding  was  underestimated  by  about  100%,  even  considering  the  fact  that  there  were  fewer  flyers 
than  was  first  assumed.  Part  of  the  under-estimation  was  due  to  inexperience  in  estimating  the  number  of 
keystrokes  that  would  be  involved  in  the  insertion  of  the  SGML  tagging.  This  miscalculation  was  compounded 
with  the  discovery  that  there  were  quite  a few  flyers  that  had  unexpectedly  substantial  textual  material.  Taken 
together  the  team  severely  underestimated  of  the  number  of  characters  to  be  keyed  which  led  in  turn  to  an 
underestimation  in  the  budget  required  to  support  this  dimension  of  the  project.  Lesson:  Take  the  time  to  do  a 
substantial  sample  of  the  textual  materials  to  ensure  that  one  has  a solid  sense  of  the  amount  of  text  to  be  keyed  - 
and  then  add  at  least  25%  for  tagging  character  strokes. 

Fortunately,  the  commitment  of  the  University  Libraries  administration  to  this  project  is  strong  and  the 
commitment  of  the  vendor  to  the  project  was  equally  deep.  Together  a negotiation  of  the  fee  to  finish  the  keying 
was  achieved  which  resulted  in  both  sides  accommodating  some  of  the  extra  cost.  Lesson:  Find  a vendor  who 
considers  the  venture  to  be  a partnership  and  with  whom  you  can  communicate  frequently.  Also,  be  very 
diligent  about  reviewing  initial  encoding  samples  for  quality 
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One  other  unanticipated  wrinkle  in  the  workflow  occurred  during  the  keying  process.  The  quality  of  the 
photocopies  to  be  used  by  the  vendor  in  the  keying  and  encoding  process  was  often  difficult  to  decipher.  The 
flyers  make  great  use  of  shading  for  background  effect  and  the  text  is  artistically  displayed  in  a variety  of  fonts 
around  and  across  multiple  images.  (This  was  one  reason  that  OCR  was  not  considered  for  the  keying  process  - 
another  reason  is  the  poor  quality  of  affordable  OCR  software  which  requires  a great  deal  of  proofreading  and 
correction.)  Photocopies  of  text  printed  in  black  font  on  a gray  or  colored  background  were  in  many  instances 
illegible  to  the  keying  staff.  It  is  especially  important  to  have  absolutely  legible  photocopies  for  keying  staff 
who  may  not  be  proficient  enough  in  the  language  of  the  text  to  decipher  poor  reproductions.  Lesson:  The 
quality  control  procedures  employed  to  ensure  adequate  photocopying  were  sufficient  for  the  preservation 
photocopying,  but  we  did  not  employ  the  same  procedures  for  photocopies  to  be  used  by  the  vendor.  Now  the 
team  needs  to  devise  a method  for  filling  in  the  <gap>  and  <illegible>  tags  from  the  source  originals. 


Intellectual  Access 

Intellectual  access  to  "Traveling  Culture"  is  provided  through  a number  of  avenues.  The  finding  aid  for 
the  collection,  which  has  been  encoded  in  both  HTML  and  SGML  according  to  the  EAD  DTD,  provides  a box 
list  of  performer  names.  Part  of  the  agreement  for  the  grant  was  the  development  of  a scholarly  historical  essay. 
Charlotte  Canning,  University  of  Texas  at  Austin  Department  of  Theatre  and  Dance,  a frequent  user  of  the 
collection,  has  agreed  to  provide  the  historical  essay  for  "Traveling  Culture.*' 

Each  flyer  is  individually  cataloged  according  to  USMARC  compatible  standards.  Strict  adherence  to 
USMARC  standards  was  not  feasible  in  determining  the  title  of  many  of  the  brochures.  In  those  instances,  the 
decision  was  arbitrarily  based  on  such  factors  as  font  size,  placement  on  the  page,  or  other  dominant 
characteristics.  Bibliographic  data  are  included  in  the  TEI  Header  and  will  also  reside  in  a Libraiy  of  Congress 
USMARC  database.  The  homepage  for  "Traveling  Culture"  will  provide  both  full  text  searching  and  access  to 
flyers  through  a browse  list  of  subject  terms  based  on  Library  of  Congress  Subject  Headings,  Thesaurus  for 
Graphic  Materials  I : Topical  Terms  for  Subject  Access  (LCTGM  /),  and  Thesaurus  for  Graphic  Materials  II: 
Genre  and  Physical  Terms  (LCTGM  2),  Subject  analysis  has  been  applied  for  the  context  of  the  piece  as  well  as 
to  its  content.  Uncontrolled  subject  terms  have  also  been  applied  when  appropriate.  For  example,  we  have  yet 
to  find  a controlled  vocabulary  term  for  bird  imitators  or  cymbolomists. 

The  Redpath  Collection  has  been  cataloged  at  the  collection  level  for  quite  a while,  but  cataloging  at  the 
flyer  level  was  a requirement  of  the  grant.  Workstation  based  packages  for  simple  MARC  cataloging  did  not  suit 
our  needs.  We  wanted  to  catalog  the  item  once  and  then  automatically  export  a MARC  file  for  both  the  Libraiy 
of  Congress  and  our  local  online  catalog,  as  well  as  a TEI  header  with  the  bibliographic  data  inserted  in  the 
correct  elements.  Finally  our  project  team  developed  a FileMaker  Pro  database  that  could  manage  both  the 
MARC  and  TEI  header  export.  The  FileMaker  Pro  database  has  been  easy  for  our  student  catalogers  to  use  and 
flexible  to  adapt.  We  have  added  special  fields  that  contain  the  file  names  (see  below)  and  the  identifying 
numbers  for  the  archival  CDs  that  contain  the  images  for  that  flyer  so  the  database  record  contains  all  the 
information  necessary  to  provide  full  access  to  any  given  flyer.  Moreover  it  now  provides  a fully  functional 
database  in  support  of  interim  subject  access  to  the  "Traveling  Culture"  collection  until  such  time  as  full-text 
SGML  searching  is  available. 

After  cataloging  each  flyer  staff  have  an  additional  task  that  provides  the  lynch-pin  for  the  entire 
process— assigning  the  file  naming  structure  to  be  used  as  the  unique  identifier  for  linking  together  the  TEI 
headers  with  the  text  files  and  the  page  images  of  the  flyers.  The  basic  directories  are  based  first  upon  file  type, 
i.e.  "sgm"  for  the  text,  "gif’  for  the  thumbnail  image,  "Jpg"  for  the  JPEG  images,  etc.  The  second  level  of  the 
directory  indicates  the  series,  performer,  sequential  number  of  unique  flyer  in  the  collection  by  that  performer, 
and  the  individual  page  image.  An  example:  sdrc/traveling-culture/chaul/img/stagg/3/5.gif  indicates  the 
Traveling  Culture  collection,  talent  flyer  series,  the  performer  named  Alonso  Stagg,  the  third  unique  talent  flyer 
in  the  box,  and  the  gif  image  of  page  5 of  that  flyer. 

Requiring  that  the  file  naming  structure  reflect  the  flyers  of  a particular  performer  has  both  advantages 
and  drawbacks.  One  advantage  is  that  it  reflects  the  traditional  structure  of  the  collection,  which  has  always 
been  based  upon  performer  name.  The  naming  procedure  makes  sense  to  human  beings  which  assists  in  the 
construction  of  the  unique  identifier  for  the  files  and  may  assist  humans  in  quality  control  when  linking  the  page 
image,  text  and  header  files  together.  However,  as  more  flyers  by  the  same  performer  are  found  or  as  more 
flyers  by  performers  with  similar  name  stems  are  discovered,  the  system  begins  to  be  subject  to  transcription 
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errors.  No  change  in  the  file  structure  naming  process  is  planned,  but  these  considerations  will  inform  our 
choices  in  the  next  project.  Lesson:  Plan  for  as  much  extensibility  in  file  structures  as  possible,  even  when 
dealing  with  collections  that  are  no  longer  growing  in  size.  It  may  be  the  case  that  machine-assigned/random 
components  to  file  names  provide  the  most  flexibility  in  the  long  run  for  large  text  collections  that  are  intended 
to  grow  and  expand  in  coverage. 


Imaging 

The  imaging  for  the  project  has  been  accomplished  in-house  by  student  staff  with  equipment  provided 
by  the  Office  of  the  Vice-President  for  Research.  Special  attention  to  simplifying  and  automating  procedures  as 
much  as  possible  has  been  a goal  of  the  project  team  to  minimize  training  time  and  error  rate. 

The  general  procedure  is  as  follows:  the  original  flyers  arrive  from  the  catalogers  with  the  photocopied 
title  page  indicating  the  unique  identifier/file  name  for  the  flyer.  Students  scan  the  flyer  pages  creating  600dpi 
32-bit  color  TIFF  images  (a  much  slower  process  than  had  been  originally  estimated)  that  are  saved  to  Jaz  disks. 
The  Jaz  disks  are  used  to  bum  archival  CDs  so  no  more  than  650  megabytes  of  data  are  saved  to  each  disk. 
Each  CD  is  labeled  sequentially  with  a note  indicating  which  performer’s  flyers/pages  are  included.  The  color 
TIFF  files  are  so  large  that  each  CD  averages  only  6 images. 

An  automated  program  creates  a 300dpi  32-bit  color  file  on  the  workstation  desktop  from  the  600dpi 
TIFF  image  after  it  is  saved  to  the  Jaz  disk.  The  300dpi  file  is  again  downsampled  into  one  300dpi  1-bit 
black/white  PDF  file,  one  150dpi  32-bit  color  JPEG,  one  72dpi  8-bit  color  GIF  image  and  one  6dpi  8-bit  color 
GIF  image.  All  images  for  a flyer  are  saved  in  one  folder  on  the  server.  The  color  300dpi  file  is  then  discarded. 
We  originally  tried  to  use  Debabelizer  for  the  downsampling,  but  that  software  could  not  rename  the  files  as  they 
were  downsampled  according  to  our  file  naming  scheme.  So  we  use  a semi-automated  procedure  based  on 
Photoscripter.  . There  is  an  as-yet-unidentified  "waffle-effect"  that  sometimes  appears  with  GIF  images,  but  not 
in  the  JPEGs.  Quality  control  on  the  imaging  will  be  the  primary  focus  of  post-production  image  processing. 

The  results  of  the  review  of  image  quality  will  inform  our  production  workflow  for  the  next  project. 

GIF  images  at  6dpi  are  used  as  thumbnail  navigational  aids  in  the  search  interface;  GIFs  at  72dpi  are  the 
default  display  images.  The  JPEG  image  is  an  option  that  may  be  selected  by  the  user  who  wishes  to  view  or 
download  a higher  quality  image  than  the  GIF  image  provides.  PDF  images  are  provided  for  ease  of  printing. 

Many  talent  flyers  contain  images  that  span  two  pages,  or  even  three.  The  project  team  made  a strategic 
decision  that  images  that  span  pages  should  be  presented  to  the  public  as  coherent  whole  images.  This  requires 
that  the  images  be  "stitched  together"  after  they  are  downsampled  - a job  that  requires  patience,  time,  and  a keen 
artistic  eye.  Sometimes  the  colors  of  each  scan  are  slightly  different  for  the  respective  image  halves  due  to 
unpredictable  (and  undefinable)  equipment  problems  that  require  extensive  staff  time  to  correct.  This  is  a task 
for  the  team  member  who  supervises  the  students  and  the  scanning  process.  Lesson:  Consider  carefully  whether 
and  how  much  one  can  create  automated  routines  to  prevent  the  need  for  extensive  training  and  lots  of  "hand- 
work" on  the  part  of  project  staff  The  budget  impact  of  the  need  for  special  handling  can  be  substantial. 


Putting  It  All  Together 

The  strategy  for  presenting  the  collection  to  the  public  was  the  subject  of  much  discussion  during  the 
grant  writing  phase.  Should  "live"  SGML  be  presented  to  the  public?  That  would  involve  customized  scripting 
to  convert  SGML  to  HTML  for  display  purposes  which  is  entirely  possible,  but  requires  staff  expertise  that  The 
University  Libraries  does  not  normally  have  on  hand.  Monies  available  through  the  grant  were  quite  limited  and 
although  the  Libraries  administration  was  prepared  for  a substantial  cost-share  there  was  a limit  to  what  we 
thought  we  should  attempt.  After  some  debate  it  was  decided  that  the  highly  visual  nature  of  the  flyers  would  be 
equally  well  served  by  presenting  only  page  images  of  the  flyers  and  using  the  encoded  text  as  the  basis  for  full- 
text  searching  behind  the  scene,  but  not  for  display  purposes.  Lesson:  This  was  the  right  decision!  As  the 
project  progressed  there  were  substantial  other  issues  to  address.  The  time  required  to  test  and  perfect  the  SGML 
encoding  and  to  write  the  scripts  to  translate  text  on  the  fly  to  HTML  would  have  been  over-burdensome  and  too 
costly. 
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The  prototype  website  for  the  collection  is  now  available  and  it  includes  a facility  for  basic  searching  on 
bibliographic  data  in  the  FileMaker  Pro  database.  The  next  technical  step  is  to  begin  to  export  the  TEI  header, 
pre-pend  it  to  the  text  files  that  have  arrived  from  the  vendor,  begin  the  process  of  linking  text  to  page  images, 
and  develop  full-text  searching  with  LiveLink  software. 

The  next  policy  step  was  to  develop  a copyright  statement  for  the  site.  The  Head  of  Special  Collections 
drafted  a statement  indicating  support  of  unfettered  use  by  researchers  of  all  ages,  but  requiring  that  permission 
be  sought  by  commercial  entities  that  might  want  to  use  the  images.  A fee  schedule  for  high-quality 
reproductions  was  developed,  with  the  copyright/use  statement,  had  to  be  reviewed  by  the  university’s  lawyers 
and  administration.  The  statement  as  well  as  an  updated  method  for  ordering  high  quality  reproductions  is  now 
on  the  website. 

General  Lessons  Learned:  Although  hiring  and  training  student  staff  is  time  consuming,  the  quality  of 
the  work  of  students  and  their  facility  with  the  technology  has  been  wonderful.  Equipment  and  scheduling 
’’hiccups"  have  occasionally  delayed  work,  but  the  scanners  and  the  server  have  been  robust.  We  could  use  more 
students,  more  equipment,  and  more  hours  in  the  day,  but  the  experience  gained  in  this  project  has  been 
invaluable  as  a staff  development  and  enrichment  activity.  Nothing  replaces  the  opportunity  to  "get  one’s  hands 
dirty." 

In  August  1999  the  project  staff  gave  a presentation  to  the  University  of  Iowa  Libraries  staff  about  the 
project.  Our  colleagues  were  impressed  with  the  complexity  and  scope  of  the  project  and  were  glad  to  learn 
more  about  exactly  what  is  involved  with  the  workflow.  But  what  impressed  us  was  their  equal  conviction  that 
the  Redpath  Chautauqua  Collection  is  indeed  a treasure  that  should  be  available  to  the  world.  We  share  that 
conviction  and  look  forward  to  offering  "Traveling  Culture"  to  people  around  the  world. 


URLs  of  Interest 

1 . Library  of  Congress/Ameritech  Competition  is  at:  http://memorv.loc.gov/ammem/award/index.html . 

2.  Library  of  Congress  American  Memory  is  at:  http://memorv.loc.gov/ammem/amhome.html . 

3.  The  Information  Arcade  is  at:  http://www.lib.uiowa.edu/arcade/index.htinl . 

4.  The  University  of  Iowa  of  Libraries  is  at:  http://www.lib.uiowa.edu/index.html . 

5.  The  Redpath  Chautauqua  Collection  Finding  Aid  is  at:  http://sdrc.lib.uiowa.edu/traveling-culture/findaid  1 .html . 

6.  Traveling  Culture  is  at:  http://sdrc.lib.uiowa.edu/traveling-culture/  . 

7.  The  University  Libraries'  policy  on  copyright  and  reproduction  of  images  is  at:  http://sdrc.lib.uiowa.edu/traveling- 
culture/copyright.html  and  http://sdrc.lib.uiowa.edu/traveling-culture/reproductions.html . 
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Abstract:  The  capacity  of  computer  technology  to  support  cognitive  and  learning  processes  is 
illustrated  in  the  Quality  in  Learning  and  Teaching  Project  (QuILT),  a case  and  problem-based 
multimedia  resource  which  presents  critical  aspects  of  classroom  practice  in  order  to  scaffold 
critical  inquiry,  observation  and  reflection.  QuILT  is  an  interactive  CD  ROM  program  that  provides 
teacher  education  students  with  a range  of  examples  of  authentic  scenarios  where  they  can  observe, 
analyse,  reflect  and  evaluate  actions,  pedagogies  and  decision-making  through  immersion  and 
engagement  in  virtual  classrooms.  QuILT  is  now  being  used  extensively  within  the  Faculty  of 
Education,  University  of  Melbourne,  Australia  as  part  of  the  teacher  education  program.  The 
software  provides  an  enabling  environment  in  which  student  teachers  can  develop  the  skills  of 
analysis,  prediction  inference  and  interpretation.  This  integration  of  technology  into  professional 
education  extends  the  range  of  experiences  available  to  teacher  educators  and  applies  multimedia 
technology  into  new  areas  of  learning  support.  This  paper  analyses  and  demonstrates  the  elements 
that  have  make  QuILT  a successful  technological  tool  in  the  professional  education  of  teachers.  The 
key  design  elements  are  identified  as  the  integration  of  authentic  classroom  scenarios  with 
opportunities  for  reflection  and  articulation  of  beliefs,  the  involvement  of  students  in  new  mediator 
roles  and  the  provision  of  a powerful  technology  supported  environment  to  create  contexts  for 
students’  professional  development. 


Aim  of  the  study 

The  aim  of  this  paper  is  to  demonstrate  the  capacity  of  interactive  multimedia  to  scaffold  teacher  practices,  inquiry 
and  decision  making  in  order  to  develop  the  skills  and  competencies  required  in  professional  practice.  Information 
technology  has  not  yet  achieved  a great  deal  of  recognition  in  the  professional  education  of  teachers  (Schrum  & 
Dehoney,  1998;  Davis,  1997,  Collis,  1994).  However,  there  are  increasingly  greater  constraints  on  education 
faculties  in  terms  of  resources  and  staff-student  ratios  with  the  consequent  problems  of  developing  teaching  and 
diagnostic  skills  that  are  transferable  into  actual  classroom  settings.  It  is  important  that  professional  development  for 
teacher  continues  to  be  embedded  in  contexts  and  practices  for  which  the  skills  are  intended.  Technology  support  for 
professional  development  through  the  interactive  multimedia  resource  of  QuILT  enables  novice  teachers  to  interact, 
reflect  and  learn  collaboratively  in  scalable  and  sustainable  ways  in  authentic  settings. 

The  design  features  of  QuILT  are  analysed  in  terms  of  a cognitive  apprenticeship  model  of  teaching,  constructivist 
learning  principles  and  scaffolded  learning  processes  (eg.  Levin  & Waugh,  1998;  Guzdial  & Keogh,  1998;  Hmelo  & 
Day,  1999).  On  the  basis  of  their  immersion  in,  and  interaction  with  a range  of  authentic  classroom  situations, 
student  teachers  are  supported  in  extending  and  modifying  their  understandings  of  teaching  processes,  decision- 
making and  interpretation  of  classroom  events. 
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Rationale  for  technology  supported  scaffolding  in  teacher  education 


The  incorporation  of  experiential  learning  tasks  to  cultivate  reflective  thinking  in  teacher  preparation  programs  at 
university  is  widely  acknowledged  in  the  literature  Stuhlman,  1998;  Collis,  1998).  Through  experience,  teachers 
learn  how  different  learning  outcomes  can  be  achieved  and  what  skills  are  required  in  particular  situations.  In 
addition  to  teaching  skills,  teachers  need  to  consider  how  different  school  settings,  events  and  resources  can  impact 
on  their  teaching.  Individual  teachers  working  in  isolation  in  schools  do  not  have  opportunities  to  share  experience 
or  to  examine  different  perspectives  on  teaching,  and  to  test  their  own  developing  skills  against  established  criteria. 
According  to  Shulman  (1987:  19),  teaching  requires  reflective  practice  that  develops  through  experience  and 
sharing  of  information  with  peers  and  mentors.  According  to  Shulman,  reflection  is  : 

..what  a teacher  does  when  she  or  he  looks  back  at  the  teaching  and  learning  that  has  occurred,  and  reconstructs, 
reenacts,  and/or  recaptures  the  events,  the  emotions,  and  the  accomplishments.  It  is  that  set  of  processes  through 
which  a person  learns  from  experience. 

For  the  development  of  critical  reflection,  student  teachers  need  to  undergo  a period  of  self-evaluation  and 
reflection  based  on  their  own  observations  of  teaching,  their  existing  belief  systems  and  their  experience.  This  stage 
in  the  development  of  professional  expertise  has  dominated  the  discussion  of  teacher  education  provision  in  recent 
years  (Bright,  1995).  Student  teachers  have  been  observed  to  undergo  a steep  learning  curve  at  the  beginning  of  their 
teaching  career,  when  they  abandon  strongly  held  beliefs  regarding  pedagogy  and  educational  values.  This  is  the 
critical  period  of  reflection  when  teachers  learn  from  others,  question  and  analyse  their  own  experiences  and 
develop  the  capacity  for  principled  knowledge  of  teaching.  For  this  to  occur,  there  needs  to  be  time,  encouragement, 
monitoring  and  feedback  in  a context  where  critical  reflection  can  unfold. 

While  teacher  education  programs  advocate  the  adoption  of  ‘reflective  approaches’  to  their  own  practice,  constraints 
imposed  by  the  resource  implications  are  impinging  on  the  provision  of  opportunities  and  time  for  development  of 
reflective  teaching  practice.  In  the  current  climate  of  decreasing  staff-student  ratios  in  many  education  faculties,  and 
the  reality  that  school  placements  of  teachers  are  widely  dispersed,  the  opportunities  and  tasks  that  foster  reflection 
are  very  limited.  New  frameworks  and  instructional  strategies  for  learning  are  called  for,  and  technology  offers  both 
cognitive  tools  and  resources  to  enhance  learning  for  professional  development  (Robson,  1999). 


Using  technology  to  scaffold  teacher  reflection:  The  QuILT  metaphor 

Communications  technology  and  multimedia  can  contribute  to  the  range  of  tools  that  contribute  to  experiences  of 
teachers  and  foster  critical  thinking.  For  example,  electronic  networks  and  e-mail  contribute  to  reflective  practice  by 
enabling  teachers  to  communicate  and  to  share  ideas  and  experiences  (Wu  & Lee,  1999;  Pearson,  1999).  In  order  to 
enhance  the  skills  of  teachers  before  they  enter  a real  school  setting,  it  is  possible  to  use  a variety  of  technological 
supports  as  facilitation.  QuILT  provides  a complete  multimedia  experiential  environment  where  teachers  can 
engage  in  problem  solving  and  diagnostic  teaching.  Through  the  scenarios  presented  in  QuILT,  teachers  are 
scaffolded  through  a range  of  interpersonal,  diagnostic  and  decision-making  tasks.  Through  technological 
scaffolding,  teachers  encounter  a broad  range  of  experiences  and  also  have  the  opportunity  to  learn  skills  in  a safe 
environment,  with  feedback  provided  by  peers,  technology  and  mentors. 

Four  specific  organisational  devices  have  been  designed  to  provide  a visual  and  conceptual  structure  to  assist  users  as 
they  work  through  the  program.  The  QuILT  program  allows  the  user  to  view  and  re-view  authentic  video  recordings 
of  teaching  episodes  (which  reflect  problem  situations)  from  four  major  lenses: 

1 . What  is  the  teacher  doing? 

2.  What  is  the  teacher  thinking  and  feeling? 

3.  What  are  the  students  doing? 

4.  What  are  the  students  thinking  and  feeling? 
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Each  of  these  lenses  is  signalled  through  colour  coding  and  the  use  of  icons  that  draw  from  aspects  of  weaving 
involved  in  quilt  making.  In  addition,  the  lenses  show  patterns  of  how  the  four  elements  interrelate  in  and  between 
teaching  sequences.  These  interrelationships  demonstrate  that  learning  and  teaching  are  complex,  multiple  and 
strongly  dependent  on  personal,  interpersonal,  and  physical  contextual  features.  Research  perspectives  attest  that  the 
nature  of  quality  learning  and  teaching  is  clearly  pluralist  and  relativist  in  nature  (Alexander,  1996)  and  this  is 
integrated  into  the  design  of  the  multimedia  environment  of  QuILT.  Metaphorically,  each  of  the  four  lenses  are  the 
fibres  (colour  coded  to  teacher-related  lenses  1 and  2,  and  student-related  lenses  3 and  4)  that,  through  weaving,  form 
the  material's  warp  and  weft.  The  Quilt  is  an  intricate,  complex  web  of  teacher  beliefs,  understanding  conceptual 
knowledge  and  action,  that  is  woven  from  experience  and  reflection. 

For  learners  engaging  with  this  virtual  classroom  environment,  an  adequate  interpretation  of  each  of  the  episodes  and 
consequent  decisions  regarding  future  personal  teaching  actions  can  only  be  achieved  by  consideration  of  the  episode 
through  many  or  all  of  these  lenses.  Encounters  with  multiple  perspectives,  decision-making  and  inquiry  into  action 
are  the  learning  processes  scaffolded  the  multimedia  environment  on  QuILT,  which  is  informed  by  constructivist 
learning  theory. 


Using  multimedia  technology  as  scaffolding:  QuILT  environment 

The  major  purpose  of  QuILT  is  to  create  a multimedia  environment  whereby  novice  teachers  achieve  particular 
goals  related  to  effective  teaching.  In  presenting  critical  aspects  of  classroom  practice  the  intended  learning  outcomes 
are  that  users: 

1 . critically  observe  sequences  of  teaching  that  present  classroom  based-problem  situations  and  then  respond  to 
these  by  considering,  determining  and  undertaking  planned  actions; 

2.  develop  or  modify  their  conceptions,  values  and  beliefs  in  ways  that  indicate  enhanced  understandings  of  quality 
through  a process  of  guided  reflection  involved  in  the  performance  of  particular  tasks; 

3.  make  decisions,  record  these  and  then  defend  their  choices  within  and  outside  the  program.  Some  feedback  will 
be  within  the  program  design,  other  feedback  will  come  from  other  users  and  from  the  teaching  staff;  and 

4.  solve  case-based  problems  in  ways  similar  to  the  problem  presented  on  the  CD  ROM  applying  strategies  and 
thinking  employed  by  effective  teachers. 

Task  design  is  based  on  the  pedagogy  of  problem  and  case-based  learning  (Savery  & Duffy,  1995:  McLoughlin  & 
Oliver,  1999).  ) Problem  and  case-based  learning  reflects  conceptions  of  learning,  knowledge  and  understanding  that 
are  very  different  from  conventional  ideas  on  learning.  For  example,  content  or  subject  experts  know  a lot  about  a 
subjects  and  their  expertise  is  judged  on  the  amount  of  content  known,  or  on  the  display  of  propositional  knowledge 
{to  know  that  such  and  such  is  the  case).  In  professional  areas  it  is  also  important  to  have  procedural  knowledge  {to 
know  how  to  do  something).  It  is  this  alternative  understanding  of  expertise  that  characterises  problem-based 
learning.  According  to  Margetson  (1997:  38)  “Expertise  is  the  ability  to  make  sound  judgements  as  to  what  is 
problematic  about  a situation,  to  identify  the  most  important  problems  and  to  know  how  to  go  about  solving  or  at 
least  ameliorating  them”. 

In  the  professions  and  in  the  teaching  world,  the  skills  of  identifying  problems,  managing  solutions  or  improving 
problem-situations  are  considered  to  be  important  competencies  that  need  to  be  scaffolded  or  assisted  in  novice 
learners.  More  recent  work  in  technology-supported  environments  illustrates  how  the  concept  of  scaffolding  has 
expanded  to  include  many  innovative  forms  of  support,  increased  responsibility  for  students  and  a fading  of  the 
directive  of  asymmetrical  aspect  of  earlier  work  on  scaffolding.  Theoretical  models  of  scaffolding  in  technology- 
supported  environments  have  been  elaborated  by  Winnips  (1998)  and  McLoughlin  (1999).  In  QuILT,  the 
multimedia  environment  enables  learners  to  engage  critically  with  authentic  cases  and  develop  professional 
knowledge  and  awareness. 


Employing  the  Technology:  The  Quilt  metaphor  as  an  architecture  for  design 

The  conceptual  and  procedural  frames  for  the  QuILT  program  are  organised  through  the  metaphor  of  a quilt.  The 
construction  of  a patchwork  quilt  involves  a number  of  processes:  the  weaving  of  the  fibres  to  produce  material  from 
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which  the  patches  are  made,  the  construction  of  the  patches,  the  arranging  and  stitching  together  of  these  and  their 
possible  fashioning  through  embroidery.  Each  of  these  stages  of  construction  is  referred  to  iconographically  as  the 
program  proceeds.  For  instance,  the  finished  quilt  is  a metaphoric  construction  of  the  nature  and  practice  of  quality 
in  classroom  learning  and  teaching.  This  construction  requires  the  user  to  create  patches  (one  for  each  of  nine  major 
areas  for  study  ),  and  then  arrange  and  cohere  these  patches  to  make  a unified  conceptual  and  methodological  frame 
for  professional  thought  and  action.  When  constructing  the  patches,  the  user  is  guided  by  utilisation  of  four  different 
'lenses*  with  which  to  explore  lesson  activities  and  contexts.  These  lenses  are  teacher  action,  thinking  and  motivation, 
and  student  action,  thinking  and  motivation. 

This  metaphor  provides  an  integrated  architecture  for  the  design  of  the  program.  Continuous  social  interactions,  a 
sequence  of  patterns  and  relationships,  and  a resultant  creative  harmony  are  necessary  for  successful  crafting  of  a 
quilt.  The  social  meanings  inherent  in  quilt  production  vary  according  to  the  cultural  setting,  the  materials  and  the 
technology  available.  Similar  qualities  and  features  are  evident  in  quality  learning  and  teaching. 


Student  Tasks:  Action,  inquiry  and  reflection  interwoven 

Progress  through  the  program  is  framed  according  to  nine  professional  focus  areas  that  include  planning  units  for 
learning,  teaching  individuals  and  groups,  student  and  classroom  management  and  evaluating  learning.  Users  analyse 
teaching  segments  to  interpret  classroom  activities,  predict  what  actions  will  follow  certain  key  points  in  teaching 
and  learning,  propose  and  justify  actions  they  would  take  themselves  in  such  a teaching  situation  and  draw  inferences 
and  conclusions  about  feasible  and  appropriate  teaching  strategies  to  foster  quality  classroom  practices.  All 
segments  are  presented  in  a way  that  requires  the  user  to  induce,  through  a process  of  guided  reflection,  substantiated 
insights  regarding  effective  and  productive  teaching  and  learning.  In  addition,  users  have  access  to  numerous 
interviews  with  the  classroom  teachers  and  students.  These  interviews  provide  perspectives,  feelings,  explanations 
and  interpretations  of  classroom  interaction  that  are  normally  not  available  to  observers. 

Interactivity  is  manifested  in  various  ways.  The  design  of  some  sequences  within  the  program  prompts  users  to 
pursue  alternative  pathways  and  assess  the  quality  of  teacher  action  and  decision  making.  As  each  pathway  is 
enacted,  the  journey  provides  immediate  feedback  to  the  user  who  has  chosen  that  particular  pathway  and  then 
justified  and  recorded  that  decision.  Other  prompts  within  the  program  require  users  to  record  their  individual 
observations,  predictions,  justifications  and  reflections.  Thus,  by  providing  support  for  inquiry,  participants  are 
scaffolded  in  developing  their  own  understandings  through  active  participation  in  a learning  event.  Each  participant 
has  the  opportunity  to  learn  within  his/her  own  zone  of  development,  and  support  is  given  to  enable  student  to 
perform  the  task  according  to  the  level  of  their  competence  (Vygotsky,  1978).  Offering  choices  of  pathways, 
multiple  perspectives  and  individual  feedback  combined  with  reflective  action  fosters  self-regulation.  In  addition  by 
scaffolding  metacognitive  processes  through  personal  reflection  and  subsequent  group  dialogue,  self  directed-inquiry 
and  decision  making  are  intended  learning  outcomes  (Boekaerts,  1995;  Shin,  1998;  Winne,  1995). 


Reflection  through  an  electronic  notepad 

Two  software  in  QuILT  tools  allow  for  systematic  staged  reflection.  An  electronic  notepad  is  available  to  users  at  all 
times  and  is  saved  to  file  server  or  local  disk  for  further  refinement.  This  notepad  allows  users  to  record  their 
predictions,  observations,  inferences  and  analyses  of  classroom  episodes  and  teacher  and  student  interviews  and  to 
take  a personal  position.  The  second  tool,  known  as  an  action  planner,  requires  users  to  reshape  and  refine  data  from 
the  notepad.  This  is  achieved  through  a variety  of  devices  that  enable  the  user  to  summarise  and  synthesise,  compare 
and  contrast,  signify  and  prioritise,  review  their  positions  and  frame  their  teaching.  Printouts  of  these  files  are 
brought  to  scheduled  class  workshops  for  sharing,  reflection,  analysis  and  synthesis.  The  classroom  becomes  a space 
for  shared  discourse  where  learners  can  negotiate  and  construct  new  understandings.  Technology  offers  the  initial 
support  for  learners  when  they  encounter  new  experiences,  and  provides  a facilitating  structure  and  tools  that  enables 
them  to  make  maximum  use  of  their  own  knowledge  and  experience.  This  form  of  scaffolding  has  been  referred  to  as 
procedural  facilitation  (Scardamalia  & Bereiter,  1994;  Scardamalia  & Bereiter,  1989). 


192 


Summary:  Categorising  the  range  of  scaffolds  in  QuILT 

An  overview  of  the  forms  of  scaffolding  provided  in  QuILT  shows  the  diverse  ways  in  which  knowledge  building 
and  professional  skills  are  developed.  Conceptual  scaffolding  is  provided  when  the  problem  under  study  is 
delineated,  contextualised  and  defined  so  as  to  focus  the  learner  towards  central  issues  and  concepts  where  there 
may  be  multiple  interpretations.  This  is  achieved  in  QuILT  through  the  presentation  of  classroom  scenarios  and 
problems.  Metacognitive  scaffolding  supports  the  underlying  processes  associated  with  learning  management  and 
reflection.  It  provides  support  for  thinking  through  enabling  tool  (an  electronic  notepad)  to  enable  students  to  record 
their  thinking  while  engaging  with  the  classroom  scenarios.  Metacognitive  support  is  also  provided  by  promoting 
and  questioning  learners  to  relate  incidents  to  personal  belief  systems.  In  addition,  QuILT  fosters  thinking  processes 
by  encouraging  users  to  link  cases  with  prior  knowledge  and  experience.  Procedural  scaffolding  supports  learners  in 
using  available  tools  and  resources.  In  QuILT,  classroom  scenarios  are  viewed  from  different  angels,  or  lenses 
(teacher  action  and  motivation,  student  action  and  motivation).  Each  of  these  lenses  is  signalled  through  colour 
coding  and  use  of  icons  that  use  the  metaphor  of  weaving,  as  in  quilt  making.  Visual  and  conceptual  forms  of 
scaffolding  are  thus  interwoven.  Strategic  scaffolding  is  afforded  by  emphasising  alternative  courses  of  actions  and 
events  that  might  be  applied  in  classroom  contexts.  The  presentation  of  multiple  scenarios,  events  and  perspectives 
in  QuILT  supports  analysis,  planning  and  decision  making,  which  are  valued  skills  in  teaching. 


Conclusion:  QuILT  as  a cognitive  tool 

Initial  assessment  of  user  responses  to  QuILT  have  been  positive.  For  novice  teachers,  the  CDROM  is  an  innovative 
and  powerful  cognitive  tool,  a multimedia  resource  that  supports  higher  order  thinking,  decision-making  and  the 
development  of  critical  reflective  practice  among  student  teachers.  This  resource  provides  experiential  learning 
environment  that  fosters  the  reflective  skills  needed  to  become  a professional  educator. 
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Abstract:  This  paper  discusses  the  potentials  and  limitations  of  web-based  Virtual  Reality  (VR)  in  operating  slide 
projector.  A VR-based  interactive  and  simulated  environment  on  web  for  immersively  learning  slide  projector 
operation  has  been  designed,  developed,  and  evaluated.  The  framework  and  functionality  of  the  system  described 
in  this  study  are  also  presented.  The  evaluation  results  report  that  this  learning  material  has  positive  effects  on 
students'  learning  outcomes.  Finally  some  indication  of  further  researches  to  be  done  will  be  given  based  on  using 
the  material  developed  in  this  study. 

Introduction 

It  has  been  reported  that  many  students  fail  to  easily  understand  the  manipulation  activity  that  occurs  at  a 
psychomotor  level  during  the  production  and  presentation  of  slides,  and  the  operation  of  slide  projector.  Students 
usually  do  not  have  enough  time  to  physically  practice  the  operation  of  slide  projector  after  the  class  due  to 
insufficient  equipment  of  slide  projectors  provided  by  the  school.  Students  are  therefore  not  well  motivated  to  learn 
this  subject  and  have  difficulty  grasping  these  complex  procedures.  Without  sufficient  simulation  and  visualized 
demonstration,  students  argued  that  transfer  of  skills  and  retention  of  knowledge  for  producing  slides  and  operating 
slide  projector  would  not  effectively  work. 

VR-based  learning  environment  on  the  Web  could  provide  alternative  to  make  presentation  of  learning 
materials  more  visualized  and  enables  their  demonstration  dynamically.  Students  would  be  more  motivated  to 
actively  learn,  and  able  to  highly  visualize  the  complex  process  of  producing  slides  and  operating  slide  projector. 
Additionally,  students  could  then  manipulate  the  VR-based  system  dynamically  and  learn  by  their  own  hands-on 
experiences  how  slides  are  produced  and  a slide  projector  is  operated  that  is  impossible  to  observe  in  real  situation. 
The  application  of  distributed  VR  to  online  simulation  for  the  hands-on  experiences  of  operating  slide  projector  on 
World  Wide  Web  (WWW)  are  worthwhile  creating  for  reasons  of  equipment,  safety,  expense,  time,  and  distance. 
VR-based  learning  environment  on  the  Web  to  learn  slides  projector  operation  (WebVR)  would  be  very  helpful  for 
students  to  acquire  and  construct  the  needed  knowledge  in  the  social  and  active  sense. 

Application  of  VR  to  Psychomotor  Learning 

Virtual  Reality  are  drawing  more  and  more  attention  in  skill  training  and  psychomotor  learning  due  to  its 
capabilities  of  real-time  interactivity,  three  dimensional  animation,  situated  simulation,  dynamic  mediated 
environment,  immersive  sense  of  presence  and  so  forth.  In  such  a artificial  3D  simulated  learning  environment  based 
on  VR  technologies,  students  can  highly  interact  with  the  virtual  world,  move  around  within  it  at  will,  highly  control 
over  their  self-paced  navigation  through  it,  freely  manipulate  virtual  objects  insides,  and  immediately  have  a 
immersive  sense  of  really  being  there.  Additional,  it  allows  students  to  visualize  abstract  concepts,  and  to  observe 
events  at  atomic  scales  that  cost,  distance,  time,  and  safety  factors  make  unavailable  (Youngblut,  1997). 

Repair  to  the  Hubble  Space  Telescope  were  simulated  in  VR  environment  to  enable  the  technicians  to 
familiarize  themselves  with  the  dangerous  manipulations  they  would  experience  before  the  actual  repairs  were  done 
(Traub,  .1994).  Safe  space  and  situation  to  practice  skills  of  repair  was  provided  in  the  virtual  environment  that  was 
risky  to  do  so  in  real  situation.  Skills  learned  in  the  virtual  environment  could  transfer  successfully  to  the  telescope 
repair  ability  of  real  world.  VR  via  the  Internet  and  World  Wide  Web  (WWW)  may  offer  the  possibilities  of  enabling 
students  being  able  to  collaboratively  create  their  online  virtual  worlds  or  contribute  to  those  created  by  others 
(Moore,  1995).  The  types  of  activities  supported  by  VR  capability  facilitate  current  educational  application  that 
students  are  better  able  to  acquire,  master,  retain,  transfer,  and  generalized  new  knowledge  whenever  they  are 
actively  engaged  constructing  their  knowledge  in  a leaming-by-doing  situation  (Youngblut,  1997). 

Design  and  Development  of  Learning  Environment 

Environmental  Framework  and  Functionality 

Students  can  easily  move  around  within  and  freely  navigate  through  the  virtual  worlds  and  do  all  kinds  of 
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actions,  such  as  freely  manipulating  virtual  objects  in  any  degrees  of  freedom.  Additional,  students  immersively 
existing  in  the  3D  environment  of  WebVR  may  experience  several  degrees  of  freedom,  including  full  freedom  in 
navigation,  the  access  to  information  through  multiple  interactions,  and  the  capability  to  figure  their  own  learning 
process. 


Figure  1 Main  menu  of  the  WebVR 


Designing  Instructional  Strategies 

The  constructivist  principles  of  learning  in  the  design  of  WebVR  are  proper  approaches  to  the  work  of  our 
instructional  strategies.  The  main  learning  approaches  to  implementing  the  constructivist  principles  in  the  3D  virtual 
worlds  are  exploration,  discovery,  problem-solving,  and  learning  by  doing.  Using  web-based  VR  browser 
capabilities,  students  are  able  to  freely  explore  the  virtual  objects,  discover  phenomenon,  practice  skills,  move 
around  and  navigate  through  the  3D  virtual  worlds  composed  of  simulated  equipment  and  tools.  The  manipulation  of 
an  object  in  a virtual  world  supports  the  practice  of  constructivism.  According  to  Salis  & Pantelidis  (1997),  object 
manipulation  in  the  virtual  worlds  is  a powerful  learning  strategy  frequently  used  for  constructing  a knowledge 
framework  that  presents  concrete  components  for  associating  some  superior  knowledge.  Additionally,  transformation 
and  rotation  of  the  objects  enable  students  clearly  to  observe  their  various  situations  that  can  be  rarely  observed  in 
traditional  situation.  The  attributes  of  virtual  objects  can  be  explored  through  direct  interaction  with  them.  In  this 
study  the  WebVR  was  using  these  strategy  to  develop  psychomotor  skills  for  high  level  of  human  interaction  and 
knowledge  association  from  a motor  answer  to  a stimulus  which  students  received  in  the  virtual  worlds. 

A self-space  learning  navigation  system  is  employed  in  the  WebVR  to  increase  the  opportunities  for 
exploration,  manipulation  of  objects,  discovery,  movements,  and  senses  of  real  or  not  real  by  using  visualized 
controlling  elements  within  virtual  worlds.  The  self-spaced  learning  navigation  system  for  users  through  the  virtual 
worlds  was  designed  and  used  as  a learning  tool  in  search  for  information  and  objects  so  as  to  highly  take  control 
over  their  exploration  and  manipulation.  In  a virtual  world  students  may  use  the  navigation  system  to  freely 
determine  to  select  and  manipulate  the  objects  that  might  be  of  interest  and  for  the  observation  at  convenience. 
Visualization,  navigation,  and  manipulation  tools  was  developed  in  the  WebVR  to  allow  students  to  move  around 
within  a virtual  world  and  to  discover  the  attributes  of  an  object. 

Designing  Learning  Content  and  Activities 

The  content  of  this  WebVR  material  for  the  VR  simulation  part  (operating  slide  projector)  was  divided  into 
four  essential  parts  based  on  the  results  from  a content  analysis  and  task  analysis,  they  are  individually  Set  Up, 
Operate,  Disassemble,  Maintain,  and  Troubleshooting.  The  VR-based  learning  world  contains  many  virtual  objects 
such  as  slide,  focus  button,  lens  barrel,  slide  tray,  lock  ring,  elevation  leg,  remote  controller,  forward  button,  reverse 
button,  and  selection  button  that  are  the  components  of  a slide  projector.  Each  object  is  made  from  many  models  and 
attributes.  Models  describe  it  outside  appearance,  while  attributes  present  its  interactivity  and  actions.  Students  are 
allowed  to  reinforce  their  experiences  of  operating  slide  projector  through  manipulating  the  virtual  objects  and 
exploring  the  virtual  worlds. 

When  students  pick  up  a virtual  object,  they  may  execute  the  actions  predefined  insides  it  by  clicking  the 
mouse  button  such  as  rotation  or  deflection.  Students  can  pick  up  and  move  the  objects,  as  well  as  observe  them  and 
their  behaviors  from  various  directions.  The  scenes  in  the  virtual  worlds  have  viewpoint  so  that  students  can  quickly 
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view  the  virtual  objects  from  the  front,  back,  left,  right,  top,  and  bottom.  Object  names  are  shown  when  students 
move  the  cursor  on  them  using  mouse,  while  the  names  disappear  when  the  cursor  is  moved  away.  By  this  way, 
students  may  observe  the  various  components  of  the  slide  projector,  and  learn  the  names,  appearances,  and  attributes 
of  the  virtual  objects.  In  other  words,  students  must  actively  manipulate,  explore,  and  interact  with  the  virtual 
objects,  otherwise  they  could  not  be  able  to  learn  their  attributes. 

In  the  VR-based  learning  world  of  operating  slide  projector,  the  functionality  and  simulated  actions  are  as 
follows  (Heinich,  Molenda,  Russell,  & Smaldino,  1996): 

1 . Slides  could  be  load  into  tray  and  locking  ring  could  be  tightened  as  users  pick  up  the  slides  and  place  them  into 
the  tray,  and  adjust  the  locking  ring  on  the  tray  by  clicking  mouse  button  and  moving  the  mouse. 

2.  The  slide  would  be  pop  up  as  users  push  and  hold  selection  button  by  clicking  mouse  button. 

3.  Slide  tray  would  rotate  clockwise  or  counterclockwise  as  the  users  press  forward  or  reverse  button  on  slide 
projector  or  remote  controller  to  project  slides  by  clicking  mouse  button. 

4.  Lock  ring  could  be  locked  on  slide  tray  as  users  tighten  it  by  clicking  and  moving  mouse  button.  If  it  is  not  locked, 
system  would  present  a warning  message  that  "slides  would  drop  out." 

5.  Lens  barrel  would  move  out  or  in  as  users  adjust  focus  knob  or  move  lens  barrel  to  focus  image,  position  it  on 
screen,  make  it  smaller  or  larger  by  clicking  mouse  button  and  moving  the  mouse. 

6. User  could  remove  slide  tray  and  remove  slides  from  the  slide  tray  buy  clicking  mouse  button  and  moving  the 
mouse. 

7 Slide  projector  could  be  viewed  from  different  directions  by  clicking  mouse  button  and  moving  the  mouse. 

8.  Names  of  virtual  objects  such  as  button,  lens  barrel,  slide  tray,  lock  ring,  elevation  leg,  remote  controller,  forward 
button,  reverse  button,  and  selection  button  would  be  shown  up  when  users  place  cursor  on  them  by  moving 
mouse. 

9.  Slide’s  image  would  upside  down  or  backwards  as  users  remove  the  slide  and  reverse  it  by  clicking  mouse  button 
and  moving  the  mouse. 

Development  Platforms  and  Tools 

Due  to  the  factors  of  cost  and  convenience,  a desktop  virtual  environment  with  less  immersion  was  developed 
in  this  study  rather  than  using  a head-mounted  display  with  higher  immersion.  In  order  to  make  interfacing  uses  as 
friendly  as  possible  and  keep  the  cost  as  low  as  possible  in  cases  of  educational  institutions'  using  our  system,  we 
chose  a standard  mouse  rather  than  a data  glove  or  a 3D  mouse.  In  addition,  data  golve  and  3D  mouse  art  still  not 
popular  to  the  public  at  large  was  taken  into  consideration  for  not  using  them.  WebVR  in  the  study  may  therefore 
support  direct  dynamic  interaction  with  the  virtual  worlds  and  objects  with  a standard  and  regular  mouse. 

Recent  advanced  technology  in  WWW  platform  have  provided  with  tools  and  browsers  for  the  capability  of  A 
desktop  Virtual  Reality  known  as  the  Virtual  Reality  Modeling  Language  (VRML).  3D  Webmaster,  the  ultimate 
software  tool  for  designing  interactive  3D  WebVR  worlds  in  the  study,  allows  developers  to  create  rich,  interactive, 
real-time  3D  worlds  quickly  and  efficiently,  as  well  as  lets  developers  add  interactive  3D  virtual  worlds  to  flat  2D 
Web  pages.  3D  Webmaster  not  only  may  support  VRML  and  3D  web  producing  but  also  can  integrate  with  HTML 
and  JavaScript  commands.  In  addition  to  hundreds  of  ready  made  components  and  Virtual  Clip  Art  objects,  there  is  a 
fully  functional  3D  Shape  and  World  Modeller  as  well  as  Sound  and  Image  Editors  included  in  3D  Webmaster.  This 
also  includes  the  powerful  VR  control  language  SCL  (Superscape  Control  Language)  for  assigning  behaviors  to 
virtual  world  objects  (see  http://www.superscape.com/).  3D  Webmaster  provide  users  interactions  and  offer  a natural 
way  for  extending  the  practice  environment  to  a virtual  3D  practice  world.  This  extension  provides  with  more 
advanced  simulation  and  enable  learning  transfer  from  a virtual  situation  to  a real  situation. 

Prototyping  WebVR  materials  including  interface  was  first  developed  by  using  Frontpage,  and  then  completed 
environments  were  further  established  on  the  WWW  by  using  3D  Webmaster  to  create  the  3D  virtual  worlds  and 
objects  relevant  to  slides  production  and  slide  projector  operation.  Superscape  Vi  scape,  the  fastest  3D  browser 
plugin,  enables  users  to  view  the  product  developed  by  3D  Webmaster  and  lets  users  experience  interactive  3D 
virtual  worlds  on  Web.  It  is  therefore  needed  to  install  for  viewing  the  WebVR  materials  in  the  study. 

Evaluation  Methodologies  and  Results 

We  evaluated  the  quality  of  the  WebVR  through  doing  qualitative  study  for  mainly  examining  the  functions, 
interface,  content,  and  learning  experiences  by  means  of  questionnaires  that  embrace  specific  questions.  The  results 
from  the  survey  of  students'  feedback  on  the  satisfactions  for  the  WebVR  reported  that  the  system  interface  is  easily 
accessible  and  usable  to  users.  They  also  indicated  that  there  are  many  learning  materials  that  can  be  delivered  more 
effectively  by  video  and  audio  media  that  are  not  provided  in  the  WebVR  materials.  Students  reported  that  they  liked 
having  hands-on  experiences  with  the  virtual  worlds,  and  they  felt  comfortable  with  the  simulated  capabilities 


197 


provided  by  the  WebVR  material. 

In  summary,  the  conclusions  drawn  from  the  evaluation  study  are  as  follows: 

1 . The  majority  of  students  had  a very  positive  response  in  the  functionality,  usability,  interface,  and  content  of 
WebVR. 

2.  Students  commonly  enjoyed  learning  within  the  virtual  worlds,  and  their  learning  process  was  highly  motivated. 

3.  Students  were  not  tolerant  of  the  low  running  space  that  sometime  occurred. 

4.  The  use  of  WebVR  could  encourage  students  self-directed  learning 

5.  The  use  of  WebVR  could  captivate  students'  attention  and  foster  active  involvement  in  their  learning  process. 

6.  The  test  scores  of  students  had  higher  learning  performances  than  those  of  traditional  instructions  in  classroom 
based  upon  the  comparisons  with  last  few  semesters. 

As  a result  of  the  evaluation  study,  the  WebVR  provides  a promising  platform  for  exposing  students  to 
actively  learn  in  the  domain  of  "Slides  Projector  Operation",  draw  a high  attraction  to  students  and  can  be  employed 
for  a wide  variety  of  instructional  purposes.  A claim  can  be  derived  from  observation  study  that  identified  a fact  that 
WebVR  does  support  the  learning  process,  reception  of  information,  and  active  participation.  WebVR  acts  as  an 
experience  facilitator  who  allows  students  observe  and  explore  phenomenon  via  their  various  senses.  In  short,  our 
key  insight  to  the  evaluation  results  of  the  WebVR  material  is  the  verification  of  its  successful  design,  creation  and 
use  experiences. 

Conclusions  and  Summary 

A VR-based  learning  environment  on  the  Web  was  developed  to  examine  the  feasibility  of  applying  VR  to 
media  education.  However,  the  material  does  not  want  to  replace  the  teacher,  but  only  wants  to  be  a teacher-assistant 
or  student-assistant  to  help  guide  the  student  during  the  learning  process.  This  means  that  if  students  experience 
difficulties  learning,  they  may  attempt  to  solve  the  problem  in  collaboration  with  their  peers  under  the  support  of 
web-based  VR  technologies.  Evaluation  results  of  WebVR  indicate  that  it  is  a helpful  tool  both  for  students  and 
teachers  in  the  learning  process.  This  is  concerned  about  the  experiences  students  gained  from  the  free  navigation 
and  high  interaction  afforded  to  them,  and  from  the  exploration  of  basic  skills  of  operating  slide  projector.  In 
summary,  this  study  not  only  refines  the  content  of  distributed  VR-based  learning  materials  on  the  Web  developed  to 
learning  slides  production  and  slide  projector  operation,  but  also  help  us  verify  a methodology  for  instructional 
designing  uses. 

We  do  believe  that  the  application  of  VR  on  web  not  only  highly  encourage  the  creative  design  of  learning 
environments,  methods,  and  tools,  but  also  bring  us  into  the  new  instructional  strategies  and  tactics.  It  is  hopeful  that 
more  studies  on  the  web-based  VR  learning  environment  will  be  conducted  to  benefit  all  students  in  teacher 
preparation  program. 

Suggestions  and  Future  Work 

The  WebVR  is  currently  a stand-alone  application  not  associated  with  the  learning  support  tools  such  as  chat 
room,  conferencing  and  discussion  board  on  the  Web.  It  would  be  greater  to  be  integrated  into  a web-based 
instructional  environment  so  that  the  high  level  of  collaborative  learning  activities  can  more  profoundly  be  developed 
and  relative  issues  can  more  deeply  be  investigated.  In  this  study,  we  experienced  difficulties  in  displaying  a large 
amount  textual  information,  however  some  textual  information  is  indispensable  for  learning  slide  projector  operation. 
Therefore,  how  to  display  large  amounts  of  textual  information  using  current  VR  techniques  is  still  a big  problem 
that  needs  to  solve.  An  experimental  research  to  examine  the  real  learning  performance  by  comparing  the  classical 
classroom  situation  with  the  WebVR  through  statistically  controlled  group  and  experimental  group  is  recommended 
being  conducted  in  the  future.  (This  research  was  supported  by  NSC  Project  No.88-2520-S-032-003  of  Taiwan) 
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Abstract:  After  an  instructor  had  used  the  Web-based  forum  in  a course  for  years,  the 
system  database  will  contain  an  accumulation  of  discussion  articles  from  different 
classes,  which  may  be  of  similar  topics.  Furthermore,  the  discussion  processes  of  some 
classes  may  be  very  heuristic  for  other  learners  in  the  same  course.  In  general,  the  Web- 
based  forum  serves  as  an  additional  discussion  channel  for  learners  in  the  absence  of  an 
instructor.  However,  an  instructor  may  want  to  guide  learners  to  read  the  meaningful 
discussion  articles  in  portfolios  of  previous  learners  when  a similar  question  is  asked. 
Hence,  it  is  necessary  to  model  the  relationships  between  discussion  articles  of  different 
classes.  Furthermore,  an  instructor  desires  no  unanswered  question,  called  dangling 
question,  in  the  Web-based  forum  because  of  the  possible  negative  influence,  such  as 
discouraging  learners  to  raise  questions.  This  paper  proposes  a concept-mapping  method 
for  representing  the  relations  between  learners’  questions  and  accumulated  discussion 
articles,  which  enable  a learner  to  find  easily  the  set  of  discussion  articles  from  the 
system  database  with  conceptual  similarity  to  his/her  question. 


Introduction 

Many  researches  use  portfolios  for  learning  assessment,  supporting  instructors’  decision  making, 
or  planning  instruction  (Wiggins,  1993;  Chang  and  Chen,  1998;  Chang,  Chen,  and  Ou,  1998).  Such  usage 
of  portfolio  is  only  from  an  instructor’s  perspective.  From  a learner’s  points  of  views,  portfolio  means 
learning  efforts  of  previous  learners.  For  instance,  portfolio  of  a previous  learner  may  include  the  previous 
learner’s  discussion,  project,  and  homework  so  on.  The  portfolios  of  a Web-based  learning  environment 
can  be  reused  as  recommended  readings  to  eliminate  dangling  questions  by  filtering,  encoding,  and 
indexing  previous  learners’  portfolios  into  a concept  similarity  representation.  This  paper  proposes  a 
methodology  for  reusing  the  discussion  articles  in  portfolios  to  answer  dangling  questions  by  the  concept 
similarity  between  dangling  questions  and  discussion  articles. 

Most  of  the  existing  asynchronized  discussion  styles  is  thread-based  on  the  Web.  This  means  that 
an  initial  article  was  first  posted,  which  followed  by  articles  of  the  same  subject.  Discussion  articles  can  be 
presented  in  different  ways,  for  instance,  by  time  order,  author  order,  or  subject  order.  However,  most 
learners  will  use  subject  order  because  they  need  to  know  others’  response  to  the  subject  before  posting  an 
article.  Sometimes,  a prior  learner  will  use  ’’please  see  a specific  article  posted  before"  as  an  answer  to  a 
question  raised.  Furthermore,  new  learners  are  very  interested  in  the  experience  of  the  prior  learners.  For 
instance,  new  learners  may  want  to  know  whether  a specific  topic  had  been  discussed  or  not.  Hence,  it  is 
necessary  to  find  a method  for  reusing  prior  learners’  discussion  articles. 

Very  often,  some  questions  are  never  answered  in  the  discussion  activity  of  the  distance  learning 
system.  These  unanswered  questions  are  called  as  dangling  questions.  The  dangling  questions  may  occur 
for  different  reasons.  For  instance,  the  dangling  question  is  so  easy  that  other  learners  ignore  it.  Sometimes, 
other  learners  may  not  be  discussing  the  topics  related  to  the  dangling  question.  Dangling  questions  are 
important  both  to  the  learners  and  the  discussion  group.  If  a dangling  question  is  noticed  and  answered,  the 
discussion  that  follows  may  be  very  abundant  in  some  cases.  Furthermore,  some  studies  have  indicated  the 
positive  effects  of  reducing  the  turnaround  time  of  learners’  question  (Rekkedal,  1983).  However,  there  is 
still  no  mechanism  for  triggering  learners  to  review  dangling  questions. 

There  are  many  advantages  for  dangling  questions  to  be  answered  by  prior  learners’  discussion 
articles  in  the  discussion  process  of  a Web-based  forum.  First,  the  participation  of  the  discussion  activity 
will  be  enhanced  because  every  learner  may  feel  that  there  are  responses  to  his/her  question.  Second,  the 
group  discussion  will  be  more  efficient  because  prior  learners’  articles  will  answer  some  types  of  questions, 
for  instance,  the  frequently  asked  questions.  Third,  all  learners  will  get  an  additional  chance  to  reflect  more 


on  the  dangling  question  in  the  light  of  prior  learners’  discussion  articles. 

Portfolios  can  also  be  used  for  active  dialogue  scaffolding.  Discussion  articles  may  contain  any 
topic,  for  instance,  discussing  a misconception,  exchanging  notes,  sharing  an  article,  and  even  checking  on 
the  deadline  for  homework.  If  there  is  a prior  learner  in  the  new  class,  he/she  will  often  use  previous 
discussion  articles  to  answer  questions  for  the  new  classmates.  The  types  of  questions,  answered  by  prior 
learners’  discussion  articles,  varied  from  frequently  asked  questions  for  novices  to  very  specific  questions 
for  experts;  depending  on  whether  the  prior  learners’  discussion  articles  contain  the  topic  of  the  new 
question  or  not.  Hence,  it  is  very  difficult  to  reuse  previous  discussion  articles  without  a prior  learner  in  the 
new  class.  The  issue  is  referred  to  as  the  reusability  problem. 

Learners  do  not  know  whether  a specific  topic  in  a course  has  been  discussed  or  not.  In  general, 
learners  will  use  the  keyword  search  method  to  probe  whether  there  are  related  discussion  articles  in  prior 
learners’  portfolios  or  not.  However,  the  results  of  keyword  search  may  be  too  much  to  handle.  Hence, 
learners  will  use  the  Boolean  system  to  improve  the  keyword  search  results.  Using  the  Boolean  system  to 
retrieve  answers  from  discussion  portfolios,  learners  have  to  provide  sufficient  syntactical  restrictions  in 
their  query  to  limit  the  number  of  articles  retrieved.  Worse  still,  these  retrieved  articles  are  not  ranked  in 
any  relationship  to  learners’  query.  However,  learners  may  not  be  familiar  with  the  terminology  of  the 
specific  domain,  which  could  be  used  in  the  discussion  articles.  Furthermore,  learners  find  it  difficult  to  get 
consistently  good  results  because  they  lack  the  training  for  expressing  the  complex  query  syntax  required 
by  the  Boolean  system.  Consequently,  some  models  for  retrieving  answers  easily  from  prior  learners’ 
discussion  portfolios  should  be  constructed.  This  issue  is  denoted  as  the  recognizability  problem. 

Prior  learners’  discussion  portfolios  are  not  meant  for  use  by  novices.  Its  organization  is  difficult 
for  a novice  to  absorb  prior  learners’  learning  experience.  Hence,  it  is  necessary  to  reorganize  prior  learners’ 
Web-based  discussion  portfolios  so  that  even  new  learners  can  benefit  from  using  them.  Although  learners 
can  retrieve  prior  learners'  discussion  articles  by  asking  questions,  they  may  want  to  know  why  prior 
learners  discussed  this  topic.  There  should  be  a mechanism  not  only  for  answering  dangling  questions,  but 
for  also  providing  a hyperlink  to  the  beginning  of  the  thread  leading  to  the  answer  (the  discussion  article) 
of  the  dangling  question.  Furthermore,  a thread  of  prior  learners’  discussion  articles  may  contain  a 
reference  to  other  portfolios.  Hence,  the  prior  learners’  portfolios  should  be  more  readable  for  novices  to 
reuse  and  benefit  from  them.  The  issue  is  called  the  readability 

This  paper  presents  a ranking  method  of  retrieving  discussion  articles  and  detailed  instructions  on 
using  ranking  technology  to  answering  dangling  questions  of  a Web-based  forum  by  the  support  of  prior 
learners’  portfolios  in  a distance  learning  system.  First,  we  solve  the  reusability  problem  by  answering 
learners’  questions  with  retrieved  articles  fi*om  prior  learners’  discussion  articles.  Then,  a concept-based 
mapping  method  is  used  to  reorganize  prior  learners’  portfolios  so  that  learners  can  benefit  from  them 
without  the  readability  problem.  Finally,  the  relevance  ranking  in  concept  vector  space  is  used  to  retrieve 
articles  from  portfolios  for  answering  questions.  Learners  need  not  know  whether  there  is  any  related 
discussion  articles  in  prior  learners’  discussion  articles  before  asking  questions.  Hence,  the  vector  space 
model  will  solve  the  recognizability  problem.  The  following  table  illustrates  the  relationships  between  the 
aforementioned  problems,  requirements  and  proposed  methodology. 


PROBLEMS 

REQUIREMENTS 

METHOD 

Reusability 

Reuse  portfolios  for  scaffolding 

Article  retrieval 

Readability 

Reorganize  portfolios 

Concept-based  mapping 

Recognizability 

Represent  portfolios  for  answering 
questions 

Vector  space  model 

Table  1:  Relations  between  problems  and  methods. 


Illustrative  Example  and  Article  Retrieval 

We  assume  that  there  are  seven  vocabularies  in  a specific  domain,  including  “factors”,  “distance”, 
“human”,  “portfolio”,  “collaboration”,  “learning”,  and  “system”.  If  a learner  asks  “Please  tell  me  about 
collaboration  in  distance  learning  systems”,  the  question  will  be  transferred  to  the  7-dimensional  vector  as 
follows: 

9 = [0  1 0 0 1 1 l] 

where  q denotes  the  question.  The  terms  in  q are  determined  by  whether  the  vocabularies  appear  in  q or 
not.  The  question  will  become  a dangling  question  if  there  is  no  answer  to  the  question  after  a few  days. 

Suppose  there  are  three  learners’  discussion  articles,  denoted  as  di,  d2,  and  d3.  Discussion  article 
di  contains  “distance”,  “portfolio”,  “collaboration”,  “learning”,  and  “system”  vocabularies.  Hence,  the  7- 
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dimensional  vector  for  di  is  as  follows: 

= [o  1 0 1 1 1 i] 

Suppose  discussion  article  d2  contains  ‘‘factors”,  “distance”,  “human”,  “learning”,  and  “system”;  while 
discussion  article  dj  contains  “factors”,  “human”,  and  “collaboration”.  Similarly,  these  two  discussion 
articles  are  denoted  as  follows: 

^2  = [i  1 1 0 0 1 i] 

= [l  0 1 0 1 0 o] 

To  determine  whether  an  article  is  a suitable  answer  to  a dangling  question,  a dot  product  of  the 
dangling  question  vector  and  each  document  vector  is  made.  The  document,  with  the  highest  result  number, 
will  be  the  answer  to  the  dangling  question.  The  q'di^  of  di,  d2,  dj  is  4,  3,  2 respectively.  Consequently, 
discussion  article  dj  is  retrieved  to  answer  the  dangling  question. 

The  above  method,  though  feasible,  also  causes  some  difficulty.  First,  the  matching  process  may 
cost  expensively  because  there  are  many  accumulated  discussion  articles.  An  alternative  is  to  cluster  the 
discussion  articles  with  the  same  concept  into  the  same  category.  Second,  the  previous  method  determines 
the  relevance  degrees  between  a question  and  accumulated  discussion  articles  by  shared  words.  However, 
there  is  a vocabulary  gap  problem  in  retrieving  discussion  articles.  The  vocabulary  gap  problem  means  that 
two  discussion  articles  may  be  related  in  concept  but  there  may  be  no  shared  words.  Consequently,  a 
concept-based  methodology  is  crucial  for  solving  the  vocabulary  gap  problem  by  vocabulary 
transformation. 


Concept-Based  Mapping  Method 

The  goal  of  answer  retrieval  is  to  post  a discussion  article  in  prior  learners’  portfolios  to  a 
learner’s  question.  The  basic  problem  is  how  to  determine  the  relevance  of  a discussion  article  in  portfolios 
to  a learner’s  question.  In  other  words,  this  is  a problem  of  relevance  ranking  of  the  discussion  articles  in 
portfolios.  Once  the  discussion  articles  are  retrieved  with  respect  to  a question,  they  can  be  ranked 
according  to  a relevance  score.  However,  the  vector  representation  of  a discussion  article  does  not  indicate 
how  the  discussion  articles  can  be  arranged  by  a relevance  score.  In  other  words,  the  vector  representation 
of  a discussion  article  does  not  consider  the  learning  requirements  of  a novice,  which  depends  on  the 
learning  domain  and  concept. 

To  determine  the  relevance  score  of  discussion  articles  to  a learner’s  question,  there  should  be  a 
ranking  method  that  is  based  on  the  conceptual  similarity,  rather  than  on  words  shared  by  the  question  and 
the  retrieved  answer.  This  method  cannot  assume  that  there  is  any  shared  vocabulary  between  the  learners’ 
questions  and  the  discussion  articles.  Figure  1 illustrates  an  example  of  representing  the  discussion  articles 
as  a matrix.  We  use  four  discussion  articles,  each  with  its  own  set  of  concepts.  There  are  two  matrices,  A 
and  B,  used  to  represent  the  concept  mapping  process.  A row  in  matrix  A means  a discussion  article  and 
the  words  contained  in  the  discussion  article.  The  rows  of  matrix  B are  sets  of  concepts  for  each  discussion 
article.  We  use  Wi  to  denote  the  weight  of  a word  because  there  are  many  methods  for  determining  the 
weight  of  each  word,  for  instance,  binary  weight,  within-document  word  frequency,  and  inverse  document 
frequency  (Salton,  1 989). 


Discussion  article  1.  “distance”  and  “learning” 
Discussion  article  2.  “human”  and  “factor” 
Discussion  article  3.  “portfolio”  and  “system” 
Discussion  article  4.  “collaboration”  and  “system” 


□ contains  Concept  {cl,  c3} 

□ contains  Concept  (cl } 

□ contains  Concept  {c3} 

□ contains  Concept  {c2} 


Word 

Weight 
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representing  the  concepts  of  a discussion  article. 


Figure  1:  The  matrix  representation  for  the  concept-mapping  process. 


Figure  1 depicts  how  to  represent  a discussion  article  with  vocabulary  (words)  by  matrix  A and 
the  concepts  of  a discussion  article  by  matrix  B.  Hence,  the  concept-based  mapping  method  can  be 
implemented  without  shared  words  if  the  “magic”  matrix  F can  be  computed  to  transfer  matrix  A to  matrix 
B.  Now  we  are  ready  to  find  a matrix  F that  can  be  used  to  determine  the  relevance  scores  between  the 
question  and  discussion  articles  by  concept  similarity  Theoretically,  the  problem  is  exactly  like  finding  the 
Linear  Least  Squares  Fits  (LLSF)  between  matrix  B and  matrix  FDA.  Under  this  assumption,  the  LLSF 
method  can  provide  a conceptual-based  transformation  of  the  original  words  of  a question  into  the 
vocabulary  of  discussion  articles  in  portfolios  (Yang  and  Chute,  1993). 


Definition:  The  LLSF  problem  is  to  find  a matrix  that  minimizes  the  sum  of  residual  squares 


where 
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and  are  given  matrices  representing  the  training  pairs; 
A ^ and  B ^ are  the  transposes; 

5.  and  b.  are  the  ith  pair  in  the  training  set; 


= I 7 is  a vector  2-norm  of  an  / X 1 vector,  and 
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If 


is  the  Frobenius  matrix  norm  of  an  I Xm  matrix. 


The  LLSF  problem  was  proven  to  have  at  least  one  solution.  The  singular  value  decomposition 
(SVD)  method  is  often  used  to  solve  the  LLSF  problem.  (Lawson  and  Hanson,  1974;  Golub  and  Van  Loan, 
1989) 

The  intuitive  meaning  of  the  solution  of  the  LLSF  problem  is  to  find  an  matrix  F that  can  be  used 
to  measure  the  “distance”  between  a learner’s  question  and  a discussion  article  in  previous  learners’ 
portfolios.  Because  the  matrix  F is  derived  from  discussion  articles  of  some  classes,  the  matrix  F is  trained 
to  assign  properly  a question  to  a set  of  discussion  articles.  The  scenario  may  be  as  follows.  A learner  asks 
a question  a few  days  ago  and  no  one  answers  or  follows  his/her  question.  Then,  the  system  uses  the  matrix 
F to  find  some  related  discussion  articles  from  prior  learners’  portfolios  as  answers  to  the  learner’s 
question.  Finally,  the  system  asks  the  learner  whether  he/she  is  satisfied  with  the  retrieved  discussion 
articles  or  not.  If  he/she  is  satisfied  with  the  retrieved  discussion  articles,  the  methodology  works.  If  not, 
the  system  will  notify  a teaching  assistant  of  the  case.  The  teaching  assistant  will  answer  the  learner’s 
question  or  manually  seek  for  some  related  discussion  articles.  Then,  the  system  will  retrain/derive  the 
matrix  F so  that  it  can  solve  a similar  case  in  future. 

Hence,  the  system  or  the  agent  can  reorganize  the  discussion  articles  according  to  the  similarity 
between  the  transformed  question  and  the  concepts  of  discussion  articles  for  a relevance  order  to  choose  a 
most  suitable  article  in  a suitable  concept  as  the  retrieved  answer.  Then,  we  can  solve  the  readability 
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problem  by  reorganizing  the  prior  learners’  discussion  articles  in  a concept-based  style. 


The  Relevance  Ranking  in  Concept  Vector  Space 

As  the  illustrative  example  demonstrates,  every  discussion  article  is  viewed  as  an  n-dimensional 
vector  space  in  the  concept  vector  space.  Hence,  a dangling  question  should  be  represented  as  an  n- 
dimensional  vector  in  the  concept  vector  space  before  the  system  begins  to  find  the  suitable  article.  Then, 
an  n-dimensional  vector  matching  operation  can  be  used  to  compute  the  similarity  between  a discussion 
article  and  a dangling  question.  Let  dj  denote  the  j-th  discussion  article  and  q refer  to  the  dangling  question 

in  concept  vector  space,  q = {^Fx^Y  . The  system  uses  the  following  formula  to  compute  the  similarity  in 
the  concept  vector  space. 


similarity  {d  j ^q) 


V /=i  1=1 


where 

td-j  means  the  i‘^  vocabulary  in  the  vector  for  the  discussion  article  dj, 

tq^  means  the  i^^  vocabulary  in  the  vector  for  the  dangling  question,  and 

n means  the  number  of  vocabularies  in  the  learning  domain. 

The  vector  space  model  has  been  proved  useful  in  many  experiments  at  Harvard  University  and 
Cornell  University  (Salton  and  McGill,  1983;  Salton  and  Buckley,  1988).  Learners’  question  will  not  be 
dangling  any  more  if  there  is  any  prior  learners’  discussion  articles  related  to  the  concepts  of  his/her 
question  by  the  similarity  in  the  concept  vector  space.  Figure  2 illustrates  the  framework  of  our 
methodology. 


Figure  2:  The  framework  of  the  concept-based  transformation  to  answer  dangling  question. 


Conclusion 

This  paper  proposes  a methodology  for  reusing  the  discussion  articles  of  learners’  portfolio  by  a 
vector  space  model.  Since  the  Web-based  forum  has  accumulated  many  discussion  articles  after  years,  an 
instructor  may  expect  all  learners  in  a new  class  to  benefit  from  prior  learners’  discussion  articles.  Because 
an  instructor  providing  a Web-based,  asynchronized  forum  for  his/her  course  becomes  continual,  the 
“concept-based”  text  retrieval  methodology  to  quick  search  topics  and  scaffold  learners  by  prior  learners’ 
learning  experience  becomes  crucial.  Consequently,  the  relations  between  the  discussion  articles  of 
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different  class  (with  the  same  course)  should  be  constructed. 

This  paper  uses  the  information  retrieval  technology  to  retrieve  a most  suitable  discussion  article 
as  the  answer  of  a learner’s  dangling  question.  Consequently,  a learner  need  not  review  all  prior  learners’ 
discussion  articles  but  can  still  benefit  from  the  prior  learners’  discussion  articles.  Most  important  of  all,  the 
Web-based  discussion  portfolios  not  only  can  play  the  passive  role  in  the  Web-based  distance  learning 
research,  but  can  also  serve  an  active  role  by  answering  dangling  questions.  Hence,  the  forum  will  be  full 
of  suggestions  that  answers  dangling  questions  from  the  perspective  based  on  the  prior  learners’  Web-based 
discussion  portfolios. 
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Abstract:  In  recent  years,  there  has  been  a worldwide  trend  toward  greater  integration  of 
technology  use  in  teaching  and  learning.  There  has  also  been  a concomitant  shift  away  from 
traditional  teacher-centered  instruction  towards  interactive,  peer  tutoring,  collaborative  group 
learning,  and  global  education.  By  leveraging  off  the  dissemination  of  technology,  we  have 
developed  a system  that  allows  multiple  geographically  remote  learners  to  participate 
simultaneously  in  a simulation-oriented  VR-based  collaborative  learning  environment.  Virtual 
environments  allow  users  to  participate  in  experientially-grounded  simulation  environments 
and  support  the  exploration  of  activities  that  may  be  impractical  to  perform  in  real  life. 
Moreover,  users  can  benefit  from  mutual  tutoring  and  interactions  with  experts. 

Our  system  development  work  is  based  entirely  upon  open  standards.  For  this  reason,  the 
system  is  called  VirtUOS  (Virtual  Reality  Learning  Environment  Using  Open  Standards). 
Adherence  to  open  standards  ensures  that  the  system  is  portable  across  hardware  platforms  thus 
benefiting  a wide  range  of  users. 


Introduction 

The  use  of  desktop  virtual  reality  (VR)  to  support  learning  presents  educational  technologists  with  exciting  opportunities  as  well  as 
significant  challenges  that  need  to  be  overcome.  Desktop  VR  is  ideally  suited  to  use  in  domains  that  benefit  from  the  use  of 
simulation,  e^qjerientially-grounded  learning,  and  socialized  learning.  However,  desktop  VR  systems  have  traditionally  been 
realized  by  various  commercial  oiganizations  using  their  own  prcprietaiy  technologies  and  file  formats.  In  view  of  this,  we  have 
developed  a desktop  VR  learning  system  directed  at  supporting  collaborative  learning  in  a simulation-oriented  environment  The 
objective  is  to  allow  leamas  to  leam  coUaboratively  in  real  time  by  sharing  learning  e^q^eriences  as  well  as  by  articulating  and 
refining  conceptual  ideas  related  to  these  learning  experiences.  In  line  with  developments  in  the  VR  field,  our  system  moves  beyond 
the  first  generation  of  navigation-oriented  VR  to  the  developmait  of  interactive  VR,  where  interactions  with  virtual  worid  objects 
give  users  richer  learning  experiences. 


Theoretical  Underpinnings 

Cbnstmctivist  theory  is  pah^s  the  most  widely  accepted  pedagogical  stanc^int  adopted  by  teachers  today.  Constructivism  views 
the  studait  “as  one  who  acts  on  objects  and  evaits  within  his  or  ha  environment  and  thaeby  gains  some  understanding  of  the 
features  held  by  the  objects  and  evaits”  (Bog^-Mehall).  Constmctivism  suggests  that  people  make  significant  progress  whai  they 
collaborate.  Moreover,  mutual  tutoring,  shared  visions,  and  collaboration  produce  substantial  impetus  to  learning.  The  Internet 
makes  this  collaboration  process  convaiiait  and  viable.  However,  as  McKenzie  (1997)  points  out,  technology  does  not  remove  the 
need  for  traditional  sources  of  knowledge.  Studies  by  Strommai  (1992)  also  show  that  technology  is  a completing  augmaitation 
rather  than  a corrq)eting  aggravatioa 

A constmctivist  oriaitation  requires  the  adoption  of  a fiesh  paspective  on  technology.  Instead  of  viewing  compute  as  a knowledge 
presentation  device,  we  should  view  them  as  tools  for  siqporting  a pedagogical  focus  on  communications  in  collaborative  learning 
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ventures  (Koschmann  et  al.,  1993).  If  we  are  able  to  bring  a group  of  peqjle  together  to  intaact  in  amodel  of  a real  aivironm^t, 
then  we  also  have  a tool  for  constructivist  learning  Imagine  a music  conposidon  class  where  the  students  can  compose  music  and 
present  their  individual  works  to  the  rest  of  their  class  in  a virtual  mode.  Or  perh25)s  a chemistry  class  where  students  can  mix 
chemicals  and  test  chemical  reactions  in  the  safety  of  a virtual  chemistry  laboratory. 

Gagnon  and  CoUay  (1996)  posit  that  a successful  Caistiuctivist  Learning  Design  (CLD)  should  provide  familiar  environments  that 
reflect  the  thinking  processes  of  the  participants;  in  such  environments  there  must  be  trust  and  public  sharing  of  knowledge  in  this 
environment  Moreover,  the  Across-Schools  Pedagogy  Issues  group  (Koschmann  et  al.,  1993)  endorses  the  “necessity  of  large-area 
networks  for  particular  contexts,  instructional  goals,  and  learner  characteristics.”  Injecting  constructivism  into  the  educational  system 
culminates  in  a revolution,  from  planning  for  teaching  to  designing  for  learning  (Gagnon  and  Collay,  1996).  Fosnot  (1996)  ri^tly 
asserts  that  the  ‘key  to  reinvaiting  our  educational  systan ...  lies  in  what  our  teachers  believe  about  the  nature  of  knowing” 

Working  within  the  constructivist  paradigm,  we  have  developed  a system  that  creates  virtual  environments  where  usa:s  can  perfonn 
e?q)eriments  and  ejq^erience  simulations  in  support  of  collaborative  on-line  learning.  In  such  environments,  usa:s  can  aigage  in 
experiments  that  may  be  expensive  or  even  (tegqous  to  pafomr  in  real  life.  Our  system  supports  user  interactions  that  facilitate 
mutual  tutoring  and  knowledge  sharing  The  system  can  be  used  by  academic  institutions  that  offer  courses  throu^  distance 
learning  or  it  may  be  used  as  a complemaitary  fonn  of  on-line  collaborative  learning  in  on-campus  educatioa  In  particular,  such 
institutions  can  conduct  laboratory  classes  with  visual  demonstrations  and  presentations  in  real-time.  The  system  can  also  be  used  to 
give  guided  tours  of  museums,  art  galleries,  and  show  rooms  of  new  houses. 


Research  Focus 

There  have  been  several  attempts  to  create  similar  VR  environments.  We  reviewed  four  representative  systems  among  these 
plications.  The  first  system,  VR forLeantingQJm,  1997),  is  limited  in  that  it  does  not  store  its  world  data  in  a database.  Moreover, 
avatars  are  static  and  non-standaixl,  and  they  float  in  virtual  worlds  rather  than  walk  The  second  system  that  we  reviewed  is  Active 
Worlds.  This  proprietary-standard  virtual  world  browser  has  a remarkably  fast  response  time.  However,  this  fast  response  time  is 
achieved  at  the  price  of  only  supporting  avatars  that  are  more  coarsely  defined  The  third  system,  Community  Pbce,  is  designed  to 
be  scalable  to  support  many  “geogr^hically  disposed  users,  interconnected  throu^  low  bandwidth,  hi^  latency  communication 
links”.  However,  Community  Place  separates  the  chat  and  whiteboard  windows  fium  the  virtual  world  navigation  window.  This 
separation  increases  the  semantic  distance  between  the  different  components  of  the  system.  The  final  system,  blaxxun,  has  the 
advantage  of  using  Humanoid  Animation  1.0  (HANIM,  Roehl,  1997)  conpliant  avatars.  However,  blaxxun' s avatars  float  instead 
of  walk.  While  the  avatars  move  very  quickly,  realism  is  conpromised 

In  order  for  our  system  to  benefit  as  many  usos  as  possible,  we  inplemented  our  work  using  open  standards.  Our  system  uses  a 
third-party  virtual  world  browser,  Cosmo  Player.  The  other  corrponents  of  the  system  are  inplemented  in  Java  Hence  our  sytem  is 
portable  to  hardware  platforms  that  sippoit  the  virtual  world  browser.  Moreover,  the  system  is  designed  to  support  a large  number  of 
users  while  rnaintairiing  reasonable  pafotmance. 

Gven  that  we  wanted  to  help  users  participate  in  learning  environments  and  not  in  learning  how  to  use  the  system,  considerable 
attention  was  given  to  designing  an  eng3ging  irteface  that  would  make  the  system  pleasurable  to  use.  A pleasant  experience  will 
attract  usas  back  to  the  virtual  learning  worlds,  resulting  in  more  experiential  learning  and  fkniliarity  with  the  learning  context 

Our  system  moves  away  fiiom  traditional  navigation-oriented  VR  to  interactive  VR  that  enriches  users’  learning  experiences.  In 
VirtUOS,  virtual  object  states  are  modified  by  manipulating  componaits  of  these  virtual  models  directly.  Each  object  has  unique 
behaviors  and  properties,  or  is  able  share  bdiaviors  and  properties  with  other  similar  objects.  This  concept  is  similar  to  Object 
Oriented  Prograrnrrung  where  each  object  has  its  own  variables  and  methods.  VirtUOS  processes  new  object  states  and  ipdates 
virtual  worlds  accordingly.  By  storing  object  states,  usas  can  collaborate  in  discussions  that  span  several  login  sessions.  Moreover, 
VirtUOS  models  avatars  realikically  in  that  they  no  longer  float  around  virtual  worlds.  Instea4  the  avatars  perfonn  life-like  walking 
motions  as  users  navigate  in  virtual  worlds.  Avatars  are  instrumental  for  promoting  user  interactions  because  they  allow  users  to 
establish  presence  in  virtual  worlds  by  simulating  the  sense  of  “being  there.” 


System  Design 

Our  design  considas  the  database,  avatar  animation,  object  irtoaction,  and  user  intaJace  aspects  of  the  systan.  A database  is 
needed  to  provide  storage  and  retrieval  of  virtual  world  states.  Avatar  animation  is  desirable  because  it  increases  the  realism  of  virtual 


worlds.  Moreover,  interaction  with  virtual  worid  objects  is  necessary  to  give  users  a rich  learning  experience.  Furthermore,  users 
should  be  able  to  use  the  system  without  any  training  or  instruction. 

Given  that  VirtUOS  is  targetted  at  Internet  users,  we  use  VRML  to  model  our  virtual  worlds  and  objects.  In  a traditional  VRML 
environment,  the  states  of  virtual  worlds  are  not  maintained  Hoice,  objects  ^>pear  at  their  same  default  locations  every  time  users 
log  in  afiesh.  Users  also  always  entCT  a world  at  the  same  location  regardless  of  where  they  were  whoi  they  last  logged  off.  Lim’s 
system  (Lim,  1 997)  simulates  a database  using  Java.  But  the  database  does  not  exist  on  a persistent  medium  or  in  a forni  that  can  be 
queried  and  iqxlated  using  standard  commands  (such  as  SQL  statemoits).  Furthennore,  Lim’s  systoi  has  to  e;q)licitly  consider 
concurrency  control  issues  that  are  inhermtly  takoi  care  of  in  a database  oivironment  As  we  believe  that  there  is  no  reason  to 
code  a SQL  database,  we  make  use  of  the  comprehaisive  set  of  functions  provided  by  Java  that  standardizes  database  access. 

The  design  of  our  world  objects  to  provide  object  behaviours  requires  our  systoi  to  be  able  to  enc^)sulate  avatar  (and  any  virtual 
object  in  general)  animations,  bdiaviours,  and  propaties  as  though  they  are  Object  Oriented  classes.  Hence,  it  is  necessary  to  have  a 
standard  structure  for  represoiting  an  avatar.  VirtUOS  uses  the  Humanoid  Animation  1.0  (HAMM)  standard  (Roehl,  1997). 
HANIM  1.0  is  an  extaision  of  the  VRML  2.0  standard  that  defines  the  scene  gr^h  of  any  humanoid  VRML  model.  Standard 
avatar  structures  allow  animations,  behaviours,  and  properties  to  be  reused  by  all  other  avatars.  Each  avatar  has  its  own  instantiation 
of  the  animations,  behaviours,  and  properties.  Moreover,  shared  animations  need  to  be  downloaded  only  once  from  the  web  because 
Internet  browsers  cache  downloaded  documents.  Subsequent  accesses  (within  the  same  browser  session)  are  read  from  cache. 

The  plethora  of  possible  objects  in  a virtual  world  makes  it  impossible  to  define  a rigid  structure  for  the  VRML  representations  of 
arbitrary  objects.  However,  it  is  possible  to  characterise  the  common  attnbutes  (e.g.  position  and  orientation)  while  letting  the  other 
attributes  be  object  specific.  For  exanple,  battaies,  resistors,  and  bulbs  have  position,  orientation,  and  resistance  attributes  (and 
associated  “methods”  to  get  and  set  ihs  attributes).  But  only  batteries  have  a voltage  attribute,  and  only  bulbs  have  a power  attribute 
and  a set  method  The  power  attribute  indicates  the  power  that  the  bulb  is  designed  for,  and  the  set  method  is  used  to  tell  the  bulb  the 
amount  of  power  that  the  circuit  is  supplying  it  wifii 

The  design  of  the  user  interface  must  ensure  that  the  systan  is  engaging  and  pleasurable  to  use.  Moreover,  the  interfece  should  be 
simple  enough  for  users  to  use  without  any  prior  training  or  instruction.  It  is  difficult  to  provide  users  with  an  extended  user  interfece, 
e.g.  popup  menus  for  interacting  with  virtual  objects,  because  the  system  makes  use  of  a third-party  application,  such  as  Cosmo 
Player  or  Cortona,  as  the  VRML  browsCT.  Consequaitly,  there  are  no  easy  means  of  extending  the  user  interface  of  the  browser  to 
support  interactions  between  users  and  virtual  objects.  In  our  system,  Cosmo  Player  is  chosen  as  the  browser  for  VirtUOS  because  it 
is  better  established  than  newer  browsers  such  as  Cortona, 


Functional  and  Performance  Requirements 

In  this  section,  we  s^  out  the  functional  and  perfonnance  requirements  that  our  system  is  designed  to  support  Most  of  the  functional 
requirements  that  we  propose  are  desirable  for  enhancing  the  realism  of  virtual  worlds  and  providing  a rich  learning  experience.  On 
the  other  hand,  the  performance  requirements  are  directed  at  supporting  virtual  world  interactions  realistically  with  rniriirrial  latency. 


Functional  Requirements 

Our  system  has  been  designed  to  support  the  following  functions: 

□ Allow  users  to  change  their  avatar  represoitations  and  to  reflect  these  changes  on  all  other  clients. 

□ Allow  the  storage  of  virtual  world  states  and  the  restoration  of  these  states  when  users  log  in  afresh.  This  feature  fk:ilitates  the 
building  of  complex  virtual  structures  spanning  sev^al  sessions. 

□ Allow  animated  avatars  that  walk  instead  of  float  The  systan  also  allows  users  to  walk  in  arbitrary  stq)  sizes  and  directions. 
Users  need  not  walk  in  full  steps,  rwris  the  walking  restricted  to  moving  forward. 

□ Allow  users  to  navigate  by  using  the  cursor  keys/mouse  to  intaact  with  a virtual  world  browso*.  The  systan  should  not  use 
navigation  controls  outside  of  the  virtual  world  because  users  may  have  difficulty  rriEqjping  such  controls  to  the  virtual  world. 

□ Allow  different  actions  to  be  associated  with  an  avatar  with  minimal  changes  to  the  system.  This  would  fiee  us  from  making 
significant  modifications  to  the  systan  when  we  include  experiments  requiring  new  avatar  behaviors. 

□ Ensure  concurrency  control  over  virtual  objects  that  users  can  manipulate.  The  system  should  not  allow  two  or  more  users  to 
update  an  object  at  the  same  time  in  order  to  maintain  consistency  of  virtual  worlds  across  all  users. 

□ Allow  the  alignment  of  virtual  objects  that  users  can  manipulate  by  snapping  to  a grid.  This  “forcing  flmctiori  ’ frees  users  fiom 
having  to  manage  minute  details  of  object  orientations  and  positions. 
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□ Provide  a user  intoiace  to  siqjport  object  interaction.  The  intoiace  provided  by  Cosmo  Player  allows  only  mouse  clicks  and 
dragging  actions.  However,  some  simulations  need  to  support  several  actions  that  depend  on  users’  choices.  Hence,  it  is 
necessary  to  extoid  Cosmo  Player's  user  interface  to  circumvent  this  restrictioa 

□ Allow  users  to  hide  the  avatars  of  otfia-  usees  whom  hey  do  not  wish  to  see.  Users  can  also  use  this  function  to  speed  up  he 
system  when  too  many  users  are  logged  on  simultaneously. 


Performance  Requirements 

It  is  desirable  for  he  ^stem  to  meet  he  following  performance  requirements: 

□ Virtual  world  iqxlates  ^ould  be  propagated  to  enou^  to  all  clients  so  that  users  can  interact  wih  one  anoher  in  real-time. 

□ The  rendering  process  ^ould  take  less  time  when  avatars  are  hidden. 

□ The  system  is  e^qjected  to  run  on  Pentium  II  PCs  wih  at  least  32MB  RAM. 


System  Description 

This  section  is  divided  into  two  parts.  First,  we  describe  our  system  by  means  of  a simulation-oriented  virtual  world  model  that  we 
have  created  In  this  description,  we  present  a typical  scenario  of  how  users  learn  coUaboratively  using  our  system.  Second,  we 
discuss  he  technical  innovations  in  he  development  of  our  system. 


Simulation-oriented  Collaborative  Learning 

VirtUOS  is  equable  of  supporting  multiple  virtual  worlds.  Each  world  has  its  own  learning  goals  and  users  can  teleport  between 
hese  worlds.  Figure  1 (Fig.  1)  ^ows  a screen  snapshot  of  VirtUOS  viewed  through  Netscape  4.5  and  Cosmo  Player  2.1 . The  Chat 
client  and  Multi-user  client  are  displayed  at  he  bottom  left  and  bottom  ri^t  of  he  virtual  world  respectively.  The  HTML  display  on 
he  ri^t  hand  side  of  he  Netscape  window  explains  he  goals  of  he  virtual  world  that  Cosmo  Player  displays. 
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One  particular  world  in  our  irtplemaitalion  uses  the  flowing  waters  and  teaning  crowds  analogies  (Goitno"  and  Gaitner,  1 983)  and 
colliding  billiard  balls  (Humpluey  and  Topping,  1949)  to  ctaionstrate  the  systan’s  ability  to  support  collaborative  learning.  We  built 
a virtual  electrical  circuit  that  contains  sev^al  batt^ies,  resistors,  and  a bulb  in  a virtual  physics  laboratory.  The  background 
infonnation  on  the  ri^t  of  tfie  virtual  world  describes  the  flowing  waters  and  teeming  crowds  analogies.  Users  arc  a^ed  to  predict 
and  compare  the  effects  of  the  circuit  using  tfie  analogies  when  batteries  and  resistors  are  placed  in  sales  and  parallel  on  the  circuit 
In  addition,  usas  can  use  the  virtual  VCR  (shown  on  the  left  side  of  the  virtual  circuit  in  Figure  1)  to  view  MPEG  movies  that 
demonstrate  tfie  tasks  that  the  e?q)erimarts  require  tfiem  to  perfonn.  The  virtual  VCR  plays  any  MPEG  movie  indicated  by  a URL 
input  by  the  user  into  a system  dialog  when  tfie  VCR  is  clicked  upon. 

In  ordCT  to  make  our  virtual  laboratory  more  argaging,  the  system  plays  soothing  spatialized  background  music.  Our  system  also 
provides  audio  feedback  when  usas  intoact  witfi  the  virtual  circuit  conponaits  and  witfi  the  billiard  balls. 

Users  can  manipulate  batteries  and  resistors  of  the  virtual  circuit  directly  by  dragging  and  dropping  the  components  using  the  mouse. 
Our  system  pronpts  users  for  the  orientation  of  the  componoits  in  the  circuit  by  using  a popi^  menu.  The  oriaitation  is  not  set 
automatically  because  orientations  of  conponaits  save  as  valid  learning  goals  as  well.  Batteries  that  are  placed  in  opposite 
directions  cancel  out  the  voltages  of  one  anotha.  On  the  otha  hand,  resistors  produce  the  same  resistance  even  when  they  are  placed 
in  opposing  directions.  Howeva,  when  eitha  batteries  or  resistors  are  placed  perpendicular  to  the  wire  of  the  circuit,  they  are 
disconnected  from  the  circuit  Hence,  they  do  not  contribute  to  the  ovoall  voltage  and  resistance  in  the  circuit 

While  some  users  are  performing  these  battoy  and  resistor  amangemats,  otha  usos  can  commat  on  their  progress.  Futhermore, 
users  can  collaborate  in  the  circuit  building  process.  Users  who  are  more  proficioit  can  help  others  by  sharing  knowledge  and 
understanding  with  the  weaka  users.  The  text  chat  freility  allows  participants  to  engage  in  collaborative  knowledge  building 
activities.  Consequently,  teaching  becomes  more  participative  and  distributed  across  users. 

The  second  simulation  in  this  virtual  world  investigates  irrpulsive  forces  using  colliding  billiard  balls.  VirtUOS  allows  usos  to 
modify  the  friction  and  restitution  between  billiard  balls  and  the  billiard  table  as  well  as  the  maximum  initial  force  applied  to  a billiard 
ball.  The  ease  with  which  users  can  change  these  avironmental  parametos  is  impossible  in  real  life.  For  exanple,  it  is  unlikely  to 
find  tables  and  billiard  balls  with  zero  frictional  force.  It  is  similarly  difficult  to  find  such  a setup  with  zero  restitution.  Moreova, 
unless  e?q)erimenters  are  expert  billiard  playos,  it  is  difficult  to  hit  billiard  balls  squarely  at  the  centa  of  gravity  all  the  time.  Althou^ 
hitting  billiard  balls  off  the  coita  of  gravity  can  be  aixjtha  modifiable  parameta,  uncontrolled  variations  of  such  parameters  are 
likely  to  inpede  usos’  investigations  of  simulations  that  they  choose  to  focus  on  (such  as  the  consovation  of  momentum). 

Users  can  vary  the  initial  force  acting  on  a billiard  ball  by  holding  onto  the  ball.  The  length  of  time  is  used  to  estimate  the  initial  force 
proportionately  with  respect  to  the  maximum  force.  Hence,  clicking  (and  holding)  onto  a ball  for  a longa  period  will  cause  the  ball 
to  have  a hi^a  initial  speed  corrpaied  to  clicking  (and  holding)  onto  a ball  for  a shorta  period 

We  intend  to  conduct  empirical  evaluations  on  users  in  orda  to  detemiine  the  usability  and  effectivaress  of  our  system.  These 
evaluations  will  include  the  time  taken  for  users  to  be  familiar  with  the  system,  the  amount  of  knowledge  retained  from  virtual  world 
learning  experiences,  and  the  willingness  of  usos  to  use  our  system  repeatedly. 


Technical  Innovations 

In  supporting  the  shift  fiom  navigation-oriated  to  intoaction-oriented  VR,  we  achieved  the  following: 

□ ^capsulation  of  object  behaviours  in  VRML.  Object  behaviours  are  created  separately  fiom  the  objects  that  use  the 
behaviours.  As  a result,  these  behaviours  can  be  shared  among  many  objects  at  the  same  time.  An  exarrple  of  shared 
behaviours  is  the  walk  animation.  Avatars  in  VirtUOS  walk  instead  of  float  in  virtual  worlds.  All  avatars  share  the  same  walk 
animation  behaviour. 

□ Usa-interface  support  for  object  intoactions.  VirtUOS  extoids  the  usa-interface  of  Cosmo  Player  2.1  by  adding  popup 
menus.  These  maus  are  di^layed  as  VRML  objects  roidered  by  Cosmo  Playa  in  virtual  worlds. 

□ VirtUOS  provides  database  access  for  VRML  objects.  VirtUOS  uses  the  database  to  store  and  retrieve  virtual  world  states. 
These  states  are  updated  as  users  intoact  with  virtual  worlds. 

□ VirtUOS  provides  network  support  for  VRML  objects.  The  syston  enables  VRML  evaits  to  be  propagated  to  cliaits  ova  the 
Internet  hence  allowing  otha  clients  to  reflect  changes  to  virtual  world  states. 

□ Concurrency  control  for  object  manipulations.  VirtUOS  uses  a database  to  provide  concurrency  control  so  that  only  one  usa 
can  update  a virtual  object  at  a time. 
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Conclusion 


K/>t(y05  siq^poits  collaborative  learning  Ihrougji  simulated  virtual  oivironments.  These  virtual  environments  allow  users  to  actively 
participate  in  coUaborative  activities.  Users  can  accelaate  their  learning  processes  by  engaging  in  mutual  tutoring  and  knowledge 
sharing  through  the  virtual  environmaits  that  provides. 

VirtUOS  extends  the  existing  usct  intofee  of  Cosmo  Player  so  that  users  can  interact  with  virtual  objects.  For  example,  VirtUOS 
displays  a popup  menu  that  can  be  used  to  control  the  orientation  of  virtual  objects.  Because  popup  menus  are  not  part  of  Cosmo 
Player's  virtual  world  rq^resaitation,  we  developed  tiiis  functionality. 

There  are  four  possible  future  enhancanents  to  our  work  First,  in  ordo*  to  increase  system  efficiency,  we  plan  to  partition  virtual 
worlds  so  that  only  visible  portions  of  the  worlds  are  iqxlated  on  every  client  computer.  Second,  it  is  desirable  to  exploit  users’ 
hearing  as  an  altaiative  perception  channel.  Consequently,  the  system  should  be  able  to  receive  audio  input  from  users  and 
propagate  that  to  other  usas.  Third,  VirtUOS  does  not  currently  indicate  which  user  is  moving  a virtual  object  Hence,  there  is  a 
detached  feeling  when  otho*  usas  see  non-living  objects  move  on  their  owa  The  systan  should  show  users  holding  or  pushing  the 
objects  that  the  users  are  moving.  Fourth,  the  system  should  allow  users  to  manipulate  virtual  world  objects  more  directly.  The 
current  implementation  of  our  systan  uses  apopip  menu  for  interactions  between  users  and  virtual  objects. 

In  summary,  our  work  deals  with  the  synthesis  of  technology  and  learning  by  using  desktop  VR  to  fk:ilitate  expaiaitially-grounded 
collaborative  learning.  Neither  component  rqDlaces  the  other,  rather,  the  two  complanent  each  otiiCT.  The  ongoing  r^id  evolution  of 
various  VRML  standards  such  as  DIS  (Brutzman,  1998),  HANIM,  and  External  Authoring  Intefece  (EAI),  means  that  VirtUOS 
has  to  be  maintained  and  updated  constantly  to  be  up-to-date  with  the  open  standards.  Fortunately,  the  highly  modular  design  of 
VirtUOS  makes  the  system  readily  modifiable  to  corifonn  to  new  standards.  Using  VirtUOS,  we  are  attempting  to  reach  out  to  a 
global  community  of  leamas  to  collaborate  in  the  exciting  new  world  of  VR-based  constnictivist  learning. 

References 

Brutzman,  D.  ( 1 998).  Distributed  Interactive  Simulatiai  DIS-Java-VRML  Working  Group.  httpyAvww.stl.rpsnavymiMsjava-vi^ 

Boger-Mdiall,  S..  BehavimstandO^ristnKtivistT^  http://www.cce.iihedu/"-siTTiehall/theo 
Fosnot,  C.  (1 996).  Constructivism:  Theory,  Perspectives,  and  Practice.  New  York  Teachers  College  Press. 

Gagnon,  G.  W.  Jr.,  & Collay,  M.  ( 1 996).  Teachers*  Perspectives  on  a Constnictivist  Learning  Design.  http:/Avww.prainbcw.oom/cld/cl^ 

Centner,  D,  & Centner,  D.  R.  (1983).  Flowing  waters  or  teeming  crowds:  Mental  models  of  electricity.  In  Centner,  D.  and  Stevens,  A L.  (ed). 
Mental  Modeb.  pp.  99- 1 29,  Lawrence  Eribaim 

Humphry,  D.  and  Topping  J.  (1949).  A SknIerIrUerrrmBate  Mechanics.  Lcaigmans,  Green  and  Co. 

Koschmarm,  T.,  Newman,  D,  Woodmfi;  E,  Pea,  R,  & Rowley,  P.  (1993).  Technology  and  Pedagogy  for  Coliabaative  Problem  Solving  as  a 
Context  foe  Learning:  Rqxst  on  a CSCW  *92  Workshop.  ACM SIGCHI BuMin,  25  (4),  pp.  57-60. 

Lim,  K.  P.  E (1997).  VR  for  Learning.  Naticnal  University  of  Singapore,  DepartmoTt  of  Infonnalion  Systems  and  Computer  ScietKe. 

McKenzie,  J.  (1997).  In  Defense  of  Texfoooks,  Lectures  and  Oflier  Ageing  Techixplogies.  From  Now  On:  The  Educational  Technology  Joianal,  6 
(8),  http://www.fit5ncwon.(3]^riT^97/defenseht^ 

Roehl,  B.  (1997).  Spedfication  fipr  a Standard  VRML  Humanoid  Version  1 .0.  http7/ece.uwaterioo.ca:80/'Ti-anirrr/^3ec.h^ 

Strommen,  E F.  (1992).  Constructivism,  Technology  and  the  Future  of  Qassroom  Learning  Bank  Street  Colley  of  Bducatioa 
http://www.iltoolunibiaedukl2/tivetext/docs^cOTStriK^ 


ERIC 


PATTERNS  OF  LEARNER-LEARNER  INTERACTION  IN  DISTANCE 

LEARNING  NETWORKS 


C.  Candace  Chou 

Interdisciplinary  PhD  Program  in  Communication  and  Information  Sciences 
University  of  Hawaii 
Honolulu,  Hawaii  96822 
chou@hawaii.edu 


Abstract:  This  study  examines  the  different  patterns  of  online  interaction  between  asynchronous 
communication  and  synchronous  communication  networks.  Discussion  transcripts  were  analyzed 
and  coded  using  Bale’s  Interaction  Process  Analysis  (IPA)  model  (revised  and  expanded  version). 

The  preliminary  findings  showed  significant  differences  in  the  interaction  between  social- 
emotional  and  task-oriented  contents.  There  was  a higher  percentage  of  social -emotional  oriented 
interaction  in  synchronous  communication.  Whereas  in  asynchronous  communication  mode,  a 
significantly  higher  percentage  of  online  interaction  were  task-oriented.  Furthermore, 
asynchronous  discussion  appeared  to  be  mostly  one-way  communication  and  synchronous 
discussion  showed  evidence  of  two-way  communication. 

INTRODUCTION 

Computer-mediated  communication  (CMC)  systems  have  become  an  integral  part  of  distance  learning. 
Distance  courses  are  conducted  over  CMC  systems  and  learners  interact  with  instructors  or  peers  through  computer 
conferencing.  Studies  on  interaction  over  CMC  networks  have  provided  educators  and  researchers  information  on 
learners  in  terms  of  both  intellectual  growth  and  social  development.  Research  findings  on  how  interaction 
contributes  to  intellectual  growth  such  as  knowledge  construction  (Gunawardena,  1997),  academic  achievements 
(Hartman  et  al,  1995),  and  critical  thinking  (Newman,  Webb,  & Cochrane,  1995)  indicates  that  cognitive 
development  can  be  achieved  over  distance  learning  networks.  Research  on  the  social  aspect  of  interaction  also 
furthers  the  understanding  of  learner  behaviors  and  dynamics  in  distance  education  (Johanson,  1996;  McDonald  & 
Gibson,  1998). 

The  majority  of  research  on  learner  interaction  was  conducted  over  asynchronous  computer  networks 
which  are  the  primary  media  for  distance  education  (Lewis  et  al.,  1999).  Few  studies  have  been  done  on  learner 
interactions  in  a synchronous  communication  network  and  even  fewer  research  projects  have  been  designed  to 
compare  both  synchronous  and  asynchronous  communication.  Factors  such  as  difficulty  in  coordinating  meeting 
time,  high  cost  in  good  quality  synchronous  communication  technology,  and  tool  stability  may  explain  the  under- 
utilization of  synchronous  CMC  systems.  Nevertheless,  with  the  improvement  in  CMC  technology  and  more 
affordable  tools  available,  synchronous  conferencing  systems  have  become  more  common  in  distance  learning 
environments. 

The  problems  of  the  previous  research  can  be  summarized  as  follow:  a),  lack  of  long-term  study:  most  of 
the  research  uses  previously  unacquainted  participants  and  short-lived  group;  b).  lack  of  study  on  synchronous 
interaction:  much  emphasis  has  been  placed  on  interaction  over  asynchronous  communication  networks;  and  c). 
lack  of  study  on  the  comparison  of  asynchronous  and  synchronous  interaction. 

Moore  (1989)  defines  three  types  of  interactions  in  distance  education:  learner-content  interaction, 
learner-instructor  interaction,  and  learner-learner  interaction.  A comprehensive  understanding  of  each  type  of 
interaction  can  help  educators  better  plan  learning  activities  and  make  use  of  the  CMC  technology  for  maximum 
effectiveness.  Most  of  the  research  on  online  interaction  draws  conclusions  from  studies  on  asynchronous 
communication  networks,  e.g.  electronic  mail,  listserv,  web-based  bulletin,  BBS,  etc.  Little  research  has  been 
conducted  over  synchronous  networks.  What  are  the  nature  and  patterns  of  synchronous  interactions?  To  what 
degree  can  learner  interactions  over  synchronous  learning  networks  contribute  to  learning?  It  is  the  purpose  of  this 
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proposed  study  to  examine  learner-learner  interactions  in  a collaborative  online  learning  environment.  The 
objectives  for  this  study  are: 

1.  To  identify  the  patterns  and  developments  of  learner  interactions  over  time,  especially  on  task-oriented  versus 
social-emotional  oriented  content; 

2.  to  determine  if  there  is  a difference  in  the  interactions  when  learners  assume  the  roles  of  moderators  and 
participants;  and 

3.  to  compare  learner-learner  interactions  between  synchronous  and  asynchronous  communication  networks, 

LITERATURE  REVIEW 

Learners’  abilities  to  interact  with  the  instructor,  the  peers,  and  the  content  can  affect  their  performance  in 
distance  learning.  Acker  and  McCain  (1993)  state  that  ’’interaction  is  central  to  the  social  expectations  of  education 
in  the  broadest  sense  and  is  in  itself  a primary  goal  of  the  larger  educational  process  and  that  feedback  between 
learner  and  teacher  is  necessary  for  education  to  develop  and  improve”  (p,  11),  Online  interactions  take  into 
consideration  the  characteristics  of  the  learners  as  well  as  the  communication  technology.  The  interactive  features 
of  the  current  CMC  systems,  such  as  two-way  video  and  instant  feedback,  have  provided  more  options  for  learner 
interactions.  Gunawardena  (1998)  interprets  interaction  as  “the  process  through  which  negotiation  of  meaning  and 
co-creation  of  knowledge  occurs  in  a constructivist  learning  environment”  (p,  141),  Wagner  (1998)  argues  that 
interaction  can  serve  as  a means  to  an  end  of  enhancing  learning  and  performance. 

The  design  of  the  learning  environments  influences  the  results  of  learner-learner  interactions, 
Scardamalia  et  al.  (1992)  found  significant  difference  in  the  verbal  scores  from  a Canadian  standardized 
achievement  test  between  elementary  students  who  used  Computer  Supported  Intentional  Learning  Environments 
(CSILE)  to  collaborate  with  teammates  and  those  who  used  CSILE  for  individual  study,  CSILE  is  a multimedia 
learning  environment  that  allows  students  to  collaboratively  contribute  to  one  another’s  learning  through  co- 
construction of  a knowledge  database.  As  suggested  by  Martens  et  al.  (1997),  interaction  is  ’’supported  by  the  user 
interface  that  enables  the  learner  to  access  and  manipulate  the  objects  in  the  knowledge  base”  (p,  46),  Good 
interface  design  provides  students  the  opportunities  to  learn  by  interaction  with  their  peers. 

One  should  not  equate  interaction  with  learning.  It  depends  on  whether  the  goal  of  an  interaction  task  or 
activity  is  for  the  purpose  of  enhancing  understanding  of  the  subject  matter  or  for  improving  interpersonal 
connections.  Learner  interactions  require  planning  and  structure  in  order  to  achieve  the  goal  of  active  learning, 
Rohfeld  and  Hiemstra  (1995)  suggest  tasks  such  as  debates,  guest  lecturers/discussants,  polling,  brainstorming,  or 
student-moderated  discussions  via  CMC  networks  to  increase  student  interactions  for  learning.  The  principles  of 
student-centered  discussion  accord  the  students  the  responsibilities  of  facilitating  online  conversations.  The  process 
serves  well  for  both  cognitive  and  affective  purposes.  On  the  one  hand,  the  process  stimulates  intellectual  growth 
and  enhances  organizational  skills;  on  the  other  hand,  it  improves  the  student  connection  at  the  interpersonal  level 
(Paulsen,  1995).  When  the  activities  and  tasks  become  an  integral  part  of  the  learning  process,  learner  interactions 
can  be  conducive  to  learning. 

Bales’  Interaction  Process  Analysis  (IPA)  (1950)  has  been  utilized  to  examine  small  group  interactions  in 
both  face-to-face  and  CMC  groups  (Rice  & Love,  1987;  Sorensen  & McCroskey,  1977),  IPA  consists  of  twelve 
categories  which  fall  into  two  main  areas:  social-emotional  oriented  and  task-oriented  contents,  IPA  was  first 
developed  in  1950  as  the  result  of  studies  of  group  processes,  e,g,  who  did  what,  who  spoke  to  whom,  and  how 
interactions  developed  and  changed  (Bales,  1 950),  Bales’  IPA  describes  well  the  process  of  interaction  and  the 
dynamic  in  small  groups.  The  twelve  categories  were  modified  for  this  study  in  order  to  better  described  the  nature 
of  online  interactions.  More  details  will  be  described  in  the  following  section. 
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To  further  the  understanding  of  online  learner-learner  interactions,  the  research  questions  will  address 
learner  interactions  in  the  following  areas:  learning  tasks,  learning  environments,  learner  characteristics,  and  the 
relationship  with  learning.  The  main  questions  are  listed  as  follows: 

A.  Learner-learner  interactions  in  social-emotional  oriented  contents  versus  task-oriented  contents:  How  do 
online  interactions  develop  over  time  in  terms  of  social-emotional  versus  task-oriented  content  as  outlined  in  Bales' 
Interaction  Process  Analysis?  Is  this  pattern  of  interaction  different  between  asynchronous  communication  mode 
and  synchronous  communication  mode? 

B.  Learning  task:  Will  role-taking,  such  as  assuming  the  role  of  a moderator  or  a participant,  make  a difference  in 
online  interactions? 

C.  Learner  characteristics:  Do  learner  characteristics  such  as  gender,  attitude  toward  computer  networks, 
experience  in  using  computer,  confidence  in  computer  skills,  affect  the  way  they  interact  online? 

D.  Learning  environments:  What  are  the  similarities  and  differences  in  learner  interactions  via  synchronous  and 
asynchronous  networks? 

METHODOLOGY 

This  study  used  mainly  qualitative  methodology  with  the  support  of  statistical  analysis  program.  Content 
analysis  was  employed  to  examine  learner-learner  interactions  and  learning.  Bales’  Interaction  Process  Analysis 
(IPA)  with  minor  modifications  was  used  as  the  basis  of  content  analysis  for  learner  interactions.  The  procedure 
was  to  systematically  code  student's  weekly  postings  and  online  seminar  transcripts  in  order  to  detect  the 
interaction  patterns. 

The  research  was  carried  out  by  surveying  distance  learners  and  analyzing  student  written  data  in  a ten- 
week  writing-intensive  online  course  titled  "Theory  and  Application  of  Computer-Mediated  Communication."  This 
course  was  delivered  via  a courseware  called  WebCT  at  the  University  of  Hawaii  in  1998.  The  data  include  weekly 
postings  in  the  bulletin  board  (asynchronous  data)  and  transcripts  from  a weekly  seminar  (synchronous  data).  In 
the  weekly  seminar,  students  met  in  a chat  room  in  real  time  to  discuss  course-related  topics  from  any  locations 
that  they  could  access  a computer.  Students  were  divided  into  five  three-member  small  groups.  Each  group  took 
turns  to  moderate  one  online  seminar  throughout  the  whole  semester.  Two  researchers  who  specialize  in 
instructional  design  worked  on  the  coding  of  the  data.  The  software  program  QSR  NUD'^^IST  was  used  for  the 
coding  of  online  transcripts.  The  unit  of  analysis  was  the  sentence.  A total  of  5,015  text  units  (sentences)  from  the 
random  sample  transcripts  were  coded.  The  phi  coefficient  for  intercoder  reliability  was  .80. 

Bale’s  IPA  was  expanded  from  12  categories  into  a total  of  16  categories  as  shown  in  table  1.  The  original 
12  categories  can  be  divided  into  two  main  areas  and  four  sub-groups.  The  two  main  areas  are  social-emotional 
contents  and  task-oriented  contents.  The  social-emotional  area  consists  of  two  groups:  positive  reactions  and 
negative  reactions.  The  task-oriented  area  consists  of  two  groups  that  are  attempted  answers  and  questions.  These 
areas  represent  the  domain  of  interactions  in  small  group  discussions.  The  categories  may  not  be  comprehensive 
enough  to  describe  the  patterns  of  online  interactions  but  the  overall  scheme  does  address  most  online  interaction 
patterns.  In  his  study  on  social-emotion  interaction  over  computer-mediated  communication  networks,  Rice  (1987) 
expanded  category  6 and  7 into  two  sub-categories.  Category  6 on  “gives  orientation,  information,  repeats, 
clarifies,  confirms”  was  divided  into  “gives  personal  information”  (socioemotional)  and  “gives  professional 
information”  (task).  Category  7 was  divided  into  “asks  for  professional  information”  (task)  and  “asks  for  personal 
information”  (socioemotional).  For  this  study,  category  6 and  7 were  furthered  divided  into  three  sub-categories: 
“gives/asks  topic-related  information”  (task),  “gives/asks  personal  information”  (socioemotinoal),  “gives/asks 
technical  information”  (task).  After  the  initial  round  of  coding,  the  researcher  found  that  the  original  category  6 
and  category  7 were  too  broad  to  reflect  the  actual  online  interaction  patterns  from  the  samples  used  for  this  study. 
Technical  questions,  topic-specific  discussions,  and  personal  information  exchanges  were  frequently  seen  in  the 
synchronous  discussions  and  yet  they  were  missing  from  the  original  IPA  model.  The  researcher  decided  to  include 
these  new  categories  in  order  to  better  reflect  the  patterns  of  online  interactions. 
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Coding  online  transcripts  is  a time-consuming  process.  In  order  to  meet  the  submission  deadline  of  the 
conference  proceedings,  only  the  data  for  the  first  research  question  is  available  for  this  report.  The  author  will 
report  the  results  of  the  data  analysis  for  the  rest  of  the  research  questions  during  the  EDMEDIA2000  conference. 
The  revised  and  expanded  version  of  Bale’s  IPA  is  shown  in  table  1.  There  were  significant  differences  in  online 
interactions  between  asynchronous  and  synchronous  communication  modes  using  analysis  of  variances.  In  terms  of 
each  individual  category,  the  differences  were  significant  in  the  following  categories:  category  4 (gives  suggestion, 
p < .01),  category  5 (gives  opinion,  p < ,01),  category  7,2  (asks  topic-related  information,  p < ,05),  category  7,3 
(asks  personal  information,  p = .01),  and  category  1 1 (show  tension,  p < .05),  In  terms  of  the  four  sub-areas,  i,e, 
positive  reactions  (p  = .05),  negative  reactions  (p  < ,05),  attempted  answers  (p  < .01),  and  questions  (p  < ,01),  the 
differences  were  significant.  In  terms  of  the  two  main  areas  of  interaction,  both  social-emotional  oriented  contents 
(p  < .05)  and  task-oriented  content  (p  = .01)  indicate  the  significant  difference  between  the  synchronous  and 
synchronous  communication  networks  (table  2). 

A closer  look  in  category  4,  5,  7,2,  7,3,  and  11  will  provide  a better  understanding  in  the  different 
interaction  patterns  between  the  two  communication  modes.  First,  there  was  virtually  no  interaction  in  category  4 
(giving  suggestion  or  direction)  in  asynchronous  discussion  mode.  The  mean  sentence  in  category  4 is  0 for 
asynchronous  discussion  and  9.36  (SD:  7.49)  for  synchronous  discussion.  Category  9 (asking  suggestions)  is  the 
counterpart  of  category  4.  The  mean  sentence  for  category  9 is  0 too.  It  is  understandable  that  in  asynchronous 
communication  most  interactions  focus  on  task-specific  discussion  and  less  on  social-emotional  oriented  contents 
since  there  was  a lack  of  immediacy.  Most  students  either  internalized  their  attempts  to  ask  for  suggestions  or 
email  the  instructor  for  advice.  The  time  lag  in  asynchronous  mode  may  have  prevented  students  from  asking  or 
providing  suggestions.  Interestingly,  the  mean  sentence  in  category  5 (giving  opinions)  is  247,33  (SD:  129,1)  for 
asynchronous  mode  and  62  (SD:  42.05)  in  synchronous  mode.  It  is  a clear  indication  that  in  asynchronous  mode, 
participants  devoted  most  of  the  discussion  to  task-related  discussion  and  less  on  social -emotional  interaction. 
Although,  there  was  no  interaction  in  giving/  asking  suggestion  (category  4 & 9),  the  interaction  in  giving/asking 
opinion  (category  5 & 8)  was  not  scarce  in  asynchronous  mode. 

Whereas  in  the  synchronous  communication  mode,  student  felt  it  more  imperative  to  ask  and  receive 
instruction  or  direction  in  order  to  participate  fully.  Synchronous  communication  mode  also  made  it  easier  to 
provide  immediate  feedback  to  information  seekers.  The  high  standard  deviation  in  synchronous  interaction  also 
reveals  the  unequal  participation  in  online  discussion.  Some  students  were  active  in  engaging  in  discussions  and 
some  students  reserved  most  of  their  opinions  until  they  were  asked  to  say  something.  The  researcher  observed  that 
when  the  students  were  divided  into  three-member  small  groups,  the  participation  was  more  even  among  members 
of  a small  group.  In  addition,  there  were  more  social-emotional  oriented  discussions  in  synchronous  mode  that  is 
indicated  in  the  difference  in  category  7.3  and  11.  In  synchronous  mode,  participants  asked  more  personal 
questions  and  revealed  their  frustration  or  needs  for  help  without  hesitation.  Personal  questions  such  as  one’s 
occupation,  schooling  history,  professional  training  were  given  more  “air  time”  in  synchronous  discussion. 


Table  1:  Bale’s  IPA  (revised  and  expanded) 


Code  1 Category 

P value 

Social-emotional  Area:  Positive  Reactions 

= .05* 

I 

Shows  solidarity,  raises  other's  status,  gives  help,  reward 

= .013 

2 

Shows  tension  release,  jokes,  laughs,  shows  satisfaction 

= .015 

3 

Agrees,  shows  passive  acceptance,  understands,  concurs,  complies 

= .047 

Task  Area:  Attempted  Answers 

<.01  * 

4 

Gives  suggestion,  direction,  implying  autonomy  for  other 

<.01  * 

5 

Gives  opinion,  evaluation,  repeats,  clarifies,  confirms 

<.01  * 

6 

Gives  orientation,  information,  repeats,  clarifies,  confirms 

6.1 

Gives  personal  information  (social-emotional) 

= .121 

6.2 

Gives  topic-related  information 

= .248 

6.3 

Give  technical  information 

= .327 

Task  Area:  Questions 

<0.1* 

7 

Asks  for  orientation,  information,  repetition,  confirmation 

7.1 

Asks  technical  information 

= .30 
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7.2 

Asks  topic-related  information 

<.05* 

7.3 

Asks  personal  information  (social-emotional) 

= .01* 

8 

Asks  for  opinion,  evaluation,  analysis,  expression  of  feeling 

= .07 

9 

Asks  for  suggestion,  direction,  possible  ways  of  action 

= .07 

Social-emotional  Area;  Negative  Reactions 

< .0.5* 

10 

Disagrees,  shows  passive  rejection,  formality,  withholds  help 

= .44 

11 

Shows  tension,  asks  for  help,  withdraws  out  of  field 

<.05 

12 

shows  antagonism,  deflates  other's  status,  defends  or  asserts  self 

= .48 

In  comparison  of  the  two  main  areas  in  social -emotional  oriented  content  and  task-oriented  content  as  shown  in  table 
2,  the  mean  sentence  for  social-emotional  content  in  synchronous  mode  (68.18)  is  significantly  higher  than  that  in  asynchronous 
mode  (28.67).  Task-oriented  content  consists  of  category  4,  5,  6.2,  6.3,  7.1,  7.2,  8,  and  9.  Social-emotional  content  consists  of 
category  1,  2,  3,  6.1,  7.3,  10,  11,  and  12.  The  breakdown  in  category  1,  2,  and  3 shows  that  the  mean  sentence  in  synchronous 
mode  (40.73,  5.36,  9.64  respectively)  is  much  higher  than  those  in  asynchronous  mode  (7.5,  1.0,  6.5).  In  synchronous 
communication  mode,  there  were  more  interactions  in  greeting,  providing  help,  joking,  and  showing  agreement.  Whereas  the 
mean  sentence  of  task-oriented  content  in  asynchronous  mode  (307.33)  in  significantly  higher  than  that  in  synchronous  mode 
(139.18).  Student  tended  to  give  their  opinions  without  asking  other’s  opinions.  It  was  done  mostly  through  one-way 
communication  in  asynchronous  discussion.  The  mean  sentence  of  category  5 is  significantly  higher  in  asynchronous  mode 
(247.33)  than  in  synchronous  mode  (62.00).  The  interaction  in  task-oriented  content  indicated  that  in  synchronous  mode  most 
of  the  discussions  were  two-way  communication  through  the  acts  of  providing  and  asking  opinions  on  course-related  subjects. 

Table  2:  ANOVA  table  for  Social-emotional  oriented  content  vs.  task-oriented  content 


Asyn.  mean 

Asyn.  SD 

Syn.  mean 

Syn.  SD 

t-value 

F-value 

p-value 

SE  content 

28.67 

20.81 

68.18 

39.95 

-2.24 

5.016 

0.041* 

TASK  content 

307.33 

159.37 

139.18 

81.93 

2.91 

8.48 

0.01** 

CONCLUSIONS 

Studying  online  interaction  can  further  the  understanding  of  how  people  communicate  over  the  networks. 
The  main  difference  between  this  study  and  other  studies  is  the  focus  on  the  comparison  of  synchronous  and 
asynchronous  communication  networks.  The  preliminary  findings  highlight  the  significant  differences  in  both 
social-emotional  oriented  interaction  and  task-oriented  interaction  between  the  two  communication  modes.  These 
differences  indicate  the  very  different  nature  of  online  interaction  in  both  modes.  Educators  may  want  to  design 
activities  that  will  ensure  a focused  online  discussion  without  diverting  to  only  interpersonal  exchanges  in 
synchronous  communication  mode.  Whereas  in  asynchronous  mode,  activity  design  to  enhance  interpersonal 
connection  may  facilitate  more  two-way  communication. 

Further  analysis  on  the  effects  of  learner  characteristics  and  learning  tasks  on  interactions  and  the 
relationship  between  interaction  and  learning  will  be  conducted  in  the  near  future.  The  research  findings  will 
contribute  to  the  knowledge  base  of  learner  interaction  in  both  synchronous  and  asynchronous  networks. 
Educators,  researchers,  and  instructional  designers  working  on  distance  education  can  all  make  use  of  the 
knowledge  to  maximize  the  effectiveness  of  distance  learning. 
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Abstract:  An  Electronic  Performance  Support  System  (EPSS),  featuring  an  extensive 
knowledge  base  of  learning  science  and  theory,  is  discussed  and  demonstrated.  The  EPSS 
is  designed  for  use  by  instructional  designer-developers  of  computer-mediated  instruction 
(CMI)  to  help  in  the  application  of  contemporary  learning  science  in  the  design  and 
development  of  instructional  materials.  The  EPSS  addresses  the  problems  faced  by 
practicing  designer-developers  in  trying  to  keep  up  with  the  rapidly  developing  literature 
of  learning  science,  and  the  heavy  cognitive  loading  inherent  in  the  CMI  design  and 
development  process.  Implemented  as  a HTML  document,  the  EPSS  features  concise, 
prescriptive,  practical  guidelines  in  an  operationalized  checklist  format.  Hyperlinks  lead 
to  expanded  checklists,  explanations,  examples,  and  Internet  resources. 


Introduction 

Instructional  designers  and  developers  often  do  not  apply  contemporary  learning  science  in  the 
creation  of  computer-mediated  instruction  (CMI)  (Brown  1997;  Cooper  1997).  Instead,  they  rely  primarily 
on  craft  and  heuristics  for  guidance  (Clark  & Estes  1998).  The  resulting  materials,  without  the  benefit  of  a 
science  basis,  are  less  effective  than  would  otherwise  be  possible  (Fletcher  1996;  Chadwick  1997; 
Luterbach  1997;  Radwan  1997;  Winn  1997;  Schmidt  1997). 

Applying  learning  science  and  theory  in  CMI  design  and  development  is  a challenge  for  several 
reasons.  First,  many  designer-developers  have  neither  formal  training  in  learning  science  nor  the  time  to 
learn  and  stay  current  with  the  rapidly  developing  learning  science  literature.  Also,  the  application  of 
learning  science  dramatically  increases  the  cognitive  loading  of  the  design  task;  using  heuristics  is  quicker 
and  easier.  Finally,  popular  authoring  tools  offer  no  support  for  the  application  of  leaning  science,  nor  are 
practical  reference  tools  available  (Brown  1997).  The  EPSS  developed  in  this  project  is  an  attempt  to  help 
overcome  some  of  these  challenges. 

To  help  designer-developers  in  applying  learning  science  and  theory  to  CMI  materials,  this  project 
developed  a prototype  Electronic  Performance  Support  System  (EPSS)  for  use  as  a real-time  reference  tool. 
The  EPSS  provides  cognitive  support  and  makes  available  for  real-time  reference  a large  amount  of 
learning  science  information  in  the  form  of  practical  guidelines.  This  type  of  cognitive  support  can  enable 
non-experts  to  perform  their  work  tasks  in  a manner  approaching  the  level  of  expertise  demonstrated  by 
expert  performers  (Gerry  1 995). 

The  underlying  knowledge  base  for  this  project  distills  a broad  sampling  of  information  from 
cognitive  constructivism  science  and  theory,  and  uses  neural  science  research  data  to  support  and  embellish 
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the  cognitive  materials.  Access  is  provided  by  three  indexes:  an  instructional  process  index,  a learning 
theory  taxonomy  index,  and  an  alphabetical  index.  Knowledge  base  contents  are  expressed  as  short, 
prescriptive  guidelines  in  an  operationalized  checklist  format.  Implemented  in  HTML,  the  EPSS  features  a 
rich  network  of  hyperlinks  to  access  increasingly  detailed  levels  of  information,  explanations,  and 
examples,  including  numerous  links  to  Internet  resources. 


Project  Background 

The  body  of  science  and  theory  selected  for  use  in  this  project  includes  cognitive  science  and 
learning  theory  and  the  neural  sciences.  The  EPSS  is  based  primarily  on  cognitive  science  literature  but 
draws  from  the  neural  science  literature  to  support  and,  in  some  cases,  to  extend  the  cognitively  based 
information.  The  two  fields  of  cognitive  science  and  neurological  science  have  historically  developed 
separately,  but  today  they  are  rapidly  merging  to  provide  a combined  physical  and  conceptual 
understanding  of  how  human  senses,  perception,  memory,  and  recall  work  as  a system  for  learning. 

The  rationale  for  using  the  cognitive  and  neural  sciences  as  the  basis  for  this  project  is  as  follows: 

1.  Cognitive  constructionism  was  selected  as  the  learning  theory  basis  for  this  project  because:  (a) 
it  comprises  the  greatest  amount  of  contemporary  work  in  progress,  (b)  it  is  based  on  scientifically 
designed  experiments  involving  objective  observation,  controls,  and  repeatability,  (c)  it  subsumes  a great 
deal  of  work  accomplished  over  the  past  50  or  60  years,  and  (d)  it  is  supportable  from  the  perspectives  of 
many  contemporary  sciences  including  the  neural  sciences  (Collins  & Smith  1998;  Osherson  & Lasnik 
1990;  Posner  1989;  Gardner  1985). 

2.  The  learning  models  and  theories  of  John  Anderson  (Anderson  & Lebiere  1998),  Jerome  van 
Merrienboer  (1998),  Robert  Gagne  (Gagne  & Medsker  1996),  and  Ruth  Clark  (1998)  are  used  to  represent 
contemporary  cognitive  learning  theory  because:  (a)  they  are  presented  in  newly  published  literature  and 
represent  the  most  recent  thinking  on  the  subject,  (b)  they  are  firmly  based  in  scientifically  designed 
research,  (c)  collectively  they  address  a broad  range  of  considerations  within  the  human  learning  domain, 
and  (d)  collectively  they  comprise  a complementary  set  of  perspectives  ranging  from  a broad  treatment  of 
human  cognition  in  John  Anderson's  theory  to  specific  instructional  design  features  in  Ruth  Clark's  work, 

3.  Neural  science  data  were  added  to  the  above  because:  (a)  they  provide  insight  into  sensory  and 
perceptual  processing  events  that  are  not  addressed  in  cognitive  theory,  (b)  they  tend  to  affirm  and 
complement  cognitive  theory,  and  (c)  they  point  the  direction  for  future  cognitive  research. 


EPSS  Design  Considerations 

Guidelines  in  the  EPSS  are  provided  in  abbreviated,  checklist  format  with  prescriptive  statements 
pertaining  directly  to  design  and  development  decisions.  For  experienced  and  knowledgeable  professionals, 
the  top-level  indexes  will  provide  sufficient  support  to  assure  that  important  considerations  are  not 
overlooked.  For  users  who  need  additional  support  second-  and  third-tier  indexes,  definitions,  explanations, 
and  additional  background  materials  are  available  by  hyperlink  connectivity. 

Several  means  are  provided  to  access  information  in  the  knowledge  base  including  a process 
oriented  index,  a learning  theory  taxonomy  index,  and  an  alphabetical  index.  The  process-oriented  index  is 
based  on  Gagne's  9 events  of  instruction  (Gagne  & Medsker,  1 998)  with  one  additional  event  added  for  this 
project.  Each  of  the  events  is  developed  by  using  material  from  several  sources.  As  an  example,  the  first 
event,  "Gain  Learner's  Attention,"  lists  5 categories  of  attention-gaining  measures  taken  from  both 
cognitive  materials,  especially  Clark  (1998),  and  neural  data  (Gazzaniga,  Ivry,  & Mangun  1998).  The 
second  event,  "State  the  Learning  Objectives,"  combines  Mager's  objective  oriented  instruction  approach 
(Mager  1984)  with  Gagne’s  knowledge  types  (Gagne  & Medsker,  1998)  and  Bloom's  taxonomy  of 
knowledge  levels  (Bloom,  1989).  The  third  event,  "Help  the  Learner  to  Recall  (and  Use)  Prior 
Knowledge,"  is  based  largely  on  Gagne's  material  as  supplemented  by  information  from  Ruth  Clark's  work 
on  how  to  encourage  recall  and  also  neural  data  on  why  it  is  important  to  do  so.  The  forth  event,  "Present 
the  Instructional  Content,"  draws  heavily  on  both  Gagne  and  Clark,  but  includes  working  memory  data 
(Chandler  & Sweller  1991;  Mousavi,  Low  & Sweller  1995;  Cooper  1997),  and  information  from  neural 
science  as  well.  The  remaining  events,  provide  learning  guidance,  elicit  performance,  assess  performance. 
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reinforce  retention,  and  reinforce  transfer  to  the  workplace,  each  draw  on  Gagne  and  Clark  as 
supplemented  by  the  other  sources  mentioned. 

The  learning  theory  taxonomy  index  is  based  in  part  on  the  sequence  of  events  that  occurs  in 
processing  sensory  inputs  into  learned  information:  sensing  and  perceiving,  working  memory  (WM),  long 
term  memory  (LTM),  and  transfer  to  the  workplace.  Additional  categories  include  the  following:  types  and 
levels  of  knowledge,  supporting  cognitive  processes,  and  individual  learner  differences. 


Implementation  of  the  SPSS 

The  EPSS  knowledge  base  is  developed  in  HTML  code  to  facilitate  the  use  of  many  useful 
features  available  in  this  format  such  as  hyperlinking.  Also,  the  HTML  format  is  usable  in  any  standard 
web  browser,  with  which  all  designer-developers  are  well  familiar.  Using  a hyperlink  format,  additional 
explanatory  information  is  readily  available  for  designer-developers  who  need  increased  levels  of  support. 
Second-  and  third-tier  data  present  expanded  checklists  and  discussions  including  explanations  and 
examples  to  support  the  checklists.  Embedded  in  the  expanded  discussion  are  hyperlinks  to  sources  of 
expanded  presentations  of  the  research  and  theoiy,  including  links  to  sources  on  the  Internet  when 
appropriate.  This  multi-level  approach  takes  after  Gerry's  description  of  an  effective  EPSS  (1995). 


Example  materials  from  the  EPSS 

The  following  excerpt  from  the  instructional  process  index  shows  how  hyperlinks  are  used  to 
access  increasingly  detailed  levels  of  guideline  checklists.  The  arrow  indicates  the  link  selected  by  a user  to 
access  the  next  index  list  shown.  The  top-level  index  is  based  on  Gagne's  nine  instructional  events,  with  the 
addition  of  a tenth  step  borrowed  from  Clark  (1998),  to  reinforce  the  transfer  new  knowledge  to  the 
workplace. 


• Instructional  Process  Index 

• Gain  Learner's  Attention  ===>  (hyperlink  to  2"^^  tier) 

• State  Learning  Objectives 

• Help  Learner  to  Recall  (and  Use)  Prior  Knowledge 

• Present  Instructional  Content 

• Provide  Learning  Guidance 

• Elicit  Performance 

• Provide  Feedback 

• Assess  Performance 

• Reinforce  Retention 

• Reinforce  Transfer  to  the  Workplace 

The  second-tier  index,  below,  is  hyperlinked  from  "Gain  Learner's  Attention,"  above,  and  expands  on 
the  concept  by  providing  several  specific  ways  to  gain  attention.  Each  of  the  methods  listed  should  be 
considered. 


• Gain  Learner's  Attention 

• Change  stimuli  suddenly  - to  gain  involuntary  sensory  attention.  ===>  (hyperlink  to  3^ 
tier) 

• Change  topic,  perspective,  or  approach  - to  gain  voluntary  cognitive  attention. 

• Present  the  logical  importance  of  the  topic  - to  gain  intellectual  attention. 

• Present  an  emotionally  impact  full  appeal  - to  gain  motivated  attention. 

• Regain  and  refocus  the  learner's  attention  at  each  instructional  step. 

A third-tier  index  in  linked  from  "Change  stimuli  suddenly,"  above,  and  expands  further  on  the 
concept  by  providing  some  specific  ideas  for  implementing  the  guideline. 

• Change  stimulus  suddenly  - to  gain  involuntary  sensory  attention. 

• Change  the  screen  contents  or  appearance  where  attention  should  be  focused. 

• Momentarily  flash  a symbol,  picture,  word,  or  other  visual  stimulus. 
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• Use  a momentary  sound  effect  as  a signal  for  the  learner  to  be  observant. 

• Use  a pointer  that  moves  to,  or  flashes  near,  the  item  that  should  be  attended  to. 

• Use  stimulus  changes  to  draw  attention  to  content,  not  distract  away  from  It. 

• Stimulus  changes  that  support  content  can  Increase  continuity  but  do  not  necessarily 
Increase  learning. 

=>  (hyperlink  to  explanations  and  examples) 

An  additional  level  of  detail,  in  the  form  of  explanations  and  examples,  is  available  for  many 
sections,  and  hyperlinks  to  Internet  resources  are  provided  for  many  topics  as  well. 


Usability  Evaluation  and  Project  Conclusions 

The  EPSS  prototype  was  field  tested  by  a number  of  graduate  students  in  instructional  technology 
and  professional  instructional  designer-developers  of  CMI  materials.  They  responded  to  the  following 
questions,  which  are  patterned  after  Gloria  Gerry’s  criteria  for  EPSS  design  (Gerry  1995).  The  purpose  of 
the  evaluation  was  to  assess  the  perceived  usefulness  and  usability  of  the  EPSS  tool: 

• Aids  ID  task  goal  establishment? 

• Structures  work  process  and  progression  through  tasks? 

• Contains  embedded  knowledge  in  the  interface  and  support  resources? 

• Uses  language  that  capitalizes  on  users'  prior  learning? 

• Is  compatible  with  the  natural  ID  work  situation? 

• Provides  alternative  views  of  the  document  interface  and  resources? 

• Provides  layers  to  accommodate  various  user  knowledge  levels? 

• Provides  access  to  underlying  logic? 

• Provides  alternative  knowledge  search  and  navigation  mechanisms? 

• Allows  customization? 

• Provides  obvious  options,  next  steps,  and  resources? 

• Employs  consistent  use  of  visual  conventions,  language,  visual  navigation,  and  other  system 
behavior? 

The  usability  and  usefulness  of  the  prototype  EPSS  was  rated  at  about  80%  - a mean  rating  of  4 on 
a Likert  scale  of  5 possible  points.  This  result  is  interpreted  as  representing  fairly  strong  support  for  this 
EPSS  concept,  especially  since  it  was  tested  only  in  a prototype  configuration.  Overall  conclusions  from 
the  project  are  as  follows:  (1)  learning  science  literature  contains  a large  amount  of  practical,  usable  data, 
(2)  such  data  can  be  expressed  in  abbreviated,  operationalized  terms,  and  (3)  field  testing  showed  the  EPSS 
document  to  be  both  useful  and  useable  in  presenting  learning  science  knowledge  for  CMI  designer- 
developers  to  use. 


Future  Directions 

The  current  EPSS  project  provides  a conceptual  starting  point  for  future  development  in  several 
directions.  Three  principle  areas  being  considered  for  future  development  are:  (1)  inclusion  of  additional 
relevant  learning  science  information,  (2)  addition  of  material  on  accommodating  to  individual  learner’s 
capabilities  and  needs,  and  (3)  providing  automated  links  to  instructional  authoring  tools. 

The  current  project  knowledge  base  was  developed  on  a framework  developed  from  cognitive 
learning  theory  literature.  This  procedure  tended  to  focus  the  literature  search  on  subjects  addressed  in  this 
literature.  Late  in  the  project,  however,  neural  science  data  pointed  to  the  important  effects  of  stress  and 
emotion  on  the  learning  process  that  are  not  developed  in  the  cognitive  learning  materials.  Stress  and 
emotion  appear  to  be  of  such  importance  to  the  learning  process  that  their  inclusion  is  seen  as  essential  in 
the  future  development  of  this  EPSS. 

Additionally,  the  neurological  basis  for  individual  differences  among  learners  was  explored  in 
sufficient  depth  to  understand  clearly  that  the  use  of  computer  capabilities  to  make  individualized 
accommodations  is  the  way  for  CMI  to  bring  a real  revolution  in  training  and  education.  Differences 
among  learners  are  not  small  or  subtle,  they  are  large  and  have  a significant  impact  on  how  each  person 
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processes  information.  The  inclusion  of  data  on  accommodating  to  individual  learners  will  be  an  important 
addition  to  the  present  project. 

Finally,  this  EPSS  will  achieve  a great  deal  of  additional  usefulness  when  it  is  linked  with  popular 
instructional  design  authoring  applications  so  that  the  learning  science  embodied  in  its  knowledge  base  can 
more  easily  be  included  in  the  development  of  CMI  materials  of  the  future. 


References 

Anderson,  J.  R.  & Lebiere,  C.  (1998).  The  atomic  components  of  thought.  Mahwah:  Lawrence  Erlbum 
Associates,  Publishers. 

Brown,  A.  (1997).  “Designing  for  learning:  What  are  the  essential  features  of  an  effective  online  course?” 
Australian  Journal  of  Educational  Technology,  13  (2),  115-126.  (Available  at: 
http://cleo.murdoch.edu.au/ajet/aJet  1 3/su97p  1 1 5.html) 

Clark,  R.C.,  (1998).  Building  Expertise:  cognitive  methods  for  traiing  and  performance  improvement. 
International  Society  for  Performance  Improvement:  Washington,  DC. 

Clark,  R.  E.  & Estes,  F.(1998).  'Technology  or  craft:  what  are  we  doing?"  Educational  Technology 
Sep/Oct 

Clark,  R.  E.  (1983).  Reconsidering  research  on  learning  from  media.  Review  of  Educational  Research, 
53(4\  445-459. 

A.  Collins,  A.  & Smith,  E.  E.  (1988).  Readings  in  cognitive  science:  a perspective  from  psychology  and 
artificial  intelligence.  San  Mateo:  Morgan  Kaufmann. 

Cooper,  G.  (1997).  Research  into  Cognitive  Load  Theory  and  Instructional  Design  at  UNSW.  Unpublished 
manuscript. 

Chadwick,  D.  K.  H.  (1997).  Computer-assisted  instruction  in  secondary  mathematics  classrooms:  A meta- 
analysis. (Doctoral  dissertation,  Drake  University,  1997).  Dissertation  Abstracts  International,  58  (09), 
A3478. 

Chandler,  P.,  & Sweller,  J.  (1991).  Cognitive  load  theory  and  the  format  of  instruction.  Cognition  and 
Instruction, 8,  293-332. 

Chandler,  P.,  & Sweller,  J.  (1992).  The  split  attention  effect  as  a factor  in  the  design  of  instruction.  British 
Journal  of  Education  Psychology,  62, 

233-246. 

Chandler,  P.,  & Sweller,  J.  (1996).  Cognitive  load  while  learning  to  use  a computer  program.  Applied 
Cognitive  Psychology,  10,  151  - 170. 

Fletcher,  J.  D.  (1996)  "Does  This  Stuff  Work?  Some  Findings  from  Applications  of  Technology  to 
Education  and  Training"  Proceedings  of  Conference  on  Teacher  Education  and  the  Use  of  Technology 
Based  Learning  Systems.  Warrenton,  VA:  Society  for  Applied  Learning  Technology  (Available  at: 
http://www.ott.navy.mil/l_2/salt.htm) 

Gardner,  H.  (1985).  The  mind's  new  science.  New  York:  Basic  Books. 

Gagne,  R.  M.,  and  Medsker,  K.  L.  (1996).  The  conditions  of  learning:  Training  Applications.  New  York: 
Harcourt  Brace  College  Publishers. 


221 


Gazzaniga,  M.  S.,  Ivry,  R.  B.,  & Mangun,  G.  R.  (1998).  Cognitive  Neurocience:  the  biology  of  the  mind. 
New  York:  W.W.  Norton  & Company. 

Gerry,  G.  (1995)  "Performance  centered  software  design."  Performance  Improvement  Quarterly. 

Lutterbach,  K.  J.  (1997).  Refining  instructional  design  theories:  A comparison  of  constructivist 
instructional  strategies  for  teaching  computer  mastery  (undergraduates,  cognitive  psychology).  (Doctoral 
dissertaion,  Indiana  University,  1997).  Dissertation  Abstracts  IntemationaK  58  (08),  A2987. 

Mousavi,  S.  Y.,  Low,  R.,  and  Sweller,  J.  (1995).  “Reducing  cognitive  load  by  mixing  auditory  and  visual 
presentation  modes.”  Journal  of  Educational  Psychology.  87(2),  319-334. 

Osherson,  D.  N.  & Lasnik,  H.  (Eds.).  (1990).  Language:  an  invitation  to  cognitive  science  (Vol  1). 
Cambridge:  MIT  Press. 

Posner,  M.  I.  (Ed.).  Foundations  of  cognitive  science.  Cambridge:  MIT  Press. 

Rad  wan,  Z.  R.  (1997).  Evaluation  of  the  effectiveness  of  a computer  assisted  intelligent  tutoring  system 
model  developed  to  improve  specific  learning  skills  of  special  needs  students  (CAI).  (Doctoral  dissertation. 
The  George  Washington  University,  1997).  Dissertation  Abstracts  Intemationak  58  (04),  A 1254. 

Schmidt,  F.  L.  (1997).  “Impact  of  instructional  strategy  on  training  effectiveness.”  (Doctoral  dissertation. 
University  of  Iowa,  1997)  [Abstract  from  UMI  ProQuest  Digital  Dissertations,  AAT  9805649] 

van  Merrienboer,  J.  J.  (1997).  Training  complex  cognitive  skills:  A four-component  instructional  design 
model  for  technical  training.  Englewood  Cliffs:  Educational  Technology  Publications. 

Winn,  W.  D.  (1997).  “Advantages  of  a theory-based  curriculum  in  instructional  technology.”  Educational 
Technology.  Jan/Feb:  34-41. 


992 


The  Design  and  Integration  of  Interactive  Multimedia  Learning  Environments 
Content,  Ergonomic,  Evaluation  and  Collaborative  Issues 


IsaBelle*,  C.,  Nkambou**,  R.  et  Dufresne***,  A. 

*Universite  de  Moncton,  Departement  d’enseignement  au  secondaire,  Moncton,  Canada 
**  Universite  de  Sherbrooke,  Departement  de  mathematiques  et  informatique,  Sherbrooke,  Canada 
***Universite  de  Montreal,  Departement  de  communication,  Montreal,  Canada 


Abstract  : The  integration  of  Information  technologies  into  the  educational  system  has  spurred  the 
creation  of  new  courses  to  effectively  train  teachers.  Hypermedia  environments  are  becoming  essential 
tools  for  the  enrichment  of  pedagogical  values  in  the  teaching  area.  WWW  technology  has  made  using 
these  environments  much  easier  since  such  hypermedia  resources  can  be  accessed  via  the  Internet. 
Hypermedia  environments,  however,  do  not  truly  meet  the  needs  of  the  teachers:  Often,  they  are  too  far 
removed  from  teachers’  true  concerns  and  needs  with  regards  to  the  integration  of  educational 
technologies.  Furthermore,  several  MILE  (Multimedia  Interactive  Learning  Environments) 
designers  are  placing  the  focusing  on  the  development  of  sophisticated  functionalities  which  seldom 
make  tasks  easy  for  the  user.  This  paper  introduces  a system  called  CINEMA*  which  is  committed  to  the 
design  and  integration  of  new  multimedia  learning  environments.  This  system  is  ideally  suited  to  the 
needs  of  teachers  who  are  eager  to  know  what  to  do  with  multimedia  and  how  to  develop  their  abilities 
using  multimedia. 


1.  Introduction 

Education  professionals  are  called  upon  to  integrate  multimedia  in  their  courses,  thereby  renewing  the 
teaching/leaming  process  (Tardif,1998)  by  designing  multimedia  teaching  materials.  They  are  confronted  with  a 
teaching  process  that  increasingly  requires  the  use  of  Information  technologies.  Teachers  are  caught  in  an  technico- 
educational  paradox  (IsaBelle,  1999):  the  general  population  demands  that  the  educational  community  offer  training 
which  meets  the  needs  of  the  today's  society;  yet,  die  majority  of  teachers  receive  little,  if  any,  training.  Much  of  the 
time,  what  little  training  they  do  get  hardly  emphasizes  the  importance  of  information  and  communication  technologies 
in  the  classroom.  While  some  interactive  learning  environments  for  distance  education  are  available,  there  are 
relatively  few  teachers  who  use  them.  First  and  foremost,  these  environments  present  contents  that  are  too  far  removed 
from  teachers' actual  concerns  and  needs  with  regards  to  the  integration  of  Information  technologies.  Secondly, 
although  the  majority  of  methods  used  to  design  Multimedia  Interactive  Learning  Environments  (MILEs)  involve  more 
explicit  reflections  of  the  learning  process  (Rouet  and  de  la  Passardiere,  1998),  several  designers  still  emphasize  the 
development  of  sophisticated  functionalities  which  often  do  not  facilitate  the  user's  task  (Nkambou,  IsaBelle  and 
Frasson,  1998;  Barthe,  1995).  This  paper  introduces  CINEMA,  a unique  system  for  the  design  and  integration  of  new 
multimedia  learning  environments,  which  is  actually  tailored  to  the  real  needs  of  teachers  who  are  eager  to  know  what 
to  do  with  multimedia  and  how  to  develop  their  abilities  . 

2.  Elements  for  effective  learning 

In  order  to  support  effective  learning,  the  person  learning  must  feel  involved  through: 

- general  knowledge  and  know-how  (the  cognitive  dimension),(Giordan  and  de  Vecchi(1987) 

- motivation  (the  emotional  dimension) 

- a relationship  with  his  peers  and/or  the  teacher,  (Aumont  and  Mesnier,  1 992). 


* CINEMA:  Conception  et  Implantation  de  Nouveaux  Environnements  Multimedias  d'Apprentissage. 
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Many  MILE  designers  neglect  the  pedagogical  dimension  when  developing  their  systems.  The  learning  process  must 
allow  the  user  to  work  in  a constructive  environment.  In  fact,  this  environment  must  promote  active,  constructive,  co- 
operative, authentic  and  intentional  learning  (Jonassen  and  al.,  1999). 


3.  Designer  Traps 

The  ergonomics  of  interfaces  deal  primarily  with  facilitating  navigation  as  well  as  addressing  the  effects  of  the 
cognitive  load  (Rouet  and  Tricot,  1995;  Duffesne,  1997).  However  there  are  some  traps  which  await  MILE  designers. 
The  Technological  trap  provides  the  user  with  powerful,  fast  and  visually  attractive  MILEs  that  often  totally  ignore  the 
pedagogical  aspects.  The  Ergonomic  trap  places  the  emphasis  on  the  ergonomic  aspects  without  taking  into  account  the 
cognitive  values  of  renewed  pedagogy  which  result  from  the  constructivism  of  Piaget  (Choplin,  Galisson  and 
Lemarchand,  1998).  On  one  hand,  ergonomic  experts  emphasize  the  user  in  order  to  facilitate  the  system’s  use  and 
decrease  the  complexity  of  the  user's  task.  On  the  other,  pedagogues  may  design  tools  that  encourage  the  ideas  of 
rupture,  deconstruction,  or  dissonance,  in  order  to  facilitate  the  development  of  the  learner’s  intellectual 
abilities(Giordan,  1996). 

Therefore,  ergonomic  experts  try,  as  much  as  possible,  simplify  and  clarify  the  use  of  computer  software.  Didactic 
experts,  on  the  other  hand,  prefer  to  create  more  difficulty,  which  is  necessary  to  any  leaming(Choplin,  Galisson  and 
Lemarchand,  1998). 

In  any  MILEs  design,  the  task  is  complex  because  the  designers  must  strive  to  balance  the  "user  side"  with  the 
"learning  side"  of  any  subject.  Can  there  truly  be  a system  which  succeeds  in  finding  the  perfect  balance  between  the 
ergonomic  and  didactic  entities  while  still  granting  important  emphasis  to  each  side  of  a subject? 

4.  The  CINEMA  system 

The  CINEMA  system  was  conceived  by  a team  made  up  of  an  educator,  a designer  and  a computer  scientist. 
Collectively,  they  have  built,  deconstructed  and  rebuilt  a model  (Gregori,  1999)  of  an  effective  interface,  a relevant 
content,  even  a content  presentation  corresponding  to  the  essential  dimensions  in  a learning  process  that  would  allow 
the  creation  of  pedagogical  multimedia  material.  The  CINEMA  program  is  available  on  the  Internet.  It  is  easy  to  use 
yet  still  allows  sufficient  questioning  during  information  processing.  The  primary  objective  of  the  site  is  to  make  it 
possible  for  users  to  create  pedagogical  material,  in  most  cases  an  educational  Web  page. 

CINEMA  provides  special  attention  to  the  application  of  constructivism  where  user  commits  him/herself  to  a process 
of  "Learning  how  to  learn"  (Bordeleau,  1994).  The  system  is  made  up  of  six  modules;  Learnings  Storyboarding^ 
Ergonomics,  Production,  System  and  Implantation  (see  figure  1).  The  first  five  modules  are  specific  to  the  creation  of 
multimedia  and  interactive  learning  materials.  The  sixth  module  details  how  to  integrate  this  environment  in  a school 
context.  In  fact,  it  is  important  to  plan  for  the  integration  of  the  developed  environment  early  on  in  the  design  phase. 
Without  addressing  this  concern,  there  is  the  risk  of  obtaining  a MILE  too  far  removed  from  the  school  needs,  and 
therefore  difficult  to  integrate  into  the  school  context  (Depover  and  Strebelle,1996). 
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4.L  Educational  content 

In  each  module,  the  user  is  presented  with  the  declarative  and  procedural  knowledge  involved.  This  is  done  through 
objectives  and  many  interactive  exercises  provided  for  the  learner. 

Learning  module 

In  this  module  (figure  2),  the  teaching/leaming  methods,  the  styles  and  the  learner  profile  are  detailed.  The  module  also 
includes  information  about  computer  applications  which  promote  the  development  of  multiple  intelligence  (Forcier, 
1999,  Gardner,  1997). 
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A la  fin  de  cette  sous-section,  vous  serez  en  mesure  de  : 


> identifier  les  elements  qui  compose  une  d6marche 
pedagogique; 

> mettre  en  pratique  la  mise  en  situation; 

>'  identifier  les  principales  caracteristiques  a considerer 
Chez  les  apprenants; 


Figure  2 : An  example  of  Learning  section  of  CINEMA 


Storyboarding  module 

In  this  module,  the  user  discovers  the  logic  which  sustains  the  elaboration  of  a storyboard,  adapted  to  the  needs  of 
"future  designers  ",  for  the  creation  of  a Web  page.  Specific  guidelines,  designed  to  help  schools  create  their  own  web 
pages,  are  also  included  in  this  module. 

Ergonomic  module 

This  section  is  specifically  devoted  to  understanding  the  human-machine  interaction.  Detailed  theories  about  human 
information  processing  are  provided.  Ergonomic  principles  and  the  eight  golden  rules  of  interface  design,  according  to 
Schneiderman  (1998),  are  explained.  These  help  future  teacher-designers  to  create  better  Web  pages  as  well  as  evaluate 
existing  systems. 

Production  module 

This  module  covers  the  basic  elements  of  an  Internet  Site,  such  as  tables,  frames,  etc.  as  well  as  ways  to  produce  them. 
The  basic  and  advanced  feature  of  an  authoring  tool  (Dreamweaver  3.0)  which  facilitates  web-page  creation  and  site 
organization  is  also  covered. 

System  module 

The  System  module  introduces  two  interactive  surveys  to  evaluate  educational  Web  sites  and  CD-ROMs.  The  first 
questionnaire  (IsaBelle  and  Zucchiatti  1999)  is  made  up  of  60  criteria  for  evaluating  the  pedagogical  aspects  of  a 
MILE.  The  second  consists  of  60  criteria,  which  allow  the  evaluation  of  interfaces  on  an  ergonomic  level.  For  briefer 
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evaluations  of  educational  and  ergonomic  aspects,  two  ” express  " surveys  of  15  criteria  each  are  also  available.  Given 
the  number  of  CD-ROMs  and  educational  Web  sites  already  on  the  market,  there  is  a great  demand  for  evaluation 
criteria  within  the  teaching  community  as  well  as  a need  for  systems  already  assessed  by  professionals.  The  System 
module  of  CINEMA  allow  users  to  evaluate  educational  CD-ROMs  and  web-sites  using  an  interactive  form.  It  is  also 
possible  for  users  to  share  the  results  of  their  evaluation  with  others. 

Implantation  module 

This  module  outlines  four  conditions  which  ensure  the  quality  of  educational  technologies: 

1 . Changing  the  beliefs  and  the  attitudes  of  the  whole  educational  community; 

2.  Incorporating  technology  into  the  pedagogical  structure  ; 

3.  Supporting  teachers  and  encouraging  collaboration  between  educators; 

4.  Transforming  teachers  into  agents  for  "change". 

Beyond  showing  teachers  a way  to  integrate  educational  technologies  into  the  practice  of  renewed  learning,  this 
module  outlines  how  to  use  the  CINEMA  system,  further  helping  to  enhance  their  multimedia  knowledge. 

The  CINEMA  system  also  offers  the  user  more  navigational  freedom.  Although  the  program's  built-in  suggests  a 
specific  browsing  and  reading  order,  the  user  is  free  to  start  with  the  module  of  their  choice.  By  offering  this  freedom, 
cinema's  creators  wanted  to  allow  the  user  to  question  him/herself  about  their  learning  process  (Choplin,  Galisson 
and  Lemarchand,  1998). 


4.2,  Ergonomic  aspects 

The  CINEMA  system  was  designed  to  be  easy  to  use,  freeing  the  user  to  concentrate  on  content.  Each  icon  was 
carefully  selected  for  its  significance,  in  order  to  represent  all  possible  types  of  activities.  For  example,  a hand  giving  a 
"thumbs  up"  indicates  the  presence  of  a positive  example.  The  "muscular"  arm  symbolizes  an  exercise  which  must  be 
performed  by  the  learner. 

All  examples,  exercises  and  tests  are  opened  in  a new,  smaller  window  in  order  to  communicate  their  context  to  the 
learner.  To  help  the  user  navigate,  the  title  of  the  current  module  is  displayed  above  the  work  area  while  the  menu  on 
the  left  indicates  sub-menus  which  have  already  been  visited.  Each  module  is  also  colour-coded  with  a specific  colour 
for  easier  identification. 

4.3,  Collaborative  learning:  communication  issues 

Tardif  (1998)  addresses  the  importance  of  vertical  and  horizontal  communication  to  support  learning.  In  distance 
education,  there  are  significant  communication  elements  which  are  considered  crucial  to  maintaining  a students' 
motivation  and  encouraging  them  in  their  learning  process.  According  to  a study  by  Moore  and  Kearsley  (1996),  as 
many  as  30%  to  50%  of  all  students  who  start  a distance  education  course  drop  out  before  finishing  it.  In  order  to 
increase  the  students'  success  ratio,  the  study  recommends:  establishing  a learning  community  between  the  students  and 
the  instructor,  as  well  as  providing  teacher-feedback  to  the  students.  This  learning  community  includes  collaborative 
training  and  socio-affective  exchanges  between  students.  Studies  (Bertrand,  1 996)  show  that  the  socio-affective  aspects 
of  the  student  are  very  important. 

Studies  further  suggest  that  collaboration  between  experts  and  other  learners  facilitates  the  acquisition  of  skills  and 
knowledge.  A Vygotsian  perspective  states  that  it  is  important  for  less  skilled  people  to  learn  from  an  expert  within  the 
zone  of  proximal  development  (ZPD).  Through  this  process,  the  learner  gradually  builds  their  own  skills  and 
knowledge  with  help  from  someone  of  greater  ability  (Vygotsky,  1978). 

Thus  CINEMA  provides  a complete  and  relevant  discussion  forum  (Figure  3).  This  includes  three  distinct  boards 
(contents,  technique  and  cafe)  allowing  the  learner  to  exchange  according  to  the  contents  and  technical  aspects  of  a 
course,  as  well  as  according  to  more  personal  aspects  (cafe). 
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Figure  3 : A view  of  Forum  of  CINEMA 


4»4»  Prototype  Evaluation 

It  often  seems  that  many  systems  are  introduced  without  being  fully  evaluated  and  yet  this  stage  remains  as  significant 
as  the  design  phase  itself  Esperet  (1998)  indicated  the  importance  of  generalizing  evaluation  and  experimentation 
methods.  Within  the  framework  of  this  project,  and  in  order  to  ensure  a use  which  truly  meets  the  needs  of  those 
concerned,  four  types  of  evaluation  were  proposed.  These  would  be  conducted  by  various  groups:  future  teachers, 
students  registered  in  a course  on  ergonomics,  teachers  studying  educational  technology  and  practicing  teachers  who 
want  to  create  educational  multimedia  environments. 

Right  now,  the  evaluation  by  future  teachers  has  taken  place  at  the  University  of  Moncton  and  at  the  University  of 
Montreal.  They  specifically  revised  the  contents  as  well  as  the  interface.  They  also  proposed  new  interactive  examples 
and  exercises.  Furthermore,  more  than  a hundred  students  registered  in  the  ’’Interface  et  multimedia”  course  at  the 
University  of  Sherbrooke  are  currently  evaluating  the  system.  This  group,in  particular,  will  evaluate  the  ergonomic 
aspects  of  the  CINEMA  using  a heuristic  evaluation  technique. 

As  part  of  the  "Training  and  the  Information  and  Communication  Technology"  course,  offered  at  the  University  of 
Moncton  during  the  fall  of  1999,  teachers  used  the  CINEMA  system  and  had  to  share  their  comments  and  evaluations, 
especially  concerning  its  pedagogical  aspects  (contents,  examples,  exercises,  pedagogical  strategies...).  By  the  end  of 
the  course,  each  "teacher  in  training"  involved  had  also  created  an  educational  Web  page. 

Finally,  within  the  context  of  a research  project  relating  to  distance  education  and  Web  page  creation,  teachers  at  an 
elementary  school  showed  great  interest  in  the  use  of  this  system  to  develop  their  school  and/or  classroom  Web  sites. 
This  use  of  the  CINEMA  program  by  the  very  users  it  targets  will  gather  invaluable  data  concerning  CINEMA’S 
unique,  symbiotic  balance  between  the  didactic  and  the  ergonomic  in  the  suggested  learning  environment. 


5.  Conclusion 

The  CINEMA  system  offers  a unique  blend  of  solid  content  and  optimized  functionalities  ideally  suited  to  the  real 
needs  of  teachers,  especially  when  it  comes  to  information  technology.  While  everyone  can  benefit  this 
comprehensive  program,  regardless  of  their  profession,  the  CINEMA  system  addresses  the  important  need  to 
adequately  prepare  and  empower  educators,  involving  them  in  the  process  of  change  to  integrate  Information 
technologies  into  the  school  system.  It  is  hoped  that  other  new  designers  of  MILEs  will  be  as  eager  to  offer  users  such 
versatile  contents  and  optimized  functionalities. 
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Abstract:  The  paper  presents  a prototype  that  manages  the  reading  paths  in  a set  of  multimedia 
documents.  It  allows  to  create  the  path  while  the  user  navigates  not  only  on  Web  pages,  but  also  on  local 
documents  or  documents  non  necessarily  on  the  Web.  The  system  stores  the  references  of  the 
information  fragments,  allowing  the  user  to  save  the  created  path,  to  review  it,  to  replay  the  sequence 
and  finally  to  distribute  the  path  to  students,  thus  avoiding  (or  limiting)  the  disorientation  phenomenon. 

The  prototype  also  supplies  some  simple  but  powerful  methods  for  manipulating  the  paths  and  creating 
new  ones.  These  primitives  consist  of  a series  of  basic  operations  on  paths,  like  the  concatenation  of  two 
paths,  the  intersection  of  a series  of  paths  in  order  to  find  common  nodes,  the  difference  of  two  paths 
and  so  on. 

1.  Introduction 

Hypertext  systems  have  been  for  long  time  the  only  software  tools  to  produce  non-linear  organization  of 
information  chunks.  Nowadays,  we  have  a plethora  of  software  tools  that  can  produce  hypermedia  documents 
(word  processors,  presentation  tools,  graphic  tools  etc.),  and  we  have  a standard  language  to  produce 
hypermedia  and  put  them  on  the  Web,  i.e.,  HTML.  The  problem  is  that,  generally,  the  integration  of  these 
different  hypermedia  documents  into  one  unified  hypermedia  is  difficult.  Building  a meta-hypertext  of  many 
different  documents  coming  from  different  software  sources  is  a task  where  technical  skills  are  necessary.  A 
good  example  of  this  difficulty  is  the  relation  between  a hypertext  system  and  another  software  data 
management  system  used  to  handle  geo-referenced  data,  i.e.,  a Geographic  Information  System  (GIS)[Colazzo 
et  al.  97].  At  the  time  of  our  experiments  in  this  field,  wc  had  to  choose  the  option  of  extending  the  functions  of 
a commercially  available  hypertext  system  by  adding  primitives  that  handle  maps,  rather  than  adding  hypertext 
functions  to  a commercially  available  GIS.  In  fact,  GIS  capabilities  of  performing  hypertext  functions  have 
little  similarity  with  the  current  standard  of  hypermedia  systems. 

Besides  the  problem  of  integrating  information  chunks  coming  from  different  sources,  the  organization  of 
these  chunks  in  different  reading  paths  (as  we  usually  do  while  building  hypertexts)  does  not  mean  necessarily 
to  cancel  the  linearity  of  the  learning  process.  In  most  hypertexts,  the  reader  does  not  proceed  randomly  in  the 
navigation.  The  author  guides  the  navigation  by  proposing  some  standard  “routes”  of  comprehension.  The 
paths  built  in  this  way  are  often  the  only  reasonable  way  of  navigation  for  the  user,  and  can  not  be  changed  by 
him/her.  However,  these  paths  could  not  satisfy  all  the  user  needs,  and  in  this  case  s/he  should  be  able  to  create 
her/his  own  path,  for  example  by  taking  information  chunks  from  different  hypermedia  and  by  combining  them 
in  a new  personalized  path.  The  paper  presents  a prototype  that  allows  the  reader  to  build  custom  paths  on 
hypermedia  documents  derived  from  different  sources,  and  to  review,  cancel,  modify  and  aggregate  these  paths 
by  using  some  specialized  primitives.  Every  path  connects  a potentially  large  number  of  information  chunks 
deriving  from  different  development  tools,  like  word  processors,  HTML  pages.  Postscript  or  PDF  files, 
presentation  slides,  technical  sketches  etc.  The  prototype  creates  generalized  lists  of  chunks  coming  from  these 
different  sources,  by  allowing  a virtual  “sequence”  among  documents  technically  different  but  logically 
connected.  This  could  be  very  useful  in  educational  environments  where  the  only  availability  of  HTML  pages 
for  creating  hypermedia  limits  the  creativity  of  the  author. 

1.1  Relevant  works 

Hypertext  paths,  also  called  tours,  are  a concept  as  old  as  hypertext  self.  Bush  [Bush,  45]  incorporated  in 
Memex  the  paths  to  provide  a personal  means  for  organizing  found  information  and  to  provide  a means  to 
communicate  that  information  to  other  persons.  Generally,  the  paths  implemented  to  help  to  solve  two  major 
problem,  namely  “user  disorientation”  in  hypertext  networks  and  the  additional  “cognitive  overhead”  needed  to 
create,  manage  and  choose  among  links  [Conklin,  87].  Paths  appear  in  many  implemented  systems:  for 
example,  Textnet  paths  [Trigg  et  al.,  86],  HyperCard  scripts  [Goodman,  87],  Guided  tours  in  NoteCards 
[Halasaz  et  al.,  87],  Hammond  & Allinson’s  guided  tours  [Hammond  et  al.,  88],  Petri  nets  in  aTrellis  [Stotts  et 
al.,  88],  Scripted  Documents  [Zellweger,  89].  There  have  been  several  different  implementations  of  paths  in 
hypertext  systems:  the  history  trail,  which  automatically  logs  every  location  the  reader  visits,  and  the  guided 
tour,  which  is  a predetermined  set  of  locations  that  can  be  stored  and  visited  over  and  over,  in  the  same  order 
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[Mylonas  et  al.,  90].  Besides,  generally,  the  hypertext  systems  are  specialized  mechanism  built  for  their  own 
customized  environments:  the  published  accounts  suggest  that  they  were  intended  for  use  by  a sophisticated 
audience  of  hypertext  authors  and  readers.  In  last  time,  the  concept  of  hypertext  paths  is  applied  to  the  WWW. 
For  example  the  Walden’s  Paths  [Furuta,  97]:  the  reason  is  the  heterogeneity  of  the  web  and  the  desirability  of 
supporting  the  teacher-student  relationship.  Walden’s  Paths  is  designed  to  enable  teachers  to  make  use  of 
materials  available  via  Internet  by  creating  directed  paths  over  the  World-Wide  Web.  The  system  consists  of 
two  main  components:  the  Path  Authoring  and  the  Path  Server.  The  first  enables  the  teacher  to  create,  modify, 
validate  and  reuse  paths:  especially,  it  supports  the  process  of  locating  sites  whit  relevant  information,  ordering 
and  annotating  those  pages  to  form  the  path.  The  path  server  is  the  implementation  means  through  which  the 
students  navigate  the  path:  when  it  receives  a request,  it  determines  the  path  and  page  number  of  the  request. 

2.  Different  chunks  from  different  tools 

Our  work  starts  from  a simple  consideration  concerning  the  current  state  of  the  process  of  distributing 
educational  material  from  producers  (teachers)  to  consumers  (students).  The  WWW  has  given  a strong  impetus 
to  the  concept  of  distance  learning,  but  in  part  it  has  also  distorted  its  meaning  and  limited  the  use  of 
multimedia  and  technical  documents,  i.e.,  what  is  not  easily  adaptable  to  an  HTML  file  or  an  GIF/JPEG  image. 
In  addition,  it  is  difficult  to  recreate  a full  interaction  with  the  user  in  Web  pages:  for  example,  a multimedia 
package  is  surely  more  suitable  than  a Java  applet  to  build  an  animation  simulating  an  industrial  automation 
process.  The  idea  presented  here  is  to  leave  the  educational  material  in  the  “best  suited”  format,  i.e.,  the 
original  format,  having  a meta-tool  that  manages  the  hypertext  chunks  by  launching  the  specific  viewer  when 
the  reader  needs  it.  Moreover,  some  attributes  can  be  assigned  to  the  elements  of  these  paths  for  subsequent 
retrieval,  and  a series  of  primitives  can  be  performed  of  these  lists  of  references.  Here  follow  two  examples  that 
show  how  useful  is  to  build  generalized  paths  on  different  sources  of  information  chunks.  The  considerations 
here  explained  are  extensible  to  other  fields,  though  both  situations  have  the  educational  material  as  common 
background. 

2.1  A hypermedia  system  used  to  project  slides 

The  construction  of  a slide  for  a lecture  is  a time-consuming  operation,  and  it  is  advisable  to  design  slides  so 
that  they  can  be  reused  several  times.  A collection  of  slides  to  expound  a set  of  arguments  has  an  intrinsically 
linear  structure:  indeed,  the  arguments  can  only  be  presented  in  a temporally  linear  sequence.  There  is  therefore 
close  connection  between  the  sequence  of  the  arguments  discussed  and  the  fragments  used  during  the  lesson. 
However,  it  is  obvious  that  not  all  the  lessons  have  the  same  audience  or  the  same  communicative  purpose. 
Moreover,  an  argument  can  be  treated  in  different  ways  according  to  the  level  of  the  students’  knowledge.  It 
follows  that,  even  if  the  structure  of  the  navigation  among  the  slides  is  linear,  the  teacher  must  reorganize  the 
slides  with  every  new  lesson.  In  principle,  the  lecturer  could  duplicate  slides  to  construct  a new  sequence,  but 
this  work  takes  a certain  amount  of  time  and  leads  to  the  proliferation  of  copies  of  the  same  slide.  If  changes 
are  made  to  one  slide,  these  changes  must  be  reproduced  in  all  the  lessons  that  use  the  modified  slide.  In  some 
cases,  the  lecturer  must  make  an  impromptu  change  to  his/her  exposition  in  order  to  deal  with  an  unexpected 
event  during  the  lecture.  The  only  thing  that  the  lecturer  can  do  is  try  to  find  the  slides  relating  to  the  topic  and 
navigate  through  the  directories  of  the  file  system  opening  different  collections  of  slides.  If  the  paths  among  the 
slides  are  managed  as  objects  endowed  with  their  own  behaviors  and  methods,  it  is  no  longer  necessary  to 
duplicate  the  slides:  all  that  is  required  is  to  create  several  paths  through  the  same  slides  and  give  different 
labels  to  them. 

2.2.  Reading  paths  in  a CD  for  teaching  purposes 

Every  path  in  a hypertext  inevitably  interweaves  with  many  others.  As  a consequence,  the  same  fragment 
may  belong  to  several  reading  paths  organized  into  difference  sequences.  The  creation  of  reading  paths  is  a 
crucial  stage  in  the  design  of  a hypertext.  The  designer  must  place  him/herself  in  the  shoes  of  the  future  user 
and  create  links  among  the  various  fragments  according  to  a unitary  and  systematic  logic,  which  prevents 
reader  disorientation.  In  this  way,  as  the  reader  follows  a route,  s/he  must  be  able  to  grasp  the  meaning  of  that 
particular  sequence. 

The  creation  of  personalized  paths  among  hypermedia  documents  has  been  the  subject  of  a joint  project 
undertaken  by  the  Department  of  Computer  and  Management  Sciences  of  the  University  of  Trento  and  I PRASE 
of  Trento  (an  educational  research  center).  The  outcome  of  the  project  is  a CD-ROM  on  medieval  history,  with 
particular  reference  to  the  Trentino  area  of  Italy. 

However,  a teacher  may  want  to  deal  with  the  material  from  a different  point  of  view:  for  example,  the 
teacher  may  decide  to  present  a sequence  of  frescoes  to  be  found  in  various  medieval  castles  in  the  area,  thus 
extracting  this  information  from  the  medieval  history  CD.  The  teacher  may  also  make  it  possible  for  the 
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students  to  upset  this  sequence  by  adding  further  information  that  they  believe  relevant  to  the  topic.  Moreover, 
the  teacher  may  have  another  hypertext  available,  which  presents,  for  example,  a collection  of  medieval 
jewelry. 

Another  example  is  taken  from  a CD-ROM  with  educational  content  regarding  botany  and  biology  (see  fig. 
2).  The  hypermedia  includes  the  navigation  tool  with  some  extensions  and  peculiarities.  In  particular: 

• It  is  possible  to  recall  the  navigator  while  selecting  a path,  instead  of  starting  the  navigation  with  the 
tool; 

• From  a simple  integration  the  tool  has  become  the  main  navigation  tool  of  the  CD.  The  opera  contains  a 
huge  number  of  different  paths,  which  can  be  modified,  added,  deleted  because  they  are  stored  on  the 
disc.  Moreover,  the  user  can  build  new  paths  based  on  his/her  own  needs. 

• The  tool  allows  the  creation  of  paths  that  include  documents  from  tools  other  than  the  original 
(Asymetrix  Toolbook),  for  example  integrating  Word  documents  with  Toolbook  hypermedia  and  HTML 
pages. 


Figura  Ja-b.  Activation  of  the  navigator  in  the  hypermedia  “Castles  ofTrentino”.  There  are  two  icons:  one  of  the 
home  page  of  the  CD,  the  other  is  a internal  page  with  the  navigator  icon  inserted  among  the  other  navigation  buttons 


Figure  2a-b.  Icon  of  the  navigation  tool  in  the  main  page  of  the  CD-ROM  and  the  navigator  active  on  a page 


3.  The  prototype 

The  prototype  is  an  application  that  enables  the  user,  or  group  of  users,  to  handle  abstract  objects  called 
Paths.  A path  is  defined  as  a class  of  objects  called  References.  A Reference  has  the  following  attributes: 

• Application:  carries  the  name  of  the  application  that  has  generated  the  file.  It  may  be  absent  in  cases  when 

the  operating  system  is  able  to  recognize  the  file  from  its  extension,  as  happens  in  MS  Windows; 

• File  location:  gives  the  path  which  identifies  the  position  of  a file  in  the  file  system; 

• File  name:  gives  the  name  of  the  file  with  its  extension; 

• relative  address  of  the  node:  fragment  identifier  allowing  to  position  on  a specific  part  of  a document; 

• script:  a piece  of  code  in  the  specific  scripting  language  of  the  application  for  precise  positioning  at  a 

certain  fragment/page/node  etc.  of  the  document. 

• Comment:  optional,  helps  user  during  navigation  by  appearing  in  the  title  bar  of  the  navigation  tool. 

• KeywordI,  2,  3:  descriptive  fields  in  which  the  user  can  insert  a keyword  for  subsequent  search.  This  is 

useful,  for  example,  when  information  chunks  have  a common  attribute  that  characterizes  them. 

The  references  in  a path  are  arranged  into  an  order  established  by  the  creator  of  that  particular  path.  As  well 
as  its  ordered  list  of  references,  every  path  is  given  a number  of  attributes  when  it  is  first  created  (figure  3): 
Title:  one  for  each  path;  Author:  the  name  of  the  creator  of  the  path;  Type  of  author:  student,  teacher. 
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researcher;  Date  of  creations  Comment:  annotations  on  the  i^dXh\Password  (optional)  to  prevent  cancellation  or 
alteration  of  the  path  by  other  users. 

The  application,  which  permits  the  creation  and  management  of  paths,  is  based  on  a stratified  architecture 
comprising  two  distinct  but  linked  components.  The  actions  performed  by  the  user  at  the  FrontEnd  are 
translated  into  actions  based  on  data  that  constitute  the  BackEnd  of  the  application 


3.1.  Creating  a path 

Once  activated  in  path  creation  mode,  the  prototype  works  in  background  and  presents  itself  as  an  icon. 
Whenever  the  user  decides  that  a particular  file  or  node  is  of  interest  to  the  path  that  s/he  wants  to  create,  s/he 
clicks  on  the  prototype’s  icon.  The  prototype  transparently  stores  the  data  relative  to  the  information  node 
chosen  by  the  user  in  the  BackEnd.  We  are  working  on  some  extensions  of  the  creation  step:  in  fact,  it  is 
possible  to  imagine  the  paths  as  being  constructed  by  an  automatic  process  (experiments  have  already  been 
conducted).  For  example,  in  products  constructed  with  multimedia  authoring  systems,  it  is  possible  to  activate  a 
process  of  automatic  creation  based  on  identification  of  objects  endowed  with  specific  properties.  The  process 
analyzes  all  the  information  nodes  included  in  the  list  of  inputs  retrieved  from  the  BackEnd  and  creates  an 
order  according  to  the  values  of  the  property  selected.  It  finally  translates  the  order  obtained  into  a sequence  of 
references. 
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Figure  3:  the  attributes  of  a path 


3.2.  Changing  and  combining  paths 

The  prototype  allows  the  user  to  inspect  the  paths  produced  and  to  act  in  two  distinct  ways:  alter  the 
references  of  individual  paths  or  combine  paths  to  produce  new  ones.  In  the  former  case,  it  is  possible  to:  1) 
change  the  characteristics  of  the  path;2)  alter  the  sequence  of  links  among  the  references;  3)  remove  certain 
references;  4)  remove  the  path  itself;  5)  insert  a new  reference  at  an  arbitrary  point  in  the  sequence. 

In  the  latter  case,  composition  functions  can  be  applied  to  the  references  constituting  two  paths.  These  are 
automatic  processes  which,  when  applied  to  two  or  more  paths,  produce  a further  one  (figure  4).  The  prototype 
in  its  current  form  comprises  the  following  composition  functions: 

1 . Intersection  of  two  paths  A and  B:  a path  formed  from  references  included  in  both  A and  B; 

2.  Union  of  two  paths  A and  B:  this  is  a path  formed  from  all  the  references  belonging  to  at  least  one  of  the 
paths  A and  B:  duplicated  references  are  eliminated; 

3.  Difference  of  two  paths  A and  B:  (in  the  order  given)  path  A references  which  are  not  in  B; 

4.  Concatenation  of  two  paths  A and  B:  a path  with  all  the  references  in  A followed  by  all  the  references  in  B; 

5.  Specific  nodes  from  two  paths  A and  B:  a path  consisting  of  nodes  which  are  not  shared  by  A and  B,  i.e., 
they  are  present  in  one  of  the  two  paths  but  not  in  both  of  them. 

It  is  similarly  possible  to  define  the  intersection,  union,  concatenation,  etc.  of  more  than  two  paths. 
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Figure  4:  example  of  combining  two  paths 
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3.3.  Following  a path 

When  the  prototype  is  used  to  follow  an  already-existing  path,  the  reader  works  with  an  interface  like  the 
one  in  Fig.  5,  reduced  to  the  minimum  so  that  it  is  as  unobtrusive  as  possible.  There  are  six  buttons:  one  to 
close  the  program,  four  to  navigate  through  the  paths  created,  and  one  to  consult  the  list  of  references  that  are 
in  the  path.  After  selecting  the  desired  path,  the  reader  uses  the  buttons  to  view  the  following  pages:  First 
reference:  shows  the  first  path  reference;  Next  reference:  shows  the  reference  following  the  current  one; 
Previous  reference:  shows  the  reference  previous  to  the  current  one;  Last  reference:  shows  the  last  reference  in 
the  path;  List  references:  shows  a listbox  with  the  references  in  the  current  paths  and  the  current  reference 
marked.  This  helps  the  user  to  understand  where  the  path  will  lead  next  (figure  6).  The  reader  can  change  path 
at  any  time,  by  only  choosing  another  path  among  those  handled  by  the  prototype.  The  reader  may  also  explore 
paths  starting  from  a specific  location.  S/he  can  obtain  a list  of  all  paths  that  traverse  the  current  reference.  For 
example,  if  the  reader  is  at  reference  X (Figure  7),  s/he  can  obtain  a list  of  all  possible  paths  which  pass 
through  that  reference  and  continue  to  navigate  by  changing  direction  and  following  one  of  the  alternative 
paths:  for  example,  paths  1,  2 or  3. 
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Figure  5.  Main  window  of  the  prototype 


Figure  6.  The  list  of  nodes  available  with  the  current  node 
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Figure  7.  Paths  intersecting  a reference  X 

Figure  8:  Architecture  of  the  distributed  version  of  the  navigation 
tool 

4.  Extending  the  prototype:  sharing  paths  among  users 

We  are  currently  testing  another  extension  of  the  prototype  that  regards  the  distribution  of  the  created  paths 
to  a centralized  database  accessible  through  the  Internet.  This  has  some  evident  advantages: 

Teachers/students  that  are  not  currently  on  their  computer  can  use  the  stored  path  wherever  they  are  taking 
the  lesson,  thus  allowing  distributing  not  the  material,  but  the  sequence  that  constitutes  the  lesson.  Under 
this  perspective,  we  can  think  to  the  prototype  as  a “sequencer’’  of  hypermedia 

the  powerful  primitives  of  the  system  can  be  applied  not  only  to  local  paths,  but  also  to  educational  paths 
that  a community  of  teachers  has  decided  to  share.  This  could  be  interesting,  for  example,  for  a multi- 
national corporate,  where  huge  amounts  of  hypermedia  presentations  are  created.  Single  teachers  use  them 
but  share  not  only  the  slides,  but  also  the  sequence  among  them.  It  could  be,  for  example,  very  useful  to 
provide  an  “intersection”  primitive  on  all  path  created  for  a presentation  of  a new  product,  in  order  to 
verify  what  the  different  salespersons  have  presented. 

The  use  of  a centralized  database  as  back-end  (figure  8)  for  storing  the  paths  has  evident  advantages  of 
administration  regarding  the  references  contained  in  the  corpus.  We  can  imagine  an  institution  / company 
that  administer  its  educational  material  in  a better  way  by  providing  a centralized  administration  not  only 
of  the  paths  but  also  of  the  physical  documents  and  slides  composing  the  educational  material.  On  the 
market  there  are  some  products  that  provide  centralized  administration  of  educational  material,  but  in 
terms  of  costs  and  complexity  of  the  system  we  are  moving  towards  a far  simpler  solution. 
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Figure  9:  the  simple  architecture  for  sharing  simple  primitives  on  a distributed  database 


The  architecture  of  the  system  is  based  on  a proprietary  though  simple  communication  protocol  that  allows 
two  remote  instances  of  the  prototype  to  exchange  commands  and  data  regarding  the  paths  required  by  the  user. 
The  protocol  is  based  on  Tcp/ip,  thus  allowing  the  connection  among  computers  connected  to  the  Internet,  but 
does  not  imply  the  sophisticated  structure  typical  of  client/server  approaches,  like  DCOM  or  CORE  A.  The 
application  is  simply  the  same,  it  is  only  activated  in  server  mode  or  client  mode  (figure  9). 

In  the  first  case,  it  does  not  navigate  by  itself,  but  receives  from  the  Internet  requests  of  executing  primitives 
on  the  database  stored  on  the  server  computer.  The  content  of  the  packets  exchanged  among  the  clients  and  the 
server  is  relatively  simple  and  small,  and  so  there  are  few  issues  regarding  network  performance. 


5.  Conclusions  and  future  developments 

We  presented  a software  tool  that  abstracts  from  the  software  that  manages  digitized  information.  The  tool 
integrates  the  path  that  an  author  creates  on  these  different  chunks  of  information,  allowing  connecting  pieces 
of  information  coming  from  different  applications  in  a hypertextual  structure.  The  prototype  manages  these 
paths  allowing:  a)  the  transportability  of  the  path  without  encapsulating  the  single  chunks  of  information;  b) 
the  abstraction  from  the  application  that  creates  the  document,  thus  allowing  the  integration  of  different  types 
of  digitized  information  in  an  unique  hypertext;  c)  the  management  of  multiple  paths,  thus  sharing  different 
hypertexts  from  different  authors;  d)  some  elementary  operation  on  the  paths,  like  intersection,  union  etc. 

The  prototype  has  been  successfully  used  in  some  informative  and  educational  CD-ROMs  and  has  other 
undergoing  projects  in  which  it  will  be  used  as  the  main  navigation  mechanism.  The  current  development  is 
oriented  to  the  integration  of  the  prototype  with  a mechanism  of  remote  archiving  of  the  paths  through  Internet. 

We  believe  that  our  prototype,  even  though  it  is  experimental  and  simple  in  its  approach,  prompts 
reassessment  of  the  role  of  ‘rationalized  sequences’  in  computer-mediated  teaching.  Good  feedback  from  users 
who  have  consulted  the  CDs  produced  with  the  navigator  demonstrates  their  natural  but  documentable 
‘familiarity’  with  the  concept  of  sequence.  It  also  demonstrates  that  teaching  materials  are  more  easily 
assimilated  if  they  can  be  consulted:  (A)  along  a path  created  by  an  author;  (B)  where  the  path  corresponds 
exactly  with  the  metaphor  of  sequence  and  the  elementary  navigation  operations  (first,  last,  next,  previous); 
(C)it  is  possible,  with  the  sufficient  knowledge,  to  operate  simultaneously  on  paths  viewed  as  sets  of  links. 
Evaluation  of  these  features  with  a complete  benchmarking  system  is  indispensable  for  clearer  understanding  of 
the  issues  involved  in  computer-supported  learning. 
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Abstract:  The  design  of  a web-based  course  is  discussed  based  upon  considerations  of  content, 
pedagogy,  learning  styles,  and  assessment.  The  learning  styles  are  examined  as  primary 
considerations  and  indeed  dictate  needed  elements  in  a web-based  course.  Assessment  is  also 
considered  in  the  initial  planning  of  such  a course.  These  design  elements  are  discussed  with 
particular  web  elements  given  for  each  design  element.  Further  illustration  of  the  design  procedure 
will  be  demonstrated  by  an  application  to  a freshman  chemistry  course.  All  considerations  may  be 
used  both  for  a web-enhanced  course  and  a course  taught  totally  online. 


INTRODUCTION 

To  design  a course  for  optimum  impact,  the  following  should  be  simultaneously  considered:  the  course  material;  the 

students'  learning  styles;  pedagogy,  which  includes  methodologies  of  presentation;  and  assessment.  These  items 

then  need  to  be  considered  as  itemized  below. 

• Regarding  course  material,  experienced  educators  know  the  needs  of  the  customer  regarding  the  content  of  the 
courses.  The  students  (learners)  need  courses  that  are  current  and  that  are  relevant  to  their  career  choices. 
Because  most  faculty  members  keep  up  with  their  disciplines  and  their  relevance  to  other  disciplines,  they 
generally  know  what  is  needed  in  the  lower-division  courses  and  in  the  upper-division  courses.  Course  content 
is  not  constant  over  the  years,  and  all  educators  must  work  to  keep  up  with  subject  advances.  However,  the  area 
of  course  content  is  usually  the  most  dependable  part  of  course  design.  But  read  below  for  web  implementation. 

• Learning  styles  often  vary  significantly  among  a typical  group  of  freshman  students.  However,  most  faculty 
members  often  omit  consideration  of  the  students’  approach  to  the  course.  The  teacher  often  will  teach  the 
course  based  upon  his/her  learning  style  and  the  way  that  he/she  was  taught.  Certainly  most  faculty  members 
usually  don't  plan  their  course  presentations  based  upon  the  myriad  learning  styles  of  their  students.  Many 
students  leave  science  partly  because  of  our  poor  planning  regarding  these  learning  styles  (1,2).  This  area  is 
usually  the  weakest  component  of  course  design. 

• Pedagogy  appropriate  for  each  particular  course  must  drive  the  development  of  the  course  (3).  Although  course 
content  may  be  well  known,  and  the  relevance  of  the  subject  matter  to  many  disciplines  well  known,  often  not 
enough  thought  is  given  to  the  pedagogy  required  for  the  best  development  of  student  learning.  Methods  of 
effective  presentation  can  vary  widely  among  disciplines.  The  white  board  is  still  an  excellent  tool  for 
presentation,  but  now  we  also  have  the  smart  board.  The  importance  of  small  group  work  is  also  well  known 
now.  Student  presentations  are  also  an  important  component  of  educational  methodology.  Faculty  members 
now  have  a wide  choice  of  technology  suitable  for  enhancing  their  teaching  of  the  courses.  Modem  technology 
can  considerably  enhance  their  applications  of  good  pedagogy.  So  we  must  consider  how  to  incorporate  these 
techniques  into  a web-based  course. 

• Course  assessment  must  also  be  considered  at  the  development  stage  of  a course  - note  in  the  title  that 
assessment  is  listed  before  implementation.  Many  books  have  been  vmtten  about  course  assessment  (4).  At  the 
beginning  of  the  course,  it  is  very  important  to  know  the  beginning  level  of  student  understanding  of  the  subject. 
It  is  also  important  that  all  of  the  students  know  their  own  learning  styles,  and  the  best  way  to  approach  learning 
based  upon  their  individual  learning  styles.  The  student  and  professor  must  also  understand  their  personality 
profiles  to  aid  in  a more  proficient  learning  experience.  Assessment  will  involve  studies  of  content  knowledge, 
before  and  after  taking  the  course;  learning  styles;  personality  profiles;  and  environment. 
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WEB  SITE  CONSIDERATIONS 


When  we  begin  our  course  development/design  considering  the  course  material  that  needs  to  be  mastered,  pedagogy, 
the  possible  learning  styles  of  our  students,  and  assessment,  a new  paradigm  of  course  design  develops.  What 
follows  can  be  used  for  either  on-campus  or  on-line  courses.  It  can  also  be  applied  to  any  other  area  of  learning 
(government,  industry,  etc). 

We  will  assume  that  the  gross  course  content  is  not  an  issue.  For  the  first-year  teaching  faculty  member,  putting 
together  the  first  set  of  course  material  can  be  quite  a daunting  task.  Also  every  year  the  faculty  member  must  fight 
against  the  tendency  to  use  the  "yellow  notes"  from  previous  years.  It  is  very  important  for  all  faculty  to  keep  their 
course  content  current,  but  we  will  assume  that  such  a consideration  is  a given.  However,  putting  the  material  on  the 
web  presents  some  additional  considerations.  We  want  to  make  the  best  use  of  the  web  medium  to  maximize  the 
constructivist  possibilities  (5).  There  will  be  some  elements  of  objectivism  in  the  presentation  because  some 
material  has  to  be  memorized  so  that  deeper  understanding  can  be  based  upon  that  memorized  base.  This  discussion 
is  now  overlapping  between  course  content  and  pedagogy,  but  such  an  overlap  is  an  essential  part  of  the  design 
process. 

The  pedagogy _2iV{d  the  method  of  presentation  have  to  be  reconsidered  every  year  by  the  faculty  member.  New 
technology  for  use  by  the  educator  is  being  developed  every  month.  A good  example  of  the  use  of  modem 
technology  is  in  the  development  of  a web  component  to  a course.  The  amount  of  web  involvement  for  a course  can 
vary  considerably  from  a very  minimal  involvement  of  putting  a course  syllabus  on  the  web  to  the  considerable 
effort  of  developing  a web-based  course.  Web-based  courses  are  courses  that  have  a major  component  of  all  aspects 
of  the  course  on  the  web.  These  web-based  courses  can  be  used  as  course  supplements  for  lectures  or  for  totally  on- 
line courses.  Because  they  can  be  used  in  both  venues,  the  development  time  is  optimized  for  its  effectiveness.  As 
stated  in  the  preceding  paragraph,  we  want  to  maximize  the  constructivism  that  can  occur  based  upon  the  design  of 
the  web  site.  In  chemistry,  we  do  want  some  objectivism  in  the  design  so  we  will  have  some  drill  exercises. 
However  we  will  want  to  clearly  use  that  memorized  base  to  lead  the  students  into  a constructivist  approach  to  their 
knowledge  development.  Abstract  topics  such  as  in  chemistry  and  physics  can  be  considerably  enhanced  by  the  use 
of  simulations  and  animations.  The  simulations  can  also  have  a student  interface  allowing  for  students  to  change  the 
operational  parameters  of  the  simulations  and  thus  lead  to  a deeper  understanding  of  the  concepts. 

The  consideration  of  the  learning  styles  of  our  students  is  usually  the  most  often  neglected  item.  Below  are  some 
learning  styles  and  the  course  elements  needed  for  those  learners. 

Learner  Type  Needed  Course  Elements 


Visual  Learners 


Graphics  (illustrations,  films,  slides,  and  diagrams),  flow  charts, 
discussion  bulletin  boards,  animations. 


Auditory  Learners 
ReadAVrite  Learners 


Kinesthetic  Learners 


Sequential  or  Global  Learning 


Films,  sound-enhanced  slides,  Microsoft  Net  Meeting 

Written  web  material,  referrals  to  other  web  sites,  written 
assignments  such  as  chapter  summaries,  suggested  exam  questions,  etc. 

Different  web  pages  for  different  materials  (makes  them  take  a needed 
break),  short  web  pages,  memorization  drills,  animations  requiring 
input,  non-web  assignments  (surveys,  lab  experiments). 

Give  a good  overview  of  each  section  and  provide  a logical  progression 
through  the  material  that  can  be  chosen  by  the  learner. 


Inductive  Approach 


Facts  and  observations  given,  principles  developed 


Deductive  Approach 


Principles  given,  consequences  and  applications  deduced 


ERIC 
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Active  Learners 


These  learners  need  group  work.  We  can  make  assignments  to  groups 
of  students  and  require  presentation  of  results  over  the  web  using  a 
course  management  system  such  as  WebCT.  Case  studies  appropriate 
to  the  discipline  can  be  used  very  effectively  in  this  component. 
Reports  of  these  assignments  should  be  due  often  (every  two  weeks). 

Reflective  Learners  These  learners  need  time  to  think  over  the  material  before  trying  it  out. 

On-line  quizzes  that  are  taken  at  the  time  of  their  choosing  appeals  to 
these  learners.  A thorough  assignment  due  at  the  end  of  the  semester 
also  appeals  to  these  students. 

Other  authors  have  also  examined  utilizing  some  of  the  implications  of  learning  theory  and  web-based 
instruction  (6). 

It  is  recommended  that  faculty  begin  work  on  designing  the  web  site  and  how  to  include  course  material  while 
considering  assessment.  The  methods  of  assessment  need  to  be  in  place  well  before  the  first  semester  of  use  begins, 
and  a specialist  in  assessment  should  be  part  of  the  team.  The  assessment  needs  to  include  comparisons  with  an  on- 
campus  class  covering  the  same  subject,  and  the  best  arrangement  is  for  the  same  professor  to  teach  the  on-campus 
and  on-line  class.  Before  the  classes  begin,  both  classes  need  to  take  a subject  matter  test,  a personality  profile  test, 
the  Learning  and  Study  Strategies  Inventory  (LASSI)  test,  and  at  least  two  different  kinds  of  learning  styles  tests.  It 
is  best  if  a standardized  subject  matter  test  can  be  found  such  as  is  offered  by  the  American  Chemical  Society  for 
chemistry  tests.  At  the  end  of  the  class  the  students  need  to  then  take  the  subject  matter  test  again.  Studies  of  the 
student  environment  involve  the  actual  class-related  experiences  of  the  student  during  the  course.  Obtaining  student 
input  on  the  environmental  issues  is  very  important.  For  a web-based  course  the  environmental  issues  are  ease  of 
site  navigation,  usefulness  of  various  sections,  ease  of  use  of  audio/video  components,  ease  of  electronic 
correspondence  with  other  students  and  faculty,  etc.  The  assessment  expert  should  help  the  faculty  member  choose 
the  proper  tests  and  opinion  surveys,  and  aid  in  the  analysis  of  the  correlations. 


WEB  SITE  CONSTRUCTION 

For  effective  use  of  everyone’s  time,  it  is  very  important  to  use  a team  approach  to  the  construction  of  the  web  site. 
It  is  unreasonable  to  expect  one  person  to  become  an  expert  in  all  needed  areas.  The  team  should  consist  of  a 
content  expert,  an  instructional  designer,  a programmer,  a web  interface  designer,  a graphic  designer,  and  an 
assessment  expert.  There  are  a number  of  good  references  for  considerations  in  the  web  design  process  (7). 

At  this  point  we  know  the  discipline  material  needed  on  the  web  site,  how  to  design  the  web  site  for  optimal  use  by 
the  students,  the  methodologies  of  presentation,  and  the  methods  of  assessment.  Now  we  can  begin  to  develop  our 
web-based  course  by  including  all  of  the  above  elements  in  our  design.  From  the  above  we  see  that  we  need: 
synchronous  and  asynchronous  means  of  communication  among  the  class  members,  video  clips,  sound  bites, 
simulations  and  animations,  photos  with  sound  and  text,  access  to  a conference  software  package  like  Microsoft  Net 
Meeting,  short  web  pages,  different  web  pages  for  different  topics,  drill  routines  for  help  in  understanding  and 
memorizing,  animations  with  input,  a good  overview  of  each  section  and  of  the  entire  site,  prepared  method  of 
progression  through  the  site  for  those  who  need  such,  methods  for  inductive  and  deductive  approaches,  short-term 
and  long-term  group  assignments,  and  on-line  quizzes  at  appropriate  locations  to  help  students  gauge  their 
educational  progress. 

Now  we  are  ready  to  design  the  web  site  for  the  course.  From  the  above  considerations,  there  are  many  needed 
components  of  the  web  site.  Now  the  team  must  examine  the  course  material  and  begin  to  storyboard  it  into  needed 
segments,  and  requisite  interactions  among  the  segments.  As  also  seen  above,  the  overall  site  should  be  designed  so 
that  a student  can  be  guided  through  it,  or  so  that  a student  can  choose  his/her  own  path.  At  this  point  it  is  a good 
idea  to  use  a hierarchy  program  and  place  the  titles  of  all  segments  in  the  spaces  and  begin  to  determine  the  best 
arrangement  and  interactions.  Because  we  are  trying  to  maximize  the  constructivist  element  of  the  web  site  design, 
the  placement  of  the  course  material  must  be  done  with  that  priority.  It  is  very  important  to  remember  that  the 
individual  web  pages  need  to  be  short,  so  a given  chapter  of  material  will  need  to  be  divided  into  appropriate  small 
sections. 
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It  is  also  time  to  begin  to  determine  what  animations  and  simulations  will  be  placed  with  what  sections  and  where  to 
place  drill  exercises.  Drills  are  to  be  placed  at  necessary  points  in  the  objectivist  portion  of  the  site  (such  as  in  the 
learning  of  significant  figures).  If  a programmer  is  part  of  the  team  then  he/she  needs  to  begin  developing  the 
needed  animations,  simulations,  and  drills.  Generous  faculty  members  at  many  universities  are  also  willing  to  share 
their  work,  which  could  save  a lot  of  time.  There  are  also  a number  of  software  packages  that  can  help  in 
constructing  these  portions  of  the  site. 

Once  the  design  of  the  web  site  has  been  determined,  the  conversion  to  html  coding  begins.  Much  of  this  work  can 
be  done  with  word  processors,  although  usually  such  conversions  have  to  be  considerably  modified  to  appear  as 
desired.  Because  most  of  us  are  never  satisfied  with  the  html  conversion  of  word  processors,  it  is  advisable  to  just 
create  all  the  code  with  an  appropriate  web  page  software  suite.  There  are  many  software  packages  available  for 
constructing  web  pages.  We  have  found  Dreamweaver  3 and  Coursebuilder  for  Dreamweaver  3 (8)  to  be  very  good 
packages  for  this  work.  Coursebuilder  can  be  used  to  add  short  quizzes  at  appropriate  places  in  the  sections  of  the 
web  material. 

The  style  of  the  writing  needs  to  be  different  from  that  in  a textbook.  The  writing  should  be  friendlier  and  written 
like  one  would  talk  when  tutoring  a student  in  the  office  for  that  is  what  a good  site  is  — an  electronic  tutor.  A good 
method  is  to  start  with  lecture  notes  and  then  fill  in  details  and  modify  the  tone  of  the  writing.  A good  reference 
point  for  the  writing  is  to  think  about  explaining  the  material  to  your  15-year  old  child  or  grandchild.  The  entire 
course  should  begin  with  a broad  overview;  each  "chapter",  and  each  "section"  of  each  chapter  should  have  an 
overview  and  learning  goals  clearly  stated.  Each  "section"  should  then  be  followed  with  a conceptual  quiz  to  help 
the  students  assess  their  progress. 

The  learning  style  considerations  revealed  that  one  needed  component  was  video,  which  presents  some  problems  if 
students  are  accessing  the  material  from  off-campus,  as  most  will  be  doing.  The  videos  should  be  short, 
approximately  ten  minutes,  and  narrowly  focussed  in  content.  We  produced  short  topical  lectures  on  many  topics  in 
freshman  chemistry  and  compressed  them  for  Real  Streaming  video  presentation  over  the  web.  However  the 
pipeline  to  homes  is  still  too  narrow  for  the  effective  use  of  such  video.  We  have  decided  that  for  the  next  few 
years,  the  best  method  for  delivering  such  videos  to  students  is  via  CD-ROM  disks.  The  video  on  the  disks  can  be 
assessed  from  the  web  course  material,  and  it  is  easy  for  the  students  to  then  watch  at  their  convenience.  The  videos 
should  be  produced  in  their  original  form  with  streaming  video  in  mind  (few  motions,  simple  background,  simple 
scenes,  etc),  so  that  they  will  compress  into  as  small  a file  as  possible.  Later,  when  the  technology  improves  for 
video  delivery  over  the  web,  the  same  videos  can  probably  still  be  used  over  the  web. 

The  synchronous  and  asynchronous  communications  among  the  students  can  be  accomplished  by  a course 
management  software  package  such  as  WebCT,  Blackboard,  Web-Course-in-a-Box,  or  needed  individual 
components  may  be  used.  We  chose  to  use  WebCT  for  the  discussion  forum,  chat  room,  and  mail  aspects  of  the 
web  course.  We  also  use  WebCT  for  on-line  tests  to  be  made  available  to  the  students  a couple  of  days  before  each 
test.  These  on-line  tests  are  of  similar  coverage  and  difficulty  as  the  real  tests.  For  some  students  who  have  the 
computer  capability,  we  also  make  available  a server  for  use  with  Microsoft  Net  Meeting.  Using  Net  Meeting  we 
can  schedule  on-line  office  hours  and  students  can  schedule  group  meetings. 

A web  site  that  has  general  information  about  the  campus  and  specifics  about  on-line  learning  should  also  be 
available  for  the  students.  We  constructed  a site  called  "ChemNet"  that  contains  links  to  the  university  homepage, 
campus  resources,  admissions,  financial  aid,  the  catalog,  the  bookstore,  library  resources,  the  university  on-line 
registration  site,  and  a site  where  the  students  can  obtain  free  e-mail  and  web  pages.  The  site  also  contains  a link  to 
a site  that  discusses  on-line  learning,  that  has  links  to  needed  software,  and  that  has  on-line  learning  style  tests.  This 
site  also  has  links  to  information  about  all  degrees  offered  on  campus,  and  links  to  the  on-line  courses  available  from 
this  campus.  Such  a site  is  essential  to  including  the  students  of  the  on-line  class  into  the  on-campus  scene. 


IMPLEMENTATION 

Now  that  the  site  is  constructed,  it  is  time  for  implementation.  It  is  very  important  to  have  an  on-campus  meeting 
before  the  course  begins.  At  this  one  mandated  meeting,  the  students  get  to  see  the  professor  and  each  other,  work 
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through  some  of  the  details  of  the  class  in  a computer  lab,  and  take  some  of  the  assessment  tests.  I also  prepared  a 
web  site  that  walks  the  students  through  the  various  aspects  of  the  web-based  course  that  is  useful  for  those  students 
who  can’t  attend  the  on-campus  meeting  and  as  a review  for  those  who  did  attend  the  orientation. 

Each  chapter  should  have  group  assignments  that  are  due  every  two  weeks.  Testing  should  be  frequent.  Optimally 
the  students  should  be  given  a test  every  two  weeks  and  group  assignments  due  every  two  weeks  with  the  timing 
such  that  something  official  happens  every  week.  For  a semester  course,  there  will  then  be  eight  proctored  tests, 
eight  on-line  quizzes,  the  final  exam,  eight  group  assignments,  short  quizzes,  and  homework  (due  the  day  of  the  test) 
all  to  be  used  in  some  combination  to  give  a grade.  It  is  preferable  to  require  that  the  tests  be  proctored  either  on  or 
off  campus.  The  validity  of  off-campus  proctors  must  also  be  carefully  checked.  If  this  sounds  like  a lot  of  work  for 
the  students  and  the  professor,  it  is!  An  on-line  course  conducted  in  this  manner  forces  the  students  to  study  as  we 
ideally  want  all  students  to  study,  but  we  know  that  probably  only  about  ten  percent  study  this  intensely.  Professors 
and  students  must  be  warned  about  the  extra  work  before  the  course  starts.  Our  data  show  that  students  finishing 
such  a course  in  freshman  chemistry  scored  57%  better  on  a standardized  test,  so  for  the  serious  student  such  an 
approach  is  worth  the  effort. 

The  professor  must  check  all  possible  student  input  at  least  daily.  It  is  critical  to  keep  the  students  involved  in  the 
course  and  motivated  to  learn.  Nothing  turns  off  a student  as  much  as  a request  for  information  to  sit  in  a mail 
hopper  for  days,  or  for  a posted  message  to  the  discussion  forum  go  totally  unanswered  for  days.  It  is  a good 
procedure  for  the  professor  to  enter  himself/herself  as  a counterfeit  student  in  the  class  so  that  the  professor  can  pose 
as  a student,  see  the  material  from  the  student’s  view,  and  discuss  issues  with  the  students  from  a different 
perspective.  Students  will  discuss  concepts  more  openly  with  other  students  than  they  will  with  a faculty  member. 
Of  course,  usually  someone  begins  to  wonder  who  the  kid  is  who  seems  to  know  all  the  answers!  It  is  also  a good 
stratagem  to  let  the  discussions  continue  on  the  bulletin  board  without  intervention  unless  the  students  are  getting 
way  off  course  on  trying  to  understand  some  concept.  Students  learn  more  when  they  teach  other  rather  than  having 
the  professor  answer  all  questions  immediately.  It  is  also  a wise  procedure  to  sometimes  enter  as  the  counterfeit 
student  and  pose  a question  to  try  to  get  the  discussions  focussed  properly. 

Assessment  tests  and  surveys  are  also  available  on-line  for  ease  of  taking  and  analyzing. 

Upon  implementation  the  professor  becomes  the  guide  through  the  material.  The  web  site  must  also  be  kept 
updated,  which  includes  checking  all  referenced  links  often,  changing  course  content  as  needed,  modifying  and 
adding  new  animations  and  simulations  as  needed,  monitoring  the  use  of  synchronous  and  asynchronous 
communication  tools  so  that  the  most  effective  methods  are  used  from  the  students’  perspective,  modifying  areas  of 
the  site  as  needed  to  meet  the  needs  of  all  learning  styles,  and  continual  use  of  assessment  data  to  make  the  site 
optimally  effective. 


CONCLUSIONS 

Assembling  a web-based  course  is  not  an  easy  task  and  requires  a lot  of  time  and  commitment  of  the  professor  and 
the  team.  After  the  initial  implementation,  the  professor  is  forever  in  a continuous  iterative  process  of  assessment, 
modification,  learning  modem  techniques,  and  implementation.  However,  this  iterative  process  has  always  been  the 
job  of  the  professor.  The  web  era  has  not  introduced  any  new  job  description,  just  different  details  and  a new 
medium  that  can  be  used  to  develop  constructivist  learning  in  a more  effective  manner  than  ever  before.  This  last 
statement  is  a rather  bold  statement,  but  we  believe  that  it  is  true,  especially  for  rather  abstract  subjects.  The  use  of 
computer  simulations  with  feedback  loops,  animations,  drills  for  memorization,  and  computer  models  of  molecules 
and  their  interactions  (again  with  feedback  loops)  afford  a method  for  active  learning  that  was  not  available  until 
recently.  The  web-based  uses  of  these  techniques  all  the  students  to  be  repeated  as  often  as  the  student  desires  rather 
than  just  once  in  class.  The  use  of  the  electronic  bulletin  boards  for  discussions  generally  afford  a deeper  level  of 
discussion  as  students  can  think  more  about  their  comments  than  most  do  in  person-to-person  interactions. 

A major  benefit  of  a web-based  site  from  the  students’  perspective  is  that  the  students  leave  such  a course  with 
better-developed  abilities  to  be  life-long  learners.  They  will  always  have  to  be  learning  on  their  own,  and  the  web  is 
a fantastic  tool  for  education.  They  will  also  have  learned  how  to  approach  the  web  cautiously  regarding  the  validity 
of  information  on  the  web.  They  will  have  developed  skills  in  working  with  other  individuals  using  various 
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technologies.  Most  corporations  now  use  the  web  for  training  employees  and  keeping  employees  modernized  in 
their  knowledge  relating  to  their  jobs.  The  successful  completion  of  a web-based  course  is  then  an  excellent  part  of 
a student’s  education  and  at  least  one  such  course  should  be  required  of  all  students. 

The  presentation  will  more  completely  discuss  all  the  elements  of  design;  the  assessment  results  of  four  semesters  of 
teaching  General  Chemistry,  a chemistry  course  for  science  majors;  and  show  some  of  the  actual  details  of  the  web- 
based  courses. 
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Abstract:  MetaMuse  is  a pedagogical  agent  that  supports  the  fast  generation  of  musical  ideas  and 
which  specializes  in  interactions  that  aim  to  foster  learner  problem-seeking  (i.e.  the  first  stages  of 
creative  problem-solving).  This  paper  provides  a overview  of  MetaMuse  and  then  gives  details  of 
an  evaluation  of  MetaMuse  with  teachers  and  learners.  The  evaluation  results  were  favorable.  In 
particular  the  evaluation  participants  rated  highly  the  learning  support  approach  taken  by 
MetaMuse  (i.e.  getting  students  to  learn  how  to  make  predictions  about  musical  ideas  and  then 
encouraging  reflection  on  what  actually  happened  when  the  idea  was  played  back). 


Introduction 

A pervasive  musical  education  problem  is  that  there  is  often  no  ’’correct”  body  of  knowledge  to  teach  (Johnstone, 
1996,  p.  247).  In  musical  composition  it  is  not  just  a question  of  solving  a problem,  it  is  more  a question  of  seeking 
out  the  nature  of  the  problem  and  then  devising  an  approach  to  solving  it.  More  generally,  Lipman  (1991)  has 
suggested  that  in  the  humanities  and  arts,  the  subject  matter  itself  is  treated  as  essentially  problematic  and  that  it  is 
more  a question  of  problem-seeking  than  problem-solving  (Lipman,  1991,  p.  175).  We  have  extended  this  notion 
(Cook,  1994)  by  proposing  that  when  reflecting  about  creative  problem-solving,  the  first  stage  is  often  to  problem- 
seek,  i.e.  to  define,  formulate,  find,  invent,  or  create  the  problem  before  a method  for  arriving  at  a solution  can  be 
identified. 

This  paper  first  presents  a pedagogical  agent  called  MetaMuse.  The  agent  is  a learning  support  tool  that  has 
been  developed  to  promote  problem-seeking  and  problem-solving  interactions  in  the  area  of  undergraduate  musical 
composition.  We  also  report  on  an  evaluation  of  MetaMuse  with  teachers  and  learners. 


MetaMuse:  A Support  Tool  for  Problem-Seeking  and  Creative  Reflection 

In  the  empirical  work  reported  in  Cook  (1998),  the  author  of  this  paper  describes  a formal  analysis  of  teacher-learner 
dialogues.  One  outcome  of  this  analysis  was  prescriptive  models  of  interactions  described  at  the  level  of 
participants’  goals  and  communicative  acts  (the  latter  being  seen  as  a way  of  achieving  goals).  The  prescriptive 
models  generated  by  the  analysis  of  empirical  data  were  used  in  the  design  and  implementation  of  an  computer- 
based  pedagogical  agent  called  MetaMuse.  The  agent  is  able  to  engage  in  interactions  that  aim  to  promote  problem- 
seeking. 

MetaMuse  can  only  deal  with  a small  task  revolving  around  the  chromatic  transposition  of  a four  note 
phrase.  Furthermore,  MetaMuse  can  not  understand  natural  language.  The  system  is  not  able  to  engage  in  dialogue 
about  the  ’free  text’  inputs  by  the  learner.  As  a result  the  system  sometimes  lets  the  user  input  some  thoughts  on  what 
they  are  doing,  stores  these  explanations  and  reflections  in  an  Interaction  History  and  then  moves  on  to  the  next  part 
of  the  session.  Interactions  with  MetaMuse  thus  typically  take  the  form  of  structured  questioning  by  the  system  and 
menu  and  button  selections  by  the  learner.  However,  within  certain  limitations,  the  system  is  able  to  comment  on  a 
musical  phrase  input  by  the  learner.  MetaMuse  also  has  various  (empirically  derived)  prescriptive  models  of 
expected  interaction  patterns  which  it  uses  as  the  basis  for  problem-seeking  interactions.  An  example  of  the 
MetaMuse  interface  is  shown  in  Figure  1 (note  that  “phrase”  in  the  text  refers  to  a musical  phrase). 
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MetaMuse  has  the  ability  to  provide  the  student  with  a Learner  Profile  (a  summary  of  the  learner  and 
teacher  interactions)  that  can  form  the  basis  for  further  reflective  engagement  and  dialogue  with  fellow  students  or 
with  the  teacher.  This  ties  in  with  one  approach  to  supporting  reflection  and  speculation  about  music  (Auker,  1991). 
Auker  has  suggested  that  students  should  be  allowed  to  develop  the  appropriate  spoken  language  by  interacting  with 
each  other.  They  can  then  adapt  and  take  ownership  of  this  language  as  they  begin  to  internalise  and  ’reflect’  on 
creative  opportunities,  and  hence  build  the  appropriate  mental  structures  of  their  creative  intentions. 


Figure  1 : An  example  of  the  MetaMuse  interface 

MetaMuse  attempts  to  engage  in  a ’mentoring’  style  of  teaching.  Mentoring  is  an  approach  to  teaching  that 
aims  to  support  learners’  creative,  metacognitive  and  critical  thinking,  these  being  essential  to  musical  composition 
and  other  open-ended,  problem-seeking  domains.  Specifically,  mentoring  intends  to  promote  learner  ’creative 
reflection’,  which  is  a constrained  type  of  metacognition.  In  the  domain  of  musical  composition,  creative  reflection 
would  be  defined  as  the  ability  of  a learner  to  imagine  musical  opportunities  in  novel  situations  and  to  then  make 
accurate  predictions  (verbally)  about  these  opportunities.  An  example  of 'imagining  a musical  opportunity’  would  be 
an  utterance  like:  ’I  don’t  want  to  repeat  the  same  note  twice  in  my  phrase’.  An  example  of  creative  reflection  would 
be  a learner  first  writing  a phrase  using  musical  notation,  then  predicting  verbally  how  that  phrase  will  sound,  then 
playing  the  phrase  back  on  a piano  and  finally  evaluating  if  the  prediction  was  accurate  or  not.  To  succeed  at 
creative  reflection  there  should  be  a correspondence  between  what  a learner  predicts  will  happen  and  what  actually 
happens. 

What  the  pedagogical  agent  ‘knows’  is  very  limited.  The  focus  for  the  pedagogical  agent  is  on  the  potential 
that  the  choice  of  musical  intervals  provides  for  musical  material  to  be  sectionalised  into  phrases.  Heuristics  on  how 
this  is  taught  have  been  taken  from  the  interaction  data  (Cook,  1998)  and  augmented  by  some  musical  grouping 
constraints  as  proposed  by  Lerdahl  (1988). 

The  pedagogical  agent  is  capable  of  certain  types  of  symbolic  reasoning  based  on  its  knowledge.  For 
example,  the  pedagogical  agent  ‘knows’  that  ‘an  interval  leap  of  23  or  more  may  indicate  a musical  phrase 
boundary’.  The  agent  also  knows  some  good  teaching  tactics;  for  example,  a tactic  called  ‘critical  probing’  involves 
an  attempt  to  get  the  learner  to  imagine  that  they  are  playing  their  phrase  on  an  instrument,  as  this  can  help  the 
learner  firm  up  their  own  perception  of  the  purpose  of  the  interval  leap  (in  metacognitive  terms  ‘go-above’  and 
regulate  their  knowledge).  This  gives  a good  example  of  the  challenge  of  mentoring  in  a problem-seeking  domain, 
i.e.  mentoring  involves  tutoring  with  non-absolute  rules,  where  there  is  no  right  or  wrong  answer.  Normally,  if  the 
MetaMuse  detects  a large  leap  then  ‘critical  probing’  would  be  selected  as  an  appropriate  intervention. 

State  Transition  Networks,  i.e,  prescriptive  models,  derived  from  dialogue  modeling  (Cook,  1998)  are  used 
to  plan  interactions.  Often,  more  than  one  exit  was  possible  from  a state  node.  Interactions  are  structured  by  offering 
all  of  these  choices  to  a learner  as  ‘Respond’  menu  options  (these  options  would  vary  from  one  node  to  the  next). 
Preferred  options  would,  it  was  decided,  sometimes  be  indicated  to  the  learner  if  there  was  a strong  basis 
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(empirically  based)  for  the  pedagogical  agent  making  a suggestion  to  the  learner.  Thus,  the  State  Transition 
Networks  are  used  to  give  options  for  choice  and  provide  expectations  of  answers.  A measure  of  confidence  in  the 
degree  of  learner  co-operation  can  be  built  up  over  time,  based  on  the  degree  to  which  a learner  fails  to  match  the 
pedagogical  agent’s  expectations  (e.g.  how  often  do  they  ignore  advice  given  by  the  pedagogical  agent?). 


Evaluation  of  MetaMuse 

The  evaluation  was  in  two  parts.  In  part  one  five  individuals  evaluated  MetaMuse.  In  part  two  six  pairs  of 
collaborating  learners  were  used  to  help  evaluate  MetaMuse.  Thus  the  total  number  of  participants  in  the  evaluation 
was  1 7.  We  will  refer  to  the  participants  in  part  1 of  the  evaluation  as  participant  numbers  1 to  5.  We  will  refer  to  the 
participants  in  part  2 of  the  evaluation  as  participant  numbers  6 to  17. 

The  Evaluation  Study  Set-Up 

The  method  that  was  used  for  the  evaluation  (parts  1 and  2)  was  questionnaire.  The  specific  aim  of  the  evaluation 
was  to  get  initial  feedback  (by  the  use  of  questionnaires)  from  music  teachers,  educational  technology  researchers 
and  students  on  six  related  questions: 

1.  Details  of  participants’  current  post,  professional  qualifications  and  experience  with  music  teaching  and 
educational  technology. 

2.  How  interesting  participants  found  MetaMuse. 

3.  How  useful  participants  found  the  way  that  MetaMuse  promotes  creative  reflection  about  a musical  phrase. 

4.  Participants’  assessment  of  the  potential  that  MetaMuse  has  for  assisting  in  undergraduate  composition  classes. 

5.  Participants’  assessment  of  how  useful  they  found  the  guidance  provided  by  MetaMuse. 

6.  Participants’  assessment  of  how  useful  they  thought  the  learning  approach  used  by  MetaMuse  was  for  musical 
composition  education  (i.e.  learning  how  to  make  predictions  and  reflecting  on  what  actually  happened). 

There  were  various  aims  behind  the  above  questions.  The  second  question  was  used  as  a general  question  to  catch 
user  impressions  of  the  pedagogical  agent.  Another  aim  of  the  questions  was  to  see  if  users  felt  that  the  agent  could 
reproduce,  at  the  formal  level,  interaction  that  were  similar  in  structure  to  a human  pedagogical  agent  whose  goal 
was  ‘creative  reflection’  (the  third  and  sixth  questions).  The  pedagogical  agent,  in  its  current  implementation,  has  no 
natural  language  understanding;  there  is  a lot  of  processing  going  on  behind  the  scenes  (although  response  time  was 
very  fast)  and  some  interface  activity.  Given  this  limitation,  the  fourth  question  attempts  to  identify  the  applicability 
of  the  system,  could  it  be  used  in  the  classroom  with  students?  The  fifth  question  attempted  to  ascertain  how  the 
participants  evaluated  the  help  screens  and  messages  provided  by  the  pedagogical  agent. 

Part  1 of  the  evaluation  was  conducted  in  London  and  at  a Department  of  Music  in  a University  College  in 
the  north-east  of  England  in  May  and  June,  1998.  This  part  of  the  evaluation  was  conducted  with  one  AI-Music- 
Education  researcher/lecturer  (in  London)  and  with  four  musical  teachers  (at  the  Department  of  Music).  Part  2 of  the 
evaluation  took  place  at  a British  University  in  the  south-east  of  England  in  March  1999.  In  part  2,  six  pairs  of 
learners  worked  with  MetaMuse  in  each  session.  The  twelve  participants  in  part  2 of  the  evaluation  ranged  from 
undergraduate  students,  postgraduate  students,  research  fellows  and  members  of  staff  (teaching  and  support).  The 
participants  in  Part  2 had  varying  levels  of  musical  ability.  For  the  whole  evaluation  eleven  participants  were  male 
and  six  were  female.  Each  session  in  both  parts  of  the  evaluation  lasted  between  30  and  50  minutes. 

The  evaluation  subjects  (one  participant  and  one  observer  per  session  in  Part  1 ; pairs  of  participants  and 
two  observers  per  session  in  Part  2)  sat  in  front  of  a composer  workstation,  which  consisted  of  a Macintosh  Centris, 
with  MetaMuse  installed,  attached  to  a midi-device  and  speakers.  Each  session  involved  the  participant  being  asked 
to  carry  out  a small  composition  task  (10  minutes  of  the  session  were  set  aside  for  participants  to  read  the  notes  and 
ask  questions  of  clarification;  in  Part  2 the  pairs  were  simply  asked  ‘to  work  together  on  the  task’).  Briefly,  the 
compositional  task  was  for  the  participant  to  attempt,  using  MetaMuse,  to  create  a phrase  by  the  repeated  chromatic 
transposition  of  an  initial  four  note  motive  (C  C#  F#  G).  When  using  MetaMuse  to  compose  a phrase,  participants 
were  asked,  in  the  task  sheet  and  by  MetaMuse,  to  limit  the  approach  used  (when  transposing  the  phrase)  to 
repetition,  contrast  and  trajectory.  The  reason  for  taking  this  constrained  approach  (of  only  allowing  repetition, 
contrast  and  trajectory)  was  that  MetaMuse  knows  more  about  how  to  interact  than  it  does  about  music  (later 
versions  will  improve  on  this  lack  of  musical  knowledge).  Thus,  constraining  the  types  of  activities  that  the 
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composer  was  allowed  to  undertake  would,  it  was  hoped,  give  MetaMuse  a chance  of  being  able  to  analyse  what  the 
participant  was  doing  (and  hence  MetaMuse  would  have  a reasonable  basis  for  mentoring  interactions). 

Participants  were  informed  that  the  current  implementation  of  MetaMuse  could  only  recognize  the  following 
approaches  (which  are  based  around  the  work  of  Lerdahl,  1988),  and  were  asked  to  limit  themselves  to  variations  on 
one  of  these  approaches  when  composing  a phrase:  octave  leaps,  descending  trajectory,  ascending  trajectory,  large 
leaps,  repeated  transposition  and  small  phrase.  Each  evaluation  participant  individually  filled  in  a questionnaire 
following  a session.  It  is  these  questionnaires  which  are  used  as  the  basis  for  our  evaluation.  Following  the  very  first 
evaluation  session  some  minor  changes  were  made  to  MetaMuse.  Consequently,  sessions  2 to  1 1 (i.e.  with 
participants  2-17)  were  conducted  with  a slightly  improved  version  of  the  pedagogical  agent. 


Results  and  discussion 

Table  1 shows  average  response  scores  (out  of  possible  maximum  score  of  5)  to  interview  questions. 


q4 

a5 

cl6 

Av.  4.06 

3.35 

3.65 

2.76 

4.35 

Table  1:  Average  Response  Scores  (out  of  5)  to  Interview  Questions 

The  number  of  study  participants,  at  17,  was  small  and  so  some  caution  is  required  when  attaching  significance  to 
these  results.  Given  this  reservation,  however,  the  results  seem  favorable:  the  total  average  response  score  for 
questions  2 to  6 was  3.63  (out  of  5).  We  will  now  discuss  the  above  results  in  the  context  of  the  underlying  purpose 
of  the  questionnaire,  which  was  outlined  above. 

Impression  of  the  Pedagogical  Agent  (q2) 

With  an  average  score  of  4.06  the  overall  impression  of  MetaMuse  seems  good.  The  comments  made  by  participant 
5 (a  former  head  of  department  of  music)  were  encouraging: 

“Compositional  value  in  teaching  very  useful.  Patterns  are  quite  limiting  - which  make  a good  test  of  both 
‘learning’  and  ‘ingenuity’.  It  would  be  further  use  in  teaching  to  [be  able  to]  play  around  with  the  pattern 
(invert,  retrograde)  and  with  tempo.”  (Participant  5) 

Participant  13,  who  has  a degree  in  Music  (and  who  specializes  in  musical  performance)  made  the  following 
comment: 

“With  the  facet  that  we  were  looking  at,  it  was  interesting  to  hear  what  the  transposed  phrase  sounded  like  ...” 
(Participant  13) 

Comments  against  this  question  (q2)  also  tended  to  make  some  suggestions  for  improvement.  Following  the  above 
comment,  Participant  13  went  on  to  make  a suggestion: 

“1  would  like  to  see  it  incorporate  rhythm  and  volume  of  sound  and  a few  musical  variations  like  staccato  and 
legato,  varying  tempi  etc.”  (Participant  13) 


Were  the  Interactions  Able  to  Promote  ^Creative  Reflection*?  (q3  and  q6) 

The  average  scores  of  3.35  (for  question  3),  and  4.35  (for  question  6)  we  feel  are  further  indicators  that  MetaMuse  is 
‘on  the  right  track’.  However,  one  comment  made  against  question  3 recognized  the  need  for  a more  sophisticated 
analysis  of  what  has  already  been  said  by  the  learner,  in  order  to  avoid  repetition: 
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“It  would  probably  be  more  helpful  if  it  were  able  to  pick  up  on  key  words  in  the  responses  and  therefore  not 
ask  questions  which  have  already  been  answered.  However,  the  process  of  evaluation  is  useful,  although  I’d  be 
interested  to  see  how  it  coped  with  larger  structures.”  (Participant  4) 

The  issue  raised  in  the  ending  of  participant  4’s  comment  above  (“I’d  be  interested  to  see  how  it  coped  with  larger 
structures”)  was  also  picked  up  by  participant  5 (“Needs  to  explore  further  how  small  components  can  contribute  to 
a larger  frame  (structure  : form)”).  Indeed,  in  her  comments  with  respect  to  question  3,  participant  8 (who  plays 
classical  guitar)  made  the  following  useful  suggestions  about  the  way  MetaMuse  tutors  phrases: 

“Not  always  clear  quite  what  was  required  — needed  “tuition”  element  — e.g.  we  are  now  going  to  look  at 
phrasing  marks;  the  effect  of  phrasing  marks  on  a particular  sequence  — now  try  it  out.  Difficult  to  say  how 
would  phrase  if  cannot  try  it  out.  Difficult  (for  me)  to  say  how  would  play  without  seeing  it  written  in  standard 
notation.”  (Participant  8) 

Future  work  could  develop  MetaMuse ’s  ability  to  work  with  larger  phrases  and  sections  (i.e.  structures).  The 
question  of  musical  notation  is  more  problematic.  In  the  design  of  MetaMuse  we  deliberately  avoided  the  use  of 
notation  and  opted  for  numerical  transposition  values.  The  reason  for  this  was  that  we  wanted  learners  with  varying 
musical  backgrounds  to  be  able  mentally  image  their  musical  ideas.  Hence  the  use  of  the  numbers  would,  we  hoped, 
assist  in  this  process  of  imaging  (we  could  not  assume  that  users  were  able  to  read  music).  However,  future  versions 
of  MetaMuse  could  include  the  ability  to  handle  musical  notation.  The  impact  of  doing  this  on  the  types  of 
interactions  and  reflections  that  would  then  take  place  is,  however,  unclear. 

The  responses  to  question  6 (which  received  the  highest  average  score  for  any  question,  at  4.35)  was  very 
positive,  and  as  such  several  selected  comments  are  included  below: 

Participant  2:  Except  on  a one  to  one  basis,  I cannot  imagine  any  other  way  of  achieving  this  experience.  This 
does  test  and  help  develop  memory  and  critical  thinking  in  a relatively  non-threatening  manner. 

Participant  3:  Most  undergraduates  would  be  at  too  high  a level.  Better  on  Primary/Secondary. 

Participant  4;  Overall,  I think  this  could  be  very  useful  in  terms  of  a learning  tool  in  encouraging  students  to 
think  about  both  local  events  and  (potentially)  larger  scale  events;  and  the  process  of  constructing  events  to 
create  cohesive  work. 

Participant  5:  Excellent  introduction  to  awareness  of  sound. 

Participant  8;  Hearing  rather  than  seeing  a representation  of  the  phrase  is  useful. 

Participant  1 1 : This  is  the  basis  for  all  my  own  compositions,  and  I am  trying  to  encourage  this  approach  in 
other  subject  areas  by  assessing  reflections  rather  than  task  solution. 

Participant  12:  Would  be  useful  to  music  education  even  at  late  primary/secondary  level. 

Participantl3:  It  is  a good  approach  because  encourages  composition  in  the  head  without  playing  first,  which  is 
a very  useful  skill. 

Participant  17:  Quite  a lot  of  potential. 


Interestingly  participant  3 (who  was  the  only  music  teacher  participant  not  to  give  question  6 a score  of  5 out  of  5)  is 
not  actually  a composition  lecturer,  unlike  participants  2,  4 and  5,  who  do  teach  composition  at  undergraduate  level 
and  who  do  see  the  ‘full’  potential  for  MetaMuse  to  teach  creative  reflection. 


Could  MetaMuse  be  used  in  the  Classroom  with  Students?  (q4) 

As  far  as  Part  1 of  the  evaluation  went  (i.e.  participants  1 to  5),  opinion  on  question  4 seemed  to  be  split  between 
those  who  do  not  teach  composition  (participants  1 and  3,  who  thought  that  MetaMuse  may  be  appropriate  in 
schools)  and  those  who  do  teach  composition  (participants  2,  4 and  5,  who  thought  it  would  be  useful  for  first  year 
undergraduates).  Participant  12  (who  is  a research  fellow  in  computing,  but  who  plays  the  bagpipes)  in  his 
comments,  quoted  above  in  the  context  of  question  6,  appears  to  agree  with  participants  1 and  3.  Subject  13  also 
added  to  the  split  decision  on  who  MetaMuse  should  be  aimed  at; 

“I  would  see  this  tool  to  be  more  appropriate  at  the  secondary  or  6^*^  form  level.  I think  it  would  be  a bit  basic  for 
undergraduate  level.”  (Participant  13) 
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If  MetaMuse  were  given  more  domain  knowledge  then  we  feel  that  it  would  appear  more  convincing  to  non- 
composition teachers  and  undergraduate  users.  Improving  MetaMuse ’s  domain  knowledge  would  provide  an 
interesting  project  for  future  work.  However,  once  again  we  should  point  out  that  the  implications  for  MetaMuse  (in 
terms  of  the  type  of  interactions  and  reflective  thinking  that  it  would  then  encourage)  of  ‘adding’  more  domain 
knowledge  remains  unclear.  The  approach  taken  in  MetaMuse  is  to  constrain  what  the  composer  is  allowed  to  do. 
This  is  a perfectly  valid  compositional  learning  technique,  often  called  a study  or  exercise. 


Evaluation  of  the  Help  Screens  and  Messages  Provided  by  MetaMuse  (q5) 

This  question  received  the  lowest  average  score  (2.76).  The  help  screens  and  the  language  used  by  MetaMuse 
clearly  need  improving.  Participant  2 (when  actually  answering  question  2)  probably  put  his  finger  on  the  weakness 
of  the  help  screens  and  messages: 

“The  language  and  display  of  text  needs  a radical  rethink!  A session  with  students  on  the  language  used  would 
be  essential  - and  a good  thing  to  do  anyway  as  this  kind  of  formal  language  use  is  not  (but  should  be) 
encouraged.”  (Participant  2) 

The  many  responses  to  question  5 provided  us  with  a useful  source  of  user  data  on  how  to  go  about  improving  the 
usability  of  MetaMuse, 


Conclusions 

This  paper  has  described  the  evaluation  of  MetaMuse,  a pedagogical  agent  that  aims  to  foster  learner  problem- 
seeking and  creative-reflection.  The  questionnaire  approach  to  evaluation  provided  us  with  a useful  snapshot  of  user 
opinions  about  MetaMuse.  However,  the  above  evaluation  of  the  teaching  agent  did  not  give  us  any  insight  into  the 
following  question:  what  are  the  interactive  means  by  which  learning  agents  engage  in  cooperative  problem- 
seeking? Future  work  will  focus  on  both  answering  this  question  (by  undertaking  detailed  dialogue  interaction 
analysis  of  part  2 of  the  evaluation)  and  on  improving  the  learning  support  provided  by  MetaMuse  by  drawing  on 
the  evaluation  results  discussed  above. 
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Abstract;  A collective  case  study  examined  the  change  in  pedagogy  that  occurred  in  a 
postgraduate  Information  Technology  in  Education  subject  that  utilised  World  Wide  Web 
technology.  Two  implementation  cycles  were  examined.  The  model  that  surfaced  from  the 
first  case  correlates  with  a Radical  Constructivist  approach  (von  Glasersfeld  1995)  and  the 
model  that  emerged  from  the  second  case  maps  with  the  Beyond  The  Information  Given 
(BIG)  constructivist  framework  (Perkins  1991).  This  paper  describes  the  parallels  of  theory  to 
practice  and  raises  implications  for  the  design  of  web-based  learning  environments. 


Introduction 

This  study  examined  the  pedagogical  change  that  occurred  in  a course  that  was  delivered  using 
network-based  technologies.  Using  a collective  case  study  method  (Stake  1997),  the  researcher  (primary 
author)  was  a participant  observer  in  a postgraduate  course  offered  by  the  Graduate  School  of  Education 
at  the  University  of  Wollongong  over  two  years.  The  research  focus  was  to  examine  the  interactions  that 
occurred  amongst  the  students  and  the  instructor.  In  addition  to  participant  observation,  data  was  gathered 
from  student  and  instructor  interviews  and  via  a collection  of  artifacts  such  as  student  work,  computer- 
mediated  communication  transcripts  and  email  messages.  The  model  that  emerged  from  each 
implementation  parallels  two  theoretical  constructivist  frameworks.  Case  One  parallels  Radical 
Constructivism  (von  Glasersfeld  1995)  whilst  Case  Two  parallels  a Beyond  the  Information  Given  (BIG) 
approach  (Perkins  1991). 

As  part  of  the  initiative  by  the  Faculty  of  Education  to  investigate  flexible  delivery  strategies,  in  1996  the 
postgraduate  course:  Implementation  and  Evaluation  of  Technology-Based  Learning  was  chosen  as  a pilot  study 
to  explore  the  use  of  internet  and  videoconferencing  technologies.  The  introduction  of  these  technologies 
enabled  students  to  experience  the  use  of  technology  whilst  learning  about  its  use  in  an  educational  context.  Two 
different  implementations  were  undertaken  one  in  each  year. 

In  the  first  case,  there  were  two  geographically  separate  classes.  Eight  students  met  on  campus  and  six 
students  met  at  another  site  80  kilometres  away.  Both  classes  were  held  on  the  same  evening  for  three  hours  over 
a semester  of  fourteen  weeks.  Videoconferencing,  a subject  Web  site  and  computer-mediated  communication 
tools  facilitated  interaction  between  the  two  sites.  The  instructor  physically  attended  each  site  on  alternate 
weeks.  The  researcher  physically  attended  every  on-campus  class. 

The  content  material  was  structured  so  that  a new  topic  was  addressed  each  week.  The  first  five  weeks 
were  structured  as  instructor-led  sessions  delivered  via  videoconferencing  and  computer-mediated 
communication  tools.  The  remaining  nine  weeks  were  structured  as  student-led  sessions.  Assessment 
requirements  were  based  on  individual  work  and  consisted  of  three  assignments.  The  first  assignment  was  a 
seminar  presentation  that  was  to  be  conducted  as  a collaborative  exercise  between  two  students,  one  from  each 
site.  Seminar  presentations  began  from  Week  Six  of  the  course.  Students  had  to  facilitate  a discussion  for  the 
entire  evening  and  had  to  create  a web  page  and  provide  online  activities  to  engage  both  sites  as  one  class.  A 
grade  was  given  based  on  a theoretical  paper  that  accompanied  the  presentation.  The  last  two  assignments  were 
due  at  the  end  of  the  semester.  One  involved  developing  a portfolio  of  resources  that  each  student  found  relevant 
in  terms  of  the  content  of  the  course  and  the  other  involved  producing  an  evaluation  report  of  the  implementation 


248 


of  the  course  as  a technology-based  learning  project  or  of  an  educational  software  package  of  the  students’ 
choice. 

The  lessons  learnt  from  this  case  were  used  to  redesign  the  teaching  and  learning  environment  for  the 
subject  offering  in  the  following  year.  For  this  second  case,  the  subject  was  implemented  using  internet 
technology.  There  again  were  two  geographically  separate  classes.  Eleven  students  met  on  campus  and  six 
students  met  at  the  remote  site.  The  two  classes  ran  on  different  evenings  for  three  hours.  During  the  fourteen 
week  semester,  students  attended  eight  class  meetings,  which  were  scheduled  approximately  every  second  week 
(Weeks  1,  2,  4,  6,  8,  10,  11  and  14),  and  participated  in  asynchronous  and  synchronous  online  discussions  during 
the  non-meeting  weeks.  A web  site  facilitated  interaction  among  the  students  and  instructor  outside  class  time. 
The  instructor  physically  attended  every  face-to-face  meeting.  The  researcher  attended  the  face-to-face  classes 
held  on  campus. 

Assessment  was  restructured  and  comprised  four  assignments.  The  first  assignment  was  due  in  Week 
Four  and  involved  writing  a paper  about  evaluation  theory  and  its  relevance  to  technology-based  learning.  A 
Web  page  on  an  agreed  topic  was  to  be  completed  by  Week  Eight.  Students  then  had  to  evaluate  all  the  Web 
pages  and  produce  an  evaluation  report  by  Week  Eleven.  The  last  assignment  involved  developing  and 
presenting  an  evaluation  proposal.  It  was  due  in  the  last  week  (Week  Fourteen)  and  was  completed  and  graded  as 
a group  project.  These  assessable  tasks  were  structured  so  that  the  first  three  tasks  could  be  used  as  resources  for 
the  final  group  project.  Content  material  was  provided  to  help  with  the  completion  of  an  assessment  task.  The 
concept  of  a Web  page  replaced  the  need  for  content  to  be  presented  in  a weekly  sequence  in  class  time  as  the 
content  was  now  available  in  electronic  form  and  accessible  at  a time  and  place  convenient  to  the  students. 

The  instructor  incorporated  several  online  activities  during  the  semester.  The  first  online  activity  involved 
students  working  in  small  groups  to  discuss  asynchronously,  the  content  required  for  the  first  assignment.  Each 
group  worked  on  different  content  and  posted  a summary  of  their  discussions  to  the  subject  web  site.  The 
intention  was  that  students  could  use  the  summaries  produced  by  the  different  groups  as  resources  for  the  first 
assignment.  Other  online  activities  included  synchronous  and  asynchronous  discussions  regarding  particular 
content. 

Constructivism — Its  Application  in  Technology-Based  Learning  Environments 

Jonassen,  Mayes  & McAleese  (1993)  argue  that  “constnictivistic  learning  environments  are  most 
effective  for  an  advanced  knowledge  acquisition  stage  of  learning”,  which  is  mostly  required  and  promoted  in 
university  education.  The  underlying  premise  that  purports  a constructivist  approach  is  the  epistemological  belief 
that  knowledge  is  not  an  objective  reality,  that  is,  exists  “outside  the  mind”  but  rather,  it  is  internally  constructed. 
“Knowledge  is  a function  of  how  the  individual  creates  meaning  from  his  or  her  experiences;  it  is  not  a function 
of  what  someone  else  says  is  true”  (Jonassen,  Davidson,  Collins,  Campbell,  & Haag  1995,  p.  11).  What 
constitutes  a constructivist  approach  to  teaching  and  learning,  particularly  within  a technology-based  learning 
environment?  Much  of  the  current  educational  technology  literature  is  devoted  to  this  issue  and  several 
frameworks  have  been  presented  (for  example:  Jonassen  et  al.  1993;  Jonassen  et  al.  1995,  p.  13-14;  Duffy  & 
Cunningham  1996,  p.  177;  Robyler,  Edwards  & Havriluk  1997,  pp.  70-72;  Jonassen  1999,  pp  2-7,  193-201). 
From  a coalescence  of  this  literature,  four  generic  principles  essentially  characterise  a constructivist  learning 
environment: 

1.  Learning  is  a process  of  construction.  Learners  are  provided  the  opportunity  to  actively  create  their  own 
meaning  from  their  current  and  past  experiences  and  prior  knowledge  rather  than  passively  learn  the 
instructor’s  interpretation  of  the  content. 

2.  Learning  occurs  through  social  negotiation  of  meaning.  Learners  are  able  to  collaborate  with  peers. 

3.  Learning  is  contextually  mediated.  Learners  are  immersed  in  authentic  contexts. 

4.  Reflective  thinking  is  an  ultimate  goal.  Learners  are  encouraged  to  think  about  their  own  thinking  processes. 

The  degree  in  which  these  generic  principles  are  manipulated  has  given  rise  to  various  interpretations  of 
constructivism  ranging  from  “mild”  (Boudourides  1 998)  or  “trivial”  (von  Glasersfeld  1 993  cited  in  Geelan  1 997) 
constructivism  to  “radical  constructivism”  (von  Glasersfeld  1995).  Perkins  (1991,  p.  20)  states:  “Almost  all 
educators  and  psychologists  are  constructivists  of  some  stripe  these  days.  But  battles  rage  concerning  just  how 
constructive  one  should  be.”  Phillips  (1995,  p.  5)  claims:  “As  in  all  living  religions,  constructivism  has  many 
sects...”  Several  frameworks  for  comparing  the  different  forms  of  constructivism  are  presented  in  the  literature 
(see  Perkins  1991;  Phillips  1995;  Geelan  1997;  Roblyer  et  al.  1997;  Boudourides  1998).  Perkins  (1991) 
differentiates  two  forms  of  constructivism:  Beyond  the  Information  Given  (BIG),  and  Without  the  Information 
Given  (WIG).  These  two  forms  are  based  on  the  amount  of  intervention  provided  by  the  instructor.  Phillips 
(1995)  presents  a complex  comparison  framework  made  up  of  three  dimensions:  (1)  The  individual  construction 
of  knowledge  versus  the  construction  of  human  knowledge  in  general  (p.  7);  (2)  “Humans  the  creators  versus 
nature  the  instructor”  (p.  7).  That  is,  'is  individual  knowledge  created  actively  in  the  mind  or  imposed  from  the 
outside?’;  (3)  The  active  construction  process  can  be  described  in  terms  of  individual  cognition  or  in  terms  of 
social  and  political  process  (p.  9).  Geelan  (1997)  provides  a synthesis  of  six  different  forms  of  constructivism 
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based  on  a two-dimensional  comparison  model:  (1)  Individual  versus  social  learning,  (2)  Epistemological  belief, 
that  is,  objectivism  versus  relativism.  Robyler  et  al.  (1997,  pp.  65-70)  describes  various  degrees  of 
constructivism  in  the  form  of  different  learning  theories  and  the  amount  of  intervention  provided.  For  instance, 
they  claim  that  cognitive  flexibility  theory  is  a form  of  radical  constructivism  as  it  suggests  little  direct 
instruction.  Boudourides  (1998)  presents  various  “streams”  of  constructivism  along  a continuum  from  “mild”  to 
“radical”.  Petraglia  (1998),  however,  presents  a different  argument  with  respect  to  the  application,  or  as  he  states 
“misapplication”,  of  constructivism.  Petraglia  states:  “educational  technologists  have  fundamentally 
misunderstood  the  challenges  posed  by  constructivism.  This  is  seen  in  an  approach  to  contextualising  learning 
that  I call  preauthentication''  (p.  58).  He  goes  on  to  explain: 

A problem  with  the  goal  of  authenticating  learning  arises  when  we  remember  that  constructivism  argues  that  the 
world  is  not  understood  independently  of  our  experiences,  and  that,  therefore,  any  sense  of  authenticity  can  be  neither 
predetermined  nor  preordained...  educational  technologists  have  responded  to  this  epistemological  dilemma  by 
creating  problems  and  environments  that  they  have  determined  to  be  authentic;  we  might  call  this  practice 
preauthentication,  (p.  58-59) 

A synthesis  of  the  different  conceptions  of  constructivism  is  provided  in  Figure  1.  It  is  a somewhat 
simplistic  view  but  merely  serves  as  an  illustration  to  situate  the  differences  in  constructivist  approaches  that 
surfaced  from  the  collective  case  study. 


Preauthenticated  learning  environments 
(Petraglia  1998) 


Cognitive  Flexibility  Theory  (Spiro,  Feltovich, 
Jacobson,  & Coulson  1991) 


Beyond  information  given  (BIG) 
(Perkins  1991) 


Radical  constructivism  (von  Glasersfeld  1995) 
Without  information  given  (WIG)  (Perkins  1991) 

► 


More  — Amount  of  Intervention  provided  — Less 
(Examples  of  interventions  include:  assistance/guidance  from  instructor;  the  nature  and  structure  of 

tasks  assigned  to  students) 

Figure  1 : Different  conceptions  of  constructivism 

Comparing  the  Two  Cases 

Interestingly,  the  model  that  emerged  from  each  course  implementation  parallels  with  the  characteristics 
of  constructivism  provided  at  the  two  extremes  in  Figure  1 : Case  One  exemplifies  a radical  constructivist 
learning  environment  whilst  Case  Two  is  an  example  of  the  Beyond  the  Information  Given  approach.  (See 
Hedberg  & Corrent-Agostinho  1999  for  a discussion  of  the  themes  that  emerged  from  each  case.)  Both  cases  are 
described  in  terms  of  how  the  four  generic  constructivist  principles  identified  above  were  implemented  in 
practice  and  how  the  technology  was  incorporated, 

1.  Learning  is  a process  of  construction 

The  following  description  illustrates  the  difference  in  emphasis  of  the  knowledge  construction  process  in 
the  two  cases.  Students  in  Case  One  experienced  an  ill-structured  knowledge  domain  (Spiro  et  al.  1991)  whereas 
students  in  Case  Two  were  provided  with  learning  scaffolds  via  the  progressive  nature  of  the  assessment  tasks. 

Case  One:  Although  the  course  content  material  was  pre-determined,  the  pilot  nature  of  the  course 
delivery  afforded  students  the  opportunity  to  experience  a “real-life”  technology-based  learning  implementation, 
that  is,  an  ill -structured  knowledge  domain.  Different  computer-mediated  communication  tools  were  introduced 
during  the  course  and  the  entire  class  (instructor  included)  was  learning  about  computer-mediated 
communication  by  experiencing  it.  The  role  that  technology  played  was  as  much  the  message  as  the  medium. 
Students  were  introduced  to  the  technology  to  Icam  about  it.  As  such,  the  technology  was  not  transparent  and 
integrated  seamlessly  in  the  course,  it  was  as  much  the  focus  as  the  content  material.  The  following  example  of 
student  feedback  highlights  the  significance  of  such  a learning  environment. 

The  subject  was  a valuable  learning  experience  despite  the  technical  glitches,  periodic  support  problems  and 
communication  difficulties.  We  learnt  as  much  about  implementation  of  technology-based  learning  through  the  failed 
attempts  to  innovate  as  we  did  through  the  more  traditional  presentations  and  reading.  There  was  a very  heavy 
workload  and  a real  danger  of  cognitive  overload  in  this  course.  It  challenged  us  to  extend  ourselves,  perhaps 
unrealistically  in  terms  of  our  part-time  status  and  work  commitments,  but  we  mostly  rose  to  the  occasion. 

Students  experienced  tolerance  for  ambiguity  (Jonassen  & Grabowski  1993).  They  had  to  manage  their 
own  learning  experience  and  had  to  facilitate  their  own  discussion  when  the  instructor  was  not  physically 
present.  Whilst  some  students  rose  to  the  occasion,  as  exemplified  in  student  comments  such  as:  “There’s  one 
thing  about  all  these  hiccups  [referring  to  the  use  of  the  technology].  They  make  you  think  and  you  really  learn 
how  to  cope  with  things  that  don’t  work  and  strategies  to  try  to  make  them  work.“  (Evaluation  Report);  others 
found  the  teaching  and  learning  process  challenging,  for  example:  “I  found  the  lecturer’s  style... vague,  loose  and 
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unstructured.  In  some  tutorials  I really  had  no  idea  of  what  was  going  on,  or  what  was  expected  of  me.  I feel  I 
was  not  alone.  I found  this  style  frustrating  and  annoying.”  (Evaluation  Report) 

All  assessment  was  based  on  individual  work  and  was  not  used  to  scaffold  the  learning  process.  Specific 
assessment  criteria  were  not  provided.  The  assignments  were  open-ended  and  tailored  to  the  individual.  For 
example,  the  portfolio  was  based  on  individual  student  relevance.  There  was  no  set  format  for  the  evaluation 
report  discussed.  These  two  assessments  were  very  individual  in  nature.  It  was  left  to  the  students  to  decide  how 
to  approach  the  task. 

Case  Two:  The  second  implementation  was  structured  so  that  the  assessment  tasks  drove  the  learning 
process.  The  assessment  tasks  acted  as  scaffolds  as  each  task  informed  the  next  subsequent  task.  Student 
feedback  from  the  end-of-subject  questionnaire  (N=15)  indicated  that  the  majority  of  students  thought  the 
sequence  of  the  assessment  tasks  was  appropriate.  There  were  several  students  that  commented  that  the  first 
assignment  was  “too  much  too  soon”  (4  students).  For  example,  one  student  commented:  “I  felt  the  first 
assignment  was  too  early  to  have  formed  any  views  in  the  subject”. 

The  technology  was  integrated  into  the  learning  process.  Computer-mediated  communication  tools  were 
introduced  at  the  beginning  of  the  subject  and  students  were  given  the  opportunity  to  learn  how  to  use  the  tools 
before  online  discussions  tool  place.  The  technology  was  used  as  a vehicle  for  students  to  interact  with  each 
other.  It  became  more  transparent  in  this  case  than  in  Case  One. 

The  themes  that  surfaced  in  this  case  suggest  that  students  experienced  a lesser  degree  of  tolerance 
ambiguity  than  in  Case  One.  The  data  suggests  the  following  factors  were  influential:  The  scaffolding  nature  of 
the  assessment  tasks,  the  instructor  being  physically  present  with  each  class  during  meeting  weeks,  and  more 
stringently  defined  assessment  criteria  than  in  Case  One. 

2.  Learning  occurs  through  social  negotiation  of  meaning 

The  nature  in  which  learners  were  able  to  discuss  content  issues  with  peers  differed  between  the  two 
cases.  In  Case  One,  the  interaction  amongst  students  was  predominantly  of  a synchronous  nature  that  occurred 
on  a weekly  basis.  That  is,  interaction  amongst  students,  both  face-to-face  and  online,  occurred  mostly  during 
class  time.  In  addition,  there  was  little  opportunity  for  students  to  reflect  and  build  on  content  discussions  as 
discrete  content  “chunks”  were  discussed  each  week.  In  Case  Two,  the  technology  was  used  to  encourage  both 
synchronous  and  asynchronous  discussions.  The  asynchronous  online  mode  afforded  students  the  opportunity  to 
build  on  the  discussion  during  the  course.  Also,  the  nature  of  the  assessment  tasks  encouraged  student 
collaboration.  For  example,  collaboration  was  encouraged  in  the  first  online  task  as  students  discussed  content  in 
small  groups  and  each  group  produced  an  online  summary.  Also,  the  final  assignment  was  structured  as  group 
project. 

3.  Learners  are  immersed  in  authentic  contexts 

The  degree  of  “authenticity”  differed  between  the  two  implementations.  The  description  below  explains 
how  students  were  afforded  with  an  authentic  context  in  Case  One  and  a pre-authenticated  context  (Petraglia 
1998)  in  Case  Two. 

Case  One:  The  “real  world”  task  was  the  course  itself.  Due  to  the  pilot  nature  of  the  course  delivery 
students  experienced  problems  that  were  not  pre-determined  by  the  instructor.  They  were  also  able  to  evaluate 
the  implementation  as  part  of  their  assessment.  Thus  both  students  and  the  instructor  were  immersed  in  an 
authentic  setting  for  the  entire  duration  of  the  course.  The  following  student  feedback  illustrates  the  perceived 
value  of  this  learning  context. 

1 guess  one  of  my  own  observations  about  this  course  is  that  most  of  us  spend  a great  deal  of  time  discussing  the 
technology  and  its  use.  This  breaks  us  from  the  'formar  side  of  discussion  in  the  classroom  and  I think  is  a statement 
about  the  power  of  what  we  are  doing.  I'm  not  quite  sure  why  this  is  happening  though. ! think  this  in  itself  is 
marvellous  as  we  all  seem  to  share  the  same  concems/thoughts  etc..  ..The  other  interesting  observation  from  my 
conversation's  with  others  is  that  everyone  seems  to  be  trying  really  hard  to  use  the  technology  to  its  capacity.  The 
last  few  seminars  I think  have  been  markedly  different  and  successful  in  their  own  rights..  ..We  really  seem  to  be 
gently  pushing  the  boundaries  and  experimenting  - great  stuff!. ..At  the  end  of  the  day,  I hope  that  the  course  and 
process  ends  up  by  being  one  huge  experiment  whereby  we  can  reflect  on  VC,  social  interaction  and  learning. 

Perhaps  we  can  also  reflect  on  what  technology  really  is  capable  of  doing  because  we've  been  there.  (Email  message 
from  student  to  researcher.  Week  9) 

Case  Two:  The  instructor  imposed  more  structure  in  the  course  via  the  four  assessment  tasks  and  the 
technology  innovation  was  at  a more  routine  form  of  use  (Hall  & Hord  1987).  Thus  the  element  of 
experimentation  that  students  experienced  in  Case  One  was  not  as  great.  The  “real  world”  context  was  confined 
within  the  assessment  tasks  particularly  in  the  final  group  project.  This  assessment  task  was  deemed  an  authentic 
task  by  the  instructor.  It  was  pre-determined  before  commencement  of  the  course.  Such  a learning  context  is 
claimed  by  Petraglia  (1998)  to  be  preauthenticated. 

4.  Reflective  thinking  is  an  ultimate  goal 

The  degree  to  which  students  exercised  metacognition  differed  between  the  two  cases.  In  Case  One, 
reflective  thinking  was  encouraged  via  the  open-ended  nature  of  the  tasks  and  the  recursive  nature  of  the  tasks. 
The  evaluation  assignment  of  the  course  itself  enabled  students  to  reflect  on  their  experience  during  the  course. 
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The  student  seminars  also  provided  opportunity  for  reflection  as  students  could  refine  their  implementation 
strategy  after  reviewing  other  seminar  strategies.  In  Case  Two,  the  reflective  thinking  component  was  isolated 
within  each  task.  The  instructor  encouraged  students  to  reflect  on  their  experience  in  completing  each 
assignment  during  the  face-to-face  class  discussions. 

Conclusion  and  Implications 

This  study  has  demonstrated  two  different  approaches  that  fall  under  the  “constructivist”  banner  and  the 
different  emphasis  in  learning  outcomes  each  approach  can  generate.  The  instructor  contends  that  effective 
learning  outcomes  were  achieved  in  both  cases,  however,  there  was  a difference  in  their  emphasis.  Case  One 
fostered  exploration  and  afforded  students  to  reflect  on  their  own  practice.  As  such,  the  learning  outcomes  were 
directly  attributable  to  the  amount  of  reflective  thinking  performed  by  the  students.  In  Case  Two,  students  were 
able  to  compartmentalise  their  learning  into  manageable  “chunks”  that  were  more  clearly  identifiable  than  in 
Case  One  and  the  sense  of  exploration  was  diminished. 

The  use  of  the  technology  in  both  cases  from  a “mechanical”  form  of  use  to  a more  “routine”  form  of  use 
illustrates  a consistency  with  the  Concerns  Based  Adoption  Model  (Hall  & Hord  1987).  This  study,  however, 
highlights  an  added  dimension  to  the  change  process  advocated  by  CBAM.  Unlike  the  change  process  being 
only  confined  to  the  instructor  of  the  innovation,  Case  One  demonstrated  that  students  were  also  engaged  in  the 
change  process.  The  experimental  nature  of  Case  One  created  a community  of  researchers  that  explored  the 
nature  of  their  own  learning  process  and  how  lessons  could  be  transferred  in  other  web-based  learning  contexts. 
This  particular  approach  is  not  often  attempted,  possibly  as  the  role  of  the  instructor  as  a co-leamer  is  not 
comfortable  and  can  be  seen  as  less  disciplined.  However,  the  resulting  enthusiasm  and  learning  growth  of  the 
students  in  this  case  demonstrated  the  potential  gains  from  this  approach.  An  implication  for  postgraduate 
education  may  be  that  instructors,  rather  than  embarking  on  a solitary  journey  of  innovation  when  teaching 
technology-based  content,  can  invite  students  to  experience  the  journey,  thus  producing  a community  of  learners. 

The  maturity  of  each  student  cohort  will  greatly  influence  the  outcome  and  the  learning  experience.  The 
course  content  and  the  focus  on  change  as  being  both  a process  and  the  content  of  the  subject  provide  a powerful 
focus  for  learning.  Unlike  much  of  the  change  literature,  if  the  concerns  of  the  instructor  are  met,  the  radical 
view  might  achieve  greater  personal  movement  in  the  students.  It  also  enables  each  student  to  address  their  own 
concerns  and  move  from  mechanical  and  routine  use  of  technology  in  the  learning  process  through  to  a renewal 
and  reassessment  of  their  own  personal  expertise  to  employ  the  technology  as  a professional  educator. 
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Abstract:  The  use  of  the  World  Wide  Web  for  education  is  increasing  rapidly,  but  the  usability  and 
utility  of  instructional  Web  sites  varies  greatly.  The  difficulty  experienced  by  designers  in  mapping 
instructional  design  activities  onto  instructional  systems  is  a major  cause  of  this.  Thus  there  is  a 
need  for  ways  to  make  instructional  design  more  accessible  to  non-instructional  designers. 
Evolving  standards  in  learning  technology  exacerbate  this  problem.  Using  patterns  in  instructional 
design  may  help.  This  paper  reports  on  work  in  progress  on  developing  an  instructional  design 
pattern  language  for  Web-based  learning  materials  and  its  planned  use  in  the  development  of  a 
post-graduate  Human-Computer  Interaction  (HCI)  course  to  be  offered  via  distance  education. 


Usability  Problems  of  Educational  Web  Sites 

Many  researchers  and  professionals  realise  that  the  usability  and  utility  of  World  Wide  Web  sites  varies  greatly 
(Nielsen  1995).  This  observation  also  applies  to  educational  Web  sites,  many  of  which  are  merely  electronic  page- 
turners.  The  use  of  the  World  Wide  Web  (WWW)  as  one  of  the  delivery  systems  for  asynchronous  and  semi- 
synchronous  distance  education  and  training  is  growing  at  an  astonishing  rate.  Institutions  of  higher  learning, 
government,  business  and  industry  will  continue  to  rely  ever  more  heavily  on  the  WWW  for  this.  Thus  it  is  in  the 
interests  of  progressive  societies  that  the  usability  and  utility  of  educational  Web  sites  be  improved,  particularly 
those  providing  formal  instruction.  By  “formal”  is  meant  “in  an  institutional  setting”.  To  improve  these  two 
qualities  will  require  using  some  systematic  way  of  reengineering  existing  educational  Web  sites  (if  this  is  feasible) 
and  constructing  new  ones,  that  goes  beyond  mere  layout  aesthetics  or  page  mechanics.  What  systematic  ways  of 
doing  this  are  there?  The  next  section  will  discuss  one  possible  solution:  instructional  design. 


The  Need  for  Instructional  Design 

According  to  Berger  and  Kam,  instructional  design  as  a process  is  “the  systematic  development  of  instructional 
specifications  using  learning  and  instructional  theory  to  ensure  the  quality  of  instruction.  It  is  the  entire  process  of 
analysis  of  learning  needs  and  goals  and  the  development  of  a delivery  system  to  meet  those  needs.  It  includes 
development  of  instructional  materials  and  activities;  and  tryout  and  evaluation  of  all  instruction  and  learner 
activities.”  (Ryder  1999) 

The  first  task  of  instructional  design  is  to  choose  an  instructional  design  model.  An  instructional  design  model  is  an 
abstract  perspective  on  how  learners  learn  from  others,  and  its  choice  will  determine  how  the  remaining  activities  of 
instructional  design  (described  above)  are  carried  out.  There  are  about  three  dozen  models  (see  the  very  good 
Instructional  Design  Models  Web  site  at  (Ryder  1999)).  Fortunately,  some  instructional  design  models  are  more 
popular  and  well  known  than  others.  Two  of  the  most  popular  models  are  Constructivism  and  Instructional  Systems 
Design  (ISD).  Constructivism  views  learning  as  an  active,  personal  construction  of  meaning  about  the  world  by 
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learners  and  is  popular  in  the  educational  realm  (schools  and  universities),  although  not  universally  so.  ISD  views 
learning  as  a matter  of  conditioning  learners  to  provide  the  correct  responses  to  stimuli,  and  is  rooted  in 
behaviourism.  It  still  dominates  in  the  training  realm  (government,  business  and  industry). 

An  instructional  delivery  system  presents  learning  content  (instructional  materials  and  activities)  to  learners,  who 
interact  with  the  content,  giving  feedback  on  their  interaction.  Hopefully  the  learners  will  emerge  from  the 
experience  changed  in  a positive  sense.  The  delivery  system  need  not  be  computer-based.  Good  instructional  design 
is  a pre-requisite  for  producing  a good  instructional  delivery  system.  But  instructional  design  is  more  complex  than 
the  short  list  of  activities  shown  above  might  suggest.  It  is  a specialist  field  and  is  difficult  to  do  well,  requiring 
expert  knowledge  and  a considerable  investment  of  time  and  effort,  usually  by  a team  of  developers  rather  than  an 
individual.  How  can  instructional  design  be  made  more  accessible  to  those  who  develop  instruction,  most  of  whom 
are  not  professional  instructional  designers?  The  following  section  will  discuss  the  concept  of  pattern  languages  and 
how  they  can  be  used  to  assist  in  the  design  process. 


Patterns  and  Pattern  Languages 

Patterns  and  Why  They  Are  Better 

The  most  important  property  of  modem  computer  software  is  the  level  of  support  that  it  provides  for  the  underlying 
user  activities.  The  interaction  between  software  and  users  takes  place  through  its  user  interface  (UI),  and  thus  a 
well-designed  UI  that  properly  supports  the  tasks  of  users  is  essential.  Considerable  empirical  research  has  been 
done  on  designing  usable  UIs  (Newman  1995).  This  research  includes  general  UI  design  guidelines  and  domain- 
specific  guidelines  for  the  WWW  and  general  hypermedia  (Lowe  1999)  (Nielsen  1995).  Despite  these  guidelines,  a 
large  number  of  usability  problems  still  occur  (Wesson  1 999).  This  is  because  design  guidelines  may  be  imprecise, 
have  limited  applicability  and  be  contradictory. 

Patterns  may  be  a better  way  of  designing  UIs  than  guidelines  (Tidwell  1999)  (Wesson  1999),  A pattern  is  a way  of 
documenting  a successful  solution  to  a recurring  problem  in  an  application-independent,  concrete  manner,  so  that 
the  solution  can  be  applied  in  many  contexts  (reused).  A pattern  is  generally  derived  from  experience  (by  induction 
through  a process  called  pattern  mining,  which  is  both  tentative  and  collaborative)  and  is  not  the  same  as  a stylistic 
rule.  Its  pre-scientific,  humanistic  approach  runs  contrary  to  the  deductive  approach  of  much  of  Computer  Science 
and  Software  Engineering.  Unlike  guidelines,  patterns  are  precise,  are  applicable  in  many  contexts  and  harmonise 
with  neighbouring  patterns.  Patterns  can  form  a framework  in  which  HCI  knowledge  can  be  structured,  and  be  a 
communication  tool  within  HCI  and  between  other  disciplines — like  instructional  design,  for  example. 


A Short  History  of  Patterns 

Pattern  languages  were  first  proposed  by  Christopher  Alexander  et  al.  in  building  architecture  (Alexander  1979) 
(Alexander  1987).  They  have  proved  useful  and  popular  in  object-oriented  design  (OOD)  (Gamma  1995),  and  in  a 
number  of  other  areas  of  Software  Engineering.  Pattern  languages  as  used  in  Software  Engineering  are  also  rooted 
in  the  literate  programming  movement  started  by  Donald  Knuth  (Thompson  1997).  Software  Engineering  patterns 
range  in  granularity  from  programming  language-specific  idioms  to  general  design  principles.  There  is  an  active  on- 
line OOD  community,  exemplified  by  the  Patterns  Home  Page  (Sane  1999).  There  is  also  a small  but  growing 
patterns  group  in  the  HCI  community  (see  e.g.  (Erickson  1999),  (Tidwell  1999)  and  (Wesson  2000))  and  among 
those  who  model  organisational  structures. 


Structure  of  Patterns  and  Pattern  Languages 

Patterns  have  a standardised  format  that  is  generally  discipline-dependent.  For  example,  the  classical  Alexandrian 
format  for  a building  architecture  pattern  consists  of  a meaningful  name  for  the  pattern,  examples  of  contexts  in 
which  the  pattern  is  used,  a description  of  the  problem  which  the  pattern  solves  and  a listing  of  the  external  forces 
which  act  to  shape  a solution.  The  last  two  parts  of  an  Alexandrian  pattern  are  a solution  that  explains  what  must  be 
done  to  solve  the  problem  and  the  context  in  which  one  would  use  the  pattern,  plus  any  additional  information 
required.  The  OOD  pattern  community  uses  a different  format  (Gamma  1 995). 


A pattern  language  (Salingeros  19—)  is  an  interconnected  group  of  patterns.  A pattern  language  may  be  visualised 
as  a graph,  made  up  of  nodes  (individual  patterns)  and  edges  (the  functional  relationships  between  patterns).  Good 
pattern  languages  have  the  properties  of  internal  consistency  and  connectivity  to  other  pattern  languages  at  their 
applicability  boundaries.  Pattern  languages  are  usually  hierarchically  structured  in  that  a particular  pattern’s 
applicability  is  limited  to  a certain  level  in  the  hierarchy,  supporting  larger-scale  patterns  and  being  supported  by 
smaller-scale  patterns  (HCI  pattern  languages  may  be  an  exception  (Tidwell  1999)).  They  provide  superior  ways  of 
understanding  and  creating  complex  systems.  The  OOD  pattern  community  does  not  make  use  of  pattern  languages, 
but  rather  of  catalogues,  which  are  lists  of  useful  programming  language-specific  patterns. 


The  Case  for  an  Instructional  Design  Pattern  Language 

Difficulties  in  Mapping  Instructional  Design  Models  onto  Instructional  Systems 

A search  of  the  Web  reveals  numerous  general  tutorials  of  varying  quality  on  Web  page  design  that  deal  with  layout 
aesthetics,  HTML,  DHTML,  XML,  Java  and  JavaScript  coding  or  the  use  of  various  Web  authoring  tools  (e.g. 
http://www.taxpolicy.com/tutorial/httutor.htm).  Tutorials  and  on-line  documents  that  focus  on  practical  issues 
connected  with  applying  instructional  design  to  educational  Web  sites,  or  hypermedia  modelling  (Lowe  1999)  for 
educational  Web  sites,  are  far  fewer  in  number.  There  are  many  good  online  resources  on  instructional  design  (e.g. 
(Kennedy  1 997)),  but  the  theoretical  treatment  of  models  like  Cognitive  Apprenticeship  and  Conversation  Theory  is 
difficult  to  translate  into  Web  sites  that  work,  particularly  for  those  who  are  not  instructional  designers.  What  is 
needed  is  a way  to  map  instructional  design  models  onto  design  activities  in  a practical,  straightforward  way. 


Evolving  Instructional  Technology  Standards 

A development  which  will  also  increase  the  need  for  a practical,  straightforward  way  to  perform  instnictional  design 
tasks  is  the  work  being  done  by  the  IEEE  1484  Learning  Technology  Standards  Committee  (LTSC)  on  the  Learning 
Technology  Systems  Architecture  (LTSA)  (Ryder  1999).  This  is  being  done  in  close  collaboration  with  Educom’s 
Instructional  Management  System  (IMS)  Project,  the  Aviation  Industry  CRT  Committee  (AICC),  and  the  US 
Department  of  Defense’s  Advanced  Distributed  Learning  (ADL)  project.  It  includes  specifying  an 
architecture/reference  model,  and  learning  objects  and  their  metadata. 

Although  the  LTSA  and  its  associated  projects  seem  to  have  a slight  training  (ISD)  emphasis,  there  seems  little 
doubt  that  they  will  change  and  expand  computer-assisted  learning,  especially  via  the  Web.  One  of  the  changes  will 
be  a push  towards  harnessing  the  creativity  of  individuals,  academic  institutions  and  businesses  in  the  production  of 
large  numbers  of  learning  objects,  each  of  which  serves  an  educational  objective  (or  a few  objectives).  These  will 
be  given  away  or  more  likely  sold.  Educational  programmes  will  be  produced  by  selecting  the  necessary  learning 
objects  and  combining  them  by  means  of  instructional  management  systems. 

As  more  effort  is  put  into  producing  instructional  materials,  the  set  of  learner  interactions  is  likely  to  become  richer, 
compared  to  the  rather  impoverished  interaction  set  generally  now  possible  with  the  low-bandwidth  WWW  (server- 
side  ASP  pages,  and  client-side  plug-ins,  applets  and  scripts  notwithstanding).  XML  and  Java  will  play  an 
important  part  in  this.  One  can  conceive  of  the  production  of  families  of  learning  objects  that  conform  to  a 
particular  instructional  design  model,  like  Andragogy  or  Computer-Supported  Collaborative  Learning.  As  this  push 
gains  momentum,  the  need  for  instructional  design  knowledge  by  learning  object  developers  will  become  greater,  as 
the  draft  standards  are  all  pedagogically  neutral.  In  addition,  knowledge  of  how  to  build  highly  usable  systems  that 
make  use  of  the  richer  set  of  learner  interactions  will  become  more  important. 

Thus  there  is  a need  for  very  practical  assistance  for  non-expert  developers  of  Web-based  learning  materials  in  these 
two  interwoven  areas  of  instructional  design  and  UI  design,  and  this  will  increase  as  the  Web  (and  the  Internet) 
expands  and  becomes  increasingly  pervasive.  An  instructional  design  pattern  language  could  be  of  assistance, 
particularly  when  one  considers  that  the  two  major  commercial  instructional  design  (“pre-authoring”)  software  tools 
support  ISD  only.  Such  a language  could  be  built  into  an  electronic  performance  support  system  for  the  use  of 
developers. 
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Related  Research 


There  are  a number  of  projects  that  are  related  to  the  work  described  in  this  paper  and  these  may  be  studied  with 
profit.  (Bernstein  1998),  (Nanard  1998)  and  (Lyardet  1998)  are  examples.  The  international  Pedagogical  Patterns 
Project  of  Manns  et  al.  (Manns  1999),  which  focuses  on  patterns  for  teaching  object  technology  (a  special  case  of 
(probably)  synchronous,  non-distance,  non-computer-assisted  learning)  and  the  pattern  languages  for  instructional 
multimedia  interface  design  project  of  Dimitrova  (Dimitrova  1999)  are  further  useful  examples.  This  list  is  not 
exhaustive. 


Examples  of  Instructional  Design  Patterns 

At  the  INTERACT’ 99  HCI  Patterns  Workshop,  a classification  scheme  for  HCI  patterns  on  the  basis  of  scale  was 
proposed  (Borchers  1999).  This  scheme,  in  decreasing  order  of  scale,  consists  of  the  following  levels:  Society 
(beyond  systems),  Multiple  Users,  Social  Position,  System,  Application,  User  Interface  Structure  (dialog). 
Components  (containers,  windows,  layout).  Primitives  (buttons  and  other  simple  widgets)  and  Physical  Properties. 
Instructional  design  patterns  (which  are  at  least  partially  HCI  patterns)  map  onto  the  System  and  Application  levels. 
The  greatest  rate  of  change  in  real-world  systems  takes  place  between  the  System  and  Components  levels,  inclusive, 
and  patterns  that  map  onto  these  levels  may  become  obsolete  quite  quickly.  It  is  wise  to  be  aware  of  this  possibility 
for  instructional  design  patterns. 

To  illustrate  how  instructional  design  patterns  can  assist  in  mapping  models  to  implementations,  two  simple 
instructional  design  patterns  for  instructional  Web  pages  follow.  These  are  products  of  a broader  research  project 
that  is  described  in  the  conclusion  to  this  paper.  These  patterns  were  mined  by  considering  two  instructional  design 
models,  examples  of  Web-based  instructional  materials  design  practice  and  descriptions  of  best  practice  by 
developers.  They  deal  with  aspects  of  the  models,  not  the  entire  models  (there  may  be  an  entire  pattern  language  for 
each  model).  Since  mining  patterns  is  a tentative  process,  it  is  customary  to  call  a newly  mined  pattern  a 
protopattem  (Sane  1 999)  until  sufficient  empirical  evidence  through  use  cases  has  been  accumulated  to  promote  it  to 
the  ranks  of  patterns. 


Example  1 : Present  Initial  Knowledge 

Present  Initial  Knowledge,  the  first  protopattem  presented,  deals  with  an  aspect  of  ISD:  how  to  present  initial 
knowledge  about  a topic  to  a novice  learner. 

Name:  Present  Initial  Knowledge 

Examples:  An  on-line  help  screen  on  how  to  customise  a printed  document,  step  by  step.  A checklist  of  activities 
required  when  preparing  to  go  to  ED-MEDIA  2000. 

Context:  The  learning  activity  supported  is  essential  for  grounding  a novice  learner  in  a new  learning  area. 

Problem:  How  should  initial  knowledge  about  a topic  be  presented  to  a novice  learner? 

Forces:  A novice  learner  learning  the  initial  facts,  definitions,  procedures,  etc.  of  some  field  must  not  be  distracted 
by  too  many  choices.  The  content  must  be  presented  in  a logical  sequence.  It  must  be  clear  to  the  novice  learner 
what  the  purpose  of  the  lesson  is,  and  how  to  control  the  display  of,  and  interaction  with,  the  content. 

Solution:  Create  a linear  structure  made  up  of  one  or  more  relatively  short  pages/frames,  limiting  the  links  to  those 
required  for  jumping  forward  and  backward  between  adjacent  pages/frames  in  the  linear  structure.  This  structure 
closely  follows  the  model  of  conventional,  linear  text,  Stmcture  the  content  on  the  pages/frames  into  logical  units 
that  follow  a sequence  from  elementary  to  more  complex.  Define  all  terms  used  and  explain  how  new  objects 
encountered  fit  into  the  overall  content.  Provide  a consistent  set  of  navigation  interaction  objects  on  each 
page/frame.  If  possible,  embed  assessment  interaction  objects  in  the  sequence  of  pages/frames  to  allow  the  learner 
to  practice  mastery  of  the  content.  It  may  be  desirable  to  control  forward  progress  to  make  it  conditional  on  content 
mastery  (e.g.  using  ASPs  or  client-side  scripts)  if  a Mastery  Learning  instmctional  design  model  is  being  used. 

The  Present  Initial  Knowledge  pattern  can  be  used  to  evaluate  the  Web-page  shown  in  Figure  1,  which  exhibits  both 
strengths  and  weaknesses.  Its  strengths  are  that  it  is  short,  its  content  is  logically  stmctured  and  that  it  does  have 
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some  navigation  interaction  objects.  Its  weaknesses  are  that  the  linearity  requirement  of  forward  and  backward  links 
is  not  satisfied,  and  that  terms  presented  are  not  defined. 


Figure  1:  A Web-page  to  evaluate  using  the  Present  Initial  Knowledge  pattern 


Example  2:  Fade  Out  Scaffolding 

Fade  Out  Scaffolding,  the  second  protopattem  presented,  deals  with  an  aspect  of  the  Cognitive  Apprenticeship 
model.  Cognitive  Apprenticeship,  based  on  the  methods  of  traditional  craft  and  trade  apprenticeships  (learning  by 
doing),  aims  to  initiate  the  novice  into  a community  of  expert  practice.  Scaffolding  is  support  provided  while  a 
learner  learns  to  carry  out  a task.  Fading  is  the  progressive  reduction  of  support  as  the  learner  become  more  and 
more  expert  at  executing  the  task.  This  pattern  was  synthesised  from  theory  and  practitioners’  exposition  of  theoiy; 
it  was  not  mined  from  Web-sites. 

Name:  Fade  Out  Scaffolding 

Examples:  A mechanic  shows  an  apprentice  how  to  change  a tyre.  The  apprentice  does  it  herself  several  times.  The 
mechanic  gives  a lot  of  feedback  during  the  first  attempt  by  the  apprentice,  but  as  she  becomes  more  proficient, 
intervenes  less  and  less  until  at  last  he  can  say,  “You’ve  got  it!” 

Context:  The  learning  activity  supported  by  this  structure  is  essential  for  grounding  a novice  learner  in  a new  skill, 
using  Cognitive  Apprenticeship  as  a method. 

Problem:  How  should  scaffolding  provided  to  the  learner  be  progressively  reduced? 

Forces:  The  learning  characteristics  of  the  individual  novice  learner  need  to  be  catered  for.  The  skills  to  be 
mastered  need  to  be  capable  of  being  practised  on  a Web  page  (manual  skills  may  be  very  difficult,  but  more 
computational  skills  are  possible).  There  may  be  limits  on  the  type  of  feedback  possible. 

Solution:  Create  a linear  structure  made  up  of  as  many  pages/frames  as  required  to  model  the  steps  or  phases  of  the 
procedure  to  be  mastered.  Limit  the  links  to  those  required  to  jump  forward  and  backward  between  adjacent 
pages/frames  in  the  linear  structure.  Place  each  step  or  phase  in  the  procedure  on  a separate  page/frame.  Provide 
scaffolding  on  each  page  that  initially  guides  the  learner  in  what  to  do  to  carry  out  that  step,  and  gathers  data  to 
measure  progress  with  mastery.  Keep  a learner  model  whose  stored  values  are  determined  by  learner  performance 
on  task.  Iterate  the  learner  through  the  linear  sequence.  As  the  learner  model  reflects  improving  performance,  fade 
the  level  and  frequency  of  scaffolding  on  each  page/frame  until  expert  performance  is  achieved,  and  then  terminate 
the  iteration.  Provide  a consistent  set  of  navigation  interaction  objects  on  each  page/frame. 
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Conclusions  and  Future  Work 


The  instructional  design  pattern  language  proposed  in  this  paper  might  make  it  easier  for  people  who  are  not 
instructional  designers  to  design  educational  Web  sites  of  high  usability  and  utility.  The  investigation  of  the 
potential  of  HCI  design  patterns  for  designing  technology-supported  distance  learning  programs,  particularly  those 
using  the  Internet,  will  continue.  The  project  will  involve  an  investigation  into  the  design  of  several  such  systems. 
The  goal  of  this  investigation  will  be  to  assess  the  usability  of  these  systems  and  to  identify  possible  design  patterns 
that  have  been  used.  This  project  also  involves  an  investigation  into  the  general  area  of  HCI  design  patterns,  and 
specifically  those  at  the  Application  and  Dialogue  level  that  could  be  applied  to  the  design  of  distance  learning 
programs.  The  instructional  design  pattern  language  will  be  used  to  design  a distance  learning  program  for  the  post- 
graduate HCI  course  at  the  University  of  Port  Elizabeth,  that  could  be  incorporated  into  a virtual  classroom 
environment.  The  usability  of  this  program  will  be  evaluated  in  order  to  validate  the  pattern  language  and  provide 
suggestions  for  improvement. 


References 

Alexander,  C.  (1979).  The  Timeless  Way  of  Building:  Oxford  University  Press. 

Alexander,  C.  et  al.  (1987).  A Pattern  Language:  Towns,  Buildings,  Construction:  Oxford  University  Press. 
Bernstein,  M.  (1998).  Patterns  of  HyperText.  HyperText  98:  Association  for  Computing  Machinery. 

Borchers,  J.,  Griffiths,  R.,  Pemberton,  L.  & Siemon,  E.  (1999).  Usability  Pattern  Language:  Creating  a 
Community:  undistributed  internal  draft  document. 

Dimitrova,  M.  (1999).  Towards  a Pattern  Language  for  Instructional  Multimedia  Interface  Design.  INTERACT  *99, 
IFIP  TC.13:I0S  Press. 

Erickson,  T.  (1999)[The  Interaction  Design  Patterns  Page],  [Online],  (1999  - last  update).  Available: 
http://www.pliant.org/personal/Tom_Erickson/InteractionPattems.html  [1999,  October  25]. 

Gamma,  E.,  Helm,  R.,  Johnson,  R.  & Vlissides,  J.  (1995).  Design  Patterns:  Elements  of  Reusable  Object-Oriented 
Software:  Addison- Wesley  Publishing  Company. 

Kennedy,  D.  & McNaught,  C.  (1997).  Design  Elements  for  Interactive  Multimedia,  Australian  Journal  of 
Educational  Technology,  13(1),  1-22,  [Online],  Available:  http://wwwasu.murdoch.edu.au/ajet/ajetl3/wi97pl.html 
[1999,  October  25]. 

Lowe,  D.  & Hall,  W.  (1999).  Hypermedia  &.  the  Web:  An  Engineering  Approach:  John  Wiley  & Sons. 

Lyardet,  F.D.,  Rossi,  G.H.  & Schwabe,  D.  Using  Design  Patterns  in  Educational  Multimedia  Applications.  ED- 
MEDIA  *98:  Association  for  the  Advancement  of  Computing  in  Education,  Charlottesville,  VA. 

Manns,  M.  et  al.  (1999).  [The  Pedagogical  Patterns  Project],  [Online],  (1999,  July  2 - last  update).  Available: 
http://www-lifia.info.unlp.edu.ar/ppp/  [1999,  August  12]. 

Nanard,  M.,  Nanard,  J.  & Kahn,  P.  (1998).  Pushing  Reuse  in  Hypermedia  Design:  Golden  Rules,  Design  Patterns 
and  Constructive  Templates.  HyperText  98:  Association  for  Computing  Machinery. 

Newman,  W.  and  Lamming,  M.  (1995).  Interactive  Systems  Design  : Addison  Wesley  Publishing  Company. 

Nielsen,  J.  (1995).  Multimedia  and  Hypertext:  the  Internet  and  Beyond:  AP  Professional. 

Ryder,  M.  (1999).  [Instructional  Design  Models],  [Online],  (1999,  October  22  - last  update).  Available: 
http://www.cudenver.edu/~mryder/itc_data/idmodels.html  [1999,  October  25]. 

Salingeros,  N.  (19—).  [The  Structure  of  Pattern  Languages],  [Online],  (19 — last  update).  Available: 
http://www.math.utsa.edu/sphere/salingar/StructurePattem.html  [1999,  October  26]. 

Sane,  A.  (1999).  [Patterns  Home  Page],  [Online],  (1999,  June  10-  last  update).  Available: 
http://hillside.net/pattems/pattems.html  [1999,  August  5]. 

Thompson,  D.  (1997).  [The  Literate  Programming  FAQ],  [Online],  (1997,  August  15  -last  update).  Available: 
http://shelob.ce.ttu.edu/daves/lpfaq/faq.html  [1999,  October  26]. 

Tidwell,  J.  (1999).  [Common  Ground:  a Pattern  Language  for  Human-Computer  Interface  Design],  [Online],  (1999, 
May  17  - last  update),  Available:  http.7/www.mit.edu/~jtidwell/  [1999,  October  25]. 

Wesson,  J.  (1999).  Integrating  HCI  with  the  Software  Development  Process.  Proceedings  of  BitWorld’99,  Cape 
Town,  30  June-2  July  1999. 

Wesson,  J.&  Cowley,  L.  (2000)  Selecting  Interaction  Objects:  A Patterns  Approach,  to  be  presented  at  the 
CHF2000  Workshop  on  Pattern  Languages  for  Interaction  Design:  Building  Momentum,  The  Hague,  The 
Netherlands,  2-3  April. 


259 


The  Best  Practices  Web-Based  Tool:  Using  Technology  and  Constructivism  to 
Create  4“*  Grade  Instruction  that  Supports  Historical  Thinking 


Maya  L.  Creedman 
University  of  California,  Los  Angeles 
Graduate  School  of  Education  & Information  Studies 
Box  95121,  Los  Angeles,  CA  90095 
creedman@ucla.edu 

Elizabeth  Wellman,  Ed.D 
University  of  California,  Los  Angeles 
Graduate  School  of  Education  & Information  Studies 
Box  95121,  Los  Angeles,  CA  90095 


Abstract:  The  proposed  Best  Practices  web-based  tool  is  designed  to  help  grade 
California  teachers  integrate  technology  and  constructivist  learning  into  existing  lessons. 
The  tool  guides  teachers  through  the  instructional  development  process  to  clarify, 
enhance,  and  reflect  on  their  instructional  practices.  Throughout  the  process,  teachers 
explore  and  build  on  specific  instructional  examples.  The  resulting  instructional  lessons 
use  technology  and  constructivist  approaches  to  engage  students  in  active  historical 
thinking. 


Introduction 

Technology  use  in  a constructivist  learning  environment  can  enhance  student  learning,  specifically,  the 
development  of  higher  order  thinking  skills.  The  Best  Practices  web-based  instructional  development  tool  will  help 
teachers  develop  constructivist  learning  activities  and  revise  existing  instruction  while  integrating  technology  and 
higher  order  thinking  skills  simultaneously.  Successful  technology-integration  not  only  teaches  students  to  use  the 
computer  as  a learning  tool,  but  can  actually  improve  the  quality  of  instruction  by  encouraging  teachers  to  apply 
constructivist  principles.  Furthermore,  higher  order  thinking  skills  provide  a necessary  foundation  for  learning 
advanced  subject  matter. 

Many  teachers  are  uncomfortable  with  using  computers  for  learning  and  are  unsure  how  to  integrate 
computers  into  their  own  lessons  successfully.  When  technology-integration  is  unsuccessful,  it  is  often  because  the 
teacher  has  not  used  the  computer  to  enhance  learning  (Cuban,  1986).  The  Best  Practices  tool  will  provide  structured 
support  for  4^^  grade  teachers  to  integrate  technology  and  constructivism  into  their  existing  history  lessons  to  support 
higher-order  thinking.  Because  the  Best  Practices  tool  is  designed  for  history  teachers,  higher-order  thinking  skills 
focus  specifically  on  historical  thinking  and  understanding. 

This  paper  begins  with  a background  on  constructivst  learning  environments,  higher  order  thinking  skills 
and  technology  as  they  relate  to  learning  and  instruction.  Then  the  challenges  facing  teachers  in  developing  effective 
technology-integrated  instruction  will  be  explored.  Finally,  I will  explain  how  the  design  and  development  of  Best 
Practices  provides  this  necessary  support  to  teachers. 

Constructivist  Learning,  Higher  Order  Thinking  and  Technology 

The  differences  between  behavioral  and  constructivist  learning  approaches  stem  from  differing  definitions  of 
the  teaching  process  (Willis  & Mehlinger,  1996).  These  differing  views  directly  affect  the  nature  of  student  and 
teacher  roles  and  the  content,  context  and  goals  of  the  learning  activities.  In  traditional  instruction  students  tend  to 
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act  as  recipients  of  information  and  the  teachers  as  the  deliverers  (Brown,  1992).  A constructivist  learning 
environment  provides  more  opportunities  and  motivation  for  learning  through  interactive,  authentic  and  student- 
centered  learning  activities.  In  a Constructivist  model,  both  teachers  and  students  act  as  additional  resources  for 
testing  and  redirecting  the  student’s  understandings.  In  a behavioral  setting,  activities  are  likely  to  be  highly 
structured  and  geared  towards  learning  a specific  set  of  skills  (Willis  & Mehlinger,  1996).  In  a constructivist 
approach,  skills  are  often  developed  because  the  learning  activity  requires  the  utilization  of  those  skills. 

Constructivist  learning  engages  and  supports  higher  order  thinking  skills  in  students.  In  fact,  constructivist 
pedagogy  places  more  importance  on  the  development  of  higher-order  thinking  skills  than  on  specific  instructional 
content  (Cadiero-Kaplan,  1999).  Common  learning  goals  of  constructivist  technology-integrated  instruction  include 
building  on  ideas,  solving  problems  and  creating  research  presentations  (Cognition  and  Technology  Group  at 
Vanderbilt,  1 996).  As  students  build  their  own  knowledge,  they  analyze,  obtain,  reconstruct  and  apply  higher  order 
thinking  skills.  The  connection  is  evident  in  Honebein’s  (1996)  seven  constructivist  learning  principles.  In  a 
constructivist  learning  environment,  students  become  more  responsible  for  determining  the  learning  process  while 
teachers  act  as  facilitators  to  support  them.  In  order  to  support  students  in  being  active  learners,  teachers’  roles 
change  from  “knowledge  givers”  to  guides  or  “co-leamers”  (Resnick,  1996).  The  focus  shifts  from  “transforming 
knowledge”  to  engaging  students  in  acquiring  learning  skills  (Minstrell  & Stimpson,  1996).  The  content,  context  and 
goals  of  constructivist  learning  activities  emphasize  multiple  perspectives  and  encourage  multiple  methods  of 
communication,  collaboration  and  student  reflection  on  the  learning  process. 

Higher  order  thinking  skills  specific  to  the  content  of  history  and  social  studies  are  well  recognized  by  both 
National  and  California  State  History-Social  Science  Standards.  These  historical  thinking  skills  are  essential  for 
students  to  be  able  to  integrate,  recount  and  analyze  in  the  context  of  time  and  place,  the  social,  political,  scientific, 
technological,  economic  and  cultural  forces  that  have  influenced  history  (National  Center  For  History  in  the 
Schools,  1 998).  The  National  History-Social  Science  standards  present  five  categories  of  historical  thinking  - 
chronological  thinking,  historical  comprehension,  historical  analysis  and  interpretation,  historical  research,  and 
historical  issues-analysis  and  decision  making.  In  other  words,  instead  of  simply  memorizing  information,  students 
should  create  historical  argument  and  narratives,  explain  the  major  events  leading  to  a war  or  analyze  their 
interrelationship. 

In  order  for  students  to  have  the  opportunity  to  develop  and  use  historical  thinking  skills,  they  must  be  exposed 
to  instruction  that  allows  them  the  opportunity  for  substantial  exploration  (Cognition  and  Technology  Group  at 
Vanderbilt,  1996).  One  approach  is  to  engage  the  students  in  authentic  history  activities  (like  those  in  which 
historians  would  engage)  with  historical  thinking  and  strategies  support  provided  (Bain,  1998).  Another  useful 
strategy  for  developing  historical  thinking  skills  involves  asking  students  “what  if’  questions  about  a presented 
situation  and  having  them  explain  their  answer  (Cognition  and  Technology  Group  at  Vanderbilt,  1996).  For 
example,  history  students  might  be  asked  what  might  have  happened  if  the  49ers  hadn’t  discovered  gold. 

The  successful  integration  of  these  strategies  can  be  enhanced  with  technology  (Cognition  and  Technology 
Group  at  Vanderbilt,  1996)  Recently  researchers  have  begun  to  explore  the  benefits  that  technology  and 
constructivist  learning  environments  bring  to  each  other  (Panel  on  Educational  Technology,  1 997).  It  appears  that 
the  combination  of  constructivist  learning  environments  and  technology  both  encourages  students  to  construct  their 
own  knowledge  and  enhances  teaching  practices  (Cadiero-Kaplan,  1999).  In  addition  to  supporting  authentic  and 
student-centered  learning,  increasing  communication  and  improving  the  student’s  ability  to  collect  and  analyze 
information  (Nickerson,  1995;  U.S.  Government,  1996),  technology  is  specifically  useful  when  it  is  used  to  support 
and  model  higher  order  thinking  skills  for  students. 

The  overall  success  of  technology  depends  on  how  it  is  used  (Cognition  and  Technology  Group  at  Vanderbilt, 
1996)  and  whether  it  involves  any  cognitive  thinking  (Salomon,  1993b,  1991).  When  technology  is  used  to  deliver 
information  to  a student,  Rogers  (1998)  suggests  that  there  is  little  or  no  increase  in  learning.  If,  instead,  technology 
is  thoughtfully  integrated  into  the  instructional  design,  curriculum  goals,  standards  and  assessment  (Kahn,  1997), 
learning  is  enhanced  “with”  computers  rather  than  “from”  them  (Salomon,  1988).  Jonassen  and  Reeves  (1996) 
determined  that  technology  is  most  effective  when  it  is  integrated  into  a constructivist  environment  as  a cognitive 
tool. 

Many  computer  applications  currently  available  in  school  classrooms  can  be  used  as  cognitive  tools. 
Applications  commonly  include  communication,  spreadsheets  database,  semantic  networks  and  presentation 
software  - all  of  which  support  student  learning  processes  such  as  collaborating  (the  Internet),  calculating, 
organizing  and  analyzing  data  (spreadsheets,  databases)  and  representing  information  (presentation  and  semantic 
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network  software).  Telecommunication  tools  provide  students  with  new  opportunities  to  discuss  and  collaborate  on 
problem  solving  (Jonassen  & Reeves,  1996).  Spreadsheets  also  support  student  problem-solving  by  helping  to 
reorganize  and  amplify  student  thinking  (Jonassen  & Reeves,  1 996).  Databases  enable  students  to  actively  manage, 
categorize,  search,  and  analyze  data  (Jonassen  & Reeves,  1996).  Semantic  Networks  (e.g.  Timeliner,  Inspiration  and 
Learning  Tool)  support  students’  organization  and  reorganization  of  information  concepts  and  relationships 
(Jonassen  & Reeves,  1 996).  Presentation  applications  allow  students  to  analyze,  construct  and  present  knowledge  in 
a nonlinear  format  (Cognition  and  Technology  Group  at  Vanderbilt,  1996).  Such  tools  also  offer  opportunities  for 
students  and  teachers  to  develop  their  own  instructional  software  (using  authoring  software  such  as  HyperStudio), 
individualize  student  learning,  and  encourage  team  work  and  self-confidence  (Office  of  Technology  Assessment 
(OTA,  1995). 

Teacher  Challenges  and  the  Best  Practices  Solution 

Many  researchers  are  beginning  to  see  technology  as  a catalyst  for  changing  teachers’  instructional 
practices  to  a more  constructivist  approach  (Cadiero-Kaplan,  1 999).  The  effective  use  of  technology  lends  itself  to 
student-centered  instruction  (Clark,  1991;  Cognition  and  Technology  Group  at  Vanderbilt,  1996),  which  is  an 
important  component  of  constructivist  learning  environments.  Current  research  suggests  that  the  effective 
integration  of  technology  requires  teachers  to  adjust  instructional  practices  toward  Constructivist  models  of  teaching 
and  learning  (Cognition  and  Technology  Group  at  Vanderbilt,  1996). 

Best  Practices  addresses  teachers’  instructional  challenges  by  encouraging  exploration,  providing 
technology  integration  and  supporting  activity  development  with  guiding  prompts.  The  prompts  provide  a 
constructivist  instructional  development  model  and  thus  scaffold  the  development  process.  Additionally,  because 
content  relates  to  the  teacher’s  existing  lessons  and  instructional  standards,  Best  Practices  is  also  flexible,  efficient 
and  relevant  to  the  teacher. 

The  OTA  report  (1995)  suggests  that  barriers  to  technology  use  include  the  lack  of  opportunities  to 
experiment  and  share  experiences,  an  understanding  of  how  and  why  to  use  technology  in  the  curriculum,  good 
models  and  training  and  support.  Best  Practices  provides  teachers  with  opportunities  to  experiment  through  the 
technology  matrix  tool,  which  provides  various  technology  solutions  and  activity  examples.  Sample  lesson  plans 
also  allow  teachers  to  share  and  learn  from  their  colleagues’  experience.  Through  guiding  prompts  and 
constructivist-based  examples,  the  tool  offers  teachers  a model  as  well  as  rationale  for  integrating  technology. 
Because  technology  integration  focuses  on  existing  lesson  plans,  it  is  both  efficient  and  effective.  Teachers  are  more 
likely  to  integrate  this  process  into  their  teaching  practice  because  they  experience  technology  integration  as  relevant 
to  them  and  their  instructional  needs.  While  Best  Practices  can’t  provide  in-person  training,  the  tool  models  and 
supports  teachers’  own  learning  process. 

The  Best  Practices  environment  is  modeled  closely  on  a particular  constructivist  learning  approach  - Problem  - 
Based  learning.  Problem-Based  learning  uses  constructivist  principles  to  focus  student  learning  around  a specific 
prolilem.  Once  the  problem  is  defined,  the  learner  must  find  a way  to  solve  the  problem  with  support  from  peers, 
teachers  and/or  other  resources.  Throughout,  learners  reflect  on  their  solutions  and  learning  process.  To  be 
successful,  the  learner  must  have  support  in  using  and  developing  their  cognitive  skills  (Savery  & Duffy,  1996). 
Best  Practices  helps  the  teacher  to  identify  and  solve  a specified  instructional  problem  encountered  in  teaching 
historical  thinking  to  students. 

Best  Practices  also  encourages  exploration.  Hughes  (1998)  suggested  that  such  barriers  might  be  overcome  by 
encouraging  technology  exploration.  The  OTA  report  suggests  that  teachers  “need  visions  of  technology’s  potential, 
opportunities  to  apply  them,  training,  just-in-time  support  and  time  to  experiment”.  The  OTA  report  found  that  the 
most  effective  technology  training  for  new  teachers  includes  watching  videos  of  actual  classroom  teaching, 
practicing  with  case  studies  and  participating  in  computer  simulations  and  electronic  networks  (such  as  newsgroups 
and  bulletin  boards).  Experienced  teachers  responded  best  to  on-line  access  to  new  technology  ideas,  good  teacher 
examples,  and  on-line  contact  with  colleagues  and  technology  use  experts  (OTA,  1996).  Through  exploration  and 
reflection  on  their  own  teaching  and  learning  processes,  teachers  with  a more  traditional  teaching  approach  can 
explore  learning  from  a constructivist  approach.  Because  teachers  often  teach  in  the  same  way  that  they  themselves 
were  taught,  teachers  often  need  to  have  the  experience  of  being  a student  in  the  same  environment  that  they  will 
create  for  their  own  students  (Fosnot,  1 989). 

Because  Best  Practices  will  be  available  on  the  Internet  access  it  is  not  limited  by  geography,  time  or 
personnel  constraints.  This  tool  could  also  be  used  within  a group  professional  development  context  to  help  clarity 
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individual  instructional  goals  and  view  examples.  Although  Best  Practices  currently  focuses  on  instruction  for  4^*^ 
grade  California  history,  it  can  be  expanded  to  include  multiple  content  areas  and  grade  levels.  This  content  and 
standards  specific  focus  further  ensures  that  the  instructional  development  process  is  meaningful  to  the  teacher’s 
instructional  goals  and  practices.  By  revising  an  existing  lesson,  a teacher  is  more  likely  to  develop  a lesson  he  or 
she  will  feel  comfortable  teaching  as  well  as  be  able  to  take  ownership  for  the  improvements  in  the  lessons. 

Best  Practices  also  provides  the  teacher  with  support  and  a framework  in  which  to  reflect  upon  his  or  her 
solution  and  learning  process.  Using  the  Technology  Matrix  at  the  beginning  of  the  Enhancement  Process  helps  the 
teacher  identity  the  specific  goals  (technology-integration  and  historical  thinking  engagement)  at  the  beginning  of 
the  instructional  development  process.  The  teachers  have  the  opportunity  to  explore  technology-integrated  activity 
examples  that  demonstrate  the  strengths  of  different  software  applications  for  engaging  historical  thinking  skills. 
The  examples  also  illustrate  how  multiple  historical  thinking  skills  can  be  addressed  in  a single  activity. 

The  Technology  Matrix  can  be  used  in  several  ways.  Some  teachers  may  want  to  explore  all  options  - and 
view  activities  for  all  historical  thinking  skills  and  software.  Other  teachers  may  be  interested  in  viewing  the  way 
different  activities  engage  different  historical  thinking  skills  for  each  software  program.  The  Technology  Matrix  also 
provides  teachers  with  the  opportunity  to  compare  solutions  to  those  of  other  teachers,  broadening  their  perspective 
and  resources  for  approaching  an  instructional  problem.  The  Technology  Matrix  should  help  teachers  see  “how”  to 
integrate  technology  to  meet  the  content  standards  (historical  thinking  skills  and  content).  The  Technology  Matrix’s 
flexibility  allows  the  teacher  to  choose  those  areas  historical  thinking  skills  and  software  applications)  that  fit  his  or 
her  needs. 


Program  Components 

Teachers  are  offered  the  tools  to  revise  their  lessons  through  a two-stage  process:  the  Clarification  Process 
and  the  Enhancement  Process.  In  the  Clarification  stage,  teachers  identify  and  clarify  the  components  of  their 
existing  lesson.  In  the  Enhancement  stage,  teachers  begin  to  revise  their  lesson  and  develop  a technology-integrated 
activity.  Teachers  choose  from  a list  of  historical  thinking  skills  and  software  applications.  The  resulting  matrix 
provides  activity  ideas  as  well  as  teacher  examples  to  match  their  preferences.  From  there,  teachers  develop 
technology-enhanced  activities  with  the  help  of  guiding  prompts.  The  teacher’s  responses  to  these  prompts 
eventually  inform  the  content  of  the  revised  lesson. 


Clarification  Process 

With  the  support  from  specific  prompts,  teachers  identify  and  enter  the  lesson  components  from  their  pre- 
existing lesson  “ such  as  student  prior  knowledge,  goals  and  objectives,  learning  activities  and  assessment  --  into  a 
specific  format.  This  format  helps  teachers  to  identify  the  specific  elements  of  their  existing  lesson  and  to  recognize 
the  implications  of  those  choices.  The  prompts  help  engage  the  teacher  in  active  reflection  and  response  to  the 
instructional  development  process.  The  teacher’s  responses  are  automatically  entered  in  the  final  Lesson  Plan.  These 
responses  provide  the  teacher  with  a preliminary  instruction  plan  on  which  to  build  and  reflect  on  as  they  begin  the 
Enhancement  process. 


Enhancement  Process 

The  Enhancement  process  is  the  central  focus  of  Best  Practices  - this  is  where  the  teacher’s  existing  lesson 
is  revised  to  incorporate  constructivist  learning  and  technology  in  a way  that  supports  historical  thinking 
engagement.  The  Enhancement  Process  consists  of  (1)  five  sets  of  prompts  that  guide  teachers  toward  developing 
the  revised  instruction  and  (2)  the  Technology  Matrix  tool , which  provides  resources  to  help  the  teacher  respond  to 
the  prompts.  The  teacher  begins  the  enhancement  process  by  using  to  Tech  Matrix  tool  to  match  computer  software 
applications  and  specific  historical  thinking  skills  with  appropriate  instructional  activities. 

The  Tech  Matrix  tool  functions  as  a first  step  in  the  Enhancement  process.  The  Tech  Matrix  tool  allows 
teachers  to  choose  historical  thinking  skills  that  they  want  to  focus  on  and  specific  software  that  they  want  to  use. 
The  tool  then  creates  a matrix  offering  descriptions  and  specific  teacher  examples  of  activities  that  meet  both 
criteria.  The  matrix  provides  both  general  and  content  specific  activity  examples.  Teachers  will  use  these  activity 
examples  to  begin  developing  their  own  technology-integrated  activities  with  the  support  of  development  prompts. 
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The  five  sets  of  prompts  - technology  integration,  historical  thinking  skills,  constructivist  learning, 
assessment  and  final  reflection  - provide  support  and  structure  as  the  teachers  interacts  with  the  Technology  Matrix 
tool  and  creates  the  content  for  the  revised  lesson.  Each  set  of  prompts  supports  the  integration  of  a specific 
instructional  goal  into  the  developing  activity.  These  prompts  support  teachers  in  integrating  technology  as  a 
“cognitive  tool”  that  promotes  historical  thinking  skills,  integrating  constructivist  principles  and  developing  an 
authentic  student  assessment  method  for  the  created  lesson.  The  reflection  prompts  provide  a time  for  teachers  to 
reflect  and  revise  their  lesson  as  a whole. 

The  prompts  include  questions  about  specific  lesson  plan  components  (e.g.  instructional  goals,  activity,  and 
student  learning  processes)  as  well  as  questions  to  encourage  reflection  on  the  teachers’  teaching  practices.  A sample 
response  and  detailed  description  are  offered  for  each  prompt.  The  teachers  reflect  on  their  instruction  throughout 
development  to  maintain  or  develop  an  awareness  of  their  own  design  process  as  well  as  their  solution.  The 
teachers’  responses  are  automatically  entered  into  the  lesson  plan  at  the  end  of  each  set  of  prompts.  The  teachers  can 
then  reflect  on  their  changes  and  additions  as  they  relate  to  the  lesson  as  a whole  as  well  as  make  additional  changes. 
This  Enhancement  process  guides  the  teachers  through  basic  principles  and  examples  for  developing  constructivist 
instruction  and  provides  a final  checklist  to  help  teachers  identify  areas  that  need  further  development. 


Conclusion 

Since  it  is  essential  that  teacher  training  programs  model  appropriate  teaching  methods.  Best  Practices 
integrates  three  important  constructivist  characteristics.  First,  the  instructional  development  is  centered  around  a 
teacher-determined  problem  (Technology  Matrix).  Second  the  teacher  has  a role  in  determining  the  process  for 
solving  the  problem.  Third,  the  environment  supports  and  challenges  the  teacher’s  thinking. 
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Abstract:  This  paper  examines  the  design  of  an  On-Line  Campus  (OLC)  and  how  it  affects 
the  learner.  It  describes  the  structure  and  analyses  the  thought  processes  that  went  into  the 
design  of  the  first  OLC  at  the  University  of  Greenwich.  It  states  the  importance  of  the 
students’  perspectives  when  designing  a virtual  learning  environment.  The  discussion  is 
focused  on  the  various  elements  that  comprise  the  OLC,  as  well  as  speculating  about  future 
developments. 


Background 

In  1998  the  University  of  Greenwich  began  piloting  an  On-Line  Campus  (OLC)  accessed  via  the  World  Wide 
Web  by  students  (gre-guns2.gre.ac.uk)  and  via  Lotus  Notes  groupware  by  staff  Until  this  time  the  university 
had  never  attempted  to  co-ordinate  teaching  via  the  Internet,  instead  individual  schools  with  appropriate 
expertise  constructed  their  own  web  pages  in  isolation  from  other  areas  of  the  university.  In  the  early  stages, 
the  OLC  was  used  primarily  for  piloting  the  school  of  Post  Compulsory  Education  & Training’s  Computer 
Mediated  Communication  (CMC)  related  courses  on  the  Internet,  which  had  previously  been  delivered  via 
Lotus  Notes  [Ryan,  1997].  Early  in  1999  the  OLC  had  progressed  to  a stage  where  the  interface  had  been 
redesigned  to  facilitate  easier  access  to  core  areas  and  more  schools  and  departments  were  involved. 


The  Design  - everything  only  ^one-ciick’  away 

When  constructing  the  OLC  we  wanted  to  mirror  the  student  experience  of  being  on  a physical  campus,  so  that 
any  department  that  existed  on  one  of  the  university’s  real  campuses  would  be  available  to  our  on-line 
students.  We  wanted  this  virtual  environment  to  be  perceived  as  an  extension  to  the  university’s  physical 
locations,  so  we  designed  it  around  the  metaphor  of  a real  campus  (Gould,  1995).  This  differed  from  the 
university’s  corporate  web  site  philosophy,  which  was  more  of  an  advert  - one  big  web-enabled  prospectus. 

The  advantage  of  the  OLC  from  the  learner’s  perspective  is  that  it  provides  a 'one  stop  shop'  for  university 
learning  and  teaching  materials  and  support.  The  use  of  Lotus  Notes  as  the  authoring  tool  and  Domino  as  the 
web  server  provided  a much  greater  degree  of  interactivity  than  many  of  the  web-based  materials  found  on  the 
corporate  site.  The  OLC  was  designed  to  be  highly  user  friendly,  particularly  for  learners  studying  at  a 
distance,  and  consisted  of  mostly  interactive,  rather  than  static  (dateable),  material.  Our  philosophy  of  using 
Information  and  Communication  Technology  (ICT)  to  provide  opportunities  for  interaction  was  based  on  our 
earlier  work,  using  CMC  to  support  adult  distant  learners  engaged  in  Continuing  Professional  Development 
(Lewis  et  al,  1 997;  Jordan  and  Ryan,  1 999) 

There  were  threes  core  rules  that  we  tried  to  apply  when  constructing  the  OLC: 
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1.  ‘One  click’  from  useful  information 


In  our  view,  so  many  web  sites  lose  sight  of  their  core  function.  Our  function  was,  and  is,  to  provide  a learning 
environment  where  access  to  information  should  be  quick  and  easy.  We  were  conscious  that  many  of  our 
students  were  novice  ICT  users  and  when  studying  at  a distance  they  were  likely  to  have  limited  access  to 
technical  support  and  be  easily  discouraged  if  presented  with  a complex  system.  We  felt  it  important  that  no 
matter  where  the  student  was  in  their  individual  school’s  web  pages  that  they  should,  with  ‘one  click’,  be  able 
to  access  many  sources  of  information.  To  make  this  ‘one  click’  approach  possible  we  opted  for  a frame-based 
design  with  a navigation  bar  running  down  the  left  side  of  the  screen,  as  can  be  seen  in  figure  1 (Brookes, 
1993;  McCormack  and  Jones,  1997;  Latham,  1998). 
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youi  Web  Browser. 
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Figure  1:  On-Line  Campus  web  site  layout  with  navigation  side  bar. 


This  allows  our  students  to  work  within  their  course  on  the  right  side  of  the  screen  and  quickly  move  from  one 
part  of  the  OLC  to  another  to  access  other  resources.  For  example,  if  a student  were  reading  a conference  entry 
by  a tutor  or  peer  and  a book  was  mentioned,  with  one  click  on  the  Student  Resource  Area  on  the  navigation 
bar,  they  would  have  access  to  the  university  library’s  search  engines  to  see  if  the  book  was  in  stock.  They 
could  then  email  or  submit  a form  requesting  the  book  be  held  or  sent  to  them. 

2.  No  prospectus  type  materials  to  be  included  on  school  pages 

Primarily,  we  wanted  the  OLC  to  be  a learning  and  teaching  site,  so  we  discouraged  the  placing  of  large 
amounts  of  course  documentation  as  this  would  duplicate  the  information  held  on  the  university’s  corporate 
site.  The  problem  was  that  pages  could  be  authored  easily  using  Lotus  Notes  with  no  prior  knowledge  of 
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HTML.  Consequently,  staff  preferred  to  use  Notes  to  place  information  on  the  OLC  as  opposed  to  the  corporate 
web  site  that  was  still  based  around  traditional  HTML  language  and  Unix  command  structures. 


3.  A fully  featured  environment  experience 

Often  web  sites  tend  to  include  materials  that  require  plug-ins  but  with  no  link  to  download  them.  By 
providing  a collection  of  plug-ins  on  the  OLC  this  saved  time  and  provided  a convenient  download  site  to 
many  of  the  users,  especially  in  the  UK  where  even  local  phone  calls  are  chargeable. 


The  Core  Areas 

The  OLC  Navigation  bar  contains  links  to  core  areas  of  the  campus.  We  kept  these  to  a minimum  to  make  the 
OLC  easy  to  use  and  by  placing  this  navigation  bar  in  its  own  frame,  it  provided  an  anchor  for  the  students. 
Wherever  they  went  (which  always  occurred  in  the  right  hand  frame)  in  the  OLC,  or  even  if  URL  links  took 
them  away  from  the  OLC,  they  would  always  be  able  to  switch  to  a different  OLC  area  with  one  click 
(Brookes,  1993). 

The  Faculty  Area  shows  a list  of  schools  and  departments,  which  provide  services  for  the  students,  this  list  is 
updated  automatically  whenever  a new  school  or  department  home  page  is  created.  The  Student  Resource  Area 
provides  a central  place  for  links  that  will  be  helpful  to  students;  at  the  top  of  this  area  is  the  link  to  the 
Student  Common  Room.  We  decided  there  should  be  a single  common  room  for  all  students,  irrespective  of 
course  or  school.  This  is  more  representative  of  a real  campus  where  all  students  mingle  in  the  canteen.  The 
News  and  Helpdesk  is  where  students  can  find  all  the  latest  information  on  the  OLC  as  well  as  any  web 
browser  plug-ins\resources  that  they  might  require.  This  area  also  houses  a helpdesk  - a bespoke  Lotus  Notes 
database  that  accepts  forms  submitted  by  a web  browser.  Helpdesk  personnel  can  email  an  answer  to  the  user 
or  compose  a technical  note  that  is  added  automatically  to  the  Frequently  Asked  Questions  (FAQ)  list  of  the 
helpdesk.  The  special  project  area  is  where  new  OLC  developments  are  tested  prior  to  launch.  Although 
students  may  view  these,  access  is  restricted  to  project  participants  and  the  final  link  is  to  the  university’s 
corporate  site. 

The  Student  Office  (SO),  which  is  part  of  Central  Registry,  have  worked  with  the  OLC  developers  from  the 
beginning.  This  has  allowed  a great  many  of  the  resources  that  are  available  for  traditional  on-site  students  to 
be  repackaged  for  OLC  users.  They  have  converted  many  of  the  print-based,  student-centred  publications  into 
a web  format  for  the  very  first  time.  These  include:  regulation  documents,  advice  leaflets,  information 
regarding  assessment  and  two  important  books  that  all  new  students  receive  when  they  first  join  the  university; 
a Guide  for  New  Students  and  a Skills  for  Learning  Handbook. 

With  the  Student  Office  taking  such  an  active  role  in  the  development  of  the  OLC,  this  has  raised  awareness 
throughout  the  university  of  the  need  to  treat  distance  students  as  equals  to  their  on-site  counterparts.  This  is 
shown  by  the  SO  constructed,  student  satisfaction  survey  form,  which  is  only  available  electronically  via  the 
web  and  makes  no  distinction  between  distance  and  local  students 


The  Educational  Elements 

From  the  tutor’s  or  institutional  view,  the  OLC  provides  an  integrated  suite  of  virtual  departments  including; 
Library,  Media  Resources,  Student  Office  and  Registry,  which  gives  the  distance  learner  the  sense  of  belonging 
to  a real  institution.  At  the  same  time  it  provides  a single  gateway  into  school-based  materials  which  remain 
under  the  school’s  control.  Because  of  the  non-html  nature  of  the  authoring  system,  even  non-technical  staff 
can  maintain  these  pages,  and  they  also  benefit  from  the  accumulated  good  practise  being  developed  in  other 
flexible  learning  initiatives  employing  the  OLC. 


ERIC 


Educational  innovation  has  primarily  come  from  use  of  the  system  by  the  School  of  Post  Compulsory 
Education  and  Training  who  have  developed  a large  area  to: 

a)  support  their  current  traditional  courses 

b)  to  develop  purely  CMC  based  courses  for  continuing  professional  development  [Jordan  8l  Ryan,  1 999]. 

With  collaboration  between  the  developers  and  tutors  of  the  OLC  it  has  been  possible  to  add  new  features  to 
the  course  areas  [McCormack  and  Jones,  1997].  For  instance,  at  the  end  of  a course  all  students  are  required  to 
complete  an  evaluation  form.  This  is  now  done  electronically  and  submitted  and  held  within  the  respective 
course  database.  Using  Lotus  Notes  workflow  features  the  course  database  has  the  ability  to  allow  the  tutor  to 
correlate  these  evaluation  results  into  percentages  for  analysis.  These  can  be  emailed  automatically  to 
appropriate  departments  in  the  university  for  inclusion  in  a variety  of  reports. 

In  1998  we  developed  our  first  validated  programme  to  exploit  the  potential  of  the  OLC.  Designed  by  the 
Natural  Resource  Institute,  this  is  a worldwide,  distance  master’s  programme  in  grain  storage  management 
that  employs  the  OLC  for  tutor  support  and  peer  to  peer  interaction. 


The  Future 


There  are  many  ways  we  would  like  to  extend  the  OLC  in  the  future.  One  would  be  to  include  a template 
database  for  creating  on-line  exams  with  automatic  results  generation,  which  could  be  adapted  easily  by  non- 
technical staff  for  any  course. 

Another  would  be  to  integrate  more  university  facilities  into  the  OLC,  possibly  in  an  on-line,  ‘multiplex’ 
lecture  theatre.  The  university’s  lecture  theatres  are  geographically  dispersed  across  many  campuses.  If  they  all 
had  video  cameras  installed  and  were  connected  to  computers,  this  would  allow  the  lecture  to  be  uploaded  to 
the  OLC  for  live  transmission  or  archived  for  future  presentation.  The  students  entering  the  ‘multiplex’  would 
be  able  to  choose  which  live  lecture  to  attend  or  go  to  the  archive  and  view  an  old  lecture,  although  we  have  yet 
to  find  a way  to  simulate  popcorn  on  the  OLC! 

Lastly,  to  integrate  real-time  collaboration  via  Lotus  Sametime  software  (Lotus,  1999),  which  would  allow 
students  to  see  if  their  tutor  is  available  on-line  to  interact  with  them  synchronously,  via  a Chat  based  system. 
This  would  also  allow  the  tutor  to  share  computer  applications  with  the  student.  Such  a system  would  also 
greatly  assist  the  helpdesk  by  allowing  the  personnel  to  demonstrate  visually  at  a distance  where  the  student  is 
having  a problem. 

Whilst  we  attempt  to  introduce  innovation  into  the  university’s  learning  and  teaching  processes  via  the  OLC, 
we  must  never  forget  it  must  be  focused  on  what  students  want  and  we  need  to  be  ever  mindful  of  the  vast 
range  of  technical  abilities  of  these  virtual  students.  We  welcome  constructive  criticism  from  all  our  users  in 
our  endeavour  to  make  the  OLC  an  even  better  virtual  experience. 
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Abstract 

The  purpose  of  the  study  was  to  examine  how  students  (n  = 14)  participated  in  a web-based  graduate  course  discussions. 
The  focus  of  this  examination  was  to  analyze  the  interactions  that  occurred  during  synchronous  (chat)  and  asynchronous 
(listserv)  modes  of  discussions  using  a coding  scheme  developed  by  the  researchers.  During  a one  week  period,  students 
were  randomly  assigned  to  either  a small  group  chat  or  threaded  discussion.  The  next  week,  the  two  small  groups  switched 
forms  of  discussion  and  were  given  another  topic  question  to  discuss.  The  study  is  qualitative  in  nature  in  order  to  find  out 
whether  of  the  students’  participation  were  substantive  (that  is,  directly  related  to  the  topic)  or  non-substantive  (messages 
were  not  directly  related  to  the  content)  in  both  types  of  discussions  and  these  were  compared  by  type  of  comment  and 
quantified.  In  addition,  the  students  were  surveyed  about  their  computer  and  Internet  experience  and  skills  as  well  as  their 
attitudes  toward  the  course  content,  its  organization  and  delivery.  Results  indicated  that  overall  students’  discussions 
included  all  8 types  of  substantive  and  non-substantive.  However,  the  participants  when  in  the  chat  showed  greater  numbers 
of  responding  and  reacting  statements  (substantive  types)  in  both  weeks  than  when  participating  in  the  threaded  discussion. 
Based  on  the  survey  results,  some  students  found  it  difficult  to  follow  the  dialogue  in  the  chat,  but  overall  they  enjoy  this 
type  of  interaction.  The  students  also  enjoyed  the  threaded  discussion,  especially  for  its  convenience. 


Introduction 

Although  various  forms  of  Distance  Education  have  been  in  existence  for  a long  time,  the  advent  of  the 
Internet  and  the  World  Wide  Web  (WWW)  has  brought  changes  to  teaching  and  learning  at  the  university  level. 
A number  of  universities  and  colleges  are  adding  or  converting  traditional  courses  and  programs  to  web-based 
instruction  (WBI)  or  online  learning  environments.  These  WBl  courses  require  not  only  a different  design  and 
delivery  of  the  instruction,  but  also  a different  form  of  engagement  on  the  part  of  students  with  the  course 
materials,  other  students,  and  the  instructor.  These  learning  environments  require  not  only  active,  but  interactive 
participation  (Davidson-Shivers  & Rasmussen,  1998  & 1999).  Interactive  learning  includes  an  interchange  of 
ideas  with  all  participants;  that  is,  the  students  and  instructor  exchange  ideas  in  a flexible  and  dynamic 
environment  (Rasmussen  &Northrup,  1999). 

This  interchange  of  ideas  may  occur  through  a synchronous  mode,  which  occurs  at  the  same  time  but  from 
different  locations,  or  through  an  asynchronous  mode,  in  that  students  and  the  instructor  communicate  to  each 
other  at  differing  times.  The  computer-mediated  communication  (CMC)  literature  documents  the  dynamics  of 
online  discussions  by  various  forms  of  communication  patterns,  processes,  and  purposes  (William  & Merideth, 
1996;  Pibum  & Middleton,  1998;  Wojahn;  1994;  Jeong,  1996;  McCormick  & McCormick,  1992;  McConnell, 
1997  Sherry,  1999;  Kara,  Bonk,  & Angel  (in  press)).  The  CMC  literature  and  the  literature  on  Web-based 
instruction  (WBI)  identifies  various  online  delivery  formats  (e-mail,  listserv,  chats,  conferences,  etc.)  and  how 
they  might  be  used  for  discussions  (Khan,  1997;  Shotsberger,  1997;  Driscoll,  1998).  Developers  of  online 
learning  environments  often  suggest  that  asynchronous  communication  may  have  advantages  over  synchronous. 
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For  instance,  Driscoll  states  that  asynchronous  methods  allow  students  more  time  for  reflection  than  do 
synchronous  delivery  formats;  with  one  example  shows  only  20%  of  a web-based  course  to  be  in  the 
synchronous  format.  In  addition,  Jeong  argues  that  “most  [not  all]  findings  hail  the  use  of  asynchronous 
communication’’  but  this  author  also  “notes  the  absence  of  the  evaluation  of  synchronous  communication  and 
their  effects”  (p.  51).  His  own  findings  suggest  that  synchronous  online  chats  have  an  advantage  of  promoting 
highly  interactive  discussions  with  a disadvantage  for  the  group  to  digress  from  the  topic  to  another  (p.62). 
However,  there  are  mixed  results  based  on  the  CMC  research  and  there  is  a lack  of  evaluation  on  synchronous 
discussion,  it  seemed  viable  to  compare  and  contrast  how  students  participate  in  both  types  of  discussions. 

Methods 

Subjects 

Participants  in  the  study  were  graduate  students  (n  = 14)  in  a required  course  for  their  degree  programs  of 
study  from  a southeastern  regional  university  in  the  USA.  Approximately  two-thirds  of  the  students  were 
female.  Based  on  the  survey  results,  the  majority  of  students  reported  that  they  had  computer  experience  with 
some  having  less  experience  with  the  Internet  and  WWW.  Participation  in  the  discussions  was  a course 
requirement.  Confidentiality  of  information  was  maintained  by  having  surveys  collected  and  coded  by  someone 
other  than  the  instructor  and  the  analysis  of  the  discussions  occurred  after  final  grades  were  posted. 


Course  content,  organization,  and  requirements. 

The  course  was  an  introductory  course  on  trends  and  issues  in  instructional  design.  The  course  was 
organized  by  weekly  topics  with  assignments  and  questions  being  posted  to  its  website.  Two  or  three  questions 
were  given  with  directions  on  how  to  post  (either  chat  or  threaded  discussion)  answers  and  replies.  Students  had 
a week  to  respond  to  any  listserv  qucstion(s)  and  were  also  required  to  reply  at  least  twice  to  other  students’ 
responses  during  the  week.  Typically  one  question  was  scheduled  for  an  hour  and  a half  chat  during  the  week. 
Chats  were  either  large  group  (whole  class)  or  small  groups  (half  of  the  class  per  chat  session).  Students  were 
also  assigned  particular  readings  as  preparation  for  discussing  the  weekly  topic.  They  were  also  encouraged  to 
draw  on  their  own  experiences,  knowledge  and  skills.  Both  threaded  discussion  and  chat  could  be  copied  and  all 
of  the  chats  were  distributed  to  all  members  of  the  class.  After  the  fifth  week  of  the  term,  students  were  assigned 
as  discussion  leaders  to  facilitate  the  weekly  discussions  in  both  chats  and  threaded  discussions  with  guidance 
from  the  instructor.  The  instructor  participated  directly  in  the  online  chats;  however,  less  so  when  another 
student  was  the  discussion  leader.  With  the  threaded  discussions,  she  added  her  comments  to  a summary  at  the 
end  of  the  week  rather  than  commenting  during  the  week. 


Procedures 

The  following  procedures  occurred  for  gathering  the  data. 

C Using  a Likert-type  questionnaire,  students  were  surveyed  three  times  during  the  term  in  order  to 
evaluate  students’  reactions  and  attitudes  toward  the  course.  The  survey  also  gathered  information 
about  the  computer  experiences  and  skills.  Data  was  kept  confidential  by  having  someone  other  than 
the  course  instructor  collect  and  code  each  set  of  questionnaires. 

C For  one  question  posted  during  the  thirteenth  and  fourteenth  week  of  the  term,  the  students  were 
randomly  assigned  to  two  groups.  Half  the  students  discussed  the  question  (it  was  the  third  question 
posted  for  each  week)  using  the  online  chat  and  the  remaining  students  used  the  threaded  discussion. 
The  process  was  repeated  the  following  week  with  a new  topic  and  question,  and  the  two  groups 
switched  discussion  modes. 

C Transcripts  of  the  discussions  for  those  two  weeks  were  then  coded  using  a coding  scheme  developed 
by  the  researchers  based  on  the  work  of  Pibum  and  Middleton  (1998)  and  Williams  and  Meredith 
(1996).  See  Table  1 for  the  coding  scheme.  The  coding  and  analyses  of  the  discussions  did  not  occur 
until  after  the  final  course  grades  were  posted. 

C The  researchers  were  trained  to  use  the  coding  scheme  and  then  coded  each  discussion  transcript 
independently.  The  transcripts  were  coded  by  each  completed  statement/thought  made  rather  than 
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using  a line-by-line  method.  Complete  sentences,  incomplete  sentences,  and  short  phrases  were 
considered  as  a statement  if  a new  or  different  thought  was  presented  within  them.  Incomplete 
sentences  or  short  phrases  were  often  used  within  the  chat  due  to  the  speed  and  interactive  nature  of 
this  format. 

C The  researchers  met  together  to  reach  agreement  on  each  line  coded.  If  any  discrepancies  were 

encountered  their  codings,  the  three  researchers  discussed  the  statement  and  then  came  to  consensus. 
At  that  time,  the  researchers  added  another  code  type,  supportive  statements;  it  became  number  9. 


Table  1 

Types  of  Discussion  Participation  Coding  Scheme 

SUBSTANTIVE:  messages  that  relate  to  the  discussion  content  or  topic. 

Code  1 . Structuring  - Statements  which  initiate  a discussion  and  focus  attention  on  the  topic  of  the 

discussion.  These  statements  are  often  made  by  the  discussion  leader  or  instructor  (i.e.  “Today  we 
are  going  to  discuss  . . .”  or  “This  week’s  discussion  will  focus  on  . . .”). 

2.  Soliciting — Any  content-related  question,  command  or  request  which  attempts  to  solicit  a response 
or  draw  attention  to  something  (i.e.  “What  do  you  think  the  author  meant  by  . . .?”  or  “Give  us 
some  support  for  that  assertion.”). 

3.  Responding — A statement  in  direct  response  to  a solicitation  (i.e.  answers  to  questions,  commands, 
or  requests).  Generally  these  are  the  first  response  to  a question  by  a given  individual. 

4.  Reacting — ^A  reaction  to  either  a structuring  statement,  to  another  person’s  comments,  but  not  a 
direct  response  to  the  question,  (i.e.  “Your  earlier  statement  got  me  to  thinking  about . . .”  or  “I 
agree/disagree  with  Bob  because  . . .”). 

NONSUBSTANTIVE:  messages  that  do  not  relate  to  the  discussion  topic  or  content 

5.  Procedural — Scheduling  information,  announcements,  logistics,  listserv  membership  procedures, 
etc. 

6.  Technical — Computer-related  questions,  content,  suggestions  of  how  to  do  something,  not  related 
to  the  topic  directly. 

7.  Chatting — Personal  statements,  jokes,  introductions,  greetings,  etc. 

8.  Uncodable — Statements  that  consist  of  too  little  information  or  unreadable  to  be  coded 
meaningfully. 

9.  Supportive — Statements  that  although  similar  to  chatting,  there  is  an  underlying  positive 
reinforcement  to  the  comment!  (i.e.  “Good  idea!”  or  “Excellent  work!”).  Note:  This  type  was 
added  when  the  researchers  met  for  consensus  of  their  coding  of  the  transcripts. 

Source:  Davidson-Shivers,  Muilenberg,  & Tanner,  1999.  Adapted  from  Pibum  & Middleton  (1998)  and 

Williams  & Meredith  (1996). 


Data  Sources 
Survey 

Data  were  gathered  by  a questionnaire  distributed  three  times  to  the  group.  The  survey  asked  students  to 
report  their  capabilities  using  technology,  web-based  technology  in  particular,  experience  with  the  discipline, 
and  other  demographic  data.  They  were  also  asked  to  critique  the  course  as  it  progressed  through  the  term  in 
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terms  of  technical  difficulties,  interest  in  topics,  preferences  to  types  of  activities,  instructional  strategies  used, 
and  amount  of  work  that  this  course  in  this  format  involved. 

Coded  Transcripts 

Two  threaded  discussions  and  two  chats  were  coded  for  types  of  participation  that  students  made  during 
online  discussion.  The  coding  scheme  used  two  main  categories  of  substantive,  messages  that  related  directly  to 
the  content  or  topic  at  hand  and  non-substantive,  messages  that  did  not  relate  to  the  discussion  topic  or  content. 
The  4 subcategories  for  substantive  were  structuring,  soliciting,  responding,  and  reacting;  the  5 subcategories 
for  non-substantive  were  procedural,  technical,  chatting,  supportive  statements,  and  uncodable.  Every  statement 
or  sentence  was  coded  using  one  of  these  9 subcategories.  ( See  above  Table  1.) 


Discussion  of  Results 

The  analysis  showed  that  participants’  comments  were  made  in  all  substantive  and  non-substantive 
categories  in  the  chats  and  the  listservs,  except  that  there  were  no  uncodable  (albeit,  unreadable  by  the 
researchers)  in  the  threaded  discussions  and  several  in  the  chat  transcripts.  In  addition,  whether  in  a chat  or 
threaded  discussion  for  either  week,  students  overall  provided  more  substantive  statements  either  by  responding 
to  a question  or  reacting  to  another’s  statements  or  comments  than  any  of  the  other  seven  categories.  By  mere 
observation  of  the  length  of  the  responses  by  participants  in  a threaded  discussion,  it  would  seem  that  they 
would  contain  more  substantive  remarks  than  those  in  a chat  discussion,  especially  since  statements  in  the  chat 
tended  to  be  shorter  in  length  and  were  often  interrupted  by  others’  comments.  However,  the  chat  discussions 
showed  greater  numbers  of  responding  and  reacting  statements  in  both  weeks  over  the  substantive  statements 
made  in  the  threaded  discussions.  There  were  also  greater  amounts  of  non-substantive  statements  in  the 
categories  of  chatting  and  supportive  comments  during  the  chats  than  in  the  threaded  discussions.  Overall  there 
were  a greater  number  of  interchanges  and  interaction  when  participating  in  a chat  than  when  involved  in  the 
threaded  discussions.  Thus,  it  appears  that  for  these  two  weeks  of  a given  course,  the  chat  discussions  were 
dynamic  with  a lot  of  interactive  and  interchanges  among  the  participants. 

The  supportive  data  suggests  that  students  liked  both  modes  of  delivery  every  time  they  were  surveyed.  In 
their  comments,  they  stated  that  the  chat  discussions  were  sometimes  difficult  to  follow,  but  still  enjoyed  them. 
One  student  commented  that  they  liked  small  group  chats  over  large  group  ones.  They  enjoyed  the  convenience 
of  the  threaded  discussion. 

When  reporting  their  computer  experience  level,  most  stated  that  they  had  experience  using  computers, 
although  some  stated  that  they  had  little  experience  using  the  Internet  and  WWW.  It  was  noted  that  with  the 
second  survey,  the  reported  level  of  computer  experience  was  lowered  slightly.  But  it  returned  to  the  same  level 
as  reported  in  the  first  by  the  end  of  the  course  (and  in  the  third  survey).  The  reported  ‘dip’  in  computer 
experience  may  be  due,  in  part,  to  technical  difficulties  of  online  services.  And  once  these  were  overcome, 
students  may  have  regained  their  confidence  in  their  computer  abilities. 

The  importance  in  this  study  is  not  so  much  as  to  determine  ‘what  group  won”,  but  rather  to  determine 
which  form  of  discussion  is  preferred  and  which  is  most  appropriate  for  the  types  of  actions  and  interactions 
that  students  need  to  perform  in  order  to  be  successful  in  such  a learning  community.  It  is  also  important  to 
realize  that  these  graduate  students,  for  different  reasons,  enjoyed  both  forms  of  discussion.  The  use  of  chats 
provided  a direct  and  interactive  environment  in  which  they  reacted  and  responded  to  the  topic  at  hand, 
chitchatted,  and  made  supportive  comments  to  each  other.  It  involved  a high  degree  of  interchange  and  was  a 
dynamic  environment,  although  sometimes  difficult  for  some  students’  to  follow  the  train(s)  of  thought.  In 
contrast,  threaded  discussions  provide  an  opportunity  for  students  to  provide  reflective,  thoughtful  responses  to 
posed  questions,  judging  by  the  length  and  wording  of  any  given  single  response.  They  were  also  able  to 
provide  insightful  reactions  to  others’  opinions  and  ideas  presented  perhaps  because  a threaded  discussion’s 
timeframe  (of  a week)  allowed  for  that.  Students  enjoyed  the  chat  because  of  the  convenience  factor.  Both 
discussion  modes — synchronous  and  asynchronous — have  merit  and  may  be  used  within  online  courses.  They 
clearly  can  be  used  for  different  purposes  and  provide  different,  but  useful,  means  for  students  to  engage  in 
learning.  It  is  suggested  that  additional  research  into  the  use  of  chats  and  threaded  discussions  be  conducted. 


274 


References 

Davidson-Shivers,  G.V.  & Rasmussen,  K.L.  (1999).  Designing  Instruction  for  WWW:  A Model.  Proceedings  for  the 
Ed-Media  1999,  World  Conference  on  Educational  Multimedia,  Hypermedia  & Telecommunications.  Seattle,  WA,  USA, 
June  19-24,  1999. 

Davidson-Shivers,  G.V.  & Rasmussen,  K.L.  (1998)  Collaborative  Instruction  on  the  Web:  Students  learning  Together. 
Proceedings  for  the  WebNet98  World  Conference,  Orlando,  FL. 

Driscoll,  M.  (1998).  Web-Based  Training.  San  Francisco,  CA:  Jossey-Bass  Publishers 

Hara,  N.,  Bonk,  C.J.,  & Angeli,  C.  (In  press).  Content  analysis  of  online  discussion  in  an  applied  educational 
psychology  course.  Instructional  Science.  Also  appears  on 

Jeong,  A.  (1996).  The  structure  of  group  discussions  in  online  chats.  Journal  of  Visual  Literacy,  16(1),  51-63. 

Khan,  B.H.  (1997).  Web-based  Instruction.  Englewoods  Cliff,  NJ:  Educational  Technology  Publications. 

McConnell,  D.  (1997).  Interaction  patterns  of  mixed  sex  groups  in  educational  computer  conferences.  Part  I -Empirical 
Findings.  Gender  and  Education,  9(3),  345-363. 

McCormick,  N.B.  & McCormick,  J.W.  (1992).  Computer  friends  and  foe:  Content  of  undergraduates’  electronic  mail. 
Computers  in  Human  Behavior,  8,  379-405. 

Pibum,  M.D.  & Middleton,  J.A.  (1998).  Patterns  of  faculty  and  student  conversation  in  listserv  and  traditional  journals 
in  a program  for  preservice  mathematics  and  science  teachers.  Journal  of  Research  on  Computing  in  Education  31(1),  62- 
77. 


Rasmussen,  K.L.  & Northrup,  P.T.  (1999,  February).  Interactivity  and  the  Web:  Making  and  Maintaining 
Contact.  Paper  presented  at  the  Annual  Conference  of  the  Association  for  Educational  and  Communication 
Technology,  Houston,  TX. 

Sherry,  L.  (1999,  April).  The  nature  and  purpose  of  online  discourse:  A brief  synthesis  of  current  research  as  related  to 
the  WEB  Project.  ITFORUM  Paper  #33.  Also  to  appear  in  The  International  Journal  of  Educational  Telecommunications. 

Shotsberger,  P.G.  (1997)  Emerging  roles  for  instructors  and  learners  in  the  web-based  instruction  classroom.  In  B.H. 
Khan,  (Ed.),  Web-Based  Instruction.  Englewood  Cliffs,  NJ:  Educational  Technology  Publications. 

Williams,  H.L.  & Merideth,  E.  M.  (1996).  On-line  communication  patterns  of  novice  internet  users.  Computers  in  the 
Schools,  12(3),  21-31. 

Wojahn,  P.G.  (1994).  Computer-mediated  communications:  The  great  equalizer  between  men  and  women.  Technical 
Communications,  41(4),  747-52. 


275 


The  Adaptation  and  Use  of  a WWW-Based  Course  Management  System  within 
Two  Different  Types  of  Faculties  at  the  University  of  Twente 


Wim  De  Boer  and  Betty  Collis 
Faculty  of  Educational  Science  and  Technology 
University  of  Twente 
Enschede,  The  Netherlands 
w.f.deboer@edte.utwente.nl,  collis@edte.utwente.nl 


Abstract:  At  the  Faculty  of  Educational  Science  and  Technology,  University  of  Twente,  in  The 
Netherlands,  the  entire  faculty  is  involved  not  only  in  the  use  of  a new  WWW-based  course- 
management  system  (called  TeleTOP)  but  more  fundamentally  in  a new  educational  approach.  In 
addition  we  are  working  with  other  faculties  to  support  the  same  progression.  How  are  we  doing  this? 
In  this  article,  the  TeleTOP  Method  and  implementation  model  are  (http://teletop.edte.utwente.nl)  are 
described  and  the  similarities  and  differences  between  the  implementation,  the  adaptation,  and  the  use 
of  the  TeleTOP  course  support  environment  in  the  Faculty  of  Educational  Science  and  the  Faculty  of 
Telematics  (computer  science  and  networks)  are  analyzed. 


Introduction:  TeleTOP  at  the  University  of  Twente 

The  University  of  Twente  in  The  Netherlands  has  a national  and  international  reputation  in  the  field  of 
telematics,  the  European  name  for  the  branch  of  computer  science  involving  the  combination  of  information  and 
communication  technologies  (in  particular,  related  to  the  Internet  and  the  World  Wide  Web  (WWW)).  A focus  is  the 
application  of  telematics  applications  to  the  teaching  and  learning  process,  what  we  call  "tele-learning".  In  our 
definition,  tele-learning  does  not  necessarily  imply  distance  education  but  instead  emphasizes  the  increased  flexibility 
that  can  come  to  the  teaching  and  learning  process  through  the  combination  of  the  new  possibilities  offered  by  the 
WWW  and  new  ways  of  teaching  and  learning.  The  most  ambitious  of  the  tele-learning  initiatives  at  the  University  of 
Twente  is  the  TeleTOP  project  in  the  Faculty  of  Educational  Science  and  Technology.  Our  core  ideas  are  extending  the 
levels  of  activity  and  engagement  of  our  students  and  extending  the  impact  and  influence  of  our  good  instructors 
(Collis,  1998). 

The  decision  made  by  the  Faculty  of  Educational  Science  and  Technology  in  1997  was  to  involve  all  courses 
in  a re-design  process,  both  pedagogically  and  including  use  of  WWW  tools  and  environments.  This  involves  the 
challenge  of  working  with  a wide  variety  of  courses  and  instructors.  In  the  academic  year  1997-98  we  re-designed  all 
the  first-year  courses,  in  1 998-99  all  the  second-year  courses,  and  by  approximately  the  end  of  the  year  2000  all  other 
courses  (for  more  information,  see  http://teletop.edte.utwente.nl,  through  which  a number  of  courses  can  be  visited  for 
inspection  and  many  publications  and  presentations  are  available). 

To  support  this  new  approach,  we  needed  a technical  platform  and  an  implementaiton  model  that  would  fit  a 
variety  of  different  kinds  of  courses  and  instructors.  We  identified  a core  set  of  requirements  for  the  WWW-based 
course-management  system  that  we  would  need  (Tielemans  & Collis,  1999).  In  our  analysis,  no  existing  system  met 
our  requirements,  so  we  capitalized  on  our  own  experience  and  in  1997  built  our  own  system,  based  upon  a Lotus 
Notes  database.  (This  system  is  described  in  detail  elsewhere;  Collis  & DeBoer,  1998;  Tielemans  & Collis,  1999). 
However,  no  matter  how  elegant  a system  is,  instructors  must  use  it,  an  administration  must  choose  to  support  its  use 
and  thus  must  make  resources  available,  the  technical  infrastructure  already  in  the  faculty  must  handle  it,  and  soft-  and 
hardware  must  be  available.  Thus  an  implementation  model  is  necessary  that  integrates  a variety  of  considerations  and 
change  entities. 

After  the  first  year  and  a half  of  implementing  the  WWW  in  our  own  faculty,  we  began  in  January  of  1999  to 
respond  to  other  faculties  who  also  wanted  to  make  use  of  our  system  and  method.  We  chose  as  the  next  faculty  to 
change  its  way  of  teaching  and  learning  a faculty  within  our  university  with  different  types  of  courses,  students,  and 
instructors  compared  to  our  own  faculty.  This  was  the  Faculty  of  Telematics,  a technical  faculty  with  many  engineering 
and  mathematics  courses.  The  adaptation  proceeded  smoothly  and  we  have  now  moved  to  the  rest  of  the  faculties  in  our 
own  university  as  well  as  faculties  in  other  universities.  In  this  article,  we  describe  the  TeleTOP  Implementation  Model 
which  underlies  all  this  work.  The  Model  indicates,  based  on  literature  and  experiences,  the  change  entities  and  areas 
which  are  of  importance  when  an  institution  is  going  to  implement  the  WWW  in  education  in  a wide-scale  way.  We 
also  describe  the  transfer  experiences  with  the  Telematics  Faculty. 
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The  TeleTOP  Implementation  Model,  Version  1 


In  an  earlier  study  (see  DeBoer  & Collis,  1999)  an  implementation  model  of  the  change  process  was  defined 
with  the  interrelationships  among  three  phases  of  a time  dimension,  five  change  areas,  and  12  change  entities  (see 
Figure  1). 


time 

1 ^ 1 ^ ► 

initiation  implementation  institutionalisation 

Figure  1:  The  TeleTOP  Implementation  Model,  Version  1 

The  Model  is  a direct  reflection  of  the  experiences  within  the  Faculty  of  Educational  science  and  Technology 
since  1997.  Some  aspects  of  the  innovation  have  already  reached  the  institutionalization  phase  (for  example,  all 
students  and  most  faculty  routinely  going  to  their  start  pages  generated  by  the  TeleTOP  system  on  a daily  basis)  while 
new  aspects  are  being  initiated  (such  as  the  capturing  of  instructor  and  student  presentations  on  digital  video  for  the 
reuse  of  segments  of  these  presentations  through  course  WWW  environments  as  video-on-demand  for  a variety  of 
learning  purposes).  The  sizes  of  the  circles  and  ovals  reflect  complexity  in  terms  of  the  number  of  change  areas 
involved.  The  importance  of  key  figures  in  the  initiation  phase,  including  all  of  the  members  of  the  faculty 
administration,  had  an  early  impact  on  all  change  areas  in  the  TeleTOP  situation. 


Experiences  at  the  Faculty  of  Telematics 

In  January  1999  it  was  decided  that  the  new  Telematics  Faculty  at  the  University  of  Twente  should  also  set  up 
a WWW-based  course-support  environment  as  was  done  at  the  Faculty  of  Educational  Science  and  Technology.  A 
project  group  (including  the  first  author)  was  set  up  which  would  be  responsible  for  the  technical  and  instructional 
implementation  of  TeleTOP  in  this  new  faculty.  A key  question  was  if  the  educational  culture  and  approach  in  one 
faculty,  as  visualized  by  the  TeleTOP  Implementation  Model  (Figure  1)  could  map  onto  another  faculty.  The  project 
group  consisted  of  two  TeleTOP  team  members  and  two  members  of  the  Telematics  Faculty.  The  Telematics  members 
became  primarily  responsible  for  the  implementation,  while  the  transfer  of  the  TeleTOP  method  and  course- 
management  system  was  the  responsibility  of  the  TeleTOP  team  members. 

Figure  2 gives  an  impression  of  how  the  TeleTOP  team  planned  the  steps  for  the  transfer  and  adaptation  of 
TeleTOP  to  another  faculty  (a  process  repeated  in  March-June  1999  for  a faculty  in  another  university).  Figure  2 can  be 
seen  as  an  operational  version  of  the  TeleTOP  Implementation  Model  shown  in  Figure  1. 
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Figure  2:  Steps  for  adapting  the  TeleTOP  Implementation  Model  and  methodology  at  the  Telematics  faculty 

The  actual  implementation  steps  visualized  in  Figure  2 will  be  summarized  in  the  following  section  (Table  1). 
Comments  will  be  given  as  to  what  was  intended  in  the  steps,  and  what  actually  occurred  in  the  transfer  process  (see 
also  Fisser,  Kamp  & Slot,  1 999), 

Table  1:  The  change  entities  used  in  the  TeleTOP  Implementation  model,  in  terms  of  the  experiences  of  the 


Telematics  Faculty 


Entities 

Degree 
of  fit 

Comments 

1. 

Educational 

target 

+/- 

This  is  corresponding  with:  L Defining  the  target  group,  context,  goals  and 
making  a plan.  In  the  Telematics  Faculty  situation,  while  most  of  these  activities 
occurred,  the  goal  of  this  innovation  was  not  made  explicitly  clear  to  all  involved. 
The  Telematics  faculty  coordinator  planned  a start-up  session  with  all  who  were 
involved  at  the  start  of  the  project,  and  indicated  that  the  initiative  was  very 
important  with  regards  to  providing  more  flexibility  for  the  students.  There  was  not 
a real  educational  target;  more  focussed  on  flexibility  towards  students  and  the 
image  of  the  new  faculty. 

2.  Fit  with 
instructional 
practices 

+ 

The  instructors  indicated  that  they  had  to  change  their  ways  of  teaching,  but  this 
was  within  reasonable  boundaries,  , , , 

3.  Quality 

hardware 

network 

++ 

This  is  corresponding  with:  II,  Setup  the  technical  infrastructure.  The  operational 
model  shown  in  Figure  2 indicates  that  the  technical  infrastructure  for  the  WWW- 
based  course-management  system  should  be  set  up  in  Months  2 and  3,  In  the  case 
of  the  Telematics  Faculty,  this  was  no  problem.  There  was  already  a sophisticated 
technical  infrastructure  in  place  in  the  faculty,  with  an  excellent  network,  and 
multi-media  computers  for  all  students  and  staff 

4.  Key 

figures  to 

initiate 

++ 

The  key  figure  to  initiate  the  educational  change  was  the  faculty  administrator.  In 
this  case  a top-down  approach  initiated  the  process;  some  instructors  already  had 
used  the  WWW  for  their  courses,  but  had  to  change  to  the  new  system. 

5,  Build/buy 

software/ 

hardware 

++ 

This  is  corresponding  with:  II,  Setup  the  technical  infrastructure.  The  only  thing 
needed  to  be  set-up  was  a Lotus  Notes  Domino  server,  and  to  guide  the  faculty 
Web  masters  in  learning  some  new  skills  for  maintaining  it.  The  faculty  invested  a 
considerable  sum  of  money  to  get  the  right  hard-  and  software.  They  chose  to 
buy/license  the  software  from  TeleTOP;  not  build  it  themselves,  because  it  was 
available  and  had  proven  itself  in  practice. 

6.  Top  down 
&.  bottom- 
up 

++ 

As  already  mentioned  in  #4  (Key  figures  to  initiate)  the  administrator  of  the  faculty 
followed  a top-down  approach  introducing  the  change.  The  way  instructors  were 
encouraged  to  explore  the  possibilities  of  the  system  and  were  able  to  use  it  so  they 
would  maximally  benefit  from  the  system  could  be  seen  as  bottom-up. 

7. 

Embedding 
of  use 

+/- 

As  the  initiation  phase  started  in  March  1 999,  the  faculty  is  not  at  the  point  where 
embedding  of  use  should  start.  This  should  happen  in  the  future  (within  1-2  years). 

8.  Innovative 
culture 

+ 

An  innovative  culture  should  make  it  easier  for  an  educational  change.  The  faculty 
is  still  very  young,  so  a characterization  is  difficult  to  give. 

9.  Budget 

++ 

The  commitment  of  the  faculty  management  was  evident,  because  extra  money  and 
resources  were  made  available.  The  planning  should  indicate  when  the  course- 

o 
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management  system  should  be  operational  for  use.  The  faculty  was  willing  to  invest 
in  human  resources  (support  groups)  and  hard-  and  software.  The  instructors 
however  did  not  get  payment  for  their  extra  time  investments. 

10.  Initiation 
target 

+ 

The  initiation  target  in  this  case  was  the  fact  that  the  new  faculty  was  just  ready  to 
start.  This  was  a good  time  to  set  up  new  courses  with  the  course-management 
system. 

1 1 . Project 
team 

++ 

This  is  corresponding  with: 

///.  Technical  support.  Helpdesk.  The  new  supporters,  already  experienced  in 
giving  support,  where  educated  in  using  the  new  software.  The  educational 
supporter  also  learned  to  work  with  the  supportive  educational  tools,  getting  hands- 
on  experiences.  The  project  team  played  an  important  role  in  the  implementation  of 
the  educational  change. 

V & VI,  Instructor  sessions  & Designing  the  course  A series  of  sessions  for  the 
instructors  were  set  up.  The  topics  dealt  with  within  the  sessions  were  technical  as 
well  as  instructional.  After  the  first  session,  during  which  members  of  the 
administration  introduced  their  plans,  a technical  session  followed.  The  course- 
management  system  was  demonstrated  and  instructors  had  the  chance  to  try  it  out 
themselves.  Two  weeks  later  a personal  session  was  arranged.  In  an  one-hour 
interview  examples  of  how  the  WWW  could  be  used  in  support  of  technical 
courses.  Every  other  week  a two-hour  hands-on  sessions  was  held.  Instructors  used 
their  own  course-management  environments,  to  try  things  out,  and  to  exchange 
ideas.  Most  instructors  attended  this  meetings,  and  thought  they  were  worthwhile. 
The  instructors  used  their  time  getting  skilled  in  the  using  the  course-management 
system  at  the  same  time  as  they  were  designing  their  own  courses.  Most  work  was 
done  a few  weeks  before  the  starting  date  of  a course. 

VII.  Student  session.  The  use  of  a course  support-management  system  which  is 
fully  based  on  the  WWW  and  does  not  make  use  of  additional  programs  other  than 
regular  e-mail  proved  to  be  a very  user-friendly  system  for  the  students.  Students 
learned  the  basics  of  the  system  and  were  able  to  work  with  the  system  within  one 
short  session.  All  students  got  a simple  manual.  They  learned  more  about  additional 
features  of  the  environments  as  they  used  them  during  their  courses. 

VIII  & IX,  Start  the  courses  & Educational  support.  The  courses  could  start  after 
the  student  session,  and  after  the  student  had  gotten  access  to  the  course- 
management  environment.  The  use  of  the  WWW  made  it  possible  to  have 
interaction  within  the  environment,  and  the  possibility  to  let  the  environment  'grow’ 
during  the  time  the  course  was  held.  During  the  time  that  the  courses  were  in 
operation,  educational  support  was  available  to  the  instructors,  as  well  as  technical 
support  (see  Step  IV).  Most  instructors  however  were  capable  of  managing  their 
own  environments. 

12. 

Structural 

support 

group 

? 

It  is  not  clear  yet  how  and  who  will  be  supporting  the  instructors  in  the  future.  This 
is  of  great  importance  for  the  successful  institutionalization  of  the  change. 

Thus,  as  summarized  in  Table  1,  the  TeleTOP  Implementation  Model  did  seem  to  reasonably  describe  the 
experiences  of  the  Telematics  Faculty  in  initiating  and  implementing  the  use  of  a WWW-based  course-management 
system. 


Differences  in  the  use  of  a WWW-Based  Course  Management 

For  almost  two  years,  courses  have  been  using  the  TeleTOP  system  at  the  faculty  of  Educational  Science  and 
Technology.  The  Faculty  of  Telematics  has  used  the  system  for  half  a year.  At  both  faculties  an  evaluation  was  carried 
out  to  look  at  the  use  of  the  systems.  It  is  interesting  to  look  at  differences  in  use  between  the  two  faculties,  of  which 
one  is  more  technical  (Telematics)  and  one  of  which  is  a social  science  (Educational  Science  and  Technology).  Are 
there  differences  in  the  use  of  the  TeleTOP  system  and  what  factors  influence  these  differences?  In  particular,  both 
groups  of  instructors  have  found  a WWW-based  matrix  (which  we  call  a roster)  as  an  organizing  structure  for  course 
content  and  activities  to  be  particularly  useful.  Figure  3 shows  the  options  that  were  chosen  by  the  instructors  of 
samples  of  26  courses  of  Educational  Science  and  Technology  and  10  instructors  of  the  Faculty  of  Telematics. 
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Figure  3:  Percentages  of  instructors  using  various  options  (for  Ed.  Science  & Technology,  N=26,  and  for  Telematics, 
N=  10). 


Central  in  the  course  environments  is  the  roster  of  a course,  integrating  study  materials,  contact-session  notes, 
assignments,  communication,  and  feedback.  Figure  4 shows  the  roster  of  Teleleaming,  a course  of  the  faculty  of 
Educational  Science  and  Technology.  The  roster  in  the  TeleLeaming  course  gives  an  overview  of  the  assignments  and 
meetings  of  the  course.  The  course  was  organised  around  contact  sessions.  Instead  of  lectures,  the  students  present 
could  interact  with  each  other  in  a variety  of  ways,  and  students  not  present  would  join  in,  within  the  next  48  hours,  via 
the  course  WWW  site.  For  students  not  able  to  be  involved  for  reasons  of  work  or  distance  as  part  of  the  real-time 
group  during  any  of  the  sessions,  participation  in  the  group  session  was  also  expected,  but  asynchronously  and  via  the 
WWW.  Figure  5 shows  the  roster  of  Switching  and  Control  Systems,  a course  given  at  the  Faculty  of  Telematics.  In 
this  course,  students  were  reasonable  for  presenting  more  than  half  the  lectures.  Students  formed  groups  and  chose  a 
topic,  about  which  they  presented  material  to  their  fellow  students  and  to  the  instructors.  All  material  was  placed  in  the 
session  row  of  the  roster  of  that  particular  week.  The  first  part  of  the  lecture  session  was  the  actual  lecture,  the  second 
part  was  intended  for  questions  and  discussions.  Also  after  the  sessions  the  discussion  would  go  on  in  the  discussion 
area  of  the  site.  The  site  was  therefore  intended  as  a real  support  site,  to  be  used  for  information,  materials  and 
communication. 
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Figure  4:  The  roster  of  Telelearning. 
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Figure  5:  The  roster  of  Switching  and  Control  Systems 


Differences  in  use? 

Looking  at  Figure  3 and  taking  into  account  the  two  examples  of  the  use  of  the  roster  shown  in  Figures  4 and  5,  it  can 
be  concluded  that  the  differences  in  use  of  the  system  between  the  two  faculties  are  not  very  large.  It  seems  that  the 
Faculty  of  Telematics  focuses  a little  more  on  communication  via  the  support  environment,  and  Educational  Science  & 
Technology  focuses  more  on  collaborative  work  through  the  support  environment.  Both  faculties  are  still  in  the 
implementation  phase,  and  have  a long  way  to  go  until  full  institutionalization.  It  is  therefore  interesting  to  follow  the 
faculties  and  see  where  they  will  be  at  in  one  or  two  years.  Maybe  the  differences  will  fade  away  or  maybe  they  will 
increase? 
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Abstract:  As  the  capabilities  of  information  technologies  increase,  the  success  of  developing 
learning  environments  adaptable  to  individual  learner’s  needs  and  preferences  depends  more  on 
design  planning  than  on  technical  features.  This  paper  focuses  on  a design  method  grounded  in 
knowledge  modelling,  evincing  its  use  in  the  design  of  a flexible  learning  environment  for  the  Web, 
aimed  at  three  different  target  audiences.  It  is  proposed  that  modelling  techniques  can  be  used  to 
represent  the  generic  parts  as  well  as  the  specific  activities  of  each  one  of  the  target  audiences  from 
four  different  perspectives:  contents,  pedagogical  strategy,  learning  materials  and  delivery  platform. 


Introduction 

In  the  past  decade,  the  potential  of  learning  environments  adapted  to  the  learners  has  been  put  forward  by  studies 
interested  in  user  centred  design  (Nikolova  and  Collis,  circa  1997;  Larin,  1997;  Hannafin  et  al,  1992).  Recent 
technological  advances  in  the  Web  have  increased  that  interest  because  it  offers  a promising  option  to  deliver  such 
environments,  thanks  to  the  capacity  to  combine  the  advantages  of  different  media,  to  handle  large  amounts  of 
interconnected  information  and  to  facilitate  social  interactions  by  linking  people  throughout  the  world  (Knowles, 
1996).  Before  making  this  potential  concrete,  however,  there  remains  an  array  of  challenges  for  instructional 
designers  since  design  methods  addressing  environments  adapted  to  learners  are  still  sparse  and/or  haven’t  been 
empirically  tested  (Pulkkinen,  1997  and  Larin  1997). 

This  paper  reports  an  experience  using  MISA,  a courseware  engineering  method,  in  the  design  of  a Web  course 
aiming  to  meet  the  needs  and  characteristics  of  three  different  target  audiences.  It  first  describes  the  issues  addressed 
by  MISA.  In  particular,  it  discusses  how  a modelling  approach  has  been  used  in  the  representation  of  four  different 
axes  (content,  pedagogical  strategy,  learning  materials  presentation  and  delivery  infrastructure).  Then,  it  explains 
how  this  method  has  been  adapted  to  represent  the  generic  parts  (shared  by  the  three  target  audiences)  as  well  as 
their  specific  conditions. 


1,  The  Method:  MISA 

For  more  than  seven  years,  a multidisciplinary  team  at  LICEF  Research  Centre  has  engaged  in  the  development  of 
tools  to  assist  the  designers  of  teleleaming  environments.  The  expertise  from  various  fields  has  resulted  in  the 
development  of  a courseware  engineering  method  named  MISA  (French  acronym  for  Learning  Systems  Engineering 
Method).  MISA  proposes  an  approach  to  learning  systems  design  based  on  4 axes  that  are  created  individually  but 
are  closely  related.  The  multidimensional  structure  of  each  axis  is  represented  through  a knowledge  modelling 
technique  (Paquette  et  al,  1998).  A software  called  MOT  is  used  to  graphically  describe  different  types  of 
knowledge  and  their  links  (Leonard  and  Lalonde,  1997). 
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Empirical  evidence  of  the  effectiveness  of  MISA  is  elicited  by  the  successful  application  of  its  principles  in  different 
courses  designed  to  be  delivered  in  telematic  contexts.  MISA  doesn’t  presuppose  a particular  pedagogical 
orientation  or  a specific  technology;  it  is  learner  centred  and  considers  eventual  differences  between  target  audiences 
according  to  their  profile.  The  attributes  of  the  target  audiences  determine  the  basis  upon  which  the  modelling  of 
axes  is  to  be  done.  To  identify  these  attributes,  MISA  offers  a set  of  electronic  forms  including  sections  to  describe 
each  target  audience.  Currently,  MISA  considers  different  target  audiences  as  belonging  to  one  single  organisation. 
This  may  be  seen  as  a constraint  in  the  design  of  learning  systems  involving  several  organisations  in  which  each  one 
of  them  brings  their  own  set  of  target  audiences.  In  order  to  analyse  this  limit,  MISA  has  been  used  in  the  design  of 
a Web  course  aimed  at  three  different  organisations,  corresponding  to  three  professional  associations:  accountants, 
managers  and  notaries.  The  next  sections  describe  the  process  of  examining  this  issue. 


2.  Target  Audience  Profile 

The  first  phase  of  MISA’s  pedagogical  engineering  process  addresses  the  learner’s  profile.  Since  MISA  considers 
different  target  audiences  as  belonging  to  the  same  organisation,  each  professional  association  (PA)  has  been  taken 
into  account  as  an  organisation  on  its  own.  In  so  doing,  it  was  possible  to  get  a general  characterisation  of  each  PA 
as  well  as  to  identify  the  profile  of  their  own  different  target  audiences.  Among  the  distinguishing  attributes  were 
the  number  of  participants,  language,  availability,  learning  preferences  and  attitudes. 

The  information  gathered  from  the  three  PAs  culminated  in  a generic  profile  of  the  target  audiences.  Generic 
attributes  were  the  academic  level  (a  university  degree)  and  similar  computer  literacy.  All  of  them  were  familiar 
with  Web  navigation.  The  characteristics  of  specific  target  audiences  were  drawn  and  compiled  in  families 
pertaining  to  a single  generic  trunk  with  branches  both  specific  and  generic.  The  target  audience  profile  would 
define  the  orientation  of  the  four  models  described  below. 


3.  Subject  Matter  Model 

The  modelling  of  the  subject  matter  has  been  done  in  the  second  phase  of  the  engineering  process.  The  challenge 
was  to  create  a unique  model  that  could  be  adapted  to  each  of  the  professional  associations.  A procedural  model 
was  chosen  in  order  to  accommodate  the  general  objective  of  the  course:  to  use  information  technologies  (IT)  to 
improve  the  effectiveness  of  professionals  in  their  tasks.  Figure  1 shows  the  first  level  of  the  subject  matter  model. 


Figure  1.  Subject  matter  model 

In  this  model,  the  main  knowledge,  “Using  IT  in  professional  activities”,  is  a process  (grey  oval)  having  as  input  the 
knowledge  unit  “IT  tools”  which,  in  turn  is  determined  by  the  type  of  task  to  be  done.  The  general  process  starts  by 
defining  the  task  to  be  accomplished  and  is  followed  by  the  identification  of  the  appropriate  tool  to  be  employed. 

Since  this  generic  model  had  to  represent  the  specificity  of  each  target  audience,  different  means  were  used.  First,  a 
ruling  principle  was  applied  to  the  main  knowledge  unit.  The  hexagon  “Professional  Context”  indicates  that  the 
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leading  process  varies  according  to  the  context  in  which  ITs  are  to  be  used.  In  this  case  the  context  is  given  by  the 
professional  associations  (A=Accountants;  M=Managers;  N=Notaries). 

Secondly,  a learning  skill  has  been  associated  with  each  knowledge  unit  according  to  the  requirements  of  different 
target  audiences.  For  example,  the  learning  skills  “Memorise”,  “Analyse”  and  “Transfer”  could  be  joined  to  the 
knowledge  unit  “IT  tools”  to  respectively  illustrate  the  needs  of  each  professional  association. 

Finally,  the  creation  of  sub-models  enabled  us  to  represent  a knowledge  unit  as  seen  from  different  perspectives. 
The  knowledge  “Type  of  tasks”,  in  figure  1,  has  been  further  divided  into  five  knowledge  units  corresponding  to  the 
five  sorts  of  tasks  identified  as  important  in  the  subject  matter  model:  self-management,  information,  production, 
collaboration  and  assistance.  These  tasks  were  created  as  a sub-model  inside  the  hexagon  “Type  of  tasks”. 

For  each  professional  association,  a series  of  tasks  embodying  their  professional  practice  were  selected.  A 
comprehensive  analysis  of  the  specific  tasks  led  to  the  classification  of  families  of  tasks  named  “common  tasks”  that 
were  shared  by  at  least  two  target  audiences.  The  common  tasks  were  integrated  into  the  generic  task  branches  of 
the  model  and  were  separated  into  new  sub-models  to  represent  the  specific  tasks. 

4.  Pedagogical  Model 

The  third  pedagogical  engineering  phase  suggests  the  description  of  the  activities  available  to  the  learners  as  well  as 
to  the  instructors  (if  any)  through  a pedagogical  model.  This  model  indicates  how  the  activities  are  organised  and 
which  learning  materials  and  tools  are  necessary  to  accomplish  them. 

The  pedagogical  model  is  nurtured  by  the  subject  matter  model  and  is  oriented  by  the  target  audience  profile 
depicted  in  the  first  phase  of  the  engineering  process.  Since  the  specific  aspects  to  be  considered  in  a learning 
environment  can  be  numerous,  the  diversity  of  pedagogical  possibilities  in  a learning  environment  adapted  to 
learners  can  be  difficult  to  represent. 

To  facilitate  this  kind  of  modelling  the  activities  were  combined  to  form  groups  named  “learning  units”.  The 
purpose  was  to  discern  different  alternatives  according  to  the  two  main  learning  preferences  of  the  target  audiences: 
guided  learning  or  autonomous  learning.  The  model  offers  different  possibilities  to  the  learners  through  seven 
learning  units  (LU).  All  LUs  are  generic,  excepted  for  LU5,  which  was  conceived  to  answer  some  specific  needs  of 
one  target  audience:  the  manager’s  professional  association  (M). 

The  first  learning  unit  (LUl)  introduces  the  general  concepts  regarding  IT  and  enables  the  learner  to  get  a glimpse  of 
the  different  decisions  s/he  can  take,  among  others,  to  select  the  content  to  learn  and  the  pedagogical  path  to  follow. 
The  pedagogical  model  is  not  linear  and  after  completing  LUl,  the  learner  can  choose  any  other  LU.  To  assist  the 
learner  in  her/his  choice,  LU2  offers  activities  to  help  her/him  identify  her/his  profile  whereas  LU3  contains 
activities  to  help  in  the  planning  of  her/his  learning  process. 

If  a learner’s  preference  is  for  autonomous  learning,  s/he  can  accomplish  the  tasks  in  LU4,  which  are  exploratory  in 
nature.  LU4  covers  the  same  subject  matter  as  LUs  5,  6 and  7,  but  the  learner  can  decide  on  the  content  to  be  learned 
and  can  progress  according  to  her/his  own  pace.  Assistance  of  tutors,  pairs  and  advisors  is  always  available. 

When  a learner’s  preference  is  for  guided  learning  s/he  can  choose  the  content  to  learn  from  LU6  or  LU7.  Whereas 
LU6  aims  to  improve  the  effectiveness  of  a professional  when  doing  a research  task  on  the  Web,  the  LU5  intends  to 
enhance  effectiveness  in  communication  and  collaborative  tasks. 

As  stated  before,  the  pedagogical  model  is  generic;  its  specificity  is  emphasised  in  the  activities  of  each  LU  through 
1)  the  principles  applied  to  the  learning  materials  and  2)  the  sub-models  in  pedagogical  scenarios  representing 
distinctive  characteristics. 

The  pedagogical  scenarios  contain  the  activities  of  a LU  describing  the  interventions  of  learners  and  the  tutors  (if 
any).  Figure  2 shows  the  representation  of  the  learners’  activities  of  LU  6. 
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Figure  2.  Learning  scenario  of  LU6 

In  this  model,  principles  (hexagons)  designating  the  target  publics  have  been  used  to  refer  to  the  nature  (generic  or 
specific)  of  the  inputs  and  products  of  the  activities.  The  principle  PA  indicates  that  the  material  exists  in  three 
versions,  (one  per  professional  association).  “N”  implies  that  the  element  applies  only  to  notaries.  The  absence  of  a 
hexagon  indicates  that  the  element  is  generic,  i.e.  is  the  same  for  the  three  target  audiences. 


5.  Media  modelling 

The  media  model  has  been  constructed  during  phase  4 of  the  MISA  method  and  according  to  the  orientation 
principles  proposed  in  phase  2.  These  principles  had  stated  the  types  of  learning  materials,  the  selected  media 
support,  their  size  or  duration  properties,  and  their  actual  status:  existing,  to  be  adapted  or  to  be  built  from  scratch. 

For  each  learning  material,  its  media  model  describes  its  main  components.  For  example,  Web  pages,  video 
sequences  or  book  chapters  are,  each  in  turn,  subdivided  into  smaller  components,  down  to  the  media  element  which 
will  correspond  to  a text,  an  image,  an  audio  or  a video  file  called  “content  objects”.  These  media  components  and 
elements  are  structured  together  by  transition  links  and  regulated  by  principles  (norms  to  guide  their  presentation). 

Each  learning  material  is  associated  with  one  or  more  target  populations.  Some  of  the  materials  are  generic, 
intended  to  be  used  by  every  target  population  even  outside  a specific  course,  for  example,  the  "User  Guide  of  the 
Virtual  Learning  Environment".  Others  are  specific  to  a target  population  such  as  the  case  studies  "Will  in  Trust"  for 
the  notaries,  "Exporting  marble  products"  for  the  managers  or  "The  Verdi  inc."  for  the  accountants;  while  others  are 
specific  to  the  course  but  common  to  all  target  populations,  such  as  “Tools  for  Efficient  Communication”. 

In  the  media  model,  learning  materials  and  their  components  are  organised  around  a metaphor,  in  this  case  an 
exposition  hall  on  information  and  communication  technology.  Such  a metaphor  makes  it  easy  to  separate  the 
generic,  common  and  specific  components,  while  keeping  them  related  in  a consistent  way.  The  metaphor  also  gives 
a concrete  meaning  to  the  pedagogical  scenarios  through  an  analogy  with  a familiar  situation. 

In  our  case,  the  exhibition  hall  contains  booths  corresponding  to  the  generic  tasks  in  the  pedagogical  scenarios:  Self 
management:  the  “Welcome  Kiosk”  (Learning  Units  1,  2 and  3 );  Exploring  tools:  the  “Smart  Explorer”  (LU  5); 
Information  search:  the  “Successful  Prospector”  (LU  6);  Production:  the  "Efficient  Communicator"  (LU  7); 
Collaboration  and  assistance:  the  “Bistro”,  the  “Auditorium”  and  the  “Idearium”  are  all  sections  of  the  exhibition  in 
which  contacts,  exchange  of  ideas  and  assistance  can  be  requested  or  given  (LU  5). 

On  that  basis,  a media  model  of  the  website  could  be  built  as  a MOT  graph  representing  media  components  and 
elements  as  concepts  (rectangular  boxes),  transition  links  as  procedures  (oval  boxes)  and  norms  as  principles 
(hexagonal  boxes).  Figure  3 presents  the  main  model,  together  with  the  corresponding  main  Web  page  of  the 
exposition  hall.  In  the  model,  the  transition  links  a,  b,  c,  d,  e and  f,  lead  to  pages  where  other  components 
correspond  to  activities  and  resources  that  will  be  accessible  in  the  future  website. 

Based  on  the  orientation  principles  and  each  target  population  profile,  we  have  first  built  a generic  model  (mainly 
independent  from  the  subject  matter  but  relevant  to  the  metaphor).  Then,  in  a second  step,  we  have  identified  the 
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specific  media  components  addressed  to  each  target  population,  and  also  the  common  components  and  their  media 


The  main  media  model  of  figure  3 shows  how  we  have  grouped  the  generic  aspect  related  to  the  metaphor,  for 
example  the  booths.  It  shows  the  common  components  linked  to  the  course  content,  for  example  "Principles  of 
communication"  in  the  Efficient  Communicator’s  booth.  Each  booth  has  transitions  towards  specific  files,  such  as 
the  assignments  for  learning  activities  using  situational  descriptions  such  as  "Mediation  familial"  (for  notaries)  or 
" PI  an  ifi  cation  strategique"  (for  managers). 

Important  constraints  have  influenced  this  media  model.  We  wanted  to  have  a single  interface  for  every  target 
population  in  order  to  minimise  the  development  costs,  while  still  ensuring  that  each  target  population  would  have 
access  to  tools  and  learning  materials  appropriate  to  them.  To  achieve  this,  the  generic  and  common  components 
have  been  integrated  in  the  main  model,  while  the  specific  components  have  been  represented  in  sub-models  of  the 
media  model  and  their  corresponding  Web  pages. 

Finally,  in  a third  step,  to  insure  consistency  between  this  media  model  and  the  knowledge  model,  each  main 
component  of  the  media  model  has  been  associated  with  a sub-model  of  the  knowledge  model  that  had  been 
developed  in  phases  2 and  3.  This  is  particularly  important  for  some  material  components  such  as  the  self-diagnosis 
grid,  which  requires  the  learner  to  answer  questions  about  his  or  her  actual  skills,  followed  by  a piece  of  advice  from 
the  system  on  the  next  activities  to  tackle. 

6.  Delivery  modelling 

Also  in  phase  4,  a delivery  model  has  been  developed  in  tandem  with  the  media  model,  based  on  the  orientation 
principles  stated  in  phase  2 and  revised  in  phase  3.  The  principles  had  identified  the  technological  infrastructure 
(networks  and  tools)  and  the  services  (for  example,  of  a training  manager)  required  for  the  delivery  of  the  course. 
The  delivery  model  is  another  specialised  MOT  graph  based  on  the  relations  between  users  requiring  services  to 
access  the  learning  materials,  tools  and  telecommunication  services. 

In  our  case,  there  were  many  challenges.  We  had  to  provide  Web-based  training  including  asynchronous  as  well  as 
synchronous  activities.  We  were  to  build  a single  website  for  three  professional  associations,  each  having  their 
specific  needs  to  reach  learners  distributed  at  home  or  at  work  within  a large  territory  in  Quebec,  using  various 
communication  services  that  would  require  the  participation  of  many  service  providers. 

The  Explora  virtual  learning  centre  model  and  system  (Paquette,  1997;  Girard  et  al,  1999)  has  been  a good  way  to 
fulfil  all  these  conditions.  This  system  is  now  in  full  operation  at  Tele-universite  and  in  other  organisations.  The 
Explora  system  supports  learners  and  the  other  actors  in  their  interactions.  Each  actor  has  a specific  environment 
enabling  him/her  to  manage  his/her  activities,  consult  and  produce  information,  engage  in  collaboration  and  receive 
assistance  from  different  sources.  Some  of  these  resources  are  linked  to  the  specific  website,  such  as  the  one 
intended  here  for  professional  associations,  to  provide  user  tracking,  reports  on  progress  and  counselling. 

Using  the  Explora  system,  we  were  able  to  provide  an  efficient  teleleaming  environment  to  each  of  the  three  target 
populations,  using  either  broadband  communication  from  Bell  Canada  for  ADSL  lines  and  from  Videotron  for 
cable-modem  lines,  as  well  as  narrower  ISDN  lines  more  available  to  some  of  the  participants. 
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7.  Results  and  Discussion 


The  four  models  developed  in  this  project  have  been  useful  to  structure  the  main  dimensions  of  a teleleaming 
environment,  producing  a course  design  that  proved  to  be  an  efficient  way  to  communicate  with  the  development 
team,  and  also  to  prepare  the  delivery  of  a Web-based  course  and  its  technological  environment. 

Graphical  modelling,  as  proposed  by  the  MISA  method,  has  been  a good  way  to  represent  the  course  content  as 
related  knowledge  and  skills,  the  learning  scenarios  with  their  activities  and  resources,  the  structure  of  the  website 
based  on  a metaphor,  as  well  as  the  delivery  infrastructure  and  services.  The  graphical  representation  of  these 
elements  has  helped  the  design  team  to  produce  a consistent  blueprint  of  the  system  in  all  its  dimensions,  while 
considering  the  many  alternatives  possible  to  fulfil  the  learner’s  training  needs.  It  has  been  helpful  in  decision 
making  on  how  to  support  different  target  populations,  types  of  learning  materials  and  deliveiy  infrastructure 
components,  within  the  same  website. 

Two  limitations  have  been  identified.  First,  the  many  models,  generic  and  specific,  each  with  their  sub-models,  even 
with  a well  developed  and  useful  representation  tool  like  MOT,  have  proved  a bit  cumbersome.  Second, 
instructional  engineering  is  a complex  iterative  process  that  requires  constant  updating  of  the  models.  Revising  one 
model  entails  modification  in  the  others,  so  the  designer  has  to  be  helped  in  his  or  her  tracking  efforts  for  possible 
inconsistency  between  models. 

To  overcome  these  barriers,  the  LICEF  Research  Centre  will  deliver  a more  powerful  knowledge  editor,  which  is  an 
extension  of  the  actual  MOT  software,  based  on  a effort  started  in  spring  1997.  This  software  has  new  capabilities  to 
link  different  models  together  so  that  the  designers  more  easily  identify  incompatibilities.  Also,  some  help  functions 
are  integrated  to  support  team  collaboration,  as  well  as  variant  scenarios  or  media  models  addressed  to  different 
target  populations. 

A further  effort  was  started  in  the  summer  of  1 999,  to  built  ADISA,  a Web  based  performance  support  system  for 
designers  that  will  cover  all  the  30  main  tasks  and  processes  of  the  MISA  method.  ADISA  will  integrate  seamlessly 
the  actual  and  the  future  MOT  modelling  tool,  together  with  forms  describing  the  main  objects  such  as  knowledge 
units,  learning  activities  and  resources,  media  components  or  deliveiy  services.  ADISA  will  enable  designers, 
possibly  working  at  distant  locations,  to  work  together  on-line  or  off-line  to  design  teleleaming  environments. 
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Abstract:  The  design  of  an  online  knowledge  community  is  described. 

A online  knowledge  community  serves  as  the  socio-technical 
infrastructure  for  tele-learning.  The  E-Study  Europe  project  is  taken  as  an 
example  of  the  design  and  development  of  an  international  online 
knowledge  community  in  an  university  setting. 


Introduction 

Developments  in  technology,  society  and  new  pedagogies  have  changed  the  education  last 
years  (Tempelaars,  Jager  & de  Vries,  1999).  New  technologies  offer  a wide  variety  of  instructional, 
information  and  communication  functionality.  With  the  development  of  new  technologies  a whole  new 
range  of  possibilities  become  available. 

The  educational  concept  we  apply  is  the  online  knowledge  community.  In  the  E-Study  Europe 
project  a knowledge  community  is  being  developed  to  foster  tele-learning  activities  in  Eastern  Europe. 
To  enable  tele-learning  there  is  a need  for  a socio-technical  infrastructure.  A socio-technical 
infrastructure  refers  to  a hardware  and  software  infrastructure  embedded  into  an  organizational 
infrastructure.  The  main  question  of  this  paper  is  how  a international  online  knowledge  community  can 
be  designed. 


Online  knowledge  communities 

We  see  an  online  knowledge  community  as  a group  of  like  minded  people  who  band  together 
for  frequent  interaction  by  means  of  (an)  online  gathering  place(s),  where  they  share  knowledge  and 
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work  together  onto  a specific  knowledge  domain  through  communication  and  knowledge  services, 
because  it  is  mutually  beneficial.  Online  implies  the  (technical)  infrastructure  that  is  used  to  unite  the 
group  of  people  and  to  enable  them  to  share  their  interests.  The  key  elements  of  an  online  knowledge 
community  are  the  members,  the  shared  interests,  and  the  instrumentality.  The  members  can  be 
described  by  role  models.  We  distinct  for  instance:  the  organization,  the  expert,  the  master,  the  external 
and  the  apprentice  role  model.  The  shared  interests  can  be  described  by  the  mission  of  a community. 
Knowledge  sharing  and  knowledge  creation  activities  in  a community  require  thoughtful  and  careful 
design.  Two  educational  views  are  used  for  the  design  of  a socio- technical  infrastructure  for  a 
knowledge  community: 

• constructivist  view  of  learning.  This  view  implies  that  knowledge  is  not  seen  as  an  accumulated 
body  of  empirically  verified  facts,  but  rather  is  derived  directly  from  observation  and 
experimentation.  In  this  view,  it  is  assumed  that  knowledge  is  something  that  is  constructed  by 
people.  Learning  concerns  the  individual  construction  of  knowledge  as  schemata,  concepts, 
attributes,  links,  forms,  etc. 

• cognitive  apprenticeship  approach.  In  this  approach,  a master  and  apprentice  work  together  on 
tasks  in  order  to  enable  the  apprentice  to  become  a master.  The  purpose  of  Cognitive 
Apprenticeship  is  to  transfer  conceptual  and  problem-solving  knowledge  in  an  integrated  and 
connected  way  (Collins,  Brown  & Newman,  1986). 

Based  on  these  educational  viewpoints  a technological  design  encompassing  the  technical 
configuration  as  well  as  more  social  requirements  like  the  functionalities,  services  and  resources 
available  is  needed.  From  this  point  of  view,  four  design  requirements  for  the  technical  system  can  be 
formulated: 

Technical  design  requirements: 

• The  system  has  to  be  functional.  The  technical  system  should  include  all  the  functionalities  users 
want  from  the  information  system  and  it  should  support  the  user  while  working  with  it. 

• The  system  structure  should  be  efficient.  The  design  of  the  system  should  provide  appropriate 
information  to  the  user  and  should  support  the  navigation  through  the  system. 

Social  design  requirements: 

• The  system  has  to  be  usable.  Usability  means  the  extent  in  which  an  user  can  work  with  the  system 
easily  and  the  amount  of  system  support  the  user  gets. 

• Users  should  acknowledge  that  the  system  is  valuable  to  work  with,  because  of  effective  and 
efficient  system  use. 

The  technical  and  social  requirements  are  presented  in  Table  1 . 


Design  requirements  for  an  online  knowledge  community 
Technical  requirements  Social  requirements 
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Functional  system 

Flexibility 

• Provide  opportunities  that  members  don’t 
have  to  be  physically  present  to  participate  in 
online  knowledge  communities. 

• Provide  continuous  communication  so 
members  could  log  on  the  community 
whenever  it  suited  their  schedules  and  where 
ever  they  are 

• Provide  member  functionalities  to  execute 
needed  (learning)  tasks  and  activities  (based 
on  function  and  task  analysis) 

Useful  system 

Support 

• Provide  a shared,  understood  goal  to 
members  of  the  community  to  learn  about. 

• Embed  learning  / knowledge  creation  in 
social  experience.  Provide  spaces  and 
communication  and  collaboration 
opportunities  that  all  could  share. 

• Present  opportunities  for  support:  for 
modelling,  coaching,  scaffolding,  fading, 
articulation,  reflection,  and  exploration. 

• Provide  both  public  and  private  feedback 
support. 

Efficient  structure 

User  acceptance 

Information-richness 

Interactivity 

• Present  needed  information  to  perform  tasks 

• Establish  clear  lines  of  responsibility  and 

and  store  and  maintain  information  in  objects 

functionalities  for  the  different  role  members 

in  a knowledge  base. 

in  the  community. 

• Provides  links  to  other  useful  (Web) 

• Use  contexts  that  reflect  the  way  the 

resources  and  make  selective  use  of  outsiders 

knowledge  and  skills  will  be  useful  in  real 

for  complementary  insights  and  information. 

life  to  make  the  knowledge  realistic  and 

• Provide  formal  and  informal  environments. 

relevant  and  meet  practical  needs  and 

Let  members  of  all  kinds  benefit  from  having 

development. 

a broader  audience,  see  other’s  work  and 

• Support  mediation  between  the  members  and 

dialogue  about  ideas  and  insights. 

provide  mutual  respect,  tolerance  and  trust. 

• Provide  navigation  tools  that  are  easy  to  use. 

The  members  must  continually  reinforce  the 
collaborative  premise  of  the  community. 

• Foster  a sense  of  collaborative  learning  and 
knowledge  creation  by  including  learning 
scenarios. 

Table  1:  Technical  and  social  design  requirements  for  an  online  knowledge  community. 

The  requirements  in  Table  1 provide  a framework  for  the  design  of  an  online  knowledge 
community  in  the  E- Study  Europe  project. 

The  E-Study  project 

The  main  goal  of  the  E-Study  Europe  project  is  the  development  of  a tele-educational 
infrastructure  for  life  long  learning  purposes  concerning  the  design  of  ICT-applications  for 
organizational  communication  purposes  as  a mean  for  setting  up  a knowledge  community.  The  project 
aims  at  the  implementation  of  an  European  socio-technical  infrastructure  for  enabling  tele-learning  in 
CCE  that  makes  it  possible  for  students  from  the  CCE  to  set  up  and  follow  personalized  curricula  based 
on  available  certificated  online  courses. 

The  project  comprises  seven  partners  from  four  different  countries  (The  Netherlands, 
Bulgaria,  Poland  and  Germany).  Every  partner  has  a specific  role  or  responsibility  in  this  two-year 
project,  for  example  being  the  project  coordinator,  scientific  coordinator,  course  developer  or  design 
expert.  In  the  first  year  of  the  project  the  knowledge  community  was  designed  and  preparations  were 
made  for  the  technical  infrastructure  (computer  network).  At  this  moment  courses  to  be  used  in  the 
knowledge  community  are  developed. 

In  the  E-Study  Europe  project,  the  concept  e-study  is  used  (Vries,  1998).  The  e-study  concept 
refers  to  a socio-technical  infrastructure  for  life  long  learning.  To  enable  life  long  learning  there  is  a 
need  for  a socio-technical  infrastructure,  that  is,  a hardware  and  software  infrastructure  embedded  into 
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an  institutional  infrastructure.  To  describe  such  an  infrastructure,  the  concepts  Interactive  Study 
Environments,  Interactive  Study  Systems  and  online  Study  Services  are  used.  These  concepts  are 
illustrated  in  Figure  1. 


workplace 


knowledge 


Figure  1:  Example  of  a socio-technical  infrastructure  based  on  the  E-study  concept. 


An  Interactive  Study  System  (ISS)  can  be  seen  as  a software  system  consisting  of  study  task 
views,  study  tools,  information  resources,  and  if  needed  additional  materials.  The  main  goal  of  an  ISS 
is  to  enable  an  interactive  study  process  between  a user  of  the  system  and  other  users  and/or  between 
the  user(s)  and  educational  resources. 

To  give  the  needed  support  to  teachers,  there  is  a need  for  Interactive  Study  Environments 
(ISE's).  An  ISE  is  a software  system  for  the  preparation,  application,  and  evaluation  of  ISS’s.  Such  an 
environment  consists  of  the  following  basic  components:  preparation,  application,  and  evaluation  tools 
and  Interactive  Study  System  Components.  The  main  goal  of  an  ISE  is  to  improve  the  effectiveness, 
efficiency  and  attractiveness  of  the  (re-)  design,  use  and  evaluation  of  ISS’s. 

In  the  E-Study  concept  teachers  and  learners  will  be  supported  by  means  of  a wide  range  of 
communication  and  information  services,  summarised  in  the  concept  online  Study  Services.  For 
instance,  educational  publishers  as  ‘content  owners’  may  act  as  a Study  Service  Provider.  They  provide 
teachers  and  learners,  who  apply  study  books  of  the  involved  Educational  Publisher  with  services 
linked  to  the  study  books.  In  the  e-study  concept  ISS’s,  ISE’s  and  study  services  are  interrelated. 

In  the  E-study  community  the  mission  is  ’the  acquisition,  creation,  management,  sharing,  and 
incorporating  of  knowledge  about  the  design  of  media  applications  for  educational  and 
communicational  use'. 

The  infrastructure  of  a knowledge  community  refers  to  the  ict-applications  which  form  a 
meeting  place  for  the  members.  This  meeting  place  enables  them  to  accomplish  their  mission. 
Application  of  modem  ICT  in  education  requires  high  performance  backbones  for  data  transmission.  In 
Figure  2 the  telecommunication  model  used  in  the  E- Study  Europe  project  is  displayed. 
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Figure  2:  E-Study  communication  model 

The  presented  telecommunication  model  enables  a broad  international  propagation  of  the 
projects's  result  and  make  easier  utilization  and  dissemination  at  European  level.  On  the  other  hand,  at 
local  level  the  high-speed  requiring  technologies  could  be  run  on  a local  network. 


Designing  the  E-study  community 

In  the  E-Study  Project  an  Interactive  Study  System  is  seen  as  a study  service  platform  for 
students  and  teachers.  A study  service  or  a closely  related  set  of  study  services  is  made  accessible  by  a 
workspace.  A workspace  is  defined  as  a framework  which  consists  of  a set  of  views  and  interactions 
(with  other  users  and/or  the  study  environment)  needed  to  make  use  of  services.  A view  in  this  context 
is  a set  of  knowledge  objects  dynamically  created  by  a specific  query  on  either  the  Knowledge  Object 
database  or  the  Person/Group  database. 

In  E- Study  we  distinguish  three  basic  types  of  workspaces:  Personal  Workspaces,  Community 
Workspaces  and  Knowledge  Workspaces.  Personal  Workspaces  contain  all  views  and  interactions 
related  to  the  personal  housekeeping  of  the  community  member.  Examples  are  the  personal 
mail/agenda  or  portfolio.  Communication  Workspaces  contain  all  views  and  interactions  related  to  the 
co-operation  and  communication  processes  between  the  community  member  and  the  other  members. 
Examples  are  group  management,  project  management  and/or  communication  tools.  Knowledge 
Workspaces  contain  all  views  and  interactions  related  to  the  creation  and  management  of  knowledge 
objects.  We  distinguish  three  types  of  knowledge  objects:  knowledge  items,  knowledge  collections  and 
knowledge  sequences.  Knowledge  items  are  media  products  containing  distinct  messages.  Examples 
are  documents  (like  a MS  Word  file,  a Quicktime  movie  or  a Webpage)  or  links  to  external  websites  or 
ftp-sites  (URLs).  Knowledge  collections  are  unstructured  collections  of  knowledge  items  or  other  (sub) 
knowledge  collections.  An  example  is  a collection  of  knowledge  items  about  a certain  topic. 
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Knowledge  sequences  are  structured  knowledge  collections  where  a certain  (non-)linear  path  exist 
between  its  components.  An  example  is  a course  which  is  sequenced  by  a time  schedule.  By 
maintaining  metadata,  created  knowledge  objects  can  be  reused  and  shared  among  members  of  the 
local  knowledge  community  or  members  of  other  knowledge  communities. 

Based  on  these  basic  workspaces,  customized  workspaces  can  be  constructed  for  certain  types 
of  interactions  and  views.  In  E-Study  we  distinguish  a Guest  Workspace  (based  on  the  Personal 
Workspace)  and  several  workspaces  based  on  the  Knowledge  Workspace:  Tutorial,  Support,  Design, 
Archive  and  Explorative  workspaces.  In  some  cases,  combinations  of  these  workspaces  are  possible. 

Workspaces  contain  a set  of  views  and  interactions.  We  distinguish  the  following  views  (and 
related  interactions)  in  E-study:  ‘What’s  new?’  View,  Identity  View,  Agenda  View,  Portfolio  / 
Showcase  View,  Records  View,  Personal  Education  Plan  View,  Notations  View,  Groups  View, 
Knowledge  Object  View,  Overview  of  relevant  tools,  Communication  tools,  Collaboration  tools. 

By  using  workspaces  as  a framework  to  organize  views,  interactions  and  tools,  study 
environments  gain  a more  dynamic  and  adaptive  way  to  accommodate  several  types  of  learning  in  the 
knowledge  community. 


Further  developments 

The  development  of  the  knowledge  community  is  still  under  way.  The  next  step  in  the  E- 
Study  project  will  be  the  development  of  courses  to  be  used  within  the  developed  knowledge 
community.  The  courses  planned  to  be  set  up  have  to  contribute  to  a trans-national  on  line  curriculum 
on  ’’Design  and  use  of  Interactive  Media  for  Corporate  Communication”.  Two  trials  are  going  to  be 
carried  out  to  identify  the  viability  of  the  concept  used  in  the  E- Study  project,  the 
effectiveness/efficiency  of  applied  tools,  services  and  approaches. 
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Abstract:  We  describe  a prototype  environment  for  programming.  The 
Specification  Oriented  Language  in  Visual  Environment  for  Instruction 
Translation  (SOL VEIT),  is  designed  to  address  the  novice  programmer’s 
problem  solving  and  programming  skills  by  providing  a collection  of  tools  that 
students  can  use  in  formulating  the  problem,  planning,  designing,  testing,  and 
delivering  the  solution. 


The  SOLVEIT  Prototype 

SOLVEIT  is  a prototype  of  an  integrated  environment  to  support  students  through  all  problem  solving 
stages,  including  formulating  the  problem,  planning  and  designing  a solution,  testing  and  delivering  that 
solution.  Students  who  use  SOLVEIT  not  only  learn  programming  but  also  gain  experience  in  problem 
solving  by  using  strategies  and  tools  that  help  them  understand  the  problem,  develop  a plan  for  solving  the 
problem,  and  then  produce  the  design  needed  to  implement  the  solution.  The  SOLVEIT  system  is  based  on 
an  underlying  process  for  problem  solving  and  program  development  (Deek  1997),  together  with  the  tools 
needed  to  support  that  process.  This  work  is  based  on  the  premise  that  the  programming  language  itself  is 
only  one  of  the  necessary  skills  in  writing  programs  and  that  there  is  a complementary  relationship  between 
efforts  to  develop  environments  and  tools  to  support  problem  solving  and  programming,  and  research  into 
understanding  the  fundamental  cognitive  skills  and  effects  of  learning  programming  (Weinberg  1971; 
Shneiderman  1980;  Mayer  1988;  Hoc,  Green,  Samurcay,  & Gilmore  1990;  Lemut,  du  Boulay,  & Dettori 
1993;  Deek,  Kimmel  & McHugh  1998;  Deek  & McHugh  1998). 

The  Process 

The  conceptual  framework  underlying  the  SOLVEIT  environment  joins  the  problem  solving  method  with 
the  tasks  of  program  development  into  a common  model  for  problem  solving  and  program  development. 
Each  stage  of  this  model  relies  on  specific  activities  to  integrate  and  support  the  tasks  of  both  problem 
solving  and  program  development  (Deek  1997).  Specifically,  the  framework  articulates  the  details  of  the 
activities  required  to  support:  formulating  the  problem,  identifying  a plan  for  solving  the  problem, 
specifying  a solution  design,  translating  the  solution  specification  into  programming  language  code,  testing 
and  delivering  the  results.  Thus,  the  framework  explicitly  unifies  problem  solving/program  development 
into  one  methodology. 

The  Tools 
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The  SOLVEIT  environment  complements  existing  tools  of  traditional  programming  environments  (syntax 
editors,  compilers,  linkers,  loaders,  code  libraries,  debugging  utilities,  tracers  and  code  watchers),  with 
additional  tools  to  support  the  process  of  problem  solving  in  problem  formulation,  planning,  designing  and 
delivering  the  solution;  translating  and  testing  are  facilitated  through  existing  programming  environment 
tools. 

Tools  are  available  for  specific  stages  of  the  process  and  are  used  to  perform  the  various  tasks  of  problem 
solving  and  program  development.  There  are  shared  tools,  such  as  project  notebook,  recorder,  reference 
database,  and  algorithmic  toolbox,  that  can  be  used  in  most  of  the  stages.  Compiling  and  debugging  tools 
are  pre-existing  utilities  that  are  already  available  in  traditional  programming  environments.  The 
SOLVEIT  environment’s  architecture  establishes  relationships  among  the  different  tools  and  components 
that  support  the  activities  within  the  stages  of  the  problem  solving  and  program  development  process,  that 
is,  the  architecture  realizes  or  embodies  the  methodology.  The  tools  for  formulation  and  planning  stages 
were  fully  developed,  i.e.  operational  prototypes,  but  the  tools  for  the  remaining  components  of  the  system 
we  only  simulations.  The  students  used  these  simulated  components  to  mimic  the  activities  of  those  stages. 
Figure  1 provides  an  overview  of  the  functionality  of  this  prototype  system. 


I Jl.  SOLVEIT 


File  SCLVEIT  Tools 


r VcAiikUion  i -V  ^ 

. \ r’ Data  :> 

r la&cmian  ; P ''UiAiwwitf  ><>  Dtta 


- Tranststmn  " 


r PncBMSheU  ;^ 
r StWtiM  Jtbdakia) 


r Int»fr«tE4  ■ V 


P Dui  An«cisilan  ~ 

P rtfkSpecifIntba.; 

P VeriScstiaa''  , ' j 


- DeffyfBfyrr 


m 


P SeWtIf*  Sta^ 
P Canf ktB' 

P Rec«Ed«r 


Figure  1:  An  overview  of  the  SOLVEIT  prototype 
Tools  for  Problem  Formulation 

Three  tools  are  used  for  problem  formulation:  (1)  the  problem  description  editor;  (2)  the 
verbalization  tool;  and  (3)  the  information  elicitation  tool.  The  problem  description  editor  is  invoked  from 
within  the  problem  formulation  stage  to  enter  and  save  the  problem  statement  into  the  system.  If  the 
problem  is  already  in  electronic  form,  the  file  can  be  opened  and  read  into  the  editor.  The  problem 
description  text  is  stored  in  the  multiple- view  reference  database. 

The  verbalization  tool  is  active  in  all  stages  of  SOLVEIT,  but  is  invoked  automatically  in  problem 
formulation  after  the  problem  description  is  saved.  Questions  are  presented  to  the  student  while  the 
problem  statement  is  still  visible  in  the  editor.  The  student  can  answer  the  questions  or  can  skip  the  entire 
verbalization  session.  These  questions  will  provoke  the  student  to  re-examine  the  problem  statement  which 
remains  displayed  throughout  the  interaction  session.  The  results  of  this  verbalization  session  are  saved, 
along  with  subsequent  verbalization  sessions,  by  the  recorder  and  are  available  to  the  student  during  the 
entire  problem  solving  process.  A complete  transcript  is  created  as  part  of  project  deliverables.  The 
verbalization  questioning  in  SOLVEIT  is  general,  but  can  be  adapted  to  fit  a specific  class  of  problems.  In 
addition  to  questioning,  users  have  access  to  a project  notebook/graphics  editor  in  order  to  draw  and  make 
notes  as  necessary.  The  interaction  transcript  is  stored  in  the  project  notebook/graphics  editor. 
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The  information  elicitation  tool  is  invoked  from  v^ithin  the  problem  formulation  stage.  It  is  used 
to  extract  and  organize  relevant  information,  found  within  the  problem  description,  in  a structure  suited  to 
perform  the  transformations  of  subsequent  stages  and  to  carry  out  the  tasks  that  will  lead  to  the  solution. 
This  technique  of  information  elicitation  will  transform  the  problem  description  from  its  text  format  into  a 
database  of  problem  facts.  Using  the  information  elicitation  tool,  the  student  returns  to  the  typed  problem 
and  extracts  the  goal,  givens,  unknowns,  conditions  and  constraints.  All  gathered  information  is  stored  in 
the  multiple- view  reference  database  and  is  accessible  from  within  other  stages  of  SOLVEIT. 

Tools  for  Solution  Planning 

In  conjunction  with  the  tools  used  for  problem  formulation,  there  are  three  tools  used  for  solution 
planning:  (1)  the  plan  definition  editor,  (2)  the  goal  decomposition  tool,  and  (3)  the  data  description  tool. 
The  plan  definition  editor  is  invoked  from  within  the  planning  stage.  The  student  uses  this  simple  text 
editor  to  describe  various  alternatives  and  the  steps  required  for  solving  the  problem,  but  it  does  not  impose 
a specific  solution  or  a rigid  sequence  on  the  student.  This  initial  plan  will  help  coordinate  student’s 
thoughts  and  actions  in  solving  the  problem.  The  plan  is  written  to  the  project  notebook/graphics  editor. 

The  goal  decomposition  tool  is  invoked  from  within  the  planning  stage,  allowing  the  student  to 
begin  transforming  the  goal  identified  in  the  information  elicitation  phase  into  subgoals  that  need  to  be 
completed  to  solve  the  problem.  The  student  refines  the  goal  into  a collection  of  subgoals  and  associates 
two  attributes  with  each  subgoal:  an  identification  name  and  a description.  A goal  can  have  multiple 
subgoals  and  subgoals  themselves  may  be  decomposed  into  smaller  subgoals.  A decomposition  tree  and 
information  about  all  the  subgoals  are  stored  in  the  multiple- view  reference  database. 

The  data  description  tool  is  invoked  from  within  the  planning  stage  to  transform  the  givens  and 
unknowns  identified  in  the  information  elicitation  phase  into  data  representation.  Givens  are  the  problem’s 
input-data  or  constants.  Unknowns  are  output-data  or  intermediate  data  elements.  Using  the  editor,  the 
student  associates  a series  of  attributes  with  givens  and  unknowns:  name,  description,  origin/source,  type 
and  structure.  A data  description  table  is  stored  in  the  multiple-view  reference  database. 

Tools  for  Solution  Design 

Three  tools  are  used  for  solution  design:  (1)  the  module  organization  tool,  (2)  the  module 
communication  tool,  and  (3)  the  module  logic  specification  editor.  The  module  organization  tool  is 
invoked  from  within  the  design  stage.  It  is  used  to  establish  the  hierarchy  of  the  subgoals,  using  a structure 
chart  approach,  already  refined  in  previous  stages  and  now  contained  in  the  reference  database.  There  is  an 
implied  left  to  right  organization  and  top  to  bottom  hierarchy  of  subgoals.  To  reorganize  or  change  the 
hierarchy,  subgoals  can  be  repositioned  vertically  and  horizontally.  New  subgoals  can  be  added,  and  others 
can  be  removed  or  merged.  Information  about  the  organization/hierarchy  of  subgoals  in  the  structure  chart 
is  also  maintained  in  the  reference  database  and  can  be  viewed  in  a convenient  manner. 

The  module  communication  tool  is  invoked  from  within  the  design  stage.  It  is  used  to  establish  the 
data  flow  relationship  between  modules.  Communication  between  modules  is  formed  using  information 
gathered  during  the  planning  and  design  stages,  now  stored  in  the  reference  database.  Additional  data  may 
be  defined  and  added  to  the  database  as  needed.  Data  elements  can  be  associated  with  specific  modules  as 
a sender  or  receiver  of  that  element.  The  result  is  a data  dictionary  table,  also  stored  in  the  reference 
database,  that  includes  the  data  element  names,  type,  description,  the  associated  goal/subgoal  and  the 
direction  of  the  data  flow. 

The  module  logic  specification  editor  is  invoked  from  within  the  design  stage.  It  is  used  once  the 
organization  and  refinement  of  the  structure  chart  and  data  flow  have  been  completed.  This  tool  enables 
the  student  to  develop  module  logic  for  each  box  of  the  structure  chart.  Module  logic  is  constructed  using 
an  algorithmic  constructs  toolbox.  A program  shell,  based  on  the  structure  chart,  is  displayed  in  a “table  of 
contents”  form  to  facilitate  module  logic  development. 

The  “table  of  contents”  gives  the  name  of  the  main  module  and  all  other  program  modules  along 
with  the  data  flow  and  the  direction  of  each.  Each  module  and  associated  entry  in  the  “table  of  contents”  is 
hypertext  sensitive  and  when  selected  it  invokes  the  module  logic  development  window. 

A SOLVEIT  algorithmic  constructs  toolbox,  which  is  used  to  describe  the  logic  for  each  module, 
contains  templates  for  a SOLVEIT-specific  algorithmic  language  consisting  of  basic  data  types,  control 
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structures,  a simple  list  structure  and  modular  abstraction.  Syntax  templates  are  selected  and  copied  into 
the  student’s  workspace. 

The  module  logic  is  saved  in  the  reference  database  as  a text  file  using  the  SOLVEIT  algorithmic 
language.  General-purpose  routines  that  are  developed  in  this  stage  may  be  stored  into  a library  of 
functions  for  reuse  in  other  problems.  Similarly,  any  modules  developed  and  saved  in  earlier  problem 
solving  sessions  that  are  appropriate  for  other  situations  may  be  integrated  and  reused.  The  algorithmic 
logic  created  in  the  specification  editor,  stored  in  the  reference  database,  can  not  be  compiled  directly.  It 
must  be  copied  into  the  syntax  editor  of  the  target-programming  environment  (i.e.  Pascal,  C,  and  C-f-f)  and 
the  student  uses  it  as  the  basis  for  code  translation  from  algorithmic  logic  into  language  syntax. 

The  next  two  stages  of  translation  and  testing  are  performed  within  the  programming  environment. 

Tools  for  Solution  Delivery 

The  file  management  tool  is  invoked  from  within  the  delivery  stage.  It  is  used  to  organize  and 
produce  the  complete  solution  package  consisting  of  the  outcome  of  each  stage  and,  if  desired,  the  content 
of  the  reference  database  and  project  notebook/graphics  editor.  The  result  is  a fully  documented  program 
maintained  in  an  organized  manner  and  can  be  printed  or  saved  on  a disk.  In  addition  to  the  specialized 
tools  in  each  stage  of  the  process,  SOLVEIT  provides  a host  of  environment-wide  tools  that  may  be 
invoked  from  within  any  stage  of  SOLVEIT. 

The  Project  notebook/graphics  editor  is  a text  and  graphics  editor  used  as  a scratch  pad  by  the 
student.  The  student  is  prompted  to  invoke  the  editor  during  the  problem  formulation  stage,  but  can  also  be 
invoked  at  any  time  during  the  problem  solving  process.  Text  and  graphics  capabilities  are  available  for 
the  student  to  make  notes  and  to  draw. 

The  recorder  monitors  the  student’s  activities  within  SOLVEIT  and  acts  as  an  event-capture  log. 
It  allows  the  student  and  teacher  to  replay  the  activities  of  a problem  solving  session  at  a later  time.  Each 
time  the  student  enters  a stage  within  SOLVEIT,  the  event  is  recorded,  and  the  task  performed  is  also 
recorded.  Upon  exiting,  a snapshot  of  the  reference  database  is  copied  to  the  recorder.  Activities  logs  are 
listed  in  the  sequence  they  occurred.  The  recorder  content  cannot  be  edited.  The  multiple-view  reference 
database  contains  the  result  of  all  data  transformation  within  SOLVEIT,  beginning  with  entering  problem 
description  through  the  development  of  the  algorithmic  solution.  The  database  organizes  this  information 
and  makes  it  available  to  the  student  at  different  stages  of  the  problem  solving  process.  The  algorithmic 
constructs  toolbox  is  a logic  template  database  represented  as  icons  and  is  accessible  from  within  the  design 
stage  to  specify  modules’  logic. 

In  summary,  the  process  of  the  SOLVEIT  environment,  the  tools  supporting  this  process  and  the 
existing  tools  of  a traditional  programming  environment  are  combined  to  form  an  integrated  environment 
for  problem  solving  and  program  development. 


Conclusions 

Students’  difficulties  with  learning  basic  programming  go  beyond  the  scope  of  language  syntax  because 
programming  is  an  activity  that  demands  the  development  of  problem  solving  skills.  Command  of 
language  syntax  must  be  acquired,  but  in  conjunction  with  problem  solving  skills  and  analysis  and  design 
methodologies.  With  the  development  of  composite  methodological/software  environments  like  SOLVEIT 
that  supports  teaching  students  programming,  the  role  of  language  syntax  in  the  overall  process  of 
programming  instruction  will  be  redefined  in  a manner  that  gives  appropriate  weight  to  both  language 
issues  and  problem  solving. 
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Abstract:  Authoring  hypermedia  is  a difficult  process  and  any  deficiencies  in  the 
authoring  process  are  likely  to  be  reflected  in  the  student’s  educational  experience. 
This  paper  is  concerned  with  providing  the  author  with  a framework  and  process  so 
as  to  reduce  the  possibility  of  producing  a poorly  structured  domain.  The  discussion 
is  centred  on  practical  experiences  with  an  educational  hypermedia  system  produced 
at  Leeds  Metropolitan  University  that  helps  the  author  to  structure  the  domain  by  the 
use  of  a semantic  network  that  is  used  to  represent  the  essential  structure  of  the 
domain.  Since  it  is  an  educational  system  tutorial  nodes  are  used  within  the 
hypermedia  to  provide  the  student  with  tasks  to  reinforce  their  learning.  The  location 
and  use  of  these  tutorial  nodes  is  discussed,  in  that  they  may  be  used  as  landmarks  to 
help  orientate  a student. 


Introduction 

It  has  been  argued  by  Pereira  et  al  (1991)  that  all  hypermedia  systems  should  have  a semantic  network 
(or  similar  structure)  defined  within  them  to  aid  both  the  author  and  the  end  user.  Jonassen  and  Wang 
(1993)  have  further  suggested  that  structures  such  as  semantic  networks  provide  the  student  with 
structural  information  that  aids  them  in  the  retention  of  hypermedia  material.  Further  studies  have 
shown  that  complex  information  and  arguments  can  be  represented  in  semantic  network-like  web  like 
structures,  termed  mind-maps  or  concept  maps  and  that  the  use  of  such  structures  can  aid  the  author  of 
the  structure  to  remember  the  information  stored  within  them  (McAleese  1998,  Holmes  1999).  Given 
this,  this  paper  discusses  the  implications  and  the  process  of  defining  a semantic  network. 

In  order  to  explore  the  issue  of  authoring  an  educational  hypermedia  domain,  and  other  issues 
described  in  Mullier  et  al  (1998)  and  Mullier  (1999),  a prototype  educational  hypermedia  system, 
called  “Hypemet”,  has  been  developed  at  Leeds  Metropolitan  University.  The  remainder  of  the 
discussion  uses  Hypemet  as  an  example  of  an  educational  hypermedia  system,  but  the  discussion  is  not 
restricted  to  Hypemet  itself 

Domain  Types 

The  exemplar  domain  designed  for  the  conception  and  implementation  of  Hypemet  was  an  astronomy 
domain.  This  domain  was  chosen  for  the  reason  that  it  is  highly  structured  and  hierarchical,  making  it 
ideal  for  application  to  this  system,  since  it  allows  the  rapid  design  of  a semantic  network.  However, 
the  goal  of  the  project  as  a whole  is  to  devise  a paradigm  that  is  as  generic  in  approach  as  possible,  so 
that  it  can  be  applicable  to  a multitude  of  domains.  In  practice,  however,  it  is  likely  to  be  the  case  that 
some  domains  are  easier  to  describe  as  a semantic  structure  than  others.  This  is  because  some  domains 
do  not  exhibit  an  obvious  and  generally  accepted  stmcture.  This  could  result  in  a new  hypermedia 
problem  similar  to  “Linkitis”  (Conklin  1987),  in  that  if  an  author  is  unsure  how  a domain  may  be 
structured  then  they  may  be  tempted  to  force  a structure  on  the  domain  that  may  not  necessarily  exist  or 
be  helpful.  This  may  result  in  the  student  becoming  confused  and  hence  render  the  domain  difficult  to 
navigate,  exacerbating  the  lost  in  hyperspace  problem.  It  is  not  an  ideal  situation,  however,  to  allow  the 
students  to  identify  the  shortcomings  of  a domain’s  design.  It  is  suggested,  therefore,  that  it  is  the 
author’s  responsibility  to  ensure  that  the  domain  is  designed  correctly  before  it  is  exposed  to  students. 
It  is  accepted  that  this  is  not  a trivial  task  however. 

The  authoring  process  is  a complex  one  and  has  often  been  overlooked  by  hypermedia  designers 
(Theng  and  Thimbleby  1998,  Conklin  1987).  It  is,  however,  fundamental  to  the  student’s  learning 
needs  that  the  domain  is  authored  to  a sufficient  standard  (Jonassen  and  Wang  1993).  This  is  especially 
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true  for  “fixed-link”  systems  (systems  that  do  not  provide  dynamic  links).  In  fixed-link  systems  it  is 
vital  that  the  author  provides  the  correct  links  for  every  possible  user  of  that  hypermedia  system.  Bell 
(1995)  argues  that  people  are  not  good  at  writing  in  a non-linear  fashion,  therefore  hypertext 
(hypermedia)  documents  tend  to  become  linear  and  do  not  model  the  true  (semantic)  structure  of  the 
domain  itself  By  producing  hypermedia  systems  that  require  a semantic  network  forces  the  domain 
author  to  consider  the  structure  of  the  domain  and  hence  is  far  more  likely  to  result  in  a domain  that 
provides  an  adequate  structure  for  the  student  to  navigate.  Further,  if  dynamic  links  are  employed  the 
student  is  provided  with  links  that  more  closely  meet  their  requirements  and  so  reduces  their  cognitive 
loading. 

Defining  A Domain 

Here  Hypemet  is  used  as  an  example  hypermedia  tutoring  system.  The  process  of  defining  a domain 
for  use  in  Hypemet  involves  the  following  steps: 

• Structuring  the  domain  into  a semantic  network  (defining  its  structure)  by  providing  typed  links 
between  class  nodes  (class  nodes  may  also  be  used  as  hypermedia  nodes), 

• Forming  a semantic  hypermedia  domain  by  attaching  instance  nodes  to  the  class  nodes  (instance 
nodes  may  be  used  as  hypermedia  nodes), 

• Storing  suitable  exercises  within  tutorial  nodes,  which  are  connected  to  the  semantic  hypermedia. 

For  a standard  hypermedia  system  it  would  usually  be  the  case  that  none  of  the  above  stages  would  be 
employed,  since  the  majority  of  hypermedia  systems  remain  unstructured  (Dillon  and  Gabbard  1998), 
Instead,  the  author  determines  for  every  node  in  the  domain  how  it  connects  to  all  of  the  other  nodes, 
typically  an  extremely  complex  task.  However,  the  application  of  the  semantic  network  encourages  the 
domain  author  to  organise  the  domain  in  a more  logical  fashion.  The  final  stage  is  unique  to  a computer 
based  learning  system,  as  opposed  to  a general  browsing  hypermedia  system.  Since  it  is  difficult  for  a 
student  to  learn  in  an  undirected  environment  (Jonassen  and  Wang  1993)  it  is  appropriate  for  the 
domain  author  to  require  the  student  to  accomplish  some  task,  at  least  until  they  have  become 


The  Semantic  Network 

A semantic  network  can  be  defined  as  a graphical  representation  relating  concepts  and  information  et  al 
Beynon-Davies  1994).  A concept  in  a semantic  network  is  defined  as  a node  and  labelled  arcs  (links) 
define  the  relationships  between  concepts.  The  example  semantic  network,  shown  in  figure  1 , describes 
part  of  a simple  astronomical  solar  system  domain.  The  oval  nodes  represent  class  nodes  and  form  the 
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overall  structure  of  the  domain.  The  square  nodes  represent  the  instance  nodes  that  are  subsequently 
attached  to  the  class  nodes.  Each  class  node  is  therefore  used  as  an  anchoring  point  for  instance  nodes. 
Therefore  every  instance  node  in  the  hypermedia  is  connected  directly  to  the  class  structure  of  the 
semantic  network.  For  example,  the  instance  node  “Earth”  is  attached  to  the  class  node  “Planet”,  thus 
all  “Planets”  are  logically  collected  around  the  “Planet”  class  node.  Each  instance  and  class  node  is 
visible  to  the  student  as  a hypermedia  node.  It  is  therefore  of  potential  use  to  the  student  and  author  to 
utilise  the  class  nodes  for  presenting  higher-level  information  that  relates  to  the  nodes  connected  to  it. 
For  example,  the  instance  nodes  “Earth”,  “Mars”  and  “Jupiter”  are  attached  to  the  class  node  “Planet”. 
The  instance  nodes  contain  information  that  relates  purely  to  the  node  itself  (the  “Earth”  node  contains 
information  about  the  Earth).  The  class  node  “Planet”  may  therefore  hold  information,  useful  to  the 
student,  about  the  higher  level  “Planet”  node  (which  may  not  have  been  present  without  the  need  for 
the  semantic  network)  and  may  provide  information  as  to  why  each  node  attached  to  it,  should  be 
attached  to  it.  In  many  hypermedia  systems  the  only  way  to  discover  this  fact  is  to  traverse  a link  that 
exists  between  the  two  and  rely  upon  the  domain  author  providing  the  information  within  both  nodes 
that  each  is  a volcano.  If  the  student  is  forced  to  undergo  this  procedure  in  order  to  discover  the 
relationship  between  two  nodes  then  they  are  further  exposed  to  the  “risks”  of  navigation  in  a complex 
information  structure.  The  semantic  network  however  reduces  the  danger  of  the  cognitive  loading 
problem  and  navigation  problem  associated  with  the  hypermedia  paradigm,  by  providing  a student  with 
contextual  information  and  by  forcing  the  author  to  structure  the  domain  more  logically  (Pereira  et  al 
1991,  Jonassen  and  Wang  1993). 

Link  Types 

This  standardised  set  of  link  types,  described  fully  in  (Beynon-Davies  et  al  1994)  are  defined  below  as: 

ISA  - relates  an  object  to  a class,  i.e.  it  defines  an  instance  of  a class.  For  example  “Mars”  ISA 
“Planet”,  “Mars”  being  the  instance  of  the  class  “Planet”.  This  link  is  unidirectional  in  that  properties 
are  inherited  in  one  direction  only.  For  example  the  “Mars”  node  inherits  all  properties  from  the 
“Planet”  node  by  virtue  of  this  link  (such  properties  or  attributes  may  include  the  fact  that  a Planet 
orbits  the  Sun,  thus  Mars  orbits  the  Sun).  However  “Planet”  does  not  inherit  properties  from  “Mars”. 
The  IS-A  relationship  usually  represents  information  towards  the  top  of  a hierarchy  and  therefore 
represents  more  general  information  (Tudhope  and  Taylor  1997). 

AKO  (a-kind-of)  - relates  a class  to  another  class,  or  may  define  a subset.  This  link  type  demonstrates 
the  relationship  between  classes.  This  type  is  fundamental  in  connecting  a class  nodes  together  to  form 
one  semantic  network.  For  example  “Planet”  AKO  “Solar  System  Body”,  meaning  that  a “Planet”  is  a 
kind  of  “Solar  System  Body”  and  that  a “Planet”  inherits  all  properties  of  the  class  “Solar  System 
Body”.  Tudhope  and  Taylor  (1997)  have  referred  to  the  AKO  link  as  “narrower-than”,  since  it 
represents  information  at  the  bottom  of  a hierarchy,  which  is  usually  more  detailed  or  narrower  in 
scope. 

PARTOF  (part-of)  - represents  how  an  object  is  composed  of  other  objects,  or  inherits  only  part  of  the 
parent  class.  This  link  type  demonstrates  how  a class  may  be  associated  with  component  parts.  For 
example,  “Geographical  Feature”  PARTOF  "Planet".  The  part-of  link  implies  that  there  is  a 
relationship  between  instances  of  classes  connected  via  a part-of  link. 

HASA  (has-a)  - relates  an  object  to  a property  or  attribute.  This  is  not  used  to  represent  structural 
information  (it  is  not  used  as  a link  type).  Instead  it  may  be  used  to  represent  knowledge  within  a class. 
For  example,  a “Planet  has-a  Diametef ’.  This  may  be  used  to  ensure  that  an  object  conforms  to  a class 
exactly  (i.e.  the  author  is  aided  in  deciding  if  a node  belongs  to  a particular  class  by  virtue  of  it  having 
the  same  attributes).  As  noted  by  Beynon-Davies  et  al  (1994),  this  link  type  also  aids  queries  in  that  a 
search  may  search  for  exact  matches,  such  as  “Planets  with  diameters  greater  than  100”. 

The  above  link  types  allow  the  structure  of  a domain  to  be  represented.  However,  further  link  types 
may  be  required  to  represent  course  material.  For  example,  it  may  be  necessary  to  represent 
dependencies,  if  one  concept  must  be  learned  before  another,  or  to  represent  counter  arguments.  Extra 
link  types  such  as  "dependent-on"  could  be  used  by  hypermedia  systems  to  determine  whether  a node  is 
relevant  to  a particular  student.  If  a student  has  not  visited  the  node  at  one  end  of  the  link  then  the  node 
at  the  other  end  is  not  relevant  and  may  be  hidden  or  greyed  out  in  accordance  with  the  hiding  method 
discussed  in  Brusilovsky  (1996).  The  "dependent-on"  link  can  therefore  be  used  to  represent  course 
information,  where  it  is  common  to  lead  a student  through  topics  that  are  dependent  upon  each  other. 
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Further  link  types  may  include  ”example-of’,  ”counter-example-of’,  “supported-by”  and  “disputed- 
by”.  Link  types  such  as  these  provide  a greater  depth  of  information  about  a particular  node.  Therefore 
it  may  be  necessary  to  separate  a node  into  several  nodes  representing  various  levels  of  detail.  It  is 
arguable  that  some  node  types  such  as  ”counter-example-of’  may  not  be  beneficial  to  lower-level 
students,  since  they  may  contain  contradictory  or  otherwise  confusing  information,  therefore  there  may 
be  a case  to  make  such  links  available  to  only  the  higher  level  students.  Further,  in  the  area  of  concept 
mapping,  which  can  be  said  to  share  some  common  attributes  with  hypermedia,  there  can  be  any 
number  of  author  (of  the  concept  map)  definable  link-types  (McAleese  1998).  In  essence  it  does  not 
matter  whether  the  domain  author  uses  a predefined  set  of  link  types  or  whether  they  use  their  own. 
Other  researchers  have  attempted  to  produce  automatic  links  differently,  depending  upon  the  link  types 
that  connect  two  remote  nodes.  Tudhope  and  Taylor  (1997)  calculate  automatic  links  by  giving  a 
stronger  relationship  between  nodes  connected  by  AKO  relationships  than  IS-A  relationships.  The 
reason  for  this  is  that  relationships  at  the  bottom  of  hierarchies,  represented  by  AKO  relationships,  are 
thought  to  be  more  closely  related  than  those  at  the  top,  represented  by  IS-A  relationships  (Tudhope 
and  Taylor  1997).  The  use  of  different  strengths  of  relationships  has  not  been  explored  as  part  of  this 
research  project,  although  it  may  prove  to  be  beneficial  as  part  of  further  work.  Further,  it  may  be 
possible  to  combine  the  information  potentially  available  from  the  student’s  browsing  pattern  to 
dynamically  alter  the  strength  of  the  link  types,  in  that  if  a user  is  determined,  by  virtue  of  their 
browsing  pattern,  to  be  interested  in  general  information  then  the  strength  of  the  IS-A  link  type  could 
be  increased  and  vice  versa. 

Adding  Nodes  to  the  Domain 

Once  a semantic  network  has  been  defined  for  the  domain  in  terms  of  class  nodes,  then  instance  nodes 
may  be  added  to  it.  The  author  must  decide  to  which  class  a new  node  belongs,  a class  being 
represented  by  a class  node  (connected  to  other  class  nodes  via  ISA  and/or  AKO  link  types).  If  there  is 
no  such  class  node,  then  the  semantic  network  has  not  been  sufficiently  defined  for  the  domain  or  the 
new  node  does  not  belong  to  the  domain.  This  is  an  important  restriction,  since  there  may  be  more  than 
one  author  for  the  domain.  It  is  possible  that  a domain  may  be  initially  defined  by  an  author,  but  extra 
nodes  are  added  to  it  subsequently,  perhaps  after  students  have  begun  using  the  hypermedia.  The 
addition  of  the  semantic  network  is  an  aid  to  all  such  authors,  since  it  has  the  potential  to  help  them  to 
decide  if  a node  should  belong  to  the  domain.  An  author  who  is  unsure  as  to  the  legitimacy  of  a node’s 
relationship  with  the  domain,  may  succumb  to  the  “Linkitis”  problem,  i.e.  if  unsure  then  link.  The 
semantic  structure  aids  such  an  author  by  providing  them  with  a structure  into  which  the  node  must  fit. 
It  is  possible  that  the  responsibility  for  deciding  whether  a node  should  be  added  to  the  domain  lies 
with  the  original  semantic  network  author,  since  it  could  be  stated  that  if  a node  cannot  be  typed  to  an 
existing  semantic  network  node,  then  it  should  not  belong  to  the  domain.  This  could  be  achieved  by 
allowing  only  certain  classes  of  author  to  modify  or  otherwise  add  to  the  semantic  network.  Thus,  the 
integrity  of  the  domain  is  protected  to  a far  higher  degree  than  would  be  the  case  with  a hypermedia 
system  without  a semantic  network. 

Once  a node  has  been  correctly  typed,  it  is  connected  to  the  semantic  network.  This  immediately 
connects  the  new  node  to  all  the  other  nodes  in  the  hypermedia  system,  via  paths  of  varying  lengths, 
thus  the  author  need  not  specify  any  other  direct  links.  It  is  likely  however,  that  an  author  wishes  to  add 
a series  of  related  nodes.  For  example,  the  author  may  wish  to  add  the  node  "Pluto"  to  the  domain.  This 
could  involve  the  addition  of  several  new  nodes,  namely  the  node  “Pluto”  representing  the  planet  itself 
and  “Charon”,  representing  Pluto’s  only  satellite.  The  author  therefore  types  “Pluto”  is-a  “Planet”  and 
“Charon”  is-a  “Satellite”.  Once  “Charon”  has  been  correctly  typed,  the  author  is  required  to  connect  it 
to  a host  “Planet”,  since  this  is  an  explicit  requirement  of  the  “Satellite”  type  (a  reasonable  requirement 
since  satellites  do  not  exist  in  isolation).  The  original  author  of  the  semantic  network  encodes  this 
requirement  within  the  semantic  node  itself 

The  semantic  network  is  therefore  a structure  that  connects  the  entire  domain  together.  It  is  vital  that 
the  author  connects  the  class  nodes  together  in  an  accurate  fashion.  As  discussed  above,  it  may  not  be 
clear  cut  decision  whether  to  connect  a node  to  a particular  class  node  or  not.  The  author  may  have  to 
make  a decision  about  connecting  two  class  nodes  together  when  the  relationship  between  them  is  not  a 
definite  one.  The  author  should  detail  this  relationship  within  the  class  node  so  that  any  future  author  is 
clear  about  the  defined  relationship.  It  may  not  always  be  the  case  that  such  a one  to  one  relationship 
can  be  defined,  i.e.  an  instance  node  may  not  fit  exactly  into  the  semantic  structure  of  the  domain,  but 
may  still  be  valuable  to  the  student  studying  the  domain.  In  this  situation  the  author  should  provide  a 
suitable  “non  specific”  semantic  node.  For  example,  the  domain  author  may  wish  to  add  information 
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about  probe  missions  to  various  planets.  The  author  may  decide  that  it  is  not  appropriate  to  add  the 
semantic  node  “Probe  Missions”  as  a-kind-of  “Planet”.  However,  it  may  be  appropriate  to  have 
information  about  the  Voyager  missions  to  Jupiter,  with  the  "Jupiter”  instance  node.  The  author  could 
therefore  define  the  semantic  node  “Related  Material”  as  part-of  “Planet”.  It  is  important  that  this 
facility  is  used  sparingly,  otherwise  the  benefits  of  structuring  the  domain  could  be  lost. 

Defining  Tutorial  Nodes 

Tutorial  nodes  are  employed  by  Hypemet  as  a means  to  aid  the  student’s  retention  of  the  domain 
contents.  Jonassen  and  Wang  (1993)  suggest  that  information  is  retained  by  providing  a context  into 
which  the  information  fits,  thus  aiding  the  student  in  determining  where  the  information  fits  within 
their  own  knowledge  representation  structures  (schema).  In  Hypemet  this  is  aided  by  providing  the 
students  with  tasks  to  complete,  via  the  tutorial  nodes.  Tutorial  nodes  may  take  the  form  of  a simple 
question  and  answer  session,  or  they  may  involve  a more  complex  task  that  requires  that  the  student  to 
visit  several  nodes.  Such  tasks  may  require  the  student  to  traverse  the  hypermedia  looking  for  various 
pieces  of  information.  Requiring  the  student  to  find  and  examine  the  relationship  between  concepts 
may  also  aid  their  retention  of  structural  knowledge  of  the  domain  (Jonassen  and  Wang  1993).  It  is 
therefore  paramount  that  tutorial  nodes  are  designed  and  placed  within  the  hypermedia  in  order  to 
render  this  transfer  of  knowledge  as  efficient  as  possible. 

The  tutorial  nodes  are  designed  to  remove  the  danger  of  passivity  from  the  system,  i.e.  the  student  is 
not  always  left  to  wander  through  a hypermedia  system,  without  a task  or  guidance,  especially  if  they 
are  a novice  of  the  domain.  It  could  be  argued  that  a human  teacher  could  present  the  student  with  a 
learning  task  before  they  are  exposed  to  the  hypermedia  system.  However,  this  would  require  the 
human  teacher  to  supply  the  student  with  a complete  task  that  would  detail  requirements  for  the 
student,  as  they  become  more  proficient  with  the  domain.  Alternatively,  it  would  require  that  the 
human  teacher  is  constantly  present,  in  order  to  gauge  when  the  student  requires  a new  more  advanced 
task.  The  latter  situation  is  not  always  possible,  since  the  human  teacher  is  an  expensive  resource  and 
the  need  for  ITSs  has  been  driven  by  this  fact  (Woods  and  Warren  1 995).  The  former  situation  is  also 
not  desirable,  since  providing  the  novice  student  with  expert  level  tasks  (for  their  future  use),  may  be 
providing  the  student  with  a level  of  complexity  that  is  not  beneficial  to  them  (Ridgeway  1989, 
Kinshuk  and  Patel  1997). 

Tutorial  Nodes  are  embedded  within  the  hypermedia,  although  they  do  not  strictly  form  part  of  the 
semantic  network,  in  that  they  do  not  represent  the  structure  of  the  domain.  A tutorial  node  may  be 
triggered  by  a simple  rule,  defined  by  the  domain  author.  For  example:  '"Activate  Tutorial  Node  1 
when  the  student  enters  the  system''.  The  student  is  therefore  given  a task  to  complete  as  soon  as  they 
enter  the  system.  It  is  advised  that  such  a task  should  concentrate  on  providing  the  student  with 
information  about  how  to  use  the  hypermedia  tutoring  system  itself  and  not  necessarily  be  related  to  the 
domain,  since  initially,  a user  who  is  new  to  a system  expends  most  of  their  “cognitive  power”  learning 
how  to  use  a new  system  . 

Linear  and  Non-Linear  Tutorial  Nodes 

Tutorial  nodes  may  exist  in  “linear  hyperspace”  and  “non-linear  hyperspace”.  They  may  be  designed  to 
be  encountered  one  after  the  other,  guiding  the  student  through  the  hypermedia  (linear  hyperspace). 
Alternatively,  they  may  be  activated  by  a student  following  their  own  interests  into  a particular  area  of 
the  hypermedia  (non-linear  hyperspace).  Linear  hyperspace  is  similar  to  Trigg’s  (1988)  idea  of  paths 
authored  into  a hypermedia  system.  Trigg’s  system,  "TexNet",  allows  the  author  to  specify  default 
paths  for  the  student  to  follow,  in  order  to  reduce  the  chances  of  them  becoming  lost.  Such  a method 
has  been  criticised  by  Jonassen  and  Wang  (1993),  for  providing  the  student  with  a path  that  they  may 
“blindly”  follow,  i.e.  they  are  encouraged  not  to  search  the  hyperspace.  The  same  criticism  cannot 
however  be  levelled  at  the  linear  hyperspace  tutorial  nodes  (LTN)  employed  by  Hypemet,  since  they 
are  not  providing  a node  to  node  path.  Instead  the  LTNs  are  providing  the  student  with  useful 
landmarks  with  which  to  orientate  themselves,  in  that  the  student  can  navigate  from  one  LTN  to  the 
next.  They  may  therefore  travel  through  the  hypermedia  in  freedom,  between  such  tutorial  nodes. 
These  nodes  are  suitable  for  novice  students,  since  they  are  guided  (as  opposed  to  forcefully  directed) 
by  the  tutorial  nodes. 

The  non-linear  hyperspace  tutorial  nodes  are  intended  for  more  advanced  students,  since  they  do  not 
provide  the  guidance  that  the  LTNs  provide.  This  is  because  the  NLTNs  do  not  appear  in  a predefined 
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sequence.  Instead,  it  is  a matter  of  when  the  student  uncovers  them.  They  therefore  do  not  offer  the 
same  landmark  qualities  as  their  linear  counterparts.  Such  nodes  are  therefore  more  suited  to  students 
of  higher  levels,  since  the  student  may  randomly  encounter  them  and  are  not  lead  from  one  to  the  next. 


The  LTNs  may  be  useful  for  lower  level  students  and  the  NLTNs  for  higher  level  students.  It  may  be 
desirable  in  some  instances  to  use  the  two  types  of  tutorial  nodes  interchangeably;  for  example,  using 
LTNs  for  an  expert  level  student  who  has  entered  an  unvi sited  section  of  the  domain.  It  may  also  be 
desirable  for  the  author  to  target  certain  tutorial  nodes  at  certain  levels  of  students,  or  to  have  slightly 
different  tutorials,  for  each  student  level,  present  within  the  same  tutorial  node.  Figure  9, 3,1, A (shown 
previously)  shows  two  NLTNs,  labelled  ’’Planet  Tutorial"  and  "Volcano  Tutorial",  These  nodes  are 
invoked  when  the  node  to  which  they  are  connected  to  via  an  "Invoked-by”  link  is  selected.  For 
example,  the  "Volcano  Tutorial"  is  invoked  when  the  "Volcano"  class  node  is  visited. 

Conclusion 

Experiences  with  Hypemet  suggest  that  the  use  of  a semantic  network  does  indeed  help  the  author  to 
represent  their  knowledge  in  a formalised  way,  suitable  for  expression  as  a hypermedia  domain. 
Whether  this  domain  representation  is  more  of  less  useful  to  the  student  is  open  to  educational  debate, 
however,  the  literature  concerning  the  use  of  semantic  networks  would  strongly  indicate  that  it  is 
(Tudhope  and  Taylor  1997,  Pereira  et  al  1991,  Jonassen  and  Wang  (1993), 
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Abstract: 

Many  colleges,  universities  and  businesses  recognize  the  rewards  of  teaching  on  the  Internet.  As  each 
organization  rushes  toward  cyberspace,  they  bump  into  a variety  of  potholes  that  cause  resistance  and 
discouragement  among  the  faculty.  Sometimes  poor  administrative  strategy  leads  faculty  members  to 
abandon  a technology  that  may  become  the  most  effective  tool  in  the  history  of  education.  This  session 
emphasizes  methods  of  successfully  introducing  faculty  to  teaching  in  virtual  classrooms,  and  points 
out  strategies  to  be  avoided. 


Interactive  media  and  web-based  instruction  have  repeatedly  proven  themselves  to  be  effective  and  efficient  tools 
for  classroom  support  and  distance  learning.  Compared  to  traditional  modes  of  instruction,  students  leam  the  same 
amount  of  material  in  half  the  time  or  less  with  equal  or  better  understanding  and  equal  or  better  retention  when 
using  interactive  learning  materials.  To  use  these  new  instructional  tools,  colleges  and  universities  need  the 
technological  infrastructure  and  a profound  cultural  change  within  the  institution.  If  funding  is  available,  the 
technological  infrastructure  is  relatively  easily  achieved. 

The  greater  challenge  is  achieving  the  necessary  cultural  change.  Simply  putting  traditional  materials  on  the  Internet 
doesn’t  work.  Using  this  tool  requires  a new  teaching/leaming  paradigm.  Faculty  must  alter  their  teaching  methods 
and  styles  to  teach  successfully  in  cyberspace.  Student  preconceptions  of  education  will  need  to  change  to 
accommodate  this  non-traditional  moderated  mode  of  learning. 

Effective  change  is  not  mandated,  but  adopted  because  of  obvious  benefits  to  the  users.  A top  down  approach  is  likely 
to  meet  resistance,  but  a bottom  up  approach  can  generate  enthusiasm.  In  the  early  stages  of  the  move  to  virtual 
classroom  use,  top  down  standardization  and  policy-making  inhibit  progress.  If  administrators  support  grassroots 
faculty  and  student  use  of  the  Internet  the  school  will  avoid  stifling  the  initiative  of  individual  faculty  members. 

The  initiative  is  likely  to  go  nowhere  if  the  faculty  senate,  administration,  union,  and  dozens  of  committees  pool 
their  ignorance  and  discuss  ad  nausium  issues  they  do  not  fully  comprehend.  During  the  early  years  administration 
should  simply  support  voluntary  initiatives  of  early  adapters.  Otherwise  peer  pressure  among  faculty  members  to 
maintain  the  status  quo  is  likely  to  subvert  use  of  technology  in  the  classroom.  If  the  administration  does  encourage 
specific  software,  it  should  be  the  simplest  to  leam  and  use  available.  Once  the  majority  of  the  faculty  has  adopted 
Internet  tools  and  understands  how  the  virtual  classroom  works,  the  peer  pressure  supports  adoption  of  this  new 
technology  rather  than  the  status  quo. 

Once  the  majority  of  faculty  is  knowledgeable  about  use  of  the  Internet  to  teach  and  leam,  discussions  are  informed, 
and  better  decisions  will  be  made.  Standardization  and  policy-making  will  be  easier  and  more  appropriate,  because 
most  faculty  members  will  be  familiar  with  the  technology  and  issues  and  will  have  learned  what  works  and  what 
doesn't.  Since  the  WWW  Consortium  already  promotes  HTML  communication  standards,  it  is  relatively  easy  to 
switch  from  one  software  package  to  another.  Switching  from  one  virtual  classroom  program  to  another  is  analogous 
to  switching  from  driving  a car  with  automatic  transmission  to  a semi-truck. 
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Faculty  training  is  the  key  to  success.  The  technology  should  be  as  simple  and  transparent  as  possible.  The  important 
issues  here  are  about  learning,  not  about  technology.  Many  faculty  members  are  attracted  to  experiences  that 
promote  teaching  and  learning  but  tend  to  avoid  experiences  emphasizing  technology. 

Time  commitment,  compensation  and  recognition  are  key  elements  in  the  adoption. 

Most  Colleges  and  Universities  recognize  the  rewards  of  teaching  on  the  Internet  if  it  is  implemented  properly.  As 
each  organization  rushes  toward  cyberspace,  they  bump  into  a variety  of  potholes  that  cause  resistance, 
discouragement,  and  sometimes  abandonment  of  what  may  become  the  most  effective  technology  in  the  history  of 
education. 

This  workshop  will  examine  the  potholes  (false  assumptions)  every  institution  should  avoid,  and  examine  strategies 
(fastlanes)  that  lead  toward  successful  implementation  of  the  virtual  classroom  and  Internet  supported  and  Internet 
based  programs.  Some  of  these  will  include: 


Pothole: 

Fastlane: 

Implementation  should  be  organized  and  standardized  from  the  top  down. 

In  higher  education,  the  top  down  approach  frequently  generates  resistance  - the  bottom  up 
approach  generates  enthusiasm. 

Pothole: 

Fastlane: 

The  greatest  obstacle  is  the  technology. 

The  greatest  obstacle  is  social  inertia  of  the  institution. 

Pothole: 

Fastlane: 

Learning  in  cyberspace  is  just  teaching  repackaged. 

Learning  in  cyberspace  requires  a fundamental  paradigm  shift  on  the  part  of  students  and  teachers. 

Pothole: 

Fastlane: 

e-Mail  is  a great  way  to  manage  distance  learning. 

e-Mail  has  its  place,  but  using  e-mail  as  the  primary  technology  to  manage  distance  learning  is 
nearly  equivalent  to  tutoring  each  student  individually. 

Pothole: 

Fastlane: 

Unions,  Administration,  and  Faculty  Senate  should  get  involved  early,  because  otherwise  their 
interests  may  be  hurt. 

Arguing  about  theories  that  no  one  has  put  into  practice  or  really  understands  prevents  progress. 

Pothole: 

Fastlane: 

This  endeavor  is  about  technology. 
This  endeavor  is  about  learning. 

Pothole: 

Fastlane: 

Faculty  training  is  best  done  by  techies,  and  implementation  of  the  virtual  campus  can  be  done  by  staff. 
Faculty  training  should  be  done  by  faculty.  Faculty  must  play  the  leadership  role  in  decision 
making  about  the  technology  after  experimenting  with  various  systems.  The  role  of  the  techie  is  to 
serve  faculty  needs  and  help  faculty  promote  student  learning. 

Pothole: 

Fastlane: 

Relationships  with  students  will  degrade  when  teaching  in  cyberspace. 
Relationships  improve,  as  the  lecturer  becomes  a coach,  mentor  and  guide. 

Pothole: 

Fastlane: 

Relationships  between  students  and  with  teachers  are  not  as  important  in  cyberteaching. 
Relationships  in  the  virtual  classroom  are  more  important  to  learning  than  in  the  traditional 
classroom.  Steps  must  be  taken  to  build  cooperative  relationships. 

Pothole: 

Fastlane: 

Create  a rich  didactic  site  with  lots  of  information  no  matter  how  much  time  it  takes. 
Use  a minimalist  approach  and  the  Socratic  method. 

Pothole: 

Fastlane: 

Use  all  the  information  available  on  the  Internet. 

Pace  and  organize  material  so  students  don't  become  overwhelmed  with  information. 

The  potholes  (false  assumptions)  and  fastlanes  (successful  strategies)  will  be  discussed  in  this  presentation. 
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Appendix:  Quotes  and  Notes 


Analysis  of  Effectiveness  and  Costs  of  IVD  Instruction  in  Defense  Training  and  Education,  summarized  of  over  600 
research  studies  of  interactive  training  programs.  This  metastudy,  done  by  the  Department  of  Defense  Analysis 
(1801  Beauregard  Street,  Alexandria,  VA  22311-1772),  concluded;  Compared  to  traditional  modes  of  instruction, 
students  learn  the  same  amount  of  material  in  half  the  time  or  less  with  equal  or  better  understanding  and  equal  or 
better  retention  when  using  interactive  learning  materials.  To  see  or  contribute  to  a summary  of  research  studies 
dealing  with  the  effectiveness  of  interactive  multimedia  and  the  Internet  follow  the  hyperlinks  to  the  Interactive 
Virtual  Classroom  Discussion  from  http://www.alsi.net. 

Some  aspects  of  the  new  paradigm  for  online  learning  are  discussed  in  Using  the  Internet  and  CD-ROM  for 
Classroom  Support  at  http://www.alsi.netAVEBLEARNING.htm 

Distance  learners  found  faculty  to  be  more  accessible  than  in  the  typical  classes.  All  types  of  learning  styles  can  be 
successfully  taught  using  distance  education  technologies.  Threlkeld,  R.,  & Brzoska,  K.  (1994).  Research  in  distance 
education.  In  B.  Willis  (Ed.),  Distance  education:  Strategies  and  tools  (pp.  41-66).  Englewood  Cliff,  NJ:  Educational 
Technology  Publication. 

To  be  effective  distance  educators,  faculty  should  participate  in  a training  program  designed  specifically  to  meet 
their  needs  (Threlkeld  & Brzoska,  1994,  p.  58). 

Clark,  R.E.  (1991)  maintains  that  instructional  strategies,  and  not  the  medium,  are  the  key  to  effective  learning. 
Unfortunately,  technology  and  production  considerations  rather  than  teaching-learning  theory  or  the  instructional 
development  process  are  often  the  driving  force  behind  distance  education  programs. 

Clark,  R.E.  (1991).  When  researchers  swim  upstream:  Reflections  on  an  unpopular  argument  about  learning  from 
media.  Educational  Technology  Research  and  Development,  Feb.  1991,  34-40. 

Jerald  G.  Schutte  of  California  State  University,  Northridge  did  a comparative  study  of  two  statistic  classes,  and 
reported:  "Quantitative  results  demonstrated  the  virtual  class  scored  an  average  of  20%  higher  than  the  traditional 
class  on  both  examinations.  Further,  post-test  results  indicate  the  virtual  class  had  significantly  higher  perceived  peer 
contact,  and  time  spent  on  class  work,  but  a perception  of  more  flexibility,  understanding  of  the  material  and  greater 
affect  toward  math,  at  semester  end,  than  did  the  traditional  class."  Virtual  Teaching  in  Higher  Education:  The  New 
Intellectual  Superhighway  or  Just  Another  Traffic  Jam?  (http://www.csun.edu/sociology/virexp.htm) 

In  Dancing  with  The  Devil,  Duderstadt  maintains  that  education  used  to  be  "just  in  case".  It  is  now  "just  in  time"  in 
corporate  America,  and  in  the  future  it  is  likely  to  be  "just  for  you". 

Relationships  between  students  and  between  students  and  faculty  are  important  to  the  learning  process.  Does  use  of 
technology  degrade  the  relationship?  Does  talking  on  the  phone  degrade  the  relationship?  Prejudices  fall  away 
online,  and  relationships  focus  on  ideas.  Students  become  more  intimate  rather  than  more  distant  when  they  learn 
online.  Professor  Joe  Schramer,  Teacher  of  the  Year  at  the  University  of  St.  Thomas  said,  "I  had  over  70  comments 
posted  by  students  in  my  virtual  class  last  night.  In  my  traditional  classroom.  I'd  be  lucky  to  have  70  student 
comments  in  an  entire  semester." 

Problem:  Students  can  become  overwhelmed  with  information.  How  do  you  pace  an  online  class?  How  do  you  help 
the  students  choose  among  the  activities  and  information? 

Thomas  J.  Donahue  teaches  communication  technology  at  the  University  of  St.  Thomas  in  Minnesota.  He  is  also 
President  of  Active  Learning  Systems,  Inc.  His  full  resume  is  online  at  www.alsi.net/content/index.htm.  ALSI  helps 
colleges  and  universities  create  a strategic  plan  for  using  technology  and  trains  faculty  to  teach  using  the  Internet. 
Copynght©Active  Learning  Systems,  Inc.  2000 
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Abstract:  The  ExploraGraph  interface  was  designed  to  facilitate  interaction  in  the  context  of 
distant  learning.  It  was  developed  as  an  alternative  to  simple  web  interaction,  in  order  to 
increase  flexibility,  visibility,  and  structure  in  the  learning  environment.  It  may  be  used  as  a 
front  end  to  existing  courses  on  the  web.  The  ExploraGraph  Navigator  makes  it  possible  to 
navigate  through  conceptual  graphs  with  automatic  arrangement  of  elements,  zoom  and  "fish 
eye"  effects.  Each  node  of  the  graph  may  have  a description  attached  to  it  and  may  give  direct 
access  to  an  application,  a document  or  an  Internet  site.  Graphic  structures  may  thus  be  used 
to  represent  the  organization  of  tools,  activities,  concepts  or  documents.  The  Navigator  offers 
each  user  a tool  to  specify  his  goals  and  the  system  can  support  him,  using  multiple 
modalities:  Hypertext,  graphical  cues  (Lee  & Lehman,  1993),  Ms  Agents  avatars,  voice, 
visual  demonstrations  and  force  feedback  guiding. 


Introduction 


LICEF  is  developing,  in  the  context  of  the  Canadian  TeleLeaming  Network  of  Centres  of  Excellence,  a 
distance  education  environment  on  the  web.  The  Center  for  Virtual  Learning  developed  at  LICEF  offers  not 
only  hypermedia  content  on  the  web,  but  different  tools  for  conferencing,  planning  meetings, 
videoconferencing,  presenting  scenarios  and  knowledge,  etc.  The  modeling  of  users  in  activity  has  illustrated 
the  difficulties  they  experience  in  learning  the  efficient  use  of  such  a complex  environment,  first  as  they  learn 
it  at  the  training  session,  and  then  on  their  own  when  they  are  at  distance.  To  offer  contextual  help  to  learners, 
we  have  designed  a supportive  interface  that  will  adapt  to  each  user,  unfolding  gradually  as  s/he  acquires 
expertise  and  offering  adaptive  support  using  different  modalities. 

The  ExploraGraph  interface  was  designed  to  facilitate  interaction  in  the  distance  learning  context.  After 
studying  learners  in  various  distance  learning  situations,  it  became  evident  that  existing  interfaces  could  be 
improved  to  better  support  the  interaction  and  the  learning  process.  Learners  have  to  navigate  between  pages 
of  web  documents,  search  engines  and  text  instruction.  They  must  use  various  communication  tools  and 
organize  their  activities  with  little  support.  As  DiSessa  (1986)  explains,  they  encounter  the  “utility  barrier”, 
not  knowing  how  the  system  should  be  used  and  how  to  modify  their  activities  to  better  use  it.  They  find  it 
difficult  to  transpose  their  activities  in  multiple  environments.  The  ExploraGraph  interface  was  developed  as 
an  alternative  to  simple  web  interaction,  in  order  to  increase  flexibility,  visibility,  and  structure  in  the  learning 
environment.  Inspired  by  the  Explora  interface  developed  at  LICEF  (Paquette  et  al.,  1996),  we  tried  to 
develop  a more  dynamic  and  flexible  interface  for  the  learner  to  access  the  Center  for  Virtual  Learning.  The 
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learner  navigates  in  graphic  conceptual  structures  of  tasks,  knowledge  elements  and  documents,  that  give  him 
a schematic  idea  of  what  is  to  be  learned  and  what  s/he  has  to  do.  Automatic  arrangement  of  elements,  zoom 
and  "fish  eye"  effects  are  used  to  increase  visibility  as  each  users  manipulates  the  graphs.  Each  node  of  the 
graph  may  have  a description  attached  to  it  and  may  give  direct  access  to  an  application,  a document  or  an 
Internet  site. 


The  Importance  of  Adaptive  Interfaces 

The  need  to  integrate  adaptive  interfaces  in  the  domain  of  learning  environments  has  become  an  important 
field  of  research  (Dufresne,  1997;  de  La  Passardiere  et  Dufresne,  1992;  Brusilovsky,  1997),  especially  as  it 
appears  to  play  an  important  role  in  reducing  navigation  disorientation  and  encouraging  exhaustive  learning 
in  hypertext  systems,  particularly  in  web  learning  applications.  Adaptation  may  take  many  forms,  modifying 
content  and/link  access  and  annotations. 

One  of  the  main  limitations  to  adaptive  interfaces  is  the  amount  of  information  obtained  by  the  system  on  user 
activity  and  knowledge.  In  order  for  adaptive  interfaces  to  increase  the  salience  of  some  elements,  the  next  best 
to  be  learned,  it  is  necessary  to  define  what  is  the  proposed  structure  for  learning,  what  has  been  seen,  but  also, 
what  does  the  user  want  to  learn  next.  If  the  structure  of  the  content  is  encyclopedic  and  easy  to  chunk,  then 
direct  manipulation  may  be  used,  giving  access  to  graph  representing  the  structure  of  concepts  to  be  learned. 
An  overlay  model  of  knowledge  may  then  be  extracted  from  the  trace  of  the  user’s  navigation:  What 
information  has  he  read?  For  how  long?  How  long  ago?  In  such  a context,  adaptive  links  annotation  can 
suffice  to  support  the  user’s  navigation  and  encourage  him  to  complete  his  navigation  and  understanding. 

However,  some  pedagogy  and  learning  theories  insist  that  it  is  better  that  learning  take  place  in  a more  active, 
situated  and  collaborative  setting.  In  this  context,  content  has  to  be  organized  in  a more  complex  way  and  this 
complicates  the  possible  support  to  navigation.  According  to  the  MISA  model  (Paquette,  Aubin,  & Crevier 
(1997),  learning  activities  are  organized  around  many  structures,  including  ; knowledge  structure,  scenarios  of 
activities  and  documents  structure.  In  this  context,  there  may  be  many  paths  to  access  knowledge  and  the  user 
needs  some  coaching  and  adaptation  at  a more  general  level  to  better  assimilate  content.  For  this  meta-level  of 
support  to  be  implemented,  the  system  must  use  strategic  information  on  both  content  structure  and  the  user’s 
higher  level  of  activity  . The  present  project  proposes  a model  and  generic  prototype  to  facilitate  navigation 
and  adaptive  support,  based  on  the  intentions  of  the  user  and  on  the  structure  of  his  activities. 

We  were  among  the  first  to  experiment  with  links  annotation  (Dufresne,  John,  & Senteni,  1990),  where  visited 
links  are  hierarchically  modified  as  the  user  finishes  visiting  the  information  within  them.  In  the  context  of 
distance  education,  our  interest  was  more  proactive,  in  order  to  attract  the  user’s  attention  to  tasks  or  contents 
that  were  more  pertinent  at  one  point  in  time.  We  were  interested  in  the  pacing  of  the  meta-level  of 
information  attached  to  a content  to  be  learned,  in  order  to  follow  the  evolution  of  user  expertise  and  to  support 
learner . 


System  Structure 

The  architecture  of  the  ExploraGraph  (Dufresne,  Cosmova.,  LeTran,  & Ramstein,  1999)  system  is  partly  local  and 
partly  shared:  it  is  an  integration  of  commercial  Office  applications  and  applications  we  developed  in  Visual 
Basic  using  an  Access  database.  Figure  1 show  the  architecture  of  the  system  and  of  the  interaction,  with  the 
different  input  and  output  paths  used  in  system  adaptation. 

The  system  main  features  are  the  dynamic  arrangement  of  graphical  structures,  the  Control  Panel  that  allows 
each  user  to  specify  his/her  intentions  and  to  delegate  actions,  and  the  system’s  various  modalities  of  support. 
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Figure  1 . Multimodal  adaptive  support  in  the  ExploraGraph  interface. 


Navigation:  A Metaphor  that  Helps  You  “Stretch  Your  Mind” 

The  ExploraGraph  Navigator  was  designed  to  facilitate  the  visualization  of  graphical  structures.  A physical 
model  of  nodes  (mass  and  repulsion),  links  (elasticity)  and  of  the  general  organization  (gravity  and  friction)  is 
used  to  simulate  the  relations  between  sets  of  elements.  As  concepts  are  added  (editor  mode)  or  as  they  are 
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traversed  (navigation  mode),  nodes  rearrange  themselves  in  order  to  maximize  visibility  and  contextual 
relation  to  what  is  at  the  focus  of  interest.  Zoom  and  ’Tisheye”  effects  make  it  possible  for  the  user  to  pop  out 
and  concentrate  on  an  area  of  the  graph  knowledge  or  task  representation.  Such  relative  accentuation  may  also 
be  driven  by  the  system  adaptive  functions,  which  may  use  it  to  highlight  areas  of  the  information  to  be 
explored;  this  corresponds  to  concept  of  map  adaptation  (Brusilovsky,  1997). 

The  ExploraGraph  Navigator  makes  use  of  a generic  representation  of  types  of  nodes  and  types  of  links  that 
may  be  changed  to  represent  structures  of  tasks,  concepts,  tools  or  documents. 
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Figure  2 ExploraGraph  Navigator  showing  the  graphical  structure,  the  Control  Panel  and  the  avatars’ 
contextual  help. 

In  figure  2 the  User  have  chosen  the  goal  “Communicate  a technical  question”. 

Depending  on  the  task  model,  the  Avatar  may  suggest  a specific  technical  problem  and  propose  help  related  to 
it.  He  may  zoom  on  the  right  discussion  Forum  and  eventually  launch  it  with  appropriate  help  depending  on 
previous  interaction  with  the  user.  Eventually,  looking  at  others  user’s  model  he  will  suggest  someone  to  ask 
for  help. 
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Activities:  Let  the  User  Choose  What  S/He  Wants  to  Do. 


In  fact,  the  context  of  distance  learning  is  one  where  the  system  is  not  at  the  center  of  the  interaction;  the 
teacher  and  the  other  learners  are  also  there,  so  the  system  is  seen  more  as  a tool  then  as  the  only  field  of 
interaction.  As  such,  the  system  cannot  assume  the  full  direction  of  the  activity:  it  must  be  transparent  to  the 
user  goals  and  activity  and  cannot  be  too  unpredictable.  In  this  context,  what  the  user  needs  is  not  as  much  a 
feedback  on  his/her  progression  in  the  content  but  a guide  as  to  how  s/he  may  use  them  to  achieve  the 
proposed  learning  objectives.  The  adaptive  interface  should  coach  the  learner,  according  to  the  schedule,  and 
to  learner  progression  and  context.  It  must  then  cue  the  learner  to  links  between  tasks  and  documents, 
between  tasks  and  the  structure  of  concepts,  between  tasks  and  his/her  utilization  of  tools,  methods  and 
external  resources  like  discussion  forums. 

For  the  interface  to  adapt  its  support  to  the  user’s  higher  level  of  activity,  a Control  Panel  is  incorporated  in 
the  Navigator,  where  he  may  choose  a general  intention:  Explore,  Plan,  Revise  his  own  progression,  Adjust  the 
system  (each  with  various  options).  Choosing  a goal  may  trigger  a set  of  actions  in  the  system,  opening 
multiple  applications,  choosing  a specific  navigation  environment.  The  Control  Panel  also  enriches  the  user’s 
model,  making  it  possible  to  adapt  system  support,  which  may  propose  different  contextual  explanations  or 
guidance  for  each  goal,  depending  on  task  schedule  and  on  user  profile. 


Support  with  Multiple  Modalities 

The  system  may  propose  different  types  of  adaptive  support,  taking  into  account  the  potential  of  each  modality 
to  guide  the  user.  Support  could  take  the  form  of: 

• Graphical  highlight  and  links  annotation,  which  give  a general  idea  of  the  structure  and  which  are  more 
passive  and  less  intrusive  than  more  formal  guiding  and  demonstration. 

• Avatars  animations  which  are  used  for  graphical  demonstrations  (attention  seeking,  pointing,  etc.)  , but 
also  for  non-verbal  communication  and  motivational  purpose. 

• Avatars  audio  voices,  which  serve  to  attract  attention  and  are  more  easily  combined  with  visual 
demonstrations  than  text. 

• Visual  guiding  to  present  to  the  user  procedures  and  actions  which  s/he  may  use  to  realize  his/her  chosen 
intentions 

• Force  feedback  guiding,  which,  combined  with  visual  demonstration,  obtains  more  attention,  takes  less 
cognitive  concentration,  facilitates  memorization  (Arcand,  1995)  and  ensures  that  the  user  follows  the 
demonstration  to  the  end. 

Avatars  were  introduced  in  the  environment  partly  “to  fulfill  the  need  for  social  context  ” when  no  other 
learners  are  on-line  (Kearley,  1993  in  Pentti,  1998;  Rickel  & Johnson,  1999;  Gregoire,  Zettlemoyer  & Lester, 
1 999)  and  to  serve  as  the  interface  to  the  advisory  system. 

Some  adaptation  of  the  support  will  eventually  be  under  user  control.  But  for  the  user  in  the  fire  of  action, 
adapting  the  interface  might  be  a tedious  and  bothersome  task,  so  we  intend  to  integrate  adaptive  functions 
that  will  integrate  both  global  and  individual  users'  feedback  on  the  help  system,  in  order  to  adjust  future  help 
both  in  general  and  individually.  Such  a mixed  adaptive  learning  environment  appears  to  be  important  in 
guaranteeing  adaptation  of  systems  to  longer  learning  situations.  A mixed  adaptive  environment  appears  to  be 
the  best  way  for  the  environments  to  be  really  adapted  to  the  user’s  longer  learning  situations  and  to  his 
evolving  wishes  (Brusilovsky  1 996). 
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Abstract:  The  technology-based  Curriculum  Integration  project  in  the  Plano  Independent  School 
District  is  presented  as  a model  for  successful  implementation  of  large-scale  technology 
integration  projects.  The  project  addressed  relevant  components:  teachers,  students,  curriculum, 
learning  theories  and  materials,  technological  infrastructure,  and  school  architecture.  It  was 
found  that  systemic  change  in  the  learning  culture  among  district  teachers  and  students 
resulted  from  a convergence  of  forces.  These  forces  included  changes  in  teaching 
methods,  the  curriculum,  and  the  physical  learning  environment,  as  well  as  the 
planning,  development,  and  implementation  of  the  project  as  a collaborative  effort 
between  Plano  educators  and  various  professionals  within  an  educational  software 
company. 


Introduction 

In  the  last  decade,  there  has  been  an  increasing  number  of  publications  reporting  the  minor  impact  that, 
despite  substantial  investments,  the  integration  of  technology  in  schools  has  on  the  teaching  culture,  the  school 
environment,  and  the  students'  attitude  to  learning  (e.g.,  Bryson  & de  Castell  1998).  In  addition,  teachers  report 
dissatisfaction  with  the  applicability  of  off-the-shelf  multimedia  products  to  their  curriculum.  For  example,  in  a 
survey  conducted  in  United  States  in  1999,  59%  of  teachers  who  use  software  for  instruction  report  that  is  it 
“somewhat”  or  “very”  difficult  to  find  software  to  meet  their  classroom  needs  (Resnick  1 999).  However,  it  is 
not  only  appropriate  software  that  is  the  issue.  Successful  technology  implementation  into  schools  requires  a 
holistic  approach  that  addresses  all  relevant  issues,  including  design  of  custom  curricular  materials,  learning 
theory,  pedagogy,  staff  development,  technology  infrastructure,  management,  finance,  and  school  architecture. 

The  technology-based  Plano  Independent  School  District  (PISD)  Curriculum  Integration  Project  is  an 
excellent  example  of  such  holistic  implementation.  The  large-scale  systemic  project  was  implemented  in  the 
PISD  between  1993-1998,  involving  approximately  40  elementary  schools  and  60,000  K-5  students.  Two  of  its 
unique  features  were  that  teachers  played  a leading  role  in  collaboration  with  an  education  computer  company 
to  design  an  interdisciplinary  curriculum  with  custom  learning  materials  based  on  a constructivist  approach  to 
teaching  and  learning  as  well  as  recommended  the  classroom’s  architecture  and  technology  infrastructure. 

The  paper  reports  the  Plano  Curriculum  Integration  Project  to  highlight  necessary  ingredients  for  a 
successful  wide-scale  integration  of  technology  in  a school  system. 

Launching  the  Plano  Project 

The  decision  to  launch  a systemic  curricular  change  in  1993  was  based  on  the  recognition  that  the 
PISD  school  system  “suffered”  from  problems  such  as  those  listed  in  Table  1 as  well  as  an  understanding  that 
introducing  technology,  without  addressing  other  educational  and  architectural  issues,  would  not  be  enough. 
Therefore,  the  project  was  designed  to  address  the  factors  in  Table  1;  these  are  subsequently  discussed  in  the 


paper.  The  PISD  contracted  Edunetics,  a multimedia  production  company,  to  help  it  redesign  its  curriculum, 
inservice  the  teachers  in  the  area  of  computer  technology,  and  develop  educational  computer  software  that 
would  serve  as  a crucial  component  of  the  new  integrated  curriculum.  In  addition,  PISD  established  the 
Curriculum  Development  Center  in  which  a team  of  30  to  40  selected  K-5  teachers,  supervised  by  PISD 
curriculum  coordinators,  cooperatively  developed  the  new  thematic  curriculum  with  the  pedagogical,  content, 
and  instructional  technology  design  experts  in  Edunetics  (Otto  1 997). 


Pre  Technology-Based  System 

Technology-based  System  Changes 

Traditional  discipline-based  curriculum 
that  adopted  mostly  behaviorist  pedagogy. 

A conceptual  redesign  to  a technology-based,  thematically 
integrated  curriculum  that  adopted  cognitive  and 
constructivist  pedagogy. 

Infrequent  and  minimal  involvement  of 
teachers  in  curriculum  design. 

Teacher  ownership  in  redesigning  the  curriculum  and 
developing  the  learning  materials. 

Limited  use  of  technology  in  schools  due 
to  scarcity  of  computers  and  an  inadequate 
technological  infrastructure. 

An  appropriate  technological  infrastructure  plan  that 
included  8 computers  and  a printer  per  classroom,  ethemet 
connectivity,  and  Internet  access. 

Very  limited  use  of  software,  partly  due  to 
its  lack  of  suitability  to  the  curriculum. 

Custom  learning  material  development,  both  computer- 
based  and  other  types  of  materials. 

Classrooms  were  designed  for  traditional 
chalk-and-talk  style  of  teaching. 

Interior  classroom  and  school  redesign  to  promote  an  open 
teaching  style,  use  of  technology,  teamwork,  and  group 
discussion. 

Lack  of  finance. 

Debenture  bonds. 

Table  1:  Decision  Factors  Influencing  Curriculum,  Teacher  Involvement,  and  Architectural  Change 


Curriculum  redesign 

Plano  Independent  School  District’s  pre-1993  segregated  curriculum  (Table  2)  was  designed  to  follow 
the  curriculum  standards  in  the  various  disciplines  (cf  Bybee  & Mclnemey  1995;  Texley  & Wild  1996).  A 
major  task  of  the  project  was  to  convert  this  curriculum  into  a thematic  one  (Table  3),  without  giving  up  the 
disciplinary  depth  of  content.  It  was  argued  that  thematic  learning  enables  the  student  to  perceive  curricular 
topics  in  a broader  and  more  interconnected  way  (Jacobs  1989,  1997;  Kovalik  & Olson  1994;  Marzano  1992). 


Kindergarten 

Grade  1 

Grade  2 

LIFE 

SCIENCE 

• Looking  at  plants 

• Looking  at  animals 

• Homes  for  living 
things 

• My  body  grows 

• Animal  body  parts 

• Grouping  plants 

• Where  plants  live 

• Living/  non-living  things 

• My  body  grows 

• Staying  healthy 

• Grouping  animals 

• Ancient  living  things 

EARTH 

SCIENCE 

• Exploring  space 

• Exploring  earth 

• Watching  the  weather 

• Weather  changes 

• Looking  at  the  sky 

• Earth  seen  from  space 

• The  sun 

• Fossils  & Dinosaurs 

• Air  and  water 

• Looking  at  weather 

Table  2:  A section  of  the  pre-Plano  Curriculum  Project  K-2  discipline-segregated  curriculum 

In  a thematic  curriculum,  the  learning  objectives  are  organized  in  themes,  or  “big  ideas,"  and  not 
necessarily  in  segregated  disciplines,  as  in  most  traditional  curricula.  Thus  the  first  stage  in  designing  the  new 
curriculum  was  to  identify  six  major  themes  (Overarching  Concepts)  under  which  the  entire  new  curriculum 
was  organized:  Communication,  Continuity  and  Change,  Interactions,  Diversity,  Systems,  and  Balance  and 
Stability  (Table  3).  The  thematic  curriculum  was  designed  as  a matrix  that  matched  a six-week  thematic 
learning  unit,  called  Organizing  Idea  (01),  for  each  of  the  K-5  grade  levels.  Each  01  belonged  to  a specific 
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theme.  The  Ols’  design  presented  the  disciplinary  content  (Table  2)  in  a thematic  way,  supplemented  and 
reinforced  with  technology  (Table  3).  For  example:  Grade  Two’s  science  disciplinary  curriculum  (Table  2) 
included  topics  such  as  grouping  of  animals  and  ancient  living  things  in  Life  Science,  as  well  as  fossils  and 
dinosaurs  in  Earth  Science.  In  the  thematic  curriculum,  these  topics  were  clustered  into  the  01,  “What 
Evidence  Tells  Us  About  Our  World?”  under  the  Overarching  Concept  (theme)  of  "Communication”  (Table  3). 
In  this  01,  the  students  use,  among  others,  fossil  evidence  to  communicate  and  draw  conclusions  about  the 
world.  This  approach  to  curriculum  design  is  a modification  of  the  original  curriculum  integration  model  of 
Jacobs  (1989)  in  the  sense  that  it  focuses  on  maintaining  the  disciplinary  learning  objectives,  but  presenting 
them  in  a thematic  manner.  In  this  way,  the  teachers,  supported  by  PISD’s  curriculum  coordinators,  designed 
an  array  of  36  Ols  (Table  3). 


Overarching  Concepts:  Each  for  a Six  Week  Period  (not  in  teaching  sequence) 

Gd 

Continuity  & 
Change 

Systems 

Interactions 

Diversity 

Communication 

Balance  & 
Stability 

K 

Change  is  All 
Around  Me 
Changes 
Around  Us 

All  Aboard 
for  Systems 

What’s  the 
Connection? 
Grady's  Weather 
Window 

What's  Alike? 
What's 
Different? 
Animals  and 
Their  Homes 

How  Do  I Learn 
About  My 
World? 

Off  to  a “Beary” 
Good  Start! 

1 

Cycles 

Events  in  My 
World 

It's  No 
Mystery, 

It's  a System 
Calling  All 
Problem 
Solvers! 

Cause  and  Effect 

Why  Classify? 
Animals  in 
Their  World 

Making 

Connections 

Solid 

Foundations 
Fun  With  Maps 

2 

What  Impacts 
Our  World? 

How  Do 
Systems  Help 
Us  Connect  in 
Our  World? 
Weather, 
Climate  & 

You 

How  Are 
Communities 
Interdependent  in 
Our  World? 

Get  the  Story! 

How  Are  We 
Alike  and 
Different  in  Our 
World? 

Many  Places, 
Friendly  Faces 

What  Does 
Evidence  Tell 
Us  about  Our 
World? 
Message  in  a 
Fossil 

Where  is 
Harmony  in  Our 
World? 

3 

A Matter  of 
Time 

The  Way  We 
Live 

Where  Would 
We  Be 
Without 
Systems? 
Yourtown 

Seeking 
Reasonable 
Solutions 
Animals  in  Our 
World 

Proud  to  Be  an 
American 

The  Fine  Arts 
of  Sharing 
Adventures  in 
Problem 
Solving 

Drawing  Upon 
Our  Resources 

4 

Changes  Over 
Time 

One  Affecting 
the  Other 
Body  Balance 

Many  Faces, 
Many  Places 
Earth:  Forces 
and  Formations 

State  of 
Diversity 

Clues  That 
Communicate 
Sun,  Earth, 
Moon 

Mankind  and 
Nature  in 
Harmony 
Our 

Environment 

5 

How 

Variables 
Cause  Change 
Go  West!  and 
American 
Historical 
Map  Room 

In  Sync  with 
Systems 

How  the  Parts 
Interact  to  Make  a 
Whole 

Strength  in 
Diversity 
Decades  of 
Change 

The  Ins  and 
Outs  of 

Communication 

It’s  a Balancing 
Act 

Frontier 

Ahead! 

Map  Room  and  Visual  Planner  (Tools)  are  used  throughout  the  Ols  from  1 st  through  5th  grades. 
Interactive  Computer  Lessons  are  utilized  through  Grades  1 -5. 

Names  of  Ols  are  in  normal  text;  names  of  custom  educational  software  programs  are  in  italics. 


Table  3:  Plano  ISD  Integrated  Thematic  Curriculum:  Organizing  Ideas  and  Custom  Edunetics  Software 

Almost  all  of  the  Ols  included  custom  multimedia  learning  products  that  were  used  as  a crucial 
component  throughout  the  six-week  learning  period  in  the  01.  The  multimedia  products  were  designed  and 
produced  by  Edunetics’  instructional  design  specialists,  with  the  full  involvement  of,  and  guidance  from,  PISD 
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teachers.  All  OIs  were  designed  to  follow  a general  basic  structure  exemplified  by  the  Grade  Two  01,  "What 
Does  Evidence  Tell  Us  about  Our  World?"  (Figure  1)  under  the  theme,  "Communication":  An  introductory  1- 
week  unit,  followed  by  3-5  interdisciplinary-based  units,  and  a 1-week  culmination  unit.  The  culmination  unit 
brought  the  thematic  activities  and  understandings  together  through  further  synthesis,  analysis,  and  application, 
providing  a "big  picture"  thematic  overview.  Plano's  curriculum  design  process  was  unique  also  in  the  sense 
that  the  teacher  teams  took  the  leading  role  in  this  design  process  thereby  ensuring  that  both  the  overall 
curriculum  structure  and  the  computerized  and  off-line  learning  materials  complied  with  the  teachers’  and 
students’  curriculum  needs. 
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Figure  1:  Basic  organization  of  a six-week  thematic  unit  (01),  using  a Grade  Two  01  as  an  exemplar. 


Developing  the  learning  materials;  A unique  teacher-multimedia  producer  interaction 

The  Plano  Project  included  the  production  of  mainly  three  types  of  learning  materials: 

1 . Hundreds  of  offline  written  and  hands-on-small-group  activities  developed  by  the  teachers. 

2.  Approximately  200  short  discipline-based  interactive  "computer  lessons"  that  addressed  major  curricular 
topics  in  Science,  Language  Arts,  Math,  and  Social  Studies.  The  computer  lessons  included  simulations, 
games,  "lectu re-type"  presentations,  simulated  labs,  and  interactive  tutorials. 

3.  26  multimedia  interactive  learning  environments  (Table  3),  mainly  microworld  simulations,  databases,  and 
learning  tools  that  served  as  crucial  components  of  the  OIs.  In  addition,  PISD  teachers  incorporated  some  off- 
the  shelf  computer  products  that  were  available  on  the  market.  However,  according  to  the  Science  Coordinator 
in  PISD  (Mainwaring  1 999,  personal  communication),  teachers  found  their  use  somewhat  limited  because  these 
products  were  not  custom-designed  for  the  project. 

To  ensure  that  the  computerized  learning  materials  dovetailed  with  the  pedagogical  concepts  and  the 
content  requirements  of  the  integrated  curriculum,  a special  interactive  production  process  was  established.  The 
process  consisted  of  the  following  stages:  preliminary  ideas,  product  specification,  prototype  alpha  and  beta 
versions,  implementation,  and  evaluation.  Review  of  the  processes  occurred  throughout  production.  Both  the 
consumers  (teachers)  and  the  producers  (Edunetics)  were  equally  engaged  in  this  process.  According  to  Hord 
(1992),  long-term  changes  occur  if  teachers  take  part  in  corporately  contributing  towards  re-educating  their 
own  values  and  attitudes  and  changing  the  school's  culture.  This  includes  collaborative  in-situ  identification  of 
issues  by  the  change  agents;  in  our  case,  these  were  the  teachers,  curriculum  coordinators,  and  Edunetics 
pedagogical  content  and  software  producers.  From  the  PISD  experience,  such  a process  requires  long-range 
planning  and  can  be  challenging  to  execute  due  to  a variety  of  difficulties.  One  of  these  is,  not  surprisingly, 
given  their  differing  professional  backgrounds,  the  different  approach  to  planning  and  design  brought  by  the 
teachers  on  the  one  hand,  and  the  software  instructional  designers,  on  the  other.  The  successful  implementation 
of  the  computer  products  in  PISD  schools  (Henderson,  Klemes,  & Eshet  2000)  illustrates  the  value  of  producing 
custom-made  products  for  such  technology  integration  projects. 


Integrating  the  software  into  the  curriculum 
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A major  impact  of  computers  and  instructional  software  on  the  learning  process  of  children  is  achieved 
only  when  they  are  integrated  into  the  curriculum  as  a vital  element  for  instruction  (Shade  & Watson,  1 990).  In 
the  Plano  project,  the  custom-developed  educational  software  played  a significant  role  in  the  instruction  within 
each  01  (Table  3)  in  supplementing  the  disciplinary  and  interdisciplinary  content,  encouraging  students  to 
engage  in  higher  order  thinking  about  the  concepts,  and  in  creating  thematic  understanding.  An  example  of 
this  approach  is  seen  with  Message  in  a Fossil  (MIF),  one  of  the  26  multimedia  products  that  were  developed 
for  the  project  (see  Figure  1;  Table  3).  MIF  is  a microworld  simulation  in  which  the  student  plays  the  role  of  a 
paleontologist  who  excavates  in  virtual  dig-sites,  discovers  plant  and  animal  fossils,  and  reconstructs  the 
prehistoric  world  by  constructing  a museum  diorama.  According  to  a second  grade  teacher,  MIF  was  ’’the 
thematic  and  content  backbone  of  the  Organizing  Idea,  ’What  Evidence  Tells  Us  About  Our  World”' 
(Henderson,  Klemes,  & Eshet  2000).  The  major  theme  of  the  01  was  gathering,  interpreting,  and 
communicating  evidence  to  solve  mysteries  and  problems,  particularly  those  that  inform  our  understanding  of 
the  past.  For  approximately  45  minutes  each  day,  the  class  worked  in  stations  where  each  small  group  activity 
integrated  the  01  and  themes  across  curriculum  areas.  One  of  these  daily  rotating  activities  was  learning  with 
MIF,  which  was  also  included  in  the  time  allocated  for  reading.  This  meant  that  each  student  used  the  software 
each  day  for  at  least  20  minutes,  for  a period  of  6 weeks.  Henderson  et  al.  (2000)  concluded  that  the  length  of 
time  students  accessed  software  was  a significant  factor  in  student  learning  outcomes. 

Each  of  the  computer  products  was  used  in  a similar  way  to  MIF,  although  their  role  in  the  01  may 
have  varied  from  one  01  to  another.  In  the  course  of  developing  the  project’s  learning  materials,  it  was  found 
that  while  it  was  relatively  easy  to  incorporate  the  science  and  social  studies  topics  in  the  thematic  curriculum, 
it  was  much  harder  to  do  so  for  topics  in  Arts  and  Language  Arts.  Therefore,  these  topics  continue  to  be  taught 
separately,  in  a disciplinary  manner,  during  part  of  the  day. 

Technology  and  class  redesign 

Papert  (1993)  advocated  the  incorporation  of  computers  inside  classrooms  rather  than  in  computer  labs 
in  order  to  increase  their  impact  on  children’s  learning  and  a reconceptualization  of  classroom  layout  to  enable 
thematic  learning  and  group  work  (Jacobs  1989;  Marzano  1992;  Brooks  & Brooks  1993).  This  influenced  the 
PISD  project  as  it  included  a complete  redesign  of  the  architecture  of  the  classrooms  and  the  each  school’s 
technological  infrastructure.  In  order  to  enable  an  open  and  more  flexible  learning  atmosphere,  the  “frontal 
chalk-and-talk  classrooms”  were  redesigned  as  spaces  that  allowed  an  easy  access  to  the  class  computers  and 
promoted  team  work  and  group  discussions.  The  large  computer  labs  were  dismantled  and  replaced  by 
networked  computers  in  each  classroom,  at  a ratio  of  three  students  per  computer,  and  in  the  library  and 
administration  areas.  Accessory  equipment  such  as  a video  player  and  a large  TV  monitor  were  included  as 
classroom  equipment  along  with  a printer  for  each  four  classrooms.  The  networked  computers  enabled  student 
access  to  the  available  on-line  learning  environments  and  utility  programs.  This  combination,  of  new  classroom 
architecture  and  new  technology  infrastructure  design,  have  contributed  to  the  establishment  of  a new  teaching 
and  learning  culture  in  the  PISD  participating  schools  (Henderson,  Eshet,  & Klemes  1998;  Kent  & McNergney 
1998;  Provenzo,  Brett,  & McCloskey  1999). 

Impact  of  the  Plano  Project 

The  Plano  Curriculum  Integration  Project  had  a decided  impact  on  the  teaching,  learning,  and 
management  culture  of  the  entire  district,  affecting  the  teachers,  students,  and  district  administration.  (Eshet, 
Klemes,  & Henderson  1999).  One  such  impact  is  provided  in  a study  of  second  grade  children  who  participated 
in  the  Plano  Project  (Henderson  et  al.  1998;  Henderson  et  al  2000).  The  authors  report  several  clear  changes  in 
the  classroom  culture.  The  new  classroom  architecture  and  the  introduction  of  computers  into  the  classroom 
promoted  team  and  small  group  work.  This  also  affected  the  teacher’s  teaching  culture  that  became  more 
student-oriented  and  problem-solving  based.  The  research  findings  also  indicate,  in  comparison  with  the  way  it 
was  taught  pre-MIF  and  the  new  thematic  integrated  curriculum,  an  increase  in  students’  engagement  in  the 
learning  processes  and  an  improvement  in  transferring  their  learning  into  other  domains  (Henderson  et  al. 
2000).  A second,  and  probably  the  biggest  impact  of  the  project,  is  best  portrayed  by  the  fact  that  today,  the  30 
to  40  teachers  who  participated  in  designing  the  thematic  curriculum,  are  engaged  in  training  teachers 
nationwide  in  developing  similar  curriculum  integration  projects.  A final  impact,  the  decision  of  PISD  to 
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extend  the  Curriculum  Integration  Project  into  their  middle  schools,  illustrates  successful  implementation  of 
systemic  change  in  a school  district. 

Conclusions 

The  technology-based  curriculum  integration  project  in  Plano  provides  a valuable  opportunity  to 
identify  the  ingredients  of  a successful  technology  integration  project.  The  following  are  highlighted: 

1.  Introducing  hardware  and  software  is  not  enough;  an  educational  technology-based  project  must  involve 
appropriate  curricular  considerations  that  might  require  the  design  of  an  entire  new  curriculum. 

2.  Learning  materials  should  be  developed  especially  to  complement  and  be  integral  to  the  curriculum.  This 
applies  particularly  to  computer-based  learning  materials. 

3.  Teachers  should  be  involved  in  agenda-setting  and  decision-making,  including  those  concerned  with  the 
curricular  approach  and  production  of  learning  materials,  both  on-line  and  off  line. 

4.  The  introduction  of  technology  into  schools  requires  consideration  of  classroom  architecture,  placement  of 
in-class  computers,  and  technology  infrastructure,  and,  not  least,  their  effect  on  the  teaching-learning 
culture  in  school  and  school-home  interconnectivity. 
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Abstract;  This  paper  is  a report  of  the  findings  of  a study  that  describes  the  development, 
evaluation,  and  revision  process  for  an  on-campus  graduate  level  Statistics  course  website. 
Focus  groups,  interviews,  and  document  analysis  were  examined  to  determine  patterns  and 
amount  of  student  use  during  the  website’s  initial  semester  of  use.  Findings  indicate  a lack  of 
integration  into  course  activities  contributed  to  minimal  student  use  of  the  website  throughout 
the  semester.  Student  expectations  about  the  currency  of  information  on  a course  website 
suggest  that  regular  review  and  updating  of  on-line  materials  should  be  planned  for  and 
accomplished.  If  on-line  conferencing  is  to  be  part  of  a course  website,  it  needs  to  be  both 
required  and  structured  by  the  instructor  to  be  successful.  Finally,  a comprehensive  review  of 
printed  course  materials  should  be  performed  in  order  to  ensure  that  website  contents  are  not 
simply  a redundant  collection  of  course  materials. 


Introduction 

Since  1994,  when  the  worldwide  web  became  available  to  most  colleges  and  universities,  the  number 
of  websites  for  on-campus  courses  has  grown  exponentially.  The  World  Lecture  Hall,  a directory  of  these 
websites  at  the  University  of  Texas-Austin,  lists  129  websites  in  the  field  of  Education  alone,  and  this  listing  is 
far  from  comprehensive.  But  too  many  of  these  websites,  we  would  argue,  are  thrown  together  quickly,  almost 
immediately  become  outdated,  and  do  not  effectively  serve  the  needs  of  the  students  taking  the  course,  their 
primary  audience.  And  once  created,  these  websites  often  tend  to  be  ignored  by  the  instructor,  becoming 
outdated  and  irrelevant,  filled  with  useless  information  about  past  semesters  and  a graveyard  of  dead  links. 

This  paper  describes  the  systematic  design  and  development  process  for  a graduate-level  statistics 
course  website,  with  a special  focus  on  the  continuing  development  and  revision  process  after  the  initial 
semester  of  use  of  the  website  is  over.  We  wanted  to  discover  how  the  website  had  actually  been  used  during 
the  course  of  the  semester,  and  how  it  met,  or  failed  to  meet,  the  needs  of  the  students  and  the  course  instructor. 
We  also  undertook  a careful  review  of  the  website  usage  logs  to  determine  patterns  of  use. 

The  design  team  consisted  of  three  advanced  doctoral  students  from  the  Instructional  Systems 
Technology  department  at  Indiana  University.  During  the  initial  design  process,  the  project  team  followed  a 
common  instructional  design  and  development  model.  Major  sections  of  the  design  effort  included:  analysis  of 
the  task  and  audience,  prototype  design  and  testing,  design  revision  and  full-scale  development,  final  usability 
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testing  and  revision,  implementation  and  summative  evaluation.  In  this  paper,  however,  we  will  focus  our 
description  and  analysis  on  our  post-instructional  website  development  activities. 


Methods 

Our  analysis  in  this  study  focused  on  two  major  questions.  The  first  question  was,  “How  (and  when, 
and  where,  and  why)  did  the  students  actually  use  the  course  website?”  The  second  question  was,  “Based  on  the 
patterns  of  student  usage  found  in  the  initial  semester  of  use,  what  changes  in  the  course  website  (in  terms  of 
content,  structure,  etc.)  need  to  be  made?”  To  answer  these  questions,  we  conducted  focus  groups  of  a random 
sampling  of  students  who  had  just  taken  the  course.  This  was  followed  up  by  a survey  of  the  entire  population 
of  students  who  had  taken  the  course.  The  instructor  was  interviewed  about  her  use  of  the  website  and  her 
perceptions  of  student  use.  Finally,  the  website  usage  logs  were  carefully  analyzed 


Focus  Groups 

A random  sampling  of  students  enrolled  in  the  Spring  1 999  section  of  the  course  were  invited  to  focus 
group  sessions  to  discuss  their  use  of  the  website  during  the  semester.  A sequence  of  questions  was  developed 
for  these  sessions  by  the  design  team,  and  additional  questions  were  added  during  the  sessions  themselves. 
These  questions  were  chosen  to  gain  an  understanding  of  several  issues:  which  sections  of  the  extensive  website 
were  most  commonly  used;  the  ease  of  use  of  the  website;  the  times  during  the  semester  that  the  website  was 
used,  and  the  tasks  associated  with  that  use;  and  those  aspects  of  the  website  that  were  incomplete  or  lacking. 


Surveys 


With  that  data  in  hand,  we  then  sent,  via  email,  a survey  to  all  of  the  students  who  had  taken  the  course 
in  the  past  semester.  The  survey  was  designed  to  help  us  confirm  the  findings,  and  assure  us  of  the  accuracy,  of 
our  focus  group  data.  The  survey  asked  the  students  questions  similar  to  those  asked  during  the  focus  groups, 
simply  revised  and  refined  versions  to  fit  the  survey  format. 


Instructor  Interviews 

Throughout  the  semester  previous  to  the  first  usage  of  the  website,  and  during  the  website’s  initial 
semester  of  use,  we  had  met  with  the  instructor  of  the  course,  a tenured  professor  in  the  Counseling  and 
Educational  Psychology  department.  We  again  met  with  her,  just  before  our  focus  groups  with  the  students,  to 
gain  an  understanding  of  her  use  of  the  course  website  during  the  semester,  how  she  had  introduced  the  website 
to  the  students,  and  her  perception  of  the  amount  and  quality  of  student  use  of  the  website. 


Document  Analysis 

We  had  access  to  the  usage  logs  for  the  course  website,  which  gave  us  information  on  how  often  each 
page  (of  the  over  100  web  pages  of  the  site)  was  accessed  and  when  it  was  accessed  during  the  semester.  We 
were  able  to  determine  the  most  popular  pages  on  the  website  and  the  days  during  the  semester  that  the  website 
was  most  often  used.  This  information  was  compared  with  the  course  schedule  to  help  understand  patterns  of 
student  use  as  they  related  to  course  assignments.  We  were  also  able  to  view  aggregate  information  about 
website  visitors.  This  information  helped  us  determine  whether  the  students  were  accessing  the  site  from 
computer  labs  on  campus,  dormitories  on  campus,  or  through  dial-up  connections. 


Findings 
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Student  Use  of  Site 


Unfortunately,  we  found  that  the  course  website  was  not  used  much  during  the  initial  semester, 
according  to  the  students  we  talked  to.  Server  logs  back  this  up,  though  use  did  increase  from  729  page  requests 
in  January  to  a high  of  5162  requests  during  March,  before  falling  back  to  765  students  again  in  May,  as  the 
course  was  ending.  For  comparison,  another  course  offered  by  the  same  department  had  10571  requests  during 
March,  almost  twice  as  much  activity  (Bonk,  2000).  As  one  student  noted  regarding  the  website,  “Didn’t  use  it 
much.  Didn’t  have  to.” 

Students  indicated  that  the  professor’s  reference  to  and  use  of  the  website  in  course  meetings  was 
minimal.  Several  mentioned  that  they  might  have  used  the  website  more  if  they  had  a better  understanding  of  its 
contents.  During  one  of  the  focus  group  sessions,  one  particular  student  discovered  many  useful  and  interesting 
resources  when  he  was  reviewing  the  site  contents.  He  commented  that,  “I  am  currently  working  on  a research 
design  and  1 can  really  use  these  resources  right  now.  The  instructor  should  have  emphasized  this  more  during 
the  class.” 


Technology  Details 

The  campus  provides  both  Windows  and  Macintosh  computers  in  student  computer  labs,  and  both 
platforms  were  used  to  access  the  site.  The  server  logs  confirmed  student  reports  that  they  accessed  the  website 
from  university  labs,  dorms  and  libraries  and  off-campus  homes.  Both  high-speed  networks  and  slower  modems 
were  used  to  access  the  site. 

Both  Internet  Explorer  and  Netscape  web  browsers  were  used  by  students  to  access  the  site. 
Interestingly,  many  students  reported  using  both  web  browsers  depending  upon  the  location  from  which  they 
accessed  the  site.  Perhaps  surprisingly,  students  reported  no  technical  problems  accessing  or  using  the  site,  even 
though  it  included  advanced  Javascript  features  on  every  page. 


Site  Content 

Students  primarily  used  the  site  for  one  purpose:  viewing  and  printing  examples  of  assignments  and 
bibliographies  (which  were  not  in  the  course  reader)  as  models  for  their  own  work.  The  article  review  samples 
and  bibliographies  were  cited  by  many  students  as  by  far  the  most  useful  elements  of  the  site.  For  example,  in 
March  and  April,  1999,  during  the  part  of  the  semester  that  the  students  did  the  majority  of  their  assignments  for 
the  course,  the  page  that  included  links  to  these  items  was  the  second  most  popular  page  on  the  site,  after  the 
home  page. 

Other  sections  of  the  site  that  were  favorably  mentioned  were:  the  links  to  information  about  the 
software  that  was  to  be  used  in  the  course,  sample  questions  for  the  test,  instructor  contact  information,  and  the 
links  to  professional  journals.  Students  reported  that  they  occasionally  recommended  the  site  to  other  students 
or  referred  peers  to  information  they  found  on  the  site,  especially  information  regarding  assignment  completion. 
One  feature  of  the  site,  the  bulletin  board,  a feature  that  can  produce  much  site  activity,  was  not  used  at  all,  and 
some  students  commented  that  they  wished  it  had  been  used.  One  student  said,  “The  discussion  section  would 
be  helpful  for  after-hours  help.”  Normally  when  the  students  sought  after-hour  help,  they  sent  email  messages  to 
the  professor.  The  professor  would  send  a reply  directly  to  the  student  and  forward  a copy  of  the  message  to  the 
whole  group.  Students  commented  that  the  communication  between  students  and  professor  might  have  been 
more  efficient  if  the  bulletin  board  had  been  used. 

Some  students  mentioned  material  that  could  have  been  included  on  the  site  but  wasn’t,  such  as:  links 
to  the  library’s  online  article  search  systems,  answers  to  lab  assignments,  lab  assignment  handouts,  case  study 
examples,  and  sample  problems  to  use  with  the  statistical  software.  One  student  complained  that  while  they  read 
from  several  books,  only  the  readings  from  one  of  the  books  were  included  on  the  schedule.  Some  of  the 
content,  specifically  examples  of  studies  cited  in  the  bibliography,  seemed  out  of  date  to  the  students  as  they 
were  several  years  old.  The  list  of  scheduled  readings  was  also  out-of-date,  referring  to  the  schedule  of  a past 
semester. 

We  got  the  strong  indication  from  students  that  because  use  of  the  site  was  not  necessary  to  be 
successful  in  the  course,  they  therefore  had  little  motivation  to  visit  it.  Several  commented  that  because  much  of 
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the  site’s  contents  were  already  in  their  printed  course  reader,  it  was  not  necessary  to  access  them  via  the 
website. 


Site  Design 

Overall,  students  reacted  favorably  to  the  look  of  the  site  and  the  utility  of  the  navigation  options. 
Several  students  commented  that  they  appreciated  the  large  image  on  the  home  page  and  the  color  scheme  of 
the  site  (Fig,  1),  One  student  said  the  site  was  “fiin  looking,”  though  another  complained  that  the  light  blue  of 
the  central  image  made  the  site  look  “faded.”  After  extensive  usability  testing,  detailed  earlier  in  this  paper, 
several  methods  of  navigating  the  site  (menu  bar,  sidebar,  search,  pulldown  menu)  had  been  provided  for 
student  use  and  each  of  these  systems  was  in  fact  used  by  some  students  in  our  focus  group  interviews,  though 
some  showed  strong  preferences  for  one  system  over  another.  For  example,  one  student  said,  “I  never  used  the 
buttons  at  the  top,  I just  used  the  links  at  the  bottom.”  Students  pointed  out  that  some  secondary  pages  lacked 
the  navigation  elements  found  on  primary  pages  of  the  site. 


^Y603:  Research  Design  4.0  Home  - Microsoft  Internet  Explorer 
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Figure  1:  Website  Main  Page 


The  website  did  not  influence  student’s  feelings  about  the  course  in  general;  however,  some  students 
felt  Dr.  Peng  was  a strong  advocate  of  instructional  technology  because  she  had  her  own  website. 


er|c 


323 


BEST  COPY  AVAILABLE 


Recommendations 


Based  on  our  findings,  we  have  made  the  following  recommendations  to  the  instructor  of  the  course. 

• A careful  review  of  the  contents  of  the  printed  materials  given  to  the  students  needs  to  be  made.  Which  of 
these  materials  would  more  appropriately  be  provided  in  print  form,  and  which  in  online  form?  Are  there 
materials  that  should  be  available  in  both  formats? 

• Students  expect  online  resources  to  be  regularly  updated,  providing  them  with  the  most  current  information 
available.  Because  of  this  student  expectation,  materials  on  the  website  should  be  periodically  reviewed  and 
updated  to  reflect  changes  in  the  course,  related  materials,  and  on-line  resources. 

• The  instructor  should  consider  methods  by  which  the  students  can  be  made  more  aware  of  the  resources 
contained  on  the  website  from  the  very  beginning  of  the  semester.  Perhaps  part  of  the  initial  class  session 
should  be  used  to  introduce  the  website.  Throughout  the  semester,  the  instructor  could  also  point  out 
occasions  when  the  website  could  be  profitably  used. 

• Course  activities  which  require  more  use  extensive  use  of  the  website  resources  should  be  integrated  into 
the  course. 

• The  instructor  should  take  advantage  of  the  interactive  potential  of  the  website  by  providing  opportunities 
for  students  to  discuss  issues  related  to  the  course.  Specifically,  if  the  bulletin  board  conferencing  tool  is  to 
be  included  as  part  of  the  website,  students  need  to  be  made  aware  of  its  existence  and  encouraged  to 
participate  in  the  discussions.  We  suggest  that  use  of  the  tool  should  be  made  a required  part  of  class 
participation,  since,  in  our  experience  with  courses  using  such  tools,  unless  use  is  mandatory,  some  students 
will  only  minimally  use  the  tool  and  many  will  avoid  it  all-together.  (ARTT:  Practical  Strategies  for  Using 
IT,  2000) 


Limitations  and  Recommendations  for  Further  Study 

While  we  do  not  claim  that  our  experiences  are  generalizable  beyond  this  particular  website  design 
project,  we  do  believe,  based  our  experiences  as  instructional  designers  for  other  course  websites,  that  our 
findings  and  recommendations  will  be  useful  to  other  instructional  designers  working  on  similar  projects. 
Regarding  the  limitations  of  this  study,  obviously  it  is  only  one  course  of  around  35  students,  and  the  particular 
subject  of  the  course,  statistics,  may  have  had  some  important  implications  regarding  the  students’  use,  or  lack 
thereof,  of  the  site.  Also,  the  fact  that  the  website  design  team  also  interviewed  and  surveyed  the  students  may 
have  had  some  influence  on  their  responses,  though  we  attempted  to  minimize  this. 


Conclusions 

Course  websites,  even  given  their  growing  numbers,  are  still  a new  resource  for  instructors  and 
students;  they  are  exploring  an  unfamiliar  realm  when  it  comes  to  using  this  new  technology  in  their  teaching 
and  learning.  As  instructional  designers,  we  need  to  keep  the  academic  needs  of  both  students  and  instructors  as 
our  main  focus  throughout  the  design  and  revision  process.  Unless  the  course  website  serves  their  needs  as 
efficiently  and  effectively  as  possible,  it  will  not  be  utilized  and  our  time  and  energy  in  developing  it  will  be 
wasted.  We  hope  that  in  this  study  we  have  added  to  the  knowledge  available  about  this  design  task,  and  hope 
that  other  instructional  designers  will  find  our  experience  useful. 
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Abstract:  The  capability  of  Hypermedia  to  exploit  associative  linking  between  contents  and  to 
provide  multiple  navigation  paths  through  them  makes  this  medium  predestinated  for  mapping  and 
representing  Imowledge  spaces  (Bush  48).  Especially  the  possibility  of  mapping  knowledge  makes 
Hypermedia  a very  interesting  medium  for  educational  purposes.  But  access  to  information,  as  e. 
g.  in  the  Internet,  does  not  equal  learning.  In  educational  settings,  we  must  adapt  the  mere  access  to 
information  to  educational  needs.  On  the  one  hand  we  need  to  support  the  authors  in  the  production 
and  maintenance  of  these  non-linear  contents,  on  the  other  hand  we  must  give  the  students  tools  to 
be  able  to  work  with  these  contents.  In  this  article,  we  present  the  core  functionality  of  an  adaptable 
learning  environment,  ALISl.  The  ALIS  project  aims  to  support  the  production  and  use  of 
interactive  Web-based  learning  applications.  We  have  developed  an  authoring  System  for 
organizing  the  contents  of  the  learning  system  and  an  editing  tool  for  the  students,  which  allows 
them  to  adapt  the  learning  material  while  working  interactively  with  the  learning  materials. 


Introduction 

In  the  recent  years  the  demand  in  computer-based  learning  systems  has  been  growing  continuously.  This  is  due  to 
the  need  of  life-long  learning  and  the  fact  that  most  of  the  students  wish  to  be  able  to  learn  any  time,  any  where. 
Using  hypermedia  systems  for  achieving  these  goals  seems  very  reasonable  because  of  the  characteristics  of  this 
medium.  A hypermedia  system  is  a good  platform  for  representing  knowledge  spaces  and  for  interacting  and 
navigating  in  them.  The  very  strength  of  hypermedia  as  a medium  to  transfer  knowledge  is  the  mapping  of  nonlinear 
content. 

Normally,  when  writing  course  material,  the  authors  convert  their  nonlinear  knowledge  into  a linear  form  to  be  able 
to  transfer  this  knowledge  on  a linear  medium  such  as  written  text.  The  learners  on  the  other  side  must  de-linearize 
the  contents  to  be  able  to  embed  this  knowledge  in  their  own  knowledge  space  (Lowe,  Hall  99). 

It  is  clear,  that  both,  the  authors  and  the  students,  would  not  need  to  go  through  this  process,  if  they  were  able  to 
work  at  the  non-linear  level.  On  the  other  hand  the  capabilities  of  hypermedia  to  exploit  associative  linking  between 
the  learning  objects  and  to  provide  multiple  navigation  paths  through  them  make  the  production  and  the 
maintenance  of  material  very  difficult.  These  difficulties  are  increased  by  the  rapid  technological  progress  and 
constant  change  in  tools  and  standards.  Especially  the  building  of  hypermedia  learning  systems  is  a cumbersome  and 
expensive  task  because  the  authors  need  various  skills  to  develop  the  solutions.  They  have  to  decide  how  to  model 
and  represent  the  knowledge  to  be  transferred  to  the  students,  how  to  organize  and  administer  this  material,  how  to 
embed  it  in  existing  technology  standards  (browsers,  operating  systems,  ...)  and  how  to  offer  the  students  tools  to 
work  effectively  with  this  learning  material. 


The  Idea 

The  idea  behind  the  ALIS  System  can  be  summarized  by  the  following  requirements  for  a learning  environment: 


1 http://www.femuni-hagen.de/ALIS/ 
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©Learning  is  the  process  of  knowledge  construction,  not  knowledge  recording  or  absorption  (Harel  & 
Papert  91) 

©The  learners  need  environments,  in  which  they  can  be  an  active  designer  rather  than  a consumer  (Fischer 
98) 

Consequently  we  need  a learning  tool  which  has  the  following  characteristics: 

©it  is  adaptable,  and  expandable 

©it  visualizes  also  the  semantic  interrelation  of  the  contents  and  gives  the  learner  the  possibility  to  use  this 
as  an  interface  to  adapt  these  structures  to  his  own  needs. 

©it  must  be  a generic  tool,  so  it  can  be  used  for  any  kind  of  domain 

The  interesting  question  at  this  point  is,  how  is  it  possible  to  build  a generic  system,  which  can  map  a knowledge 
space  independent  of  its  contents  As  a solution,  we  have  chosen  to  give  the  learner  always  two  visual  perspectives  of 
the  knowledge  space.  On  the  one  hand  we  have  the  presentation  of  the  particular  content  (at  the  present  time  made 
of  HTML  pages)  in  which  other  media  types  (e.g.  JPEG,  GIF,  MPEG,  applet)  are  integrated.  Thus  one  special  aspect 
is  covered  and  shown  in  a browser  window. 

On  the  other  hand  we  have  got  an  overview  of  all  pieces  of  content  and  their  relations  within  the  whole  context.  For 
this  purpose  we  use  the  mind  map,  which  is  a specialized  form  of  a concept  map. 

A concept  map  is  a well-known  formalism  for  representing  structural  knowledge  (Jonassen  93).  Concept  maps  tend 
to  be  much  easier  for  human  users  to  understand  than  other  knowledge  representations  such  as  pure  text  or  predicate 
logic  (Nosek  & Roth  90).  It  is  a directed  acyclic  n-dimensional  graph  consisting  of  a set  of  m concept  labels 
{Cl...Cn}  and  a non-empty  set  of  r relationships  between  them.  Different  diagrammatic  representations  are  used  to 
visualize  such  graphs.  One  of  the  most  popular  study  aids  in  educational  settings  are  mind  map  (Buzan  93).  Their 
advantages  for  the  learner  are  described  e.g.  in  (McAleese  98,  Novak  & Gowin  84).  McAleese  stresses  the  role  of 
concept  maps  for  the  important  self-regulatory  study  process,  which  he  calls  "reflection". 


The  Learning  Tool 

We  have  developed  a Java  and  XML-based  learning  tool,  which  aims  to  provide  more  flexibility  for  students  while 
working  with  courses  and  allow  them  to  personalize  them.  It  can  be  used  online,  off-line  or  in  a mixed  way. 

Its  user  interface  provides  two  separate  windows:  one  of  them  is  a normal  browser  window  presenting  the  content, 
the  other  one  is  a structural  navigation  tool  as  shown  in  Fig.  1 . The  navigation  window  includes  a concept  map 
(right  and  lower  left  window  pane),  a navigation  tree  (upper  left  pane),  and  a pyramid  shaped  depth  indicator 
(middle  left  pane). 
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Figure  1 : The  learning  tool 


Functionality  of  the  Learning  Tool 

With  ALIS,  the  learners  are  able  to  access  the  content  directly  in  the  browser  window  and  navigate  from  content 
page  to  content  page.  Alternatively,  they  can  manipulate  a visual  concept  map.  The  manipulations  of  ALIS'  concept 
include:  shifting  the  focus  to  the  topic  of  interest,  folding  or  unfolding  concepts  nodes,  adding  nodes  or  associative 
links  among  nodes  that  are  not  member  of  the  same  sub-hierarchy,  or  select  presentation  pages  to  be  shown  in  the 
browser  window  in  a point-and-click  fashion.  In  addition  the  learners  can  easily  insert  their  own  notes  and  select 
bookmarks  in  the  map  and  save  same  a personal  bookmark  register  in  the  navigation  tree.  The  navigation  tree,  which 
shows  a folder  hierarchy,  allows  the  learner  to  access  the  content  by  clicking  into  a selected  folder  hierarchy  and 
selecting  a leaf  item  denoting  a content  page  opened  up  in  the  browser.  The  nodes  or  folders  in  the  tree  are  color 
coded.  Red  indicates  an  unread  item,  blue  indicates  items  visited,  and  green  refers  to  a page  or  collection  of  pages 
the  user  confirmed  explicitly  to  have  understood.  There  is  also  the  possibility  for  the  user  to  create  context  sensitive 
views  on  the  content  in  different  trees,  which  can  be  accessed  by  separate  registers.  For  instance,  a student's 
learning  application  might  include  a tree  for  examples,  for  related  links,  for  exercises,  and  for  definitions  allowing 
him/  her,  for  example,  to  quickly  refresh  the  content  before  entering  an  examination.  Another  pane  of  the  navigation 
tool  shows  a 3D  visualization  of  the  focus  depth  hierarchy.  The  idea  behind  this  view  is  to  indicate  the  level  of  detail 
a knowledge  piece  as  belonging  to.  The  assumption  here  is  that  the  deeper  the  content  is  placed  in  the  hierarchy  the 
more  specific  the  selected  information  is.  The  pyramid  window  also  includes  a learning  progress  indicator  (the  bar 
on  the  right  side  of  this  pane),  which  shows  the  learning  progress  percentage-wise  using  the  same  color  coding  as  in 
the  tree  representation. 

The  different  views  on  the  course  material  are  always  synchronized.  This  ensures  that  the  structural  navigation  tool 
always  refers  to  the  page  shown  in  the  browser  window  and  vice  versa.  The  learning  tool  supports  also  a merge 
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functionality.  This  means  that  the  students  are  able  to  synchronize  their  own  extensions  with  other  students  or  with 
the  update  information  about  the  course  on  our  servers.  This  functionality  gives  them  the  possibility  of 
collaborating  with  other  students. 


Implementation 

The  first  version  of  the  navigation  tool  is  operational  and  in  use. 


The  Authoring  System 

According  to  Elliot  and  al.  (Elliot  et  al.  95)  "if  an  authoring  environment  were  to  allow  developers  to''  draw  "their 
subject  domain  on  a screen  as  a semantic  net  and  then  add  the  hypermedia  features  it  would  considerably  facilitate 
development".  In  the  ALIS  project,  we  are  developing  an  authoring  system,  which  allows  authors  to  use  this 
framework  to  gather  pieces  of  information  and  weave  them  into  a hypermedia  system.  For  doing  so,  ALIS  offers 
several  views  upon  the  content  and  its  structure.  Our  main  goal  by  developing  the  authoring  system  was  to  give  the 
authors  the  possibility  to  concentrate  on  forming  and  designing  the  subject  domain  and  to  be  able  to  store  this 
knowledge  into  a data  base. 

In  the  first  pane  the  authors  define  the  educational  objectives  to  be  able  to  control,  whether  all  of  the  relevant  themes 
are  included.  In  a second  pane,  they  use  the  concept  map  designer  to  structure  and  organize  the  information  space. 
While  working  with  the  concept  map,  they  are  able  to  compare  the  consistency  of  the  contents  with  the  educational 
objectives  and  check  the  associations  between  contents.  After  putting  up  the  main  semantic  structure  of  the  contents, 
the  authors  can  use  this  scaffold  and  fill  it  with  contents  and  style  information  by  clicking  the  nodes  of  the  concept 
map  and  adding  the  descriptions  to  this  particular  topic.  After  collecting  and  structuring  the  information  in  this  way, 
it  is  possible  to  automatically  generate  and  build  up  the  learning  application  out  of  the  data  base.  The  advantage  of 
this  approach  is  a high  level  of  separation  between  knowledge,  its  presentation  and  its  implementation  in  a 
hypermedia  application.  The  authors  don't  need  to  take  care  about  the  presentation  of  the  contents,  the  navigational 
support,  and  other  facilities  of  an  learning  application. 


Implementation 

The  first  version  of  the  authoring  system  is  operational.  It  supports  the  building  and  organizing  the  of  contents  at  the 
semantic  level.  The  screen  units  can  be  edited  with  external  HTML  Editors.  In  the  next  level  of  development  we  will 
also  separate  the  style  and  markup  information  from  the  contents  to  store  them  also  in  the  data  base.  Doing  so,  the 
authors  can  assemble  the  course  and  the  contents  with  their  style  and  markup  information,  at  the  time  of  delivering. 


Conclusion 

In  this  article  we  have  illustrated  the  learning  environment  ALIS  with  its  generic  functionality.  ALIS  lets  the 
students  adapt  the  learning  material  to  their  own  needs  by  adding  links  to  up-to-date  information  and  integrating 
their  own  elaborations.  Students  collaborating  in  geographically  dispersed  teams  can  make  their  personal  extensions 
accessible  to  their  peers  through  a simple  update  function  that  just  exchanges  the  differences  in  the  individual  link 
structures.  Flexible  navigation  mechanisms  support  multiple  perspectives  on  the  course  material  and  allow  them  to 
find  their  own  tour  through  it.  Personal  notes  and  links  to  related  material  of  their  choice  can  be  added  by  drag-and- 
drop  functions. 

We  know  that  the  author's  task  of  developing  such  flexible  applications  is  difficult  and  hardly  supported  by  suitable 
tools  reaching  the  comfort  of  traditional  CASE  tools.  Commercial  authoring  tools  such  as  Director,  Toolbook,  or 
Authorware  can  at  best  be  considered  low  level  design  and  programming  tools. 

Few  tools  and  techniques  are  available  at  present  that  help  authors  to  elicit  the  requirements  for  a new  course 
(including  learning  objectives,  subject  topics  to  be  taught,  appropriate  didactic  approach,  learning  material  and 
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presentation  media  suitable  for  the  clients,  navigation  structure)  and  systematically  evolve  such  requirements  into  a 
hypermedia  application.  The  extensions  we  plan  for  ALIS  aim  to  overcome  some  of  these  difficulties  in  the  near 
future. 
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Abstract:  This  paper  describes  research  being  conducted  at  Rutgers  University  into  the  design 
of  Distributed  Learning  Systems  for  asynchronous  delivery  of  multimedia  educational 
material  to  students  via  broadband  networks.  A pilot  system  called  Aristotle  comprising 
videotaped  lectures,  text  transcripts  and  textbook  figures  all  organized  into  short  synchronized 
presentations  has  been  successfiilly  implemented  and  is  described  in  detail.  The  experience 
gained  during  the  project  has  fueled  ongoing  research  in  the  areas  of  automated  speech 
recognition,  automated  question-answering,  and  development  of  tools  for  generating  and 
organizing  media  content,  all  briefly  described  herein. 


1.  Introduction 

This  paper  describes  the  design  and  implementation  of  the  Aristotle  system,  a web-based  educational 
tool  intended  to  supplement  classroom  instruction  for  an  introductory  course  in  General  Biology.  The  Aristotle 
system  is  a Distributed  Learning  System  that  makes  a multimedia  database  containing  segmented  lecture 
material  augmented  with  text  and  graphics  available  to  students  via  a broadband  network.  Section  2 of  this 
paper  presents  the  design  approach.  The  implementation  of  the  Aristotle  system  is  described  in  sections  3 and 
4,  which  contain  an  architectural  overview  followed  by  a more  detailed  discussion.  Section  5 describes  the 
deployment  of  the  system  and  the  design  of  an  ongoing  study  to  determine  its  effectiveness.  The  paper 
concludes  in  Section  6 with  a summary  of  proposed  enhancements,  which  are  currently  under  development. 


2.  Approach 

In  the  development  of  Aristotle  as  well  as  its  ongoing  work,  the  project  team  considered  three  main 
objectives  : 

1 . to  present  information  to  the  student  in  a highly-integrated  manner 

2.  to  organize  information  content  in  a way  that  facilitates  learning 

3.  to  develop  tools  and  processes  for  efficient  capture  of  data 

These  objectives  form  the  basis  of  an  approach  to  the  design  of  Distributed  Learning  Systems  which  is 
discussed  more  fully  in  the  following  paragraphs. 

Integrated  Presentation  of  Information 

We  seek  to  provide  students  with  a learning  experience  that  is  convenient,  engaging  and  saves  time. 
The  presentation  of  information  will  be  attractive,  and  facilitate  the  cognitive  processes  used  in  learning.  To 
achieve  these  goals,  the  system  will  support  user  interaction  via  all  the  sensory  modalities  employed  during 
traditional  classroom-style  instruction:  visual/auditory  (while  listening  to  the  lectures),  visual/tactile  (recording 
notes),  verbal/auditory  (asking  questions),  visual-only  (reading  textbook).  Moreover,  all  interaction  can  occur 
at  a single  time  and  location,  by  contrast  with  conventional  learning  where  information  delivery  is  disjoint 
(lectures,  reading,  and  visits  to  library  and  instructor  must  be  done  at  different  times  and  places)  and  the  student 
must  expend  extra  effort  to  integrate  these  experiences  into  a consistent  framework. 
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Content  Organization 

We  will  introduce  new  logical  structures  for  defining  and  linking  objects  within  multimedia  databases. 
The  objects,  which  we  call  “knowledge  containers”,  will  present  educational  information  pertaining  to  concepts 
or  ideas.  The  links  will  describe  relationships  between  the  objects  and  will  facilitate  learning  and  rapid 
information  retrieval.  For  example,  a student  learning  for  the  first  time  about  viruses  may  obtain  a clearer 
understanding  by  referring  to  a description  of  a specific  virus  such  as  influenza  or  by  contrasting  it  with  a 
disease  caused  by  infection,  such  as  scarlet  fever.  However,  the  student  may  not  initially  know  enough  about  a 
high-level  concept  to  search  for  the  correct  terms  or  phrases.  In  this  example,  the  "virus”  container  could  be 
connected  to  the  "influenza”  container  via  a link  of  type  "example",  and  to  the  "scarlet  fever"  container  via  a 
link  of  type  "counter-example". 

These  new  types  of  data  structure  will  also  facilitate  so-called  “meta-cognition”  or  “meta-leaming”  - 
that  is,  a student’s  conscious  awareness  of  the  process  by  which  he  or  she  acquires  knowledge.  Through  a 
visual  representation  of  the  knowledge  containers  and  their  relationships,  a student  can  track  his  or  her  progress 
in  learning  a subject,  and  to  make  better  decisions  regarding  which  topics  or  ideas  to  consider  next. 

Content  Generation 

A key  design  goal  will  be  to  minimize  the  labor  required  for  capture  and  organization  of  lecture 
material.  We  will  seek  methods  for  unobtrusively  capturing  lectures  as  they  are  delivered  within  the  normal 
classroom  setting.  Automated  tools  and  methodologies  will  be  developed  for  video  capture,  speech 
transcription,  and  multimedia  presentation  authoring.  These  tools  will  be  custom-built  for  the  distributed 
learning  application  in  order  to  ensure  that  they  will  be  simple  and  efficient. 

3.  Project  Overview 

The  Aristotle  project  is  a pilot  program  whose  purpose  is. to  explore  the  technologies  and  concepts 
described  in  the  introduction.  For  this  project,  a web  site  was  created  containing  multimedia  instructional 
material  derived  from  an  introductory  Biology  course  (http://lilac.rutgers.edu/bio/aristotle.html). 

Twenty-five  lectures  from  the  course  were  videotaped  at  an  on-campus  television  studio.  Using 
written  outlines  of  the  lectures,  the  project  team  prepared  many  brief  multimedia  presentations  that  could  be 
indexed,  and  retrieved  using  a personal  computer.  Software  utilities  were  written  to  generate  the  hundreds  of 
files  associated  with  the  presentations  automatically.  The  presentations  include  video  and  sound  clips  from  the 
lecture,  textbook  figures  that  pop  up  to  illustrate  concepts  referred  to  in  the  lectures,  scrolling  text  transcripts  of 
the  lecture,  and  written  definitions  of  biology  terms.  Students  view  the  material  using  a standard  web  browser. 
Headsets  were  provided  in  order  to  permit  students  to  listen  to  lecture  audio. 

The  multimedia  presentations  were  classified  into  two  types:  topics,  and  key  words.  Topic 
presentations  contain  all  the  material  relevant  to  a concept  covered  in  the  lecture,  while  key  word  presentations 
are  very  short  and  concise,  designed  to  give  only  the  definition  of  the  term  in  question.  Students  search  for 
topics  using  an  index  based  on  the  lecture  outlines  just  as  they  would  search  the  table  of  contents  of  an  online 
book.  Key  word  indices  are  also  provided  for  each  lecture,  but  are  organized  in  alphabetical  order  instead  of 
the  order  in  which  the  material  was  covered  by  the  instructor.  Information  can  also  be  located  using  full  text 
search. 

4.  Implementation  Details 

4.1  Major  components 

The  course  content  resides  on  a computer  located  in  the  research  center’s  computing  facility  (Fig.  1). 
Media  files,  such  as  JPEG  and  MPEG  files,  are  managed  by  a streaming  media  server  (the  RealServer  from 
Real  Networks  Corp.)  while  HTML  files  are  handled  by  a standard  web  server  (Apache).  Students  access  the 
course  content  from  a computer  lab  on  campus.  A 100  Mb/s  Ethernet  connection  links  the  two  facilities.  In  the 
future,  access  will  be  extended  to  include  any  facility  served  by  the  Rutgers  University  Network  (RUNet  2000). 

A web  browser  configured  to  handle  streaming  of  compressed  video,  can  be  used  to  access  the  web 
site.  Two  plug-ins  are  required:  the  Real  Networks  RealPlayer  G2  plug-in,  and  the  “MPEG  by  Bitcasting” 


plug-in  - designed  specifically  for  streaming  MPEG-1  system  files  (files  containing  audio  and  video  data 
interleaved)  at  high  bit  rates. 

Management  of  the  web  server,  and  video  server  is  accomplished  using  Apache  Administrator  and 
RealAdministrator,  respectively.  These  are  web-based  interfaces  that  allow  configuration  and  control  of  the 
respective  servers  from  any  browser  capable  of  connecting  to  the  server  computer.  Using  these  interfaces, 
system  administrators  can  start  and  stop  the  servers,  set  bit  rate  limits,  enable  or  disable  access  control,  and 
collect  site  usage  statistics. 


Server  Network  Client 

Figure  1:  Major  Components  of  Aristotle 


4.2  Web  Site  Design 

When  students  access  the  web  site,  they  begin  with  the  Aristotle  Welcome  Page  which  contains  a brief 
description  of  the  project,  as  well  as  a link  to  the  Virtual  Classroom  page  which  lists  the  biology  lectures  that 
are  available  for  viewing.  Links  from  the  Virtual  Classroom  page  lead  to  HTML  pages  containing  topic  and 
key  word  indices  which,  in  turn,  lead  to  the  core  of  the  web  content  - the  multimedia  presentations  for  each 
topic  and  key  word  (Fig.  2). 

Topic  pages  contain  both  standard  links  and  embedded  objects.  On  the  left  side  of  the  page  are  links 

which: 

1 . lead  back  to  the  lecture  outline  and  key  word  index 

2.  connect  to  the  web  search  feature 

3 . lead  to  the  previous  topic  or  subsequent  topic. 

4.  generate  a pop-up  list  of  the  figures  associated  with  the  topic,  if  any  are  available 

Helpful  prompts  describe  each  link  when  the  mouse  arrow  is  placed  over  the  link  text.  The  remainder  of  the 
page  contains  the  multimedia  window,  which  is  split  into:  video  region  (view  of  professor  and  blackboard), 
transcript  region  (contains  scrolling  text),  and  illustrations  region  (contains  pop-up  figures  - also  contains  a link 
to  an  expanded  view  of  the  figure  which  opens  in  a separate  window).  Across  the  bottom  of  the  multimedia 
window  there  are  playback  controls  and  a playback  status  display. 

Key  word  pages  are  identical  to  topic  pages,  apart  from  the  fact  that  the  links  region  on  the  left  hand 
side  contains  a link  which,  when  selected,  activates  a pop-up  window  containing  a text  definition  of  the  key 
word  in  question). 

The  key  component  of  a multimedia  presentation  for  a topic  or  key  word  is  a file  containing  code 
written  in  the  Synchronized  Multimedia  Integration  Language  (SMIL).  SMIL  (Hoschka,  1998)  is  a markup 
language  for  multimedia  presentations  that  can  be  interpreted  by  compatible  media  players  (such  as  RealPlayer, 
in  this  instance).  Using  SMIL,  it  is  possible  to  define  screen  regions,  associate  media  objects  with  the  regions, 
and  synchronize  the  appearance  of  media  objects  (for  example,  to  define  when  figures  appear  and  disappear). 
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Figure  2:  Example  Topic  Page 


4.3  Search  Engine 

Students  can  access  a site-specific  text  search  engine  via  links  embedded  within  several  of  the  Virtual 
Classroom  pages.  To  search  for  a word  or  phrase,  without  using  lecture  outlines  or  key  word  indices,  they  can 
go  to  the  search  page,  type  in  their  query  and  obtain  a list  of  links  to  relevant  pages  within  the  web  site. 

4.4  The  Lecture  Production  Process 

The  Biology  course  comprises  25  lectures  amounting  to  over  36  hours  of  video  and  audio,  250  figures, 
and  approximately  150,000  words  of  transcripts  and  other  data.  The  web  site  contains  presentations  on  400 
topics,  and  1 800  key  words.  Clearly  the  process  of  generation  of  this  material  needed  to  be  as  streamlined  as 
possible.  Through  trial  and  error,  a reasonably  efficient  approach  was  adopted  (Fig.  3): 

1 . video  tape  lectures  in  studio  (Betacam  format) 

2.  edit  lectures  using  an  AVID  Express  non-linear  editor 

3.  encode  video  in  MPEG-1  format  using  a hardware  encoder  board  (Optibase  MovieMaker) 

4.  review  lecture  video  and  identify  time  codes  for  figures,  and  topic/key  word  boundaries 

5.  scan  (digitize  and  compress)  textbook  figures 

6.  generate  topic/key  word  files  (HTML,SMIL,  RealText) 

7.  generate  transcripts 

8.  transfer  content  to  server 
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Figure  3:  Lecture  Production  Process 

5.  DEPLOYMENT  AND  EVALUATION 

A study  is  now  in  progress  to  determine  the  effectiveness  of  the  Aristotle  system.  In  the  fall  of  1999, 
informal  anecdotal  feedback  was  obtained  from  a pilot  group  of  one  hundred  students  who  were  given  access  to 
the  web  site  during  its  development.  A more  formal  study  involving  a group  of  similar  size  is  planned  for  the 
summer  of  2000.  Final  grades,  a user  satisfaction  survey,  and  web  site  usage  statistics  will  be  used  to  measure 
the  quality  of  the  learning  experience.  Final  results  will  not  be  available  until  the  end  of  the  semester; 
however,  initial  feedback  has  been  very  encouraging.  In  the  meantime,  several  important  lessons  have  been 
learned  simply  through  the  process  of  developing  the  Aristotle  system,  which  help  to  point  the  way  for  further 
research.  These  lessons  and  proposed  enhancements  for  the  Aristotle  system  are  described  in  the  following 
section. 

6.  ONGOING  WORK 

Lecture  Transcription 

Currently,  the  process  of  generating  lecture  transcripts  is  by  far  the  most  time-consuming  and  labor- 
intensive  step  in  generation  of  course  content.  Transcripts,  however,  are  necessaiy  for  incorporating  full-text 
search  as  a means  of  locating  specific  information;  therefore,  future  generations  of  the  DLS  must  incorporate 
more  efficient  methods  for  their  generation.  Experiments  with  a commercial  speech  transcriber  indicate  that 
automatic  speech  recognition  is  not  yet  sufficiently  accurate  for  the  task.  One  reason  for  relatively  low 
accuracy  is  that  the  standard  language  model  is  not  well  suited  to  the  biology  domain.  Although  a 
significant  error  reduction  (40%  relative)  could  be  achieved  by  training  a biology-specific  language  model 
(using  a biology  textbook  containing  approximately  600,000  words  as  training  text),  the  amount  of  the  training 
data  was  not  sufficient  to  bring  the  recognition  error  rate  to  the  desired  level.  Analysis  of  the  transcription 
errors  suggests  that  a significant  benefit  could  be  realized  by  employing  long  range  language  models  that 
attempt  to  deduce  the  current  context  based  on  words  recently  spoken  and  using  the  results  to  condition  the 
predictions  of  the  short-range  trigram  models  that  are  currently  employed.  At  present,  our  research  is  focused 
on  investigation  of  such  techniques. 
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Multimedia  Authoring 

We  will  seek  to  develop  tools  and  procedures  to  make  content  preparation'  as  painless  as  possible.  As 
a first  step,  we  are  developing  multimedia  authoring  software  specifically  designed  for  systems  like  Aristotle. 
Where  standard  authoring  or  editing  tools  use  time  lines  and  frame-by-frame  playback  of  video  content  for 
segmentation  and  synchronization  of  media  content,  our  tool  will  be  based  on  lecture  transcripts.  The  author 
will  be  able  to  use  text  search  to  quickly  locate  relevant  sections  in  a video  clip,  then  will  employ  a “point-and- 
click”  interface  to  select  words  from  the  transcript  that  bracket  a topic  segment  or  figure  reference. 

To  ftirther  enhance  the  functionality  of  the  tool,  we  are  investigating  ways  to  automatically  determine 
the  location  of  topic  boundaries  within  the  transcript  text.  The  segmentation  algorithm  will  use  lecture  outlines 
and  passages  from  the  course  textbook  that  have  been  labeled  with  the  name  of  the  topic  discussed  therein. 
Using  labeled  passages  as  templates,  Local  Context  Analysis  (Xu  & Croft,  1996)  will  be  applied  to  a large 
corpus  of  unlabeled  biology  text  in  order  to  find  words  that  are  strongly  indicative  of  a given  topic.  Short 
segments  from  within  the  lecture  transcripts  can  then  be  categorized  by  topic,  and  systematically  clustered  into 
longer  segments,  based  on  the  presence  or  absence  of  such  characteristic  words. 

Question-Answer  Capability 

Future  versions  of  the  DLS  will  allow  students  to  find  information  by  asking  questions,  as  they  would 
of  a professor.  We  envision  a user  interface  in  which  the  student  interacts  with  an  animated  character  that  is 
capable  of  speaking  and  responding  to  voice  input.  Research  into  question-answering  techniques  is  being 
conducted  using  data  and  guidelines  developed  for  the  question-answering  track  of  the  National  Institute  of 
Standards  and  Technology  (NIST)  Text  Retrieval  Conference  (TREC).  The  initial  research  objective  is  to  find 
optimal  methods  for: 

1 . Question  analysis  with  a view  to  predicting  the  type  of  answer  required. 

2.  Full-text  passage  retrieval  (using  the  question  as  a query). 

3.  Analysis  of  the  retrieved  text  passages  to  select  the  most  relevant  given  the  question  and  answer  type 
prediction. 

Organizing  Content:  the  Knowledge  Web 

Our  future  Distributed  Learning  Systems  will  go  beyond  the  use  of  topics  and  key  words  for 
segmenting  and  organizing  information.  A new  data  entity  called  a Knowledge  Web  will  be  developed  based 
on  the  extensible  Markup  Language  (Bray,  et  al,  1998).  The  Knowledge  Web  will  define  Knowledge 
Containers,  and  the  relationships  that  link  them.  Currently,  fundamental  software  components  are  being 
developed  that  will  allow  experimentation  with  various  web  structures.  These  components  include  a graphical 
editor  for  creating  and  modifying  containers  and  links,  and  a hierarchical  visualization  tool  called  a Knowledge 
Map  for  viewing  and  traversal. 
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Abstract:  This  paper  is  a report  in  progress  on  a longitudinal  case  study  over  six  years  at  Apollo 
Parkways  Primary  School.  Apollo  Parkways  is  an  elementary  school  in  middle  to  lower-middle 
class  socioeconomic  suburb  in  Melbourne,  Victoria  Australia.  The  study  examines  the  changes  in 
formal  and  informal  conversations  as  the  school  evolved  from  a 'community  of  learners’  to  a 
'learning  community’  (Sergiovanni,  1994)  when  a technology  rich  environment  was  created.  The 
nature  of  the  conversations  is  providing  evidence,  which  supports  the  concept  that  conversation  can 
be  an  indicator  and  catalyst  of  change.  More  importantly  this  study  is  identifying  elements  in  the 
school,  which  have  supported  sustainable  change. 


In  today’s  society  greater  demands  are  being  placed  on  education  systems  at  all  levels  to  produce  citizens  who  can  use  knowledge 
in  new  domains  and  different  situations.  Members  of  society  at  every  level  are  being  asked  to  demonstrate  advanced  levels  of 
problem  solving  to  retain  their  utility  as  employees.  Learning  to  think  critically,  to  analyse,  to  synthesise  information,  to  solve 
problems  in  a variety  of  contexts  and  to  work  effectively  in  teams  are  crucial  skills  for  modem  employees,  and  yet  there  is  little 
evidence  that  our  education  systems  are  developing  these  skills  in  our  children  (Bransford,  Goldman  & Vye,  1991).  Some 
(Schank  & Cleary,  1995)  argue  strongly  that  our  educational  systems  are  not  developing  these  essential  skills.  The  Cognition  and 
Technology  Group  at  Vanderbilt  (CTGV,  1993)  cite  Whitehead’s  (1929)  notion  of  “inert”  as  opposed  to  “active”  knowledge  as  a 
consequence  of  conventional  approaches  to  schooling  and  teaching.  Berryman  (1991)  has  delineated  several  erroneous 
assumptions  on  which  much  of  the  teaching  and  schooling  process  is  based. 

Schools  now  faced  with  mandates  for  the  inftjsion  of  information  and  communication  technology  (ICT)  throughout  the 
curriculum  may  feel  they  are  at  a crossroad.  Technology  is  not  universally  seen  as  a panacea  for  current  educational  woes.  As 
long  ago  as  1984  Cuban  warned  “there  should  be  a page  in  the  Guinness  Book  of  Records  on  failed  classroom  reforms,  for  few 
ever  seem  to  have  been  incorporated  into  teachers’  repertoires”.  Like  past  revolutions  in  education,  it  will  go  the  way  of  previous 
technologies  unless  there  are  changes  to  the  educative  framework. 

In  Australia,  similar  concerns  were  raised  in  discussions  with  senior  departmental  officers.  In  March  1 99 1 Dr  Terry  Burke, 

Deputy  Director-General,  NSW  Department  of  Education  and  Training  raised  serious  issues  regarding  the  implementation  and 
long  term  viability  of  the  Technology  High  School  program  (18  months  into  the  program).  In  November  1 996,  Mr  Phillip  Arthur, 
Manager,  Schools  of  the  Future,  Victorian  Department  of  Education  emphasised  the  need  to  model  the  Navigator  School  program 
on  key  findings  from  recent  studies  into  the  use  of  technology  in  teaching  and  learning. 

In  many  similar  programs,  managers  and  administrators  have  simply  said,  “Pretty  simple,  just  train  them  (the  teachers)  to  use  the 
computers  - as  experts  in  curriculum  and  instruction,  the  teachers  will  do  the  rest”.  In  his  critical  review  of  computers  in  schools 
and  universities,  Bork  (1995)  argued  that  the  effective  use  of  new  instructional  paradigms  required:  "a  shift  in  teachers’ 
pedagogical  approaches  and  software  that  supports  modes  of  instruction  with  alternative  frameworks".  The  failure  of  many 
programs  to  bring  about  sustainable  change  in  the  classroom  may  have  been  linked  to  ignorance  of  the  need  to  change 
pedagogical  approaches,  and  with  them,  the  nature  of  interactions  and  conversations  among  teachers. 

This  paper  outlines  a study  in  progress  of  what  occurred  within  one  school  as  it  traversed  the  crossroad  ~ it  is  a study  of  whole 
school  change  through  the  integration  of  technology,  using  a model  based  on  a decade  of  ACOT  (Apple  Classrooms  of 
Tomorrow)  research.  The  implementation  model  began  with  the  key  findings  of  twenty-two  published  ACOT  reports,  and  the 
venue  was  Apollo  Parkways  Primary  School. 
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The  Setting 

Apollo  Parkways  Primary  (Elementary)  School  is  a government  school  located  in  the  north  east  outer  suburbs  of  Melbourne, 
Victoria  Australia.  Socioeconomic  area  -middle  class  Australia.  Prior  to  1995,  technology  in  the  school  consisted  of  one  or  two 
computers  in  some  classrooms.  In  1995  the  school,  with  support  from  the  local  community,  installed  a single  computer 
laboratory,  increasing  the  total  number  by  16  computers.  In  October  1 995,  the  school  was  named  'Navigator  School  of  the  Future' 
by  the  Victorian  Department  of  Education.  In  August  1996,  it  was  also  announced  as  the  eighth  ACOT  site  in  the  world.  As  part 
of  the  ACOT  and  Navigator  school  programs,  in  July  1996,  the  school  received  over  200  computers,  file  servers,  network  and 
Internet  connections.  The  computer  student  ratio  had  decreased  from  1 :30  to  1:3.  During  this  time,  the  school  population 
continued  to  grow  by  5-10%  per  annum.  Currently,  there  are  nearly  700  pupils,  34.2  teaching  staff,  3 executive  staff  and  4.2 
administrative  and  support  staff 

The  challenge  seemed  simple  - share  key  ACOT  research  findings  (Fisher,  Dwyer,  & Yocam,  1 996;  Baker,  Gearhart  & Herman, 
1993;  Ferguson-Smith,  1998)  with  staff,  give  students  and  teachers  immediate  access  to  computers,  show  them  how  to  use  the 
technology,  and  introduce  them  to  open-ended  software. 

The  Study 

From  a research  perspective,  the  key  question  was  how  to  capture  the  essence  of  what  occurred  in  some  meaningful  way  so 
others  could  benefit.  The  formal  and  informal  conversations  within  a school  were  initially  considered  in  their  capacity  as  an 
indicator.  Lambert’s  (1 995)  'typology  of  conversation’  (dialogic,  inquiring,  sustaining  and  partnering)  in  a constructivist 
leadership  environment  suggested  a vital  role  for  conversations  in  the  change  process,  thus  elevating  the  status  of  conversation 
from  indicator  to  focus. 

The  purpose  of  the  study  is  to  examine  the  change  in  conversation  at  the  school  before,  during  and  after  the  introduction  of 
technology  rich  classroom  learning  environments.  It  addresses  three  main  questions: 

1 ) How  has  the  conversation  changed? 

2)  How  has  the  classroom  practice  changed? 

3)  How  have  teachers’  views  of  learning  changed? 

A qualitative  approach  to  the  research  is  followed  in  this  ethnographic  study  - the  researcher  is  a participant  observer  in  a 
naturalistic  paradigm  (Lincoln  and  Cuba,  1985)  with  an  interpretivist  goal  (Willis,  Thompson  & Sadera,  1999).  A variety  of 
techniques  have  been  employed  for  data  collection  and  analysis,  informed  by  researchers  such  as  Dwyer  (1 985)  - understanding 
stories,  and  Beittel  (1973)  - using  artefacts  and  observation.  Data  sources  have  included  official  documentation,  observations, 
unstructured  discussions,  interviews,  questionnaires  and  personal  diaries. 

The  analysis  of  all  the  school  official  documents  and  publications  including  daily  bulletins,  newsletters,  annual  magazine,  the 
web  site,  teacher  journals,  student  portfolios,  student  observations  and  reports  involved  identification  then  tallying  of  themes 
evident  in  these  formal  conversations. 

The  identified  themes  have  generated  questions  to  be  included  in  interviews,  questionnaires  and  unstructured  discussions  with 
community  members  including  the  school  executive,  teaching  staff,  administrative  and  support  staff,  the  students  and  parents, 
visiting  educators  and  participants  of  the  different  professional  development  programs. 

School  Change 

During  the  first  visit  to  the  school,  at  the  after  school  staff  meeting,  the  vision  of  ACOT  and  the  key  research  finding  were  shared. 
An  underlying  assumption  was  that  the  findings  would  translate  from  the  American  experience  to  an  Australian  school.  Over  the 
next  twelve  months,  each  of  the  35  staff  members  participated  in  3000  hours  of  professional  development.  These  activities 
covered  a wide  range  of  topics.  The  exercise  provided  clear  support  for  many  key  research  findings,  such  as  stages  of  adoption 
(Hall,  1997),  the  nature  of  learning  (Roblyer,  Edwards  & Havriluk,  1997),  constructivism  (Cobb,  1994;  Fosnot,  1996)  and 
reflective  practice. 

Working  in  collaboration  with  the  school  executive  and  project  coordinator,  key  issues  of  professional  development  (Hargreaves 
& FuIIan,  1992;  McLaughlin  & Marsh,  1983)  were  revisited.  These  discussions  soon  highlighted  many  of  the  issues  and  concerns 
associated  with  school  change  (Hargreaves,  1994;  Fullan  & Hargreaves,  1996;  Schlechty,  1997,  Stoll  & Fink,  1996;  and 
Sandholtz,  Ringstaff,  & Dwyer,  1996).  Reorientation  towards  constructivist  learning  environments  (Cobb,  1994;  Fisher,  Dwyer, 

& Yocam,  1996;  Fosnot,  1996;  and  Lambert,  1995)  coupled  with  adjustments  to  the  physical,  technical  and  social  ramifications 
of  increased  technology  created  a complex  pattern  akin  to  Rogers'  (1995)  diffusion  of  innovation  model. 

With  a developing  awareness,  understanding  and  appreciation  of  key  research  findings  from  several  technology  based  programs 
around  the  world  (ACOT  in  the  USA  and  River  Oaks  School  in  Canada)  Apollo  Parkways  implemented  support  structures  to 
ensure  a successful  and  sustainable  experience  which  could  improve  student  outcomes  and  learning. 

The  successes  and  challenges  faced  by  the  teachers  in  the  altered  instructional  settings  resulted  in  a mix  of  readiness  by 
individuals  to  accept  and  implement  technology.  Their  willingness  to  accept  and  adopt  change  is  described  by  Rogers’  (1995) 
adoption  and  diffusion  of  innovation,  "from  first  knowledge  of  an  innovation  to  forming  an  attitude  toward  the  innovation,  to  a 
decision  to  adopt  or  reject,  to  implementation  of  the  new  idea,  and  to  confirmation  of  this  decision".  Researchers  studying  teacher 
development  have  found  that  all  teachers  who  are  implementing  educational  innovations  progress  through  a series  of  stages: 
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survival,  mastery,  and  impact  (Stoll  & Dean,  1996;  Hall  & Loucks,  1979).  According  to  Renyi  (1996),  teachers  make  or  break  a 
school: 


‘‘To  improve  schools  we  must  focus  on  the  teachers.  Schools  can  only  be  as  good  as  the  teachers  in  them.  This  is 
something  that  all  other  so-called  'reform  efforts'  have  missed.  It's  what  teachers  know  and  can  do  that  will  make 
the  difference  in  improved  student  performance''  (http://www.nfie.org/takechar.htm) 

Preliminary  Data  Analysis 

Intuitive  analysis  of  the  formal  publications  have  so  far  identified  some  key  sub-questions  that  will  be  used  to  build  the  interview 
and  questionnaire  protocols: 

• How  did  you  implement  whole  school  change? 

• What  protocols  (strategies  or  formal  elements)  have  been  employed? 

• What  were  their  frequencies? 

• Were  they  subjective  or  objective?  Why? 

• Were  there  any  hidden  messages?  If  so,  how  did  they  involve  forward  planning  and  reflective  practice? 

• Who  did  it  impact? 

Intuitive  analysis  of  the  informal  themes  of  conversation  identified  the  following  key  concepts: 

• Spontaneous 

• Subjective 

• Circumstantial 

• Current  or  immediate  future  or  past  (here  and  now) 

• Formative 

• Focused  on  the  individual  perspective 

The  complete  data  set  for  the  formal  conversations  is  too  voluminous  to  interpret  in  its  entirety,  so  sampling  methods  have  been 
applied  to  certain  documents.  Bulletins  have  been  sampled  from  the  months  of  Term  1 - January,  February  and  March  each  year. 
Table  1,  provides  the  analysis  framework  for  each  publication  type.  This  involves  a 3-D  graph  for  each  year  (1994-2000),  with 
the  following  axes; 

X axis  = dimensions  of  the  publications 
y axis  = characteristics  of  the  formal  conversation 

z axis  = qualitative  assessment  of  how  these  have  changed  over  time  (judgement  / depth)  or  rate  of  change  of  the 
changes  (using  a rating  scale  1-5,  where  1 is  the  lowest) 


Table  1 : Depth  of  assessment  of  the  characteristic  and  dimension  of  conversation 
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The  links  between  the  dimension,  characteristics  and  depth,  from  Table  1,  will  help  formulate  the  themes  and  provide  the  basis 
for  some  of  the  questions  to  be  raised  in  the  interviews  and  questionnaires.  Teacher  interviews  will  be  conducted  with  5 to  6 
teachers  who  have  been  with  the  school  throughout  the  study,  and  3 to  4 teachers  who  were  new  to  the  environment  in  1999  and 
2000.  Interviews  with  other  key  staff  include  members  of  the  executive,  parent  and  citizen  association,  school  council  and  a small 
sample  of  visitors. 

Interview  questions  will  originate  from  four  sources: 

• Analysis  of  data  (above)  - to  triangulate  findings  from  analysis  of  written  documents 

• Theory  - to  test  the  teachers’  understanding  of  certain  theory,  and  to  validate  patterns  suggested  by  research  literature 

• The  structural  framework  for  data  analysis  - to  identify  key  features  of  formal  and  informal  conversations  across  several 
pre-specified  themes  (see  below). 

• Intuitive  ideas  of  the  researcher  who  participated  in  the  design  of  professional  development  experiences  for  staff  within  the 
time  frame  of  1996  - 1999. 

Table  2,  outlines  some  pre-specified  arbitrary  categories  that  have  been  chosen  as  a starting  point  to  identify  patterns  and  trends 
in  data.  The  preliminary  results  from  the  analysis  of  formal  and  informal  conversation  before  and  after  the  introduction  of 
technology  in  the  school  is  providing  evidence  that  supports  the  conversation  as  an  indicator  and  catalyst  of  change. 


Table  2 : Initial  framework  for  data  analysis 


Themes  pre-specified  to  begin 
data  analysis 

Formal  Aspects 

Informal  Aspects 

School  organisation 

What  changed  formally  in  the  school 
organisation?  What  led  to  these 
changes? 

How  much  did  teachers  initiate  changes  to  school 
organisation  on  an  informal  basis?  What  sort  of 
changes  made  a difference  within  individual 
classrooms? 

Pedagogy 

What  formal  changes  were  made  to 
curriculum  planning  and  integration  of 
ICT? 

Who  did  teachers  talk  to  on  an  informal  basis? 
How  were  these  critical  friends  found?  Did  this 
increase  over  time? 

Staff  Skill  acquisition 

What  PD  activities  were  run  formally 

How  much  did  teachers  learn  informally  from 
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for  skill  acquisition  and  what  software 
was  used? 

peers  and  others? 

Technology  diffusion  (hardware) 

How  was  hardware  formally  distributed 
for  the  duration  of  the  study? 

Did  teachers  attempt  to  use  hardware  out  of 
school  time?  Have  staff  members  purchased  their 
own  equipment  for  home  use? 

Conversations 

What  themes  can  be  identified  in  the 
formal  conversations  monitored  in  the 
school? 

What  themes  can  be  identified  in  the  informal 
conversations  monitored  in  the  school? 

Researcher  as  PD  Designer 

What  programs  did  the  researcher 
design? 

What  was  the  nature  of  the  researcher’s  informal 
input  to  staff  members? 

The  underlying  reality  that  is  being  sampled:  What  happened  at  Apollo  Parkways  Primary  School  between  1996  and 

2000. 

The  school,  during  1996/7  implemented  several  guiding  principles,  highlighted  by  recent  research  about  learning  and  technology. 
These  principles  can  be  summaries  by  the  following: 


• Provide  continued  and  extensive  professional  development  that  was  situated  in  the  contexts  of  practice. 

• Adopt  a constructivist  learning  approach  in  the  classroom 

• Create  learner  centred  and  technology  rich  learning  environments 

• Engage  teachers  from  the  school  in  an  ongoing  conversation  and  reflection  about  their  practice,  their  students,  theories 
of  learning,  how  they  learn,  alternative  forms  of  assessment,  the  appropriation  and  integration  of  technology  and  what 
they  would  change  or  not  change  in  their  classrooms. 

• Teachers  in  teams,  develop  integrated  units  of  work,  lesson  plans  and  other  learning  tasks  that  were  cross  curricula  used 
technology. 

Conclusion 

There  is  evidence  to  suggest  that  a new  way  of  thinking  about  change  and  technology  is  required,  based  on  a thorough  knowledge 
of  the  processes  and  the  people,  blending  research  and  experiential  knowledge,  Fullan  (1991).  Effective  change  management  is 
facilitated  by  supportive  management  structures.  The  constant  communication  of  outcomes  and  results  is  essential  in  order  to 
maintain  support  and  it  is  advisable  to  keep  personal  political  agendas  transparent. 

This  study  may  provide  some  insight  into  the  key  elements  contributing  to  effective  change  management.  The  question  is,  do 
successful  agents  of  change  have  a characteristic  stance,  a capacity  for  creating  personal  vision,  inquiry,  technology  mastery  and 
collaboration  in  addition  to  a sizeable  'repertoire  of  procedural  knowledge'? 

The  implications  of  this  whole  school  study  can  provide  important  lessons  and  strategies  for  administrators,  educators,  schools, 
districts  and  education  systems  in  an  Australian  setting.  Those  at  the  crossroads  would  do  well  to  listen  to  the  conversations  of 
those  who  have  successfully  traversed  the  terrain. 
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Abstract:  In  the  past  years  there  has  been  a tremendous  movement  in  the  area  of 
teleteaching/teleleaming.  Many  institutions  try  to  find  solutions  for  distance  learning  that  could  be 
used  especially  with  the  Internet.  Today’s  solutions  cover  synchronous  scenarios  like  teaching  using 
videoconference  systems  and  asynchronous  scenarios  like  putting  teaching  material  on  the  web. 
JaTeK  (Java  based  Teleteaching  Kit)  is  a system  for  both  teleleaming  and  teleteaching. 
Teleteaching,  for  us,  means  composing  teaching  material  and  using  tools  for  self  learning,  seminars 
as  well  as  synchronous  and  asynchronous  communication  and  cooperation.  JaTeK  is  completely 
implemented  in  Java  and  consists  of  client  components  and  the  JaTeK  servers  managing  material, 
users,  groups,  and  permissions  as  well  as  communication  and  evaluation).  First  experiences  with 
JaTeK  have  shown  that  there  is  strong  demand  for  the  system  on  offline  media.  This  article 
describes  the  realized  approach  of  a hybrid  version  of  JaTeK  that  solves  the  problems  related  with 
its  availability  off-line. 


Introduction  to  JaTeK 

JaTeK  (Java  Based  Teleteaching  Kit)  is  a three-tier  client/server  system  with  an  underlying  object  oriented  database 
(see  fig.l).  As  both  JaTeK  and  the  DBMS  are  completely  implemented  in  Java,  the  system  is  applicable  on  all 
platforms  with  a Java  virtual  machine  installed. 


Figure  1.  JaTeK  Architecture 

Application  data  are  managed  by  ObjectStore  PSE,  an  object  oriented  DBMS  completely  implemented  in  Java  that 
makes  its  functionality  available  via  the  database  API.  The  JaTeK  server  provides  high  level  functions  to  create  and 
login  users  as  well  as  to  create,  write,  read  and  delete  objects  implementing  the  server  API.  The  JaTeK  clients  allow 
to  display  and  administer  the  teaching  content  (e.g.  documents,  exercises,  and  experiments),  and  encapsulate  the 
JaTeK  server  interface.  They  consist  of  the  actual  applications  realizing  the  graphical  user  interface  and  the  data 
objects  representing  the  teaching  contents  that  will  be  read  from  the  server. 

When  a user  logs  into  the  server-based  system,  the  client  application  gets  a reference  to  a user-specific  server  object 
via  RMI  naming  service.  That  manages  all  subsequent  user  requests  to  data  objects  on  the  server.  The  data  objects 
are  read  from  respectively  written  to  the  database  by  the  JaTeK  server.  The  teaching  content  is  implemented  in  form 
of  different  material  templates  that  use  methods  of  the  JaTeK  API  to  communicate  with  the  client  application  or 
with  the  server,  respectively  (e.g.  for  receiving  or  saving  material  data). 

For  the  hybrid  system  to  work,  JaTeK’s  functionality  had  to  be  enhanced  by  a new  component  responsible  for 
controlling  the  hybrid  mechanism,  storing  of  data  in  the  local  database,  and  data  equalization  (see  fig  2).  The  hybrid 
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control  supplies  the  application  with  data  objects  by  loading  them  from  either  the  local  database  or  the  remote 
server,  depending  on  the  current  application  mode.  Communication  services  that  are  part  of  the  JaTeK  system  do  not 
access  the  local  database.  Only  if  the  remote  JaTeK  server  is  reachable  their  data  are  handed  over. 


Scenarios  of  the  hybrid  approach 

Originally,  the  JaTeK  system  was  required  to  be  available  over  the  Internet.  However,  our  experiences  and  the 
results  of  evaluation  cycles  have  shown  that  more  flexibility  regarding  Internet  connectivity  is  needed.  To  meet  the 
requirements  of  a wide  ranged  usage  the  hybrid  system  has  to  cover  the  following  scenarios: 

1 . Usage  as  non-hybrid  system  via  HTTP/RMI-server  without  client-side  storing 

An  educational  institute  offers  teaching  material  on  a server  that  students  may  access  over  the  Internet.  Mostly 
they  use  the  JaTeK  clients  from  within  a university  computer  laboratory,  so  a fast  network  connection  can  be 
assumed.  All  data  - that  are  kept  in  a database  on  the  server  - may  be  transferred  via  the  network, 

2.  Local  usage  without  communication-  and  write-requests 

A student  without  network  access  can  use  the  standalone  version  of  JaTeK.  All  data  are  stored  locally  and  the 
system  will  be  started  from  CD-ROM  or  local  hard  disc.  Since  CD-ROMs  are  read-only  media,  in  this  case  just 
read  operations  can  be  executed.  For  write  operations  a medium  that  can  be  written  to,  e.g.  a hard  disk,  is 
required. 

3.  Hybrid  usage 

A user  with  but  a slow  network  connection  would  still  want  to  access  the  most  recent  data.  Using  the  system  in 
the  hybrid  modus,  the  user  would  transmit  as  few  data  as  possible  via  the  network  - thus  minimizing  network 
traffic  and  delays. 

In  this  scenario,  the  system  is  installed  locally  on  the  hard  disk.  For  regular  work,  only  data  locally  stored  are 
accessed.  They  will  be  updated  only  if  the  data  on  the  server  have  been  changed  since  the  last  equalization. 
Write  requests  are  performed  on  the  server  and  on  local  data. 

4.  Data  equalization 

Normally,  if  a local  database  is  used  it  is  not  necessary  to  hold  a network  connection  alive  all  the  time  one 
works  with  JaTeK.  For  instance,  course  material  could  be  created  in  the  standalone  modus.  When  the  work  is 
finished,  the  server  database  can  be  updated  with  that  material. 

This  scenario  supplements  that  of  local  usage  without  communication  and  write  requests. 
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The  scenarios  described  above  are  summarized  in  table  1 . The  need  of  connecting  the  server  in  one  of  the  scenarios 
implies  to  respect  its  user  and  permission  management.  The  permissions  have  to  be  applied  to  the  local  database  in 
order  to  avoid  inconsistencies  between  the  server  and  client  databases. 


Scenario 

Database  access 

Data  equalization 

1 . Usage  as  non-hybrid  client/server  system 

server 

No 

2.  Standalone 

local 

No 

3.  Hybrid 

hybrid 

Yes 

4.  Equalization 

hybrid 

Yes 

Table  1.  Summary  of  the  scenarios  covered  by  the  hybrid  mechanism 


The  form  of  database  access  inevitably  derives  from  the  scenario  currently  used.  Exclusively  in  scenario  3 and  4,  the 
databases  are  equalized  with  each  other. 

System  requirements  of  the  hybrid  approach 

A system  that  covers  all  of  the  scenarios  described  above  has  to  meet  the  following  requirements: 

* Work  as  "pure"  client  server  system 

The  system  should  be  able  to  deactivate  the  hybrid  functionality  and  to  switch  to  a mode  not  using  this 
additional  functionality.  That  means  the  system  must  be  workable  without  a local  database. 

* Data  equalization  in  both  directions  client  server  and  server  client 

Usually,  only  client  data  will  be  updated  with  newer  server  data.  However,  if  local  data  are  changed  in  the 
standalone  mode,  these  changes  will  have  to  be  processed  in  the  server  database,  too. 

* Functioning  without  a server 

To  use  the  system  without  network  access,  it  has  to  be  able  to  work  in  standalone  mode  with  just  the  client 
database.  Therefore,  all  necessary  components  (i.e.  database,  class  files,  etc.)  have  to  reside  locally,  without 
network  based  communication  services  being  available. 

* Keeping  resource  consumption  low 

Since  the  client  computers  running  the  hybrid  component  usually  have  fewer  resources  than  the  server 
computer,  the  hybrid  component  must  not  consume  resources  excessively. 

* Automatic/manual  start  of  the  full  data  equalization 

Full  data  equalization  can  be  run  automatically  at  the  start  and/or  at  the  end  of  a session,  or  be  started  manually 
to  keep  the  entire  database  up  to  date  while  working  in  standalone  mode. 

* Data  equalization  on  demand 

If  the  client  is  working  online,  local  data  will  not  be  equalized  until  they  are  accessed  by  the  application.  Thus 
several  small  data  transfers  are  performed,  consuming  less  time  than  one  big  transfer. 

* Transparent  data  access 

Accesses  to  the  local  and  the  remote  database  have  to  be  realized  transparently  to  a large  extent.  For  the  user 
and  the  application,  there  should  be  no  difference  between  a query  to  the  local  or  the  remote  database.  Only  the 
hybrid  component  has  to  know  where  the  needed  data  are  located  and  use  this  information. 

* No  separate  application  for  data  equalization 

In  order  to  keep  the  user  interface  simple  the  hybrid  functionality  is  integrated  into  the  existing  client  software. 
No  external  application  is  needed  for  data  equalization. 
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Database  access  in  the  hybrid  system 

Different  possibilities  of  access  have  to  be  considered  to  determine  the  form  of  database  access: 

* Database  access: 

* direct  via  the  database  API 

* via  a server  process 

* Database  Access  by 

* a Single  user 

* multiple  users  concurrently 

* Using  a database 

* on  the  local  computer 

* on  a remote  computer 

Since  the  used  DBMS  does  not  support  a remote  use  of  the  database  API,  it  is  impossible  to  directly  access  a remote 
database  API  (e.g.  via  RMI).  Neither  is  accessing  a local  database  via  a server  process  effective  as  this  approach 
consumes  resources  and  brings  no  advantages  in  the  context  of  the  system.  The  scenarios  best  applicable  in  the 
hybrid  system  are,  therefor,  the  use  of  a local  database  directly  via  its  API  for  accessing  cached  data,  and  of  a remote 
database  via  a server  process  for  updating  data. 


Access 

Local 

Remote 

Single  user 

Multi  user 

Direct  via  database  API 

Local  solution 

Concurrency 

problems 

Impossible 

Via  server  process 

Incommensurate 

Server  solution 

Table  2.  Scenarios  of  database  access 


As  the  server  process  is  capable  of  handling  multiple  users,  remote  access  by  multiple  users  has  not  to  be  regarded 
separately.  A direct  access  to  the  database  API  in  multi  user  mode  could  result  in  the  database  being  opened  by 
multiple  processes.  That,  in  turn,  could  lead  to  undefined  database  behavior  since  JaTeK  uses  the  DBMS 
ObjectStore  PSE,  which  does  not  provide  cross-process  concurrency  control  mechanisms. 

Therefore,  the  hybrid  system  uses  the  database  API  to  access  the  local  database  directly.  The  multi  user  server 
process  is  used  for  accessing  the  remote  server  database  which  in  this  case  is  shared  with  other  clients. 

Assigning  client  objects  to  server  objects 

For  equalizing  the  objects  of  the  local  client  and  the  remote  server  database  in  the  hybrid  system,  corresponding 
objects  have  to  be  assigned.  To  unambiguously  assign  an  object  of  the  server  database  to  one  of  the  client  database  a 
unique  external  link  must  exist.  Equivalent  objects  reside  at  the  same  position  in  the  object  tree  of  their  databases. 
Client  and  server  objects  are  thus  identifiable  database-independently  by  the  object  path  - indicated  by  the  names  of 
the  nodes  - from  the  root  to  that  object.  To  find  the  corresponding  object  one  has  to  navigate  along  the  path  from  the 
root  to  the  object. 

Another  approach  would  be  to  use  unambiguous  identifiers  - “external  references”  - for  persistent  objects  in  the 
server  database.  They  contain  information  about  the  database,  the  segment  in  which  an  object  is  stored,  and  the 
location  of  that  object  in  a segment.  The  problem,  here,  is  that  segment  and  location  information  are  not  identical  for 
objects  of  the  client  and  the  server  databases.  Storing  the  external  reference  of  the  server  objects  together  with  the 
object  data  of  the  client  database  could  solve  this. 
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Path 

External  reference 

Advantages: 

* Database  independent 

* Fast  access  to  server  database  objects 

* Only  the  object  and  perhaps  the  parent 
object  have  to  be  known 

Disadvantages: 

* Navigation  through  whole  path 
necessary 

* Whole  path  has  to  be  known 

* Problems  after  renaming  nodes 

* Database  dependent 

* May  vary  after  reorganizing  database 

Table  3.  Comparison  of  methods  to  find  equivalent  objects  in  the  client  and  the  server  databases 


The  two  possibilities  to  allocate  objects  in  the  client  and  server  databases  considered  here  are  compared  in  table  3.  If 
the  server  database  will  never  be  reorganized,  both  segment  and  location  of  objects  of  the  server  database  remain  the 
same  during  the  entire  life  span  of  the  persistent  server  object.  Thus,  corresponding  objects  can  be  easily  allocated 
and  identified  by  the  external  reference  of  persistent  server  objects.  As  this  is  faster  and  more  reliable  than 
identifying  objects  by  path,  this  method  is  used  in  the  hybrid  system. 

Timestamps 

Another  important  aspect  of  equalizing  data  is  the  method  of  determining  their  age.  The  local  data  will  have  to  be 
updated  if  comparing  the  timestamps  of  local  and  remote  data  shows  that  the  server  data  are  newer. 

For  that  reason  every  object  (in  the  server  and  client  databases)  contains  a timestamp  representing  the  time  of  its  last 
modification.  If  the  application  changes  an  attribute,  the  timestamp  will  be  set  to  the  current  time.  As  the  real  time 
clocks  of  the  clients  are  not  synchronized,  they  may  differ.  Therefor,  timestamps  are  not  created  by  each  client, 
individually,  but  by  the  server.  This  timestamp  is  sent  to  the  client  during  data  equalization  and  applied  to  the 
according  object. 

The  consideration  of  using  timestamps  led  to  the  discussion  of  three  different  approaches: 

1 . To  have  only  one  timestamp  per  object  would  lead  to  the  whole  object  tree  having  to  be  searched  by  comparing 
the  timestamps  of  every  client  object  with  that  of  the  corresponding  server  object.  Especially  for  full  data 
equalization,  that  consumes  too  much  time. 

2.  To  reduce  network  traffic  and  dispensable  operations,  objects  have  collective  timestamps  (called  tschiid) 
summarizing  the  timestamps  of  the  child  objects  reachable  by  the  current  object.  The  collective  timestamp 
shows,  for  server  objects,  the  timestamp  of  the  youngest  and,  for  client  objects,  the  timestamp  of  the  oldest  child 
object.  If  the  collective  timestamps  of  the  server  object  is  not  younger  than  that  of  the  corresponding  client 
object,  no  data  equalization  of  the  child  objects  is  necessary.  If  this  is  valid  for  the  root  object,  the  client 
database  is  up  to  date  and  no  data  equalization  is  needed. 

If  one  object  in  the  tree  was  modified,  the  collective  timestamp  has  to  be  changed  up  to  the  root.  Although  this 
approach  offers  a better  basis  for  optimizing  data  equalization,  it  does  not  produce  enough  information  to 
unambiguously  recognize  modifications.  That  led  to  the  following  approach: 

3.  To  avoid  having  to  compare  all  nodes  of  the  path  another  timestamp  (tsscir)  - representing  the  time  of  the  last 
modification  of  the  current  object  - is  introduced.  For  instance,  if  a Material  object  has  been  changed  or  added, 
tsscif  of  this  object  and  tschiid  of  all  parent  objects  on  the  server  are  set  to  the  current  time.  If  a Material  object 
has  been  deleted,  only  tschiid  of  all  former  parent  objects  are  set.  For  a better  handling  collective  timestamps  are 
divided  into  one  for  eveiy  type  of  an  object’s  children. 

Timestamps  are  not  used  for  data  equalization  on  the  server.  Only  in  hybrid  mode,  all  local  changes  will  be  put 
directly  to  the  server.  Otherwise  all  local  changes  that  cannot  be  performed  immediately  on  the  server  (e.g.  client  is 
started  in  standalone  mode)  are  recorded  to  be  processed  when  the  server  equalization  is  started. 
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Instead  of  timestamps  for  client  equalization  changes  may  be  recorded  by  the  server  and  sent  to  the  clients.  But  there 
is  an  indefinite  number  of  clients  whose  local  databases  could  differ  considerably,  if  they  exist  at  all.  Trying  to  keep 
track  of  changes  by  recording  them  would  therefor  mean  to  store  records  of  all  changes  since  the  creation  of  the 
database  for  an  unlimited  period.  As  this  is  inexpedient,  the  method  of  timestamps  is  used  for  updating  the  clients  so 
they  can  create  (if  no  local  database  exists  yet)  and  update  their  local  database  by  requesting  just  the  needed  server 
objects  and  comparing  timestamps. 

Outlook 

The  hybrid  JaTeK  system  has  been  integrated  in  the  real  learning  environments.  It  now  has  to  prove  its  stability  and 
usability.  Until  now  the  hybrid  system  can  only  be  administered  by  editing  a property  file  (e.g,  naming  the  local 
database  and  determining  the  mode,  i.e.  local,  hybrid,  or  server  only).  However,  a more  convenient  graphical  user 
interface  will  be  implemented  and  embedded  suitably  into  the  user  interface  of  the  clients. 

While  work  at  the  project  is  going  on  not  only  course  data  but  also  Java  classes  are  subject  to  modification,  A 
mechanism  for  updating  Java  class  files  would  ensure  that  all  client  components  are  up  to  date. 

Some  data  used  by  teaching  content  has  to  be  accessed  via  the  network  - even  if  the  client  database  is  up  to  date.  In  a 
future  version,  these  data  must  be  stored  in  the  local  database  to  be  available  in  the  standalone  mode,  too. 
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Abstract:  Development  and  implementation  of  systems  for  online  learning  is  a major  task  for 
institutions.  Policy  must  be  based  on  a sound  pedagogical  knowledge  and  supported  by  strategic 
decisions  at  different  levels.  This  paper  examines  two  studies  considering  organizational  strategies 
for  IT  policy.  These  views  are  related  to  the  experiences  of  the  individuals  in  a collaborative 
development  of  online  learning  materials.  This  development  has  simultaneously  informed  the 
design  of  a generic  architecture  for  customizable  online  learning.  Projects  such  as  this  are  the 
direct  outcome  of  University  funding  policy.  This  is  high  cost^igh  risk  approach  not  suited  to 
every  institution  and  it  is  therefore  essential  to  develop  understanding  of  the  relationship  between 
policy,  project  and  participants  in  order  to  maximize  benefits  to  the  institution.  It  is  proposed  that 
grounded  analysis  of  representative  projects  can  be  used  to  more  fully  inform  strategic  thinking  by 
management,  academics  and  developers. 


Background 

Transformation  of  curricula  into  more  flexible  forms  is  a critical  process  for  institutions  in  an  increasingly 
competitive  education  market.  Strategies  adopted  must  balance  pedagogical  needs  with  rapidly  changing 
technologies  and  should  reflect  the  culture  of  the  institution.  The  Multimedia  Education  Unit  (MEU)  works  in 
conjunction  with  faculty  units  in  providing  educational  design,  evaluation,  media  production,  programming  and  staff 
development  services  to  faculties.  From  this  position,  MEU  is  well  placed  to  identify  and  promote  strategic 
approaches  to  educational  technology  in  the  institution.  The  operational  strategies  to  bring  about  centralized 
teaching  and  learning  systems  will  impact  at  different  levels  in  the  university.  For  the  purposes  of  this  paper,  these 
levels  involve  the  organizational  'climate'  created  by  strategic  management  counterpoised  against  the  working 
experience  of  teachers  at  the  educational  coal  face,  and  somewhere  between,  a supporting  technical  infrastructure 
that  ties  together  both  sets  of  requirements.  This  paper  contends  that  the  complexity  of  decision  making  associated 
with  institutional  approaches  to  online  earning  environments  requires  active  consideration  and  reflection  on  the 
interaction  between  top  down  and  bottom  up  processes.  In  attempting  to  do  this,  the  paper  examines  a model  for 
strategic  management  of  change  in  IT  (Yetton  1997),  its  extension  to  a more  functional  level  guide  for  online 
systems  (Reid  1999)  and  relates  these  views  to  the  actual  experience  of  ‘producers’  engaged  in  development  and 
teaching.  The  case  examines  one  project,  the  direct  outcome  of  central  funding  strategies,  from  the  perspectives  of 
the  developer  of  scalable  online  learning  systems  and  teacher  creating  innovative  learning  environments. 


Models  for  Managing  IT-based  Strategic  Change 

U The  EIP  Study 
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A study  of  the  strategic  approaches  to  IT  in  twenty  Australian  universities  has  been  undertaken  for  the  Department 
of  Employment,  Education,  Training  and  Youth  Affairs  (Yetton  1997).  In  this,  the  institutions’  performance  is 
analyzed  against  five  key  factors:  strategy,  structure,  management  process,  roles  and  skills,  and  technology.  The 
study  uses  a qualitative  methodology,  using  in-depth,  semi-structured  interviewing  and  document  analysis  to 
investigate  management  issues.  The  strength  of  the  approach  is  to  ground  the  analysis  in  the  trends  and  patterns 
observed  in  field  research,  rather  than  testing  only  against  pre-determined  theoretical  frameworks. 

Strategic  choices  made  by  institutions  were  found  to  depend  on  the  historical  context  and  capabilities  of  that 
institution,  recognizing  factors  such  as  age,  size,  resources,  location  and  reputation.  To  individually  prioritize  and 
manage  the  five  key  factors  would  a complex  task.  What  emerges  from  the  study  however  are  certain  organizational 
forms,  or  patterns  of  factors,  that  serve  to  reduce  the  overall  complexity  of  decision  making.  Three  distinct 
approaches  were  identified,  each  related  to  the  different  contexts  of  the  institutions  (Tab.  1).  Although  more  than 
one  approach  might  co-exist  within  an  institution,  it  would  appear  that  pressure  to  reduce  differentiation  would  be 
experienced.  Yetton  also  anticipates  that  new  models  are  likely  to  emerge  with  changing  economic  and  technical 
environments.  In  looking  at  the  organizational  configuration  as  a whole,  IT  plays  a central  role  in  each  model  as  an 
enabler,  not  driver. 


Model 

Description 

Difficulties 

The  ‘Old’ 
University 

Cottage  industry 

IT  used  to  support  its  elite  learning  community.  Based  on  maintaining 
the  established  base  while  funding  independent  new  ventures. 
Successful  ventures  grow  while  feeding  the  institution  with  their 
innovations  in  teaching,  learning  and  research.  Attracts  high  status 
academic  risk  takers  & innovators. 

Requires  strong 
funding  base. 

Costly,  not  scalable 

Provides  Control  over  IP 

The  ‘Divisional’ 
University 

Single  product 
approach 

IT  supports  the  success  of  semi-autonomous  faculties.  Enabled  by 
powerful  central  IT  structure,  devolved  powerful  faculties  develop  their 
own  IT  technologies,  skills  & processes  according  to  their  interests  & 
competencies. 

Low  startup  cost 

Difficult  to  manage 

Blackbox  problem, 
not  scalable, 
inefficient 

The  ‘New’ 
University 

Mass  production 
approach 

IT  is  central  and  critically  underpins  the  strategic  agenda.  IT  based 
teaching  and  learning  delivered  from  a separate,  centrally  resourced 
unit,  building  new  core  competencies.  Designed  to  deliver  quality  and 
reliability  to  large  student  numbers. 

Low  startup  costs,  consistent  look  and  feel 

Inflexible,  high  staff 
development  costs, 
loss  of  control  of  IP 

Table  1:  Yetton’s  emerging  organizational  forms  in  universities  managing  the  introduction  of  IT  into  educational 
programs  and  administration.  Additional  interpretations  by  Reid  are  in  italics. 

The  University  of  Melbourne,  to  this  point  at  least,  has  fitted  the  old  university,  or  cottage  industry,  model  with  its 
strong  research  culture  and  tradition  of  intellectual  independence  of  faculty.  A Multimedia  Development  Fund  has 
seen  the  distribution  of  large  amounts  of  money  into  ‘innovative’  curriculum  projects  over  a number  of  years.  These 
have  been  primarily  driven  by  individuals  or  small  groups  and  are  diverse  in  their  technical  and  pedagogical 
approaches.  Individual  involvement  is  also  encouraged  by  policies  that  recognize  creative  input  in  any 
commercialization  of  intellectual  property  and  the  contribution  of  good  teaching  practice  to  staff  appraisals.  Such  a 
climate  has  encouraged  innovative,  and  perhaps  higher  risk,  application  of  new  teaching  technologies  by  faculty 
staff  (Taylor  1998).  Yetton's  model  illustrates  broad  institutional  policies  to  create  an  organizational  climate  in 
which  innovation  is  fostered,  although  offers  little  in  the  way  of  a decision-making  framework  for  the  new  online 
learning  infrastructures. 


2.  Reid’s  University  Strategy  for  Online  Instruction 

Reinterpretations  and  extensions  to  Yetton’s  model  reflect  the  emergence  of  highly  scalable  and  interoperable  online 
learning  technologies  (Reid  1999  p24).  Reid  identifies  difficulties  for  each  of  Yetton’s  models  (Tab.  1)  and  proposes 
a fourth  model,  ‘Mass  Customization’,  to  address  problems  of  scalability  and  cost. 
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Mass  customisation  Central  IT  framework  provides  academic  with  in  house  developed  tools  and  templates  - 
model:  interoperable  technology  supports  innovation  and  experimentation. 

In  this  model,  corporate  IT  systems  support  extensible  frameworks  to  facilitate  innovation  and  flexibility  in  the 
development  by  the  user.  For  example,  templates,  simple  authoring  tools  or  communication  components  might  be 
developed  to  meet  particular  local  learning  needs.  These  facilitate  efficient  development  of  learning  materials  and 
add  value  to  the  provider  IP  by  leveraging  off  the  functionality  of  corporate  delivery  and  administrative  systems. 
Reid  defines  criteria  for  system,  producer  and  user  issues  for  online  instruction  (Tab,  2).  With  these  guidelines 
Universities  can  respond  rapidly  over  time,  according  to  the  economic,  social  and  technical  environment.  These 
criteria  provide  another  perspective  from  which  to  generate  policy  for  the  development  of  online  resources, 
extending  Yetton’s  broader  organizational  models  to  address  system  issues  in  greater  detail. 


System 

Issues 

1 , Extensibility  and  interoperability: 

Open  systems,  rather  than  ‘black  lx)xes’,  allow  flexibility  in  structuring  and  efficient  use  of  materials. 
Online  frameworks  are  supported  by  corporate  IT  systems. 

2,  A critical  choice:  Functionality  vs.  interoperability  and  extensibility 

Degree  of  functionality  as  a determinant  for  product  selection  comes  only  after  conditions  for  integration 
with  corporate  systems  and  broader  adoption  across  the  university  are  met, 

3,  Security 

Consistent  authentication  and  administration  systems  across  the  university 

Producer 

Issues: 

1 . Role  of  subject  developers: 

Sustainable  processes  by  online  subject  developers  and  support  by  IT  support  staff. 

2,  Role  of  application  developers 

Continuing  creation  of  system  components  and  applications  according  to  accepted  standards  provided  by 
central  systems  and  online  frameworks 

User 

1.  Access  to  materials 

Issues: 

2.  Usability: 

Minimized  complexity  for  staff  and  students 

Table  2:  Reid’s  criteria  forjudging  policy  for  development  of  online  resources. 


The  Online  Courseware  Component  Architecture  (OCCA) 

OCCA  is  an  online  component  framework  under  development  at  the  MEU  that  displays  many  of  the  elements 
identified  by  Reid  in  his  Mass  Customisation  model.  It's  aim  is  to  support  creation  of  rich  interactive  learning 
customized  to  local  requirements,  while  ensuring  that  both  software  and  pedagogical  techniques  are  scalable  and 
transferable  (Fritze  & Welch  in  press).  The  component  architecture  is  viewed  here  from  Reid’s  perspectives  of 
functionality,  extensibility  and  scalability. 


Functionality 

From  a teaching  perspective,  OCCA  provides  a mechanism  for  academic  staff  to  create  various  forms  of  online 
learning  activity  to  solve  their  local  teaching  problems  and  create  novel  and  flexible  forms  of  learning  activity. 
These  could  involve,  for  example,  customized  interactive  interfaces,  techniques  for  reflective  & open-ended 
questioning  or  group  interaction  environments.  Educational  requirements  targeted  include: 

1 . customized  interactive  user  interfaces  to  discipline  knowledge; 

2.  provision  of  customized  feedback  in  response  to  a student's  actions; 

3.  support  for  learning  activities  involving  subjective  knowledge; 

4.  capacity  to  design  learning  sequences  that  can  incorporate  elements  of  face  to  face  interaction;  and 

5.  provision  of  reflective  summaries  and  overviews  of  the  learning  experience  for  both  student  and  teacher. 

Such  requirements  have  been  derived  directly  from  the  experiences  gained  working  with  faculty  teaching  staff  on 
many  real  course  projects  over  a number  of  years. 
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Extensibility  and  Interoperability 


As  a technology  for  online  component  scalability  and  inter-operation,  the  OCCA  specification  regularizes 
communication  between  conforming  Web  components  devised  in  Shockwave,  Java,  JavaScript  or  HTML.  It  uses 
open  protocols  for  information  transfer  including  XML  and  a ‘State  Description  Protocol’  (SDP),  with  which 
’snapshots’  of  arbitrary  interactive  Web  pages  can  be  saved,  recalled  or  analyzed. 

The  online  OCCA  browser  activity  may  involve  user  interface  components  such  as  forms  elements  and/or 
interactive  Java  or  Shockwave  interface  objects,  supported  by  generic  OCCA  ‘glue’  components  (Fig.  1).  The  key  to 
the  inter-operation  of  these  components  is  the  SDP  description  of  the  page.  As  well  as  providing  the  means  to  save 
and  restore  the  state  of  customized  pages  to  a database,  additional  functions  can  be  independently  developed  to 
provide  specialized  feedback,  navigational  management  or  access  to  SDP  database  records.  For  example,  SDP 
records  can  be  recalled  from  the  database  and  embedded  dynamically  into  other  OCCA  pages. 


Figure  1:  OCCA  component  and  protocol  framework.  Visible  browser  page  objects  are  in  bold  text. 


Perspectives  on  the  development  and  implementation  experience  of  OCCA 

In  order  to  better  understand  the  full  range  of  processes  by  which  a frameworks  such  as  OCCA  occurred,  a reflective 
history  of  the  development  of  OCCA  and  the  earlier  Learning  Engines  project  is  currently  being  undertaken.  This  is 
intended  to  make  visible  the  coal  face  complexity  of  a real  project  development  and  to  reveal  models  by  which  this 
complexity  can  be  managed.  This  research  is  intended  to  complement  that  of  Yetton  and  Reid  by  relating  the  bottom 
up  experience,  discovered  through  a grounded  theory  methodology,  with  top  down  strategic  models  (Glaser  & 
Strauss  1967).  Preliminary  findings  are  given  from  the  perspectives  of  the  MEU  developer  focussing  on  scalable 
online  solutions  and  teaching  staff  engaged  in  creating  innovative  learning  environments.  While  the  experience  of 
this  involvement  is  unique  and  does  not  by  itself  define  any  model  to  be  adopted,  the  intention  of  this  research  is  to 
better  inform  management  decisions  for  IT  at  an  institutional  level. 


Framework  Developer  Perspective 

The  current  implementation  has  evolved  out  of  prior  work  in  the  Learning  Engines  project  (Fritze  & Ip  1998),  the 
Virtual  Apparatus  framework  (Ip  et  al  1997)  and  Component-based  Object  Library  (Ji  1998).  Development  has  also 
depended  on  collaborations  with  numbers  of  independent  faculty  projects,  but  involved  only  a small  number  of 
MEU  staff  until  the  later  stages  of  the  project.  Some  characteristics  that  emerged: 

1 . The  basic  aim  of  the  project,  to  provide  a component  based  online  component-based  framework  for  rich  learning 
activities,  on  the  surface  remained  constant,  but  understanding  of  the  pedagogical  structures  and  specific  technical 
implementations  evolved  significantly. 
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2.  The  framework  progressed  through  a number  of  distinct  phases,  identified  by  different  collaborative 
involvements,  technical  and  pedagogical  approaches,  descriptive  vocabularies  and  personal  experiences. 

3.  The  researcher/developer  maintained  substantial  control  of  the  framework  technical  and  functional  design  until 
the  transition  and  production  phases  of  the  project,  when  substantial  shifts  in  roles  occurred. 

4.  The  continuous  pedagogical  input  from  involvement  with  a number  of  independent  curriculum  projects  played  a 
large  part  in  the  shaping  of  the  functional  operation  of  the  framework  (Fritze  et  al  1998b). 

5.  Although  the  project  is  now  being  integrated  into  general  University  IT  strategy,  the  development  process 
progressed  along  a number  of  'dead  end'  directions,  at  times  causing  stress  as  different  approaches  were  explored 
and  rejected. 

6.  The  online  environment  necessarily  involves  of  a number  of  parallel  technologies  and  is  fraught  with  technical 
challenges  not  present  in  equivalent  stand  alone  developments.  These  are  particularly  prominent  at  the  interfaces 
between  technologies,  for  example  between  Shockwave,  Java  and  Javascript. 


Teacher  Perspective 

One  of  the  main  faculty  projects  associated  with  Learning  Engines  and  later,  OCCA,  involved  the  development  of 
tutorial  systems  for  the  department  of  Physiology.  The  project  involved  the  writing  of  online  materials  extending 
across  24  weeks  of  campus-based  tutorial  workshops  to  make  up  time  otherwise  spent  in  lectures.  The  online 
techniques  complemented  the  use  of  commercial  applications  and  locally  developed,  highly  interactive,  single  use 
CAL  programs.  The  purpose  of  the  customized  online  materials  was  to  create  an  environment  for  timetabled 
workshops  that  would  stimulate  student  thinking  in  a manner  that  had  proved  difficult  with  other  approaches,  such 
as  using  the  TopClass  learning  framework.  The  aim  was  to  develop  the  techniques  that  could  be  implemented  as  an 
ongoing  process  by  local  teaching  staff  The  experience  of  collaborating  in  the  development  of  these  online 
techniques  was  characterized  by: 

1 . The  need  to  work  pragmatically  with  different  versions  of  the  software,  adapting  to  the  incomplete  functionality 
where  necessary,  while  still  addressing  the  desired  pedagogical  approach 

2.  The  online  component  technique  made  possible  forms  of  interactions  not  possible  with  alternative  approaches, 
e.g.  particular  question  activities  that  involved  sorting  complex  processes  or  use  of  a free  form  graphing  object 

3.  Unexpected  benefits  could  arise  from  situations  where  limits  were  reached  with  software  implementations, 
leading  for  example,  to  improved  understanding  of  computer  supported  collaborative  classroom  environments. 

4.  Teaching  ideas  developed  as  a result  of 'negotiation'  with  developer  over  desired  vs.  possible  functionality. 

5.  On  campus  collaborative  learning  within  supported  timetabled  workshops  has  great  potential.  A close  hands  on 
involvement  with  classes  needs  to  be  maintained,  for  example  to  appreciate  the  cultural  variation  across  student 
groups. 

6.  The  availability  of  funding  was  a prime  enabler  for  the  exploration  and  development  of  new  ideas  for  teaching. 

7.  Participation  in  conferences,  seminars  and  academic  papers  provided  valuable  opportunities  for  reflection, 
development  of  ideas  and  documentation. 


Discussion 

For  the  University,  OCCA  fits  the  many  of  the  criteria  proposed  by  Reid  for  functionality,  extensibility  and 
scalability  in  an  online  learning  environment.  This  is  the  ‘mass-customization’  model  in  which  a centralized 
framework  supports  continuing  innovation  and  customization  of  materials  at  various  levels  of  involvement. 
Systems-focussed  criteria  however,  while  interesting  to  apply  retrospectively,  or  in  selecting  between  existing 
products,  are  fairly  blunt  instruments  with  which  to  guide  the  development  of  generic  and  innovative  strategic 
frameworks.  Reid’s  suggestion  that  functionality  follows  interoperability  and  scalability  as  ‘the  major  determinant 
for  product  selection’  is  against  our  experience.  We  have  found  that  the  rapid  evolution,  problematic  nature  and 
complexity  of  online  technologies  demand  a close  connection  between  developer  and  academic  teacher.  The 
complexity  of  this  connection  was  illustrated  in  the  development  history  of  OCCA  and  further  grounded  studies  of 
this  type  should  be  used  to  inform  strategic  decisions  coming  from  above.  The  balance  between  individual,  faculty 
and  institutional  control  will  continue  to  fluctuate  in  response  to  changes  in  technology  and  environmental  pressures. 
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The  ‘Old’  University  approach  can  foster  an  environment  in  which  ideas  and  innovations  can  be  successfully 
incubated.  It  is  important  however,  that  successes  from  this  environment  are  effectively  transferred  into  broader  use 
within  the  institution  or  beyond.  The  development  of  OCCA  was  enhanced  by  ability  to  link  developers  and  faculty 
teaching  staff  in  real  projects  across  a cross-section  of  discipline  environments.  The  developing  model  could  be 
subjected  to  a wide  variety  of  probing  from  discipline,  pedagogical  and  technical  requirements.  This  approach 
however  was  characterized  by  directional  changes  and  frustrations.  In  adopting  such  approaches,  it  is  important  for 
the  both  institution  and  participants  to  acknowledge  the  nature  of  these  processes  and  the  inherent  risks  involved. 

The  EIP  study  is  an  interesting  snapshot  of  strategic  IT  practices  of  a cross-section  of  Australian  universities  and 
further  broad  studies  like  it  are  required  to  maintain  pace  with  technological  and  social  change.  Deeper 
understanding  of  institutional  development  cultures  and  activities  is  also  required  to  optimize  strategies  for 
transformation  of  teaching  and  learning  with  IT  most  appropriate  for  each  institution.  Grounded  theory  analysis  of 
representative  projects  is  one  way  in  which  strategic  thinking  by  management,  academics  and  developers  can  be 
informed. 
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Abstract:  This  paper  presents  theoretical  issues  about  the  role  of  metacognition  in 
learning,  as  well  as  the  architecture  and  design  of  a learning  system  dedicated  to 
monitoring  and  helping  students  to  use  their  metacognitive  skills.  This  system  is 
called  the  Reflection  Assistant  and  is  supposed  to  be  used  together  with  problem 
solving  or  case-based  learning  systems.  Thus,  through  interactions  with  the 
Reflection  Assistant  during  problem  solving  activities,  students  can  also  improve 
their  metacognitive  skills  (such  as  planning,  self-monitoring  and  assessing),  as  well 
as  their  knowledge  in  the  specific  domain.  The  purpose  of  this  research  is  to 
investigate  better  means  to  increase  metacognitive  knowledge,  including  awareness 
on  the  learning  process  and  capacity  of  transferring  skills  to  new  learning 
situations.  The  motivation  for  this  research  is  that  it  is  necessary  to  focus  as  much 
on  the  metacognitive  processes  of  problem  solving  as  on  the  cognitive  ones  to 
develop  good  problem  solvers. 


1.  Introduction:  Theoretical  Issues 

1.1  Metacognition  and  Learning 

The  argument  that  drives  this  research  is  that  metacognition  plays  a major  role  in  cognitive 
development.  Thus,  enabling  students  to  develop  a conscious,  explicit  model  of  their  metacognitive 
skills  by  means  of  reflective  activities  facilitates  the  improvement  of  both  cognitive  and  metacognitive 
expertise. 

Research  in  the  area  of  cognitive  psychology  (Flavell,  1976,  1979,  Schoenfeld,  1987,  Derry,  1992, 
amongst  others)  shows  that  an  important  type  of  knowledge  underlying  the  construction  of  complex 
representations  and  solutions  strategies  is  metacognitive  knowledge.  Metacognition  refers  to  a 
cognitive  system's  intelligence  about  itself  and  its  ability  to  regulate  and  control  its  own  operation.  So, 
knowing  about  one's  own  cognition  forms  the  basis  for  most  metacognitive  abilities.  These  abilities 
include  knowledge  about  one's  perceptions,  memories,  decisions,  and  actions  (Shimamura,  1994,  p. 
253). 

Metacognitive  knowledge  can  be  divided  into  three  related  but  distinct  categories  of  intellectual 
behavior  (Schoenfeld,  1987): 

Knowledge  about  thought  process 

This  is  related  to  people's  ability  to  accurately  assess  their  knowledge  and  knowledge  acquisition 
process.  In  a learning  situation  one  of  the  final  goals  is  to  help  students  develop  good  study  skills. 
Those  skills  depend,  in  part,  on  students'  ability  to  make  realistic  assessments  of  what  they  can  learn. 
Hence,  it  is  important  to  know  how  likely  it  is  that  students  will  reflect  on  their  thinking  and  how 
accurate  those  reflections  will  be. 

Control  and  Self-regulation 

This  aspect  of  metacognition  is  related  to  the  monitoring  of  cognitive  activities.  One  way  to  look  at 
this  aspect  of  metacognition  is  to  think  of  it  as  a management  issue,  i.e.,  how  well  one  manages  her 
time  and  effort,  as  she  is  involved  in  cognitive  tasks.  In  the  context  of  learning  self-monitoring  or  self- 
regulation refers  to  an  individual's  ability  to  conduct  on-line  self-checks  of  the  problem  solving 
process,  for  example;  a skill  that  is  particularly  important  when  the  student  deals  with  an  ambiguous 


or  confusing  problem.  Thus,  this  self-regulation  could  provide  the  student  with  information  to  guide 
her  problem  solving  actions. 

Beliefs  and  Intuitions 

These  are  related  to  the  ideas  about  a learning  topic  that  the  student  brings  to  her  work,  and  how  it 
shapes  the  way  the  student  faces  the  topic. 

Another  important  aspect  of  learning  is  transfer.  Educators  hope  that  students  will  transfer  learning 
from  one  problem  to  another  within  a course,  from  one  year  in  school  to  another,  between  school  and 
home,  and  from  school  to  workplace.  Transfer  can  be  improved  by  helping  students  become  more 
aware  of  themselves  as  learners  who  actively  monitor  their  learning  strategies  and  resources  and 
assess  their  readiness  for  particular  tests  and  performance.  Metacognitive  approaches  to  instruction 
have  been  shown  to  increase  the  degree  to  which  students  transfer  to  new  situations  without  the  need 
of  explicit  prompting  (Bransford  et  al.,  1999). 

1.2  Problem  Solving,  Metacognition  and  Reflection 

According  to  Schoenfeld  (1987),  problem  solving  activities  can  be  divided  into  three  conceptual 
stages,  named:  Preactive  (activities  performed  before  starting  to  solve  the  problem).  Interactive 
(activities  performed  as  the  student  works  on  the  problem)  and  Postactive  (all  those  activities 
performed  after  finding  a possible  solution  to  the  problem).  Considering  these  conceptual  stages,  three 
important  metacognitive  processes  can  play  a major  role:  planning  (mainly  in  the  preactive  and  also 
in  the  interactive  stage),  self-monitoring  (in  the  interactive  stage)  and  assessing  the  problem  solving 
efforts  (in  the  postactive  stage). 

Metacognitive  processes  are  important  contributors  to  problem  solving  performance  across  a wide 
range  of  domains  (Davidson  et  al.,  1994).  In  problem  solving  situations  there  are  always 
metacognitive  processes  involved,  such  as  (1)  identifying  and  defining  the  problem,  (2)  mentally 
representing  the  problem,  (3)  planning  how  to  proceed,  and  (4)  evaluating  what  you  know  about  your 
performance. 

The  successful  application  of  these  metacognitive  processes  depends  on  characteristics  of  the  problem 
(if  it  is  a well-structured  or  an  ill-structured  problem),  the  problem  solver,  and  the  context  in  which 
the  problem  is  presented.  To  the  extent  that  what  is  desired  is  to  identify  and  develop  good  problem 
solvers,  what  is  needed  is  to  focus  as  much  on  the  metacognitive  processes  of  problem  solving  as  on 
the  cognitive  ones  (Davidson  et  al.,  1994). 

Hence,  it  is  important  that  students  can  recognize  and  explore  their  own  metacognitive  processes. 
Reflective  activities  encourage  students  to  analyse  their  performance,  contrast  their  actions  to  those  of 
others,  abstract  the  actions  they  used  in  similar  situations,  and  compare  their  actions  to  those  of 
novices  and  experts.  The  reflection  on  the  metacognitive  skills  can  bring  awareness  of  their  thought 
process  and  increase  their  capacity  of  transferring  these  skills  to  new  learning  situations.  Fig.  1 
summarizes  the  interrelationship  between  the  problem  solving  conceptual  stages,  the  metacognitive 
processes,  and  the  role  of  reflection  in  this  context. 


Figure  1-  Metacognition,  Reflection  and  Problem  Solving 


2.  The  Design  of  the  Reflection  Assistant 
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The  Reflection  Assistant  is  be  a specialized  system  that  monitors,  analyses  and  improves  student's 
metacognitive  skills  in  problem  solving  situations.  It  is  still  in  the  design  phase  (it  has  not  been 
implemented  yet).  It  is  designed  to  be  used  in  combination  with  different  problem  solving  systems 
(e.g.:  mathematics  problem  solving  systems,  case-based  medical  system,  geometry  system,  etc.). 
Indeed,  the  Reflection  Assistant  is  disjoint  of  the  specific  domain  learning  system,  and  it  has  to  be 
configured  accordingly  to  the  specific  domain  that  it  will  be  applied. 

The  two  major  goals  of  the  Reflection  Assistant  are: 

(1)  Create  learning  environments  that  afford  and  encourage  metacognitive  processes. 

(2)  Improve  metacognitive  skills  development  in  general  and  thereby  enable  the  transference  of  these 
skills  to  future  learning  situations. 

Through  graphical  tools  and  specific  activities  the  Reflection  Assistant  makes  perceptible  to  the 
student  certain  metacognitive  skills  used  in  distinct  moments  of  instructional  activities.  Thus,  initially 
students  can  use  these  mechanisms  to  monitor  their  learning  process  and  become  aware  of  the 
metacognitive  strategies  they  are  using  when  engaged  in  problem  solving  activities.  After  that,  in  a 
subsequent  phase  they  can  use  appropriate  mechanisms  to  help  them  to  improve  these  metacognitive 
strategies. 

The  important  points  this  research  aims  to  investigate  are: 

(a)  The  appropriate  timing  for  providing  metacognitive  scaffolding  and  reflection  on  the  problem 
solving  process. 

(b)  The  effective  balance  between  reflection  focused  on  the  learning  task  (domain  specific)  and 
reflection  on  general  metacognitive  skills  (domain  independent)  for  a particular  problem  solving 
situation. 


2.2  Description  of  the  Reflection  Assistant 


The  Reflection  Assistant  contains  four  main  components.  The  general  architecture  is  shown  in  Fig.  2 
and  the  function  and  characteristics  of  each  component  are  explained. 
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Figure  2 - Architecture  of  the  Reflection  Assistant 


Metacognitive  Knowledge  Base 

This  component  has  detailed  information  about  metacognitive  skills  used  in  problem  solving 
situations.  Each  metacognitive  skill  is  described  in  the  Reflection  Assistant  in  terms  of  variables, 
which  may  have  their  values  modified  according  to  the  importance  that  the  specific  skill  plays  in  the 
specific  domain. 

Student  Metacognitive  Model 

This  is  responsible  for  recording  the  student's  responses  and  keeping  track  of  the  student’s 
performance.  The  Student  Metacognitive  Model  uses  information  from  the  Student  Log  and  from  the 
Metacognitive  Knowledge  Base  to  generate  a structured  model  of  the  student's  metacognitive 
capabilities  and  progress. 

This  model  informs  the  kind  of  assistance  that  should  be  provided  to  a particular  student  and  also 
serves  as  an  inspectable  model  for  the  student  (in  other  words,  the  student  can  also  view  her  use  of 
strategies). 
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Interaction  Generator 

This  component  combines  information  provided  by  the  Student  Metacognitive  Model  and 
Metacognitive  Knowledge  Base  to  elaborate  the  content  and  format  of  the  metacognitive  activity 
proposed. 

Translator 

This  is  the  communication  mechanism  (or  protocol)  that  will  make  possible  the  integration  of  the 
Reflection  Assistant  with  the  learning  system.  There  is  a two-way  communication  between  the  two 
systems:  (1)  the  learning  system  provides  information  about  the  actions  of  the  student  during  her 
interaction  (in  the  format  of  a log  file);  then,  the  Translator  interprets  this  information  and  a 
metacognitive  log  is  created  in  the  Reflection  Assistant.  (2)  The  reflection  assistant  sends  suggestions 
of  possible  new  actions  to  the  learning  system,  so  adapting  the  teaching  strategy  according  to  the 
student  metacognitive  model. 

Despite  being  decoupled  from  the  learning  system,  the  Reflection  Assistant  is  not  completely 
independent  of  it.  It  keeps  an  intermediate  level  of  interaction  with  the  system  through  a 
communication  channel.  This  interaction  is  reflected  in  the  architecture  of  the  system,  where  its  main 
components  need  to  contain  several  parameters  that  are  modified  depending  on  the  circumstances 
under  which  the  assistant  is  used  (i.e.,  the  specific  learning  system  to  which  the  Reflection  Assistant 
will  be  associated  with  or  the  kind  of  reflective  activities  relevant  to  a specific  domain  or  for  certain 
problems). 

2.3  The  Different  Types  of  Reflective  Assistance 

Following  the  idea  of  dividing  the  problem  solving  activities  into  three  conceptual  stages  (as 
explained  in  section  1.2)  the  Reflection  Assistant  can  be  used  in  different  configurations.  The 
reflection  activities  can  happen  either  before  starting  a new  problem/case;  or  during  the  interaction 
with  the  problem/case;  or  even  after  finishing  the  problem/case.  A specific  visualization  method  will 
be  selected  depending  on  both  the  timing  of  the  reflection  and  the  particular  metacognitive  skill 
emphasized  (c.g.:  planning,  monitoring,  etc.).  Indeed,  the  combination  Metacognitive  skills  - Timing 
- Visualization  method  (or  approach)  is  determined  by  the  characteristics  of  the  domain  and  task  and, 
also  the  student's  preferences.  Besides  that,  the  degree  of  specificity  of  the  behavior  of  the  assistant 
depends  on  the  level  of  integration  between  the  Reflection  Assistant  and  the  Learning  Environment. 
This  integration  is  generated  by  the  amount  of  information  provided  by  the  problem  solving  system  to 
the  Reflection  Assistant's  Translator. 

To  organize  these  different  arrangements  the  system  provides  three  basic  types  of  assistance, 
corresponding  to  a particular  moment  of  the  interaction.  Before  solving  a new  problem  (or  case)  the 
Preparation  to  Reasoning  Assistant  helps  the  student  to  recall  previous  knowledge  or  studied  cases. 
While  the  student  is  trying  to  solve  the  problem  he  can  interact  with  the  Support  to  problem  solving 
Assistant.  And,  finally,  after  finding  a solution,  the  student  could  use  the  Reflective  Follow-up 
Assistant  that  provides  means  to  reflect  on  the  performance  in  the  task  and  the  general  progress. 
These  assistants  aim  at  providing  structured  support  to  the  student,  establishing  "check-points"  to 
reflect  over  the  learning  process.  The  kinds  of  assistants  that  comprise  the  Reflection  Assistant  and 
which  metacognitive  skills  (or  processes)  they  are  mainly  related  to  are  detailed  below. 

Preparation  to  Reasoning  Assistant 

Questions  like  What  prior  knowledge  can  help  me  with  this  particular  task?  What  do  I need  to  know 
before  I can  successfully  deal  with  this  task?  What  should  I do  and  in  which  order?  How  much  time 
do  I have  to  complete  the  task?  drive  the'"activities  proposed  by  this  assistant.  The  idea  is  to  anchor 
new  concepts  into  the  learner's  existing  cognitive  knowledge  to  make  them  retrievable.  The  aim  of 
this  assistant  is  to  prepare  the  student  for  the  problem  solving  activity,  promoting  reflection  upon  the 
necessary  skills  to  solve  the  problem,  more  specifically,  the  capacity  to  recall  previous  knowledge  and 
make  generalizations. 

Sometimes  the  reflection  process  does  not  occur,  in  part  because  students  are  not  really  motivated  to 
perform  reflection  after  solving  the  problem  and  in  part  because  the  current  problem  solving  medium 
(paper  and  pencil)  does  not  really  lend  it  self  to  this  activity  (Collins,  A.  & Brown,  J.  S.,  1988). 
Because  the  Reasoning  Assistant  is  going  to  be  used  before  the  learning  activity  takes  place,  it  may 
contribute  to  bypass  problems  of  low  motivation. 

Support  to  Problem  Solving  Assistant 

This  assistant  offers  a means  for  monitoring  the  student's  plan  of  action.  It  helps  the  student  during 
the  cognitive  activity  with  mechanisms  to  reflect  on  the  necessary  skills  to  solve  a problem,  making 
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them  think  about,  among  other  things:  How  am  I doing?  Am  I on  the  right  track?  How  should  I 
proceed?  Should  / move  in  another  direction?  What  do  I need  to  do  if  I don^t  understand? 

Also  an  important  point  is  to  use  the  Reflection  Assistant  as  a means  to  aid  the  student  when  she  feels 
stuck.  Thus,  the  Support  to  problem  solving  Assistant  should  provide  support  for  students  to  seek  for 
clarification  and  to  reexamine  their  problem  solving  strategy. 

Reflective  Follow-Up  Assistant 

After  concluding  the  cognitive  task  the  student  can  use  this  assistant  to  verify  her  performance  and 
compare  it  with  previous  interactions.  Here,  generalizations  are  again  used  to  help  the  student  to 
organize  the  case  or  problem  studied  into  a general  knowledge  framework.  Examples  of  common 
points  to  reflect  on  in  this  moment  can  be: 

Did  my  particular  course  of  thinking  produce  more  or  less  than  I had  expected?  What  could  I have 
done  differently?  How  might  I apply  this  line  of  thinking  to  other  problems? 

2.4  The  Design  of  the  Reflective  Follow-up  Assistant:  First  Prototype 

There  are  several  points  to  take  into  account  in  the  development  of  such  a learning  system.  The 
abstract  nature  of  the  metacognitive  processes  is  indeed  a major  concern.  Also,  the  graphical  tools 
used  to  represent  these  processes  have  to  be  adequately  developed.  Moreover,  these  tools  and 
metacognitive  processes  probably  vary  from  one  student  to  the  other,  depending  of  their  own 
characteristics. 

So,  in  order  to  collect  information  about  all  these  factors  that  influence  the  whole  system,  some  initial 
studies  are  being  performed.  As  part  of  this  initial  phase  a first  version  of  the  Reflective  Follow-up 
Assistant  is  being  designed  and  will  be  used  with  the  PATSy  system.  PATSy  (Patient  Assessment 
Training  SYstem)  is  a case-based  interactive  learning  environment  that  keeps  a database  of  medical 
cases  in  the  domain  of  aphasiology  (Lum  & Cox,  1998,  Cox,  1999).  PATSy  provides  a context  in 
which  students  may  practice  diagnosis  via  deductive  reasoning.  Thus,  the  Reflection  Assistant  will 
help  students  to  monitor  their  reasoning  process.  PATSy  keeps  an  electronic  log  of  all  actions 
performed  by  the  student  (sequence  of  clinical  tests  chosen,  cases  studied,  etc.). 

The  first  version  of  the  Reflective  Follow-Up  Assistant  will  provide  two  types  of  information: 
quantitative  assessment  and  qualitative  assessment  of  the  metacognitive  skills  used  during  interaction 
with  the  PATSy  system.  Fig.  3 (a,b  and  c)  show  mock-ups  of  this  prototype. 

Quantitative  Assessment 

After  each  session  with  the  PATSy  system  students  will  see  a graph  showing  the  time  spent  in 
cognitive  and  metacognitive  activities.  For  example.  Fig.  3a  shows  a sample  graph  of  a student. 
Using  that,  the  student  can  see  graphically  the  amount  of  time  spent  analysing  the  problem,  planning, 
studying  previous  cases  or  solutions  and  attempting  to  solve  the  problem.  Other  information  provided 
are  the  moments  where  the  student  reflected  on  her  studying  process.  PATSy  already  provides  some 
questions  that  should  help  students  to  step  back  and  ask  what  they  want  to  do  next.  Further 
information  generated  is  the  summary  of  the  total  amount  of  time  spent  in  the  task. 


Figure  3a  - Quantitative  Assessment  (sample  1)  Figure  3b  - Quantitative  Assessm.  (sample  2) 
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Another  graph  from  the 
quantitative  assessment  is 
shown  in  Fig.  3b.  It  is  a 
graphical  comparison 
between  the  performance 
in  the  present  case  and  in 
previous  cases.  Another 
possible  comparison  could 
be  between  the 
performance  of  one  student 
with  those  of  other 

students. 

Qualitative  Assessment 

This  kind  of  assessment  aims  at  providing  more  detailed  explanations  to  the  student  about  how  to 
improve  her  metacognitive  skills.  It  is  achieved  by  asking  questions  about  the  student’s  performance. 
For  example.  Fig.  3c  shows  a qualitative  assessment  of  the  planning  skills  of  one  student.  The  student 
is  invited  to  explain  why  she  failed  in  using  the  planning  strategies  she  proposed  to  use  at  the 
beginning  of  the  task.  The  idea  is  to  make  the  student  to  reflect  on  the  importance  of  a good  plan  of 
action  that  has  to  be  followed  during  the  problem  solving.  In  addition,  the  system  will  to  provide 
scaffolding  to  increase  the  planning  abilities  of  that  student. 

3.  Conclusions 

This  research  investigates  better  means  to  make  students  learn  important  self-monitoring  and  other 
metacognitive  skills  in  computer-based  problem  solving  environments.  The  final  goal  is  the  design  of 
a 
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new  learning  environment  that  extend  the  possibilities  of  interactive  learning  systems,  giving  students 
opportunities  for  reflection,  revision  and  feedback  over  their  problem  solving  efforts. 

This  paper  has  presented  the  design  of  the  Reflective  Follow-up  Assistant  Module  to  be  used  in  the 
postactive  stage  of  problem  solving.  It  focuses  on  both  quantitative  and  qualitative  assessments  of 
students’  performance.  More  work  has  to  be  done  in  the  design  of  the  other  modules  of  the  Reflection 
Assistant:  the  Preparation  to  Reasoning  Assistant  and  the  Support  to  Problem  Solving  Assistant. 

A distinguished  characteristic  of  the  Reflection  Assistant  is  that  it  is  both  a domain  independent 
system  and  also  maintains  an  intermediate  level  of  interaction  with  the  learning  system  that  uses  it. 
The  granularity  of  interaction  is  an  interesting  point  of  investigation.  This  relates  to  the  tension 
between  teaching  metacognitive  skills  within  a particular  context  and  ensuring  their  transferability  to 
other  contexts.  Some  initial  studies  have  been  done  and  the  first  prototype  is  under  development. 
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Abstract:  Classroom  teachers'  comfort  and  familiarity  with  computers  and  software  influences 
student-computer  use  in  the  schools.  Too  often  teachers  and  students  remain  mired  in  repetitive 
preoccupation  with  basic  software  mechanics  to  the  neglect  of  interaction  with  the  content  at  hand. 
To  offload  these  tasks  onto  the  software,  an  On-  Line  Assistant  (OLA)  was  developed  to 
accompany  the  commercial  hypermedia  software,  HyperStudio.  This  OLA  introduced  the  software, 
provided  a framework  for  students  to  storyboard  projects,  sequentially  introduced  software 
mechanics,  and  suggested  basic  principles  of  good  design.  The  impact  of  this  OLA  on  the  nature  of 
classroom  interactions  and  on  the  quality  of  projects  produced  was  investigated.  Control  group 
students  used  the  hypermedia  software  alone  and  the  experimental  group  used  the  software  with  the 
OLA.  Projects  employing  the  OLA  were  consistently  rated  higher  than  those  using  software  alone. 
Student  usage  of  the  OLA  significantly  reduced  interactions  dealing  with  basic  software  mechanics 
while  significantly  increasing  interactions  concerning  design,  stack  issues,  and  content.  Interactions 
significantly  increased  for  the  experimental  group  in  the  areas  of  advanced  software  issues,  design, 
stack  issues,  and  content.  The  OLA  did  not  diminish  the  teacher's  role  but  perceptibly  shifted  the 
nature  of  interactions  to  those  areas  requiring  higher  order  cognition. 


Introduction 

The  use  of  computer  technology  in  today's  schools  is  often  limited  by  educators  who  feel  overwhelmed  by  the 
demands  placed  upon  them  to  become  proficient  at  the  hardware,  the  software,  and  curriculum  development 
simultaneously  (Howe  & O'Shea  1980).  The  majority  of  educators  are  familiar  with  computer  usage  at  a second 
state  level  of  technological  innovation  (Maes  1994).  At  this  level  the  computer  is  used  in  conjunction  with  existing 
technologies  to  do  what  can  already  be  done  albeit  more  efficiently.  The  majority  of  software  developed  over  the 
past  thirty  years  has  reflected  this  second  level  of  innovation.  Computer  assisted  instruction  has  long  focused  on 
replicating  the  traditional  classroom  model  of  top-down  teaching,  drill  and  practice,  and  teacher  directed  learning 
(Naisbitt  1982;  Glasser  et  al.  1988).  However,  later  research  began  to  acknowledge  learning  as  a constructive 
process  (O'Shea  1979;  Druin  1998;  Ennals  1983).  Recent  innovations  in  intelligent  agents  have  pointed  towards  a 
collaborative  partnership  between  computer  and  student  (Lockard  et  al.  1997). 

New  epistemologies  of  learning  are  emerging  that  indicate  a dichotomy  between  theorists'  accounts  of  cognition 
and  the  actual  practices  of  people  in  learning  activities  (Perkins  1993).  It  is  increasingly  evident  that  cognition  does 
not  reside  entirely  in  the  individual  mind  but  may  have  portions  of  it  distributed  among  other  people  and  simple  or 
complex  tools  (Papert  1980).  One  subset  of  the  current  work  on  distributed  cognition  is  off-loading  which  occurs 
when  a person  places  some  of  the  cognitive  burden  for  performing  a task  onto  another  person  or  tool  (Salomon 
1993).  This  can  be  a positive  condition  such  as  when  one  creates  a list  of  items  rather  than  trying  to  carry  those 
items  in  memory.  However,  it  can  be  a negative  experience  in  the  classroom.  Educators  have  long  assumed  portions 
of  students'  cognitive  burden  when  they  make  all  the  choices  about  what  to  study,  how  to  study  it  and  what 
resources  to  use.  In  computer  usage,  this  off-loading  is  part  of  what  makes  meaningful  integrated  computer  usage  so 
overwhelming  to  teachers.  It  is  simply  too  much  to  be  the  expert  on  the  hardware,  the  software,  and  the  curriculum 
while  simultaneously  planning  the  implementation  of  the  project  with  integrated  computer  support. 

The  use  of  an  on-line  assistant,  or  OLA,  can  facilitate  off-loading  some  of  the  teacher’s  responsibilities  onto  the 
OLA  smoothing  the  way  for  better  computer  integration  into  the  learning  experiences  of  students.  With  this  support, 
students  are  able  to  assume  responsibility  for  learning  the  mechanics  of  the  software  and  for  planning  projects.  This 
study  focused  on  the  addition  of  an  on-line  assistant  to  commercial  hypermedia  software  (HyperStudio)  and  the 
subsequent  impact  on  the  quality  of  the  completed  project,  the  nature  of  teacher/student  interactions,  and  the  nature 
of  student/student  interactions. 
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Study  Design 

A quasi-experimental  design  was  used  in  this  study.  Two  existing,  intact  4^^  grade  classrooms  of  nine  year  olds  from 
an  urban  school  were  used  as  the  experimental  and  control  groups.  Random  assignment  of  subjects  to  groups  was 
precluded,  A nonequivalent  posttest-only  control  group  design  was  employed.  Both  classrooms  and  their  teachers 
were  pretested  to  determine  familiarity  with  the  basics  of  the  commercial  software  program  to  be  used  in  the  study. 
In  addition,  NCE  scores  on  the  Iowa  Test  of  Basic  Skills  were  compared  between  the  two  groups.  These 
comparisons  determined  initial  equivalence  of  the  experimental  and  control  groups. 

Both  groups  used  HyperStudio  to  develop  a project  consisting  of  a stack  with  a minimum  of  five  cards.  Both 
groups  had  access  to  HyperStudio  software  as  well  as  their  classroom  teacher.  The  control  group  of  22  students  used 
the  software  in  its  unedited  form  while  the  experimental  group  of  23  students  had  access  to  an  on-line  assistant 
(OLA)  developed  by  the  researcher  to  assist  students  in  planning  their  project,  in  learning  the  HyperStudio  software, 
and  in  developing  awareness  of  rudimentary  design  elements.  Quality  of  the  finished  HyperStudio  projects  was 
assessed  for  both  groups  to  determine  effectiveness  of  the  treatment. 

Teacher/Student  and  Student/Student  interactions  were  evaluated  in  both  groups.  Interactions  in  the  following 
seven  areas  were  tallied  daily:  Software  issues  - basic  (new  cards,  buttons,  text  boxes);  Software  issues-  advanced 
(graphics,  transitions,  new  button  actions);  Design  issues  (color,  font,  layout);  Stack  issues  (linking  and  layout); 
Spelling  and  Grammar;  Content  material  (appropriate  content);  and.  Content  composition  (appropriate  discourse 
form). 

Hypermedia  projects  completed  during  the  research  period  were  collected,  copied,  and  distributed  to  raters.  The 
MEDIA  HCI  Design  Evaluation  (Druin  1998)  was  used  to  assess  four  areas:  Reaction  to  Visual  Design  (card  layout 
and  design,  organization  of  information,  and  sequence  of  cards);  Reaction  to  Stack  Construction  (user  interface  with 
the  stack);  Reaction  to  Content  (content  and  presentation  of  information);  and  the  Overall  Reaction  to  Stack 

The  teacher  of  the  classroom  using  the  software  with  OLA  was  interviewed  to  discuss  her  reactions  to  the 
presence  of  the  on-line  assistant,  how  effective  she  felt  it  was  with  her  students,  and  her  general  reactions  to  helping 
students  develop  a project  using  HyperStudio.  Her  students  were  surveyed  to  assess  their  acceptance  of  the  OLA. 
Additionally,  an  exit  interview  was  held  with  the  teacher  of  the  classroom  that  used  the  software  alone  to  discuss  her 
reactions  to  helping  students  develop  a project  using  HyperStudio. 


The  On  Line  Assistant 

The  researcher  developed  an  OLA  whose  persona  was  based  on  Addy,  a dog  that  already  periodically  appears  in 
existing  dialogue  boxes  in  HyperStudio,  It  was  felt  that  the  appearance  of  Addy  both  in  dialogue  boxes  and  as  the 
OLA  would  maintain  consistency  within  the  software.  The  name  of  the  OLA  service  was  Ask  Addy  and  it  first 
assisted  the  students  in  designing  a HyperStudio  stack  by  introducing  a storyboard  approach  to  design.  This 
component  of  the  OLA  sent  the  students  off-line  to  develop  a storyboard.  Students  consulted  with  their  teacher  on 
the  storyboard's  links  and  content.  The  completed  storyboard  served  as  the  working  plan  to  be  followed  when 
students  returned  to  the  software  to  begin  stack  construction. 

Upon  returning  to  the  software  to  begin  stack  development,  the  OLA  sequentially  guided  the  student  through  a 
series  of  steps.  First,  students  created  each  card  with  only  text  boxes  and  content  inserted  in  those  boxes.  Next,  the 
portion  of  the  OLA  was  brought  on-line  to  assist  students  in  adding  buttons.  After  students  had  demonstrated  their 
stack  to  their  teacher  the  next  portion  of  the  OLA  was  added  which  assisted  students  in  adding  graphics  to  their 
existing  cards.  Finally,  the  OLA  offered  advice  on  modifying  existing  text  and  button  objects.  Concurrent  with  the 
on-line  components  that  addressed  adding  and  modifying  buttons  and  graphics,  the  availability  of  design  hints  were 
added.  These  design  hints  covered  such  items  as  choice  and  variety  of  fonts,  color  selection,  and  button  transitions. 

The  researcher  developed  the  OLA  using  HyperStudio  software.  The  OLA  was  implemented  as  a series  of 
independent  stacks,  each  linked  to  a menu  card  that  was  in  turn  linked  to  the  cards  in  the  students’  stack.  This 
enabled  the  researcher  to  bring  various  components  of  the  OLA  on-line  sequentially  throughout  the  experimental 
period.  To  manage  memory  size,  the  OLA  contained  only  text  and  graphics.  No  voice  or  animation  features  were 
included.  This  sequential  introduction  of  the  OLA  enabled  a certain  amount  of  control  over  the  students’  process. 
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Research  Results 


Results  of  the  study  suggest  that  the  presence  of  an  OLA  significantly  impacts  the  quality  of  students  ’ completed 
hypermedia  projects.  There  was  a significant  difference  between  the  holistic  quality  scores  of  the  completed  projects 
of  the  group  using  software  with  the  OLA  and  the  quality  scores  of  the  group  using  software  alone.  The  results  of 
the  MEDIA  HCI  Design  Evaluation  forms  showed  that  the  mean  holistic  score  of  the  stacks  completed  by  students 
in  the  group  using  the  OLA  was  38.9  points  higher  than  the  mean  of  those  students  using  the  software  alone.  See 
Table  1 at  the  following  URL:  http://www.coehs.uwosh.edu/facultv/garcia/addvres.htm. 

To  measure  the  nature  of  teacher/student  and  student/student  interactions,  seven  areas  were  chosen  for 
consideration.  These  seven  areas  formed  a hierarchy  of  possible  interactions.  At  a rudimentary  level,  the  frequency 
of  interactions  concerning  basic  software  issues  about  mechanics  of  the  HyperStudio  software  and  spelling  and 
grammar  was  measured.  Then  interactions  about  advanced  software  skills  such  as  importing  graphics  and 
specialized  button  actions  were  measured.  Next,  interactions  were  measured  that  required  the  application  of  more 
complex  thinking  skills  such  as  design  issues,  stack  issues,  content  material  and  content  composition.  Finally,  the 
sum  of  the  total  number  of  interactions  in  all  seven  areas  was  computed  and  analyzed. 

Results  of  the  study  suggest  that  the  presence  of  an  OLA  positively  impacts  teacher/student  interactions  by 
increasing  the  number  of  interactions  concerning  areas  that  require  higher-order  thinking  skills  while  reducing  the 
number  of  interactions  concerning  basic  software  issues.  There  was  no  significant  difference  in  the  total  number  of 
interactions  between  teacher/students  in  the  group  using  software  with  the  OLA  and  the  group  using  the  software 
alone.  However,  a shift  in  the  nature  of  teacher/student  interactions  could  be  seen  as  indicated  by  the  frequencies  of 
interactions  occurring  at  the  differing  levels  of  complexity.  The  observed  pairs  in  the  group  using  software  with  the 
OLA  had  significantly  fewer  interactions  with  their  teacher  about  basic  software  issues,  while  there  were 
significantly  more  interactions  in  the  complex  areas  of  design  issues,  stack  issues,  content  material,  and  content 
composition.  See  Table  2 at  the  following  URL:  http://www.coehs.uwosh.edu/faculty/garcia/addvres.htm. 

Further  results  of  the  study  suggest  that  the  presence  of  an  OLA  positively  impacts  student/student  interactions 
by  increasing  the  number  of  interactions  between  students  concerning  areas  that  require  higher-order  thinking 
skills.  There  was  no  significant  diiference  in  the  total  number  of  interactions  between  student/student  in  the  group 
using  software  with  the  OLA  and  the  group  using  the  software  alone.  However,  a shift  in  the  nature  of 
student/student  interactions  could  be  seen  as  indicated  by  the  frequencies  of  interactions  occurring  at  the  differing 
levels  of  complexity.  There  were  significantly  more  interactions  between  students  in  the  observed  pairs  in  the  group 
using  software  with  the  OLA  in  the  areas  of  advanced  software  issues,  design  issues,  stack  issues  and  content  issues 
which  require  the  application  of  complex  thinking  skills.  See  Table  3 at  the  following  URL: 
http://www.coehs.uwosh.edu/facultv/garcia/addyres.htm. 

The  study  results  suggest  that  the  presence  of  an  OLA  is  accepted  by  students  as  most  useful  in  helping  design  a 
hypermedia  project  and  is  an  acceptable  source  for  information. _W\  students  in  the  group  using  software  with  the 
OLA  completed  a Post  Treatment  Survey.  From  the  responses  on  this  survey  it  may  be  concluded  that  a majority  of 
students  found  AskAddy  to  be  helpful,  easy  to  understand,  and  quick  to  get  help.  A majority  of  students  would  like 
Ask  Addy  to  be  able  to  offer  more  help  than  was  available  in  the  prototype.  However,  the  majority  of  students 
reported  that  they  still  had  to  ask  their  teacher  for  help  even  after  using  Ask  Addy  and  the  majority  of  students  felt 
that  they  turned  to  AskAddy  less  often  than  they  did  their  teacher.  Over  fifty  percent  of  the  students  indicated  that 
Ask  Addy  was  most  helpful  in  helping  them  learn  how  to  design  a HyperStudio  stack.  Secondarily,  Ask  Addy  was 
most  helpful  in  assisting  students  in  learning  how  to  make  text  boxes.  See  Table  4 at  the  following  URL: 
http://www.coehs.uwosh.edu/faculty/garcia/addvres.htm 

Finally,  results  of  the  study  suggest  that  there  is  a positive  relationship  between  student  acceptance  of  an  OLA 
and  the  quality  of  their  completed  hypermedia  project.  _Dai3,  analysis  confirmed  that  there  is  a moderate  correlation 
between  student  acceptance  of  the  OLA  and  the  quality  of  their  completed  hypermedia  stack.  This  acceptance  of  the 
OLA  was  indicated  by  ninety  percent  of  the  students  indicating  that  they  found  AskAddy  helpful.  Over  fifty  percent 
of  the  students  also  indicated  that  they  would  like  to  be  able  to  use  Ask  Addy  for  getting  more  help.  Students 
indicated  that  they  found  Ask  Addy  to  be  most  helpful  in  the  design  process  and  in  adding  text  boxes.  This 
overwhelming  student  acceptance  of  Addy’s  assistance  positively  correlated  with  raters’  assessment  of  the  quality  of 
students’  stacks.  See  Table  5 at  the  following  URL:  http://www.coehs.uwosh.edu/facultv/garcia/addyres.htm. 

Discussion  of  Results 
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The  completed  projects  of  the  students  in  the  group  using  software  with  the  OLA  were  consistently  rated  higher  than 
those  completed  by  students  in  the  group  using  software  alone.  Although  both  groups  of  students  had  been 
instructed  by  their  teacher  to  create  a minimum  five-card  stack,  those  students  in  the  group  using  software  with  the 
OLA  went  well  beyond  that  five-card  minimum.  The  greater  number  of  cards  in  these  stacks  and  the  more  web-like 
construction  may  be  attributed  to  the  storyboarding  process.  The  researcher  observed  that  the  use  of  storyboarding 
did  not  occur  naturally  to  either  teachers  or  students. 

The  teacher  of  the  group  using  the  OLA  said,  “I  was  pleasantly  surprised  by  the  fact  that  the  five-card  minimum 
was  never  a discussion  point  with  these  kids.  They  got  all  carried  away  with  their  storyboards  and  connections  and 
put  down  so  many  more  cards,  I thought  they’d  never  complete  it  but  since  they  had  this  plan  (storyboard)  that  they 
had  committed  to,  they  did  it!”  It  is  important  to  note  the  italics  in  the  prior  quote.  The  students  had  committed  to 
the  plan  because  they  created  it.  The  teacher  did  not  impose  the  plan;  the  product  was  not  the  emphasis.  The 
storyboard  places  the  emphasis  on  the  process  instead.  The  creation  of  a storyboard  allowed  the  students  to  assume 
responsibilities  or  executive  functions  (Perkins  1993)-  deciding  on  content,  locating  resources,  arranging  those 
resources-  that  would  have  traditionally  been  off-loaded  to  the  teacher.  The  storyboarding  period  provided  a rich 
environment  for  teacher/student  and  student/student  interactions  on  the  content  of  the  material  to  be  presented  in  the 
stack.  The  group  using  the  OLA  showed  a significantly  higher  number  of  teacher/student  and  student/student 
interactions  about  content  material.  The  bulk  of  these  interactions  occurred  during  the  storyboarding  period. 

The  storyboarding  process  also  provided  opportunities  for  the  teacher  and  students  to  talk  about  how  their  stack 
was  constructed.  The  teacher  of  the  group  using  the  OLA  averaged  thirty-seven  interactions  with  groups  on  the  stack 
construction  and  layout  while  the  teacher  of  the  group  using  the  software  alone  averaged  only  five  interactions.  This 
lack  of  interaction  was  reflected  in  the  smaller  stack  size  and  linear  design  of  the  students  using  the  software  alone. 
Ask  Addy  was  able  to  significantly  reduce  the  number  of  teacher/student  interactions  that  dealt  with  basic  software 
issues.  These  mechanics  form  the  foundation  for  all  work  in  HyperStudio  and  are  relatively  easy  to  learn  but  require 
much  repetition  from  the  classroom  teacher  in  a lab  setting. 

Total  facility  with  any  software  is  a matter  of  practice  and  experience.  Educators  usually  lack  the  time  to 
develop  that  facility  and  are  therefore  hesitant  to  move  students  ahead  in  a project  using  unfamiliar  software.  Ask 
Addy  was  brought  on  line  sequentially  in  a manner  that  facilitated  students’  completion  of  their  storyboard  plan. 
Initially,  Ask  Addy  only  provided  help  in  creating  new  cards  and  adding  text  boxes  in  which  to  enter  text.  Ask  Addy 
asked  that  students  make  all  the  cards  in  the  stack  first  before  adding  buttons  and  graphics.  Because  this  involved  the 
bulk  of  the  on-line  work  (word  processing)  the  teacher  of  this  group  became  concerned.  “Will  they  actually  finish 
this  project?”  she  asked,  eight  days  into  the  data  collection  period.  Having  spent  three  to  four  days  storyboarding 
and  then  another  four  to  five  days  creating  the  rudiments  of  the  cards  for  the  stack,  it  did  indeed  seem  that  they  were 
lagging  behind.  However,  the  finished  products  demonstrated  that  the  sequential  introduction  of  skills  facilitated  that 
completion  of  the  stacks  as  originally  envisioned  by  the  student. 

It  is  important  to  note  from  the  data  analysis  of  teacher/student  interactions  that  the  addition  of  an  OLA  did  not 
significantly  affect  the  total  number  of  teacher/student  interactions.  However,  it  did  impact  the  nature  of  the 
teacher/student  interactions.  As  previously  discussed,  the  storyboarding  period  required  by  Ask  Addy  did 
significantly  increase  the  number  of  teacher/student  interactions  related  to  stack  issues,  and  content  material.  In 
addition,  there  were  significantly  more  teacher/student  interactions  concerning  content  composition  and  design 
issues  during  the  time  spent  on-line  with  HyperStudio  stack  development.  Students’  self-assessments  of  the  presence 
of  the  OLA  also  supported  the  conclusion  that  the  need  for  a teacher  is  not  diminished  by  the  on-line  assistance. 
Fifty-seven  percent  of  the  students  indicated  that  they  did  not  Ask  Addy  for  help  more  than  they  asked  it  of  their 
teacher.  The  same  percent  of  students  also  reported  having  to  ask  their  teacher  for  help  even  after  asking  Addy  for 
assistance. 

The  total  number  of  student/student  interactions  was  also  not  significantly  different  in  observed  pairs  in  both 
groups.  Again,  however,  the  nature  of  that  interaction  was  different  as  observed  in  the  frequency  of  interactions  in 
the  areas  of  advanced  software  issues,  design  issues,  stack  issues,  and  content  material.  As  previously  discussed,  the 
story  boarding  period  provided  an  environment  in  which  students  could  have  intense  discussions  on  stack  design  and 
content  material.  During  the  actual  stack  production  period  on  the  computer,  student/student  interactions  in  the 
group  using  the  OLA  gradually  shifted  to  design  issues,  advanced  software  issues,  and  appropriate  discourse  form 
for  this  media.  These  students  actively  debated  over  the  quantity  of  print  versus  the  ability  of  pictures  to  tell  a story. 
They  discussed  what  Ask  Addy  listed  as  good  principles  of  basic  design,  deciding  whether  to  listen  to  her  advice  or 
discard  it  in  the  interest  of  curiosity  and  personal  taste. 

An  another  interesting  phenomena  occurred  as  individual  groups  were  freed  by  the  OLA  to  move  ahead  with 
more  advanced  techniques.  Groups  or  individuals  began  to  emerge  as  resident-experts  on  specific  techniques  which 
resulted  in  an  impromptu  II  (Slavin  1990)  method  emerging.  Again,  this  emerged  from  the  student  process. 
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It  did  not  require  that  the  teacher  assume  the  responsibility  for  building  in  the  peer  tutoring  as  part  of  the  process. 

Student  responses  on  the  Post  Treatment  Survey  when  correlated  with  the  MEDIA  HCI  Design  Evaluation  form 
holistic  scores  indicated  a moderate  correlation  between  student  acceptance  of  the  OLA  and  the  quality  of  the 
finished  project.  Students  demonstrated  a purposeful  acceptance  of  Ask  Addy.  The  majority  of  students  indicated  that 
they  did  find  Ask  Addy  helpful  particularly  in  the  areas  of  design  and  adding  textboxes.  These  two  areas  are 
indicative  of  student  concern  with  the  complex  issues  of  content  and  process.  The  Ask  Addy  prototype  was  devoid  of 
anthropomorphic  qualities;  Addy’s  assistance  was  straightforward  and  presented  in  a formal  manner.  Therefore, 
students  acceptance  of  Ask  Addy  was  based  on  the  quality  of  help  offered  rather  than  an  attraction  to  the  character, 
Addy. 

Students  did  indicate  a fascination  with  the  possibilities  of  an  OLA.  In  discussions  throughout  the  production 
period,  they  discussed  their  visions  of  what  Ask  Addy  could  be.  Students  wanted  much  more  than  what  the  prototype 
could  offer.  They  wanted  Ask  Addy's  bank  of  assistance  to  be  much  broader.  Some  students,  familiar  with 
Microsoft's  Office  97,  wanted  more  personas  available  as  well. 


Conclusion 

An  OLA  can  help  students  and  teachers  facilitate  a process  and  produce  better  quality  work.  It  can  also  suggest  a 
learning  environment  that  is  conducive  to  a different,  more  complex  nature  of  teacher/student  interactions.  While  the 
act  of  directing  students  to  make  a storyboard  prior  to  the  production  process  did  not  require  an  OLA,  it 
demonstrates  that  an  OLA  can  provide  valuable  guidance  in  a suggested  process  for  using  that  software.  This  would 
help  a teacher  who  is  unfamiliar  with  the  basic  hypermedia  software  in  the  implementation  of  a project. 

The  role  of  a teacher  is  not  diminished  by  the  presence  of  an  OLA  but  that  role  is  perceptibly  altered  in  the 
interactions  between  teacher  and  student.  The  on-line  assistant  is  able  to  assist  the  students  in  learning  the 
rudimentary  mechanics  of  a hypermedia  software  program  while  making  make  them  sensitive  to  the  discourse  form 
of  hypermedia.  The  OLA  may  stimulate  children  to  take  more  responsibility  for  higher  level  activities  such  as 
planning  strategies,  locating  resources,  and  organizing  information  that  have  typically  been  off-loaded  onto  the 
teacher.  This  in  turn  can  free  teachers  to  interact  with  students  concerning  more  complex  content  and  process  issues 
while  guiding  their  discovery. 
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Abstract:  Computer  Algebra  Systems  (CAS)  are  complex  computer  systems  that  have  not  been 
designed  for  learning  situations.  In  order  to  enhance  their  didactic  capabilities,  we  study  the 
possibility  to  graft  mathematics  interactive  learning  environments  onto  them.  We  present  in  this 
paper  our  approach  that  is  centered  on  student  interaction  and  we  illustrate  it  with  LIMITS  a 
learning  environment  that  helps  students  to  study,  work  out  and  justify  limits  of  rational 
functions.  We  describe  three  activities  involved  in  limits  study:  entering  and  selecting  sub- 
expressions, transforming  expressions,  and  calculating  the  limit  by  applying  theorems.  We 
show  how  teaching  goals  have  determined  the  characteristics  of  these  activities. 


Introduction 

Computer  Algebra  systems  (CAS)  such  as  MAPLE,  DERIVE  or  MATHEMATICA  are  powerful  software  tools 
able  to  perform  a large  range  of  calculus.  At  the  same  time,  more  and  more  computers  are  now  available  in 
secondary  schools.  Consequently,  the  question  of  how  mathematics  teaching  can  integrate  this  kind  of  tools  is 
set. 

Some  research  works  pointed  out  specific  phenomena  which  occur  when  students  are  working  with  a CAS 
(Artigue  1997).  Others  focused  on  a long  time  utilization  of  a CAS  with  students  (Lagrange  1996).  These  works 
emphasized  that  the  utilization  of  CAS  in  mathematics  learning  is  not  so  obvious.  The  powerful  calculus 
capabilities  of  CAS  help  students  in  the  technical  part  of  their  work,  but  do  not  automatically  lead  them  to  master 
concepts  they  work  on  (Tall  1996). 

We  explore  a new  way  of  using  CAS  with  students.  CAS  were  not  designed  with  a pedagogical  purpose  and  we 
intend  to  include  them  in  learning  environments.  These  learning  environments  try  to  fit  the  calculus  capabilities 
of  CAS  to  the  students  levels  and  to  provide  them  with  relevant  presentations  and  manipulations. 

In  this  paper,  we  present  the  learning  environment  LIMITS  which  is  based  on  MAPLE  and  allows  the  students  to 
work  on  the  limit  concept.  We  describe  this  environment  and  discuss  about  some  issues  this  work  deals  with. 
The  first  version  of  our  environment  allows  students  to  work  on  limits  of  rational  functions. 


Learning  Limit  Concept 

In  the  paper-and-pencil  context,  working  on  limits  implies  to  carry  out  two  different  tasks.  The  first  one  consists 
in  applying  some  transformations  when  the  limit  evaluation  leads  to  an  indeterminate  form  (such  as  0 x infinity 
or  +infinity-infinity).  The  second  one  consists  in  applying  theorems  and  properties  in  order  to  calculate  the  limit. 
Teachers  note  numerous  difficulties  when  students  carry  out  these  two  tasks,  such  as  calculus  errors  in  the 
transformations  on  expressions  or  misapplication  of  theorems. 

Using  CAS  to  perform  limit  calculus  is  likely  to  help  the  students  to  verify  their  results  and  to  focus  on  the 
strategic  part  of  their  work.  Unfortunately,  CAS  utilization  is  not  so  obvious.  Available  functions  are  sometimes 
too  powerful  and  students  may  have  some  difficulties  in  applying  a precise  transformation  in  a given  sub- 
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expression.  Other  transformations  are  merely  impossible  to  perform  with  a CAS  because  too  simple  functions 
(from  a technical  viewpoint)  do  not  exist  in  a powerful  software.  Furthermore,  CAS  results  can  sometimes  be 
quite  useless  for  students.  For  example,  CAS  can  display  any  limit  values  but  this  indication  does  not  suggest 
any  transformation  to  apply  in  order  to  eliminate  an  indetermination. 

We  think  that  the  design  of  a learning  environment  based  on  a CAS  is  likely  to  solve  difficulties  that  students 
encounter  both  in  paper-and-pencil  and  CAS  context.  Our  approach  consists  in  specifying  clear  pedagogical 
goals  and  in  designing  an  interaction  that  reaches  these  goals.  Students  will  work  with  an  environment  that 
reifies  identified  properties  and  provides  well  suited  functions  and  convenient  calculus  facilities. 

From  now  on,  we  describe  the  LIMITS  environment  and  explicit  the  teaching  goals  we  try  to  meet. 


Architecture 

The  Learning  Environment  LIMITS  is  composed  of  two  units.  The  first  one,  called  the  Interaction  Unit,  interacts 
with  the  student  in  order  to  specify  his/her  request.  The  student  can  enter  an  expression,  request  a transformation 
upon  an  expression  or  a sub-expression  or  carry  out  a limit  calculus. 

On  the  basis  of  the  student's  demand,  the  Interaction  Unit  sends  a request  to  the  Adaptation  Unit.  The  part  of  this 
second  unit  is  to  adapt  the  student’s  demand  to  the  CAS  and  to  collect  the  result  from  the  CAS  before  sending  it 
back  to  the  first  unit  (see  Fig.  1).  These  units  have  been  developed  in  DELPHI  and  the  CAS  we  used  is  MAPLE. 


Interaction  Unit  ** ► Adaptation  Unit 

k i 

L 

Student  Computer  Algebra  System 

Figure  1:  Architecture  of  the  LIMITS  environment. 

We  describe  below  three  activities  required  to  study  limits:  entering  and  selecting  sub-expressions,  transforming 
expressions,  and  calculating  the  limit  by  applying  theorems.  We  show  how  didactic  goals  have  determined  the 
characteristics  of  these  activities. 


Entering  and  Selecting  Expressions 

When  a student  enters  an  expression  using  its  linear  form,  the  Interaction  Unit  parses  this  expression  and 
displays  it  in  its  usual  bidimensional  form  (Fig.  2). 
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Figure  2:  Entering  an  expression  in  LIMITS. 
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The  bidimensional  display  is  achieved  by  using  nested  boxes  which  include  operators  (such  as  +),  variables, 
numbers  or  sub-expressions.  In  order  to  calculate  a limit  or  perform  a transformation  the  student  has  to  select 
expressions  or  sub-expressions.  The  LIMITS  environment  allows  to  select  expressions  or  sub-expressions  with 
the  mouse  directly  on  the  bidimensional  representations  of  expressions.  Our  assumption  is  that  such  a capability 
will  lead  him/her  to  a better  understanding  of  expressions  structures. 


Some  Difficulties 

Different  ways  of  selecting  sub-expressions  were  possible.  We  chose  to  give  a particular  importance  to  root 
symbols  because  their  identifications  are  necessary  to  deeply  understand  expressions  structures.  For  example,  we 
chose  to  select  the  whole  expression  x^2+x+l  by  clicking  on  the  first  or  on  the  second  + symbols. 

Some  situations  were  however  more  difficult  to  solve  than  others.  For  example,  some  operating  symbol  are  not 
displayed  in  the  usual  bidimensional  representation.  That  is  the  case  with  the  multiply  operator  in  3x  for 
example.  We  planed  to  insert  some  space  between  the  3 and  x boxes,  so  that  students  can  select  the  whole 
expression  3x  by  clicking  in  this  space.  Another  difficulty  we  had  to  deal  with  came  from  the  particular 
conventions  concerning  the  minus  symbol.  For  example,  the  real  syntactic  structure  of  the  x^2-3x+2  expression 
is  x^2+(-3x)+2.  We  had  so  to  decide  what  sub-expressions  will  be  selected  when  clicking  on  the  minus  symbol 
of  the  previous  expression.  We  have  in  this  case  two  possible  results,  either  the  -3x  sub-expression  (if  we 
consider  that  the  + symbol  of  the  whole  expression  is  omitted),  or  the  x^2-3x+2  expression  (if  we  consider  that 
this  minus  symbol  is  at  the  same  level  in  the  expression  that  the  plus  symbol).  We  chose  the  second  possibility  in 
order  to  have  a mean  to  select  the  whole  term  in  expressions  such  as  x^2-3x.  A space  between  the  minus  and  3 
symbol  allows  the  user  to  select  the  sub-expression  . 

Another  difficulty  was  to  explain  to  the  user  how  to  select  sub-expressions  with  respect  of  our  particular 
conventions.  We  designed  a little  tool  that  allows  the  student  to  see  what  expression  or  sub-expression  he/she 
points  out  with  the  mouse. 

When  passing  the  mouse  all  over  the  expression  without  clicking,  different  moving  frames  are  drawn,  indicating 
the  different  sub-expressions  that  would  be  selected  by  clicking  (see  Fig.  3). 
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x?-3:@.+-2 
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a|  + 2 

x^-3x+-i 
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Figure  3:  Visualization  of  different  sub-expressions  (using  the  mouse). 


Adaptations  Needed  in  the  CAS 

Expressions  and  sub-expressions  are  sent  by  the  Adaptation  Unit  to  MAPLE  in  order  to  calculate  transformed 
expressions  or  their  limits.  But,  working  with  MAPLE  led  us  to  take  some  precautions. 

For  example,  some  operators  do  not  exist  within  the  MAPLE  representation.  That  is  the  case  with  the  expression 
2/x  which  is  parsed  by  MAPLE  as  2*x^-l,  because  quotient  does  not  belong  to  the  MAPLE  operators  set. 
Another  difficulty  comes  from  the  simplification  algorithm  that  MAPLE  applies  as  soon  as  it  receives  any 
expression.  This  algorithm  is  tedious  because  it  may  eliminate  some  important  characteristics  of  the  expression 
structure.  For  example,  the  sub-expression  3x+5x  is  immediately  simplified  by  MAPLE  in  8x,  whatever  the 
place  in  an  expression  it  takes. 

It  was  not  possible  to  return  to  the  student  expressions  or  sub-expressions  so  deeply  changed  by  MAPLE  that  the 
student  may  not  recognize  the  original  object  he/she  works  on.  Consequently,  we  designed  a tool  within  MAPLE 
which  aims  at  protecting  expressions  from  the  powerful  algorithms  of  this  CAS.  This  tool  encodes  expressions 
as  soon  as  they  are  received  from  the  Adaptation  Unit.  After  working  on  these  expressions,  it  decodes  the 
resulting  expressions  and  sends  them  back  to  the  Adaptation  Unit. 
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Figure  4:  A sequence  of  transformations. 


Transforming  Expressions 

In  order  to  calculate  limits,  students  have  sometimes  to  perform  transformations,  specially  when  the  calculus  of 
the  current  expression  leads  to  an  indeterminate  limit  (such  as  0 x infinity  or  0 divided  by  0).  In  LIMITS,  to 
transform  an  expression,  a student  has  to  select  a sub-expression  and  the  transformation  he/she  wants  to  apply. 
The  figure  4 shows  the  different  transformations  a beginner  can  do  in  order  to  calculate  the  limit  of  a rational 
function.  He/she  first  factors  out  x^2  in  the  denominator,  then  x^2  in  the  numerator,  before  simplifying  the 
whole  expression  by  x^2.  The  limit  of  the  whole  expression  can  then  be  calculated. 


Another  Way  of  Transforming  Expression 

We  aim  at  providing  students  with  convenient  capabilities  for  transforming  expressions  quickly  and  precisely. 
Consequently,  we  designed  another  way  of  transforming  expressions  by  direct  manipulation,  particularly  well 
suited  for  elementary  syntactical  transformation,  such  as  inverting  two  terms  of  a sum  (Avitzur  1998). 

This  way  of  performing  transformations  requires  that  the  student  selects  a sub-expression  with  the  mouse  and 
drags  it  then  over  the  expression.  The  environment  shows  to  the  student  what  possible  transformation  he/she 
would  perform  if  he/she  relaxes  the  mouse  button.  To  do  that,  the  environment  displays  (when  possible)  a label 
pointing  out  the  transformation  corresponding  to  the  place  the  mouse  indicates  (Fig.  5). 
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Figure  5:  By  dragging  x^2  after  x,  the  two  terms  commute. 
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Adaptations  Needed  in  the  CAS 


MAPLE  functions  are  generally  not  well  suited  for  students  work.  For  example,  some  functions  are  too 
powerful.  The  function  expand  works  not  only  on  polynomial  expressions  (as  expected)  but  also  on 
trigonometric,  logarithmic  or  exponential  expressions.  It  will  be  better  from  a pedagogical  viewpoint  to  break 
this  powerful  function  into  several  more  precise  ones  working  on  specific  expression  forms. 

We  also  noticed  that  some  useful  functions  do  not  exist  within  MAPLE,  which  (as  any  other  CAS)  does  not 
require  too  simple  functions  because  its  efficient  algorithms  don't  use  them.  For  example,  no  direct 
transformation  in  MAPLE  rewrites  x^2-3x+2  into  x^2*(l-3/x+2/x^2). 

Consequently,  we  had  to  design  a library  of  specific  functions,  closely  corresponding  to  the  student  actions  and 
respecting  the  types  of  expressions  the  student  is  working  with. 


Calculating  a Limit 

Several  ways  were  possible  to  allow  the  student  to  calculate  limits  of  expressions  in  the  LIMITS  environment. 
We  noticed  that  when  calculating  limits  students  have  often  difficulties  in  applying  theorems  and  properties. 
Consequently,  we  make  students  indicate  the  theorem  they  want  to  apply  before  proposing  any  result  in  LIMITS. 
In  the  example  of  figure  6,  the  student  has  first  selected  the  expression  1 and  determined  its  limit  by  choosing 
"reference  limit".  Then  he/she  has  made  the  same  actions  with  the  expressions  1/x  and  l/x^2.  Finally,  he/she  has 
selected  the  whole  numerator  and  chosen  a sum  theorem. 


Elude  do  Hrnile 


SBE3 


r'"-or"n|: 


Etude  de  la  liinite  de ..  ^ ^ de  ief«ence  i . . 

1— \ Limke  derw  loncikH^  cortr^  I 

^ Unite 

Quotiertt  > Uniite-irif  pixit  c«fleirw;fmpou 


T o»i$  les  tne  limiie  lin« 


lim. 

L.  ......  .....  

lim  — .^0 


lim 


I ■■  X"' 


Figure  6:  A limit  calculus. 


Distinguishing  different  teaching  levels 

Different  levels  of  justifications  corresponding  to  different  teaching  levels  are  available  in  LIMITS  (Fig.  7).  At 
the  first  level,  exhaustive  justifications  are  required,  but  at  higher  levels  some  justifications  can  remain  implicit. 
At  level  2,  making  explicit  the  limits  of  x^2,  x,  and  of  the  constant  terms  is  not  required.  Furthermore,  unlike  at 
the  first  level,  the  sum  justification  is  not  divided  into  several  sub-cases.  The  only  case  is  "sum  without 
indetermination".  The  third  level  includes  a supplementary  choice  allowing  the  student  to  apply  a new  property 
concerning  the  limit  of  a polynomial  function  when  the  variable  approaches  infinity.  Thanks  to  this  property,  the 
calculus  can  be  performed  in  only  one  step. 
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Figure  7:  Different  levels  of  justification 


Conclusion  and  future  work 

In  this  paper,  we  presented  a Learning  Environment  based  on  a CAS.  We  specified  clear  pedagogical  goals  and 
designed  a software  providing  the  student  with  well  suited  calculus  facilities.  We  hope  consequently  that  our 
Learning  Environment  LIMITS  will  prevent  the  student  from  usual  paper-and-pencil  errors  and  will  provide  him 
relevant  calculus  tools. 

We  intend  now  to  realize  some  experiments  in  observing  and  analyzing  some  students  sessions  and  the  errors 
they  will  make.  We  aim  so  at  knowing  how  to  improve  our  interaction  with  the  student,  for  example  by  asking 
him/her  some  numerical  results.  A second  direction  of  work  is  to  extend  the  mathematical  domain  we  deal  with. 
Our  intention  is  firstly  to  design  another  interaction  allowing  the  student  to  calculate  limits  with  the  help  of 
relevant  inequalities.  This  domain  is  a more  difficult  one,  because  some  useful  inequalities  cannot  be  found 
automatically  and  requires  to  specify  what  kind  of  help  to  provide  to  the  students. 
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Abstract:  Hypertext  clearly  has  the  potential  to  change  the  way  students  learn.  However,  the 
World-Wide  Web  lacks  many  of  the  features  necessary  for  truly  interactive  hypertext;  while 
hypertext  pundits  speak  of  students  annotating  existing  pages,  building  new  pages,  and  linking 
them  into  existing  hypertexts,  the  World-Wide  Web  typically  limits  interaction  to  simple  point-and- 
click  browsing.  At  the  same  time,  students  encounter  a quite  different  problem  when  they  venture 
beyond  course  webs:  they  get  “lost  in  hyperspace”.  That  is,  students  have  trouble  determining 
where  they  are  and  where  to  go  next. 

In  this  paper,  we  suggest  one  solution  to  these  two  problems:  Blazer,  a system  for  extending 
the  Web  with  trails,  named  sequences  of  annotated  pages.  When  a student  loads  a page  on  a trail, 
she  sees  not  just  the  page,  but  also  information  about  the  trail.  Trails  can  give  students  increased 
interactivity  while  grounding  navigation. 


1.  Introduction:  Why  the  Web  Needs  Trails 

The  term  “hypertext”  suggests  a rich  environment  in  which  readers  could  not  only  navigate  between  pages,  but  also 
extend  and  modify  their  own  reading  spaces.  Hypertext  visionaries  suggested  systems  in  which  readers  could  add 
their  own  pages,  make  links  between  pages,  and  create  trails  between  pages  (Bush  1945)  (Nelson  1974)  (Landow 
1997).  Consider  Nelson's  first  description  of  hypertext,  in  which  he  writes  “links  may  be  artfully  arranged 
according  to  meaning  or  relations  in  the  subject,  and  possible  tangents  in  the  reader's  mind”  (Nelson  1974,  p. 
DM19,  emphasis  ours).  Similarly,  Bush  envisioned  a convenient  mechanisms  for  constructing  “useful  trails  through 
the  enormous  mass  of  the  common  record”  (Bush  1945,  Section  8).  Clearly,  linking  and  trailblazing  are  essential  to 
interactive  hypertext. 

Compare  this  rich  model  to  the  typical  Web  site.  Readers  can  click  on  links  to  select  the  next  page  they  view, 
but  that  is  the  limit  of  the  opportunity  they  have  to  interact  with  the  hypertext.  While  some  sites  provide  more 
interactivity,  few  provide  the  rich  environment  the  hypertext  pioneers  envisioned.  In  particular,  while  it  is  certainly 
possible  to  link  to  a site,  few  sites  let  you  link  from  them.  But  without  such  links,  how  can  readers  arrange  links 
according  to  their  “meaning  [,]  relations  [...]  and  possible  tangents”? 

Are  there  ways  to  permit  Web  readers  to  make  and  share  their  own  links?  Are  there  ways  for  teachers  to  give 
their  students  appropriate  guidance  for  traversing  the  morass  of  information  on  the  Web?  To  help  students,  teachers 
often  want  to  write: 

To  learn  more  about  this  subject,  read  the  following  Web  pages  in  this  specified  order.  I've 
included  notes  on  each  page  to  help  you  understand  what  is  on  the  page.  I've  also  suggested  a few 
alternate  sequences  for  those  of  you  with  different  backgrounds. 

Similarly,  a teacher  might  wish  to  provide  a sequence  of  readings  and  questions,  with  each  element  building  on  the 
previous  ones.  A teacher  or  student  might  also  trace  the  evolution  of  an  idea  or  design  principle  through  a sequence 
of  pages.  These  trails  may  also  include  a number  of  subtrails  (which  Bush  calls  “side  trails”). 

How  can  a teacher  implement  such  sequences?  It  is  certainly  possible  to  write  a separate  Web  page  that  lists  the 
elements  in  the  sequence  and  links  to  each  element.  However,  such  an  implementation  is  difficult  for  students  to 
navigate,  as  they  must  go  back  and  forth  between  the  sequence  and  linked  pages,  often  forgetting  where  they  are. 
There  are  also  a number  of  ad-hoc  solutions.  For  example,  a teacher  might  build  a sequence  of  framed  pages,  each 
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Experienced  computer  users  preferred  the  “table  of  contents  at  the  left”  or  the  “separate  contents  window” 
interfaces.  Less-experienced  users  preferred  the  simpler  interfaces.  However,  less-experienced  users  also  noted  that 
they  wanted  some  way  to  “jump  around”  the  trail.  Almost  all  users  noted  that  the  “table  of  contents”  interfaces  used 
a lot  of  screen  space.  Because  we  were  testing  on  large  monitors,  they  did  not  find  this  a problem.  However,  many 
worried  what  would  happen  when  used  smaller  monitors. 


Figure  1;  Three  trail  mockups  (Numbered  “Blue  Dot”,  Just  arrows,  and  Framed). 

Many  comments  also  served  to  validate  our  initial  design  goals.  For  example,  when  an  interface  seemed  to 
provide  less  context,  readers  asked  for  “more  context,  as  in  the  other  interfaces”.  Readers  also  wanted  clearer 
next/back  arrows  in  all  but  the  simplest  interface. 

Because  different  readers  find  different  interfaces  more  intuitive,  we  decided  to  include  configuration 
mechanisms  in  the  program.  That  is,  each  reader  can  select  which  way  trails  are  presented  for  her. 

This  decision  also  gives  us  the  opportunity  to  do  somewhat  less  direct  and  more  realistic  user  testing.  In 
particular,  we  can  see  how  (and  whether)  readers  switch  trail  interfaces  as  they  become  accustomed  to  using  trails. 
Such  testing  is  planned  for  spring  2000. 


3.  Architecture 

The  architecture  of  our  system  is  fairly  simple  and  takes  advantage  of  the  Web  Raveler  page-transformation 
infrastructure  (Kensler  and  Rebelsky  2000).  WebRaveler  is  an  account-based  system  that  mediates  communication 
between  Web  browsers  and  Web  servers.  Each  time  a reader  requests  a page,  Web  Raveler  intercepts  the  page 
before  it  is  returned,  looks  up  information  on  the  reader,  and  transforms  the  page  according  to  the  needs  of  the 
reader.  Because  Web  Raveler  supports  a plug-in  architecture,  we  were  able  to  develop  trails  as  a plug-in  for  Web 
Raveler.  (Fig.  2)  illustrates  the  interaction  between  Web  Raveler  and  Blazers 
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of  which  includes  the  remote  page,  notes,  and  a link.  Unfortunately,  the  construction  of  such  solutions  is  often  time- 
consuming  and  difficult. 

We  have  developed  a suite  of  tools  to  make  it  easier  for  faculty  and  students  to  build  and  use  trails  of  pages, 
both  local  and  remote.  The  suite  includes  a variety  of  trail-authoring  tools  intended  for  both  novices  and  expert 
users.  Similarly,  there  are  a number  of  ways  that  students  can  view  trails,  so  that  each  student  can  choose  the  way 
that  best  suits  her.  Trails  have  an  added  benefit:  they  ground  students  in  their  reading  of  the  Web.  Students  on  a 
trail  know  that  there's  always  one  good  place  to  go  next:  the  next  page  on  the  trail.  The  trail  name  also  gives 
students  some  information  as  to  why  they're  reading  the  page.  Finally,  because  our  trails  can  include  notes  from  the 
trail  author,  the  instructor  can  give  additional  context  within  the  notes. 

In  this  paper,  we  discuss  selected  issues  in  the  design  and  implementation  of  our  system,  particularly  our 
observations  during  close  user  testing. 


2.  Using  Trails 

While  it  is  necessary  to  blaze  trails  before  you  can  use  trails,  the  technique  for  blazing  trails  is  somewhat  secondary  to  the 
interface  for  using  trails,  since  trails  will  be  more  frequently  used  than  blazed.  Hence,  our  work  began  with  the  consideration 
of  possible  user  interfaces  for  presenting  trails  to  students. 

Our  work  began  with  an  analysis  of  the  necessary  components  of  any  interface.  We  decided  that, 

• students  must  always  know  their  current  position:  what  trail  they're  on,  where  they  can  go  next,  where 
they  can  go  back  to; 

• students  must  be  able  to  reach  an  overview  of  each  trail; 

• a displayed  page  must  include  not  only  the  page  content,  but  also  the  trail  context  and,  possibly,  notes 

from  the  trail  creator; 

• pages  on  a trail  must  use  standard  URLs;  and 

• each  page  may,  but  need  not,  include  links  to  all  of  the  pages  in  the  trail. 

These  design  guidelines  gave  us  a great  deal  of  freedom  in  the  ways  to  present  these  different  attributes.  There  are 

clearly  a number  of  design  choices  relating  to 

• positioning  of  the  trail  information,  which  might  appear  above  the  page  material,  to  the  side  of  the 
page  material,  or  even  in  a separate  window; 

• the  number  of  links  to  other  pages  in  the  trail,  from  just  two  links  (next  and  previous)  to  a complete 
list  of  links,  or  even  a small  window  on  the  trail; 

• the  level  of  detail  in  the  links  to  the  other  pages,  which  might  give  the  full  name  of  the  other  page,  or 
simply  a position  number;  and 

• the  presentation  of  links  to  other  pages,  which  might  appear  as  a menu  or  series  of  textual  links. 

At  this  stage  of  the  work,  our  goal  was  to  evaluate  how  students  might  react  to  different  presentations  of  the  trail. 

We  began  with  four  mockup  interfaces  that  explored  different  combinations  of  these  choices.  While  it  might  have 
been  more  appropriate  to  test  each  design  choice  independently,  it  was  also  clear  that  some  combinations  were  more 
natural.  For  example,  if  the  trail  is  presented  in  a second  window,  it  makes  sense  for  that  window  to  include  many 
or  all  of  the  nodes  in  the  trail  with  their  name,  and  not  just  the  next  and  previous  nodes  or  numbers  for  each  node. 

The  four  mockups  were 

• A simple  interface  which  included  the  full  list  of  pages  in  the  trail  at  the  top  and  bottom  of  each  page. 

The  links  were  numbers,  giving  the  position  of  the  page  within  the  trail.  To  give  the  reader  some 
context,  we  included  a blue  dot  at  the  current  position  within  the  trail.  The  first  picture  in  (Fig.  1) 
shows  this  interface  with  a course  page. 

• A simpler  interface  which  included  only  trail  name  and  next  and  previous  links  as  arrows.  The  second 
picture  in  (Fig.  1)  shows  this  interface  with  an  institutional  home  page. 

• A framed  interface  which  presented  a textual  “trail  table  of  contents”  to  the  left  of  the  page.  The  third 
picture  in  (Fig.  1)  shows  this  interface  with  a set  of  major  guidelines. 

• A separate  window  listing  the  textual  table  of  contents. 

In  almost  every  case,  the  design  strategy  for  the  mockup  bore  some  resemblance  to  typical  site  interfaces  on  the 
Web.  The  two  simpler  interfaces  resemble  the  lists  of  pages  of  search  results  typically  given  by  search  engines  like 
Altavista.  Similarly,  the  table  of  contents  at  the  left  format  is  a typical  navigation  tool  for  sites,  one  that  is 
sometimes  implemented  with  HTML  tables  and  sometimes  with  frames. 
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Figure  2:  The  Relationship  between  Blazer  and  Web  Raveler. 

A student  using  Web  Raveler  with  trails  first  logs  in  to  Web  Raveler.  She  then  surfs  the  Web  as  normal.  After 
intercepting  the  page  from  the  server,  Web  Raveler  sends  information  to  the  Blazer  plug-in  (6a)  about  the  student, 
the  URL  of  the  page,  and  the  content  of  the  page  to  Blazer.  The  Blazer  plug-in  then  sends  the  URL  to  the  Blazer 
server  (6b)  to  determine  if  the  page  is  part  of  a trail.  The  server  returns  (6c)  either  a list  of  trails  and  associated  trail 
information,  or  a note  that  the  page  is  not  on  a trail.  If  the  page  is  not  on  a trail,  the  plug-in  returns  it  (6d) 
unchanged.  However,  when  the  server  indicates  that  a trail  is  available  for  the  page,  the  plug-in  modifies  the  page  to 
add  information  about  the  trail  and  then  sends  the  page  back  to  Web  Raveler  (6d).  Web  Raveler  may  then  pass  the 
page  through  other  plug-ins  (7)  and  then  returns  the  “trailed”  page  to  the  Web  browser  (8).  The  Blazer  server  uses  a 
simple  database  to  associate  trails  with  each  URL. 

One  unresolved  issue  in  this  architecture  is  how  to  resolve  situations  in  which  multiple  URLs  are  associated 
with  the  same  page.  Many  Web  servers  return  a default  page  when  the  URL  requested  is  a directory.  For  example, 
http://www.math.grinnell.edu/  and  http://www.math.grinnell.edu/index.html  are  the  same  page.  Ideally,  Blazer 
would  add  the  same  trail  for  each  URL.  Unfortunately,  the  overhead  incurred  in  checking  equivalence  of  URLs  is 
currently  quite  great,  so  Blazer  and  Web  Raveler  treat  them  as  separate  pages. 


4.  Blazing  Trails 

While  there  are  some  clear  guidelines  for  how  interfaces  should  present  a trail  (or  at  least  what  interfaces  should 
present),  the  design  of  trail  blazing  tools  is  somewhat  less  specified.  Just  as  different  Web  authors  have  radically 
different  techniques  for  building  pages,  from  “raw  HTML”  to  graphical  editors  like  PageMill,  from  custom 
applications  to  “save  Word  as  HTML”,  so  will  different  trail  authors  have  different  preferences  for  the  ways  in 
which  they  build  trails. 

To  support  different  trailblazing  techniques,  we  use  a standard  file  format.  Just  as  some  authors  write  raw 
HTML,  so  too  can  some  authors  write  “raw  trails”.  Authors  can  include  comments  by  beginning  a line  with  a pound 
character  (#).  An  entry  in  the  file  begins  with  the  URL,  followed  by  a vertical  bar  (|),  followed  by  the  Web  page  title 
as  it  should  appear  to  a person  browsing  the  trail,  followed  by  another  bar  (|)  followed  by  any  comments  that  the 
author  wants  to  share  about  that  particular  Web  page. 

But  few  authors  will  want  to  create  trails  in  this  way.  Hence,  we  chose  to  include  two  other  mechanisms  for 
building  trails.  Trailmaker,  the  authoring  tool,  for  novices,  runs  in  any  Web  browser.  The  tool  presents  two  frames, 
one  on  top  of  each  other.  The  bottom  frame  is  used  for  a preview  of  the  page  to  be  included  in  the  trail,  and  the  top 
frame  shows  the  additional  information  that  will  be  recorded  in  the  trail.  The  top  frame,  used  for  authoring,  includes 
sections  for  the  URL  (which  may  be  dragged  and  dropped  from  the  top  of  the  browser  window),  the  author’s  name, 
page  title,  and  comments.  In  addition,  a preview  of  the  complete  trail  is  shown  in  a separate  Web  browser  window. 
Trails  can  be  built  from  scratch  or  modified  from  existing  trails. 

While  this  interface  is  particularly  appropriate  for  those  whose  primary  Web  experience  is  browsing,  another 
interface  might  be  more  appropriate  for  those  used  to  authoring  Web  pages.  Hence,  we  built  a second  tool  that  reads 
the  links  in  a file  and  turns  those  links  into  a trail.  Since  every  page  in  a trail  includes  both  title  and  comments,  the 
link  text  is  used  as  the  comment  and  the  title  of  the  destination  document  is  used  as  the  title  in  the  trail.  Authors 
accustomed  to  building  pages  with  links  can  use  whatever  system  they  prefer  to  build  a "list  of  links"  page  and  then 
run  this  tool  to  build  their  trails.  This  tool  has  been  successfully  used  to  build  some  very  large  trails,  including  one 
with  over  three  hundred  pages. 

We  tested  the  two  interfaces  on  a variety  of  potential  trailblazers,  including  some  whose  main  use  of  computers 
is  browsing  and  some  experienced  programmers.  Surprisingly,  users  found  little  to  recommend  one  over  the  other. 
Most  found  it  comfortable  to  create  trails  in  either  way. 


5.  An  Application:  Student  Trailblazing 

Why  build  trails?  In  the  introduction,  we  suggested  two  important  reasons:  trailblazing  empowers  students  and  trails 
ground  students,  thereby  reducing  the  chance  of  getting  lost  in  hyperspace.  Yet  trail-blazing  has  other  educational 
uses.  In  particular,  a teacher  might  ask  students  to  create  trails  based  on  a particular  subject.  This  activity  differs  in 
many  ways  from  the  related  “build  a Web  on  this  subject”. 
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First,  students  must  do  more  than  collect  resources;  they  must  also  order  those  resources.  While  hypertext 
encourages  multidimensional  structures,  students  must  also  learn  the  ordering  and  relating  that  is  necessary  for 
coherent  arguments  and  narratives.  Second,  students  must  annotate  these  resources.  Again,  this  requires  students  to 
do  more  than  just  decide  “this  page  is  interesting  and  about  my  topic”.  They  must  explicitly  indicate  how  the  page 
fits  into  their  “narrative  trail”. 

Our  initial  experiments  with  asking  students  to  build  trails  have  suggested  that  students  find  the  task  quite 
challenging.  Since  their  primary  experience  on  the  Web  is  gathering,  and  not  sequencing,  they  find  it  difficult  to 
think  about  how  one  might  sequence  these  pages  that  are  related  by  topic  but  not  by  order. 


6.  A Risk:  Page  Modifiers  from  the  Author’s  Perspective 

While  there  are  many  benefits  to  trail-blazing  tools,  there  are  also  some  significant  concerns  raised  by  the  use  of 
tools  that  modify  pages  on  both  local  and  remote  sites.  Consider  the  case  of  ThirdVoice  (ThirdVoice  1999) 
(Warner  1999),  a browser  plug-in  that  allows  readers  to  add  and  share  annotations.  Although  there  are  many 
appropriate  uses  for  this  tool,  such  as  instructors  posting  notes  on  remote  pages  for  students  to  read,  many  made 
inappropriate  use  of  the  tool,  such  as  posting  links  to  pornography  sites  on  common  locations  like  whitehouse.gov 
and  disney.com.  Even  without  these  abuses,  many  page  authors  were  concerned  that  ThirdVoice  usurped  their 
literal  authoriiy,  permitting  others  to  “modify”  their  pages  (Oakes  1999).  Any  page  modification  tool  must  consider 
such  issues. 

Because  we  rely  on  Web  Raveler  (Kensler  and  Rebelsky  2000)  for  our  infrastructure,  we  are  able  to  provide 
some  support  for  page  authors.  In  particular,  Web  Raveler  includes  a language  that  permits  page  authors  to  limit  the 
modifications  done  to  pages.  Hence,  authors  can  indicate  that  they  do  not  want  trails  added  to  their  pages,  or  limit 
trail  use  to  certain  domains.  If  authors  permit  trails  for  a page,  they  cannot  specify  that  certain  pages  (or  types  of 
pages)  should  not  be  included  in  trails. 

During  early  user  testing,  we  built  some  prototype  sites  using  local  copies  of  remote  pages,  to  support  faster 
access  to  the  locations  in  the  trail.  However,  we  did  not  always  copy  images.  In  one  case,  a subject-generated  trail 
created  some  controversy,  since  the  pages  in  that  trail  included  images  from  another  site.  The  owners  of  that  site 
claimed  that  we  were  attempting  to  “steal”  both  their  pages  and  their  bandwidth.  Fortunately,  we  were  able  to 
convince  those  authors  that  we  were  working  on  a useful  research  project  and  not  attempting  to  create  a-mirror  of 
their  site.  In  subsequent  work,  we  turned  off  remote  access  for  any  locally  generated  trails.  However,  such  issues 
remain  a concern. 


7.  Related  Work 

In  spite  of  the  importance  of  trails  to  the  World  Wide  Web,  there  seem  to  be  surprisingly  few  systems  that  permit 
readers  to  add  links  and  trails  to  pages.  The  most  notable  trail-blazing  system  is  Walden's  Paths  (Shipman  et  al. 
1998)  (Shipman  et  al.  1999).  At  times,  trailblazing  is  subsumed  under  a notion  of  annotation,  as  in  (Roscheisen  et 
al.  1995). 

Our  Blazer  system  differs  from  Walden's  Paths  in  a number  of  ways.  First,  Blazer  provides  more  mechanisms 
for  authoring  trails.  Walden's  Paths  provides  one  tool,  which  has  some  similarities  to  our  novice  interface.  Because 
we  use  a simple,  open,  textual,  trail  format,  it  is  possible  to  design  new  interfaces.  Second,  Blazer  gives  readers 
many  more  opportunities  to  customize  their  display  of  trails  and  have  employed  user  testing  to  develop  this 
interface.  Walden's  Paths  uses  one  fixed  display  that  consumes  a significant  amount  of  screen  space.  Third,  Blazer 
users  standard  URLs  and  informs  students  about  the  availability  of  paths  when  they  view  pages  that  happen  to  be  on 
paths.  Walden's  Paths  assumes  that  students  intentionally  start  on  a path,  and  also  uses  special  URLs  for  the  pages 
on  the  path.  Finally,  because  we  rely  on  Web  Raveler  (Kensler  and  Rebelsky  2000),  we  have  the  opportunity  to  use 
information  about  the  student  in  presenting  path  information  and  to  interact  with  other  Web  Raveler  plug-ins.  For 
example  we  might  automatically  present  a different  path  based  on  information  about  the  student’s  skills  or  browsing 
history. 

Admittedly,  Walden's  Paths  has  some  advantages  over  our  system.  It  has  been  used  much  more  widely  and  has 
been  used  successfully  for  a number  of  different  applications  at  different  levels.  We  hope  to  compare  the 
applicability  of  the  two  trail-blazing  systems  for  similar  activities. 
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Both  Walden's  Paths  and  the  Blazer  system  have  many  advantages  when  compared  to  annotation-based  trails. 
In  annotation-based  trails,  page  annotations  can  include  links  to  the  next  and  previous  pages  on  a trail.  However, 
such  trails  are  clearly  difficult  to  build  and  maintain.  For  example,  inserting  a page  into  an  annotation  trail  requires 
that  you  modify  the  annotations  in  three  different  pages.  While  trails  provide  an  interesting  application  of 
annotations,  a separate  annotation  system  is  clearly  preferable. 


8.  Conclusions  and  Future  Directions 

For  the  Web  to  empower  readers,  it  must  permit  readers  to  do  more  than  follow  links  and  build  pages  that  link  to  and 
never  from  other  pages.  The  Blazer  system  empowers  readers  by  letting  them  build  trails  through  the  Web,  using 
both  local  and  remote  pages.  It  therefore  provides  an  important  resource  for  teachers,  not  only  letting  them  build 
trails  to  help  guide  their  students  through  the  Web,  but  also  providing  context  to  browsing  and  empowering  students 
to  create  their  own  trails. 

Yet  our  work  is  not  done.  While  we  have  done  some  user  testing  for  various  parts  of  the  interface,  we  need  a 
comprehensive  study  of  how  readers  prefer  to  see  trails  (and  forthcoming).  We  also  need  to  study  more  closely  how 
students  and  faculty  will  use  trail-blazing  tools  and  whether  the  assignment  to  “build  a trail”  can  really  make 
students  think  about  linear  structure. 
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Abstract:  Recent  technological  innovations  have  substantially  altered  the  ways  in  which  colleges 
and  universities  deliver  essential  services.  Among  the  academic  units  most  notably  affected  by 
technological  change  has  been  the  academic  library.  This  paper  will  consider  the  impact  of  recent 
and  emerging  technologies  on  the  services  provided  by  academic  libraries  and  on  the  evolving 
needs  students  have  for  library  instruction  in  relation  to  these  new  technologies.  1 will  argue  that 
making  digital  libraries  usable  for  patrons  requires  the  development  of  effective  and  innovative 
forms  of  online  reference. 


INTRODUCTION 

Recent  technological  innovations  have  substantially  altered  the  ways  in  which  colleges  and  universities 
deliver  essential  services.  Among  the  academic  units  most  notably  affected  by  technological  change  has  been  the 
academic  library.  This  paper  will  consider  the  impact  of  recent  and  emerging  technologies  on  the  services  provided 
by  academic  libraries  and  on  the  evolving  reference  and  instructional  needs  of  library  patrons  in  the  new  digital 
environment. 

Since  the  development  of  the  World  Wide  Web  in  the  mid-1990s,  colleges  and  universities  have  been 
undergoing  a major  shift  in  the  nature  of  their  communication  with  their  various  audiences.  Now  college  web  sites 
often  serve  as  gateways  for  information  to  students,  faculty  and  staff,  prospective  students,  alumni,  parents,  and  the 
general  public.  In  addition  to  providing  a platform  for  administrative  functions,  the  web  is  effecting  substantial 
changes  in  the  way  educational  services  are  being  offered.  It  has  become  the  major  vehicle  for  delivering  distance 
education  courses.  Even  locally  within  a campus,  more  and  more  courses  are  offered  either  partially  or  exclusively 
over  the  web,  and  many  other  more  traditional  on  campus  courses  integrate  the  web  as  an  integral  part  of  the  course 
(see,  for  example,  Schulman  & Sims  1 999). 

At  the  heart  of  the  educational  and  research  functions  of  the  university  is  the  academic  library,  and  there  is 
no  part  of  the  university  whose  services  have  been  impacted  by  technological  change  more  than  the  library. 
Technological  innovations  have  been  radically  altering  the  kinds  of  service  academic  libraries  offer  and  the  way 
they  do  business  since  perhaps  the  mid-1970s.  In  order  to  properly  consider  the  impact  of  the  digital  environment 
on  the  kinds  of  services  academic  libraries  provide  and  the  way  they  deliver  these  services,  it  will  be  useful  to  clarify 
the  mandate  of  the  academic  library.  While  the  digital  environment  indeed  poses  new  challenges  that  must  be  met 
and  possibilities  to  strive  for,  the  basic  mission  that  guides  and  informs  our  work  remains  unchanged:  “to  support 
teaching  and  research  by  acquiring  and  preserving  materials  in  all  formats,  and  by  providing  access  to  these 
materials  through  systematic  organization’’  (Jayne  & Vander  Meer  1997: 123).  However  we  redesign  our  services  in 
order  to  deploy  the  capabilities  of  the  digital  library  to  maximal  effectiveness,  we  must  do  so  in  a way  that  is 
consistent  with  the  library’s  overriding  mandate. 

The  potential  of  the  technological  revolution  (often  referred  to  as  the  information  superhighway)  has  been 
touted  for  quite  some  time.  It  is  often  claimed  that  libraries  in  North  America  are  experiencing  a major  shift  in  their 
mandate,  which  traditionally  has  been  to  collect,  maintain,  store,  conserve,  and  preserve  the  printed  artifacts  that 
represent  the  recorded  knowledge  of  our  culture  and  society.  Although  libraries  have  always  responded  to  the 
development  of  other  media  (LPs,  CDs,  videocassettes,  etc.),  talk  of  a substantial-even  a paradigmatic- 
transformation  of  the  very  concept  of  a library  began  with  the  rapid  development  of  computer  and  communications 
technology. 
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The  concept  of  a seamless,  electronic  web  of  information,  universally  available  on  a twenty-four-hour-a- 
day,  seven-day-a-week  (24/7)  basis,  is  regularly  put  forward  as  the  natural  next  step  on  the  evolutionary  ladder, 
inevitably  and  inexorably  replacing  the  traditional  physical  library  in  which  things  are  stored  that  have  concrete 
properties  which  are  described  in  a cataloguing  record.  This  argument  is  made  from  several  quarters,  spanning  all 
types  of  libraries,  although  academic  libraries  are  the  most  common  focus.  To  take  just  a few  examples  from  the 
library  literature,  Herron  (1986)  was  among  the  early  predictors  of  a virtual  library,  making  no  distinction  as  to  type 
of  library.  Day  (1998)  makes  the  argument  with  respect  to  school  (K-12)  libraries,  and  Seiler  & Surprenant  (1993) 
make  it  with  respect  to  academic  libraries.  Denton  & Redmond  (1994)  predict  that  the  days  are  numbered  in  which 
librarians  will  be  needed  as  search  intermediaries,  since  library  “customers”  get  needed  information  by  posting  a 
query  to  a discussion  list  or  by  running  a search  on  an  Internet  search  engine. 

This  sort  of  futuristic  or  utopian-according  to  how  one  chooses  to  view  it— forecasting  may  seem 
increasingly  feasible  because  of  the  wealth  of  material— information,  disinformation  and  everything  in  between— that 
is  available  over  the  Internet  to  anyone  with  a PC.  Library  catalogues  from  all  over  the  world  can  be  searched  via 
the  World  Wide  Web  or  a telnet  connection  from  home.  Even  proprietary  textual  and  bibliographic  databases, 
which  have  customarily  been  available  only  from  an  IP  address  of  a library  that  subscribes  to  them  (or  perhaps  from 
other  PCs  within  a university  campus),  are  now  increasingly  becoming  available  remotely  by  means  of  some  form  of 
password  verification  or  proxy  server. 

REMOTE  ACCESS  AND  INFORMATION  LITERACY 

We  in  the  library  community  can  take  genuine  pride  and  satisfaction  at  the  expanding  level  of  service  we 
can  provide  our  patrons,  service  that  extends  well  beyond  the  walls  of  the  library  into  homes,  offices,  community 
centers,  etc.,  and  beyond  the  traditional  hours  of  library  service,  toward  a 24/7  model.  We  can  not  only  make  our 
catalogues  and  the  databases  we  subscribe  to  available  from  outside  the  walls  of  the  library,  but  our  patrons  can  in 
many  cases  ask  asynchronous  reference  questions  via  e-mail  Ask-A-Librarian  services,  and  some  libraries  have  been 
experimenting  with  providing  synchronous  reference  service  via  a MOO  (Multi  User  Object  Oriented)  environment. 
For  other  examples  of  remote  reference  services,  see  Sloan  (1998).  See  also  the  University  of  Leicester’s  Electronic 
Library  at  http://www.le. ac.uk/li/distance/eliteproiect/elib/,  which  provides  links  to  numerous  sites  offering  various 
types  of  remote  reference,  including  the  Remote  Referencing  Help-Desk  Project  at  the  library  of  the  University  of 
Edinborough.^*^ 

On  the  other  hand,  as  a counterpoint  to  this  optimistic  vision  and  to  forecasts  that  libraries  as  we  know 
them  are  destined  to  wither  away  and  die,  academic  librarians  confront  daily  the  reality  of  students  who  do  not  know 
how  to  define  the  topic  they  are  researching  (beyond,  say,  educational  psychology  or  history)^  much  less  how  to 
break  it  down  into  key  words  using  Boolean  logic,  or  how  to  use  the  controlled  vocabulary  of  a database.  Sloan 
(1998)  notes  that  discussions  of  digital  libraries  tend  to  focus  on  technology  and  information  sources,  rather  than  on 
service.  However,  Sloan  points  out,  “[tjechnology  and  information  resources  on  their  own  cannot  make  up  an 
effective  digital  library”  (Sloan  1 998).  Constant  access  to  data  does  not  by  itself  make  that  data  accessible  and  it 
certainly  does  not  transform  it  into  information  or  knowledge.  Information  is  accessible  to  patrons  only  if  they 
know  at  a minimum  1)  how  to  define  their  information  needs,  2)  how  to  translate  those  needs  into  effective  search 
strategies,  and  3)  how  to  evaluate  the  information  they  locate. 

CHALLENGES  AND  OBSTACLES 

Let  us  consider  some  of  the  issues  that  might  affect  a patron’s  ability  to  define  an  information  need,  to 
develop  search  strategies  and  to  evaluate  information.  For  the  moment,  let  us  assume  as  an  abstraction  a fairly  level 
technological  playing  field,  in  which  we  take  as  a given  that  most  library  patrons  have  easily  accessible  Internet 
connections,  can  negotiate  remote  access  to  proprietary  databases,  have  up-to-date  hardware,  sufficiently  speedy 
levels  of  connectivity,  and  browsers  that  are  capable  of  handling  (and  handling  within  a patron’s  natural  life 
expectancy)  Java,  multimedia,  graphics,  and  whatever  other  features  are  used  in  the  databases  they  need.  Of  course, 


Many  thanks  to  Ilene  Frank,  Danielle  Hinton,  Laura  Kortz,  Patricia  Memnnott,  and  Diann  Smothers,  who  shared  much 
information  on  remote  reference,  in  response  to  a query  I posted  to  the  DIG^REF  Listserv  (see 
http://www.vrd.org/Dig_Ref/dig_refhtml). 
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this  is  a major  abstraction,  and  one,  which  is  not  likely  to  become  a reality  anytime  soon.  Nevertheless,  even  given 
all  this,  let  us  imagine  the  confusion  of  the  novice  patron  when  faced  with  our  impressive  selection  of  bibliographic 
and  textual  databases,  not  to  mention  the,  to  all  intents  and  purposes,  infinite  possibilities  of  the  Internet. 


QUERY  FORMA  TION 

This  patron  may  well  not  realize  that  in  order  to  address  the  specific  topic  in  question,  he  or  she  should  be 
looking  for  articles  on  feminist  interpretations  of  the  works  of  Thomas  Hardy,  rather  than  Victorian  literature,  or  for 
information  on  eating  disorders  in  adolescent  girls,  rather  than  psychology.  Despite  the  naive,  rather  fantastical 
assurance  of  many  people  that  the  answer  to  the  unformed  question  is  “in  the  computer,”  just  waiting  to  extract 
itself,  the  fact  remains  that  defining  the  query  is  the  essential  kernel  of  the  research  process.  Stem  (1999)  notes  that 
regardless  of  how  powerful  and  sophisticated  a search  engine  is,  and  no  matter  how  much  it  can  do  for  the 
researcher,  the  most  difficult  part  of  information  retrieval  is  forming  the  appropriate  question.  He  observes  further 
that  as  yet  “[tjhere  is  no  algorithmic  substitute  for  good  critical  thinking”  (Stem  1999:79-80). 


CHOOSING  DA  TARASES 

Once  the  issue  of  query  formation  has  been  resolved,  the  patron  is  faced  with  the  problem  of  how  to  choose 
the  most  appropriate  resource  from  those  that  are  available.  That  the  differences  between  resources  are  not  apparent 
to  inexperienced  people  is  clear  to  any  academic  librarian  who  has  been  told  by  a student,  in  response  to  a 
suggestion  to  search  a particular  database,  that  he  or  she  has  already  “looked  on  the  computer”  and  hasn’t  found 
anything  relevant.  When  the  librarian  asks  the  student  whether  he  or  she  searched  on  the  Internet  or  on  one  of  the 
library’s  databases,  often  the  student  does  not  know.  Alternatively,  if  the  student  says  that  he  or  she  found  the 
information  by  following  a link  from  the  library’s  list  of  electronic  resources,  the  student  often  does  not  know  which 
database  he  or  she  searched.  Inexperienced  users  (as  well  as  more  experienced  ones  with  complex  questions)  need 
guidance  in  selecting  databases  that  are  appropriate  to  their  needs. 

Often,  in  the  ideal  case,  the  patron  would  have  the  assistance  of  a trained  information  specialist.  Certainly, 
as  will  be  discussed  below,  libraries’  web  pages  can  and  should  walk  patrons  through  the  steps  of  the  research 
process  and  provide  assistance  in  database  selection.  Stem  (1999)  provides  examples  of  navigational  tools  that  will 
point  to  metasearch  indexes,  subject  specific  indexes,  web  sites,  etc.  Still,  as  Buttenfield  (1999)  points  out,  what 
connects  remote  users  with  available  resources  is  a software  interface,  an  interface  that  cannot,  at  this  point,  guide 
users  to  the  sources  that  are  most  relevant  to  their  topic  and  most  suitable  to  their  level.  Nor  can  an  interface  make 
eye  contact  with  patrons,  and  respond  appropriately  to  looks  of  confusion,  disappointment  or  frustration.  Moreover, 
as  Shaw  (1996)  notes  in  her  evaluation  of  the  Internet  Public  Library’s  pilot  project  providing  synchronous  reference 
in  a MOO  environment,  the  librarians  involved  in  providing  the  service  reported  difficulties  that  resulted  from  the 
lack  of  the  feedback  present  in  face-to-face,  or  even  telephone,  communication.  Librarians  observed  that 
information  was  lost  as  a result  of  the  absence  of  feedback  from  voice  inflection  or  body  language. 


CONSISTENCY  OF  INTERFACES 

The  third  issue  we  will  consider  in  relation  to  a patron’s  ability  to  effectively  exploit  the  digital  library  is 
the  consistency  of  the  interface  itself  It  is  the  interface,  after  all,  that  enables  users  to  extract  (or  not,  as  the  case 
may  be)  the  information  they  need,  and,  as  Buttenfield  notes,  for  many  users  “the  interface  IS  the  system” 
(Buttenfield  1999:42).  Since  the  rise  in  public  use  of  personal  computing  applications,  software  has  undergone 
substantial  progression  toward  greater  user-friendliness  and  standardization,  evolving  from  often  arcane  command- 
driven  interfaces  to  point-and-click  G(raphical)  U(ser)  I(nterfaces).  Since  the  advent  of  CD-ROM  databases  first 
made  enduser  searching  feasible,  the  interfaces  for  electronic  databases  have  undergone  similar  progressions.  The 
early  DOS-based  CD-ROM  products  relied  on  a set  of  often  idiosyncratic  and  counter-intuitive  menus  and 
commands  that  were  challenging  even  for  professionals  to  use.  With  the  development  of  the  Windows  operating 
system  and  GUIs  came  the  possibility  of  greater  user-friendliness  and  consistency.  During  the  last  few  years,  we 
have  witnessed  the  mass  migration  of  electronic  databases  to  the  World  Wide  Web,  with  the  possibility  of  even 
greater  consistency,  given  the  properties  of  HTML,  the  easy  identification  of  hyperlinks,  and  people’s  growing 
familiarity  with  the  protocol  of  the  web.  However,  the  potential  of  consistency  remains,  to  a large  extent,  elusive. 
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since  there  are  still  substantial  and  often  contusing  dissimilarities  and  inconsistencies  among  the  products  of 
different  vendors.  Among  the  searching  capabilities  users  have  to  learn  for  each  product  are  what  fields  are 
searchable,  how  Boolean  operators  are  applied,  what  the  wildcard  and  truncation  symbols  are,  whether  or  not  the 
user  has  access  to  the  thesaurus,  and  whether  or  not  nesting  is  possible.  All  of  these  can  vary  considerably  from 
product  to  product. 

If  we  take  just  a brief  look  at  the  interfaces  of  popular  products  from  three  well-known  vendors  (InfoTrac, 
produced  by  Gale,  WilsonWeb,  by  the  H.W.  Wilson  Company,  and  WebSpirs  by  SilverPlatter),  we  can  see  that  in 
each  case  the  search  screens  are  quite  different.  The  terms  that  are  used  to  identify  the  different  types  of  searching 
available  are  different  for  each  case,  and  they  are  not  prominently  defined.  The  user  has  to  do  some  digging  through 
the  online  help  in  order  to  get  information. 

Fig.  1 shows  the  opening  search  screen  for  the  InfoTrac  product,  Expanded  Academic  ASAP.  The  default 
screen  is  a subject  guide.  However,  since  the  user  has  no  access  to  a thesaurus,  he  or  she  must  simply  guess  what 
terms  are  descriptors. 
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Figure  1:  The  default  search  screen  for  the  InfoTrac  databases.  The  arrow  indicates  the  option  available 
from  this  screen  to  limit  the  search  to  articles  with  full  text  and/or  refereed  publications 
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Now  in  the  Wilson  databases,  the  basic  search  screen  does  not  give  the  user  the  option  to  limit  the  search,  as  we  can 
see  in  Fig.  2. 
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Figure  2:  WilsonWeb  Basic  Search  Screen 

Finally,  Fig.  3 shows  the  opening  screen  for  SilverPlatter’s  WebSpirs  databases,  which  has  a completely  different 
look  and  different  options  from  either  InfoTrac  or  Wilson  Web.  The  user  is  presented  with  a Search  Builder,  and  the 
option  to  “Set  Other  Limits,”  neither  of  which  is  defined  anywhere. 
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Figure  3:  The  opening  screen  for  SilverPlatter’s  WebSpirs  databases. 


CONCLUSION 

When  sitting  in  front  of  a computer  at  home  or  in  a dormitory  room  and  visiting  an  academic  library’s  web 
site,  our  user  encounters  an  impressive  array  of  resources  that  extend  the  walls  of  the  library  well  beyond  its 
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physical  limits.  Nevertheless,  as  we  have  seen,  there  are  significant  obstacles  that  confront  our  user,  obstacles  that 
are  all  the  more  significant  for  someone  who  is  under  the  impression  (or  who’s  professor  is)  that  the  challenges  of 
the  research  process  melt  away  in  a brave  new  cyberworld.  As  noted  above,  the  academic  library  has  at  the  heart  of 
its  mandate  the  responsibility  “to  support  teaching  and  research  by  acquiring  and  preserving  materials  in  all  formats, 
and  by  providing  access  to  these  materials  through  systematic  organization.”  To  this,  it  is  appropriate  to  add  a 
teaching  responsibility:  the  responsibility  to  teach  students  how  to  locate,  use  and  evaluate  information  in  all  formats 
for  learning  and  research. 

If  we  truly  want  to  extend  the  walls  of  our  libraries  out  to  cyberspace,  and  to  accommodate  the  researcher 
who  is  doing  a literature  search  at  2 AM,  it  is  essential  that  we  design  library  web  sites  so  that  this  researcher  has  a 
place  to  go  for  answers  when  he  or  she  runs  into  obstacles.  A library’s  homepage  should  be  designed  as  a self-paced 
tutorial.  It  should  provide  convenient  and  easy  ways  to  locate  links  to  all  of  its  services  as  well  as  to  tutorials  on 
how  to  use  them.  We  should  make  it  easy  to  locate  information  on  the  research  process,  from  the  formulation  of  a 
topic,  to  tips  on  how  to  develop  search  strategies,  to  principles  and  procedures  for  documentation.  The  library’s  web 
site  should  guide  users  on  how  to  search  electronic  databases  in  general,  as  well  as  information  on  the  unique 
characteristics  of  each  one.  We  need  to  build  on  the  efforts  of  some  academic  libraries  to  offer  user-friendly, 
synchronous  real-time  reference  that  can  guide  users  on  database  usage  and  selection.  The  library  of  the  University 
of  Edinborough,  for  example,  lists  on  its  web  page  the  scheduled  times  that  librarians  who  are  experts  on  specific 
databases  are  available  for  interactive  reference  (see  http://www.remote.lib.ed.ac.uk/).  Building  a digital  library  that 
is  faithful  to  the  finer  traditions  of  librarianship  requires  not  just  that  we  develop  technology,  but  that  we  deploy  that 
technology  in  order  to  achieve  excellence  in  connecting  people  with  information. 
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Virtual  Study  Groups  (VSG),  an  approach  to  networked 

collaborative  learning 

1.  Introduction 

The  Open  University  of  Catalonia  (UOQ  is  a new  university  model  that  provides  distance  learning 
through  a Virtual  Campus.  With  the  application  of  new  information  and  communications  technologies, 
the  ’Virtual  campus”  becomes  an  alternative  to  traditional  correspondence  education.  Students  are  in 
permanent  contact  with  University  services,  professors  and  other  students  from  their  own  home,  via  a 
Virtual  Campus  and  a personal  computer.  In  the  distance  learning  system  that  UOC  offers,  students 
have  a wide  range  of  institutional  elements  to  stimulate,  guide  and  help  them  during  their  process  of 
leamii^:  counsellors,  tutors,  and  support  centres.  They  also  have  other  common  virtual  spaces  where 
to  pose  and  solve  doubts:  the  community  forum,  a space  for  debates  and  a classroom  forum. 

In  this  context,  one  of  the  student’s  critical  problems  is  the  feeling  of  isolation.  Lacking  every  day’s 
feedback,  contact  with  other  students  and  teachir^  staff,  student’s  motivation  level  can  decrease 
substantially.  Moreover,  not  having  someone  with  whom  to  share  impressions,  difficulties,  and  projects 
around  the  studies,  can  dirniriish  the  desire  for  learning.  During  their  studies,  students  need  a 
considerable  amount  of  support.  They  need  aid  with  the  understanding  of  study  goals,  clarification  of 
doubts,  problem  resolution,  evaluation  activities,  and  keeping  up  the  level  of  motivation. 

From  this  point  of  view,  collaborative  work  could  be  considered  as  a strategic  tool  to  improve  the 
study  conditions  and  the  quality  of  the  learning  context.  In  addition,  collaborative  work  can  be  an 
important  element  of  support  and  motivation  front  of  the  challenges  students  face  at  the  beginning  of 
theirs  university  life.  In  a broad  sense,  through  a process  of  collaboration,  students  could  share  the 
discovery  of  a new  virtual  environment  and  build  together  a new  reality  that  will  surround  them  along 
their  studies.  This  support  between  equals  will  allow  the  construction  of  social  interactions  and  co- 
operative networks  that  will  be  basic  to  break  the  isolation  that  often  implies  distance  learning  systems. 

In  this  particular  scenario,  our  research  focuses  on  co-operative  learning  in  virtual  environments.  Our 
experience  about  VSG  seeks  to  improve  the  formalisation  of  models  of  collaboration  in  virtual  spaces 
of  communication.  The  main  goal  of  this  paper  is  to  describe  the  concept  of  virtual  study  groups 
(VSG)  as  the  key  element  of  our  virtual  learning  system,  as  well  as  a central  piece  of  the  backup  system 
for  keeping  student’s  motivation  high.  A Virtual  Study  Group  is  created  because  students  with  similar 
types  of  needs  form  groups  with  the  overall  purpose  of  supporting  each  other  educational  needs,  and, 
jointly,  collaborate  in  solving  learning  problems. 


2.  Virtual  Study  Group 

Collaborative  leamii^  methods  were  developed  within  the  social-psychological  tradition  based  on  the 
idea  that  students  working  together  are  responsible  not  only  for  their  own  learning  but  for  the  learning 
of  others  students  as  well.  From  this  perspective,  we  seek  a methodological  approach  that  will  allow 
students  give  support  and  advice  to  each  other  as  well  as  increase  the  degree  of  involvement  in  sharing 
knowledge  and  expertise.  Different  modek  and  strategies  have  been  proven  in  the  past  in  pursuing  the 
same  goak.  The  aims  and  work  distribution  of  the  Virtual  Study  Group  are  in  part  similar  to  the  ones 
described  by  Slavin  for  the  group- investigation  (Slavin  et  alt.,  1985). 
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The  main  goal  of  a VSG  is  to  create  a learning  structure  in  which  students  can  study  the  contents  of  a 
subject  matter  in  a better  and  richer  manner  that  they  would  do  otherwise.  We  expect  students  workir^ 
coUaboratively  improve  their  achievement  outcomes.  The  virtual  study  team  is  created  by  the  students 
that  gather  together  with  the  shared  objective  of  helping  each  other  to  find  the  resolution  of  a problem 
generated  around  a specific  learning  problem.  The  concept  of  VSG  claims  for  concretions  related  to 
the  number  of  students  per  team,  group  creation,  working  and  dissolution  rules,  and  procedures  and 
collaboration  strategies. 

For  practical  purposes,  within  the  learning  environment  of  our  Virtual  Campus,  we  define  a Virtual 
Study  Group  as  a group  between  2 and  5 students,  who  have  a reserved  space  in  the  Campus  where 
they  can  communicate  between  themselves  and  swap  doubts,  solutions,  advises,  recommendations, 
suggestions,  notes,  etc.  In  this  learning  environment,  students  have  the  possibility  to  work  together 
even  if  they  don't  meet  at  the  same  space  or  time. 

In  that  regard,  our  previous  experience  shows  us  that  during  the  training  time  and  previously  to  the 
definitive  constitution  of  the  team,  it  is  essential  to  clarify  the  goals  and  the  level  of  commitment  in 
order  to  establish  good  collaboration  practices.  There  is  need  for  putting  in  place  the  correct  strategies 
from  the  very  beginning.  This  allows  students  to  realise  their  strong  or  weak  points,  and  build  a 
positive  attitude  towards  their  role  in  the  new  virtual  study  group.  In  this  work  we  describe  the  useful 
strategies  we  have  put  into  practice  so  far. 

Our  research  aims  to  design  instructional  and  learning  systems  that  could  allow  students  creatir^ 
groups  been  fully  aware  of  the  advantages  and  disadvantages  of  it,  and  of  the  whole  meaning  of  that 
way  of  learning.  Student  centred  models  of  learning  vary  in  their  degree  tutoring  is  centred  on  student, 
that  is  on  the  amount  of  intervention  and  gmdance  provided  by  the  tutor. 

The  VSG  could  be  situated  in  a continuum  that  ranges  from  high  self- management  by  the  students  to 
high  group  control  by  the  tutor.  The  table  below  illustrates  the  different  models  that  could  be 
employed  in  order  to  facilitate  students’  collaborative  learning  in  the  virtual  environment: 


Type 

Level  of 
Tutor’s 
Involvenicnt 

Details  of  process 

1 

High 

Tutors  present  a structure  for  the  teaching/learning  process.  This  includes 
detailed  identification  of  the  purpose  of  each  section,  the  duration  of  each 
session,  detailed  readings  that  the  students  are  required  to  have  read  prior 
to  each  session,  definition  of  the  extension  activities  from  each  session  that 
students  will  be  involved  in  and  a clear  description  of  the  outcomes  that  are 
expected  from  each  session. 

2 

Medium 

Tutors  identify  the  broad  structure  of  the  content  of  study.  They  provide 
an  initial  set  of  readings  and  other  sources  of  information.  Students 
become  aware  of  their  role  in  the  learning  process  and  the  tutors’ 
expectations  of  them.  Throughout  the  teaching/leaming  process,  tutor  is 
not  in  charge  but  takes  notice  of  the  thinking  of  the  students  and  seeks, 
where  possible,  to  incorporate  their  contributions  into  her  or  his  teaching. 

3 

Low 

Tutors  identify  the  broad  focus  of  the  content  of  study.  Students  with  the 
support  and  guidance  (where  appropriate)  of  the  tutors  define  the  initial 
structures  that  they  will  use  in  order  to  approach  the  subject.  Tutors  act  as 
sources  of  expertise  and  mentors  or  facilitators.  Tutors  only  intervene 
where  there  is  a crisis  in  the  process,  this  crisis  may,  for  example,  be  where 
students  step  outside  the  natural  boundaries  defined  by  the  subject  or 
where  there  is  tension  in  the  collaborative  process. 

4 

No 

Students  organise  their  study  in  groups  by  themselves  without  requiring  the 
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involvement 


tutor's  intervention.  That  is,  students  choose  whatever  learning  activities 
suit  better  to  their  learning  needs  and  objectives. 


The  main  benefits  for  the  students  working  co-operatively  in  a virtual  environment  can  be  twofold. 
First  of  all,  co-operative  learning  creates  the  potential  for  cognitive  and  metacognitive  benefits.  On  the 
one  hand,  it  reinforces  and  improves  the  study  and  learning  of  the  subject-matter  contents.  On  the 
other  hand,  it  er^ages  students  in  a situation  that  requires  them  to  make  explicit  both  the  process 
followed  to  carry  out  a leamir^  activity  and  the  strategies  applied  to  resolve  a problem.  Making  these 
facts  available  to  everybody  in  the  group  can  give  rise  to  further  discussion  and  reflection  about  issues 
of  the  learning  process  itself. 

Second,  co-operative  learning  promotes  affective  and  social  benefits.  In  particular,  it  increases  the 
student  interest  and  value  that  gives  to  the  subject  matter.  It  also  increases  positive  attitude  and  social 
interactions  amor^  students,  which  results  very  positive  and  enriching  for  students  with  different 
knowledge  levels  and  others  characteristics. 

Next  section  describes  the  two  experiences  we  carry  out  in  the  subjects  of  Multimedia  and 
Communication  (MiQ  and  Information  Structure  (IS)  in  the  OUC  The  first  of  them  involves  high 
self- management  by  the  students  whether  the  seconded  one  is  highly  conducted  by  the  tutor. 

3.  Initiating  Virtual  Study  Groups  within  the  subjects  of  MiC  and  IS 

Multimedia  and  Communication  at  the  UOC 

Multimedia  and  Communication  at  the  UOC  is  a specific,  transversal  and  compulsory  subject  to  any 
student  that  joins  the  university.  This  subject  has  a double  value  for  the  students:  it  reflects  on 
Information  and  Communication  technologies  and  promotes  the  strategies  and  tools  students  need  in 
order  to  study  in  the  virtual  campus.  This  is  fimdamental  for  searching,  organising,  sharing  and 
communicating  information. 

The  main  objective  is  that  the  students  improve  the  conditions  of  study  by  collaborating  in  the  process 
of  learning  in  order  to  obtain  the  objectives  of  the  subject.  The  creation  of  study  groups  is  suggested  to 
help  themselves  in  the  resolution  of  the  inherent  difficulties  to  the  practical  work  of  the  subject. 

The  characteristics  of  the  subject  are: 

• The  students  have  just  entered  the  university  and  the  Virtual  Campus  is  new  for  them. 

• They  do  not  know  the  implications  of  studying  in  a virtual  environment,  and  they  do  not  know 
how  to  use  its  resources. 

• They  have  probably  never  met  before.  Consequently,  interaction  and  collaboration  level  is 
really  low. 

• Students  lack  strategies  for  virtual  work.  In  addition,  they  show  a reserve  attitude  towards 
helping  others  with  the  resolution  of  problems. 

In  MiC  students  use  the  Campus  v3  group  tools.  For  information  exchange  purposes,  students  access 
two  spaces  called  Forum  and  Debate.  They  also  have  access  to  a workspace  where  to  send  and 
download  files  and  a Unk  to  a web  page  that  can  be  modified  as  they  work  progresses.  This  web  page 
can  also  be  used  as  a tool  to  Unk  the  workspace  files  and  organise  information. 
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The  VSG  main  objective  is  to  support  the  achievement  of  the  subject  goals.  The  group  self-manages 
and  creates  its  own  dynamics.  The  group  is  only  an  instrument  for  giving  to  each  other  support  for 
solving  individual  activities.  The  work  produced  by  and  within  the  group  is  not  evaluated.  The  tutor 
doesn't  propose  concrete  activities  to  develop  in-group.  Dependii^  of  students’  level  of  autonomy, 
tutor  plays  an  important  role  in  helping  the  group  with  study  questions  and  guiding  its  dynamics.  The 
procedure  to  start  a VSG  is  the  following: 

• Creation  of  groups: 

The  tutor  makes  a proposal  in  a face  to  face  meeting,  and  also  in  the  teacher  virtual  board.  For  creatir^ 
groups  students  just  introduce  themselves  using  the  virtual  forum  and  not  a concrete  study  activity  is 
suggested.  Students  exchange  information  and  explain  proposals  in  the  Forum.  Once  a group  is 
created,  one  of  its  members  informs  the  tutor. 


• Cohesion  and  development  of  the  group: 

The  tutor  informs  about  strategies,  clarifies  the  conditions  and  promotes  feedback  During  that  phase, 
students  should: 

Share  experiences,  create  a common  culture,  develop  common  rules  and  construct  frameworks  in 
which  create  a new  reality.  Moreover,  the  group  should  develop  activities  specifically  directed  to 
reinforce  the  interaction  amor^  members. 

Reinforce  group  cohesion  as  a factor  to  increase  internal  solidarity  and  decrease  tensions  and 
problems  between  group  members. 

While  they  consolidate  themselves  as  a virtual  study  group,  they  reach  the  objective  of  helping  each 
other  studying  the  subject.  In  order  to  “pass”  the  subject  of  MiQ  students  have  to  complete  five  lor^ 
evaluation  activities.  Although  these  activities  are  to  be  done  and  presented  individually,  VSG  should 
be  a tool  for  helping  students  with  the  difficulties  that  may  arise  during  the  resolution  of  these 
educational  practices.  In  other  words,  students  do  not  have  a functional  group  goal,  but  to  help  each 
other  to  reach  their  personal  goals  within  the  subject. 


Information  Structure 

In  this  section,  we  describe  the  methodology  followed  to  implement  the  pilot  experience  of  VSG  in  the 
course  of  Information  Structure.  First,  we  describe  the  tool  used  for  its  implementation. 

Based  on  our  previous  experience  with  cooperative  learning  and  problem-solving  in  virtual 
environments  (Guitert,  M.  et  alt.,  1999),  we  chose  the  BSCW  tool  (http:/ / www.gmd.gd/bscw).  BSCW 
is  a tool  for  cooperative  work  that  offers  a range  of  possibilities  such  as:  group  management, 
asynchronous  discussions,  document  management  and  an  automated  daily  workspace  activity  report.  It 
also  provides  a help  which  explains  how  to  use  the  tool  (Bentley  R et  alt.,  1997;  Horstmarm  T.  and 
Bentley  R,  1997).  In  addition,  we  also  provided  the  students  a brief  help  document  with  the  most 
important  functionalities  of  BSCW  so  that  to  facilitate  a faster  and  easier  interaction  and  involvement 
with  the  tool. 

Given  the  above  conditions,  we  propose  a methodology  which  consists  of  a sequence  of  actions  that 
we  believe  necessary  to  realise  the  experience.  The  type,  goals  and  the  way  these  actions  are  performed 
specify  the  organisation,  content  and  structure  of  the  interactions  that  take  place  in  the  shared 
workspace.  The  planning  of  the  actions  is  explicitly  presented  to  the  whole  class  by  means  of  a work 
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schedule  which  describes  the  type,  goal  and  the  timing  of  each  action  planned.  Next,  we  describe  the 
three  main  phases  of  the  approach  followed  to  carryout  the  experience:  mifiating,  fommg  and peffommg 

• Initiatuig:  set  up  a debate  of  a case  about  virtual  study  group 

The  realisation  of  a case  study  constitutes  the  first  action  of  our  approach  and  its  main  goal  is  to  initiate 
the  students  into  the  new  experience.  To  that  end,  a specific  case  is  presented  to  the  students  for 
asynchronous  discussion.  In  fact,  this  discussion  constitutes  an  initial,  practical  step  towards  the 
realisation  of  a group  activity  through  an  exchange  of  opinions  and  ideas,  which  can  further  motivate 
and  push  the  students  to  be  engaged  in  the  real  group  study  afterwards. 

• Forming:  arrange  group  formation 

This  phase  involves  three  main  actions:  introducing,  negotiating  and  normir^.  In  particular,  students 
first  introduce  themselves  by  presenting  information  that  their  classmates  could  use  to  reach  a decision 
about  making  a contact  to  possible  candidate  group  members.  Then,  an  interaction  between  possible 
candidates  begins  in  order  to  form  a group.  The  purpose  of  this  discussion  is  to  reach  an  agreement  of 
the  definite  group  normative  (rules  about  group  functioning,  organization  and  planning  of  the  work).  If 
the  negotiation  phase  has  been  sincere  and  effective,  establishment  of  an  accepted  normative  can  be  an 
easy  and  fast  process. 

• Perfotming;  virtual  study  group  realisation 

After  a group  is  formed  and  its  normative  is  clearly  specified,  it  is  the  right  time  for  performing:  group 
members  should  be  engaged  in  a collaborative  study  and  learning  of  the  subject  matter.  The  first  goal  is 
to  make  students  carry  out  learning  activities  that  include  the  elaboration  and  resolution  of  ass^nments 
related  to  the  concepts  and  notions  of  the  theory  to  be  studied.  The  second  goal  is  to  realise  a learning 
task  (a  small  project);  this  is  the  case  of  a practical  problem-solvir^  whose  purpose  is  to  demonstrate 
that  the  students  have  assimilated  the  conceptual  contents  of  the  subject  matter  and  are  able  to  apply 
them  in  a real  problem. 

Moreover,  the  timing  and  the  order  of  carrying  out  these  learning  objectives  is  a very  important  factor 
for  the  success  of  the  group  study  and  for  obtaining  the  benefits  drawn  from  it. 

Finally,  cooperative  learning  methods  are  most  likely  to  enhance  leamir^  outcomes  if  they  combine 
group  goals  with  individual  accountability.  That  is,  each  group  member  will  be  held  accountable  for 
accomplishing  the  learning  goals  of  an  activity  or  task  For  example,  students  should  know  that  any 
member  of  the  group  may  be  called  on  to  answer  anyone  of  the  group’s  questions.  For  this  reason,  to 
increase  and  make  explicit  students  responsibility  in  group  study,  a particular  workspace,  called 
Reflection/ revision  of  group  work,  has  been  designed  and  put  at  the  disposal  of  all  groups.  This  utility 
gives  students  the  chance  to  be  involved  in  follow-up  reflection  activities,  which  manifest  and  clarify 
the  way  students  study  in  group,  contribute  to  the  group  and  leam.  Makii^  this  explicit,  it  helps 
students  critique  and  revise  the  process  followed  and  thus  it  proves  valuable  as  a way  of  engaging 
students  in  meaningful  leamir^. 


4,  Reflections  from  the  experience 

We  are  at  the  initial  phase  of  experiencir^  with  VSG.  The  research  we  are  doing  is  based  on  a 
methodological  procedure  of  Research  - Action  and,  consequently,  we  are  taking  decisions  while  the 
experience  is  going  through,  paying  a special  attention  and  reflectir^  on  the  following  elements: 
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Group  Objectives:  What  is  the  objective  of  the  group.  It  has  to  be  concrete  and  precise  and 
should  imply  a concrete  formalisation  level  shared  by  all  members  of  the  group. 

Level  of  tutors  and  students  involvement:  Depending  on  the  group  objective,  tutor’s  role  may 
change:  managing,  democratic,  making  easy,  etc.,  which  at  the  same  time  will  imply  a student's  role, 
more  or  less  self- managing. 

Cohesion  level  of  the  group:  The  commitment  level,  the  intensity  of  the  interaction  and  the  type 
of  communication  can  influence  or  determine  the  achievement  of  the  group  goals. 

- Work  methodology:  In  a virtual  work  context,  the  notions  of  planning,  task  distribution,  the  tool 
used  and  the  way  it  is  used  are  the  key  elements  for  efficient  group  working. 

Effects  on  the  learning  process:  Since  improvement  of  learning  and  achievement  of  better 
outcomes  are  key  elements  for  every  student,  it  is  necessary  to  investigate  how  virtual  study  groups 
affect  and  promote  effective  learning.  For  this  reason,  it  is  important  to  keep  in  mind  the  following 
influencing  factors:  motivation,  satisfaction,  support,  accomplishment  of  learning  objectives. 
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Abstract:  This  paper  proposes  a comprehensive  model  for  editing  and  exploiting  curriculum  based  ITS,  relying 
strongly  on  visual  information  properties.  In  this  system,  a curriculum  is  a capability  transition  network  that 
consists  of  two  kinds  of  nodes:  capability  nodes  representing  teaching  outputs  and  transition  nodes  organizing 
tutoring  activities.  The  bridge  between  capability  nodes  and  transition  nodes  is  the  prerequisite  links  from 
capability  nodes  to  transition  nodes  and  the  output  links  from  transition  nodes  to  capability  nodes. 

Based  on  the  domain  capability  transition  network  built  by  a curriculum  author,  a learner  can  visually  view  the 
domain  knowledge  structure  and  establish  learning  goals.  The  system  then  creates  multiple  alternative  paths, 
each  permitting  the  achievement  of  the  learner’s  learning  goals.  After  the  learner  chooses  his/her  preferred 
course  to  learn,  the  system  dynamically  identifies  necessary  tutoring  activities  and  orders  these  activities 
according  to  the  current  context. 


1.  Introduction 

A curriculum  is  an  essential  component  of  recently  proposed  ITS  (Intelligent  Tutoring  Systems) 
architectures  (Gauthier  89,  Frasson  et  al.  92,  Gecsei  & Frasson  94,  Nkambou,  96,  Guo  et  al.  98a,  Zhou  et 
al.  96).  In  essence,  it  is  a structured  representation  of  all  kind  of  knowledge  needed  in  the  course  of  a 
tutoring  session  ( Gagne  92,  Merrill  87,  91,  93,  Finch  86).  In  their  work,  the  curriculum  becomes  the 
prominent  part  of  the  ITS. 

The  traditional  GUIs  (Graphic  User  Interfaces)  are  not  well  adapted  to  the  exploitation  of  complex 
information  structures  such  as  a curriculum.  Visualization  technology  (Gershon  & Eick  95)  (Gershon  & 
Brown  96)  points  to  interesting  alternatives.  Curriculum  authors  and  students  alike  now  require  much  more 
understanding  of  the  information  structures  they  are  manipulating. 

This  paper  describes  interactive  visual  ways  of  guiding  learning  process  including  establishing  learning 
goals,  creating  alternative  courses  for  the  learning  goals,  and  dynamically  guiding  the  learning  process.  In 
the  first  part  we  briefly  describe  the  actual  structure  of  a proposed  curriculum  model.  In  the  second  part  we 
address  specifically  the  visual  properties  for  guiding  learning  process. 


2.  Representation  and  Organization  of  the  Curriculum 

This  section  briefly  introduces  a model  for  organizing  various  types  of  knowledge  in  a curriculum.  In  this 
model,  we  combine  Gagne’s  capability  and  instructional  event  theory  with  Bloom’s  objective  theory  to 
define  a kind  of  transition  nodes  that  organize  tutoring  activities  to  achieve  domain  capabilities  (Guo  98). 

2.1.  Tutoring  Outputs:  Capabilities  and  Capability  Levels 

Based  on  the  analysis  of  human  learning  processes,  Gagne  classified  human  knowledge  into  following  five 
kinds  of  capabilities:  Intelligent  Skills,  Verbal  Information,  Cognitive  Strategies,  Attitudes,  and  Motor 
Skills  (Gagne  et  al.  85,  92,  & 93).  The  usability  of  such  classification  lies  in  that  different  teaching 
strategies  can  be  used  to  teach  them  respectively.  This  theory  is  the  foundation  of  our  further  discussion. 


Capability  Levels 


The  capabilities  defined  by  Gagne  point  to  the  general  goals  of  topic  teaching.  In  fact,  teaching  and 
acquisition  of  any  capability  contain  some  gradual  sub-processes.  For  example,  in  the  context  of  a course 
on  Web  page  design,  “Recognizing  a Web  page’*  is  a capability.  Some  gradual  sub-processes  to  teach  the 
capability  may  be  (1)  teaching  the  Web  definition,  (2)  teaching  basic  Web  features,  and  (3)  teaching 
advanced  Web  features.  Corresponding  to  these  sub-processes,  the  mastered  degree  to  the  capability  may 
be  divided  into  three  incremental  levels  including  (1)  stating  Web  definition,  (2)  stating  basic  Web  features, 
and  (3)  stating  advanced  Web  features. 

Our  model  associates  tutoring  activities  with  mastering  levels  of  capabilities  and  students  can  choose  any 
specified  capability  level  as  their  desired  learning  goal. 


2.2.  Tutoring  Activities:  Transition  Nodes 

A major  task  in  curriculum  modeling  is  to  organize  tutoring  activities  effectively  to  support  the 
achievement  of  teaching  outputs.  We  define  a kind  of  transition  node  for  this  purpose,  which  combines 
well-known  instructional  theories  from  Gagne  and  Bloom.  Bloom  (Bloom  69,78)  proposed  six  levels  of 
teaching  objectives,  including  acquisition,  comprehension,  application,  analysis,  synthesis,  and  evaluation. 
These  objective  levels  can  be  used  to  define  various  educational  levels  and  position  sub-processes  of 
capability  acquisition  at  a certain  educational  level. 

Gagne  (Gagne  et  al.  85,92)  defined  nine  instructional  events  for  capability  transition.  They  are  (1)  gaining 
attention,  (2)  informing  the  learner  of  the  objective,  (3)  stimulating  recall  of  prerequisite  learned 
capabilities,  (4)  presenting  the  stimulus  material,  (5)  providing  learning  guidance,  (6)  eliciting  the 
performance,  (7)  providing  feedback  about  performance  correctness,  (8)  assessing  performance,  and  (9) 
enhancing  retention  and  transfer.  This  theory  outlines  detailed  sub-processes  of  human  teaching.  Gagne 
uses  these  events  for  individual  capabilities. 

In  order  to  support  the  acquisition  of  multiple  level  and  aggregated  capabilities,  we  use  Bloom’s  objective 
level  theory  to  organize  tutoring  units.  For  each  objective  level,  Gagne’s  instructional  event  theory  is  a 
good  way  to  organize  sub-units.  Our  transition  nodes  are  defined  based  on  both  theories. 

• Transition  nodes 

A transition  node  is  a sequence  of  tutoring  units  (Fig.  1 ).  Each  tutoring  unit  consists  of  a sequence  of  sub- 
units and  relates  to  certain  output  capability  levels.  These  units  correspond  to  and  are  not  limited  to 
Bloom’s  objective  levels.  Each  sub-unit  contains  one  or  more  groups  of  resources  and  is  where  the  actual 
teaching  is  taking  place. 


Strategy  C:  Gagne’s  Instructional  Events 

► 

Figure  1 Organization  of  a transition  node 
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One  activated  condition  of  a transition  node  is  that  all  its  prerequisite  capabilities  be  mastered  at  the 
required  levels.  When  a tutoring  unit  is  activated,  a series  of  interactive  activities  takes  place.  The  output  of 
these  activities  is  the  acquisition  of  some  output  capability  levels. 

The  Fig.  2 is  an  example  of  transition  node  for  teaching  “Stating  what  is  a Web  browser”.  This  transition 
node  consists  of  three  tutoring  units  that  are  ‘Teach  browsers’  description”,  “Teach  browsers’  job”  and 
“Teach  popular  browsers”.  Each  unit  contains  a sequence  of  sub-units.  The  prerequisite  capability  of  the 
transition  node  is  “stating  Web”  with  3 mastered  levels:  “stating  Web  description”,  “stating  basic  Web 
features:,  and  “stating  advanced  Web  features”.  The  output  capability  of  the  transition  node  is  “stating  Web 
browsers”  up  to  3 levels:  “stating  description  of  browsers”,  stating  Job  of  browsers”  and  “stating  popular 
browsers”. 


In  a transition  node,  each  tutoring  sub-unit  contains  one  or  more  groups  of  didactic  resources  that  consist  of 
various  organized  media,  simulation  programs  and  guidance  programs. 
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Figure  2 Example  of  transition  node 


Figure.  3 Capability  transition  network  for  teaching 
apart  of  UML 


2.3  Capability  Transition  Networks 

A curriculum  author  can  use  prerequisite  links  to  establish  the  relationships  from  capability  nodes  to 
transition  nodes  and  use  output  links  to  associate  transition  nodes  with  output  capabilities.  Such  constructed 
network  is  known  as  a domain  capability  transition  network,  simply  called  Tnet: 

The  Fig.  3 is  an  example  of  capability  transition  network  for  teaching  a part  of  UML.  In  the  example,  there 
are  four  transition  nodes  (big  nodes  in  Fig.  3):  “analyze  Scenarios”,  “analyze  Use  Cases”,  “analyze  Objects 
(temporal)  and  “analyze  Objects  (spatial).  The  four  transition  nodes  support  the  achievement  of  four 
capability  nodes  (small  nodes  in  the  Fig.  3)  respectively:  “define  Scenarios”,  “Identity  Use  Cases”,  “create 
Sequence  Diagrams”  and  “create  Collaboration  Diagrams”. 


3.  Visual  Guidance  in  Learning  Process 

The  originality  of  the  visual  representation  in  our  curriculum  model  lies  in  the  visual  properties  associated 
with  the  various  graph  components.  These  properties  are  highly  dynamic,  changing  for  instance  with  a 
student’s  learning  progress  and  with  alternative  information  requests. 


3.1.  Visually  Declaring  Known  Capabilities  and  Establishing  Learning  Goals 

In  our  curriculum  model,  a learning  goal  is  a specific  level  of  a capability.  For  example  the  capability 
(shown  in  the  Fig.  4)  “Applying  Links  in  HTML”  contains  four  incremental  levels:  “creating  links”. 
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“linking  local  documents”,  “linking  other  Web  documents”  and  “linking  specific  place  within  a document”. 
A learner  can  choose  any  one  of  the  levels  as  one  of  his/her  learning  goals. 


Applyini 

g Links  in  HTML 

Creating 

Links 

Linking 

LocalDocuments 

Linking 

OtherWebDocuments 

LinkingSpecificPIace 

WithinADocument 

Figure  4 A capability  in  HTML  curriculum 

In  order  to  help  students  declare  their  known  capability  levels  and  establish  their  learning  goals,  the  system 
defines  a group  of  rules  for  dynamically  infer  students’  current  knowledge  states. 

Another  group  of  rules  is  defined  for  setting  learning  goals.  An  example  for  setting  a goal  capability  and  its 
levels  is  as  follows.  Let  C be  the  overall  cell  of  a capability  icon,  L be  the  ordered  set  of  all  level  cells  in  the 
capability,  Li  (/=  7, ...,«)  be  a level  cell,  and  Lg  be  the  index  of  the  goal  level  in  the  capability  icon,  one  of 
the  rules  for  the  state  transformation  of  goal  capabilities  is  defined  as: 

Rg5:  if  C is  not  a goal  & 

Li  (/  < n)  is  clicked  & 

C.state  ==  CAP_UNCONCERNED 
then  C.state  CAP_NOTHiNG_MASTERED  & 

L/.state  LEVEL_GOAL  & 

for  all ij  <=  n,  Ly.state  LEVEL_NOT_MASTERED. 


3.2  Visualizing  Alternative  Courses 

In  order  to  individualize  domain  capability  transitions,  we  define  a course  as  a sub-network  of  the  domain 
capability  transition  network,  in  which  only  the  selected  learning  goals  (i.e.  some  capabilities  and  levels) 
and  some  necessary  prerequisite  capabilities  are  included.  Our  curriculum  model  provides  an  approach  to 
create  alternative  courses  for  the  same  group  of  learning  goals  (the  details  of  course  generation  will  be 
discussed  in  a separated  paper.). 

As  soon  as  the  learner  selects  a course,  the  system  assigns  the  selected  course  as  the  learner’s  current 
course.  With  the  help  of  tutoring  delivery  module  in  ITS,  the  learner,  then,  can  leaner  the  course. 

The  following  Fig.  5 presents  an  example  of  creating  multiple  alternative  courses  based  on  the  previous 
Tnet  in  the  Fig.  3.  A learner  sets  the  first  level  of  the  output  capability  "create  Sequence  Diagrams"  as  his 
learning  goal.  The  learner  can  choose  one  of  the  four  alternative  courses  as  his/her  current  course  to  learn, 
according  to  the  information  about  effort,  resource  types  and  involved  sub-goals  provided  by  the  system. 


3.3  Sequencing  Tutoring  Activities  by  State-Driven  Reasoning 

In  order  to  identify  the  availability  of  a tutoring  unit  in  a learning  process,  we  define  five  kinds  of  states  for 
transition  nodes,  tutoring  units  and  sub-units,  which  are  recommended,  enabled,  partlylEnabled,  disabled 
and  passed.  If  a unit  or  sub-unit’s  state  is  recommended,  it  is  the  best  tutoring  unit  or  sub-unit  for  the 
learner  to  learn  immediately.  If  its  state  is  enabled,  it  is  not  the  best  tutoring  unit  or  sub-unit  though  it  can 
be  learned  currently.  If  its  state  is  pardyEnabled,  only  parts  of  the  unit  or  sub-unit  may  be  learned.  If  its 
state  is  disabled,  the  learner  cannot  learn  the  unit  or  sub-unit  immediately.  If  its  state  is  passed,  the  unit  or 
sub-unit  has  been  passed  successfully  by  the  learner. 

• States  of  Transition  Nodes 
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We  define  19  rules  to  identify  the  states  of  tutoring  units  and  sub-units,  in  which,  for  example,  let  (/  be  a 
tutoring  unit,  the  rule  No.  5 is 

Ra5:  if  U Is  enabled  & U and  the  last  executed  unit  have  the  same  parent  goal,  then  U is 
recommended. 

Based  on  these  rules,  the  system  dynamically  refers  the  current  state  of  teaching  activities  and  guides  the 
learner  with  rich  visual  properties. 


Figure  5 Four  alternative  UML  courses 
• State  of  Capability  Nodes 

The  states  of  capability  nodes  reflect  the  mastering  degree  of  domain  knowledge  of  a learner.  We  define 
two  groups  of  states  for  capability  nodes  and  their  levels  respectively,  in  which  each  group  consists  of  five 
states:  UNCONCERED,  NOTHING_MASTERED,  PARTLY_MASTERED,  GOAL_MASTERED,  and 
FULL_MASTERED.  Based  on  the  evaluation  to  the  results  of  teaching  activities,  the  system  uses  a group 
of  rules  to  dynamically  change  the  states  of  capabilities  and  their  levels.  Meanwhile,  a group  of  visual 
properties  related  to  capabilities  and  their  levels  are  used  to  guide  the  learners. 


Conclusion 

The  paper  focuses  on  the  visual  navigation  for  learning  processes  based  on  a curriculum  model  proposed. 
The  model  supports  aggregated  domain  capabilities  so  that  it  simplifies  the  overall  system  structure.  Its 
visualization  properties  and  the  integration  of  well-known  instructional  theories  achieve  a practical  and 
useful  application  of  these  theories. 

A student  can  use  various  visual  properties  to  establish  his/her  learning  goals.  The  system,  then,  creates  and 
displays  multiple  alternative  paths  to  achieve  these  learning  goals.  In  the  learning  process,  the  system 
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dynamically  catches  and  displays  the  learner’s  current  states  and  displays  the  next  recommended  step.  The 
proposed  approach  is  implemented  as  a Java  application  called  VITCAM  {Q\xo  et  al.  98b). 
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Abstract;  This  paper  describes  a distance-based  collaborative  process  undertaken  to  evaluate 
an  asynchronous  web-based  course  designed  to  teach  web  design  skills  to  students  enrolled  in 
a distance-based  Master’s  program.  Several  forms  of  collaboration  were  used  to  allow  two 
expert  reviewers  to  share  their  opinions  on  the  course  without  actually  meeting  face  to  face. 
Comments  and  suggestions  were  passed  between  the  reviewers  and  the  course  designer, 
culminating  in  a comprehensive  review  document  containing  aggregated  suggestions  for 
revision  of  the  course.  Although  the  process  proved  successful,  care  should  be  taken  when 
undertaking  such  a process  to  avoid  stumbling  blocks  brought  about  by  the  psychological 
distance  involved. 


Introduction 

The  purpose  of  this  paper  is  to  describe  a distance  collaborative  evaluation  method  employed  to  analyze  an 
asynchronous  web-based  course.  This  course  is  offered  through  the  Instructional  Technology  department  at 
Virginia  Tech  as  part  of  a distance  education  Master's  program.  An  initial  Expert  Review  of  the  course  was 
commissioned  by  the  Virginia  Tech  Instructional  Technology  department  as  part  of  an  assignment  for  a 
graduate  course  in  product  evaluation.  A first  year  doctoral  student  from  Virginia  Tech  followed  specific 
guidelines  provided  by  the  department  to  complete  the  review  process.  Several  leading  instructional  design 
models  were  utilized  to  conduct  this  initial  review  and  will  be  described  below.  After  completion  of  this  initial 
review,  a second  doctoral  student,  this  one  from  the  University  of  Georgia,  reviewed  both  the  web-based  course 
and  the  first  reviewer's  comments.  The  cumulative  review  of  the  first  two  students  was  then  passed  on  to  the 
course  developer.  The  course  developer  added  his  own  comments  and  incorporated  student  feedback  into  this 
collaborative  review.  All  of  the  information  was  then  re-circulated  and  a consensus  reached  that  incorporated  all 
the  information  into  a collective  list  of  evaluative  recommendation.  All  correspondence  between  the  three 
reviewers  was  accomplished  using  a variety  of  distance-based  collaborative  processes,  including  phone,  email 
and  videoconferencing.  The  background  behind  the  initial  online  course  offering  will  now  be  discussed. 


Background 

With  the  recent  adoption  and  mandate  of  the  Virginia  Standards  of  Learning  (SOLs),  k-12  instructors 
throughout  the  state  are  now  required  to  take  course  work  for  re-certification  that  demonstrates  a set  of  core 
technology  competencies.  In  efforts  to  support  this  statewide  endeavor,  Virginia  Tech  developed  and 
implemented  an  Instructional  Technology  Distance  Education  Masters  Program  (ITMA).  This  online  program 
consists  of  13  unique  ITMA  modules  that  can  be  taken  independently  of  each  other  as  needed  to  satisfy 
Virginia  SOL  re-certification  requirements,  or  in  total  toward  the  Virginia  Tech  Master's  degree  program.  The 
instructional  delivery  methods  Virginia  Tech  utilizes  employ  an  assortment  of  strategies  such  as  intensive 
workshop-style  courses  offered  within  individual  school  districts  (on-site);  interactive  television  courses  offered 
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through  a number  of  state-wide  facilities;  web-based  courses;  week-long,  on-campus  summer  courses;  open- 
studio  development;  and  individual  computer-based  programs. 

Our  evaluation  concerned  itself  specifically  with  ’’Module  4:  Web  Site  Development”  (available  at 
http://psyche.ed.vt.edu/module04/).  The  intent  of  this  module  is  to  provide  learners  with  basic  HTML  design 
skills  using  Netscape  Composer,  such  that  they  develop  a 7-8  page  standalone  web  site.  Should  the  learners  not 
be  taking  this  module  independently,  but  as  part  of  the  overall  Virginia  Tech  ITMA  program,  their  product  will 
be  integrated  into  a larger  electronic  portfolio.  There  were  51  students  who  participated  in  the  first  iteration  of 
the  module.  These  students  were  divided  into  three  cohort  groups  based  on  their  geographic  location  within  the 
state  of  Virginia.  They  constituted  a diverse  mixture  of  elementary  and  middle  school  teachers,  librarians,  and 
instructional  technologists  in  the  public  school  system.  The  initial  Expert  Review  evaluation  process  will  be 
discussed  next. 


The  Initial  Review  Process 

The  overall  model  of  the  Expert  Review  was  patterned  after  the  procedure  outlined  in  the  book  Planning  and 
Conducting  Formative  Evaluations  (Tessmer,  1998).  Tessmer  delineates  the  steps  and  values  behind 
conducting  the  following  types  of  reviews:  One-to-One,  Expert  Review,  Small  Group  Evaluation,  and  Field- 
testing.  The  Expert  Review  conducted  on  Module  4 focused  specifically  on  the  following  areas:  General 
product  description,  instructional  components,  instructional  contexts  and  functions,  and  overall  instructional 
message  display.  A brief  description  of  these  areas  will  now  follow. 

The  general  product  description  delineated  the  developers,  clients,  module  objectives,  learning  environment  and 
media  characteristics  incorporated  in  Module  4.  The  ’’instructional  components”  section  evaluated  the  overall 
structure  of  the  module  from  a Walt  Dick  and  Lou  Carey  instructional  design  perspective  (1996).  In  addition, 
several  other  ID  research  theories  were  employed  which  will  be  discussed  later.  The  instructional  context  of  the 
module  was  examined  to  evaluate  the  synergism  between  all  the  factors  influencing  the  appropriateness  of 
context  chosen  and  under  what  functions  it  was  intended  to  facilitate  instruction.  The  overall  message  display 
was  additionally  evaluated  with  respect  to  the  following:  layout,  color,  density  of  information,  perceived  units 
of  information,  rate  of  information  presentation,  and  appropriateness  of  images  and/or  audio  used.  Now  that  an 
understanding  of  evaluation  components  has  been  described,  an  elaboration  of  the  ID  research  models 
employed  will  be  discussed. 

To  evaluate  the  overall  module  from  a ’’Big  Picture”  perspective  the  Walt  Dick  and  Lou  Carey  model  was  used 
(Dick  & Carey,  1996).  The  following  components  were  examined  in  the  module  for  their  inclusion  of 
appropriate  sub-component  characteristics:  Introduction,  activity,  practice  & feedback,  review,  assessment  and 
transfer.  Within  this  overall  structure,  the  individual  instructional  objectives  were  examined  using  the 
methodology  cited  in  Sullivan  & Higgins  1983  book:  Teaching  for  Competence.  Thus,  instructional  objectives 
were  examined  with  respect  to  observable  and  measured  behaviors,  with  proper  attention  to  conditional  givens 
and  standards. 

The  work  of  Tessmer  & Richey  (1997)  was  used  to  classify  and  evaluate  the  module  with  respect  to  the 
appropriateness  of  context  and  its  function  within  the  overall  design  stages  of  orienting,  instructional,  and 
transfer  (Tessmer  & Richey,  1997).  Module  4 employed  the  contexts  of  ’’Real”  and  ’’Tutorial”  appropriately. 
J.M.  Keller’s  ARCS  model  of  instructional  design  was  utilized  to  evaluate  the  module  from  a motivational 
perspective,  attempting  to  identify  if  the  following  strategies  were  implored:  Attention,  Relevance,  Confidence, 
and  Satisfaction  (Keller,  1987).  Finally,  the  work  of  Gagne  and  Driscoll  was  utilized  to  measure  the 
appropriateness  of  instructional  objectives  to  specific  tasks  or  lessons  at  hand  (Gagne  & Driscoll,  1 998).  Now 
that  the  research  behind  the  initial  Expert  Review  has  been  presented,  the  process  of  the  distance  collaborative 
Expert  Review  model  will  be  discussed. 


The  Collaboration  Process 
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The  original  Expert  Review  (ER)  paper  was  circulated  to  the  second  distant  expert  reviewer.  The  second 
reviewer  then  visited  the  Module  04  web  site,  reviewed  the  ER  paper,  and  evaluated  the  initial  reviewer’s 
criteria  against  the  originally  commissioned  guidelines  and  the  actual  module,  inserting  comments  into  the 
original  ER  paper  using  a different  colored  font.  The  updated  ER  paper  with  comments  was  then  circulated  to 
the  module  developer  and  the  initial  reviewer. 

Next  the  developer’s  implementation  perspectives  were  incorporated  as  a separate  section  at  the  bottom  of  this 
evaluation  paper,  describing  the  technical  difficulties  encountered  during  the  module's  first  field  test,  including 
student  impressions  of  Module  4’s  effectiveness.  This  updated  document  was  then  sent  to  both  reviewers  who 
additionally  commented  on  the  developer’s  implementation  analysis.  Finally,  having  now  seen  both  sets  of 
Expert  Review  comments  in  a single  document,  the  developer  offered  his  impression  of  the  expert  reviews. 
Needless  to  say,  this  asynchronous  distance  evaluation  approach  involved  several  iterations  of  circular  review, 
providing  a thorough  analysis  of  the  module  from  an  ID  Expert  Review  perspective. 

The  techniques  employed  in  the  collaboration  process  utilized  several  forms  of  distance  communication.  The 
review  document  was  created  in  Microsoft  Word  and  passed  between  reviewers  via  email  attachment. 
Discussion  and  debate  between  the  reviewers  took  place  using  email,  teleconferencing,  and  videoconferencing. 
The  important  realization  to  come  out  of  this  process  is  that  without  in-person,  face-to-face  contact  there  is  a 
higher  probability  for  miscommunication  to  occur  between  participants,  especially  when  those  participants  have 
never  met  face-to-face.  At  several  points  during  this  collaborative  project  there  was  confusion  due  to 
misunderstandings  created  by  the  psychological  distance  between  the  three  reviewers.  Next  discussed  will  be  a 
synopsis  of  the  aggregated  conclusions  found  between  the  three  independent  reviews. 


Aggregated  Conclusions 

Both  expert  reviewers  agreed  that  overall  the  site  was  exemplary.  The  module  was  well  designed,  easy  to 
understand,  implement  and  execute.  The  site  was  additionally  aesthetically  pleasing  from  a graphical  design 
standpoint.  Most  importantly  the  asynchronous  module  was  successful  in  its  instructional  goal  of  facilitating  the 
learner  with  the  creation  of  standalone  web  sites.  Both  the  director  of  the  ITMA  module,  Dr.  Greg  Sherman, 
and  the  participants  commented  favorably  in  survey  reviews  at  the  completion  of  the  module. 

From  an  instructional  design  standpoint,  only  a few  suggestions  were  put  forth,  which  may  improve  the 
effectiveness  and  efficiency  of  the  module.  The  comments  that  follow  were  the  collective  agreement  from  all 
three  parties.  First,  there  may  be  a benefit  to  incorporating  an  official  assessment  for  each  sub  lesson  within  the 
overall  unit  by  way  of  a rubric.  This  rubric  would  specify  evaluation  criteria,  thus  providing  a way  for  the 
learner  to  check  their  work  prior  to  submission.  In  addition,  the  Composer  tutorial  should  include  instruction 
specific  to  the  Macintosh  platform.  This  could  be  done  by  incorporating  Macintosh  examples  in  the  existing 
tutorial,  or  by  creating  a new  tutorial  specific  to  the  platform. 

Also,  a separate  lesson  dealing  specifically  with  tables  may  be  warranted,  incorporating  additional  instruction 
and  a student-2-student  collaborative  constructivist  model.  Finally,  the  overall  module  may  be  made  more 
effective  should  all  lessons  utilize  more  examples  of  student  work  tied  to  reflective  collaboration.  It  should  be 
made  clear  though  that  these  suggestions  are  unproven  at  this  juncture,  and  their  success  must  still  be  borne  out 
in  future  revisions  through  evaluative  field-testing. 
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Abstract:  Explorations  evolve  from  the  concept  of  developing  modular  designed  learning  software. 
As  the  prototype  modules  are  for  a specific  subject  field,  they  represent  an  interactive  environment, 
which--independent  of  a didactical  global  concept — can  be  flexibly  embedded  in  currently  avail- 
able web-based  learning  environments.  Unlike  a classical  simulation,  one  has  the  possibility  whilst 
carrying  out  given  experiments,  medical  processes  or  technical  construction  problems  to  individu- 
ally customise  the  structure  or  even  produce  one’s  own  constructions  or  parameters — ^not  only  to 
adjust  parameters.  With  it,  an  instructor  is  able  to  set  constraints  in  order  to  achieve  a specific 
learning  goal,  and  in  turn  their  adherence  can  be  proven.  The  immediate  response  allows  the  learner 
to  gather  experience  in  an  easy  and  playful  way,  and  it  guarantees  direct  and  clear  access  to  a re- 
gion of  imparted  knowledge. 


Introduction 

A very  important  problem,  for  which  a need  of  information  exists  that  should  not  be  underestimated,  is  the  harden- 
ing of  arteries.  According  to  a number  of  statistical  surveys  growing  prosperity  in  the  industrial  nations  gives  rise  to 
heart  and  circulation  diseases,  which  represent  the  most  common  cause  of  disease  and  death  in  such  countries.  Dam- 
age to  the  artery  system  is  usually  the  culprit,  where  in  such  cases  cholesterol  (see  Roche,  1998)  has  been  the  major 
cause  of  the  contraction  of  the  artery. 

An  excessive  intake  of  the  ’’nasty”  cholesterol  ”LDL”  (Low  Density  Lipoprotein)  cannot  be  counteracted  fast 
enough  with  the  good  cholesterol  ”HDL”  (High  Density  Lipoprotein),  and  thus  it  lingers  in  the  internal  layer  of  the 
artery  (Initima),  and,  after  a short  period  of  time,  it  is  modified  by  a chemical  oxidation.  HDL  is  no  longer  able  to 
transport  modified  LDL  away,  and  so  the  so-called  ”eat-cells”  (macrophages)  must  absorb  the  modified  LDL 
through  a process  called  phagocytosis.  If  the  macrophages  absorb  too  much  of  the  modified  LDL,  they  die  and  mu- 
tate into  Xanthom-cells  and  deposit  plaque  on  the  walls  of  the  arteries.  The  balance  between  LDL  and  HDL  is  cru- 
cial, which  should  not  rise  the  limit  of  3.5.  Malnutrition,  smoking,  lack  of  physical  exercise  and  an  excessive  con- 
sumption of  alcohol  give  rise,  among  other  things,  to  a dangerous  surplus  of  LDL. 
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Figure  1:  Cholesterol,  C27H46O  (left)  from  (Emsley  1996),  right  picture  (CPRRC  1999):  "Cross  section 
(right)  is  that  of  an  artery  wall  showing  atherosclerosis  plaque  (middle  layer)  with  cholesterol  'clefts'.  The 
cholesterol  'clefts'  are  shown  as  the  white  streaks  throughout  the  plaque  layer" 
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A second  fundamental  cause  of  the  hardening  of  the  arteries  is  the  natural  deposition  of  calcium  in  the  arterial  sys- 
tem. It  is  possible  to  dissolve  the  calcium  attached  to  the  artery  wall  with  particular  injections.  The  calcium  is  then 
broken  down  in  the  kidneys  and  later  excreted.  A risky  surgical  operation,  involving  the  pumping  up  of  the  artery 
with  the  use  of,  for  example,  a balloon,  is  thus  unnecessary. 

With  high  blood  pressure  over  150  mmHG  the  danger  exists  that  apertures  form  in  the  artery  walls.  A large  amount 
of  blood  platelets  (thrombocytes),  which  are  responsible  for  the  blood  clotting,  travel  to  the  apertures  and  attempt  to 
close  them  up.  The  accumulation  of  the  thrombocytes  leads  to  a so-called  thrombus,  where  a blood  clot  forms  within 
a blood  vessel  and  remains  attached  to  its  place  of  origin,  hindering  the  blood  flow. 

Over  time  these  causes  lead  to  an  artery  blockage,  which  can  then  lead  to  an  insufficient  supply  of  oxygen  to  the  or- 
gans. If  arteries  attached  to  the  heart  or  near  to  the  brain  are  affected,  in  the  worst  case  it  can  lead  to  an  acute  myo- 
cardial infarction  or  stroke. 

Information  for  people  affected  and  for  those  interested  in  effective  preventative  measures  is  ample.  Due  to  its  de- 
gree of  complexity  and  current  relevance  this  medical  application  is  ideal  to  use  for  the  development  of  a three- 
dimensional  exploration  (for  the  concept  of  explorations  see  (Hampel  & Nowaczyk  1999)  and  (Hampel  1999)). 


Figure  2:  The  Athero  main  window 

Developing  a three  dimensional  exploration 

The  exploration  environment  “Athero”  developed  from  this  specification  can  provide  information  about  a person  at 
any  time.  Thus  exploring  with  the  system  is  made  possible,  in  which  the  user  is  able  to  input  the  initial  situation  and 
continuously  adapt  it  to  his/her  way  of  life  during  the  simulation  at  their  will.  Finally,  the  user  is  able  to  observe 
which  processes  within  the  arteries  are  provoked,  depending  on  the  input  parameters.  Through  interactive  interven- 
tion in  the  scene,  the  user  is  able  to  gather  information  about  single  blood  components  in  the  form  of  HTML  texts 
and  images  as  well  as  in  the  form  of  audible  documents.  These  forms  of  media  are  opened  in  an  additional  browser 
window.  It  is  expected  that  the  provision  of  a greater  number  of  interactive  possibilities  will  provide  the  user  with  a 
better  understanding  of  the  problematic  nature. 

To  ensure  that  the  public  at  large  have  unrestricted  access  to  “Athero”,  it  should  be  made  available  via  the  internet. 
In  order  to  do  so  it  was  necessary  to  develop  “Athero”  in  such  a way  that  it  could  be  loaded  into  a Web-Browser 
such  as  “Netscape"  or  "Internet  Explorer”  (At  the  moment  with  a Java  plugin  as  current  browser  do  not  support  the 
newest  Java-version  (see  Java  Plugin). 

In  order  to  provide  a clearer  understanding  of  the  3D-System,  and  to  reach  a suitable  speed,  at  which  the  complex 
3D-scenes  will  be  displayed,  the  visualisation  is  limited  to  the  effects  of  the  LDL,  such  as  the  natural  hardening  of 
the  arteries  at  age  of  40  and  the  creation  of  thrombocytes  in  the  case  of  endothelium  related  injuries.  The  informa- 
tion system  can,  however,  be  extended  at  any  time.  For  example,  additional  objects  could  be  inserted  into  the  scene, 
which  could  also  lead  to  arteriosclerosis,  without  sacrificing  its  clarity.  The  rules  used  to  calculate  the  HDL  and 
LDL  concentrations,  with  relation  to  the  other  adjustable  parameters,  are  easily  interchangeable. 
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The  model  and  the  scene 

One  section  of  a pulsating  artery  is  displayed  in  a graphic  formation,  in  which  different  components  of  the  blood 
flow;  such  as  thrombocytes  (responsible  for  clotting),  erythrocytes  (red  blood  cells,  responsible  for  the  transporta- 
tion of  oxygen),  leucocytes  (white  blood  cells,  responsible  for  the  repulsion  to  penetrate  cells  during  an  infection), 
macrophages  (so  called  eat-cells)  and  LDL.  The  user  can  see  that  under  normal  external  conditions,  the  erythrocytes, 
on  which  it  mainly  depends,  as  they  supply  the  organs  with  essential  oxygen,  still  flow  unobstructed  through  the  ar- 
teries. If  there  is  an  increase  of  artery  blockage  through  the  formation  of  calcium,  plaque  or  thrombocytes,  the  flow 
of  blood  cells  is  restricted,  and  the  cells  are  left  to  revolve  on  the  spot  for  a while  to  allow  a smaller  number  of  eryth- 
rocytes to  flow  out  the  other  end  of  the  artery.  This  leaves  the  organs  with  a lack  of  oxygen.  It  is  possible  to  influ- 
ence the  external  factors  such  as  appearance  of  age,  height,  weight,  exercise,  consumption  of  alcohol  and  cigarettes, 
and  blood  pressure.  The  individual  parameters  are  analysed  by  the  program  and  displayed  accordingly  in  the  30- 
scene  with  the  increases  and  decreases  of  LDL  and  HDL.  On  the  other  hand  this  has  an  effect  on  the  other  compo- 
nents of  the  blood  as  well  as  on  the  artery  walls. 

With  the  use  of  the  mouse  the  user  is  able  to  interact  with  the  3D-scene  from,  for  example,  expanding  the  artery  to 
giving  a virtual  injection.  With  a single  mouse  click  accurate  information  regarding  individual  objects  can  be  ob- 
tained. In  order  to  describe  the  physiological  construction,  the  task  and  possibly  the  current  condition  of  the  selected 
object  more  closely,  a text,  graphic,  video-sequence,  or  an  internet  link  to  a web-site  will  be  displayed  in  the 
browser.  In  another  control  window  the  user  has  the  possibility  to  select  a number  of  different  viewing  positions  for 
the  artery.  And  finally,  a "camera  tour"  can  be  initialised,  in  order  to  view  some  objects  more  closely. 

As  mentioned  earlier,  the  LDL  and  HDL  concentration  is  a very  important  factor  for  the  development  of  arterioscle- 
rosis. Unfortunately,  it  is  not  possible  to  draw  up  "ten  golden  rules"  for  the  calculation  of  LDL  or  HDL  concentra- 
tions, rather  the  values  rise  and  fall,  depending  on  the  probabilities  to  be  calculated.  To  determine  the  concentration 
of  cholesterol  details  such  as  age,  height,  weight,  physical  behaviour,  cigarette  and  alcohol  consumption,  blood- 
pressure,  and  a person’s  vitamin  E content  need  to  be  given. 

Depending  on  these  parameters  updates  and  "calculations"  are  carried  out  at  regular  intervals  in  an  update  method, 
which,  in  accordance,  controls  the  processes  in  the  arteiy. 


Figure  3:  Erythrocytes  (red  blood  cells,  responsible  for  the  transportation  of  oxygen),  leucocytes  (white  blood  cells,  responsible 
for  the  repulsion  to  penetrate  cells  during  an  infection)  and  thrombocytes;  from  (DRK  Lichtenstein  1 999) 

Establishing  the  factors  that  influence  the  rise  in  LDL  by  calculating  the  BMI  value  (Body  Mass  Index:  determined 
from  the  age,  height  and  weight  and  used  to  determine  whether  a person  is  overweight)  is  among  the  steps  belonging 
to  the  updating  process.  Another  is  the  evaluation  of  cigarette  and  alcohol  consumption  and  adjusting  the  LDL  val- 
ues accordingly.  Reducing  the  LDL  values  and  increasing  the  HDL  values  through  exercise  as  well  as  a low  alcohol 
consumption  leads  to  an  LDL  reduction  which,  in  turn,  leads  to  increased  HDL  values. 

After  adjusting  the  LDL  and  HDL  values,  the  LDL-HDL  relationship  is  calculated,  and  depending  on  the  result  a 
modification  probability  of  the  LDL  particles  that  are  momentarily  present  in  the  internal  artery  layer  is  calculated. 
The  LDLs  are  then  possibly  modified,  absorbed  by  the  tissue  or  are  taken  away  with  the  HDL.  In  the  latter  case  the 
cholesterol  concentration  must  be  reduced  in  the  tissue. 

In  addition  to  the  LDL-HDL  relationship,  the  probability  for  an  endothelium-related  injury  caused  by  high  blood 
pressure  is  calculated  and  an  instruction  to  close  the  endothelium-related  injury  up  is  passed  on  to  the  thrombocytes. 
From  an  age  of  40  years  the  probability  for  calcium  deposits  that  occasionally  need  to  be  produced  increases. 

A serum  can  be  administered  with  a virtual  injection,  which  prevents  the  development  and  the  growing  of  calcium 
deposits.  If  this  serum  is  available  in  the  blood  it  will  be  reduced  over  time. 

Leucocytes,  which  are  responsible  for  the  transportation  of  bacteria,  are  produced  when  an  infection  exists  in  the 
body  that  gives  a higher  reading  than  37.5  C on  the  thermometer.  On  transportation  the  body  temperature  is  reduced 
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slightly,  showing  the  fight  that  the  leucocytes  put  up  against  the  infection.  If  modified  LDL  is  found  in  the  blood, 
macrophages  are  produced,  which  eliminate  them. 

The  demand  for  three-dimensional  representations  of  objects,  increased  interaction,  exploring  with  the  system,  pub- 
lication in  the  internet  and  last  but  not  least  the  possibility  to  develop  the  software  in  an  object-orientated  way  has 
led  us  to  the  use  of  Java3D. 


Figure  4:  The  Athero  main  and  navigation  window 


Developing  Athero  in  Java  3D 

Ever  faster  processors  and  graphic  cards  offering  ever  more  memory  enable  complex  and  detailed  real-time  three- 
dimensional  universes  to  be  displayed.  Going  under  the  idea  of  "Learning  with  multimedia"  one  can  build  virtual 
models,  which  provide  the  building  blocks  for  three-dimensional  exploration  systems.  Both  Java3D  (see  (Java  3D 
API),  (Brown  & Petersen  1999)  and  (Sowinzal  et  al  1997))  and  VRML  (see  Web3D  Consortium)  are  suitable  for  the 
representation  of  these  animations. 

Java3D  is  a class  library  for  Java.  It  makes  an  extensive  range  of  commands  and  3D-features  available  to  the  devel- 
oper, as  well  as  high-  and  low-level  graphic  programming  to  create  interesting  three-dimensional  models.  It  should 
not  be  forgotten  that  VRML  is  a description  language  that  is  totally  independent  of  a programming  language.  It  is  a 
file-format  for  3D-data  in  WWW  that  allows  complex  fixed  3D-universes  to  be  presented.  The  more  animations, 
interactions  and  calculations  are  integrated  into  the  3D-models,  the  more  suitable  Java3D  is. 

The  Java3D  API  (Application  Programming  Interface)  is  a library  of  Java  classes  for  the  presentation  of  3D- 
universes.  It  contains  a graphical  run-time  system  that  offers  an  object-orientated,  high-level  programming  model  to 
the  developer,  with  which  they  can  develop  sophisticated,  interactive  3D  Java  applications  and  web-based  applets. 

As  Java3D  is  a class  library  of  Java  it  offers  the  developer  all  graphical  support  within  the  three-dimensional  area, 
with  which  they  are  able  to  construct  3D-objects  as  well  as  different  structures,  with  which  the  objects  are  rendered 
ready. 

On  the  one  hand  it  is  a matter  of  suitable  data  structures,  while  on  the  other  hand  it  is  matter  of  methods  used  to  dis- 
play data.  The  fundamental  concept  of  this  implementation  is  to  cover  the  hardware  specific  details  with  a general 
interface.  Java  3D  relies  on  low-level  technologies  such  as  OpenGL  (see  SGI  OpenGL),  Direct3D  (see  Microsoft 
DirectX)  and  inherits  ideas  from  QuickDraw3D  (Apple's  implementation  of  a 3D  graphics  libraiy,  see  (Apple 
Quicktime)). 

Java3D  fits  seamlessly  into  the  whole  concept  of  Java.  A great  advantage  of  this  programming-language  is  the  flexi- 
bility offered  by  such  platform  independence.  Implementation  is  currently  provided  for  Windows,  Solaris  and  Linux. 
During  the  development  a lot  of  weight  was  placed  upon  the  performance,  which  was  achieved  with  parallel  ren- 
dering. In  addition,  Java3D  was  optimally  designed  for  the  future  development  of  hardware  such  as  cluster  and  mul- 
tiprocessor systems,  as  one  stays  on  a comparatively  abstract  level  whilst  programming  and  allows  Java3D  to  do  the 
optimising. 

Another  advantage  is  the  support  of  different  file  formats.  For  this  purpose,  among  others,  the  portfolio-package  by 
NCSA  (see  NCSA  Portfolio)  offers  loader-classes  of  all  current  3D-graphic  formats,  such  as  3D-Studio,  True-Space, 


405 


BEST  COPY  AVAILABLE 


Digital  Elevation  Map,  Lightwave  3D  and  many  more.  During  run-time  in  the  3D-scene  additional  complex  VRML- 
universes  can  be  loaded  with  a so  called  VRML-loader  (see  VRML  loader)  available  from  Sun  Microsystems. 
Java3D  is  not  exclusively  for  laying  out  static  3D-universes,  it  also  explicitly  supports  the  drafting  of  dynamical  3D- 
scenes.  Applications  are  able  to  react  to  user  inputs,  perform  calculations,  construct  3D-objects,  manipulate  them  in 
any  way,  and  add  or  delete  them,  all  during  run-time.  These  objects  (3D-models  and  sounds)  are  placed  in  an  virtual 
universe,  which  is  then  rendered  functional.  To  increase  the  multimedia  quality  of  the  applications  they  can  be  fitted, 
without  great  effort,  with  extra  characteristics  such  as  video  sequences  and  pictures. 

Apart  from  its  **Write  once,  run  anywhere**  philosophy  3D  API  has  other  advantages.  Because  it  is  a member  of  the 
Java  family  it  is  entirely  designed  in  an  object-orientated  way.  Although  the  object-orientated  design  usually  offers 
itself  to  the  programming  of  graphics,  in  previous  packages  such  as  Open  GL  this  aspect  was  generally  dealt  with  in 
a secondary  fashion.  Thus  the  job  of  development,  servicing  and  the  extension  of  existing  software  has  been  enor- 
mously relieved. 

The  scene  graph  based  3D-modell,  which  is  based  on  Java3D,  helps  even  the  inexperienced  developer  with  the  inte- 
gration of  multimedia  and  graphic  programming  and  3D-effects  into  its  applications.  Despite  the  wide  scope  of 
functions  offered  by  Java3D  an  experienced  Java  programmer  will  find  it  easy  to  learn — in  contrast  to  low-level, 
procedural  APIs  such  as  Open  GL.  The  rendering  of  objects  is  handled  in  an  automatic  way  and  with  the  insertion  of 
threads  it  is  possible  to  work  on  different  flinctions  in  parallel.  They  can  also  be  carried  out  with  the  use  of  different 
computers.  Apart  from  simplifying  the  development  of  software  with  Java3D,  the  package  also  offers  sufficient  per- 
formance for  computers  currently  available.  TTiis  is  quite  remarkable  if  you  stop  to  think,  that  Java  belongs  to  the 
interpreted  languages. 

A Java3D  program  creates  Java3D  objects  and  places  them  in  a scene-graph.  The  scene-graph  resembles  a tree  data 
structure  constructed  by  different  objects;  it  describes  both  the  content  of  the  virtual  universe  as  well  as  the  render- 
ing of  it.  In  order  to  optimise  the  scene-graph  it  is  possible  to  compile  it  after  its  creation.  In  order  to  manipulate  the 
scene-graph  during  run-time,  so  called  capability  bits  are  used,  which  define  the  manipulations  that  are  allowed  to  be 
made. 

As  a result  Java3D  is  ideal  to  be  used  in  interactive  3D-environments  such  as  games,  simulations  and  explorations. 
Effective  animations  can  be  produced  with  the  Java3D  class  within  a short  time. 

While  experimenting  with  Java3D  a simple  driving  program  was  programmed  within  a few  hours.  The  Java3D  pro- 
gram creates  a street  with  an  asphalt  texture  and  with  median  strip  that  moves  to  the  direction  of  the  viewer.  Street 
poles  pass  rapidly  on  both  sides  of  the  street  to  give  the  impression  that  the  user  is  really  driving.  To  give  the  ap- 
pearance of  driving  around  curves  the  horizon  moves  from  side  to  side  of  the  street;  just  like  the  horizon  created  by  a 
cylinder  is  produced  with  a scenery  texture.  The  whole  scene  can  be  rotated,  zoomed  in  or  out  and  moved  by  the 
mouse  on  its  axis.  On  a Pentium  PC  the  animation  runs  astonishing  fast.  (A  Java  3D  program  runs  on  Microsoft 
DirectX  (see  Microsoft  DirectX)  or  OpenGL  graphics  devices  and  libraries  installed). 

To  represent  the  street  a simple  rectangular  plain  was  created  with  a 3D-geometry  modelling  program  and  converted 
according  to  VRML97.  In  order  to  reproduce  the  streets  as  true  to  life  as  possible,  the  street  is  given  an  asphalt  tex- 
ture, also  showing  the  markings  on  the  streets.  Every  single  straight  element  on  the  street  is  given  a "position  be- 
haviour", which  moves  the  element  on  a straight  route  in  the  direction  of  the  viewer.  A number  of  elements  lined  up 
create  an  effect  of  driving. 

The  curves  are  also  created  with  these  street  elements,  but  instead  of  being  manipulated  with  a position  behaviour, 
they  are  manipulated  by  a **rotation  behaviour**,  which  rotates  them  slowly  from  left  to  right.  Because  the  elements 
of  the  curves  are  very  close  to  the  horizon,  making  them  barely  recognisable  to  the  spectator,  a shift  in  direction  is 
avoided.  The  background  is  created  with  the  help  of  a cylinder  at  the  zero  point.  As  a large  enough  radius  and  land- 
scape texture  are  provided,  the  spectator  receives  the  perfect  illusion. 

The  architecture  of  Athero — overview 

In  Athero  the  3D-capabilities  of  Java3D  API  were  used  extensively  in  order  to  develop  an  attractive  exploration. 
The  ever  present  object-orientated  programming  makes  maintaining  and  extending  the  application  considerably 
easier  and  simplifies  the  transferring  of  real  objects  with  their  tasks  and  characteristics  to  the  computer  model. 
Extending  is  particularly  important,  as — out  of  time  and  technical  reasons — the  simulation  is  not  limited  to  itself, 
rather  continual  development  in  the  knowledge  of  hardening  of  the  arteries  is  necessary.  The  possibility  should  exist 
to  integrate,  for  example,  new  blood-particles  that  participate  in  the  hardening  of  arteries  in  the  simulation. 

Moreover,  it  is  possible  to  change  the  method,  with  which  the  LDL/HDL-concentrations  are  analysed,  by  current 
calculation  possibilities.  The  behaviour  of  the  single  objects  always  tends  to  direct  to  this  method. 
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The  application  has  two  frames.  The  main  frame  includes  a panel  with  the  3D-canvas,  on  which  the  3D-simulation  is 
displayed,  and  beside  it  is  a panel  with  sliders  and  buttons  to  adjust  the  parameters. 

The  second  frame  is  equipped  with  a 3D-canvas,  on  which  VRML-models  of  video-cameras  are  placed  around  a 
cylinder  representing  an  artery.  The  cameras  are  situated  at  four  different  positions  in  the  virtual  room,  from  which 
the  viewer  can  follow  the  scene  at  the  main  window.  The  use  of  a menu  was  avoided,  as  the  influence  of  the  simula- 
tion is  limited  to  changing  the  parameters  of  different  objects  with  the  help  sliders  and  buttons  and  also  from  mouse 
clicks. 

All  3D-models  used  such  as  the  artery,  the  erythrocytes,  the  leucocytes  etc.  are  loaded  as  VRML-models  and  can 
therefore  be  replaced  by  the  user.  On  slow  computers  for  example  3D-objects  can  be  exchanged  with  models  with  a 
low  level  of  detail.  Athero  runs  equally  well  as  an  applet  in  a web  browser  and  as  a stand-alone  application  in  a 
DOS-Box. 

Outlook 

In  the  future  smaller  exploration  systems,  in  which  constructive  theories  are  transferred,  can  be  used  for  teaching  in 
schools  or  in  universities.  Due  to  the  ease  and  quickness  of  production,  not  everything  depends  on  developing  life- 
long simulations.  If  an  application  proves  to  be  useless,  it  can  be  simply  done  away  with. 

It  is  realistic  to  imagine  that  pupils  and  students  will  develop  applications  on  their  own  with  the  help  of  Java3D  ex- 
plorations or  work  with  them  to  gain  a better  understanding  of  complicated  issues.  The  availability  over  the  net  also 
offers  an  advantage.  People  wanting  to  learn  have  the  possibility  to  do  it  simultaneously  without  being  restricted  to  a 
specific  place  and  through  it  gain  experience  with  mutual  interaction  and  a better  learning  affect.  These  kinds  of  ex- 
ploration environments  could  find  an  application  in  the  school  classroom  or  in  local  or  even  intercontinental  semi- 
nars. 

The  use  of  such  systems,  such  as  the  simulation  of  the  hardening  of  the  arteries,  is  conceivable  in  hospitals  or  ther- 
apy centres.  They  could  contribute  to  a better  understanding  of  diseases  and  thus  help  a doctor  or  therapist  to  explain 
a diagnosis  or  its  effects. 
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Abstract:  The  initiative  sTeam — structuring  of  information  within  a team — in  Paderbom  is  devel- 
oping an  open  and  co-operative  platform  in  order  to  equally  enable  learners  as  well  as  lecturers  and 
scientists  to  organise  and  structure  knowledge  in  a co-operative  way.  The  following  article  presents 
the  possible  use  of  such  an  environment  in  form  of  application  scenarios.  In  this  context  special 
emphasis  is  placed  on  integrating  the  developed  tools  and  methods  into  a way  of  co-operative 
teaching  that  will  also  work  in  everyday  life,  i.e.  the  development  of  the  so-called  learning  suppor- 
tive infrastructures  will  be  accelerated. 


Introduction 

Since  the  emergence  of  the  Internet  the  application  scenarios  of  this  new,  exciting  and  constantly  expanding  medium 
have  continuously  increased.  Apart  from  the  most  well-known  application,  the  WWW,  different  technologies  have 
been  developed  such  as  chats,  video-conferencing  or  network-supported  multimedia  databases  which  have  gained 
broad  acceptance  by  users.  Current  developments  such  as  the  expansion  of  e-commerce  solutions  or  extensive  net- 
work-supported  libraries  are  unfortunately  characterised  by  a limited  commitment  of  users  to  contribute  individual 
abilities  or  actively  participate  in  organising  the  presentation  of  information. 

The  WWW  has  emerged  as  mere  medium  of  information  providers  for  the  consumer.  What  is  less  important  for  on- 
line shops  and  web  performances  of  companies  turns  out  to  be  the  decisive  criterion  for  the  use  of  the  WWW  as 
teaching  and  learning  environment.  Today’s  WWW  offers  only  marginal  opportunities  for  the  consumer  to  structure 
the  presented  information,  to  work  on  own  documents  or  contribute  them.  Already  at  an  early  stage  it  had  been  real- 
ised from  different  angles  that  the  possibility  to  contribute  annotations  and  individual  comments  on  the  pages  and 
texts  is  an  important  prerequisite  for  the  successful  application  of  such  a medium  in  teachings  (see  Engelbart  1984). 
If  the  network  is  to  be  used  as  co-operation-supporting  tool  it  will  be  essential  for  students  and  lecturers  as  well  as 
for  all  users  to  have  personal  structures  in  the  jungle  of  the  WWW.  Bookmarks  as  browser-dependent  rudimentary 
structures  are  rather  unsuitable  procedures  since  it  is  neither  easy  nor  unproblematic  to  provide  them  with  annota- 
tions, pass  them  on  (i.e.  supporting  co-operation)  or  structure  them  sufficiently.  For  this  reason  the  so-called  “lost  in 
hyperspace”  syndrome — a synonym  for  an  inherent  navigation  problem  in  the  network — is  still  omnipresent.  It  is 
true  that  the  ever  increasing  number  of  Internet  users  has  led  to  a shooting  up  of  information,  but  the  way  of  co- 
operation among  users,  i.e.  the  exchange  of  documents  and  information  is  furthermore  limited  to  the  sending  of 
documents  and  web  addresses  (URLs)  per  e-mail. 

sTeam — create  an  open  and  flexible  platform  for  knowledge  management 

The  following  article  aims  at  outlining  new  ways  of  using  the  Internet  and  its  services  (mail,  news,  ftp,  WWW)  to 
support  teaching  and  is  based  on  experiences  of  many  years  by  the  group  “computers  and  society”.  This  working 
group  has  dealt  with  questions  concerning  the  organisation  of  infrastructures  for  an  active  support  of  human  learn- 
ing. Basing  on  a number  of  existing  technologies  the  sTeam  approach  contains  the  claim  that  existing  services  and 
systems  can  be  integrated — by  justifiable  efforts — into  an  open  user-centered  platform. 
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During  the  last  two  years  essential  efforts  have  been  made  to  approach  our  target  regarding  an  open  and  individu- 
ally-made platform  of  existing  and  future  technologies.  The  last  paragraph  of  this  article  outlines  the  technological 
requirements  to  such  an  environment  and  makes  clear  that  the  only  practicable  way  will  be  the  implementation  on  an 
open-source  basis. 

In  order  to  obtain  an  exact  idea  regarding  the  functionality  of  a platform  that  is  supporting  teaching  at  the  university 
as  well  as  scientific  work,  the  focus  of  interest — as  already  mentioned  in  (Hampel  1999) — will  not  be  the  architec- 
ture of  such  a system,  but  rather  a possible  scenario  of  the  application  in  everyday  teachings  whereas  this  scenario 
will  function  as  guiding  criterion  for  creating  the  conception  of  a core  architecture  of  the  sTeam  approach — “struc- 
turing information  within  the  team”.  Already  existing  prototypes  of  the  sTeam  system  exactly  aim  at  the  fulfillment 
of  the  above-mentioned  scenarios  and  will  continuously  be  extended  for  realisation. 

The  lectures  offered  by  the  “computers  and  society”  group  such  as  “software  ergonomics”  aim  at  the  active  collabo- 
ration of  students.  Apart  from  lectures,  students  will — in  interactive  exercises — be  encouraged  to  develop  own  soft- 
ware-ergonomic organisation  processes  and  to  evaluate  existing  systems.  In  the  course  of  the  work  with  a broad  va- 
riety of  sources  of  knowledge  such  as  laws,  standards,  style-guides  or  scientific  essays,  a number  of  systems  have  to 
be  evaluated  according  to  a number  of  software-ergonomic  criteria.  Experiences  that  have  been  gained  in  events 
during  the  last  years  show  that  students  are  only  able  to  actively  understand  and  internalise  the  handling  of  a multi- 
tude of  knowledge  when  they  co-operatively  use  different  media.  In  addition  to  this  materials  should  be  available  at 
any  learning  places  (see  Keil-Slawik  & Selke  1998).  Already  for  some  years  sTeam  has  been  working  at  the  re- 
search of  concepts  and  tools  to  ensure  the  availability  of  documents  at  each  learning  place  for  the  support  of  the  stu- 
dents. 

Scenario  of  the  application  during  lectures 

In  the  scenario  of  the  “software  ergonomics”  lecture  the  students  are  in  the  foreground.  The  lecture  takes  place  as 
reference  event  in  one  of  the  interactive  lecture  halls  (digital  infrastructure  for  computer-supported  co-operative 
learning)  at  the  University  of  Paderbom\  sTeam  as  infrastructure  and  a number  of  co-operation-supporting  tools 
provides  so-called  semantic  spaces  for  the  topic  “software  ergonomics  lecture”  (see  Henderson  & Card  1985).  Se- 
mantic spaces  contain  all  sTeam  objects  such  as  links,  documents,  texts,  graphs,  tools,  annotations  and  any  applica- 
tion objects  (presentations,  spread-sheet  calculation  etc.).  In  relation  to  the  example  of  the  software  ergonomics  lec- 
ture, current  slide  presentations  as  well  as  background  information  of  the  lecture  can  be  presented,  i.e.  all  docu- 
ments, graphics  and  external  resources  the  lecturer  is  referring  to. 

This  inherently  accessible  stock  of  documents  enables  e.g.  the  lecturer  to  have  direct  access  to  a number  of  sources 
of  knowledge  during  the  lecture,  to  illustrate  them  actively  or  to  react  to  checkbacks  and  suggestions  of  the  students 
by  additional  illustrations  or  modifications  of  the  material.  Participating  students  are  able  to  make  annotations,  i.e. 
adding  personal  notes  to  slides  and  documents  or  write  down  own  texts  or  simultaneously  link  documents  by  hyper- 
links. 

The  process  of  developing  own  mental  structures  during  lectures  has  to  be  actively  supported  by  sTeam  and  can — in 
contrast  to  most  other  available  systems — also  take  place  co-operatively.  The  hierarchy  of  groups  and  rights  that 
refers  to  all  objects  within  the  sTeam  system  allows  a complex  specification  of  the  access  to  annotations,  links  and 
documents.  Such  a concept  allows  students  to  meet  in  smaller  groups  and  to  more  easily  obtain  specific  information 
or  to  write  down  notes  co-operatively  as  learning  group.  Apart  from  being  a presenting  event,  the  lecture  also  ac- 
tively integrates  spatially  absent  participants  by  means  of  communication  mechanisms  such  as  audio-,  video  chats, 
that  have  to  be  integrated  into  sTeam  in  advance.  Thus  students  or  lecturers  are  able  to  ask  questions,  make  com- 
ments or  contribute  documents  to  the  lecture  within  the  sTeam  room  via  Internet.  However,  the  sTeam  approach  will 
be  most  efficient  if  synchronous  procedures  change  to  asynchronous  procedures,  e.g.  on  supporting  scenarios  of 
practising  individuals  or  groups,  whereas  the  system — with  all  the  documents  and  structures  stored  serves  as  basis 
for  individual  and  group  work.  The  students  have  access  to  learning  material  and  their  personally  written  down  texts 
from  different  places  (practising  room,  library,  personal  working  place). 

Application  scenarios  during  practising  include  a fluent  change  of  teamwork  and  individual  work  phases.  The  pro- 
cedure of  practising  software  ergonomics  today,  includes  the  active  commitment  of  students  to  organisational  mat- 


’ DISCO  (digital  infrastructure  for  computer-supported  co-operative  learning),  an  advanced  infrastructure  in  Paderbom,  sup- 
ports teaching  and  learning  by  means  of  multimedia.  It  comprises  interactive  lecture  and  seminar  room,  places  of  production 
for  multimedia  documents  as  well  as  working  places  that  are  connected  with  each  other  via  networks,  servers  and  services 
(more  details  in  Keil-Slawik  1999). 
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ters.  Solution  proposals  concerning  exercises  such  as  the  software  ergonomical  evaluation  of  user  interfaces  can 
smoothly  be  performed  by  the  sTeam  concept. 

Students  insert  their  solution  proposals  into  the  system  in  advance  or  find  solutions  by  linking  existing  documents, 
e.g,  screenshots  of  user  interfaces  or  criteria  catalogues  with  personal  solution  proposals  within  the  learning  and 
document  environment.  This  process  can  be  realised  individually  or  even  better  in  a co-operative  way  by  combining 
different  solution  alternatives.  The  actual  practising  can  be  described  as  interrelation  of  presenting  and  discussing 
solution  alternatives.  After  a short  presentation  of  different  alternatives  and  mutual  constructive  criticism  and 
evaluations  the  found  solution  proposals  will  be  linked  as  well  as  stored  permanently  and  archived  within  the  sTeam 
system. 

In  order  to  facilitate  preparations  for  the  exam  later  on,  it  will  be  very  important  to  permanently  store  the  found 
structure.  Thus  the  students  are  e,g.  able  to  evaluate  co-operatively  different  e -commerce  solutions  of  the  WWW 
(e,g.  by  a shared  browser)  during  practising  “software  ergonomics”  whereas  comments  will  only  be  visible  for  the 
participants.  In  a parallel  working  step  links  are  installed  to  software  ergonomical  criteria  or  available  laws  and 
standards  of  the  learning  environment  (e,g,  European  Union  ergonomic  guidelines).  Important  passages  out  of  tex- 
tual chats  can  be  used,  i,e,  translated  as  annotations  or  new  documents  by  means  of  drag&drop  mechanisms.  Organi- 
sation proposals  done  by  the  students  can  be  filed  in  the  same  way  and  without  difficulties  in  personal  and  mutual 
working  fields  (semantic  spaces).  Thus  the  students  are  able  to  develop  drafts  and  prototypes  of  user  interfaces  in 
Java  by  using  tools  for  organising  the  surface  (interface  builder,  e.g,  by  visual  programming)  and  file  them  in  con- 
text to  corresponding  lecture  documents. 

The  concept  of  the  so-called  semantic  maps  (see  Klemme  et  al.  1998)  which  was  developed  by  the  group  “comput- 
ers and  society”  has  proven  to  be  one  further  indispensable  instrument  to  media-supported  learning  and  has  also 
been  integrated  into  scenarios  of  the  sTeam  concept. 

Students  use  semantic  structures — provided  by  the  lecturer — to  navigate  or  make  cooperatively  semantic  survey 
maps  of  the  software  ergonomics  as  part  of  an  instructed  exercise  or  for  their  own  orientation.  These  maps  may  e.g. 
serve  as  criteria  catalogues  for  the  design  of  interactive  systems  or  to  point  out  the  integration  of  this  special  field 
into  other  disciplines  of  the  “Human  computer  interaction  HCI”  while  the  procedure  itself  demonstrates  essential 
coherences.  Further  application  scenarios  are  the  realisation  of  virtual  office  hours  as  periods  of  presence  of  respon- 
sible persons  within  documents  and  communication  environments.  The  virtual  office  hours  will  as  a consequence  be 
able  to  contribute  to  an  unproblematic  clarification  of  content  and  organisation  matters. 

A close  linkage  of  the  synchronous  communication  mechanisms  to  an  awareness  information  and  the  world  of 
documents  creates  a new  quality  concerning  computer-supported  communication.  Thus  co-operative  partners  can  be 
ascertained  at  any  time  while  taking  essential  personal  rights  into  consideration  (all  mechanisms  are  available  on  a 
voluntary  basis),  as  well  as  documents  and  sources  of  knowledge  they  are  currently  working  on  or  (by  consulting  the 
temporal  history)  who  has  performed  which  action  (see  Nomura  et  al.  1998),  [Parallel  to  this  it  is  possible  to  directly 
address  individuals — ^by  presenting  the  so-called  Avatars^ — or  a group  of  participants,  thus  encouraging  discus- 
sions.] 

(The  term  “Avatars”  is  not  necessarily  connected  with  the  concept  of  “spatial  navigation”^,  which  is  a spatial  real- 
world  navigation).  A shared- whiteboard  supplies  documents,  tools  or  graphical  elements  to  participating  students. 
The  students  are  able  to  transfer  elements  of  their  desktop,  graphs,  links  and  documents  by  drag&drop  to  a synchro- 
nous desktop  without  any  difficulties,  i,e,  this  desktop  will — in  virtual  space — be  visible  for  all  participants  at  the 
same  time.  Designed  to  be  an  object-oriented  whiteboard,  i.e.  some  kind  of  desktop  that  acts  with  plain  objects,  such 
an  approach  allows  to  work  with  any  digital  media  and  resources  within  the  team.  Thus  the  dragging  of  a link  from  a 
browser  leads  directly  to  installing  a link  object  within  a sTeam  space  that  has  been  assigned  to  the  whiteboard  or 
documents  will  directly  be  transferred  to  the  database  and  are  available  to  other  users — in  dependence  of  their 
documents — for  co-operative  processing  and  editing. 


^ The  term  “Avatars”,  i.e.  the  image  of  a user  in  a virtual  world  is — by  sTeam — not  understood  as  three-dimensional  repre- 
sentation of  users  as  it  is  in  so-called  Avatar-worlds  (see  Darner  1998). 

^ The  concept  of  “spatial  navigation”  has  experienced  formative  influence  by  Greenhalgh  and  Bendford  and  their  treatises 
concerning  the  MASSIVE  system.  It  is  about  a three-dimensional  world  where  Avatars  move  in  “natural”,  real-world  forms 
and  communicate  by  textual-  and  audio  chat.  This  communication  is  controlled  by  the  “spatial  model  of  interaction”,  i.e. 
communication  will  be  conducted  by  the  so-called  “focus”,  “nimbus”  and  “aura”  of  objects,  i.e.  their  relative  position  and  di- 
rection to  one  and  another.  The  so-defined  directions  and  spheres  of  activity  of  user  awareness  establish  concrete  fields  of 
interest  as  well  as  differentiate  between  communicating  groups  and  interactive  persons  (see  Greenhalgh  & Benford  1995). 
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Scenario  B:  sTeam  as  working  place  for  scientists 

The  sTeam  approach  also  offers  new  qualities  regarding  the  use  of  the  system  as  prospective  working  place  for  sci- 
entists. Analogous  to  the  support  of  learning  processes,  certain  functionalities  concerning  the  filing,  semantic  struc- 
turing and  linking  of  articles,  publications,  essays  and  documents  are  useful  for  the  daily  work  of  scientists.  The 
discussion  of  different  positions  (articles  and  treatises)  and  its  linkage  beyond  the  limits  of  one  server  as  well  as  the 
inclusion  of  any  other  sources  of  knowledge  of  the  scientific  network  like  other  national  and  international  research 
institutes  permit  a new  form  of  high-quality  co-operation  in  research  and  science. 

In  support  of  such  co-operations  it  will  be  essential  to  have  the  opportunity  to  annotate  different  sources  of  knowl- 
edge in  close  connection  to  network-supported  synchronous  and  asynchronous  communication  forms.  Besides  the 
co-operative  processing  of  documents,  the  essential  criteria  for  the  acceptance  of  the  tool  strived  for  are  conceptions 
of  the  development  of  different  views  within  the  sources  of  knowledge,  i.e.  the  possibility  of  dividing  a text  into 
different  paragraphs  in  order  to  sort  them  and  make  individual  sections  available  for  colleagues  and  co-operation 
partners.  The  individual  marking  of  texts  (“textmarks”)  is  also  an  essential  criterion  to  guarantee  the  acceptance  of 
the  tool.  Facilities  exist  to  work  with  texts  and  graphics  actively and  connect  forum  discussions  and  textual  online 
communication  forms  to  the  capabilities  of  document  management. 

Approaches  have  been  made  in  the  field  of  graphical  navigation  and  orientation  maps  (see  supportive  learning  sce- 
narios). These  maps  represent  the  semantic  structure  of  learning  materials  and  help  to  give  learners  and  teachers 
orientation  in  various  sources  of  knowledge.  Therefore  they  also  compensate  the  inherent  problem  of  losing  orienta- 
tion in  large  hypertext  structures  such  as  the  WWW.  ^ 

Our  approach-sTeam-as  open-source  initiative 

The  spectrum  of  concepts  and  approaches  that  have  to  be  considered  is  very  large  when  taking  into  account  an  open 
platform  with  a multitude  of  services,  tools  and  architectures.  Due  to  this,  the  following  representation  of  current 
technological  aspects  only  provides  a brief  and  incomplete  outline.  The  architecture  of  sTeam  is  based  on  a CVE — 
Collaborative  Virtual  Environment — and  can  neither  be  characterised  as  a mere  synchronous  CSCW  (Computer 
Supported  Cooperative  Work)  tool  (see  (Roseman  & Greenberg  1996),  (Patterson  et  al.  1990)  or  (Chabert  et  al. 

1 998) ),  nor  as  mere  management  of  documents  or  workflow-oriented  tool. 

At  the  same  time  sTeam  combines  important  concepts  concerning  the  structure  of  paths  through  existing  contents  of 
networks — thus  aims  at  the  navigation  support  of  the  user  (concerning  other  similar  approaches  compare  (Wexelblat 

1999)  or  (Furuta  et  al.  1999)).  By  coupling  one  or  several  databases,  managing  abilities  of  users  and  objects  will  be 
supplied.  In  this  sense  sTeam  impresses  less  as  closed  application  but  rather  by  its  open  and  expandable  framework 
architecture. 

In  this  connection  programming  interfaces  to  individual  tools  and  objects  as  well  as  a broad  adaptability  will  be  sup- 
plied to  the  users.  Objects  will — in  sTeam — be  provided  with  access  rights  and  filed  in  this  database  while  being 
able  to  access  them  by  the  network.  sTeam  offers  the  possibility  by  different  access  tools  (frontends,  clients)  to 
file — ^in  a structured  way — objects  such  as  documents,  graphics,  annotations  or  tools  like  shared  whiteboards  or 
message  boards  in  semantic  spaces.  Within  semantic  spaces  users  are  able  to  create  their  own  links,  modify  any  ob- 
ject and  insert  it  again  in  dependence  of  their  access  rights.  It  is  important  that  all  abilities  concerning  the  annotation 
of  documents  and  inserting  of  links  do  not  refer  to  locally  filed  documents  within  the  sTeam  databases.  For  this  pur- 
pose solutions  are  searched  that  allow  to  include  most  technologies  of  the  Internet.  Thus  it  will  be  very  important  to 
intensify  the  integration  of  existing  Internet  formats  and  services.  Within  the  application  spectrum  of  the  supplied 
communication  mechanism,  the  conception  of  sTeam  will  on  the  one  hand  be  designed  for  a mere  asynchronous  use 
(working  with  documents,  supplying  information  by  annotations)  and  on  the  other  hand  also  for  an  intensive  syn- 
chronous use  (online  co-ordination  and  online  discussion,  chat).  In  order  to  meet  this  requirement  the  supplied 
awareness  information  also  has  to  be  adjusted  to  this  situation.  The  information  degree  that  users  reveal  and  receive 


This  process  e.g.  intends  to  integrate  different  functionalities  out  of  the  field  of  office  systems  such  as  data  processing,  pic- 
ture processing  or  spread-sheet  calculation  by  providing  Java-based,  i.e.  browser-oriented  services  (e.g.  the  StarPortal  ap- 
proach of  the  SUN  company:  http://www.sun.com/products/staroffice/starportal/) 

^ Compare  the  “lost  in  hyperspace  syndrome”,  e.g.  (TTieng  et  al.  1 996)  "In  general,  the  lost  in  hyperspace  phenomenon  refers 
to  any  of  the  following  conditions:  users  cannot  identify  where  they  are;  users  cannot  return  to  previously  visited  informa- 
tion; users  cannot  go  to  information  believed  to  exist;  users  cannot  remember  what  they  have  covered;  and  users  cannot  re- 
member the  key  points  covered.  " 
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of  other  users  (where  they  are,  what  they  perform  at  present)  has  to  be  designed  for  flexible  use  and  adjusted  to  the 
application  context  of  the  corresponding  client. 

An  open  platform  requires  the  realisation  of  different  clients  and  the  use  of  specific  tools  within  semantic  spaces, 
both  in  dependence  of  the  users'  needs.  Thus  tools  for  accessing  materials  have  to  be  designed  with  different  ap- 
pearances and  functionalities  with  regard  to  the  number  of  users  co-operating  at  the  same  time.  The  appearance  of 
the  client  concerning  the  integration  into  a conventional  Internet  browser  as  well  as  the  adjustment  of  a working 
environment  and  its  fusion  with  the  client  (drag&drop  of  application  objects  into  the  sTeam  environment)  also  have 
to  fulfil  the  above-mentioned  prerequisites.  The  procedure  of  realising  the  idea  of  sTeam  can  briefly  be  outlined  as 
follows: 

In  the  last  two  years  the  first  step  was  taken  by  developing  prototypes  for  the  scrutiny  of  the  conception.  On  the  one 
hand  these  prototypes  try  to  realise  the  idea  of  semantic  spaces  in  practice  or  to  discuss  details  of  the  performed 
coupling  of  a CVE  with  a database  and  its  network-supported  access.  On  the  other  hand  these  prototypes  are  to 
guarantee  the  suitability  of  such  an  architecture  for  everyday  use  in  teaching  and  research.  Parallel  to  first  pilot  ap- 
plications the  creation  of  an  open-source-initiative  (-community) — concerning  sTeam — is  planned  in  expectation  of 
a broad  integration  of  existing  standards  and  tools.  Moreover  an  open-source  community  concerning  sTeam  seems 
to  be  a permanent  alternative  to  commercial  solutions  and  to  guarantee  constantly  the  topicality  of  the  developed 
platform.  Under  the  aspect  of  topicality,  expectations  exist  to  place  sTeam  on  a broad  basis  with  co-operating  and 
discussing  project  partners  as  well  as  collaborating  developers  and  thus  being  able  to  come  up  to  the  rapid  develop- 
ment of  current  Internet  standards.  Corresponding  examples  are  the  document  format  XML^  or  the  ISO-standard 
“Topic  Maps”^  which  provides  a standardised  notation  for  interchangeably  representing  information  about  the 
structure  of  information  resources  used  to  define  topics,  and  the  relationships  between  topics. — These  new  tech- 
nologies have  to  be  considered  within  the  conception. 

In  order  to  realise  an  open-source  initiative  it  will  be  essential  to  define  interfaces  carefully  and  to  write  sufficient 
documentations  to  be  able  to  co-ordinate  a high  number  of  developing  initiatives.  This  can  be  pointed  out  as  primary 
tasks  for  the  next  months.  In  this  sense  the  expectations  are  to  break  with  usual  practices  of  clarifying  research 
questions  by  prototypes  and  the  consequences  of  not  being  available  for  a wide  range  of  users.  The  bundling  of  ideas 
and  efforts  within  the  community  of  developers  of  computer-supported  co-operative  work  tools  and  Internet  tech- 
nology can  only  be  realised  by  synergies  of  an  open-source  community.  From  this  point  of  view  sTeam  represents  a 
consistently  open  platform  for  bundling  the  released  energy. 
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Abstract:  If  a “picture  is  worth  a thousand  words,”  then  why  have  attempts  over  the  past  decade  to 
use  pictures  and  animations  to  replace  or  supplement  traditional  instructional  methods  for  teaching 
algorithms  produced  such  disappointing  results?  In  an  earlier  paper  (Hansen  et  al.  1998)  we 
described  a research  project  based  on  the  premise  that  a rethinking  of  algorithm  animation  design  is 
required  in  order  to  harness  its  power  to  enhance  learning.  The  key  insight  was  that  for  algorithm 
animations  to  be  effective,  they  had  to  be  “chunked”  and  embedded  within  a context  and 
knowledge  providing  hypermedia  information  environment.  In  this  paper,  we  report  on  studies 
designed  to  discover  which  aspects  of  the  prototype  hypermedia  visualization  system  that  was 
developed  (called  Hal  Vis)  contributed  to  student  learning.  These  preliminary  studies  led  to  a 
surprising  discovery  that  interactive  and  animated  analogies  appear  to  significantly  prime  learning 
about  abstract  and  dynamic  algorithm  behaviors  from  subsequent  visualizations.  This  paper  briefly 
describes  the  interactive  features  and  learning  modules  of  Hal  Vis,  then  presents  the  results  of  two 
series  of  experiments  conducted  using  HalVis  that  give  insight  to  the  use  of  animated  analogies  to 
enhance  learning. 


Introduction 

Over  the  past  decade,  numerous  studies  and  experiments  have  been  conducted  to  evaluate  whether  graphically 
animating  algorithm  behavior  improved  student  learning  of  this  dynamic,  abstract,  and  difficult  subject  (e.g.  Byrne 
et  al.  1996;  Stasko  et  al.  1993).  While  the  addition  of  pictures  and  animations  are  enthusiastically  received  by 
students,  these  studies  have  not  proven  conclusively  that  algorithm  animations  actually  improve  learning  (see 
Hundhausen  1996  for  an  excellent  survey). 

We  believe  that  previous  attempts  at  using  algorithm  animations  as  learning  tools  were  unsatisfactory  not  due  to  any 
flaw  with  animation  as  a technique,  but  because  of  the  approach  used  to  convey  information  using  the  animations. 
Animations  by  themselves,  even  when  accompanied  by  some  textual  feedback  and  interactive  control,  may  not  be 
enough.  In  earlier  papers  (Hansen  et  al.  1998,  Hansen  et  al.  1999),  we  presented  research  projects  aimed  at 
rethinking  algorithm  animation  design.  The  prototype  visualization  system  we  designed  is  called  HalVis  (for 
Hypermedia  Algorithm  Visualization  System).  It  is  not  a mere  animation  presenter  in  the  mold  of  prior  algorithm 
animation  research  prototypes.  In  some  ways,  its  architecture  resembles  that  of  the  algorithm  animation  system 
described  in  (Recker  et  al.  1995)  containing  multiple  representations  in  addition  to  the  animation  itself.  Following 
the  design  principles  outlined  in  (Narayanan  & Hegarty  1998),  HalVis  incorporates  four  unique  features:  (1) 
Providing  three  kinds  of  animations  to  illustrate  different  views  of  algorithm  behavior  - an  animated  analogy  that 
illustrates  the  operational  characteristics  of  an  algorithm,  a micro-level  animation  that  focuses  on  details  of  the 
algorithm’s  behavior  on  small  data  sets,  and  a macro-level  animation  that  shows  the  algorithm’s  aggregate  behavior 
on  large  data  sets;  (2)  Embedding  animations  within  a hypermedia  visualization  that  also  employs  textual 
descriptions,  audio  narratives  and  static  diagrams  to  provide  contextual  information;  (3)  Presenting  animations  in 
discrete  chunks  accompanied  by  explanations  of  the  specific  actions  being  accomplished;  and  (4)  Encouraging 
student  participation  by  allowing  rich  interactions  with  the  animations  and  using  probes  or  questions  to  stimulate 
critical  thinking. 


414 


Two  empirical  studies  demonstrated  that  students  using  HalVis  significantly  outperformed  those  learning  from  a 
traditional  textbook.  Later,  a third  experiment  compared  learning  about  an  algorithm  from  HalVis  to  learning  from  a 
compilation  of  the  best  algorithm  descriptions  and  illustrations  (generated  by  examining  19  textbooks  published 
between  1974  and  1997)  and  subsequently  solving  a set  of  typical  problems  on  the  algorithm.  In  this  case  no 
significant  differences  were  found  between  the  group  of  students  who  learned  by  interacting  with  HalVis  and  the 
group  which  perused  the  printed  materials  and  then  solved  several  exercise  problems,  though  the  HalVis  group  had  a 
higher  post- test  mean  score.  Two  subsequent  experiments  compared  learning  by  interacting  with  HalVis  against 
learning  from  a typical  classroom  lecture  and  learning  from  an  algorithm  animation  typical  of  prior  research  on  this 
topic  (Stasko  1997),  In  both  cases  statistically  significant  performance  improvement  was  found  for  the  groups  of 
students  working  with  HalVis,  Thus,  five  empirical  studies  (summarized  in  Hansen  et  al,  1999)  involving  133 
undergraduates  have  thus  far  demonstrated  that  the  HalVis  system  significantly  improves  student  learning. 

These  results  led  us  to  ask  the  question:  Which  individual  or  combinations  of  features  are  producing  the  observed 
learning  benefits?  This  paper  describes  the  results  from  two  experiments  we  carried  out  to  help  answer  this 
question.  These  studies  led  to  a surprising  discovery  that  interactive  and  animated  analogies  appear  to  significantly 
prime  learning  about  abstract  and  dynamic  algorithm  behaviors  from  subsequent  visualizations.  First,  we  present 
the  interactive  features  and  learning  modules  of  HalVis,  Then  we  describe  the  experiments  conducted  on  HalVis. 
Finally,  we  conclude  with  a discussion  of  the  implications  of  the  results  of  these  experiments. 


Architecture  of  HalVis 

HalVis  is  structured  as  three  primaiy  modules  and  two  secondaiy  modules.  The  primary  modules  are  called  the 
Conceptual  View,  Detailed  View  and  Populated  View.  The  secondaiy  modules  are  called  Fundamentals  and 
Questions.  The  Fundamentals  module  provides  basic  knowledge  and  a context  for  understanding  the  algorithm 
being  presented  by  the  primary  modules  using  illustrated  and  (in  some  cases)  animated  explanations  of  basic 
algorithmic  concepts  such  as  complexity,  recursion  and  iteration.  This  module  is  accessible  from  all  of  the  primary 
modules  via  hyperlinks.  For  example,  if  the  primary  modules  are  presenting  a recursive  algorithm,  all  appearances 
of  the  word  “recursion”  in  these  modules  will  appear  as  underlined  hyperlinks  to  the  corresponding  descriptions  and 
depictions  in  the  Fundamentals  module.  The  Questions  module  provides  the  learner  with  an  opportunity  to  assess 
understanding  of  the  algorithm  by  posing  a series  of  questions  and  providing  immediate  feedback.  These  two 
modules  were  not  the  focus  of  the  studies  described  in  this  paper. 

The  Conceptual  View  (CV)  introduces  a specific  algorithm  in  very  general  terms  using  a real  world  analogy.  The 
analogy  is  interactive  (students  can  manipulate  it),  animated,  and  accompanied  by  text  and  audio  explanations.  Each 
analogy  highlights  the  central  conceptual  elements  of  the  corresponding  algorithm.  For  instance,  the  analogy  for  the 
Merge  algorithm  (Figure  1)  is  animated  playing  cards  which  are  divided  into  stacks  and  merged  to  create  a sorted 
sequence  of  cards.  A gaming  mode  is  provided  for  the  student  to  try  to  ‘merge’  the  cards  with  fewest  errors  and 
minimal  time.  The  animated  analogy  for  the  Quicksort  algorithm  is  that  of  a coach  ordering  a set  of  players  by 
height,  using  a series  of  moves  illustrated  through  smooth  animation  that  capture  the  essence  of  the  Quicksort 
algorithm  in  terms  of  its  fundamental  operations.  The  Shortest  Path  algorithm  used  an  analogy  involving  airfares 
between  various  cities,  a typical  problem  facing  travelers. 

The  analogies  in  HalVis  can  be  described  in  several  dimensions:  (1)  dynamics,  referring  to  the  sophistication  of  the 
animition;  (2)  fidelity,  referring  to  how  closely  the  features  of  the  analogy  maps  to  the  central  characteristics  of  the 
algorithm;  (3)  interactivity,  referring  to  the  level  of  immersion  allowed  to  the  user,  ranging  from  simple  pacing  to 
competitive  simulation;  and  (4)  familiarity,  referring  to  how  common  the  objects  of  the  analogy  tend  to  be.  The 
notion  is  that  more  is  better  for  each  of  these  dimensions  to  reduce  the  cognitive  load  imposed  by  the  analogy.  The 
more  dynamic,  the  better  the  fidelity,  the  more  interactive,  the  more  familiar,  the  better. 
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Introduction  toMcr^ng 


Figure  1.  The  Conceptual  View  Screen  of  the  Merge  Aigorithm 


The  Detailed  View  (DV)  is  a module  that  takes  the  multiple  representation  approach  to  presenting  detailed 
information  about  how  the  individual  steps  of  an  algorithm,  when  executed  in  sequence,  manipulate  data  and 
accomplish  a specific  task  (e.g.,  sorting  a set  of  numbers).  It  presents  four  windows  containing  information  that  are 
synchronously  updated  as  the  algorithm  processes  user-selected  input  data  sets:  (1)  the  Execution  Animation 
window  shows  how  steps  of  the  algorithm  modify  data  structures  using  smooth  animation  of  micro-level  operations; 
(2)  the  Execution  Status  Message  window  provides  comments  and  textual  feedback  to  the  student  about  key  events 
and  actions  occurring  in  the  animation,  also  available  as  an  audio  commentary;  (3)  the  Pseudocode  window  shows 
statement- level  algorithm  operations  by  highlighting  the  statement  being  executed;  and  (4)  the  Execution  Variables 
window  contains  a scoreboard-like  panorama  of  the  variables  involved  in  the  algorithm  and  their  changing  values. 
These  four  windows  are  shown  in  Figure  2a,  which  is  a screen  capture  of  one  frame  of  the  DV  micro-level 
animation  s for  the  Shortest  Path  algorithm.  The  Execution  Animation  Window  appears  on  the  upper  left  (below  the 
control  buttons)  of  this  figure,  the  Pseudocode  window  appears  on  the  upper  right,  the  Execution  Variables  window 
appears  on  the  lower  left,  and  the  Execution  Status  Message  window  appears  on  the  lower  right 


Figure  2.  The  Detailed  View  and  Populated  View  Screens  of  the  MergeSort  Aigorithm 


The  Populated  View  (PV)  module  presents  a macro-level  animated  view  of  the  algorithm’s  behavior  on  large  data 
sets.  A novel  feature  of  this  module  is  a facility  for  the  student  to  make  predictions  about  different  parameters  of 
algorithm  performance,  and  then  compare  those  against  the  actual  performance  when  the  animation  is  running.  A 
sample  screen  of  this  module  appears  in  Figure  2b. 
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Experiments 


The  number  of  novel  features  and  modules  that  distinguish  HalVis  from  prior  algorithm  animation  approaches  led  to 
encouraging  results  but  begged  the  question:  Which  individual  or  combinations  of  features  produced  the  observed 
learning  benefits?  Our  approach  to  answering  this  was  to  build  different  versions  of  HalVis  by  selectively  disabling 
individual  or  groups  of  related  features,  and  experimentally  evaluate  student  performance  using  these  versions 
against  the  original.  We  designed  and  carried  out  two  such  studies.  The  first  experiment  disabled  one  of  the  three 
views,  giving  the  student  two  views  with  which  to  learn  the  algorithm.  The  second  experiment  disabled  two  of  the 
three  views,  providing  the  student  with  only  one  view,  as  shown  in  Table  1 . Each  experiment  is  described  below. 


Experiment  1 Groups 

(l-view  removed) 

Experiment  11  Groups 

(2  views  removed) 

CDP 

CD 

DP 

CP 

CDP 

C 

D 

P 

Conceptual  View 

Y 

Y 

Y 

Y 

Y 

Detailed  View 

Y 

Y 

Y 

Y 

Y 

Populated  View 

Y 

Y 

Y 

Y 

Y 

Table  1.  Views  Available  to  Groups  for  Experiments  I and  II 


Experiment  1 Procedure:  The  first  experiment  involved  32  undergraduate  computer  science  students  enrolled  in  a 
third  year  algorithm  analysis  course  at  Auburn  University.  Four  matched  groups  of  students  interacted  with  one  of 
four  versions  of  HalVis  illustrating  the  Quicksort  algorithm:  (1)  a complete  version  with  all  three  primary 
modules/views  enabled  (referred  to  as  the  CV-DV-PV  group  or  CDP  for  short);  (2)  a version  with  the  CV  disabled 
(referred  to  as  the  DP  group);  (3)  a version  with  the  DV  disabled  (the  CP  group);  and  (4)  a version  with  the  PV 
disabled  (the  CD  group).  A pre-test/post-test  combination  measured  individual  learning  and  improvement  with 
questions  that  probed  conceptual  and  procedural  knowledge  about  the  algorithm.  Students  were  tested  on  their 
ability  to  recognize  and  reorder  pseudocode  descriptions  of  algorithms,  mentally  simulate  algorithmic  operations, 
and  predict  resulting  data  structure  changes.  Each  group  received  a brief,  navigation-only  orientation  to  the  version 
of  HalVis  they  were  to  use,  then  were  assigned  to  a computer  and  instructed  to  interact  with  the  visualization  until 
they  felt  they  understood  the  algorithm.  No  time  limits  were  imposed  for  the  tests  or  the  visualization. 

Experiment  I Results:  Our  hypothesis  was  that  the  most  important  view  was  the  most  information  rich  view  - the 
Detailed  View — and  the  groups  that  interacted  with  this  view  would  outperform  any  group  that  was  denied  this 
view.  Furthermore,  we  expected  the  Populated  View  would  follow  in  significance  and  that  the  contribution  of  the 
Conceptual  View  would  be  the  least.  Figure  3 (left  side)  shows  the  average  group  improvement.  As  expected,  the 
group  that  received  all  three  HalVis  views  performed  the  best.  However,  these  results  also  reveal  a surprising 
impact  of  the  Conceptual  View.  Contrary  to  our  expectations,  the  groups  that  performed  best  were  not  the  ones 
exposed  to  the  Detailed  View  but  rather  the  groups  that  interacted  with  the  Conceptual  View.  The  improvements  of 
the  groups  that  received  the  Conceptual  View  with  another  view  were  more  than  twice  the  improvement  of  the  DP 
group.  Table  2 shows  a statistical  summary  of  this  experiment,  presenting  data  for  pairwise  comparisons  between 
each  of  the  groups  with  access  to  the  Conceptual  View  and  the  one  group  -DP  - that  did  not  have  this  access.  Note 
that  all  comparisons  yielded  statistically  significant  results.  Perhaps  the  most  noteworthy  observation  from  the 
results  of  this  experiment  was  the  effect  of  the  Conceptual  View  in  priming  the  learning  of  information  presented  in 
subsequent  views.  The  groups  that  interacted  with  the  Conceptual  View  in  any  combination  with  other  views 
performed  better  than  the  group  that  lacked  the  Conceptual  View.  The  impact  of  the  Conceptual  View  was 
examined  further  in  the  next  study  that  elided  two  views  at  a time. 
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Table  2.  Statistical  Summary  for  Experiment  I 
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Experiment  II  Procedure:  This  experiment  was  designed  to  isolate  each  of  the  three  views  of  our  algorithm 
visualization  framework  to  measure  their  respective  impact  and  effectiveness.  Procedurally  identical  to  experiment 
I,  twenty-seven  undergraduate  students  enrolled  in  a third  year  algorithm  analysis  course  at  Auburn  University 
participated  in  four  matched  groups:  (1)  the  CDP  group  worked  with  a full  version  of  HalVis  illustrating  a graph 
algorithm  to  find  shortest  paths  between  nodes;  (2)  the  C group  worked  only  with  the  Conceptual  View  of  this 
algorithm;  (3)  the  D group  worked  only  with  the  Detailed  View  of  this  algorithm;  and  (4)  the  P group  worked  only 
with  the  Populated  View  of  this  algorithm.  Our  hypothesis  was  that  the  Detailed  View  would  prove  to  be  the  most 
valuable  because  of  the  amount  of  information  it  provided.  We  were  uncertain  how  the  performances  of  the  other 
views  would  get  ranked,  since  neither  the  Populated  View  nor  the  Conceptual  View  contained  the  volume  or  depth 
of  information  available  in  the  Detailed  View. 

Experiment  II  Results:  Figure  3 (right  side)  shows  average  improvements  observed  in  each  of  the  groups.  As 
expected,  the  CDP  group  outperformed  the  others,  followed  closely  by  the  D group.  Interestingly,  the  C group 
outperformed  the  P group  by  21%.  It  is  illuminating  to  note  how  well  the  C group  did  with  the  limited  amount  of 
information  that  they  received.  Table  3 summarizes  the  various  statistical  comparisons  between  the  full-version 
group  and  groups  with  individual  views,  and  between  groups  with  individual  views.  Except  for  the  CDP/D 
comparison,  the  results  are  all  statistically  significant. 
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Figure  3.  Improvement  by  Group  for  Experiments  I and  II 
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Table  3.  Statistical  Summary  for  Experiment  II 


The  importance  of  the  Detailed  View  was  confirmed,  as  was  the  value  of  the  Conceptual  View.  We  were  surprised 
at  the  level  of  improvement  observed  in  the  group  that  only  interacted  with  the  Conceptual  View,  and  this  suggests 
that  having  a good  analogy  can  produce  surprisingly  positive  results.  The  performance  of  the  Populated  View  group 
lagged  behind  the  others.  Yet  the  interaction  logs  we  collected  revealed  that  in  each  of  the  experiments,  the 
animation  in  this  view  was  executed  about  the  same  number  of  times  as  the  animations  in  the  other  two  views. 
Perhaps  this  serves  as  confirmation  that  some  animations  are  merely  ‘candy  for  the  eyes’  in  that  they  are 
entertaining  to  observe  but  not  particularly  informative.  It  should  also  be  noted  that  the  animation  in  the  Populated 
View  closely  resembles  algorithm  animations  developed  in  prior  research.  Interestingly,  students  generally 
commented  most  favorably  about  the  animation  in  the  Populated  View.  It  appears  that  they  thought  they  were 
learning  more  from  the  Populated  View  than  was  actually  the  case. 
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Conclusion 


While  we  initially  believed  that  the  analogy/real-world  example  was  a minor  educational  addition  and  feature  of  our 
framework,  it  appears  to  have  a pronounced  effect  in  peaking  student  interest  and  priming  them  for  learning  from 
subsequent  views  of  greater  detail.  There  exists  a significant  body  of  psychological  literature  on  the  role  of 
analogies  in  communicating  a target  concept  (e.g.  Centner  1989).  Researchers  have  also  investigated  the 
educational  value  of  paired  interactive  simulations  as  analogies  which  complement  each  other  (Brophy  & Schwartz 
1998).  However,  we  believe  this  is  the  first  time  that  the  role  of  animated  analogies  in  understanding  computer 
algorithms  from  visualizations  has  been  empirically  studied.  The  power  of  animated  analogies  in  this  domain  arises 
from  the  fact  that  these  provide  conceptual  bridges  between  familiar  scenarios  and  abstract  components  of 
algorithms.  It  has  been  noticed  that  students  tend  to  employ  analogies  in  describing  how  algorithms  operate 
(Douglas  et  al.  1995;  Stasko  1997)  and  analogies  can  serve  to  provide  a form  of  scaffolding  (Hmelo  & Guzdial 
1996)  for  subsequent  learning.  Results  reported  here  lead  us  to  speculate  that  concrete  dynamic  analogies  can  prime 
students  to  learn  better  from  visualizations  of  abstract  dynamic  phenomena.  If  confirmed  by  additional  experiments, 
this  has  important  implications  for  learning  technology.  Therefore,  our  future  research  is  aimed  at  seeking 
additional  empirical  evidence  regarding  this  conclusion.  Finer  grained  ablation  studies  are  being  planned  to 
examine  the  differential  effects  these  features  have  on  the  ability  of  analogies  to  prime  subsequent  learning. 
Additional  studies  looking  at  the  other  features  - animation  chunking,  animation  embedding  and  rich  interactivity  - 
are  also  on  the  drawing  board. 
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Designers  of  interactive  learning  environments,  applying  contemporary  views  of  learning,  are  now 
employing  design  models  that  incorporate  the  variety  of  ideas  that  are  based  on  constructivist  frameworks. 
Designers  are  attempting  to  give  learners  control  over  their  actions  and  place  them  into  worlds  that  can  be 
manipulated  and  interrogated.  These  environments,  if  well  designed,  can  support  learner  construction  of 
knowledge  through  structured  or  ill-structured  problem  solving  experiences.  Such  frameworks  are  based 
upon  arguments  that  learners  should  be  placed  in  authentic  environments  that  incorporate  sophisticated 
representations  of  context  through  such  constructs  as  "virtual  worlds".  Within  these  modem  constructions, 
learners  are  often  given  a rich  set  of  resources  to  construct  artefacts  which  reflect  their  problems  solutions. 

However,  researchers  such  as  Seymour  Papert  have  long  called  for  more  open  environments  drawn  from  a 
theoretical  view  of  learning  termed  constructionism  which  is  based  on  two  different  senses  of 
‘construction’.  It  is  grounded  in  the  idea  that  people  learn  by  actively  constructing  new  knowledge  as  well 
as  asserting  that  learners  are  particularly  likely  to  create  new  ideas  when  they  are  engaged  in  ‘constructing’ 
personally  meaningful  artefacts.  Multimedia  authoring  tools  have  long  been  used  to  address  the  latter 
component  of  constructionism.  The  challenge  for  researchers  and  developers  is  to  find  the  balance  between 
resource  rich  ‘virtual  worlds’  with  embedded  tasks  and  open  ended  construction  that  may  not  be  goal 
driven,  both  resulting  in  personally  meaningful  artefacts,  ie  are  we  designing  for  learning,  or  are  we 
learning  by  designing?  This  paper  examines  three  projects  that  demonstrate  these  issues. 
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Abstract:  In  recent  years,  virtual  reality  (VR)  has  become  an  increasingly  powerful  and 

inexpensive  tool  for  developing  interfaces  for  a variety  of  applications.  Even  though  there  has 
been  an  increase  in  the  use  of  virtual  reality  (VR),  there  is  a lack  of  software  that  bridges  the  gap 
between  existing  application  programs  and  VR.  This  type  of  middleware  is  needed  to  manage 
the  navigational  interaction  between  existing  application  programs  and  VR.  This  paper  describes 
a language  for  describing  navigation  behaviors  in  3-dimensional  worlds  that  could  potentially 
serve  as  middleware.  The  formal  language  described  in  this  paper  permits  precise  navigation 
path  specification  and  can  be  used  as  the  means  of  mediation  with  application  programs.  The 
language  can  be  used  in  achieving  tasks  such  as  exploring  virtual  worlds,  programming  the 
navigation  and  routing  strategies  for  avatars,  simulating  the  trajectories  of  virtual  objects,  and 
providing  a formal  description  language  for  documenting  user  navigation  in  VR  environments. 


Introduction 

In  recent  years,  virtual  reality  (VR)  has  become  an  increasingly  powerful  and  inexpensive  tool  for 
developing  interfaces  to  applications  such  as  architectural  modeling,  computer-aided  design,  data  visualization, 
telecommuting,  and  environments  for  simulation  and  entertainment.  Interaction  in  applications  that 
incorporate  VR  as  an  interface  is  dynamic  by  nature.  Users  immerse  themselves  in  3 -dimensional  (3D) 
environments  using  specialized  peripheral  devices  including  head-mounted  displays,  data  gloves,  3D-trackers 
and  body  suits.  These  devices  enable  users  to  experience  rich  stimuli  and  give  them  control  over  where  they  go 
and  what  they  do  within  the  confines  of  an  application.  This  freedom  is  enabled  by  the  ability  to  navigate  in 
the  world. 

Navigation  is  an  aspect  of  behavior  that  causes  an  object  to  change  its  location,  orientation,  and/or 
relationship  to  other  objects  in  its  environment  (Matsuba,  1996).  User  interface  designers  have  previously 
investigated  navigation  in  2-dimensional  (2D)  spaces  and  developed  design  principles  that  guide  them  in  the 
development  of  effective  and  efficient  interfaces  (Beasley,  1992;  Beccue,  1994).  As  designers  work  with  3D 
worlds,  it  will  be  useful  to  know  whether  the  principles  and  findings  of  research  related  to  two-dimensional 
navigation  can  be  applied  and  extended.  Since  virtual  worlds  represent  real  worlds,  the  impact  of  real  world 
navigational  behaviors  on  interface  design  must  also  be  studied. 

A review  of  the  literature  related  to  navigation  patterns  in  human  computer  interfaces  indicates  that 
research  on  navigation  in  3D  environments  focuses  on  three  areas:  navigational  awareness,  spatial  ability,  and 
wayfinding  (purposeful,  oriented  movement  during  navigation)  (Satalich,1995).  Darken  and  Sibert  found  that 
the  organization  of  elements  in  a virtual  world  and  the  provision  of  an  absolute  frame  of  reference  improved 
navigation  performance  (Darken,  1995).  In  a subsequent  study.  Darken  and  Sibert  also  reported  an 
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improvement  in  the  performance  of  navigation  tasks  in  virtual  worlds  that  had  been  created  using 
environmental  design  principles  (Darken,  1996).  Among  their  observations  were  that  (1)  path  following  (a 
form  of  wayfinding)  is  a natural  spatial  behavior,  (2)  the  lack  of  adequate  directional  cues  inhibited  wayfinding 
performance,  (3)  a structure  must  be  imposed  on  large  complex  worlds  in  order  to  navigate  in  them,  and  (4)  a 
conceptual  coordinate  system  is  often  imposed  on  the  world  in  an  attempt  to  structure  it.  Each  of  these  studies 
required  that  the  researcher  track  the  users’  navigational  patterns  in  a virtual  world.  Vila,  et  al.  (Vila,  1997) 
reported  on  a tool  that  provides  a template  and  efficient  data  collection  capabilities  for  studying  the 
navigational  behaviors  of  users.  Additional  efforts  to  improve  navigation  performance  by  addressing  issues 
such  as  the  psychology  of  navigation,  the  use  of  navigational  aids  (e.g.,  maps),  the  effects  of  different  types  of 
worlds  on  navigation,  and  the  design  of  an  environment  for  navigability  were  the  subject  of  discussions  at  the 
1997  ACM  Computer  Human  Interaction  Conference  (Furnas,  1997).  In  reviewing  the  literature,  the  authors 
found  virtually  no  reports  on  tools  for  connecting  an  existing  application  program  to  a virtual  world.  This  could 
be  resolved  with  the  development  of  middleware  to  manage  the  navigational  interaction  between  an  application 
program  and  a virtual  world. 

The  purpose  of  this  paper  is  to  describe  the  prototype  for  a language  for  navigating  in  3D  worlds  that 
has  the  potential  to  serve  as  a mediator  between  an  application  and  a virtual  world.  Mariner y the  language 
discussed  in  this  paper,  was  designed  using  formal  language  specification  techniques  and  implemented  in  a 
Web  environment  using  Javascript  and  Virtual  Reality  Modeling  Language  (VRML). 

Virtual  worlds  provide  a forum  for  developing  simulations  of  real  world  situations  for  education  and 
training  that  are  less  costly  than  real  experiences.  Applications  like  these  need  to  be  able  to  interact  with  a 
virtual  world.  Mariner  provides  an  initial  step  in  connecting  existing  applications  and  VR  worlds.  Because 
Mariner  provides  a formal  method  for  specifying  navigation,  it  may  also  prove  useful  to  researchers  in  the  field 
of  human  computer  interaction  who  are  interested  in  describing  and  studying  navigation  behaviors  in  3D 
worlds  by  documenting  how  users  navigate  in  VR  environments. 


Development  of  the  Mariner  Language 

The  authors’  objectives  in  the  design  and  implementation  of  Mar/>ier  were  to: 

(1)  define  the  specific  navigation  behaviors  in  a 3D  virtual  world  that  need  to  be  modeled, 

(2)  use  formal  language  specification  techniques  to  define  the  syntactic  and  semantic  structure  of  Mariner y 

(3)  implement  a prototype  for  an  interpreter  using  standard  compiler  design  techniques  that  would 
translate  Mariner  language  statements  into  VRML  statements  allowing  the  visualization  of  an  object 
as  it  navigates  in  a 3D  world, 

(4)  determine  the  effectiveness  of  Mariner  by  evaluating  the  accuracy  of  programs  in  Mariner  that 
generate  navigation  actions  for  an  object  defined  in  a VRML  world. 

Defining  Behaviors 

The  first  step  in  developing  Mariner  was  to  determine  what  type  of  navigation  behaviors  occur  in  3D 
worlds.  The  authors  identified  movement , rotation,  collision  detection,  conditional  behavior  and  repetitious 
behavior  as  the  behaviors  to  model  in  the  language. 

Movements  in  a 2D  world  are  usually  expressed  using  a grid  coordinate  system.  Movements  can 
either  be  direct  (e.g.,  moveto  138,34)  or  relative  (e.g.,  move  forward  10).  Some  parameters  to  commands  may 
be  implied;  “move  forward  10”  implies  that  the  object  is  facing  in  a specified  direction  and  will  continue  to 
move  in  that  direction  when  it  moves.  In  a 3D  world,  some  2D  concepts  have  the  same  meaning.  For  example, 
movements  may  also  be  expressed  using  a grid  (now  a 3D  grid)  and  the  concept  of  moving  forward  a specified 
number  of  units  remains  the  same.  However,  the  movement  of  objects  is  no  longer  defined  in  a plane;  objects 
have  the  freedom  to  move  (and  rotate)  in  all  of  the  directions  that  can  be  defined  by  a 3D  space. 

Whether  in  a 2D  or  3D  world,  objects  have  relationships  to  other  objects  in  the  world.  An  object  may 
be  to  the  right  of  another  object  or  in  front  of  another  object.  As  an  object  navigates,  its  relationship  to  other 
objects  changes.  Correct  and  incorrect  behaviors  must  be  defined.  In  navigating  within  a world  with  multiple 
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objects,  for  example,  it  would  not  be  correct  for  an  object  to  move  forward  “through”  another  object  as  if  it  did 
not  exist.  An  object  should  be  able  to  detect  and  avoid  an  impending  collision  with  another  object. 

Objects  in  virtual  worlds  often  exhibit  repetitive  behavior  or  behavior  that  is  based  on  conditions  that 
must  be  evaluated  before  performing  the  behavior.  For  example,  not  to  move  if  there  is  an  object  directly  in 
front  of  the  object  to  be  moved.  Hence,  control  structures  that  allow  for  conditional  or  repetitious  behavior  are 
also  needed. 

Syntax  and  Semantics 

After  defining  the  behaviors  to  model,  the  syntax  and  semantics  of  the  Mariner  were  specified. 
Because  languages  that  have  been  unambiguously  defined  using  Backus  Naur  Form  (BNF)  can  often  be  easily 
implemented  using  recursive,  top  down  analysis  techniques  and  automatic  compiler  development  tools,  the 
syntax  for  Mariner  was  specified  using  BNF  (Alblas,  1996).  The  BNF  for  Mariner  appears  on  page  5. 

The  semantics  were  implemented  as  paragraphs  of  text  describing  the  behavior  of  each  statement.  For 
the  most  part  the  meaning  of  program  statements  can  be  inferred  from  their  syntax.  While  developing  the 
semantics,  it  proved  useful  to  identify  potential  errors  in  syntax  and  behavior  so  that  error  handling  routines 
could  be  later  developed. 

Currently,  the  syntax  represents  a language  in  which  small  programs  can  be  written  to  manipulate  a 
limited  number  of  objects  in  a 3D  world.  The  objects  that  can  be  manipulated  include  a sphere,  cone,  cylinder 
and  cube.  The  current  syntax  and  semantics  will  form  the  base  for  implementing  the  language  as  a form  of 
middleware  to  interact  with  other  applications.  An  example  of  a small  Mariner  program  is  shown  in  Figure  1 . 
Note  that  the  object  that  has  been  defined  is  a cone  and  all  subsequent  movements  apply  to  that  object.  In  its 
current  form,  there  are  no  variables  in  Mariner,  so  there  is  no  way  to  reference  the  cone  in  later  segments  of 
code  that  are  separated  from  its  initial  definition. 


Cone: 

Size  = 2 
Height  = 4 
Color  =120  0 0 
Movement  1 

Move  5 Units  Along  X Axis 
Move  0.0  Units  Along  Y Axis 
Move  0.0  Units  Along  Z Axis 

Rotate  Counter  ClockWise  Along  Z Axis  by  90  Degrees 
Movement  2 

Move  0 Units  Along  X Axis 

Move  5 Units  Along  Y Axis 

Move  0.0  Units  Along  Z Axis 

Rotate  Clockwise  Along  Z Axis  by  90  Degrees 


Figure  1:  Program  Example 


The  Manner  Language  Development  Environment 

The  Mariner  language  development  environment,  shown  in  Figure  2 below,  has  two  components,  the 
Mariner  code  generator  and  the  Mariner  interpreter.  The  Mariner  code  generator  allows  the  user  to  define 
objects  and  their  navigation  paths  within  a virtual  world  through  a user-friendly  interface.  The  user  can  select 
pre-defined  options  for  each  object  for  which  Mariner  statements  will  be  generated  and  displayed  in  the  text 
area,  or  he/she  can  type  Mariner  language  statements  directly  in  the  text  area.  The  language  development 
environment  was  constructed  as  a Web  page  using  Javascript  and  VRML  2.0. 
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The  code  generator  permits  the  user  to  define  an  object  geometry  that  includes  traditional  VRML 
nodes  such  as  cones  and  cubes.  In  addition,  the  user  can  define  navigational  movements  for  each  defined  object 
using  translation  and  rotation  transformations.  All  object  dimensions  and  movement  distances  are  specified  in 
pixel  units.  After  completing  a Mariner  program,  the  user  can  choose  to  have  it  interpreted  into  VRML 
statements  which  can  be  executed  in  a Web  browser. 


VRML  Creator 
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Summary 


A prototype  for  a 3D  navigational  language,  Mariner^  was  described  in  this  paper  which  has  the 
potential  to  serve  as  a link  between  3D  worlds  and  software  applications.  The  language,  written  in  Javascript 
and  VRML,  currently  generates  VRML  code  which  can  be  viewed  within  a Web  browser.  Future  development 
includes  the  transformation  of  the  language  into  code  segments  that  can  be  embedded  within  an  existing  3D 
world  and  act  upon  existing  or  newly  created  objects  within  the  world. 


Mariner  BNF 

<Program_Unit> 
<Object_Declarati  on  > 


<Cone> 

<Box> 

<Cylinder> 

<Sphere> 

<Base_Measu  remen  t> 
<Height_  Measurement> 
<Color_Measurement> 
<Statement_Sequence  > 


<Movement_Number> 

<Statement> 


<Move_Statemen  t> 


<M  o veJStatemen  t_X> 
<Mo  ve_Statem  en  t_  Y> 
<M o ve_Statem  en  t_Z> 
<M o veto_Statemen  t> 
<Rotation_Statement> 


<Spin_Mode> 


“>  <Object_Declaration>  <Statement_Sequence> 

I <Object_Declaration> 

^ <Cone> 

I <Cylinder> 

I <Box> 

I <Sphere> 

Cone:  <Base_Measurement>  < Height_Measurement>  <Color_Measurement> 
^ Box:  <Base_Measurement>  <Color_Measurement> 

^ Cylinder:  <Base_Measurement>  < Height_Measurement> 
<Color_Measurement> 

^ Sphere:  <Base_Measurement>  <Color_Measurement> 

^ Size  = <Number> 

^ Height  = <Number> 

^ Color  = <Color_Number> 

^ <Statement_Sequence>  <Statement> 

I <Movement_Number>  <Statement_Sequence>  <Statement> 

I <Movement_Number>  <Statement> 

I <Statement> 

^ Movement  <Number> 

<Move_Statement> 

I <Rotation_Statement> 

I < If_Statement> 

I < While_Statement> 

I <For_Statement> 

<Move_Statement_X> 

I <Move_Statement_Y> 

I <Move_Statement_Z> 

I <Moveto_Statement> 

Move  <Number>  Units  Along  X Axis 
Move  <Number>  Units  Along  Y Axis 
Move  <Number>  Units  Along  Z Axis 
Moveto  <X_Coord>  <Y_Coord>  <Z_Coord> 

Rotate  <Spin_Mode>  Along  X Axis  by  <Number>  Degrees 
I Rotate  <Spin_Mode>  Along  Y Axis  by  <Number>  Degrees 
I Rotzte<Spin_Mode>A\ong  Z Axis  by  <Number>  Degrees 
^ Clockwise  | Counter  Clockwise 
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<If_Statement> 


if  <Condition>  then 


<Statement_Sequence> 


endif 

I if  <Condition  > then 


<StatementJequence> 

else 


<While_Statement> 


<Statement_Sequence> 

endif 

“>  while  (<Condition>) 


<Statement_Sequence> 


<Condition> 
<For_Statemen  f> 


end  loop 
“>  Path_Clear 

for  (<Number>)  to  <Number> 


<Statement_Sequence> 


I for  (<Number>)  downto  <Number> 


<Statement_Sequence> 


<Digit> 

<Color_Number> 

<Color_Value> 


<X_Coord> 

<YjCoord> 

<Z_Coord> 

<Number> 


“>  <Number> 

<Number> 

“>  <Number> 

“>  <Number><Digit> 

I <Digit> 

^ 0|1|2|3|4|5|6|7|8|9 

“>  <Color_value><Color_value><Color_value> 

^ 0/1  12/3... 12 5 5 
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Abstract;  This  study  investigated  how  differences  in  task  affected  learner  navigation  choices,  transfer  of  knowledge,  and  attitude 
toward  the  topic  when  using  hypertext  materials  about  sexual  harassment  that  were  designed  using  cognitive  flexibility  theory  as 
a theoretical  basis.  Based  upon  the  findings  of  the  study,  it  appears  that  the  two  tasks  (juror  verdict  and  policy  creation)  did  not 
have  significant  impact  upon  learner  criss  crossing  or  transfer.  Therefore,  it  was  also  apparent  that  task  had  no  relationship  to 
transfer.  Post  hoc  analyses  on  other  variables,  including  learner  beliefs  about  knowledge,  were  also  not  significantly  related  to  the 
outcomes  regarding  navigation  and  transfer.  However,  there  did  appear  to  be  a relationship  between  task  type  and  learner 
attitude  changes  from  the  beginning  to  the  end  of  the  study,  suggesting  a possible  affective  dimension  to  the  use  of  cognitive 
flexibility  hypertexts  that  has  not  been  previously  reported  or  considered  in  the  research  literature. 


Introduction 

This  study  investigated  the  practical  application  of  cognitive  flexibility  theory  (Spiro,  Vispoel,  Schmitz, 
Samarapungavan,  & Boerger,  1987)  as  the  basis  for  designing  case-based  hypertext  learning  materials.  Results  of 
this  research  could  aid  in  furthering  the  utilization  of  this  approach  for  facilitating  learning  within  instructional 
settings. 

Cognitive  flexibility  theory  posits  that  effective  learning  of  complex  knowledge  in  ill-structured  domains 
cannot  be  supported  by  those  cognitive  theoretical  frameworks  that  support  learning  in  well-structured  domains 
(Spiro  et.  al.,  1987).  This  is  due  to  the  fact  that  those  frameworks  focus  more  on  the  retrieval  of  information  and 
necessitate  fairly  rigid  structuring  of  the  domain  knowledge,  compartmentalization  of  concepts  vs.  interconnecting 
concepts,  simplification  of  complex  concepts  for  instruction,  and  assume  knowledge  will  be  applied  consistently 
within  the  domain.  These  approaches  inhibit  the  new  transfer  of  learning  within  ill-structured  domains  where 
flexible  knowledge  assembly  is  required  based  upon  the  context  of  the  situation  in  which  the  knowledge  is  applied 
(Spiro  et.  al.,  1987). 

Cognitive  flexibility  theory  assumes  that  in  order  to  affect  the  optimal  cognitive  processes  for  near  transfer 
within  the  learner,  instruction  must  stress  '‘flexible  reassembly  of  pre-existing  knowledge  to  adaptively  fit  the  needs 
of  the  new  situation”  (Spiro  et.  al.,  1987)  by  mirroring  that  flexibility  within  the  instructional  materials.  This  is  done 
using  a case-based  approach  in  which  learners  are  shown  multiple  representations  and  perspectives  on  each  domain- 
specific  case,  with  cases  interconnected  across  themes  and  perspectives.  The  theory  contends  that  this  prepares  the 
learner  for  transfer  through  a complex  understanding  of  the  complexity  of  the  domain  and  the  interconnectedness  of 
the  domain  knowledge  (Spiro  et.  al.,  1987).  This  more  complex  understanding  comes  from  learners  reconciling  the 
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perspectives  and  contexts  of  the  cases,  and  theoretically  should  support  the  construction  of  individual  knowledge 
representations  that  are  flexible  and  adaptable  to  new  situations  (Jonassen,  Dyer,  Peters,  Robinson,  Harvey,  King,  & 
Loughner,  1997). 

Transfer  in  cognitive  flexibility  literature  is  considered  the  application  of  new  knowledge  in  new  situations 
(Spiro  et.  al.,  1987).  Spiro  et.  al.  (1987)  argue  that  the  problem  with  applying  new  knowledge  in  ill-structured 
domains  is  that  the  original  presentation  of  that  knowledge  is  usually  simplified,  thus  resulting  in  simple 
understandings  of  the  domain  by  the  learner  The  problem  is  that  such  simple  understandings  do  not  readily  allow 
for  application  when  the  domain  is  inherently  complex.  Therefore,  it  is  necessary  to  present  the  domain  knowledge 
in  a form  that  retains  the  complexity  of  the  domain.  Complex  understandings,  the  theory  holds,  should  then  be  more 
flexibly  reassembled,  or  reinterpreted,  in  order  to  address  a wide  range  of  new  situations  based  upon  the  context  of 
each  situation.  Transfer  in  ill-structured  domains,  according  to  cognitive  flexibility  theory,  is  then  considered  to  be 
a product  of  the  learner  being  able  to  think  about  and  structure  domain  knowledge  in  accordance  with  the  new 
situation 

To  date  the  research  has  supported  the  transfer  claims  of  cognitive  flexibility  theory  by  showing  an 
improved  transfer  of  knowledge  application,  as  measured  by  essay  answers  given  by  learners,  when  approaches 
based  on  cognitive  flexibility  theory  which  emphasize  complexity  and  the  goal  of  flexible  assembly  of  knowledge 
are  compared  to  other  instructional  techniques  which  promote  simpler  understandings  and  outcomes  such  as 
retrieval  of  knowledge  (Spiro  et.  al.,  1987;  Jacobson  & Spiro,  1995;  Jacobson,  Maouri,  Mishra,  & Kolar,  1996). 
However,  the  learners  had  to  be  managed  through  the  instruction  by  providing  a list  of  only  those  thematic  links  that 
were  considered  by  the  designers  to  be  relevant  to  the  particular  case  being  viewed  From  a design  and 
implementation  viewpoint,  it  would  seem  that  managing  the  learner  would  be  unnecessary  if  the  learning  task  could 
promote  criss  crossing  by  the  learner  as  effectively. 

While  research  has  provided  some  additional  theoretical  justification  for  the  potential  benefits  of  cognitive 
flexibility  hypertexts,  it  is  necessary  to  extend  understanding  of  the  application  of  the  theory  through  research  into 
the  methods  through  which  cognitive  flexibility  is  applied  to  the  design  and  implementation  of 
hypertext/hypermedia  learning  materials.  The  research  study  conducted  sought  to  accomplish  this  goal  by 
examining  the  combination  of  cognitive  flexibility  hypertexts  with  two  general  types  of  learning  tasks,  one  which 
promoted  looking  at  cases  individually  and  another  which  was  believed  to  promote  cross-navigation  of  cases. 


Task  and  Learning  Outcomes 

The  relationship  between  the  nature  of  the  learning  task  and  learning  outcomes  has  been  shown  to  be 
significant  in  several  studies  of  text  processing.  Pichert  and  Anderson  (1977)  found  that  asking  learners  to  take  a 
certain  perspective  when  reading  a story  improved  the  memorization  and  retrieval  of  information.  Pichert  and 
Anderson  concluded  that  the  improvements  were  the  result  of  the  learner  placing  an  emphasis  on  certain  ideas 
presented  in  the  text  based  upon  the  perspective  they  were  taking.  Wittrock  and  Alesandri  ( 1 990)  studied  the  impact 
of  generating  summaries  and  analogies  and  found  that  learners  were  more  analytic  in  their  understanding  of  text 
passages  than  if  they  only  read  the  passages  with  no  specific  task.  The  body  of  research  on  text  processing  suggests 
task  plays  a primary  role  in  how  people  learn  when  reading. 

However,  the  analysis  of  some  of  the  findings  regarding  task  are  flawed  in  the  view  of  cognitive  flexibility 
theory.  While  text  processing  researchers  have  looked  at  the  task  as  impacting  information  processing  and/or 
schemata,  Spiro  et.  al.  (1987)  posit  that  in  ill-structured  domains  there  is  a need  for  flexible  schema  organization. 
Rather  than  merely  “matching  the  information  in  the  message  to  the  slots  in  the  schema”  (Pichert  & Anderson, 
1 977),  the  learner  needs  to  gain  an  understanding  of  the  situational  nature  of  the  information  in  order  to  reassemble 
their  knowledge  to  fit  a given  instance  within  the  domain.  Therefore,  task  is  important  not  because  it  cues  relevant 
schemata,  but  facilitates  construction  of  relationships  between  domain  knowledge  and  situations. 


Description  of  the  Study 

Specifically,  the  present  study  sought  to  identify  the  effects  of  two  different  types  of  task  on  the 
navigational  paths  of  learners  using  a cognitive  flexibility  hypertext  concerning  issues  of  sexual  harassment.  In 
addition,  the  effects  of  the  navigation  path  on  transfer  of  learning  were  also  examined.  Finally,  the  type  of  task, 
navigation  path,  and  transfer  test  scores  were  studied  for  significant  interactions.  In  this  manner  some  of  the 
primary  assumptions  of  cognitive  flexibility  theory  were  be  tested,  including  the  role  of  criss  crossing  navigation  as 
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a primary  component  for  improving  transfer,  and  the  constructivist  nature  of  the  theory  in  terms  of  using  a 
meaningful  task  to  promote  active  learning. 

The  hypotheses  of  the  study  were  that: 

1 . The  two  tasks  would  facilitate  two  different  types  of  navigational  paths,  either  a non-criss  crossing  path  the 
juror  task,  or  a criss  crossing  path  for  the  policy  task. 

2.  Learners  taking  a criss  crossing  navigation  path  through  the  hypertext  would  significantly  enhance  their  transfer 
of  learning  as  compared  to  learners  who  take  a non-criss  crossing  path. 

3.  Difference  in  transfer  would  be  the  result  of  the  policy  task  engendering  a more  criss-crossed  navigation  path 
through  the  hypertext  than  the  juror  task. 

In  order  to  test  the  hypotheses,  the  study  was  designed  as  an  experimental  study  with  subjects  grouped  by 
task.  The  participants  were  adult  graduate  management  students  enrolled  in  a course  on  business.  Learners  were 
randomly  assigned  to  one  of  two  task  types.  One  task  involved  the  learner  in  the  role  of  juror,  in  which  they  were 
instructed  to  look  at  each  individual  case  and  render  a decision  as  to  whether  it  involved  sexual  harassment.  The 
jury  verdict  task  was  expected  to  facilitate  a linear,  case-by-case  path  since  it  directed  the  learner  to  examine  each 
case  in  its  own  context.  The  second  task  asked  the  learner  to  devise  a policy  for  dealing  with  sexual  harassment 
within  a fictitious  company,  and  directed  the  student  to  develop  a complete  picture  of  the  issue  as  it  relates  to  the 
workplace.  This  was  expected  to  facilitate  a less  linear,  and  broader  investigation  of  the  case  information  by  the 
learner  due  to  the  comprehensive  nature  of  the  task. 


Attitude  Factor 

Attitude  regarding  sexual  harassment  was  measured  both  just  prior  to  the  treatments  and  again  prior  to  the 
transfer  test.  Beauvis  (1986)  considered  attitude  to  be  a combination  of  knowledge,  sensitivity,  and  awareness  of  the 
issue  of  sexual  harassment.  In  her  study  attitude  changes  were  scored  by  comparing  participants  answers  to  how 
they  matched  the  answers  that  would  be  expected  based  upon  the  desired  goals  and  outcomes  of  the  training 
program.  In  the  present  study,  a similar  method  was  used  for  scoring  attitudinal  differences  between  pre  and  post 
treatment  answers  on  an  attitude  survey,  comparing  participant  answers  with  those  of  an  expert  from  the  Affirmative 
Action  Office  of  a university  who  was  highly  knowledgeable,  sensitized  and  aware  regarding  the  issue  of  sexual 
harassment.  The  resulting  difference  was  used  for  statistical  analysis  of  the  impact  of  treatment  variables  on  attitudes 
towards  sexual  harassment. 


Measuring  Criss  Crossing 

Using  the  description  of  “criss  crossing”  as  entailing  “revisiting  the  same  material,  at  different  times,  in 
rearranged  contexts,  for  different  purposes,  and  from  different  conceptual  perspectives”  (Spiro  et.  al.,  1992),  it  was 
decided  that  criss  crossing  would  be  evident  from  any  navigational  choice  which  resulted  in  one  of  three  types  of 
criss  crossing: 

• Within  case  criss  crossing  - revisiting  the  same  information  on  a case  previously  viewed  immediately  after 
viewing  other  information  about  the  same  case. 

o Internal  criss  crossing  - revisiting  any  part  of  the  case  information  immediately  after  reading  case  specific 
theme,  perspective,  or  legal  commentary  related  to  the  same  case. 

• External  criss  crossing  - revisiting  case  information  or  case  specific  commentaries  immediately  after 
viewing  overall  theme  commentaries  or  information  in  other  cases. 

Using  the  three  types  of  criss  crossing  to  determine  how  many  of  the  participants’  navigational  choices  were 
characteristic  of  criss  crossing,  the  next  step  was  to  determine  what  methodology  to  use  in  order  to  measure 
navigation  style  for  further  statistical  analysis.  The  most  direct  and  appropriate  measure  was  considered  to  be  the 
total  number  of  criss  crossing  choices  made  by  the  participant.  This  represented  the  simplest  and  most  direct 
measure  of  the  basic  premise  of  cognitive  flexibility  theory  that  criss  crossing  impacts  transfer.  Statistically,  it 
allowed  the  entire  sample  to  be  maintained  as  a single  group  for  the  purposes  of  measuring  transfer.  Using  the  raw 
number  of  criss  crossing,  analysis  of  the  data  was  conducted  regarding  each  of  the  hypotheses  of  the  study, 
beginning  with  the  effect  of  task  on  criss  crossing. 
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Measuring  Transfer 

Transfer  was  measured  using  essay  questions  that  presented  new  cases  and  asked  the  learner  to  apply  what 
they  had  learned  to  make  a determination  regarding  the  case.  Three  independent  raters,  including  the  primary 
researcher,  rated  the  transfer  question  essays  according  to  a five  point  rubric.  The  final  transfer  score  for  each 
learner  was  the  combined  total  points  for  each  question,  with  a maximum  possible  score  of  1 0 points.  Interrater 
reliability  for  the  transfer  scoring  was  91.8  percent  between  raters  2 and  3,  91  percent  between  raters  1 and  3,  and  93 
percent  between  raters  1 and  2. 


Summary  of  Results 

The  results  regarding  the  primary  hypothesis  revealed  no  significant  impact  for  task  type.  Specifically,  the 
results  for  each  hypothesis  were: 

• An  ANOVA  revealed  no  significant  relationship  between  task  and  criss  crossing  (F=1.61,  P>.05,  N=34),  which 
means  the  hypothesis  that  the  type  of  task  would  affect  navigational  style  in  this  study  was  not  proven. 

• Analysis  showed  no  significant  effect  between  criss  crossing  and  transfer  (F=.083,  P>.05,  N=34).  Therefore,  the 
total  number  of  times  that  a student  criss  crossed  the  hypertext  had  no  statistically  significant  impact  on  the 
transfer  of  knowledge. 

• An  ANOVA  was  run  which  showed  task  had  no  significant  relationship  to  transfer  (F=.324,  P>.05,  N=34). 

However,  the  findings  regarding  attitude  did  show  a significant  effect  for  task  type.  A repeated  measure  statistical 
analysis  revealed  that  there  was  a significant  interaction  between  pre-treatment  to  post-treatment  attitude  survey 
scores  and  the  type  of  task  (F=4.86,  DF=1,33,  MS=2.431,  P<.05).  Inspection  of  the  data  revealed  that  the  policy 
task  group  (N=16)  showed  a mean  positive  change  of  1.25  points  on  the  attitude  survey  from  pre-treatment  to  post- 
treatment, while  the  juror  task  group  (N=17)  had  a mean  negative  change  of -0.41 . 


Discussion  of  Results 

Task  and  Criss  Crossing 

It  is  possible  that  the  lack  of  differences  can  be  attributed  to  the  policy  and  juror  tasks  being  equal  in  how 
they  were  perceived  by  the  learner.  It  seems  clear  from  the  navigation  data  that  a learner  was  just  as  likely  to  criss 
cross  if  they  were  engaged  in  the  juror  task  as  they  were  in  the  policy  task.  In  fact  most  of  the  learners  in  the  study 
chose  a relatively  low  amount  of  criss  crossing  (average  of  only  ten  criss  crosses  out  of  fifty  total  navigational 
choices)  regardless  of  task  type. 

A more  theoretical  explanation  of  the  results  is  that  learners  have  a predilection  towards  certain  kinds  of 
navigation  patterns.  The  findings  of  Lawless  and  Kulikowich  (1996)  support  this  interpretation.  In  their  study, 
hypertext  learners  appeared  to  fall  into  three  types;  those  who  explored  the  navigational  options  of  the  hypertext, 
those  who  were  concerned  primarily  with  seeking  out  particular  information,  and  apathetic  users  who  simply  did  not 
spend  time  in  the  hypertext.  Beasley  and  Waugh  (1996)  also  reported  that  learners  seemed  to  approach  hypertext 
systems  with  a general  navigation  pattern  in  mind,  one  that  emphasized  coverage  of  material  rather  than  underlying 
structures  or  important  information.  In  the  present  study,  it  is  possible  that,  regardless  of  the  nature  of  the  task  and 
the  navigation  options  of  the  hypertext,  learners  used  their  preferred  style  of  navigation  to  study  the  hypertext 
information  and  complete  the  task  instead  of  adapting  their  navigation  to  the  task  and/or  the  prevalent  hypertext 
structures. 


Task  and  Transfer 

As  was  the  case  for  task  and  criss  crossing  navigation,  it  is  likely  that  the  tasks  were  not  different  enough  in 
nature  or  importance  to  engender  strong  changes  in  learner  reactions  or  understandings.  This  explanation  is  in 
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keeping  with  previous  research  regarding  task  and  perspective.  Pichert  and  Anderson  (1977)  used  two  very  different 
perspectives  (home  owner  vs.  burglar)  and  found  that  learners  attended  to  different  information  when  reading.  In 
the  present  study,  it  may  be  that  learners  simply  took  a very  similar  perspective,  perhaps  viewing  the  cases  from  a 
legal  viewpoint,  in  both  tasks. 

Other  hypertext  studies  involving  task  differences  have  also  had  difficulty  showing  a significant  link 
between  task  impacts  on  transfer  via  navigation  variables.  Beasley  and  Waugh  (1996)  found  no  differences  in 
learner  scores  based  upon  the  number  of  times  they  revisited  information  when  given  the  goal  of  reviewing  material. 
Given  equal  goals,  but  different  amounts  of  return  visits  in  a hypertext,  it  appears  that  learners  scored  relatively 
equally  regardless  of  navigation.  Jonassen  and  Wang  (1993)  found  that  without  tasks  that  focused  learners  on 
different  aspects  related  to  the  structure  of  the  knowledge  being  studied,  no  significant  differences  could  be  found 


Task  and  Attitude 

An  apparently  strong  relationship  existed  between  the  pre-treatment  and  post-treatment  scores  on  the 
survey  of  attitudes  towards  sexual  harassment.  Students  who  engaged  in  the  jury  task  actually  appeared  to  become 
somewhat  less  sensitive  to  the  issue,  while  policy  makers  gained  a greater  understanding  of  sexual  harassment.  One 
possible  rationale  for  this  difference  is  that  the  natures  of  the  tasks,  while  not  different  in  terms  of  learning  transfer, 
were  different  in  terms  of  affective  outcomes. 

Beauvis  (1986)  notes  that  sexual  harassment  training  is  designed  to  change  attitudes  by  challenging 
trainees’  opinions  on  the  issue.  The  goal  is  to  make  participants  more  aware  and  more  sensitive,  which  means  they 
may  need  to  be  exposed  to  information  contradictory  to  their  beliefs.  Some  research  regarding  the  exposure  of 
learners  to  information  inconsistent  with  their  prior  knowledge  suggests  that  the  accommodating  of  the  contradictory 
information  is  a necessary  step  in  learning  through  attitude  change. 

Chinn  and  Brewer  (1993),  based  upon  a study  of  adults  presented  with  contradictory  information  regarding 
physics  concepts,  theorized  that  learners  take  four  different  approaches  to  dealing  with  contradictory  information. 
One  was  to  simply  reject  the  new  information,  while  another  was  to  reinterpret  it  to  make  it  fit  with  their  present 
beliefs.  Both  of  these  meant  that  the  person  was  unwilling  to  move  from  their  beliefs  as  a result  of  the  exposure  to 
the  new  knowledge.  The  final  two  methods  were  partially  changing  their  beliefs  so  as  to  allow  for  the  new 
information  without  totally  giving  up  old  beliefs,  and  finally  to  restructure  their  beliefs  around  the  new  information. 

In  explaining  the  results  of  the  present  study,  it  seems  reasonable  that  perhaps  the  two  tasks  were  different 
in  how  participants  encountered  contradictory  viewpoints  on  sexual  harassment.  Learners  asked  to  provide  a verdict 
in  each  case  were  perhaps  more  likely  to  hold  on  to  their  beliefs  in  the  face  of  new  information,  since  they  were 
stating  and  defending  their  personal  opinions  in  light  of  legal  standards.  This  could  be  construed  as  meaning  they 
were  forced  to  either  accept  or  reject  new  information  on  the  basis  of  their  personal  understandings  regarding  sexual 
harassment  law. 

On  the  other  hand,  the  learners  developing  policy  may  have  been  taking  a more  open  perspective  on  the 
information  they  encountered,  since  developing  a comprehensive  policy  meant  taking  into  account  as  many  different 
possibilities  and  perspectives  as  possible.  In  order  to  do  this,  these  learners  may  have  been  acting  less  on  personal 
belief  than  those  in  the  juror  group,  and  therefore  they  were  more  willing  to  accommodate  information  that 
contradicted  their  beliefs. 

Considering  that  ill-structured  domains  tend  to  involve  a great  degree  of  variability,  with  no  true  “right”  or 
“wrong”  answers,  then  the  findings  suggest  a possible  new  direction  for  studying  cognitive  flexibility  theory. 
Findings  of  previous  CFH  research  examined  beliefs  regarding  the  nature  of  knowledge  (Jacobson  & Spiro,  1995; 
Jacobson  et.  al.,  1996),  but  not  specifically  the  nature  of  the  domain  being  studied.  For  ill -structured  domains  in 
which  the  learning  outcomes  may  be  affective,  it  may  be  that  the  nature  of  the  task  could  play  a significant  role  in 
learning  by  virtue  of  how  the  learner  approaches  information  contradictory  to  their  beliefs. 


Conclusion 

Cognitive  flexibility  theory  has  been  widely  touted  as  a strong  basis  for  the  design  of  hypertext  learning 
environments.  However,  to  date  little  research  has  been  conducted  on  this  claim.  One  contribution  of  the  study  is  to 
begin  to  examine  the  importance  of  the  type  and  nature  of  the  learning  tasks  associated  with  CFH  implementation. 
It  appears  that  the  amount  of  criss  crossing  that  occurs  is  not  by  itself  an  important  variable  in  achieving  the  learning 
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outcome  of  transfer  that  the  theory  supports.  In  this  study  it  has  been  suggested  that  perhaps  the  task  must  be 
meaningful  to  the  learner,  such  that  they  are  intentional  in  their  navigational  choices.  This  implication  may  extend 
to  other  theory-based  hypertext  designs,  in  that  if  learning  is  intentional  then  the  task  must  fit  the  goals  and  contexts 
of  the  learner  as  well  as  the  theoretical  basis  for  the  hypertext  structure. 

Perhaps  the  most  significant  finding  of  the  study  was  that  task  might  be  very  important  when  considering 
the  types  of  ill-structured  domains  for  which  a CFH  is  developed.  When  dealing  with  domains  which  involve  strong 
emotional  opinions,  and  are  perhaps  controversial  for  learners,  then  the  task  may  need  to  be  devised  such  that 
learners  remain  open  to  the  complexity  and  ill-structured  nature  of  the  knowledge  being  represented 
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Abstract 

This  paper  reports  on  a course-support  system  for  English  learners  of  Greek  which  is  built 
around  an  Intelligent  Language  Tutoring  System  (ILTS).  Meaningful  interaction  between  the 
learner  and  the  system  is  achieved  by  a number  of  means:  First,  students  provide  natural 
language  input  rather  than  selecting  exclusively  from  among  pre-deflned  answers.  Second,  the 
ILTS  uses  Natural  Language  Processing  to  analyze  the  learner’s  input  and  to  provide  error- 
specific  feedback.  Third,  the  system  contains  a Student  Model  which  keeps  a record  of 
students’  performance  histories.  The  information  about  each  learner  in  the  Student  Model 
determines  the  level  of  specificity  of  feedback  messages,  clues,  and  exercise  difficulty.  In 
addition  to  vocabulary  and  grammar  exercises,  the  system  also  contains  oral  dialogues  with 
translations,  a glossary,  and  cultural  information.  The  system  is  designed  for  introductory 
learners  of  Greek  and  is  implemented  on  the  World  Wide  Web. 


1.  Introduction 

As  the  Web  begins  to  mature  and  technologies  for  adapting  existing  content  in  language  learning  converge  with 
increased  bandwidth,  the  desire  grows  to  develop  uniquely  Web-based  applications.  Convenient  access, 
currency  and  variety  of  material,  and  integrated  multi-media  can  extend  traditional  language  instruction,  but 
truly  new  forms  require  a more  profound  degree  of  interactivity  than  provided  by  conventional  forms. 

Interactivity  has  become  a key  term  in  Hyper/Multimedia  and  although  it  is  used  extensively,  its  definition  and 
scope  is  still  not  precisely  determined.  However,  Laurel  [1991]  provides  a useful  definition  by  making  a 
distinction  between  frequency,  range,  significance,  and  participatoriness.  With  respect  to  frequency  and  range, 
we  may  count  the  number  of  times  a user  can  interact  with  a system  and  how  many  choices  are  available, 
respectively.  Significance  and  participatoriness  both  require  a qualitative  analysis  to  determine  how  the  user’s 
choices  affected  matters  and  to  which  degree  the  user  is  participating  in  ongoing  system  decisions  [Ashworth 
1996].  But  to  achieve  a high  degree  of  interactivity  with  respect  to  significance  and  participatoriness, 
software  requires  intelligence  to  consider  user’s  individual  differences.  For  example,  students  learn  at  a 
different  pace  and  have  different  sets  of  skills.  These  student  variables  need  to  be  considered  for  a system  to  be 
highly  interactive. 

The  course-support  material  described  in  this  paper  differs  from  others  in  that  it  is  built  around  an  interactive 
Intelligent  Language  Tutoring  System  (ILTS).  Interactivity  is  achieved  in  a number  of  ways.  The  system 
provides  a wide  range  of  exercises  covering  vocabulary  and  grammar  practice.  For  some  tasks,  students  provide 
natural  language  input  rather  than  selecting  exclusively  from  among  pre-defined  answers.  The  tasks  are 
independent  of  each  other  and  the  exercise  types  range  from  drill-and-practice  to  more  game-like  tasks.  In 
addition,  the  system  contains  oral  dialogues,  a glossary,  and  cultural  information.  Significance  and 
participatoriness  is  achieved  through  Natural  Language  Processing  (NLP)  and  Student  Modeling.  The  system 
consists  of  a grammar  and  a parser  which  analyzes  student  input  and  provides  error-specific  feedback.  The 
system  also  maintains  a Student  Model  which  keeps  a record  of  students’  ongoing  performance,  alters  system 
decisions  accordingly,  and  provides  learner  model  updates. 

In  the  following,  we  will  first  discuss  the  goals  of  the  system  and  address  the  grammar,  parser,  and  Student 
Model  components.  We  will  then  describe  the  modules  involved  in  analyzing  student  input.  Further  we  will 
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illustrate  the  exercise  types  of  the  system  and  provide  examples  of  learner  modeling  for  each  of  them.  Finally, 
we  will  conclude  with  suggestions  for  further  implementations. 

2.  Goals  and  System  Requirements 

The  goal  of  the  ILTS  for  Greek  is  to  provide  meaningful  and  interactive  vocabulary/grammar  practice  for 
English  learners  of  Greek.  Meaningful  tasks  and  interactivity  require  intelligence  on  the  part  of  the  computer 
program.  Unlike  existing  course-support  systems  which  use  simpler  grammar  practice  and  feedback 
mechanisms,  the  ILTS  for  Greek  emulates  two  significant  aspects  of  a student  teacher  interaction:  it  provides 
error-specific  feedback  and  allows  for  individualization  of  the  learning  process  [see  also  Heift,  1998].  In 
example  [1]  the  student  provided  an  incorrect  Greek  sentence: 

( 1 ) O^oq  o Koa^oq  ^aGaiveiq  EA,A,r|viKa. 

OAx)q  o Koa^oq  ^aGaivei  EX,A,r|viKa. 

The  whole  world  is  learning  Greek. 

In  such  an  instance,  the  system  will  detect  an  error  in  subject-verb  agreement  and,  in  addition,  will  tailor  its 
feedback  to  suit  the  learner’s  expertise.  Tailoring  feedback  messages  according  to  student  level  follows  the 
pedagogical  principle  of  guided  discovery  learning.  According  to  Elsom-Cook  [1988],  guided  discovery  takes 
the  student  along  a continuum  from  heavily  structured,  tutor-directed  learning  to  a point  where  the  tutor  plays 
less  of  a role.  Applied  to  feedback,  the  pedagogy  scales  messages  on  a continuum  from  least-to-most  specific 
guiding  the  student  towards  the  correct  answer. 

There  are  three  learner  levels  considered  in  the  system:  novice,  intermediate,  and  advanced.  For  example  (1), 
the  novice  will  receive  the  most  detailed  feedback:  “You  have  made  a subject-verb  agreement  error.”,  while  the 
intermediate  learner  will  be  informed  that  an  error  in  agreement  occurred.  In  contrast,  the  advanced  learner  will 
merely  be  told  of  an  error  in  the  sentence.  The  central  idea  is  that  the  better  the  language  skills  of  the  learner, 
the  less  feedback  is  needed  to  guide  the  student  towards  the  correct  answer.  This  analysis,  however,  requires: 

a)  an  NLP  component  which  can  analyze  ill-formed  sentences,  and 

b)  a Student  Model  which  keeps  a record  of  the  learner. 

The  NLP  component  consists  of  a grammar  and  a parser.  The  system  is  written  in  LOPE,  an  ALE  style 
extension  of  Prolog  [see  Carpenter  and  Penn,  1994].  LOPE  is  a phrase  structure  parsing  and  definite  clause 
programming  system  in  which  the  terms  are  typed  feature  structures.  Definite  Clause  Grammars  (DCGs),  like 
other  unification-based  grammars,  place  an  important  restriction  on  parsing,  that  is,  if  two  or  more  features  do 
not  agree  in  their  values,  a parse  will  fail.  However,  in  a language  learning  system,  these  are  the  kinds  of 
mistakes  made  by  learners.  To  parse  ungrammatical  sentences,  the  Greek  grammar  contains  rules  which  are 
capable  of  parsing  ill-formed  input  (buggy  rules)  and  which  apply  if  the  grammatical  rules  fail  (see  also 
[Schneider  & McCoy,  1998],  [Liou,  1991],  [Weischedel,  1983],  [Carbonell  & Hayes,  1981]).  The  system  keeps 
a record  of  which  grammatical  violations  have  occurred  and  which  rules  have  been  used  but  not  violated.  This 
information  is  fed  to  the  Student  Model. 

The  Student  Model  is  a representation  of  the  current  skill  level  of  the  student.  For  each  student  the  Student 
Model  keeps  score  across  a number  of  error  types,  or  nodes,  for  example,  grammar  or  vocabulary.  For  instance, 
the  grammar  nodes  contain  detailed  information  on  the  student’s  performance  on  subject-verb  agreement,  case 
assignment,  clitic  pronouns,  etc.  The  score  for  each  node  increases  and  decreases  depending  on  the  grammar’s 
analysis  of  the  student’s  performance.  The  amount  by  which  the  score  of  each  node  is  adjusted  is  specified  in  a 
master  file  and  may  be  weighted  to  reflect  different  pedagogical  purposes. 

The  Student  Model  has  two  main  functions: 

First,  the  current  state  of  the  Student  Model  determines  the  specificity  of  the  feedback  message  displayed  to  the 
student.  A feedback  message  is  selected  according  to  the  current  score  at  a particular  node.  A student  might  be 
advanced  with  regard  to  vocabulary  but  a beginner  with  passive  voice  constructions.  Hence,  a feedback 
message  about  vocabulary  would  be  less  detailed  than  a feedback  message  about  passive-voice  constructions. 

Second,  the  difficulty  of  the  exercises  presented  to  the  student  is  modulated  depending  on  the  current  state  of 
the  Student  Model.  For  example,  if  a student  is  rated  as  advanced  with  respect  to  vocabulary,  then  some  of  the 
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vocabulary  exercises  are  made  more  challenging.  The  same  thing  can  be  found  with  some  of  the  grammar 
exercises. 

In  the  following,  we  will  discuss  the  error-checking  mechanism  performed  by  the  system  in  analyzing  student 
input. 

3.  Error-Checking  Mechanism 

In  addition  to  the  grammar  and  the  parser,  the  system  contains  additional  error-checking  modules  which  get 
evoked  when  processing  a student  answer.  For  example,  consider  the  task  in  (2a)  where  the  student  was  asked 
to  make  a sentence  with  the  words  provided. 

(2a)  lOTiaviKa  |iiA.aco  Niicri 
(2b)  Niicri  |xiA.oc£i  lOJiaviKa 
niki  - speak  - Spanish 

The  student  answer  given  in  (2b)  contains  a number  of  mistakes:  an  error  in  subject-verb  agreement,  a spelling 
mistake,  and  a missing  article.  While  the  grammar  will  detect  the  agreement  error,  the  remaining  mistakes  will 
be  discovered  by  additional  modules  of  the  system.  Figure  [1]  illustrates  the  modules  of  the  Natural  Language 
Process  d^gtem. 

Student  input 


Spell  Check 


Answer  Check 


Extra  Word  Check 


Word  Order  Check 


Grammar  Check 


Match  Check 


Feedback  Message  to  the  Learner 
Figure  1 : Error-checking  Modules 

The  first  module  in  analyzing  student  input  is  a spell  check.  During  the  spell  check,  the  system  also  extracts  the 
base  forms  of  each  word  from  the  grammar.  The  uninflected  words  are  needed  to  determine  whether  the  learner 
followed  the  task,  that  is,  whether  the  student  answer  contained  the  words  which  were  provided.  Such  errors 
cannot  be  determined  by  the  grammar  and  the  parser  because  the  parser  can  only  judge  whether  a sentence  is 
correct  or  not. 

In  defining  an  exercise,  we  store  possible  answers  of  a given  task  and  the  Answer  Check  module  determines  the 
most  likely  answer  (MLA)  the  student  intended.  The  Answer  Check  module  further  matches  the  extracted  base 
forms  with  the  MLA.  If  any  of  the  words  in  the  task  are  not  contained  in  the  student  answer,  the  system  will 
report  an  error. 


435 


BEST  COPY  AVAILABLE 


The  following  two  checks,  Extra  Word  Check  and  Word  Order  Check,  refer  to  additional  words  in  the  student 
answer  and  errors  in  word  order,  respectively.  While  these  two  checks  are  commonly  handled  by  the  grammar 
[Schwind,  1995],  preliminary  testing  of  our  system  showed  that  the  system  performs  faster  if  these  two  error 
types  are  treated  outside  the  grammar.  Naturally,  the  speed  is  also  influenced  by  the  grammar  formalism  used. 

The  Grammar  Check  is  the  most  elaborate  of  the  modules.  Here  the  sentence  is  analyzed  by  the  parser 
according  to  the  rules  and  lexical  entries  provided  in  the  Greek  grammar.  Currently,  the  grammar  covers  a wide 
range  of  grammatical  concepts,  from  early  beginner  constructions  (verb  ‘to  be’  and  the  concept  of  null  subjects) 
to  fairly  advanced  structures  (passive  voice).  Development  is  still  ongoing  to  achieve  a complete  coverage  of 
Greek  grammar. 

The  Match  Check  looks  for  correct  punctuation  and  capitalization  by  string-matching  the  student  answer  with 
the  MLA.  By  this  time  in  the  evaluation  process,  it  is  very  unlikely  that  the  sentence  still  contains  any  errors 
other  than  punctuation  or  capitalization.  If  the  sentence  passes  the  Match  Check  successfully,  the  sentence  is 
correct.  If  not,  an  error  is  reported  to  the  student. 

The  system  is  organized  in  a way  that  if  a module  detects  an  error,  further  processing  is  blocked.  As  a result, 
only  one  error  at  a time  will  be  displayed  to  the  learner.  This  was  implemented  mainly  to  avoid  overloading  the 
student  with  extensive  error  reports  in  the  case  of  multiple  errors.  According  to  van  der  Linden  [1993], 
displaying  more  than  one  feedback  message  at  a time  makes  the  correction  process  too  complex  for  the  student. 
After  correcting  the  error,  the  student  restarts  the  checking  mechanism  by  clicking  the  CHECK  button. 

In  the  last  section,  we  will  briefly  describe  the  six  exercise  types  implemented  in  the  system  and  discuss  the  role 
of  the  Student  Model  for  each. 

t 4.  Exercise  Types 

The  system  consists  of  three  vocabulary  and  three  grammar  exercise  types:  Guess  the  Word,  Find  the  Word, 
Which  Word  is  Different,  Word  Order  Practice,  Fill-in-the-Blank,  and  Build  a Sentence,  respectively.  Student 
performance  in  each  exercise  contributes  to  the  Student  Model.  However,  the  nodes  that  get  updated  for  each 
exercise  differ.  In  addition,  there  are  two  ways  in  which  this  information  is  used  by  the  system: 

1 ) feedback  messages  tailored  to  student  expertise 

2)  difficulty  of  exercise  presented  to  student 

In  the  following,  we  will  discuss  the  nodes  which  get  updated  and  illustrate  the  feedback  message  and  exercise 
difficulty  modulation  for  each  exercise  type. 

Vocabulary 

Guess  the  Word 

This  is  a slight  variation  of  the  vocabulary  game  ‘Hangman’  where  students  guess  a word  by  entering  one  letter 
at  a time.  To  avoid  making  this  a purely  guessing  game,  a clue  is  provided  for  each  word.  This  clue  may  be  the 
English  equivalent  of  the  answer,  or  a picture  or  sound  representing  the  answer.  For  each  correct  word,  there  is 
an  increment  for  the  vocabulary  node,  for  each  word  missed,  the  system  registers  a decrement.  In  this  exercise 
type,  the  feedback  messages  are  the  same  for  all  users.  However,  advanced  students,  that  is,  students  with  a high 
vocabulary  score  get  fewer  tries  to  complete  the  task. 

Find  the  Word 

In  this  exercise  type,  students  need  to  find  several  words  in  a grid.  The  system  increments  the  vocabulary  node 
for  each  word  found  and  decrements  for  each  incorrect  guess.  The  maximum  increments  and  decrements  per 
game  are  equal  to  the  number  of  words  in  the  table.  The  feedback  messages  are  the  same  for  all  students.  The 
exercise  difficulty  is  modulated:  advanced  students  receive  clues  in  English,  while  other  students  receive  clues 
in  Greek. 
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Which  Word  is  Different 

This  exercise  displays  a number  of  words  all  except  one  of  which  belong  to  the  same  category.  The  student  task 
is  to  identify  the  one  which  differs  from  the  others.  The  divergent  word  may  differ  syntactically,  semantically  or 
pragmatically  from  the  remaining  words.  There  is  an  increment  for  each  correct  word  selected,  for  each 
incorrect  selection  a decrement  is  recorded.  The  feedback  messages  are  the  same  for  all  students  and  there  is  no 
difficulty  modulation. 

Grammar 

Word  Order  Practice 

In  this  exercise,  students  practice  Greek  word  order  with  a ‘drag  and  drop’  task:  words  have  to  be  arranged  in  an 
appropriate  order  to  form  a grammatical  Greek  sentence.  For  this  exercise  type,  the  word  order  node  in  the 
Student  Model  is  updated.  For  a sentence  with  incorrect  word  order,  the  system  records  a decrement,  else  an 
increment.  There  is  no  feedback  or  difficulty  modulation  for  this  exercise. 

Fill-in-the-Blank 

The  student’s  task  here  is  to  complete  sentences  by  filling  in  any  blanks  that  appear  in  the  example.  The  Student 
Model  records  each  grammar  node  that  is  detected  in  the  student  input.  For  example,  if  students  are  asked  to 
supply  the  correct  conjugation  of  the  verb,  then  the  system  records  an  increment/decrement  for  subject-verb 
agreement.  The  feedback  messages  are  modulated  according  to  the  level  of  the  learner.  Also,  advanced  learners 
obtain  more  difficult  tasks.  For  instance,  novice  and  intermediate  students  will  find  only  one  blank  per  example. 
Advanced  students  may  be  presented  with  examples  that  contain  more  than  one  blank. 

Build  a Sentence 

In  this  exercise  type,  students  are  provided  with  a set  of  words.  Their  task  is  create  a grammatical  Greek 
sentence  using  all  the  provided  words.  This  was  illustrated  earlier  in  example  (2).  All  grammar  nodes  activated 
during  the  processing  of  the  student’s  input  are  updated  in  the  Student  Model.  An  increment  is  recorded  if  this 
aspect  of  the  student’s  answer  was  correct.  A decrement  is  recorded  for  any  grammatical  errors  made  by  the 
student.  Feedback  messages  reflect  the  current  state  of  the  learner’s  expertise,  as  represented  by  the  Student 
Model.  There  is  no  difficulty  modulation. 

It  becomes  apparent  that  the  modeling  process  has  to  be  decided  for  each  exercise  type  independently.  Not  only 
does  it  depend  on  the  task  but  also  the  possibilities  the  exercise  offers  in  adjusting  the  level  of  difficulty. 

5.  Future  Work 

We  plan  to  extend  the  range  of  exercises  available  to  students.  Since  students  are  particularly  appreciative  of  the 
opportunity  to  practice  grammar  in  such  a constructive  environment,  we  will  focus  on  developing  further 
exercises  that  exploit  the  Natural  Language  Processing  and  Student  Model  functionality  that  we  have 
developed.  Two  exercises  under  construction  are  Dictation  and  Translation. 

In  the  Dictation  exercise,  students  listen  to  an  audio  file  and  then  type  the  text  they  have  heard.  The  student’s 
input  is  analyzed  by  the  grammar  and  feedback  is  provided  for  any  grammatical  errors.  Feedback  is  modulated 
according  to  the  current  state  of  the  Student  Model.  In  the  translation  exercise,  students  are  provided  with  an 
English  phrase  or  sentence  and  are  required  to  translate  this  into  grammatical  Greek.  Again,  feedback  is 
modulated  according  to  the  current  state  of  the  Student  Model. 

6.  Conclusions 

In  this  paper,  we  discussed  an  interactive  course-support  system  on  the  WWW.  Interactivity  is  achieved  by  a 
wide  range  of  exercise  types  which  are  arranged  in  an  independent  order.  NLP  and  Student  Modeling  also 
contribute  to  the  interactivity  of  the  whole  system  by  providing  error-specific  feedback  and  adjusting  to 
different  learners.  The  Student  Model  is  based  on  the  performance  history  of  the  student.  The  information  about 
each  learner  is  recorded  in  the  Student  Model  and  determines  the  level  of  specificity  of  feedback  messages  and 
exercise  difficulty. 
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Preliminary  testing  of  the  content  of  two  modules  and  the  overall  functionality  of  the  system  has  taken  place. 
While  the  performance  of  the  system  is  accurate  with  respect  to  feedback  and  student  modeling,  student 
responses  are  currently  being  analyzed  and  results  will  be  reported  during  the  presentation.  In  the  meantime,  the 
system  is  being  further  developed  by  adding  more  content  and  implementing  further  grammatical  constructions. 
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Abstract:  The  current  trend  to  use  the  World  Wide  Web  as  a vehicle  for  the  delivery  of 
distance  education  units  has  required  educators  to  adopt  innovative  approaches  to  the  design 
of  learning  materials  for  this  medium.  However,  the  temptation  to  place  large  tracts  of  text 
online  is  deeply  entrenched.  This  paper  describes  a project  where  guidelines  for  the  design  of 
authentic  activities  have  been  derived  from  a broad  literature  base.  The  guidelines  have  been 
used  in  the  development  of  Masters  subjects,  where  authors  are  encouraged  to  use  learning 
activities  as  the  central  focus  of  study — the  activity  does  not  supplement  the  unit,  it  is  the  unit. 
The  emphasis  is  placed  on  the  design  of  activities,  integrated  assessment  of  the  activities, 
identification  of  resources  and  establishment  of  collaborative  tools.  Examples  of  activities, 
together  with  the  instructional  design  guidelines,  are  provided,  and  the  use  of  theory  and 
research  to  inform  the  instructional  design  process  is  discussed. 


Authentic  activities 

Activities,  investigations  and  problems  are  at  the  heart  of  student  involvement  in  meaningful  learning  contexts. 
Teachers  provide  such  activities  to  enable  students  to  interact  with  the  learning  environment,  and  to  learn, 
apply  and  practice  newly  acquired  skills.  Activities  are  defined  by  Brophy  and  Alleman  (1991)  as:  ‘Anything 
students  are  expected  to  do,  beyond  getting  input  through  reading  or  listening,  in  order  to  learn,  practice,  apply, 
evaluate,  or  in  any  other  way  respond  to  curricular  content’  (p.  9). 

This  paper  describes  the  development  of  a university-based  online  project  where  activities  are  used,  not  only  as 
opportunities  for  students  to  learn,  practice,  apply  and  evaluate,  but  also  as  central  organising  devices  for  the 
design  of  entire  online  units  of  study.  Clayden,  Desforges,  Mills  and  Rawson  (1994)  point  out  that  the  kind  of 
activities  frequently  used  in  education  simply  lead  to  an  enculturation  into  the  practices  of  classrooms  rather 
than  the  real-world  transfer  teachers  expect.  They  note  that  students’  efforts  to  make  sense  of  classroom 
experiences  generally  lead  them  to  focus  on  working  practices  rather  than  abstract  ideas.  ‘What  they  learn  ...  is 
how  to  do  work,  how  to  be  neat,  how  to  finish  on  time  ...  and  how  to  tidy  away’  (p.  164).  While  these 
comments  are  most  appropriate  for  school  (and  to  some  extent  university)  classrooms,  the  same  conclusions 
may  be  drawn  for  the  design  of  online  learning  environments.  Students  frequently  learn  to  invoke  ‘sub-optimal’ 
schemes  to  enable  them  to  proceed,  rather  than  deal  with  the  content  in  a way  that  promotes  true 
understanding.  Many  of  these  online  programs  are  so  ‘well  designed’,  they  fail  to  account  for  the  nature  of  real- 
world  problem  solving,  where  the  solution  is  rarely  neat  and  the  salient  facts  are  rarely  the  only  ones  at 
students’  disposal.  In  contrast,  a number  of  authors  suggest  that  authentic  activities  should  be  ill-defined — 
students  find  as  well  as  solve  the  problems.  Learners  need  to  have  the  opportunity  to:  explore  a situation  with 
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all  the  complexity  and  uncertainty  of  the  real  world,  have  a role  in  determining  the  task  and  how  it  might  be 
broken  up  into  smaller  tasks,  select  relevant  information,  and  find  solutions  that  suit  their  needs. 

The  Cognition  and  Technology  Group  at  Vanderbilt  (1990b)  stress  the  importance  of  complexity  and  the 
necessity  to  provide  an  environment  capable  of  sustained  examination.  They  describe  authentic  tasks  as 
‘generative’  because  the  completion  of  the  task  requires  the  students  to  generate  other  problems  to  be  solved. 
They  draw  a distinction  between  these  authentic  tasks  and  simple  word  problems  that  already  define  the 
problem,  such  as:  Tf  you  travel  150  kilometres  at  90  kph,  how  long  will  the  journey  take?’  By  comparison, 
Reeves  and  Laffey  (1999)  describe  a complex  learning  environment  where,  for  an  entire  semester  unit,  the 
students’  task  is  to  establish  a research  station  on  Mars,  including  designing  an  energy  plant  to  sustain  life  once 
a station  is  established.  Similarly,  Pennell,  Durham,  Orzog  and  Spark  (1997)  describe  a web-based 
environment  where  students  learn  business  communication  skills  by  accepting  temporary  employment  in  a 
virtual  recording  company.  They  are  given  a complex  task  to  complete,  and  in  order  to  do  it,  they  make 
appointments  and  keep  a diary,  ‘interview’  the  director  and  other  employees,  and  write  letters,  memos  and 
reports. 

Several  authors  have  attempted  to  delineate  characteristics  of  authentic  activities.  For  example,  Jonassen  (1991) 
defines  authentic  activities  as  tasks:  that  have  real-world  relevance  and  utility,  that  integrate  those  tasks  across 
the  curriculum,  that  provide  appropriate  levels  of  complexity  and  that  allow  students  to  select  appropriate  levels 
of  difficulty  or  involvement  (p.  29).  Similarly,  Bransford,  Vye,  Kinzer  and  Risko  (1990)  Young  (1993)  Lebow 
and  Wager  (1994)  and  Savery  and  Duffy  (1996)  among  others  have  nominated  criteria  of  authentic  activities. 

Recommended  design  features  of  authentic  activities 

It  is  possible  to  use  these  findings  of  the  research  and  writing  on  authentic  activities  to  produce  guidelines  for 
the  design  of  learning  environments.  Accordingly,  authentic  activities: 

• have  real-world  relevance  (e.g..  Brown,  Collins,  & Duguid,  1989a;  Cognition  and  Technology  Group  at 
Vanderbilt  [CTGV],  1990a;  Jonassen,  1991;  Resnick,  1987;  Oliver  & Omari,  1999) 

• are  ill-defined  (e.g.,  Brown  et  al.,  1989a;  CTGV,  1990a;  Young,  1993) 

• comprise  a single  complex  task  to  be  investigated  by  students  over  a sustained  period  of  time  (e.g., 

Bransford,  Vye  et  al.,  1990;  CTGV,  1990b;  Jonassen,  1991;  Savery  & Duffy,  1996) 

• require  students  to  define  the  tasks  and  sub-tasks  required  to  complete  the  activity  (e.g.,  Bransford,  Vye 
et  al.,  1990;  CTGV,  1990b;  Collins,  Brown,  & Newman,  1989;  Young,  1993) 

• provide  the  opportunity  to  examine  the  task  from  different  perspectives,  (e.g.,  CTGV,  1990a;  Young, 
1993;  Spiro,  Feltovich,  Jacobson,  & Coulson,  1991;  Lebow  & Wager,  1994;  Savery  & Dutfy,  1996) 

• enable  students  to  detect  relevant  from  irrelevant  information,  (e.g.,  CTGV,  1990;  Savery  & DutJy, 
1996) 

• provide  the  opportunity  to  collaborate  (e.g..  Young,  1993;  Lebow  & Wager,  1994) 

• can  be  integrated  across  subject  areas  (e.g.,  Bransford,  Sherwood,  Hasselbring,  Kinzer,  & Williams, 

1990;  Bransford,  Vye  et  al.,  1990;  Jonassen,  1991). 

• are  seamlessly  integrated  with  the  assessment  (e.g..  Reeves  & Okey,  1996;  Herrington  & Herrington, 
1998). 

These  principals  have  been  incorporated  into  the  design  of  activities  within  online  units  currently  being 
designed  for  a suite  of  Masters  level  courses. 

The  Masters  Online  project 

Fifteen  professionally  oriented  Masters  courses  are  presently  being  developed,  in  areas  such  as  health,  teaching, 
business,  finance,  computing  and  information  science.  Students  in  these  courses  are  mostly  experienced  in  the 
workplace  and  authentic  learning  activities  are  likely  to  be  highly  valued  by  these  students.  Many  of  the 
students  live  geographically  distant  from  the  university,  whilst  others  have  such  busy  lifestyles  that  on-campus 
study  is  impractical.  The  university  has  an  established  tradition  of  providing  external  courses  for  these  students. 
The  use  of  computer  technologies  to  offer  enhanced  learning  environments  to  external  students  is  a natural  next 
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step.  However,  it  is  a significant  challenge  to  provide  the  required  staff  development  to  support  such  effective 
change. 

In  total,  the  15  Masters  courses  offer  over  250  subjects  or  units  and  involve  over  150  academic  staff,  some  of 
whom  have  limited  experience  of  working  in  the  online  environment.  The  project  has  an  initial  development 
phase  of  three  years.  During  the  first  three  years,  all  units  will  be  adapted  for  use  online,  but  at  differential 
levels: 

1.  New  units  receive  the  highest  level  of  attention.  Lecturers  are  funded  through  time  release  or  additional 
payments  to  design  the  new  unit.  They  are  supported  by  an  instructional  designer  and  a multimedia 
development  team.  These  units  are  designed  to  use  authentic  activities  as  a central  focus  of  the  unit. 
Authors  are  supported  through:  information  workshops;  demonstration,  development,  discussion  and 
evaluation  workshops;  group  and  one-to-one  consultations;  and  an  information  and  example(s)  web  site. 

2.  Major  revision  units  (existing  distance  units  in  print  mode)  where  the  focus  is  on  developing  sections  of  the 
unit  to  include  authentic  activities,  and  also  encouraging  students  (and  staff)  to  use  the  research  and 
communication  capabilities  of  the  Internet. 

3.  Minor  revisions  (existing  distance  units  in  print  mode)  receive  minimal  content  change  but  provide  the 
staff  and  students  with  enhanced  communications  facilities  (bulletin  boards,  chat  rooms.  Email,  etc.). 

Designing  authentic  online  activities 

As  Salomon,  Perkins  and  Globerson  (1991)  have  pointed  out:  ‘No  important  impact  can  be  expected  when  the 
same  old  activity  is  carried  out  with  a technology  that  makes  it  a bit  faster  or  easier;  the  activity  itself  has  to 
change,  and  such  a change  cannot  take  place  in  a cultural  vacuum’  (p.  8).  The  Masters  Online  project  has  a 
cultural  history  in  the  domain  of  the  traditional  printed  distance  education  unit.  The  temptation  for  many  of 
these  authors  is  to  produce  blocks  of  text,  similar  to  chapters  of  an  external  unit,  and  to  design  a variety  of 
activities  to  accompany  the  text,  with  separate  assignments.  Encouraging  the  writers  of  new  units  not  to  put 
copious  ‘content’  on  the  web  generally  requires  a huge  cultural  shift,  and  a substantial  rethinking  of  what  they 
want  students  to  achieve  and  how  they  could  enable  that  learning  to  occur. 

In  designing  the  new  units,  a complex  and  sustained  activity  (with  strong  teacher  support  and  peer 
collaboration)  is  the  focus  of  the  entire  unit.  Students  use  a purposeful  activity  to  organise  their  study,  to  give 
meaning  to  their  acquisition  of  information  and  to  provide  a framework  for  the  creation  of  a realistic  product. 
There  is  no  attempt  to  divide  the  course  into  discrete  fragments  of  easily  digested  information.  In  this  sense,  the 
activity  does  not  supplement  the  unit — it  is  the  unit. 

The  notion  of  the  activities  effectively  being  the  unit  is  one  that  is  quite  foreign,  and  one  that  requires  a major 
change  in  direction  for  these  teachers.  The  first  suggestion  from  an  instructional  design  point  of  view  has  been 
to  encourage  the  writers  not  to  begin  with  the  traditional  scope  and  sequence,  followed  by  the  division  of  the 
content  into  chapters  or  modules.  In  the  first  instance,  we  ask  the  writers  to  plan  the  activities  and  to  consider 
that  their  units  might  comprise  one  or  two  of  these  sustained  and  lengthy  activities.  In  order  to  assist  the  design 
process,  and  to  enhance  the  likelihood  that  the  importance  of  the  activity  as  a central  organising  device  in  the 
unit  is  prominent,  the  following  design  matrix  (Table  1)  is  given  to  each  writer,  with  its  four  main  planning 
areas,  together  with  space  for  them  to  jot  down  their  initial  ideas.  These  are  then  ‘brainstormed’  and  refined  in 
consultation  with  the  instructional  designer  and  the  project  team. 

The  design  of  the  activities,  and  supporting  resources  and  collaborative  tools,  in  many  cases  require  the  authors 
to  create  scenarios  which  call  for  the  students  to  adopt  a role  or  persona.  While  the  option  of  creating  real 
problems,  as  recommended  by  Savery  and  Duffy  (1996)  is  always  preferable,  simulated  situations  are  also 
prepared  and  are  considered  an  acceptable  vehicle  for  authentic  student  learning  (McLellan,  1994).  Table  2 
provides  an  example  of  a plan  for  a simulated  activity  together  with  its  assessment,  resources  and 
communication  strategies.  The  activities  being  designed  for  the  units  vary  enormously  (across  15  courses). 
Further  examples  of  the  kinds  of  activities  being  produced  include: 
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• In  a computer  graphics  unit,  students  produce  a piece  of  software  for  a particular  target  group,  together 
with  documentation  and  users’  manual.  Each  assignment  represents  a stage  in  the  completion  of  the  final 
product. 

• In  an  environmental  management  unit,  students  evaluate  and  prepare  a report  on  whether  a proposed 
housing  estate  development  will  pose  a risk  to  nearby  ecosystems. 

• In  an  educational  theory  unit,  students  write  journal  articles  in  groups  and  submit  their  articles  to  the 
editorial  board  of  a journal  (effectively  the  other  student  groups  for  peer  review). 

Table  1:  Web-based  learning  activities:  Instructional  design  guidelines 


Process 

Guidelines 

Advice 

Examples 

1.  Designing 
the  activities 

Design  activities  which: 

• have  real-world  relevance 

• are  ill-defined 

• comprise  a single  complex 
task  to  be  in-vestigated  by 
students  over  a sustained 
period 

• provide  opportunity  to  define 
tasks  & subtasks 

• Reflect  the  kind  of  problem  students 
would  face  in  real-life 

• Choose  a problem  which  enables 
students  to  apply  the  knowledge  you 
want  them  to  learn  in  your  unit 

• Let  the  task  be  the  central 
organising  device  for  the  students’ 
learning — don't  provide  explicit 
directions  and  sub-tasks 

• Case  studies  and 
role  play 

• Decision-making 

• Dilemmas 

• Presentations  to 
stakeholders 

• Public  defence  of 
position 

• Reports  and 
proposals 

2.  Designing 
the 

assessment 

Design  activities  which: 

• provide  the  opportunity  to 
collaborate 

• are  seamlessly  integrated 
with  the  assessment 

• Try  to  integrate  the  task  and 
assessment. 

• Set  group  tasks,  and  arrange 
students  in  collaborative  groups 
where  possible 

• Use  whole  student  group  to  evaluate 
each  other's  work. 

• Group  tasks 

• Peer  evaluation 

• Authentic 
assessment 

3,  Identifying, 
locating 
and/or 
producing 
resources 

Design  activities  which: 

• provide  the  opportunity  to 
examine  the  task  from  a 
number  of  different 
perspect-ives,  and  to  be 
able  to  detect  relevant  from 
irrelevant  information 

• can  be  integrated  across 
subject  areas 

• Create  your  own  resources  where 
necessary,  as  appropriate  to  the  task 

• Link  to  outside  sources  to  provide 
different  perspectives  and  access  to 
expert  thinking 

• Textbooks,  other  books  and  library 
journals  may  be  necessary  because 
you  can  find  no  online  equivalent 

• Video,  audio 

• Online 
documents 

• FAQs  etc 

• Websites 

• Online  journals 
and  databases 

• Textbooks 

• Books,  journals 

4.  Selecting 
collaboration 
coaching 
and 

communi- 
cation tools 

• Design  activities  which: 

• provide  the  opportunity  for 
students  to  examine  the 
task  from  a number  of 
different  perspectives,  and 
to  be  able  to  detect  relevant 
from  irrelevant  information 

• provide  the  opportunity  to 
collaborate 

• Encourage  collaboration  to  enable 
students  to  support  each  other's 
learning 

• Participate  in  online  discuss-ions  so 
that  your  voice  can  be  heard  through 
this  channel 

• Encourage  students  to  participate  in 
established  list-serves  to  enable 
them  to  be  part  of  worldwide 
discussions 

• Use  current  and  emerging 
technologies  to  communicate  with 
your  distant  students 

• Email 

• Discussion 
boards 

• Chat  sessions, 

• Online  tutorials 

• Listserves 

Lebow  and  Wager  (1994)  contend  that  ‘Human  learning  is  essentially  a matter  of  self-regulation’  (p.  239). 
However,  this  does  not  mean  that  students  can  be  given  complex  and  authentic  tasks  with  no  support  or 
coaching  to  guide  their  learning.  Wade  (1994)  points  out  that  the  promotion  of  learner  autonomy  means 
increased  responsibility  for  the  student  which,  if  it  is  to  succeed,  requires  ‘a  strong  framework  of  support  and 
guidance  for  the  student  from  the  outset’  (p.  13).  The  support  to  be  provided  by  the  tutor  of  the  online  unit  is 
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also  planned  concurrently  with  the  design  of  the  learning  activities,  together  with  opportunities  for  students  to 
collaborate  with  each  other  in  their  learning.  It  is  hoped  that  development  through  group  activity  and  dialogue 
will  be  more  enjoyable,  more  meaningful  and  more  effective. 


Table  2:  Example  of  plan  for  authentic  online  activity 


Stage  of 
development 

Example  activity:  The  ethics  of  research 

1.  Designing 
the  activities 

“For  this  activity,  you  are  required  to  assess  the  ethical  conduct  of  a proposed  piece  of  research.  You  are  to 
imagine  that  you  are  on  the  Committee  for  Ethical  Conduct  of  Research  with  Human  Subjects  in  your 
University.  Members  decide  on  whether  research  meets  ethical  standards,  but  the  whole  group  never  meet  in 
person.  You  decide  each  case  after  an  email  discussion.  (You  will  be  on  this  committee  and  the  other 
members  are  students  from  your  group  also  studying  this  unit.) 

The  committee  receives  an  outline  of  a research  proposal  from  a mature  student,  who  is  also  a teacher  In  a 
primary  or  elementary  school.  The  student  will  use  her  Year  6 class  as  subjects  for  the  proposed  research. 
The  students’  research  proposal  is  given  below: 

Title  of  research:  The  influence  of  stated  teacher  belief  on  student  assignment  writing 
Subjects:  Entire  group  of  Year  6 students  in  Social  Science  class 

Research  overview:  The  researcher  will  adopt  the  position  of  participant-as-observer  in  order  to  conduct 
the  research..  In  order  to  test  the  effect  of  stated  teacher  belief  (as  opposed  to  a genuinely-felt  belief) 
on  the  students’  assignments,  the  researcher  will  conduct  a short  survey  of  students  beliefs  on  the 
environment  at  the  beginning  of  the  school  year.  In  the  second  term  or  semester,  the  teacher  will  state 
a strongly  pro-logging  position  with  regard  to  the  logging  of  forests,  prior  to  the  commencement  of  a 
new  topic  area  in  social  science  class.  The  topic  will  include  discussion  of  environmental  issues 
relating  to  deforestation  and  its  Impact  on  ecosystems.  The  assignment  given  to  the  class  will  require 
an  evaluation  of  the  pros  and  cons  of  logging,  together  with  an  overall  assessment  of  whether  or  not 
logging  of  old  growth  forests  should  be  permitted.  The  Influence  of  the  teacher’s  stated  belief  will  be 
tested  against  the  survey  results  obtained  earlier  in  the  year. 

After  reading  the  brief  research  proposal,  you  need  to  respond  to  the  members  of  the  group  with  a summary 
of  the  ethical  issues  involved  and  the  conditions  under  which  the  research  could  be  supported.  Your  task  is 
to  frame  a considered  response  to  the  student.” 

2.  Designing 
the 

assessment 

“Because  the  Ethics  Committee  would  otherwise  be  snowed  under  with  responses,  a word  limit  of  1000 
words  has  been  placed  on  all  committee  members  for  each  response. 

“Send  your  evaluation  to  the  Unit  Online  Discussion  and  participate  in  the  discussion  to  decide  as  a group 
how  the  Ethics  Committee  should  respond  to  the  student.  You  will  be  assessed  on  your  initial  response 
and  your  final  group  response  to  the  student." 

3.  Identifying, 
locating 
and/or 
producing 
resources 

Resources  created,  e.g. 

• Create  email  message  for  members  of  the  committee,  and  student’s  research  proposal 
Links  to  outside  sources,  e.g. 

• Nuremberg  and  after: http://www.csu.edu.au/faculty/arts/humss/bioethic/resethic.htm 

• The  Ethics  Resource  Center:  http://www.ethlcs.org/ 

Books  and  journals:  e.g. 

• American  Psychological  Association.  (1993).  Ethical  principles  in  the  conduct  of  research  into  human 
participants.  American  Psychologist,  57(1),  33-51 . . 

4.  Selecting 
collaboration 
coaching 
and 

communi- 
cation tools 

Communication  tools:  e.g. 

• Online  threaded  discussions  board 

• Email 

Established  listserves:  e.g. 

• Ethics  Update  Discussion  Forum:  http://ethics.acusd.edu/kant.html 

By  the  end  of  the  initial  three  year  phase,  all  units  in  the  Masters  courses  will  be  available  in  an  online  mode 
and  a significant  number  will  use  authentic  activities:  they  will  act  as  models  for  the  future.  Teachers  will  have 
developed  the  technical  skills  necessary  for  online  teaching  in  a real  but  supported  context,  and  engaged  in  a 
range  of  problem-based,  reflective  activities  which  seriously  probe  the  nature  of  teaching  and  learning. 

Conclusion 
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The  adoption  of  online  learning  environments  has  frequently  been  an  ad  hoc  affair  in  universities,  often  led  by 
one  or  two  computer  enthusiasts  or  early  adopters  (Bates,  1999).  While  the  individual  approach  has  the 
advantage  of  commitment  and  speed  towards  implementation,  there  can  be  a tendency  for  early  users  to  simply 
replicate  existing  teaching  and  learning  practices  in  the  online  environment.  Many  staff  development  programs 
provide  technical  assistance  to  academic  staff  wishing  to  apply  new  communication  and  learning  technologies 
to  their  teaching.  Technical  advice  alone,  however,  seldom  challenges  teaching  staff  to  reflect  deeply  on  the 
nature  of  learning  and  to  reconsider  the  learning  goals  and  values  which  underpin  their  choice  of  teaching 
methods.  The  project  described  here  was  developed  specifically  to  encourage  academic  staff  to  move  beyond 
replication  of  existing  ideas  and  to  restructure  their  units  according  to  the  recent  theory  and  principles  of 
authentic  learning. 
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Abstract:  A novel  system  for  teaching  computer  science  online  is  presented.  The 
ProgrammingLand  MOO  is  a multiple  user  environment  where  students  may  learn 
content,  exercise  skills,  collaborate  with  other  students  and  demonstrate  competencies. 

It  is  intended  to  deliver  content  in  a learner  centered  fashion  that  is  both  engaging  and 
attractive  to  students.  The  system  is  designed  for  distance  education  but  has  also  been 
used  with  traditional  classroom  instruction.  It  has  been  used  for  introductory 
programming  classes  as  well  as  for  certain  junior  level  classes. 

Introduction 

The  ProgrammingLand  MOO  (Hill  & Slator  1998;  Slator  & Hill  1999),  is  a computer  aided  instructional 
system  to  teach  students  the  introductory  ideas  of  Computer  Science.  It  has  been  developed  and  used  in 
conjunction  with  traditional  classroom  instruction.  However,  its  goal  is  for  distance  learning  and  non- 
traditional  classes.  It  is  intended  to  deliver  the  content  that  would  normally  be  obtained  from  a lecture  or 
textbook,  yet  have  many  of  the  attractive  qualities  of  games  and  other  learner  centered  activities. 


Content  in  ProgrammingLand 

A typical  session  for  a student  starts  with  the  execution  of  a client  program.  There  are  several  applets  that 
allow  the  student  to  use  their  web  brower  for  this,  as  well  as  many  good  standalone  clients.  The  student 
connects  to  the  MOO  and  is  registered  for  tracking  purposes.  The  MOO  itself  is  structured  into  rooms, 
with  exits  being  the  path  from  one  room  to  another.  Each  student  has  a home  that  is  usually  the  main 
entry  way  of  the  MOO,  but  the  student  may  change  their  home  if  they  desire.  The  MOO  is  built  on 
Version  2.0.1  of  the  enCore  database  (Holmevik  & Haynes  1997). 

The  motif  of  ProgrammingLand  is  that  of  an  Exploratorium  style  museum.  Therefore  the  term  exhibit  is 
usually  used  instead  of  room.  Whenever  an  exhibit  or  room  is  entered,  the  MOO  displays  a description. 
Typically  this  is  a paragraph  or  two  of  instruction  on  some  topic.  The  MOO  will  also  indicate  what  exits 
exist  and  where  they  lead.  The  student  is  always  free  to  choose  which  path  to  take  and  what  to  look  at 
next.  Although  an  exit  is  a one  way  path  from  one  room  to  another,  they  often  come  in  pairs  so  that  a 
student  may  return  to  an  exhibit  conveniently.  The  student  entering  the  Compound  Statement  Room 
would  see  the  following  display. 


The  Compound  Statement 

The  compound  statement  is  not  a flow  of  control  statement,  however  it  is  used  in  most  flow  of 
control  statements  and  is  essential  to  the  Structured  Programming  model. 
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In  the  Structured  Programming  model  there  is  the  notion  of  a block.  The  block  in  C++  is  the 
compound  statement.  It  is  a wrapper  that  binds  several  statements  into  one.  It  is  also  the  block 
that  greatly  affects  the  scope  of  variables. 

You  may  choose  any  of  the  following  exhibits  to  consider  next. 

a)  The  syntax  of  the  compound  statement 

b)  Scope  of  variables  in  the  compound  statement 

c)  The  compound  statement  and  other  statements 
or 

x)  Return  to  the  main  exhibit  on  flow  of  control 

Figure  1 : The  Compound  Statement  Room 

In  summary  the  student  meanders  through  the  museum,  picking  and  choosing  what  they  want  to  read  and 
also  choosing  the  order  to  consider  topics.  If  this  were  the  extent  of  the  techniques  used  in  the  MOO  it 
might  be  useful,  but  would  not  be  superior  to  a series  of  web  pages.  What  must  be  considered  next  are  the 
objects  that  work  together  to  make  this  experience  more  educationally  beneficial. 


Fundamental  Objects 

In  a MOO  every  thing  is  an  object.  Rooms  are  objects,  exits  are  objects,  players  are  objects  and  everything 
else  that  we  will  later  consider  are  objects.  These  objects  have  properties  that  may  carry  information  and 
methods  that  may  perform  useful  functions.  The  student  is  oblivious  to  most  of  this,  they  walk  through  the 
museum  without  much  thought  to  the  processing  the  server  is  doing.  The  first  objects  that  need  to  be 
considered  are  the  student’s  character  and  the  rooms. 

When  a student  logs  into  the  MOO,  the  student’s  character  is  activated.  This  character  is  the  object  that 
the  server  uses  to  manage  everything  that  is  known  about  the  player.  There  are  several  important 
properties  that  are  attached  to  the  student  object  that  have  an  impact  on  the  educational  use  of  the  MOO. 
The  first  is  list  of  every  room  that  the  player  has  visited.  There  are  certain  objects  that  mark  events  and 
make  awards.  The  student  scores  points  in  this  way,  either  by  visiting  an  exhibit,  taking  a quiz,  or 
achieving  some  other  accomplishment.  Finally,  a student  has  a goal,  which  is  what  they  should  be 
working  on  currently. 

There  are  two  types  of  room  in  the  MOO.  The  first  type  is  called  the  lecture  room,  since  it  delivers  a 
paragraph  or  so  of  the  virtual  lecture.  This  is  the  object  that  tags  the  student  character  with  the  fact  that  it 
had  been  visited.  Its  description  property  contains  the  text  displayed  when  entered.  It  also  contains  a 
property  that  holds  quiz  questions,  discussed  below.  The  second  type  is  called  a workroom.  The  lecture 
room  is  the  only  type  of  room  that  records  a visit  on  the  student  character  and  is  the  most  common  type  of 
room.  A workroom  is  only  used  to  house  some  other  types  of  interactive  object. 

Currently  there  are  two  such  objects  that  are  frequently  used  in  workrooms:  the  code  machine  and  the 
workbench.  At  any  one  time  any  number  of  students  may  be  present  in  the  MOO.  They  may  interact  with 
each  other,  since  every  room  of  the  MOO  is  a chat  room.  However,  when  two  or  more  students  try  to  use 
either  of  these  types  of  machines  at  the  same  time  there  will  be  confusing  results.  Neither  student  will  be 
able  to  see  what  the  other  has  done.  Therefore,  a workroom  only  allows  one  student  into  itself  at  a time. 
Furthermore,  when  the  student  leaves  the  workroom,  the  workroom  will  reset  the  contained  machine  to  its 
starting  state.  Most  objects  other  than  rooms,  exits  and  players  can  be  picked  up  and  moved.  Few  students 
realize  this,  but  those  that  do  can  move  a machine  far  from  its  intended  location.  Therefore,  if  a student 
does  pick  up  the  code  machine  or  workbench,  the  workroom  will  remove  it,  when  they  leave. 


Interactive  Objects 

Some  of  the  objects  discussed  in  this  paper  could  be  generally  used  in  many  disciplines  but  the  code 
machine  and  workbench  are  specific  to  the  programming  discipline.  The  code  machine  contains  a piece  of 
programming  code,  which  it  will  display,  explain  or  trace.  The  code  machine  may  display  the  code  with  or 
without  line  numbers.  The  line  numbers  are  important  for  the  explanation  and  trace.  The  code  without  the 
line  numbers  allows  the  student  to  copy  the  code  from  the  MOO  client  and  then  paste  it  into  an  edit 
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window  and  actually  compile  the  code.  The  explanation  of  the  code  works  on  a line  by  line  basis.  The 
numbered  line  is  displayed  with  an  explanation.  The  student  then  requests  the  next  line.  The  trace  of  the 
code  is  a simulated  execution.  The  code  machine  displays  the  lines  that  are  executed  along  with  a 
description  of  what  is  happening  at  run  time,  including  the  new  contents  of  any  variables  that  are 
changed.  When  a student  completes  either  the  explanation  or  trace  of  a code  machine,  an  event  is  posted 
to  their  event  property.  The  event  captures  the  particular  code  machine  in  question  and  whether  this  was 
an  explanation  or  trace. 

The  MOO  is  not  used  to  compile  student  programs.  The  compiler  resides  on  the  student’s  own  computer 
or  perhaps  in  a publicly  available  machine.  Hence,  some  of  the  practice  that  would  be  desirable  for  a 
programming  class  must  be  done  out  of  the  MOO.  However,  the  workbench  represents  an  attempt  to  place 
part  of  this  practice  into  the  MOO.  A workbench  is  a table  driven  parser,  with  enough  of  the  tokens  of  a 
program  fragment  to  allow  the  student  to  test  whether  they  have  the  syntax  of  a contruct  correct.  The 
student  builds  a statement  or  program  fragment  from  the  pieces  that  are  inserted  into  the  workbench. 
When  complete  the  student  can  ask  the  workbench  to  determine  if  the  fragment  is  correct  or  not.  In  very 
simple  instances  the  workbench  may  also  interpret  the  code  and  run  the  program  fragment.  Like  the  code 
machine  a successful  parse  of  the  code  produces  a point-scoring  event  on  the  student  character. 


Lesson  Structure 

ProgrammingLand  is  not  a random  collection  of  rooms  containing  instructional  material  any  more  than  a 
textbook  is  a random  collection  of  pages.  There  is  an  order  imposed  that  organizes  rooms  into  related 
clusters  of  rooms  called  sub-lessons.  Sub-lessons  have  no  required  shape  or  size  but  often  have  some 
common  characteristics.  They  are  always  a group  of  rooms  on  a single  topic.  There  is  often  only  one 
entrance  in  to  the  group  and  usually  just  one  or  two  exits  out.  The  first  exhibit  in  the  group  is  usually  an 
entry  way  room  with  a short  introductory  paragraph  and  then  a menu  of  available  rooms.  Often  one  of  the 
rooms  has  only  text  that  attempts  to  motivate  the  student  or  explain  how  the  topic  fits  in  the  overall 
scheme  of  things.  There  is  often  a workroom  in  a sub-lesson  as  the  final  example.  A sub-lesson  may  be 
nested  within  a larger  sub-lesson. 

The  following  objects  were  created  to  support  the  lesson:  a sub-lesson,  a sub-lesson-exit,  a quiz  room,  a 
dispatcher  and  the  roving  goalie.  These  work  together  to  post  events  on  the  student  object  and  test  the 
events.  When  students  have  enough  point-scoring  events,  they  can  be  given  an  "out  of  MOO" 
programming  assignment  to  finalize  their  learning. 


Sub-lessons 

The  head  of  the  structure  is  the  sub-lesson  exhibit.  It  is  a descendent  of  the  lecture  room  so  contains  all  the 
properties  and  methods  of  that  object.  It  contains  two  additional  properties  as  well,  the  all_rooms  property 
and  the  requirements  property.  The  all_rooms  property  is  a list  of  all  the  exhibits  in  the  sub-lesson.  If  a 
student  is  given  credit  for  the  lesson  then  the  rooms  on  this  list  are  removed  from  the  student’s  history  of 
rooms  visited.  This  is  done  to  shorten  what  can  be  a long  list  of  rooms,  under  the  assumption  that  mastery 
of  the  sub-lesson  is  more  important  than  any  subset  of  rooms  visited.  The  requirements  property  states 
what  is  expected  for  a student  to  gain  credit  for  the  sub-lesson.  It  is  a list  of  lists.  If  they  satisfy  any  of  the 
sublists  they  are  given  credit  for  the  sub-lesson.  However,  to  satisfy  a list  they  must  satisfy  every 
requirement  of  the  list.  Each  of  the  sublists  may  require  any  combination  of  simple  room  visits  or  specific 
events  recorded.  The  order  that  these  requirements  are  satisfied  is  irrelevant,  only  that  the  students  did 
each  of  the  items  on  one  of  the  sublists.  A requirements  property  does  not  have  to  contain  multiple 
sublists,  but  that  is  often  the  case. 

The  sub_lesson_exit  is  a descendent  of  the  normal  exit  object  but  behaves  in  a rather  different  way.  When 
a student  chooses  an  exit  that  will  leave  the  subjesson,  their  progress  towards  satisfaction  of  the 
requirements  is  checked.  If  the  student  has  previously  met  the  requirements,  the  exit  moves  the  student  to 
their  intended  destination  with  no  action  out  of  the  ordinary.  If  any  of  the  sublists  of  the  requirements  of 
the  sub-lesson  have  been  fully  met,  then  the  exit  tells  the  student  that  they  have  completed  the  subjesson. 
It  also  posts  a completion  of  subjesson  event  to  their  event  list.  If  they  have  not  completed  the  subjesson 
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they  get  a different  set  of  messages.  The  first  sublist  of  the  requirements  is  checked  and  one  or  two  unmet 
requirements  are  displayed.  They  are  then  asked  if  they  want  to  continue  to  their  destination  and  finish  the 
subjesson  later  or  if  they  want  to  prove  their  mastery  with  a quiz.  If  they  opt  for  a quiz  they  are 
transported  to  a quiz  room. 


Quiz  Rooms 

The  quiz  room  is  a special  room  that  is  a descendent  of  the  regular  MOO  room  and  not  any  of  the 
aforementioned  rooms.  It  always  has  a single  exit,  which  is  the  destination  that  a student  usually  arrives  at 
when  leaving  the  subjesson.  It  has  no  entrances;  the  only  way  in  is  to  take  the  quiz  option  when  using  a 
subJesson_exit,  It  has  a single  new  method,  which  administers  the  quiz  to  the  student.  The  quiz  is 
randomly  generated  based  on  the  requirements  of  the  subjesson  and  the  state  of  the  students  progress 
toward  the  satisfaction  of  the  requirements.  The  quiz  generator  determines  the  rooms  that  the  student  has 
not  visited  by  looking  at  the  requirements  of  the  subjesson  and  the  history  of  the  rooms  from  the  student. 
Each  lecture  room  has  a property  that  contains  quiz  questions.  These  questions  should  be  about  the 
content  of  that  room  only.  It  then  gathers  the  available  quiz  questions  from  the  unvisited  rooms  and  makes 
up  a quiz  of  no  more  than  five  questions.  If  there  are  not  enough  questions  in  those  rooms  it  will  look  at 
the  subjesson  for  a store  of  more  general  quiz  questions  to  get  the  total  up  to  five. 

A quiz  consists  of  up  to  five  questions.  Each  question  is  a four  or  five  answer  multiple  choice  question. 
The  questions  stored  in  the  lecture  rooms  are  coded  with  three  distinct  pieces:  the  problem  statement, 
possible  right  answers  and  possible  wrong  answers.  The  quiz  generator  will  randomly  select  one  of  the 
right  answers  and  four  of  the  wrong  answers.  Therefore  it  is  desirable  to  form  questions  that  have  several 
right  answers  and  many  wrong  answers  to  prevent  students  from  communicating  these  questions  to  their 
peers.  The  randomization  of  the  answers  disallows  certain  types  of  multiple  choice  answers  like  both  b) 
and  c)  and  discourages  answers  like  all  of  these  and  none  of  these.  When  the  student  answers,  they  are 
given  immediate  feedback,  either  that  the  answer  was  correct  or  the  letter  of  the  correct  answer.  If  the 
student  answers  correctly  a sufficient  number  of  questions  then  the  quiz  posts  the  subjesson  completion 
event  on  the  student  character. 


Lesson  Dispatcher 

The  subJesson_exit  or  the  quiz  room  may  give  the  student  the  event  for  completing  the  subjesson.  This 
notice  is  put  in  the  student’s  events  property.  It  also  notifies  the  dispatcher  object  of  the  student  and  the 
event.  Certain  subjessons  are  allowed  to  change  the  goal  of  a student.  These  subjessons  are  noted  in  the 
dispatcher.  When  the  dispatcher  receives  the  subjesson  notification,  it  compares  the  subjesson  with  a 
list  of  subjessons  and  roving  goalie  objects.  If  this  subjesson  is  on  the  list,  then  the  matching  roving 
goalie  is  activated.  It  will  wait  for  three  seconds  and  then  move  to  the  location  of  the  student.  It  will  assign 
them  a goal  and  tell  them  some  messages  pertaining  to  that  goal. 


Goalies 

A roving  goalie  is  an  object  with  several  important  properties  for  the  interaction  with  the  student. 
Generally,  it  is  intended  to  give  each  student  a separate,  personalized  assignment,  usually  a programming 
assignment.  This  is  accomplished  by  three  message  properties.  The  first  is  the  prefix  message.  This 
congratulates  the  student  for  completing  the  subjesson  and  usually  starts  to  introduce  the  assignment. 
The  second  message  is  selected  from  a list  of  equivalent  assignments.  The  roving  goalie  maintains  an 
index  which  records  the  next  assignment  to  be  given.  When  an  assignment  is  given,  this  index  is 
circularly  incremented.  If  there  are  more  assignments  than  students  each  student  will  get  a different 
assignment.  If  there  are  more  students  then  several  may  receive  the  same  assignment.  The  third  message 
adds  whatever  concluding  information  may  be  appropriate.  Every  student  receives  the  same  first  and  third 
message,  but  the  second  message  will  vary  between  students.  This  message  may  be  quite  lengthy,  not  just 
a single  sentence.  If  the  student  ever  wants  to  reread  the  assignment,  the  goal  and  roving  goalie  are  noted 
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on  their  character.  They  have  the  showgoal  command  which  shows  them  the  three  messages  again. 
Moreover,  the  roving  goalie  records  the  student  which  received  each  assignment  index.  A student  may 
only  receive  one  such  goal  from  a particular  sublesson,  since  the  sub_lesson_exit  first  checks  that  they  do 
not  have  the  subjesson  completion  event. 


Toys  in  the  Attic 

In  the  spirit  of  the  Exploratorium,  the  ProgrammingLand  MOO  is  populated  with  a range  of 
demonstrations,  toys,  robots,  and  interactive  exhibits.  These  artifacts  are  intended  to  engage  a visitor  in 
the  exploration  of  the  content  stored  in  the  museum.  It  is  hoped  that  these  playful,  interactive  objects  will 
serve  to  both  engage  and  teach. 


Demonstration  Machines  and  Checker  Machines 

Demonstration  machines  were  built  for  Lisp  functions  as  an  in-class  project  in  the  summer  of  1998.  A 
class  of  50+  students  were  each  assigned  a unique  Lisp  function,  and  instructed  to  create  a machine  with  a 
’demo’  function  that  would  illustrate  the  operation  of  the  function.  For  example,  the  Lisp  ’cons  function’ 
room  contains  a description  of  the  ’cons’  function,  written  on  the  wall,  a ’cons'  machine  that  will  give  a 
demonstration  of  how  the  ’cons'  function  works,  and  a 'cons  checker’  machine  where  players  can  plug 
expressions  in  to  see  if  they’re  well  formed. When  the  student  plugs  a correct  value  into  the  machine  a 
congratulation  message  is  returned  and  an  ’award’  is  added  to  the  player’s  history.  When  the  player  makes 
a mistake,  the  feedback  includes  the  correct  answer. 


The  Recursive  Leprechaun 

Since  recursion  is  one  of  the  most  difficult  concepts  for  students  to  master,  it  is  important  to  expose  the 
students  to  recursion  as  often  as  possible.  One  approach  is  to  implement  a recursive  leprechaun,  which 
resides  in  the  Realm  of  Recursion.  The  recursive  leprechaun  demonstrates  a recursive  counting  function  in 
a visually  descriptive  manner.  For  example,  the  Leprechaun  gives  demonstrations  of  recursive  counting  by 
producing  new  leprechauns  from  a sack,  and  handing  them  an  N-1  value  to  count;  when  N reaches  zero, 
the  leprechauns  shout  their  value,  pass  it  to  the  next,  and  jump  back  in  their  neighbor’s  sack. 


The  Ring  Toss  Game 

The  Ring  Toss  game  is  intended  to  provide  an  amusing  challenge  in  the  area  of  associating  programming 
languages  with  their  historical  antecedents.  The  goal  of  the  ring  toss  game  is  to  associate  languages  with 
people  and  other  concepts.  More  than  one  ring  can  be  tossed  on  a single  peg. 


The  History  Jukebox 

The  History  Jukebox  is  a device  for  summarizing  programming  language  history  in  an  entertaining  and 
on-demand  fashion.  If  a player  were  to  'press  1959  on  History  Jukebox’  they  would  be  told  about  John 
McCarthy  at  MIT  and  the  origins  of  Lisp. 


Tutor  Robots 
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Tutor  Robots  were  implemented  to  make  the  function  rooms  in  the  LambdaMOO  wing  more  active  and 
engaging.  They  are  created  from  a prototype  Turing  Robot  developed  by  Ken  Schweller,  based  on  the 
Eliza  model  (Weizenbaum  1966)  which  was  inspired  by  Turing  (1950).  The  Robots  are  user 
programmable  and  capable  of  matching  key  words  and  sentence  patterns,  and  can  be  implemented  with 
random  responses  and  question  responses.  However,  Tutor  Robots  can  display  both  remarkable  responses, 
and  remarkably  obtuse  responses. 


Conclusion 

ProgrammingLand  was  never  designed  to  be  completely  self-contained.  The  student  will  always  need 
access  to  a language  system.  Thus  the  goals  assigned  by  the  roving  goalie  are  externally  satisfied.  Related 
MOOs  such  as  the  Geology  Explorer  (Slator  et  al.  1999)  have  a goal  system  that  is  completely  internal. 
When  a student  is  interacting  with  the  Geology  Explorer,  they  always  have  a goal,  usually  the 
identification  of  some  mineral  on  the  virtual  planet.  When  they  find  that  goal  they  are  congratulated  and 
given  a new  goal.  The  system  itself  may  monitor  the  progress  toward  the  goal.  If  they  need  a particular 
instrument  to  properly  distinguish  a mineral  from  others  that  may  appear  to  be  similar  and  they  leave  the 
starting  point  without  it,  a tutor  may  appear  and  give  them  some  instruction.  Similarly,  if  the  goal  mineral 
is  in  a room  and  they  leave  without  identifying  it,  then  a tutor  may  appear  and  give  them  some  instruction 
about  that  as  well.  These  advantages  stem  from  the  fact  that  the  goal  is  issued  by  the  MOO  and  progress 
towards  the  goal  may  be  monitored  by  the  MOO.  Though  there  are  many  parallels  between  Geology 
Explorer  and  ProgrammingLand,  there  are  some  differences  as  well.  Specifically  Geology  Explorer  is  a 
virtual  laboratory,  the  student  should  gain  experience  and  hone  skills.  ProgrammingLand  is  designed  to 
replace  either  the  textbook  or  the  lecture  or  both.  However,  its  laboratory  features  are  substantially  weaker, 
due  to  the  topic.  ProgrammingLand  provides  some  facilities  such  as  code  machines  and  workbenches  but 
the  bulk  of  the  experimentation  must  be  outside  of  the  MOO. 

Student  evaluations  of  a course  using  ProgrammingLand  were  generally  quite  high;  in  one  survey,  92% 
of  the  students  responding  rated  the  quality  of  the  course  as  either  above  or  much  above  average. 
Similarly,  88%  of  the  students  believed  their  understanding  of  the  course  content  was  either  good  or  very 
good. 

The  ProgrammingLand  MOO  is  an  interactive  learning  environment.  It  attempts  to  balance  content 
material  and  immersive  exploration.  It  is  more  interactive  than  a textbook  or  a lecture,  it  keeps  better 
records  than  web  instructional  pages,  it  also  productively  uses  the  information  that  it  records. 
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Abstract:  Increasingly  Educational  Hypermedia  Systems  (EHSs)  are  using  the  World  Wide  Web 
(WWW)  as  the  medium  of  delivery.  Many  of  these  systems  use  methods  and  techniques  from 
Adaptive  Hypermedia  Systems.  This  paper  focuses  on  the  interchange  of  information  about  the 
student's  knowledge  between  EHSs  over  the  WWW.  A conceptual  framework  based  on  a 4- 
dimensional  representation  is  introduced.  A proposed  implementation  of  this  conceptual  framework 
within  the  extensible  Markup  Language  (XML)  is  described.  The  paper  concludes  with  a brief 
summary  of  future  work. 


Keywords 
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Introduction 

The  field  of  education  has  always  been  a research  area  of  Adaptive  Hypermedia  Systems  (AHS)  [Brusilovsky, 
1996].  Every  Educational  Hypermedia  System  (EHS)  that  not  only  displays  prepared  content  to  a student  but  also 
changes  certain  styles  of  display  or  paths  through  the  system  is  an  AHS.  Increasingly  hypermedia  projects  use  the 
WWW  as  their  delivery  medium  [Brusilovsky  et  aL,  1998;  De  Bra  & Calvi,  1998;  Specht  & Oppermann,  1998; 
Nkambout  & Gauthier,  1996].  Most  of  them  lack  the  capability  to  share  information  about  the  users  or  students  with 
other  EHSs  and  are  therefore  isolated  or  closed.  The  result  of  this  isolation  is  that  every  EHS  has  to  organise 
information  about  the  users  by  itself.  To  overcome  this  isolation  we  introduce  an  approach  to  “open”  the  EHSs.  The 
approach  will  concentrate  on  information  about  the  student  but  will  also  identify  different  areas  where  openness  of 
the  EHSs  would  be  possible.  A conceptual  framework  for  an  n-dimensional  User  Profile  Interchange  (UPI)  is  shown 
and  future  work  in  this  area  is  outlined. 
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How  could  Educational  Hypermedia  Systems  be  open? 

EHSs  that  use  adaptive  techniques  are  AHSs  [Brusilovsky,  1996].  So,  it  can  be  assumed  that  EHSs  implement  the 
basic  processes  of  an  AHS,  which  are: 

• Gather  information  about  a student.  This  can  be  done  fully  automatically  or  collaboratively  with  the  student.  It 
is  also  possible  in  an  educational  context  that  the  teacher  provides  the  system  with  information  about  the 
student. 

• Create  a user  profile  of  the  student  based  on  the  information. 

• Adapt  the  content,  the  content  presentation,  the  teaching  strategy  or  the  navigational  support  to  the  students 
profile. 

It  would  be  beneficial  if  an  EHS  could  be  open  in  each  of  the  aspects  named  above.  This  means  that  the  information 
about  the  student  as  well  as  the  teaching  and  learning  strategies  and  the  adaptation  processes  might  be  exchanged 
between  the  different  systems.  However,  in  this  paper  we  will  concentrate  on  the  interchange  of  the  information 
about  the  student’s  knowledge. 

User  Models  in  Educational  Hypermedia  Systems 

Data  about  the  student  is  important  for  an  EHS  because  the  adaptation  process  depends  on  this  information.  Most 
EHSs  are  require  information  such  as: 

• Learning  preferences:  What  kind  of  teaching  or  learning  strategies  does  the  student  prefer,  e.g.  case  studies, 
problem  solving,  and  so  on? 

• Presentation  preferences:  Has  the  student  any  preferences  for  specific  presentation  styles  or  particular  media- 
types? 

• Background  and  experience:  What  background  and  experience  does  the  student  have? 

• Goals:  Which  goals  or  learning  goals  does  the  student  have?  What  goals  does  the  teacher  have  for  the  student? 

• Knowledge:  What  knowledge  does  the  student  have? 

Additional  information  about  the  student  can  be  gathered  during  the  usage  of  the  system.  The  system  could,  for 
example,  analyse  the  path  taken  by  the  student  through  the  document  or  content  space.  It  could  analyse  answers  to 
tests  or  interaction  during  a problem-solving  module.  The  system  is  then  able  to  update  the  user  profile  during  the 
presentation  and  adapt  the  presentation  on  the  fly. 

The  information  about  a student  is  processed  by  an  EHS  to  build  a user  profile  for  the  student.  If  the  same  student 
were  also  working  in  another  hypermedia  system,  then  this  system  would  also  have  to  build  a new  user  profile  for 
the  student.  A very  challenging  part  of  the  user  profile  is  the  representation  of  the  knowledge  of  a student.  Therefore 
we  concentrate  on  this  part  of  the  user  profile.  Different  approaches  have  been  tried  in  the  past  by  different  systems: 

• Stereotype  [Boyle  & Encamacioni,  1994],  e.g.  students  are  classified  into  groups  rather  then  being  classified  as 
individuals. 

• Boolean  [De  Bra  & Calvi,  1998],  e.g.  several  variables  are  used  to  classify  a student.  Each  variable  represents  a 
specific  knowledge  the  user  has  or  has  not. 

• Percentage  values  [da  Silva  et  al.,  1998],  e.g.  variables  represent  the  users  knowledge  about  a certain  domain. 
The  variable  has  iurthermore  a graded  value  that  holds  the  graded  knowledge  of  the  student  about  that  domain. 

The  information  about  the  user  could  also  be  represented  within  an  n-dimensional  framework  that  will  be  now 
introduced. 

N-Dimensional  knowledge  representation 

A feature  of  the  above  user  modeling  techniques  is  that  they  differ  in  the  level  of  detail  used  to  describe  a student. 
The  more  accurately  the  individual  student  and  his  knowledge  is  to  be  modeled  the  more  variables  are  needed.  It  is 
rather  difficult  to  convert  one  of  the  user  models  into  another  whilst  they  are  in  their  native  state.  For  a working  UPl 
model  we  have  to  build  a framework  that  allows  us  to  describe  all  user  model  structures,  such  as  stereotype, 
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Boolean,  and  so  on.  For  this  task  we  introduce  an  n-dimensional  user  knowledge  representation  scheme.  Each 
dimension  represents  the  extension  of  a user  model  with  the  capability  for  a more  fine-grained  representation  of  the 
students’  knowledge. 

The  following  table  explains  the  different  dimensions: 


Dimensions 

Description 

Corresponding  user  model 
concepts  in  AHS 

Non-adaptive  Model 
(0-dimensional) 

No  explicit  user  modeling  technique  is  used.  The 
systems  user  model  is  static  and  implicit.  The  user 
model  can  not  be  changed  and  the  presentation  is 
not  adapted. 

Every  non-adaptive  EHS 

Single  Variable 
(1 -dimensional) 

The  student’s  knowledge  is  modeled  only  with  one 
variable.  The  users  are  classified  into  groups  rather 
then  being  classified  individually. 

Stereotype  user  model 

Multiply  Variable 
(2-dimensional) 

The  model  of  the  knowledge  is  extended  by  one 
dimension.  There  are  now  multiple  variables 
available  to  model  the  student’s  knowledge.  The 
additional  dimension  indicates  whether  the 
knowledge  modeled  by  one  variable  is  present  or 
not. 

Boolean  approach 

Value  dimension 
(3-dimensional) 

Each  of  the  variables  used  by  the  2 dimensional 
approach  can  be  extended  with  an  individual  or 
range  of  different  values.  These  additional  values 
allow  an  even  more  detailed  description  of  the 
users  knowledge. 

Scalar  or  Percentage  approach 

Time  dimension 
(4-dimensional) 

There  is  not  much  research  work  available  about  4- 
dimensional  approaches.  A promising  idea  is  to 
introduce  time  as  a fourth  dimension  [De  Bra, 
1999].  Every  value  of  the  variable  could  be  marked 
with  a time-stamp,  after  some  time  the  knowledge 
of  the  student  begins  to  degrade. 

Knowledge  degrades  over  time. 

Possible  future 
dimensions 
(n-dimensional) 

Each  additional  dimension  offers  the  system  an 
even  more  detailed  view  of  the  student’s 
knowledge.  Future  research  in  the  area  may 
discover  more  dimensions  that  need  to  be 
considered.  The  problem  with  too  many 
dimensions  is  that  the  implementation  overhead 
grows  enormous  with  every  new  dimension. 

Theoretical 

Mapping  approaches 

The  UPI  between  n-dimensional  Open  Educational  Hypermedia  Systems  (OEMS)  is  difficult.  After  analysing  the 
different  dimensions  it  is  important  to  realise  that  there  are  just  3 different  approaches  to  map  the  information  about 
the  user  from  one  OEHS  to  another.  Depending  on  the  dimension  of  user  knowledge  representation  of  the  OEHS  is 
it  necessary  to  perform  a simple  mapping,  downsize  or  expand  on  the  information  about  the  user.  The  different 
approaches  will  be  now  introduced  in  more  detail. 

Simple  Mapping 

The  approach  of  simple  mapping  covers  all  the  cases  of  UPI  of  OEHSs  that  have  the  same  dimensional  structure  of 
their  user  model.  No  dimensional  conversion  has  to  be  done.  The  information  about  the  user  can  Just  be  copied  from 
one  OEHS  to  the  other.  The  problem  of  what  to  do  if  the  different  user  systems  do  not  have  the  same  knowledge 
domain  structure  remains. 
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Downsizing 


Downsizing  the  user  information  from  one  OEHS  into  another  is  necessary  whenever  the  source  OEHS  has  a higher 
dimensional  user-model  than  the  destination  OEHS.  During  the  transfer  of  the  user  profile  information  from  the 
source  OEHS  with  a higher  user  profile  dimension  to  a destination  OEHS  with  a lower  user  profile  dimension  it  is 
quite  obvious  that  there  will  be  a loss  of  information  about  the  user.  The  loss  of  information  is  larger  the  larger  the 
dimensional  gap  between  the  two  OEHSs  is.  A user  profile  that  is  transferred  from  a 4-dimensional  OEHS  into  a 1- 
dimensional  OEHS  loses  all  the  information  of  dimensions  2-4. 

The  downsizing  process  itself  has  either  to  be  done  manually  or  with  a sophisticated  automated  process.  This  will 
not  be  easy  and  requires  further  development  of  the  technologies  and  the  framework. 

Expansion 

The  user-model  of  the  source  OEHS  has  to  be  expanded  if  the  user-model  has  a lower  dimension  than  the 
destination  OEHS.  The  expanding  process  is  therefore  the  opposite  approach  to  the  downsizing  approach.  The 
difficulties  within  this  approach  lie  in  the  need  for  creating  additional  information  about  a user.  This  is  the  biggest 
problem  found  in  any  of  the  three  named  approaches. 

The  following  chart  visualises  our  n-dimensional  mapping  approach  of  UPI  in  OHES: 


Dimension  of 
Destination  AHS 


The  problem  of  different  knowledge  domains 

We  have  to  differentiate  between  two  cases  of  user  profile  interchange.  In  the  first  case  the  two  AHS  have  the  same 
structure  for  representing  the  knowledge  domain.  Furthermore,  not  only  the  representation  structure  is  the  same,  but 
also  the  perspective  of  the  domains  is  the  same.  For  example,  the  source  and  destination  AHSs  have  the  domain 
“multimedia”  and  the  domains  have  the  same  meaning  and  scope  in  both  AHSs. 

In  the  second  case  the  structure  of  the  knowledge  domains  of  the  two  AHSs  are  different.  Either  the  different 
domains  are  named  differently  or  the  have  the  same  names  but  not  the  same  meaning  for  the  two  AHSs.  For 
example,  the  domain  knowledge  concept  multimedia  symbolises  a complete  knowledge  about  multimedia  in  the  one 
AHS  but  only  coverage  of  video,  audio  and  text  elements  in  the  other  AHS.  The  mapping  between  non-identical 
knowledge  domains  in  this  second  case  is  very  difficult  to  implement  and  is  left  at  the  moment  for  further  research. 
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The  technique  of  the  user  profile  interchange  and  the  implementation  will  however  stay  the  same  in  both  cases.  The 
only  area  where  the  two  approaches  differ  is  the  grade  of  sophistication  in  the  mapping  of  the  user  knowledge  from 
one  OEMS  to  the  other. 

Implementation 

To  describe  the  different  dimensions  of  the  students  knowledge  we  use  the  extensible  Markup  Language  (XML) 
[XML,  1998].  XML  is  a markup  language  that  allows  the  creation  of  customised  markup  Languages  and  is  used  in 
several  projects  for  describing  educational  metadata  [ARIADNE,  1999;  Ram  et  al^  1999].  We  propose  the  creation 
of  a new  XML  Document  Type  Definition  (DTD)  specially  designed  for  User  Profile  Interchange  over  the  WWW. 
This  DTD  would  offer  the  possibility  of  describing  every  category  of  the  user  model  of  an  EHS.  It  would  be 
designed  so  that  it  can  be  extended  dynamically  to  describe  new  dimensions  of  the  user  model. 

If  two  OEHSs  want  to  exchange  information  about  a specific  student  they  would  create  XML  files  based  on  the  UPI- 
DTD  and  exchange  these  XML  files  over  the  WWW. 

As  soon  as  the  destination  OEMS  receives  the  UPI-DTD  tagged  information  about  the  student  it  can  start  to  import 
the  user  profile.  The  technical  realisation  of  this  import  process  could  be  done  with  any  program  language  capable  of 
reading  XML  files.  The  implementation  has  to  provide  the  different  mapping  approaches  to  the  OEMS  introduced 
above.  The  authors  are  planing  to  use  JAVA  for  this  task,  because  it  is  particularly  suited  to  working  with  the 
WWW  and  secondly,  there  are  tools  and  classes  being  developed  to  provide  XML  capability. 

Advantages 

The  biggest  benefit  of  this  approach  is  that  the  n-dimensional  UPI  builds  a meta-framework  for  student  knowledge 
representation.  The  XML  implementation  will  permit  an  interface  for  every  EHS  that  wants  to  exchange  user 
specific  information  over  the  WWW.  Other  advantages  are: 

• Information  about  a student  and  his  knowledge  will  no  longer  be  isolated  into  separate  systems  but  can  be 
exchanged  between  them. 

• It  provides  an  easy  and  dynamic  extendable  framework  that  is  able  to  describe  the  entire  present  user  modeling 
techniques  in  AHSs  and  EHSs. 

• XML  allows  the  easy  and  standardised  implementation  of  the  n-dimensional  UPI  model  for  the  WWW. 

Problems 

Despite  the  advantages  of  this  new  framework  covers  there  are  still  some  problems  which  have  to  be  solved.  The 
biggest  one  is  probably  the  one  about  the  different  knowledge  domains  of  the  OEHSs.  This  situation  is  difficult  to 
solve  without  additional  information  about  the  different  systems.  The  problem  is  not  so  big  if  one  of  the  two 
systems,  either  the  sending  or  the  receiving  one  knows  the  user  model  structure  of  the  other  EHS.  In  this  case  the 
system  that  knows  the  others  user  model  structure  can  export  or  import  the  user  information  in  the  proper  way. 
Some  other  problems  that  need  more  research  are: 

• At  the  moment  there  is  a lack  of  a common  terminology  for  describing  student  knowledge  variables. 

• How  can  variables  of  certain  dimensions  be  converted  to  variables  of  other  dimensions? 

• How  are  individual-based  user  modeling  techniques  transferred  into  group-based  modeling  techniques  and  vice 
versa? 

Conclusion  and  Future  Work 

In  this  paper  we  have  introduced  a new  conceptual  framework  for  User  Profile  Interchange  in  Open  Educational 
Hypermedia  systems.  This  new  framework  is  based  on  an  n-dimensional  projection  of  user  models.  The  framework 
will  be  implemented  in  XML  and  evaluated  in  our  future  work.  Subsequent  work  will  concentrate  on  the  interchange 
of  teaching  and  learning  strategies,  content  organisation  and  adaptation  processes  in  Open  Educational  Hypermedia 
Systems. 
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Abstract:  This  paper  explores  the  consequences  for  the  design,  development  and  evaluation  of 
educational  software  of  a model  of  understanding  information  technology  based  upon  teachers’ 
concepts  and  beliefs  about  this  technology.  The  model  was  the  result  of  a case  study  and 
considers  two  dimensions.  The  first  dimension  is  the  role  of  the  computer,  conceptualizing  it  as  a 
teaching  resource  that  helps  teachers  to  develop  their  teaching  strategy,  replacing  them  in  the  role 
of  managing  students’  rehearsal  of  materials  and  serving  as  a classroom  management  tool.  The 
second  dimension  is  the  pedagogy  that  teachers  embedded  into  the  characteristics  of  the  software, 
such  as  classroom  atmosphere  in  the  human-computer  interface  scenarios  and  elements,  the 
teaching  routines  in  the  design  of  the  navigation  structure  of  the  software,  and  the  learning 
strategy  in  the  human-computer  interaction.  The  main  consequence  of  this  model  is  in  illustrating 
a specific  integration  of  pedagogy-related  concepts  to  the  design,  development  and  evaluation  of 
educational  software,  thereby  demonstrating  the  need  and  usefulness  of  such  integration. 

Introduction 

This  paper  explores  the  consequences  of  a model  of  understanding  information  technology  in  the  processes  of 
designing,  developing  and  evaluating  educational  software.  The  model  is  the  result  of  a research  study  aimed  at 
understanding  teachers’  concepts  and  beliefs  about  educational  software  (Hinostroza,  1999).  The  research  was 
designed  as  an  instrumental  case  study  (Stake,  1994)  to  provide  insight  into  teachers’  concepts  and  beliefs  about 
computers.  The  process  of  definition  of  the  study  followed  (Yin,  1994)’s  recommendations.  The  ‘case’  was  a 
process  of  educational  software  development,  in  which  two  primary  teachers,  a software  engineer,  a psychologist 
and  a graphic  designer  were  committed  to  the  development  of  a piece  of  educational  software  over  a period  of 
seven  months.  Each  session  of  the  development  process  was  observed  and  video  recorded.  The  data  was 
analyzed  from  both,  qualitative  and  quantitative  stand  points,  using  systemic  networks  (Bliss  & Ogbom,  1 979) 
to  organize  and  give  a structure  to  the  categories  of  analysis  and  the  software  QSR  NUD*IST  to  support  the 
coding  process. 

The  unit  of  analysis  was  defined  as  a sequence  of  utterances  made  by  members  of  the  team  in  which  they  refer  to 
one  particular  aspect  of  the  software.  Where  they  referred  to  two  (or  more)  aspects  simultaneously  (in  parallel), 
these  have  been  considered  as  two  (or  more)  separate  units.  This  unit  was  named  ‘Discussion  about  Software 
Design’  and  was  described  through  the  systemic  network. 

The  two  main  attributes  that  characterized  the  unit  of  analysis  are:  the  ‘Topic’  discussed  and  the  ‘Participants’  in 
the  discussion.  The  ‘Topic’  of  discussion  was  divided  into  three  main  subjects  related  to  the  software  design. 
The  first  one  was  related  to  the  dimensions  of  software  engineering  (‘Subject  Areas’,  ‘Content  Organization’, 
‘Navigation’,  ‘Interaction’  or  ‘Interface  Element’);  the  second  one  was  about  the  characteristics  of  the  user 
(‘Teacher’,  ‘Pupil’  or  ‘Group’);  and  the  third  one  dealt  with  ‘Pedagogic  Issues’  that  were  related  to  software 
and/or  computers  (‘Aim’,  ‘Teaching  Strategy’  or  ‘Actions’). 

The  data  classified  was  analyzed  using  three  methods:  (i)  participation  analysis,  aimed  at  establishing  individual 
participation  profiles  during  the  development  process;  (ii)  sequence  analysis,  aimed  at  establishing  the  inter- 
relations among  the  different  components  of  the  software  based  on  the  analysis  of  the  patterns  of  sequences  of 
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units  in  the  data  for  the  group  and  for  each  team  member,  and  (iii)  contents  analysis,  aimed  at  looking  for  the 
meanings  expressed  by  the  teachers  about  the  different  dimensions  of  the  piece  of  educational  software.  These 
analyses  were  used  in  a complementary  way,  enabling  the  researcher  to  focus  on  relevant  issues  and  to  enable 
some  tri angulation  of  the  implications  drawn. 

The  main  implications  of  this  study  were  represented  through  a model  of  understanding  information  technology 
that  is  presented  in  the  next  section. 

The  Model  of  Educational  Software 

This  model,  firstly,  shows,  what  the  participating  teachers  conceived  of  as  the  role  of  the  computer,  and, 
secondly,  presents  the  characteristics  of  the  educational  software  that  these  teachers  designed  and  shows  the 
dimensions  of  their  teaching  strategies  that  were  embedded  in  these  characteristics. 

In  discussing  the  role  of  the  computer  in  the  classroom,  the  teachers  indicated  how  important  for  them  the  role  of 
the  computer  as  rehearsal  tool  , this  was  the  role  that  they  naturally  picked  up  and  expanded  on.  They  designed 
this  role  by  integrating  it  into  a larger  teaching  strategy  that  separated  teaching  into  two  stages:  learning  new 
concepts  and  rehearsing  these  concepts.  This  model  of  use  of  the  computer  is  represented  in  Fig  1 . 


Figure  1.  Model  of  the  role  of  the  computer 


In  this  model  the  computer  is  acting  as  a resource  for  the  teacher  that  helps  him/her  carry  out  some  of  their 
current  tasks  more  effectively.  Some  ground  rules  in  this  model  are  that  (Hinostroza  & Mellar,  2000): 

• the  teacher  teaches  new  concepts  in  the  classroom  and  not  in  the  computer  room; 

• the  teacher  does  not  interact  with  the  computers; 

• the  teacher  when  working  in  the  computer  room  guides  the  pupil’s  construction  of  knowledge  through 
managing  the  classroom  and  solving  computer  problems,  but  (s)he  does  not  engage  with  the  contents  to  be 
learned; 

• the  teacher  manages  the  pupil's  access  to  the  computers,  and  by  incorporating  them  within  the  punishment- 
reward  system  of  the  classroom  (s)he  is  able  to  use  them  to  manage  the  class. 

Regarding  the  design  of  the  piece  of  software,  it  was  found  that  they  transferred  several  teaching  principles,  or 
classroom  routines,  to  characteristics  of  the  educational  software.  These  principles  can  be  classified  as  the  three 
different  types  of  classroom  routines  described  by  (Leinhardt,  Weidman,  & Hammond,  1 986): 

• Management,  this  includes  housekeeping,  discipline  maintenance  and  people  moving  tasks. 

• Support,  that  is,  specific  behaviors  and  actions  necessary  for  a learning-teaching  exchange  to  take  place,  for 
example  ‘how  to  pass  in  papers’. 

• Exchange,  that  is,  the  interactive  behaviors  that  permit  the  teaching-learning  exchanges  to  occur.  They  govern 
the  language  contacts  between  teachers  and  students  - for  example,  routines  for  choral  responses. 

The  correspondence  of  these  routines  and  the  software  characteristics  is  presented  in  Tab.  1 (Hinostroza  & 
Mellar,  1999). 
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Teaching  Routines  Domain 

Software  Design  Domain 

Management:  In  this  case  classroom  atmosphere  and 
tone. 

The  human-computer  interface  elements  (scenarios, 
backgrounds,  characters  and  particular  functionality  of  these 
elements)  and  the  overall  organization  of  the  subjects. 

Support:  In  this  case,  lesson  flow,  contents  provision, 
and  control  of  the  user’s  progress. 

The  navigation  structure  of  the  software  and  the  response  to 
certain  situations  (for  example,  feedback  on  errors). 

Exchange:  In  this  case,  learning  strategies  or  theories. 

Particular  human-computer  interaction  and  user’s  actions. 

Table  1.  Correspondence  between  teaching  routines  and  software  design  domains. 


The  model  was  grounded  in  two  sources  of  evidence,  first,  the  data  collected  in  the  research  and  second  the 
evidence  reported  in  research  studies  about  teachers’  everyday  practices  in  the  classroom,  (i.e.  their  strategies, 
routines,  roles  and  beliefs).  This  latter  source  of  evidence  expands  the  theoretical  framework  of  the  study, 
covering  not  only  the  area  of  information  technology  in  education,  but  also  the  area  of  pedagogy.  In  this  sense, 
this  model  of  educational  software  constitutes  an  interesting  starting  point  for  further  research  in  this  area.  We 
now  turn  to  the  discussion  of  the  consequences  of  this  model  for  the  design,  development  and  evaluation  of 
educational  software. 

Implications  for  Educational  Software  Design 

Educational  software  is  still  an  arena  for  debate  and  controversy,  and  as  yet  there  are  no  clear  design 
prescriptions  that  could  ensure  its  actual  use  nor  its  effectiveness  (see  for  example  discussions  in:  Cuban,  1997; 
Johnson,  Cox,  & Watson,  1994;  Lowther  & Sullivan,  1994).  We  (also)  suggest  that  much  of  the  reason  for  this 
continuing  controversy  is  that  little  attention  has  so  far  been  paid  to  a consideration  of  teachers'  actual  classroom 
practices. 

Based  on  the  model  presented  in  the  previous  section  it  is  possible  to  argue  that  in  order  to  design  software  one 
important  issue  to  address  is  the  role  of  the  software  in  the  teaching  strategy  that  the  teacher  will  implement.  In 
this  case  the  software  was  defined  in  such  a way  as  to  have  a role  in  the  rehearsal  stage  of  the  teaching  strategy, 
but  it  is  also  possible  to  imagine  that  it  could  have  a role  in  the  stage  where  the  teacher  explains  new  concepts 
(for  example  a piece  of  simulation  software  would  probably  fit  into  this  stage). 

The  implication  of  this  definition  is  that  the  software  is  designed  to  play  this  and  no  other,  role  while  teaching. 
This  helps  to  focus  the  design  of  the  software  to  meet  a particular  set  of  learning  requirements  only,  without 
pretending  to  support  the  complete  teaching  and  learning  process. 

Given  the  role  of  the  computer  in  a given  teaching  strategy,  the  other  consequences  are  related  to  the  design  of 
the  characteristics  of  the  software  (Human-Computer  Interface,  Navigation  Structure  and  Human-Computer 
Interaction)  and  will  be  presented  in  the  following  sections. 

Human-Computer  Interface  Elements 

It  was  argued  that  this  characteristic  of  the  software  was  related  to  the  design  of  management  routines, 
particularly  to  the  creation  of  a ‘classroom  atmosphere  and  tone’.  Regarding  the  latter,  (Woods  & Jeffrey,  1996) 
describe  how  primary  teachers  create  ‘atmosphere’  and  use  different  ‘tones’  in  interaction  with  the  children  to 
achieve  different  effects,  they  describe  it  as  having  the  following  characteristics; 

• Anticipation  and  expectation:  teachers  are  skilled  in  the  construction  of  situations  and  in  their  sense  of  timing. 

• Relevance:  teachers  are  able  to  ensure  that  pupils  feel  a strong  sense  of  involvement  by  making  the  pupils  identify  and  get 
involved  in  the  atmosphere. 

• Achievement  and  success:  there  is  a sense  of  high  teacher  expectations  and  confidence  in  children’s  abilities  to  meet  them. 

• Satisfaction:  Pupils  feel  satisfaction  and  the  sense  of ‘a  job  well  done’  because  of  the  achievement. 

These  characteristics  are  part  of  the  techniques  that  teachers  use  during  their  lessons.  (Woods  & Jeffrey,  1996) 
also  describe  the  different  tones  that  teachers  use  during  the  lesson  as  “Andante  (to  be  performed  in  moderately 
slow  time)”,  “Legato  (smoothly  and  connectedly,  no  gaps  or  breaks)”  and  “Spiritoso  (with  spirit)”. 
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These  techniques  can  be  considered  in  the  design  of  the  human  computer  interface  (HCI)  elements  of  the 
software,  for  example: 

• Creating  the  background  of  the  HCI  as  a story,  so  that  students  can  get  involved  in  it  and  believe  it. 

• Designing  the  storyboard  so  that  it  can  be  reviewed  in  one  lesson  and  that,  independently  of  the  particular  path  followed  by 
the  users,  they  find  a coherent  end  to  the  story. 

• Designing  the  graphic  representation  of  the  characters  and  scenarios  of  the  HCI  so  that  they  are  known  to  the  users. 

• The  organization  of  the  ‘contents’  of  the  software  should  take  care  that  users  can  review  the  complete  story  in  one  lesson. 

The  Navigation  Structure 

It  was  argued  that  the  design  of  the  navigation  structure  was  related  to  the  support  routines,  in  this  case  this 
implies  that  the  software  should  implement  the  ‘flow’  of  the  lesson.  With  respect  to  this,  (Hammersley,  1990) 
says  that  teachers  present  the  lesson  “pitched  at  a certain  level  of 'difficulty'  according  to  the  co-ordinate  position 
of  the  class  in  relation  to  age  and  ability”  (p.  47).  He  argues  that  teachers  gradually  provide  more  and  more  clues 
to  ‘what  the  answer  is’  in  order  to  pace  the  lesson  in  such  a way  as  to  get  the  answer  to  emerge  towards  the  end 
of  the  lesson.  Further  on,  he  gives  some  reasons  why  they  do  this:  “this  pre-setting  is  designed  not  only  to  ensure 
that  pupils  are  taught  something  ‘new’,  that  they  ‘keep  moving’,  but  also  that  they  have  the  resources  to 
understand  what  the  teacher  is  to  teach”  (p.  47). 

This  strategy  can  be  transferred  to  the  design  of  the  navigation  structure  of  the  software  using  a schemata  such  as 
the  one  presented  in  Fig.  2. 

Individual  screens  of 


Figure  2.  Navigation  structure  of  the  software. 

In  such  a structure  users  can  navigate  through  different  sequences  of  ‘contents’  accordingly  to  their  ‘level’  of 
achievement  and  will  have  the  opportunity  to  learn  something  new  and  to  keep  moving  as  Hammersley  describes 
students  as  doing  in  a teacher  taught  lesson. 

Human-Computer  Interaction 

It  was  argued  that  the  design  of  the  human-computer  interaction  was  related  to  the  exchange  routines,  in  this 
case  that  is  to  the  teachers’  learning  theories  (conceptions)  that  govern  the  teacher-student  dialogue.  The 
literature  describes  the  teacher-student  interaction  as  based  mostly  on  dialogues  that  have  the  structure  of 
question,  answer,  and  feedback.  About  the  former,  (Gall  & Artero-Boname,  1995)  classifies  teachers’  questions 
during  a lesson  as:  lower  or  higher  cognitive  questions;  recitation  and  discussion  questions;  or  as  test-relevant 
questions. 

The  feedback  provided  to  students  has  also  been  studied.  (Mayer,  1995)  describes  the  educational  uses  of 
feedback  as  corresponding  to  academic  learning  tasks,  behavior  management  tasks  and  skill  learning  tasks.  In 
the  first,  feedback  provides  information  concerning  the  correctness  of  students'  performance,  in  the  second,  it 
provides  information  concerning  the  appropriateness  of  students'  behavior  and  in  the  third  it  provides 
information  concerning  the  accuracy  of  students'  behavior. 


461 


BEST  COPY  AVAILABLE 


These  two  areas  provide  a rich  framework  for  designing  the  human-computer  ‘dialogues’  that  take  into 
consideration  the  type  of  questions  that  the  software  should  consider  and  the  different  types  of  feedback  that 
need  to  be  considered. 

Implications  for  Educational  Software  Development 

Looking  at  the  literature  that  describes  educational  software  development  (for  example:  Eraut,  1996;  Galvis, 
1994;  Squires,  1996),  there  are  few  references  to  issues  such  as  ‘Classroom  Atmosphere’,  ‘Lesson  Flow’,  or 
‘Classroom  Management’.  Although  some  of  these  reviews  mention  ‘pedagogy’  as  a dimension  that  needs  to  be 
considered  (for  example:  Squires,  1996),  they  do  not  expand  on  these  issues.  Whilst  it  is  true  that  teachers  have 
sometimes  been  incorporated  into  the  software  development  process,  they  have  generally  been  asked  to 
contribute  to  the  learning  and  curriculum  design  issues,  and  the  issues  of  actual  classroom  practice  have  been 
neglected. 

This  lack  of  material  discussing  these  issues  indicates  the  low  importance  that  seems  to  have  been  given  to 
knowing  and  considering  the  actual  use  of  the  software  in  order  to  design  it.  Through  a study  of  the  literature 
relating  to  pedagogy  it  was  possible  to  identify  at  least  three  theoretical  areas  that  could  influence  the 
development  process  of  educational  software  (i.e  reported  techniques  for  classroom  atmosphere,  lesson  flow  and 
learning). 

These  findings  have  clear  implications  for  the  software  development  process,  in  so  far  as  they  show  the  need  to 
incorporate  these  dimensions  during  the  development  of  a piece  of  educational  software.  Some  concrete 
consequences  are  for  example: 

• The  need  to  define  a teaching  strategy  first  and  then  analyze  in  which  stage  the  software  should  play  a role. 

• The  need  to  define  what  teaching-learning  problem(s)  will  be  addressed  through  the  software.  For  example,  improvement 
of  the  efficiency  and/or  effectiveness  of  the  teaching-learning  process. 

• It  places  the  teacher  as  a relevant  user  of  a piece  of  educational  software  and  therefore  the  software  engineer  needs  to  take 
into  consideration  a new  area  of  expertise  (i.e.  pedagogy)  to  inform  the  decisions  during  the  development  process. 

• The  test  procedures  during  the  development  process  should  contemplate  observations  of  teachers  using  the  software  in  a 
normal  lesson. 

Implications  for  Educational  Software  Evaluation 

As  mentioned  previously,  the  design  of  educational  software  to  date  has  been  primarily  focused  on  learning  and 
has  paid  little  attention  the  teaching  dimension,  and  as  a result,  teachers  have  difficulty  incorporating  it  into  their 
practice.  Software  evaluation  has  paid  little  attention  to  this  area,  and  has  only  recently  begun  to  give  serious 
consideration  to  the  teaching  dimension  (for  example  see:  Wishart  & Blease,  1999). 

So,  it  could  be  argued  that  the  evaluation  and  selection  of  a piece  of  educational  software  could  be  improved  if 
these  issues  were  considered  and  ‘evaluated’,  defining  additional  standards  to  judge  and  characterize  a piece  of 
software.  For  example,  to  evaluate  the  interface  elements  of  a piece  of  software  by  considering  the  design  of  a 
‘Classroom  Atmosphere’,  the  navigation  structure  should  be  evaluated  taking  into  consideration  Hammersley 
(l990)’s  descriptions  of  the  strategies  that  teachers  use  in  the  classroom  and  the  human-computer  interaction 
should  be  evaluated  considering  the  principles  of  what  the  literature  defines  as  good  practice  in  questioning  and 
providing  feedback.  Some  specific  implications  for  this  area  are,  for  example: 

• Prior  to  the  evaluation  of  a piece  of  software,  the  teaching  strategy  should  be  defined,  this  in  order  to  specify  the  context 
(teaching  aims,  environment  and  techniques)  in  which  it  would  be  used. 

• The  evaluation  criteria  should  consider  the  teaching  routines  used  by  teachers  of  the  subject  area  and  school  level  in  which 
the  software  is  to  be  used. 

• The  evaluation  should  know  and  consider  the  users’  previous  knowledge  (abilities,  skills),  given  the  stage  of  the  teaching- 
learning  process  in  which  it  will  be  used. 

• The  evaluation  should  consider  measurements  to  qualify  and/or  quantify  the  improvements  in  the  teaching-learning 
process.  For  example  in  terms  of  the  efficiency  of  the  process,  this  is,  instead  of  investing  five  lessons  to  teach  something, 
with  the  software  it  can  be  done  in  three  lessons  with  the  same  (or  better)  results. 
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Conclusions 


These  implications  for  educational  software  design,  development  and  evaluation  indicate  new  issues  that  should 
be  considered.  They  show  that  there  is  a need  for  further  research  that  could  help  to  define  the  specific  teaching- 
learning  strategies,  principles  and  routines  that  should  be  known  and  used  to  define  and  evaluate  the  particular 
characteristics  of  a piece  of  educational  software.  If  considered,  they  can  change  the  very  conceptualization  of 
such  products  and  eventually  transform  them  into  teaching  aids  or  ‘professional  tools’  for  the  teachers. 

Finally,  it  must  be  considered  that  whilst  our  intention  here  was  to  present  one  model  of  educational  software 
and  thereby  to  illustrate  how  the  prescriptions  of  this  model  could  be  use  to  improve  the  processes  of  designing, 
developing  and  evaluating  educational  software,  it  must  be  kept  in  mind  that  this  model  is  the  result  of  a small 
case  study  and  all  that  it  enables  us  to  do  is  to  propose  this  model  and  its  consequences  tentatively,  recognizing 
that  it  requires  much  wider  verification. 
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Abstract:  This  paper  explains  how  a WWW  site  was  designed  to  assist  preservice  teacher 
education  students  in  using  a reflective  framework  to  study  how  they  learned  in  university 
classes.  There  were  three  phases  in  the  reflective  framework:  (i)  analysis;  (ii)  synthesis;  and 
(iii)  theorizing.  In  this  last  phase  the  students  developed  a metaphor  to  represent  an  optimum 
classroom  learning  environment.  The  web  site  was  designed  using  a FileMaker  Pro  database 
with  a template  for  each  category  of  learning — personal,  teaching,  peer  and  situational — ^to 
assist  students  in  managing,  sharing  and  theorizing  about  their  reflections.  Students  claimed 
that  the  web  site  supported  their  reflections  and  the  metaphor  helped  them  to  conceptualize  the 
dynamics  of  classroom  relationships. 


Introduction 

The  notion  of  being  reflective  means  rethinking  experiences,  particularly  ones  which  are  problematic,  to 
make  clearer  sense  of  them  and  to  gain  insights  for  subsequent  experiences  (Dewey,  1933;  Schon,  1983;  1987). 
It  is  now  common  practice  in  preservice  teachers  education  programs  to  encourage  students  to  use  reflective 
journals  to  rethink  and  analyze  their  experiences  in  university  subjects  (Loughran,  1995;  McRobbie,  1994; 
Wilson,  Hine,  Dobbins,  Bransgrove,  & Elterman,  1995).  This  includes  reflecting  upon  education  readings,  class 
discussions,  personal  thoughts  or  school  experiences.  However,  the  focus  of  the  reflection  is  often  about  w^hat 
students  understand,  what  they  don’t  understand  and  to  specify  areas  which  require  further  readings  or 
explanations.  Rarely  are  preservice  teachers  asked  to  reflect  upon  how  they  learn,  how  they  are  being  taught,  or 
how  they  interact  with  other  students. 

During  their  training,  preservice  teachers  are  exposed  to  a wide  range  of  teaching  strategies  and  content 
as  well  as  interacting  with  a variety  of  students.  This  exposure  to  a variety  of  learning  situations  provides  a 
wonderful  opportunity  for  preservice  students  to  study  how  they  learn.  When  trainee  teachers  gain  these  insights, 
they  can  deduce  implications  for  their  own  pedagogy  based  on  personal  experiences.  Although  this  idea  was 
raised  over  30  years  ago,  it  is  a rare  practice  in  teacher  education  programs: 

The  student  in  the  process  of  becoming  a teacher  is  not  made  acutely  aware  of  how  he  is  learning,  that 
is,  to  utilize  himself  as  a source  of  understanding  of  the  nature  of  the  learning  process.  . . . one  of  the 
major  reasons  so  many  teachers  are  dissatisfied  with  themselves  in  their  work  is  that  their  training  did 
not  illuminate  the  nature  of  their  learning  process  and  how  this  relates  to  and  affects  the  learning 
process  of  their  pupils.  (Sarason,  Davidson,  & Burton,  1962,  p.  118) 

When  preservice  students  are  asked  to  reflect  upon  complex  interactions  such  as  how  they  are  learning, 
technologies  such  as  the  World  Wide  Web  can  be  a useful  tool  to  support  students  in  documenting,  analysing 
and  sharing  insights  (Bennett,  1998).  The  purpose  of  this  paper  is  to  explain  a reflective  framework  which 
encourages  preservice  teachers  to  study  how  they  learn  in  university  classes  and  how  a World  Wide  Web  site 
was  designed  with  templates  to  help  students  manage  and  theorize  about  their  reflections. 


Using  WWW  Templates  to  Support  a Reflective  Framework  for  Studying  Learning  in 
University  Classes 

Over  the  last  four  years,  a reflective  framework  has  been  developed  to  guide  preservice  teachers  in 
studying  how  they  learn  in  university  classes  (Hoban,  1995,  1997,  1998).  This  framework  encourages  each 
preservice  teacher  to  be  “a  researcher  in  the  practice  context”  (Schon,  1983,  p.  68)  with  the  context  being  their 
experiences  as  learners  in  their  university  classes.  There  are  three  phases  in  the  framework  which  encourage 
students  to  analyze  their  experiences,  synthesize  key  factors  which  influence  learning  and  to  develop  a metaphor 
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to  represent  an  optimal  learning  environment.  The  first  phase  occurs  for  the  duration  of  the  subject  and  the 
second  and  third  phases  are  completed  by  students  towards  the  end  of  the  subject,  although  they  can  be 
completed  concurrently  by  students. 

In  1 999,  a WWW  site  was  designed  with  a FileMaker  Pro  database  as  a tool  to  help  preservice  students 
use  the  reflective  framework.  The  site  was  designed  with  the  intention  of  minimising  the  cognitive  load  on 
students  when  reflecting  by  using  a screen  design  which  '‘promotes  understanding  by  allowing  the  reader  to 
focus  on  new  information  rather  than  devoting  time  and  energy  to  variations  for  format”  (El-Tigi  & Maribe 
Branch,  1997,  p.  25).  In  addition,  the  web  site  assisted  students  in  organizing  and  theorizing  about  their 
reflections  to  deduce  a metaphor  to  represent  an  optimal  learning  environment.  In  spring  semester  1999,  25 
trainee  teachers  used  the  WWW  site  in  a preservice  science  methods  subject  which  lasted  for  13  week  in  an 
elementary  teacher  training  program.  The  students  had  a three  hour  class  each  week — two  hours  were  spent  with 
the  students  doing  hands-on  science  experiments  and  a third  hour  was  spent  reflecting  upon  how  they  learned  in 
the  class  using  the  web  site.  The  three  phases  of  the  reflective  framework  are  now  described  as  well  as  features 
of  the  web  site. 

Phase  1 Analysis.  After  each  university  class  students  logged  onto  the  WWW  site  and  had  to  reflect  upon  their 
class  experiences  to  identify  the  personal,  social  (teaching  and  peer)  and  situational  factors  which  influenced 
his/her  learning.  There  was  a template  for  each  category  which  is  consistent  with  a constructivist  perspective  that 
views  learning  as  an  individual  process  of  knowledge  construction  that  is  supported  by  social  interactions  with 
the  outside  world  (Duffy  & Cunningham,  1993).  The  templates  were  labelled  according  to  the  following  four 
categories: 

1 . personal  factors  attributed  to  each  student,  such  as  prior  knowledge,  feelings,  self  esteem,  motivation 
and  personal  learning  strategies; 

2.  teaching  factors  attributed  to  the  instructor/tutor,  such  as  class  organisation,  teaching  strategies,  class 
organisation  goals,  and  rapport; 

3.  peer  factors  attributed  to  other  students  such  as  how  they  encourage  each  other,  share  ideas  and 
cooperate  in  tasks;  and 

4.  situational  factors  attributed  to  the  task,  setting  and  environment. 

After  each  class,  the  students  identified  factors  which  enhanced  or  inhibited  their  learning  and  documented  these 
processes  or  strategies  in  the  templates  on  the  WWW  site.  The  following  is  an  example  of  a student’s  weekly 
reflections  as  written  in  the  four  templates  of  the  web  site  in  week  2 of  the  subject.  It  should  be  noted  that 
students  noted  a positive  factor  with  a “+”  and  a negative  factors  with  a and  highlighted  key  words  using 
upper  case: 

Personal  Factors 

- 1 am  not  confident  with  teaching  science  in  front  of  an  audience  therefore  I feel  this  elective  will  be  very 
rewarding  as  I will  gain  KNOWLEDGE,  CONFIDENCE  and  UNDERSTANDING  of  how  to  teach  science  to  a 
young  audience. 

+ I work  better  in  an  environment  where  I am  comfortable  and  feel  free  to  exchange  ideas  and  questions  without 
the  worry/fear  that  others  will  criticize  me.  This  style  of  teaching  suits  the  way  I learn. 

+ As  this  was  the  first  real  science  lesson  I had  no  idea  of  what  to  expect  from  this  elective  so  I was 
INTERESTED  and  WILLING  to  get  involved  with  the  debates  and  answer  any  questions  thrown  at  me. 

Teaching  Factors 

+ Today’s  teaching  strategies  were  very  suited  to  my  learning  and  understanding  of  science.  I enjoy  a 
RELAXED  environment  where  I feel  free  to  contribute  to  the  class  discussions. 

+ It  was  amazing  how  pre-knowledge  that  I have  gained  over  time  was  used  in  this  lesson.  I didn’t  realize  simple 
experiments  can  make  things  clear. 

- Initially,  I didn't  feel  it  was  necessary  to  write  a reflection  of  the  lesson  on  the  internet.  I prefer  to  share  and 
discuss  how  I feel  in  some  cases  I feel  it  easier  to  talk  about  problems  or  ideas  rather  than  writing  it  down. 

- I’d  prefer  writing  on  the  white  board  to  be  set  out  more  clearly  and  more  information  for  each  point  when  an 
idea  is  raised. 

+ It  was  interesting  that  there  is  no  right  way  to  go  about  teaching  a particular  subject,  everyone  goes  about 
learning  differently.  I enjoyed  the  many  different  styles  that  were  discussed  (the  frameworks). 

+ the  instructor  seemed  to  deliver  the  information  clearly  and  effectively  providing  relevant  FEEDBACK  in 
order  for  us  (the  students)  to  improve. 

Peer  Factors 

- Initially,  everyone  in  the  class  was  a little  daunted  at  what  to  expected  from  this  science  lesson. 

+ From  this  lesson  I feel  confident  with  raising  issues  and  asking  questions.  Everyone  in  the  class  was  willing  to 
accept  everyone’s  ideas  openly  and  ask  for  any  queries. 

+ The  class  members  continually  gave  positive  ENCOURAGEMENT  to  those  students  who  didn't  understand 
the  work  covered.  Everyone  tried  to  help  everyone  understand  what  the  lesson  was  about  (TEAMWORK). 
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+ I enjoyed  working  in  small  groups  to  achieve  each  activity  everyone  had  their  own  idea  on  what  they  thought 
was  right. 

+ Already  I have  met  new  friends  from  this  lesson  and  together  we  have  worked  together  to  achieve  the  work  set 
out. 

Situational  Factors 

+ This  class  is  not  large  therefore  where  the  instructor  was  teaching  or  standing  was  always  close  by.  There 
were  no  heads  in  the  way,  there  was  enough  material  to  go  round,  the  labs  which  are  well  lit  and  everyone  had  a 
great  attitude  to  work  (I  suppose  this  was  because  this  was  the  first  week). 

+ It  was  great  having  food  and  coffee  provided  for  us  which  also  seemed  to  break  up  the  lesson. 

+ As  the  materials  were  well  set  out  it  was  easy  to  get  straight  into  the  hands-on-activities  and  get  started. 

+ Everyone  in  the  class  seemed  to  get  on  well  with  everyone  (CO-OPERATION)  therefore  it  was  easy  to  reap 
the  rewards  of  each  activity. 

Phase  2 Synthesis.  Towards  the  end  of  the  subject,  students  collated  factors  documented  in  their  weekly 
reflections  according  to  the  same  four  categories  identified  in  phase  1.  After  the  students  completed  their 
collations,  they  used  an  iterative  process  of  “constant  comparative  analysis”  (Glasser  & Strauss,  1 967,  p.vii)  to 
compare,  combine  and  synthesize  factors  resulting  in  the  identification  of  several  key  factors  for  each  of  the  four 
categories.  The  WWW  site  helped  students  to  synthesize  their  reflections  because  it  was  designed  to  collate 
similar  categories  for  all  the  weeks  so  that  students  could  scan  them  to  identify  the  most  influential  factors.  For 
example,  the  data  base  within  the  web  site  linked  the  templates  from  week  to  week  enabling  each  student  to  see 
the  personal  factors  across  all  the  weeks  on  one  screen.  This  aggregation  of  the  reflections  assisted  the  students 
to  identify  the  key  factors  within  each  category  and  these  were  summarized  in  a table  called  a “Learning 
Profile”.  Collectively,  the  key  factors  highlighted  in  their  Learning  Profile  represent  a student’s  identification  of 
factors  which  would  establish  an  optimal  learning  environment  for  them  in  a university  class.  It  should  be  noted 
that  an  optimal  learning  environment  would  only  be  possible  if  all  of  the  enhancing  factors  (or  nearly  all)  were 
present.  The  following  documentation  represents  a student’s  Learning  Profile  which  is  the  synthesis  of  key 
factors  for  each  category  for  the  duration  of  the  subject: 

Personal 

- knowledge,  confidence  and  understanding 

- interested  and  willing  to  get  involved 

- preparation  is  necessary  in  order  to  benefit  from  the  lesson 

- motivation 

- sense  of  achievement 

- prior  knowledge:  you  need  to  do  the  readings 

- "what  you  put  in  is  what  your  get  out!' 

Teaching 

- creating  a relaxed  environment 

- positive  feedback/reinforcement  for  us  (the  students)  to  improve 

- information  given  to  the  students  is  clear  and  concise 

- providing  relevant  examples  to  explain  concepts 

- classroom  management 

- teacher  involvement 
Peer 

- positive  encouragement 

- working  as  a team  (teamwork). 

- group  dynamics 

- group  motivation  when  needed,  eg.  'We  can  do  anything,  come  on  girls!!!' 

Situational 

- all  class  members  working  together  (co-operation) 

- the  relaxed  atmosphere  created  by  the  teacher  and  students 

- materials  safe  and  accessible  (all  materials  are  available  when  needed) 

- the  preparation  of  each  activity  e.g.  layout  and  materials  needed 

- timing  of  the  lesson  e.g.  activities 

Phase  3 Theorising.  Each  preservice  student  considered  the  key  enhancing  factors  identified  in  phase  2 and 
theorized  about  the  relationships  between  them  to  devise  a metaphor  (Lakbff  & Johnson,  1980)  that  represents  an 
optimal  learning  environment  for  a university  class.  Although  the  students  did  not  use  the  web  site  for  designing 
their  metaphor,  it  was  labeled  with  key  factors  identified  in  their  Learning  Profile  from  phase  2.  Nonetheless, 
the  process  of  theorising  was  assisted  by  having  the  reflective  data  presented  systematically  and  collectively  in 
the  templates.  This  thinking  is  consistent  with  (Strauss  & Corbin,  1994)  interpretation  of  a theory  as  '"'‘plausible 
relationships  proposed  among  concepts  and  sets  of  concepts  ” (p.  278,  italics  in  original).  Alternatively,  students 
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may  conceptualise  the  metaphor  earlier  from  their  reflections  in  the  subject  and  then  use  the  factors  from  their 
profile  to  label  the  diagram. 

The  student  whose  reflections  have  been  shown  in  phases  1 and  2 theorized  to  develop  a metaphor  of 
“learning  to  snow  ski”  to  represent  her  optimal  learning  environment.  This  metaphor  was  sketched  and  labeled 
with  personal  factors  such  as  “prior  knowledge,  confidence,  preparation  and  motivation”,  teaching  factors  such 
as  “guidance,  feedback,  clear  and  concise  instruction”  and  peer  factors  such  as  “encouragement,  teamwork  and 
group  motivation”.  She  explained  how  her  metaphor  of  a learning  to  ski  represented  an  optimal  learning 
environment  and  drew  implications  for  her  classroom  teaching: 

There  are  many  steps  involved  before  turning  into  a proficient  skier  similar  to  the  fact  that  it  takes  time 
and  practice  before  you  are  a capable  student  in  any  area  of  learning.  It  is  important  that  the  student  sets 
a goal  on  what  they  want  to  achieve,  however,  there  are  decisions  to  be  made  on  the  plan  of  attack.  The 
ski  shop  is  an  integral  part  in  the  learning  process  as  preparation,  organization  and  understanding  are 
necessary  to  commence  learning.  Comparing  this  to  a classroom  situation  is  being  prepared  with 
homework,  equipment  and  prior  knowledge  so  that  you  are  not  disadvantaged  to  the  rest  of  the  class. 
The  most  important  factor  is  the  instructor/teacher  who  provides  correct  information,  positive 
reinforcement/feedback,  guidance  and  instruction.  In  most  classroom  settings  the  teacher  is  the  main 
focus  for  students  in  preparing,  organizing  and  implementing  various  lessons. 

Generating  a metaphor  is  like  producing  a mental  model  of  teaching  and  learning  and  other  students  in  the  class 
generated  a range  of  metaphors  such  as  playing  an  instrument  in  an  orchestra,  learning  to  be  an  acrobat  in  a 
circus,  playing  in  a one-day  cricket  match,  going  on  a mountain  hike,  being  in  a rocket  blasting  off  into  space, 
playing  in  a fun  park  and  playing  a game  of  baseball.  In  addition,  students  are  expected  to  use  these  insights  into 
an  optimal  learning  environment  to  draw  implications  for  their  future  role  as  teachers. 


Discussion  and  Conclusion 

In  any  one  year,  teacher  education  students  attend  over  300  hours  of  formal  class  time  at  university  and 
are  exposed  to  a wide  range  of  experiences  across  different  subjects.  Yet  little  opportunity  is  provided  for 
students  to  reflect  upon  these  authentic  experiences  to  analyze  how  they  learn  in  different  types  of  learning 
environments.  In  particular,  trainee  teachers  often  do  not  get  the  opportunity  to  study  their  own  experiences  as 
students  in  class  to  gain  a first  hand  understanding  of  the  relationship  between  teaching  and  learning.  This  is  a 
valuable  insight  for  trainee  teachers  as  designing  an  optimal  learning  environment,  which  is  the  main  outcome  of 
the  reflective  framework,  takes  into  account  the  type  of  teaching  and  the  type  of  learning  and  the  type  of  social 
interaction  with  peers.  Developing  such  an  awareness  of  the  dynamics  in  any  educational  setting  can  support 
students  in  participating  in  reflective  conversations  about  teaching  and  learning  which  is  a useful  skill  for  their 
future  role  as  teachers. 

Furthermore,  it  is  important  that  trainee  teachers  are  not  simply  passive  recipients  of  formal  theory  at 
university,  but  engage  in  theorising  about  their  own  experiences.  This  means  systematically  using  a framework 
to  reflect  on  experiences  by  documenting,  analysing  and  seeking  patterns  within  the  data  for  generating  personal 
theories.  This  means  that  preservice  students  can  deduce  implications  for  their  own  pedagogy  based  on  the 
analysis  of  their  own  experiences  of  learning.  Although  students  previously  used  the  reflective  framework  with 
pen  and  paper  (Hoban,  1997,  1998),  this  is  the  first  time  that  students  have  documented  their  reflections  on  a 
World  Wide  Web  site.  Previously,  students  had  to  write  their  reflections  each  week  in  a journal  and  then  scan 
manually  across  many  pages  to  synthesise  the  key  factors  for  each  category.  Also  the  students  had  to  seek 
patterns  within  the  data  and  deduce  their  metaphor  to  represent  an  optimal  learning  environment.  Doing  this 
manually  was  time  consuming  and  conceptually  difficult  for  some  students. 

Using  the  WWW  site  has  enabled  the  students  to  manage  and  theorize  about  their  reflections  more 
effectively  which  has  reduced  their  cognitive  load  in  using  the  framework.  In  an  end  of  subject  evaluation,  some 
of  the  students  made  comments  about  using  the  web  for  their  reflections: 

The  idea  of  using  the  web  was  good,  it  was  convenient  for  everyone  and  everything  was  in  front  of  you 
in  the  one  place  which  made  it  easier  to  collate  etc.  I would  prefer  to  use  the  web  but  have  the  option  of 
doing  it  in  my  own  time. 

I preferred  to  use  the  web  because  at  the  end  it  was  easier  to  bring  everything  together.  I also  found  the 
reflections  were  more  to  the  point  and  there  wasn’t  as  much  rambling  on.  The  only  negative  was  the 
inconvenience  of  getting  onto  the  net  whenever  we  needed  to  reflect  and  also  when  the  web  page  was 
down. 

It  was  better  using  the  web.  It  was  a great  way  to  group  the  information  into  the  different  factors  and  it 
was  easier  to  see  a clear  pattern  of  learning. 

Two  issues,  however,  became  evident  when  preservice  teachers  used  the  web  site  for  the  reflective  framework. 
First,  initial  use  of  the  framework  was  conceptually  difficult  for  preservice  students  as  they  had  not  reflected 
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upon  different  learning  influences  before  and  were  unsure  about  what  to  look  for  and  how  to  do  it.  For  this 
reason,  several  reflective  journals  produced  by  students  in  previous  subjects  were  shown  at  the  beginning  of  the 
subject  as  examples  of  how  to  analyze  their  learning  experiences.  Furthermore,  this  needed  to  be  revisited 
several  times  in  the  first  few  weeks  with  discussions  about  what  the  students  were  documenting.  Also,  towards 
the  end  of  the  course  several  students  commented  that  this  was  the  first  course  which  gave  them  a specific 
framework  to  guide  them  in  reflecting  on  their  class  experiences.  Previously  they  had  undertaken  reflective 
journals  in  other  subjects,  but  were  not  provided  with  a framework  and  students  stated  that  in  many  cases  they 
just  wrote  about  “what  the  lecturer  wanted  to  hear*’.  It  appears  from  the  students’  data  that  the  web  site  made 
the  reflective  framework  easier  to  use  because  it  focused  students  in  their  reflections  and  helped  them  to  manage 
the  data  for  phases  2 and  3 of  the  framework. 

A second  issue  is  that  I am  the  only  person  who  saw  all  of  the  students’  reflections  on  teaching  and 
learning  for  the  subject  as  I marked  their  journals.  Certainly,  the  WWW  site  made  it  easier  to  mark  the  students’ 
journals  as  all  the  data  were  more  accessible  on  the  web  site  rather  than  screening  large  hand  written  journals. 
Also,  it  was  a valuable  insight  for  me  to  understand  how  different  students  responded  in  different  ways  to  the 
same  learning  experience.  In  some  weeks,  the  same  science  activities  were  praised  by  some  students  and 
criticized  by  others.  This  highlights  the  problematic  nature  of  teaching  and  it  would  be  valuable  for  the 
preservice  students  to  become  more  aware  that  people  experience  learning  in  different  ways.  Although  students 
could  access  a summary  of  each  student’s  weekly  reflections  on  the  web,  they  could  not  access  the  Learning 
Profiles  or  metaphors  produced  by  other  students.  Perhaps  the  web  site  needs  to  be  modified  to  enable  more 
sharing  of  products  of  students’  reflections,  however,  there  are  ethical  issues  that  need  to  be  considered  as  the 
students  are  documenting  personal  experiences.  Nonetheless,  it  is  a valuable  lesson  for  any  teacher  to  realize 
that  students  do  not  perceive  the  same  class  experience  in  the  same  way. 

Note 

The  web  site  discussed  in  this  paper  is  available  at  http://www.edonline.uow.edu.au/edus224 
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Abstract:  A medical  information  system  at  Graz  University  contains  three  millions  of  diagnostic 
reports,  which  form  the  basis  of  patient  care  and  of  scientific  work.  Complex  retrieval  systems  are 
prepared  in  a time  consuming  dialogue  between  clinical  researchers  and  IS-experts.  In  this  paper 
we  describe  the  development  and  questions  of  evaluation  of  an  intranet  Web-based  query/answer 
system  with  reference  to  aspects  of  human-computer  interaction.  The  system  formulates  a well 
structured  and,  ideally,  machine  interpretable  retrieval  request  in  interaction  with  the  user  (i.  e. 
medical  professionals,  bio-statistical  researchers,  ...)  and  stores  it  in  an  existing  request- 
management-application.  The  user  learns,  incidentally,  how  to  use  the  hospital  information  system 
for  his  needs.  For  optimal  interpretation  of  the  gathered  information  exact  formulation  of  the 
questions  is  essential.  An  intelligent  guidance  during  the  dialogue  is  obligatory  since  we  cannot 
assume  any  technical  skill  on  the  part  of  the  medical  users.  In  the  near  future  this  service  will  be 
extended  to  the  countrywide  hospital  information  system.  By  using  this  system,  the  quality  of  the 
retrieval  and  therefore  the  quality  of  scientific  research  and  medical  studies  is  raised.  Less  iterations 
mean  a lower  system-workload.  Last  but  not  least,  we  expect  to  lower  the  response  time  without 
increasing  the  need  of  human  resources. 

"When  we  write  programs  that  "learn",  it  turns  out  that  we  do  and  they  don't.  ” 

(Alan  J.  Perils,  Yale  University) 


1.  Introduction 

The  information  systems  at  the  departments  of  radiology  and  pathology  at  the  2.300  bed  University  Hospital  in  Graz 
support  activities  in  patient  care  and  serve  as  a basis  for  scientific  research,  not  only  for  radiology  and  pathology  but 
also  for  all  other  clinical  departments  which  refer  to  these  systems  in  connection  with  their  o\vn  patient  data.  Since 
the  data  are  partly  in  standardized  form  (codes  for  examination  types,  organizational  entities)  and  partly  in  natural 
language,  scientific  retrievals  require  complex  strategies  to  yield  optimal  results.  As  will  be  shown  below,  the  scope 
of  the  retrieval  request  is  defined  in  an  interactive  discussion  between  a clinical  researcher  and  an  IS-expert.  This 
procedure  will  be  replaced  as  far  as  possible  by  the  proposed  system.  The  system  will  primarily  be  used  by  medical 
doctors  or  assistants.  To  improve  the  quality  of  data  preparation  it  is  necessary  to  provide  precisely  formulated 
questions.  Since  the  typical  user  does  not  have  detailed  technical  knowledge,  intelligent  guidance  is  required  during 
the  dialogue. 

The  user  starts  the  dialogue  with  the  System  by  stating  his  question  in  medical  terminology  for  administrative  and 
future  purpose  only.  During  the  subsequent  dialog,  unconsciously  he  will  adopt  the  dialogue’s  behavior  in 
formulating  further  questions.  Succeeding  dialogues  must  always  be  based  on  information  already  ascertained,  thus 
avoiding  ‘silly  questions’.  Following  the  indicated  evaluation  of  criteria  and  features,  only  useful  selections  of  data- 
reports  should  be  offered.  In  this  project  the  main  emphasis  is  on  ‘the  dialogue’.  The  dialogue  will  help  the  user 
understand  and  benefit  from  the  functional  possibilities  of  the  information  system.  Furthermore  it  will  help  them  to 
accept  limitations  and  guide  them  to  provide  structured  information.  The  high  level  of  the  system’s  operating 
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comfort  aims  at  persuading  the  user  not  to  make  complicated  and  time  consuming  telephone  discussions  or  use 
written  inquiries  to  collect  required  information.  This  system  provides  a remarkable  potential  of  savings  in 
administrative  expenditures  by  means  of  good  integration  into  existing  administration  facilities  and  adding  to  the 
quality  of  data  evaluation. 

The  system  itself  has  to  adapt  to  the  user’s  formulation  of  a question  not  only  per  session  but  also  in  the  long  term, 
thus  adapting  to  the  importance  and  priority  of  knowledge  objects  in  general. 

2.  Background 

The  Information-System  used  for  research  and  patient  care  was  originally  developed  by  the  Institute  of  Medical 
Informatics,  Statistics  and  Documentation  (IMI)  at  Graz  University  for  use  in  Radiology,  Pathology,  Neurosurgery 
and  Pediatrics,  and  has  been  steadily  refined  ever  since  the  early  seventies.  The  IS  Data  Base  contains  approximately 
three  million  (!)  medical  documents,  which  have  been  gathered  since  1971.  Patient  information,  technical 
parameters  and  performance  data  are  saved  in  a thoroughly  structured  form,  anamnese,  examination  descriptions  and 
diagnoses  are  available  in  free-text.  Besides  hundreds  of  simple  routine  retrievals  for  patient  care,  there  are,  on 
average,  about  one  or  two  highly  complex  retrieval  requests  per  day  for  scientific  purposes,  which  requires  the 
knowledge  of  an  IS  expert.  Quite  often  the  huge  possibilities  and  potentialities  of  filtering,  structuring  and 
representing  are  not  familiar  or  even  known  to  the  medical  researcher.  High  competence  by  the  IS  expert  in  medical 
and  technological  fields  is  needed  to  assess  the  real  scope  of  the  information  required  by  the  clinical  researcher  for 
his  work.  The  formulation  of  the  retrieval  needs  are  elaborated  in  a personal  discussion  between  the  clinician  and  the 
IS  expert,  a time  consuming  process,  requiring  patience  and  perseverance.  Due  to  the  personnel  shortage  and  the 
increasing  demands  based  on  increasing  standards  in  quality  management,  output  documentation,  health  reports,  etc. 
we  consider  an  automation  of  this  process  is  necessary  to  enable  continued  functioning  in  the  future.  The  following 
excerpt  of  a conversation  between  an  IS-expert  and  a client  shows  that  several  areas  of  knowledge  are  necessaiy  to 
lead  an  intelligent  dialogue: 

Client:  I would  like  to  have  all  reports  of  the  angiography  with  interventional  cases  from  1998  up  to  the  present. 
IS-Expert:  You  mean  all  types  of  examinations  concerning  the  angiography  ? 

Client:  Yes,  because  the  examinations  are  all  entered  with  different  coding  abbreviations. 

IS-Expert:  That  is  clear  to  me.  How  can  I identify  interventions  ? What  does  the  examination  report  say? 

Client:  ...  well,  the  criteria  of  the  contexts  are  ...  (speaks  slowly  and  pause  for  a moment) ...  A.carotis,  A.vertebralis, 
A.basilaris,  embolisation,  neuroembolisation,  stent,  baloon  dilatation,  PTA, ... 

IS-Expert:  ...  hmm,  that  appears  to  be  a lot  to  me  ...  if  I ...  for  instance  only  look  for  A.carotis  or  A.vertebralis  ...  then 
I will  find  a vast  amount  of  data-sets  ...  should  these  criteria  really  all  combined  by  ‘OR’  ? 

Client:  ...  no,  no  ...  I only  mean  neuroembolisation  OR  embolisation  etc.  with  A.carotis  OR  A.vertebralis, ... 
IS-Expert:  ...  ah,  I see,  ...  well  that  is  O.K.,  ...  I am  looking  for  interventions  in  the  examination  reports  such  as 
embolistion,  neuroembolisation,  stent,  balloon  dilatation  and  PTA  and  within  the  diagnosis  for  A.carotis  OR  A. 
vertebralis,  OR  A.  basilaris 
Client:  yes  exactly  ... 

IS-Expert:  ...  would  you  like  to  read  all  the  documents  of  the  interventions  ...  or  are  you  primarily  interested  in  the 
patience  convalescence  ... 

Client:  No,  we  are  looking  for  eventual  complications  which  might  have  occured  later ... 

IS-Expert:  Ah,  yes  ... 

It  would  be  extremely  difficult  and  exhaustive  to  put  the  whole  special  medical  knowledge  into  the  system.  For 
instance  the  fact,  that  a balloon  dilatation  is  already  an  intervention,  on  the  other  hand  A.  basilaris  is  an  organ,  where 
the  intervention  actually  has  been  made.  Also  the  knowledge  about  the  structure  of  the  IS  (types  of  examinations  are 
divided  into  coding  abbreviations,  these  depict  the  radiological  technology,  but  not  an  interventional  operation)  and 
meta-knowledge  would  be  extremely  difficult  to  put  into  the  system.  With  the  strategy  mentioned  above,  one  can 
find  the  initial  intervention,  but  to  find  more  about  complications  requires  other  strategies. 

These  problems  are  solved  by  using  a combinatorial  approach.  Every  fact,  which  can  be  appear  as  a filter,  a 
classification  item  or  as  representation  feature  in  the  resulting  request,  is  represented  as  an  object.  One  method, 
implemented  in  this  objects  enables  the  dialogue  with  the  user  (e.  g.  a question  about  an  examination  type  within  a 
meaningful  subset  represented  as  HTML  Structure).  The  reactions  which  alter  the  factual  knowledge  basis  of  a 
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particular  session  and  that  modifies  the  persistent  user  depended  state  of  learning  are  important  methods  which  are 
also  implemented  in  these  objects.  By  extending  a good  Meta-Data-Strategy  with  actions  that  represent  semantic 
rules  in  medical  context,  we  implement  a small,  dedicated  part  of  global  behavior.  These  actions  are  'triggering 
questions'  and  'generating  answer-like  facts'. 

3,  The  system  assembled  as  an  Intelligent  Tutoring  System 

3. L  What  is  an  ITS  ? 

Intelligent  Tutoring  Systems  (ITS)  are  remarkable  with  regard  to  their  methods  and  the  fundamental  theories  used. 
Their  origin  comes  from  the  field  of  Computer  Assisted  Learning.  Generally  the  “Intelligence”  in  ITS  is  traced  back 
to  Carbonell  in  1970.  Carbonells  prototyp  SCHOLAR,  which  was  actually  an  interactive  program  for  computer- 
aided  instruction  based  on  semantic  networks  as  the  representation  of  knowledge,  was  primarily  designed  for 
learning  geography.  He  implemented  a socratic  dialogue  in  an  artificial  tutor. 

ITS  generally  provides  a high  level  of  guidance  and  control  interactive  processes  in  great  detail.  Possible  navigation 
decisions  by  the  users  are  controlled  by  the  system.  There  is  no  clear  border  between  adaptive  systems  and  those 
generally  called  ITS  (cf.  Sleeman  & Brown  (1982)). 

Recently,  ideas  from  both  intelligent  tutoring  systems  and  from  hypermedia  have  been  brought  together.  This  has 
lead  to  interactive  and  adaptive  hypermedia,  which  we  use  in  our  system.  This  synthesis  responds  to  the  specific 
strengths  and  weaknesses  of  both  approaches.  A domain  model  representing  all  facts  of  the  field  to  be  learnt  usually 
forms  the  background  for  a model  of  the  learner's  knowledge  and  knowledge  acquisition. 

3.2  The  problematic  of  intelligence 

Constant  misuse  and  misinterpretation  of  the  term  “Intelligence”  and  in  particular  “Artificial  Intelligence”  has 
made  it  increasingly  undesirable  to  label  systems  with  this  designation. 

It  is  generally  accepted  to  refer  to  an  “Intelligent  Tutoring  System”  if  the  system  is  able  to 

a)  build  a more  or  less  sophisticated  model  of  cognitive  processes, 

b)  adapt  these  processes  consecutively  and 

c)  is  based  on  these  fundamentals  to  control  an  question-answer-interaction. 

The  basics  of  Intelligent  Tutoring  Systems  are  most  suitable  for  our  special  purpose. 

3.3  Interface  and  Interaction 

Initially,  an  ITS  should  be  able  to  analyze  the  current  process  of  knowledge  acquisition.  Based  on  this  information 
the  ITS  should  be  able  to  build  instructions  for  the  user.  An  ITS  is  generally  considered  to  be  “intelligent”  if  it  is 
able  to  react  to  the  process  of  communication  in  a flexible  and  adaptive  manner.  Kearsley  (1987)  distinguishes 
basically  between  five  different  types  of  interfaces  for  IT-Systems:  1)  socratic  dialogue,  2)  coaching,  3)  debugging, 
4)  microworlds,  5)  explainable  expert-systems.  In  our  system  we  use  the  principles  of  the  “socratic  dialogue”: 

The  system  questions  the  user  and,  on  the  basis  of  the  questions  and  answers,  guides  the  user  through  a controlled 
interaction.  A specific  answer  of  the  user  is  a starting  point  for  the  next  question.  The  interaction  happens  close  to  a 
natural-language  dialogue  (cf  Kearsley  (1993)). 

4.  The  System,  its  features  and  its  implementation 

4.1.  Prerequisites 

Concerning  the  implementation  of  this  system,  there  were  several  preconditions  to  be  taken  into  consideration. 
Firstly,  the  system  is  supposed  to  work  in  cooperation  with  the  Meta-Data  strategy  of  existing  department 
information  systems  that  currently  work  on  proprietary  databases.  As  mentioned  the  information  systems  are  about 
to  be  replaced  with  a countrywide  information  system  within  the  next  year.  That  means  that  our  solution  needs  to 
provide  an  interface  that  can  be  easily  extended  or  replaced. 
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Secondly,  we  had  to  keep  in  mind  that  we  needed  to  find  a way  of  implementing  the  system  so  that  the  result  would 
provide  as  open  an  architecture  as  possible  with  regard  to  interfaces  and  interoperability  with  other  systems  and 
platforms. 

Thirdly,  the  system  should  also  be  capable  of  easy  expansion,  e.g.  additional  functionality  in  future  at  very  low 
expense. 

Fourthly,  in  the  event  of  migration  to  other  hardware  and/or  software  platforms  in  future,  it  should  be  easily  possible 
to  port  the  system  without  endangering  the  stability  and/or  functionality.  Only  minor  changes  or  adaptations  should 
be  necessary  by  that  time.  This  means  that  portability  also  had  to  be  taken  as  a prerequisite. 

4.2.  Implementation  Framework:  The  Java  2 Enterprise  Edition 

We  ran  across  Java  and  the  Java  2 Enterprise  Edition  (J2EE)  pretty  quickly.  The  J2EE  provides  all  the  functionality 
and  tools  that  fulfill  the  preconditions  of  the  project  listed  above  and  impresses  the  developer  with  its  "write  once  - 
run  anywhere"  functionality.  In  addition,  the  Java  Enterprise  Components  give  us  the  advantage  of  minimum 
possible  implementation  time  and  low  costs  combined  with  a maximum  outcome.  Networking,  multithreading, 
security  and  even  database  connectivity  are  available  via  standard  APIs  and  do  not  have  to  be  written  again  from 
scratch.  Since  the  release  of  Java  in  1996,  a very  large  community  of  Java  developers  (Java  Developer  Connection, 
about  1 Mio.  members)  has  evolved,  who  provide  a wide  variety  of  Servlets  and  Beans  (reusable  components) 
almost  for  free  which  can  easily  be  integrated  in  any  project  without  having  to  “reinvent  the  wheel”. 

When  considering  an  open  system  that  is  expected  to  provide  as  much  interoperability  and  expandability  as  possible, 
the  choice  for  a Multi-Tier  architecture  approach  is  obvious.  Using  a Multi-Tier  architecture  also  ensures  that  the 
whole  system  is  built  to  work  in  a distributed  scalable  environment,  which  was  another  prerequisite  of  the  project. 

4.3.  Multi-Tier  Architecture 

Taking  into  consideration  a Multi-Tiered  architecture  relating  to  the  general  condition  explained  before,  we  expect 
the  following  framework  to  meet  our  needs. 

Presentation  Layer 

The  user  communicates  with  the  system  through  an  HTML  - Browser  that  is  driven  by  a Thin  - Client  Servlet.  The 
dialogue  phases  should  be  fast  and  simple.  In  addition  to  the  claim  that  the  dialogue  should  usually  be  similar  to  a 
natural  language  dialogue,  (Spada  and  Opwis  (1985)),  the  possibilities  of  graphical  user  interfaces  should  improve 
the  performance  of  the  human  computer  interaction. 

Business  Layer 

Here  we  implement  all  application  specific  components  and  the  session  management  using  state- full  Session  Beans. 
It  is  necessary  to  convert  the  stateless  http-communication  into  a persistent,  re-activateable  dialogue  by  means  of  the 
user  context.  So,  if  the  user  looses  the  dialogue  with  the  system  for  what  ever  reason,  next  time  he  authenticates 
himself  to  the  system,  he  has  the  opportunity  to  continue,  without  loss  of  previous  dialogue. 

Technical  Layer 

The  association  of  the  fact-,  semantic-  and  knowledge  objects  builds  the  didactic  components  in  this  layer.  The 
dependency  of  classes  from  the  database  layer  and  the  over  all  didactic  strategy  builds  the  classes. 

Database  Layer 

Here  we  find  the  object  representation  of  the  Meta-Data  in  the  Data-Dictionaries  and  their  semantic  meanings,  as 
well  as  the  user  specific  model,  his  habits,  behavior,  last  session  status  and  previous  accesses.  Also  the  knowledge 
model  is  represented  as  a class  of  this  layer. 

With  respect  to  the  manageability  of  all  growing  information  components,  all  permanent  data  will  be  stored  on 
database  servers.  Flexibility  is  acquired  by  means  of  using  only  JDBC-interfaces  to  the  servers. 
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5.  Research 


A hospital  information  system  provider  looks  ahead,  hoping  to  gain  experience  in  covering  the  complexity  of  huge 
information  archives  under  retrieval  aspects.  For  the  cognitive  scientist,  the  greatest  advantage  of  that  approach  will 
consist  of  the  feedback,  that  will  be  provided  by  the  quality  improvement  for  retrieval  requests.  During  our  talk  at 
ED‘MEDIA  2000  we  will  present  some  answers  to  questions  asked  in  the  following  paragraphs. 

5.1.  Aspects  of  Informatics 

5.1.1  Meta  Data  Dictionary  (for  details  see  h ttv://wmv-an2>  kfun israz. ac. athholzinse/its/m dd) 

The  main  goals  during  design  of  the  Meta  Data  Dictionary  were  simplicity  and  extendibility,  but  focusing  on  an 
intelligent  way  of  storing  knowledge  (actual  legacy  database  and  future  SAP  R/3).  How  can  the  IS-Meta-Knowledge 
be  stored  and  accessed  in  a intelligent  way  to  get  an  optimum  out?  What  has  to  be  prepared  to  be  able  to  change  the 
database  engine  itself  whenever  the  MDD  has  to  be  moved  to  another  platform  (what  about  XML)? 

5.1.2  Distributed  System  Design  (for  details  see  http://w\\fw-an2.  kf'mtisraz.  ac.  athholzinse/its/dsd) 

Thinking  about  a distributed  application  framework,  it  is  obvious  that  one  has  to  be  prepared  for  different  platforms 
and  will  result  in  a multi-tier  architecture.  The  main  questions  in  this  context  are:  How  can  the  system  be  kept 
portable  for  future  platforms  (JAVA  Enterprise  Edition)?  What  will  happen  when  one  component  in  the  network 
cannot  be  reached  and  will  not  deliver  an  answer?  What  can  be  done  to  ensure  good  performance  of  the  system  so 
that  the  user  will  be  satisfied? 

5.1.3  Intelligent  Engine  (for  details  see  http kfuni^raz. ac. atMiolzinse/its/i te) 

How  should  the  engine  decide  in  case  of  equal  weighted  questions  (aspects  of  fuzzy  logic)?  How  can  the  user  profile 
be  integrated  as  a decision  criteria  (adaptive  behavior)? 

5.2.  Aspects  of  Human-Computer-Interaction 

The  knowledge  representation,  as  a framework  of  objects  that  cover  facts,  semantic  information  and  parts  of 
common  knowledge  is  especially  important,  as  are  the  methods  used  to  keep  the  users’  interest,  motivation  and 
attention  on  the  actions  to  keep  the  dialogue  process  alive.  As  an  experimental  design  we  have  chosen  a pre- 
test/post-test  control -group  design,  assisted  by  qualitative  analysis  via  interviews.  The  experimental  sample  to  be 
examined  includes  24  people:  12  experts  (including  MD's)  and  12  novices  (students  of  medicine).  The  research 
questions  are  divided  into  Dialogue,  Usability  and  Learning. 

5.2.1  Questions  concerning  the  evaluation  of  the  System  (Dialogue) 

How  do  users  (experts  versus  novices)  handle  the  machine-dialogue,  in  contrast  to  the  human-dialogue  provided  by 
an  IS-expert  ? How  extensively  do  the  users  proceed  within  the  dialogue  ? At  which  time  do  they  quit  the  dialogue  ? 
Why  do  the  users  break  off  the  dialogue  ? How  often  do  we  get  useless  or  irrelevant  information  ? 

5.2.2  Questions  concerning  the  evaluation  of  the  Graphical  User  Interface  (Usability) 

How  does  the  GUI  influence  the  user  behavior  ? What  is  the  difference  in  handling  between  experts  and  novices  ? 
What  screen  contents  and  hints  are  useful  for  novices  and  experts  ? 

5.2.3  Questions  concerning  the  evaluation  of  the  Intelligent  Behaviour  (Learning) 

How  big  is  the  learning  achievement  by  using  this  system  ? How  does  incidental  learning  (cf  Holzinger  & Maurer 
1999)  increase  the  efforts  of  the  users  ? How  do  motivation,  attention  and  arousal  directly  influence  the  necessary 
actions  to  keep  the  dialogue  process  alive  ? 
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6.  Practical  Aspects 


Eventually,  when  the  IS  is  extended  to  the  countrywide  hospital  information  system,  it  will  no  longer  be  possible  to 
include  the  whole  system  information  in  one  human  to  human  consultation  call.  The  findings  and  experiences  of  this 
project  will  lead  to  a suitable  knowledge  representation,  which  provides  an  interactive  dialogue  suitable  for  a more 
complex  system.  Finally  the  quality  of  requests  for  scientific  medical  research  can  be  increased  by  reducing  the 
response-time  and  lowering  the  system  work  load  through  minimizing  iteration  cycles. 
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Abstract:  During  the  period  from  September  1st  1998  to  December  31st  1999  the  first  and  second 
authors  jointly  managed  a project  funded  under  European  Social  Funding  objective  2,5b  and  the 
Regional  Technology  Initiative  entitled  "Multimedia  Training  Materials  for  the  Community 
University  of  the  Valleys".  The  aim  of  the  project  was  to  develop  learning  materials,  in  a variety  of 
subject  areas,  using  a variety  of  media,  for  use  in  the  communities  of  the  South  Wales  Valleys.  A 
variety  of  materials  were  produced  and  piloted  by  Community  Tutors.  The  most  notable  successes 
being  (1)  an  interactive  multimedia  package  to  teach  Multimedia  Principles  delivered  on  CD-Rom 
and  (2)  a paper-based  Career  Planning  package.  A more  general  achievement  is  a wider  awareness 
of  the  need  for  training  in  the  use  of  Information  and  Communication  Technology  (ICT)  in  the 
South  Wales  Valleys. 


Introduction 

The  "Multimedia  Training  Materials  for  the  Community  University  of  the  Valleys"  project  involved  the  creation, 
delivery  and  evaluation  of  multimedia  education  and  training  materials  in  the  areas  of  Multimedia,  the  European 
Computer  Driving  License  and  Employability  Skills.  The  project  is  funded  under  European  Social  Funding  (ESF) 
objective  2,5b  and  the  Regional  Technology  Initiative  from  1st  September  1998  to  31st  December  1999.  The  paper 
begins  with  an  outline  description  of  the  CUV  and  a variety  of  its  initiatives  and  a description  of  the  context  in 
which  this  particular  project  took  place.  It  then  describes  the  nature  and  structure  of  the  materials  development 
project  and  concludes  with  a discussion  of  the  key  lessons  learned  and  of  the  ways  in  which  the  initiative  could  be 
sustained. 


The  Community  University  of  the  Valleys 

Context 

Wales  is  a country,  part  of  the  United  Kingdom,  lying  West  of  England.  The  communities  of  the  South  Wales 
Valleys  lie  north  of  the  more  affluent  cities  in  South  Wales,  i.e.  Newport,  Cardiff,  and  Swansea.  The  area  is 
relatively  deprived  compared  with  other  areas  in  the  UK,  having  suffered  economically  from  the  loss  of  much  heavy 
industry,  particularly  coal  mining.  The  Community  University  of  the  Valleys  (CUV)  is  a unique  partnership  between 
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the  University  of  Glamorgan,  the  Swansea  University  and  the  Open  University  in  Wales  representing  part  of  the 
regeneration  strategy  for  valley  communities  in  Industrial  South  Wales.  It  aims  to  provide  for  lifelong  learning  and 
widening  participation  in  Higher  Education  (HE)  through  an  active  partnership  of  public,  community,  voluntary  and 
private  sectors.  At  Glamorgan  the  CUV  involves  a series  of  projects  and  courses,  ranging  from  state  funded  delivery 
through  to  European  and  projects  funded  by  Private  industry.  Approximately  2000  learners  within  the  community 
are  involved  each  year  in  various  educational  programmes  delivered  in  the  community  by  CUV  tutors.  The  CUV 
project  has  been  recognised  internationally,  by  the  European  Union,  and  nationally  in  the  Bearing  report  (NCIHE, 
1997). 

The  CUV  operates  in  a variety  of  environments  but  the  bastions  of  the  community  provision  are  (i)  the  workplace, 
(ii)  School  and  (iii)  local  Community  Centres.  Some  examples  of  CUV  provision  are  as  follows: 

Continuing  and  Adult  Education  (CAE) 

Since  1994/95,  many  community  centres  have  been  used  for  the  delivery  of  modular  and  non-accredited  courses. 
The  campus  has  also  been  used.  The  CUV  team  in  partnership  with  FE  providers  is  now  addressing  the  need  to 
define  and  market  progression  pathways  into  HE.  Curriculum  areas  cover  languages,  arts,  business,  study  skills, 
technology,  social  science  and  astronomy. 

Glamorgan  Summer  School 

The  annual  residential  summer  schools,  (held  at  the  Glamorgan  Campus)  attract  between  400-600  learners.  This  is  a 
stimulating  and  high  profile  activity  offering  valued  educational  opportunities  to  low  income  learners  (through 
subsidy  from  the  Rhondda  Cynon  Taff  unitary  authority)  as  well  as  more  affluent  students.  The  Summer  School  hcis 
gained  a national  reputation  for  its  jazz  courses. 


Compacts 


Compacts  (developmental  partnerships)  are  operating  with  schools  in  a number  of  Local  Education  Authorities  and 
involve: 

• student  tutoring, 

• records  of  achievement  initiatives, 

• aiming  for  a college  education  days, 

• ‘A’  level  master  classes  and 

• GCSE  revision  sessions. 


In  1998  the  Compact  initiative  was  selected  by  the  Committee  of  Vice  Chancellors  & Principals  (CVCP)  as  a 
national  case  study  for  good  practice  in  widening  access  to  higher  education. 


Internet  Simulation 


A project  funded  through  the  European  Union  'Socrates'  programme,  established  an  internet-based  journalism 
activity  for  adult  learners  within  the  Rhondda  Cynon  Taff  area.  The  learners  developed  IT  expertise  as  well  as 
awareness  of  media  issues  and  news  values  based  on  a newspaper  editing  project.  This  also  involved  students  in 
Finland,  Scotland  and  France.  The  project  led  to  a more  extensive  electronic  newspaper  simulation  involving  local 
schools,  thanks  to  sponsorship  via  the  British  Telecomm  University  Development  Award  for  1998. 

Partnership  Framework 

A University  strategy  on  employability  has  been  agreed  by  Academic  Board,  leading  to  the  establishment  of  a 
Work-Based  Learning  Office  that  develops  links  with  work  based  learners.  The  employability  initiative  recognises 
the  importance  of  accrediting  prior  experience  and  distance  learning,  as  well  as  projects  within  the  workplace,  and 
offers  educational  opportunities  to  employees  who  would  not  otherwise  be  able  to  attend  traditionally  delivered 
programmes  on  campus  or  in  the  community. 
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It  is  against  this  background,  of  an  urgent  need  for  the  development  of  ICT  (Information  and  Communications 
Technology)  Skills,  that  this  project  was  conceived  and  was  successful  in  obtaining  funding  from  the  European 
Union, 


Multimedia  Training  Materials  for  the  CUV 

Project  Rationale 

Nationally  and  internationally  patterns  of  employment  are  changing,  characterized  by 

(a)  a shift  away  from  labour  intensive  heavy  industry  to  ‘information  age’  organisations, 

(b)  a shift  away  from  the  concept  of  a job  for  life  towards  more  short  contract  and  part  time  work,  and 

(c)  increasing  numbers  of  female  employees. 

These  changes  are  clearly  applicable  to  the  South  Wales  Valleys  labour  market.  In  this  context  the  UK  Government 
and  the  Welsh  Assembly  have  recognised  an  urgent  need  to  develop  Information  and  Communications  Technology 
(ICT)  skills  amongst  the  existing  and  potential  work  force. 

Three  recent  publications  evidence  the  concerns  of  Government. 

1 . The  Wales  Information  Society  report  produced  by  the  Welsh  Development  Agency  from  which  the 
following  quotation  is  taken. 

**If  Wales  is  to  sustain  a competitive  position  within  the  increasingly  information-based  European  and 
global  economy,  it  must  move  towards  a knowledge-based  economy.  The  new  iCTs  provide  the  tools  to 
enable  this  to  happen.  ” (Welsh  Development  Agency,  1 998) 

2.  The  Lifelong  Learning  is  for  Everyone  (LIFE)  green  paper  presented  to  Parliament  in  1998.  The  paper 
provides  evidence  that  supports  the  following  claims  for  Wales: 

• there  is  a serious  learning  deficit; 

• educational  achievements  are  lower; 

• there  is  a greater  shortfall  in  basic  numeracy  and  literacy  skills; 

• barriers  to  learning  stifle  access  and  participation; 

• education  and  training  are  not  always  appropriate  to  the  needs  of  the  individual  or  the  economy; 

• existing  patterns  of  provision  are  not  achieving  the  best  results; 

• the  nature  of  the  qualifications  framework  is  inflexible;  and 

• there  are  persistent  deficiencies  of  skill  and  capability  in  all  walks  of  life  and  throughout  Wales. 
(Secretary  of  State  for  Wales,  1998) 

3.  The  Future  Skills  Wales  report,  commissioned  by  the  largest  partnership  of  organisations  assembled  in 
Wales  including  the  Careers  Service,  CBI  Wales,  FEFCW  and  HEFCW,  Wales  TUC,  WDA  and  the  Welsh 
Office.  The  report  indicates  that  in  the  future  the  following  skills  will  be  most  important. 

• Basic  IT 

• Understanding  Customers  needs 

• Ability  to  Learn 

• Communication 

(MORI  Research  and  Business  Strategies  Wales  Ltd,  1998) 

With  these  requirements  in  mind,  it  is  unfortunate  that  some  studies  have  shown  that  many  adults,  particularly  those 
from  socio-economic  groups  D and  E,  are  not  attracted  to  using  modem  day  ICT.  It  may  also  be  the  case  that  many 
people: 

(a)  do  not  perceive  or  understand  the  need  to  develop  skills  in  these  areas,  and 

(b)  do  not  have  the  financial  resources  to  become  ICT  literate. 
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There  is  thus  a pressing  need  to  extend  and  refocus  CUV  activities  into  these  areas.  The  Multimedia  Training 
Materials  project  is  intended  to  begin  to  break  down  these  barriers  by: 

• Delivering  citizen-level  ICT  skills  through  the  vehicle  of  the  nationally  recognised  European  Computer 
Driving  License  - a British  Computer  Society  supported  initiative. 

• Developing  a web-based  collaborative  learning  environment  providing  access  to  a variety  of  new  and 
legacy  learning  resources. 

The  European  Union  has  recognised  the  needs  of  the  South  Wales  Valleys  by  classifying  the  region  as  objective  1, 
but  it  is  clear  that  initiatives  such  as  the  CUV  need  to  do  much  more  than  simply  developing  materials  and  offering 
courses.  They  also  need  to  work  in  partnership  with  schools,  employers  and  Government  agencies  on  changing 
attitudes  and  raising  awareness  of  the  need  for  lifelong  learning  and  more  specifically  the  need  for  the  development 
of  ICT  skills. 

Project  Description 

Aims  and  Objectives 

The  aim  of  the  project  itself  is  to  deliver  a variety  of  educational  materials  to  community  learners.  The  objectives  are 
threefold: 

1 . To  deliver  specific  materials  to  the  community  in  the  following  areas: 

• Multimedia 

• European  Computer  Driving  License 

• Employability  Skills 

2.  To  provide  technology  training  to  lecturers  and  community  tutors 

3.  To  develop  and  maintain  a technical  and  managerial  infrastructure  which  will  enable  further  materials  to  be 
incorporated. 

Project  Structure 


Figure  1:  Project  Structure 

A Project  Structure  was  required  to  facilitate  the  co-ordination  of  the  various  sub-activities  within  the  project.  The 
project  steering  group  developed  the  structure  described  in  figure  1 that  distinguishes  four  related  types  of  activity. 

Materials  Development 

Some  team  members  were  mainly  concerned  with  identifying  and  developing  education  and  training  materials.  In 
the  area  of  Multimedia  a specialist  team  is  developing  a multimedia  Computer-Based  Learning  package  to  teach 
multimedia  principles  and  techniques.  Staff  at  Glamorgan  are  collaborating  with  Swansea  to  produce  web-based 
portfolios  of  materials  for  each  of  the  7 modules  of  the  ECDL.  A further  team  are  working  on  portfolios  in  6 
Employability  Skills  areas  including,  Communications,  Numeracy,  Using  IT,  Problem  Solving,  Working  with 
Others  and  Improving  Learning  and  Performance. 
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Delivery  Technology 

Other  team  members  are  largely  concerned  with  developing  the  web-based  collaborative  learning  environment.  This 
environment  is  accessible  by  registered  learners  and  through  an  easy  to  use  interface  the  learner  builds  up  a Personal 
Academic  Menu  (PAM)  containing  all  the  links  required  for  their  individual  programmes.  A different  interface  to 
the  environment  is  provided  that  allows  development  staff  to  create  a standard  look  and  feel  for  their  materials.  The 
efficiency,  effectiveness  and  appropriateness  of  this  environment  is  a key  issue  for  the  sustainability  of  the  project. 

Community  Delivery 

This  is  the  activity  the  project  stands  or  falls  by.  It  is  absolutely  essential  that  any  problems  experienced,  by  the 
tutors  or  learners,  with  the  materials  (e.g.  with  inaccuracies  or  irrelevance)  or  with  the  delivery  technology,  are 
reported  back  to  the  project  steering  group  for  action.  The  key  initiatives  in  this  area  are  concerned  with  establishing 
reporting  and  feedback  mechanisms  to  ensure  that  the  provision  works  in  practice. 

Technology  Training 

The  remaining  team  members  are  concerned  with  providing  any  technology  training  required  by  the  members  of  the 
other  teams.  In  Swansea  the  emphasis  so  far  as  CUV  is  concerned  is  on  the  Humanities  rather  than  IT  and  thus  many 
of  the  community  tutors  themselves  have  limited  ICT  experience.  Therefore  the  approach  will  be  to  train  lecturers 
and  tutors  in  the  development  of  flexible  learning  materials  and  the  use  of  a variety  of  technologies  such  as  video 
and  computer  conferencing.  The  emphasis  is  on  providing  lecturers  and  tutors  with  the  technological  skills  to  create 
materials  for  use  in  the  community. 


Lessons  Learned 

1.  Close  collaboration  within  and  across  departments  and  particularly  between  campus-based  development  staff 
and  community-based  tutors  is  critical  to  the  success  of  such  initiatives. 

2.  Developing  high  quality  educational  materials,  as  is  already  well  reported  (e.g.  Marshall  et  al  1997),  and  based 
on  the  authors  experience,  is  difficult  and  time  consuming.  However  it  became  apparent  that  the  community 
tutors  would  prefer  early  access  to  small  chunks  of  materials  for  evaluation  purposes,  through  the  course  of  the 
project  rather  than  waiting  to  receive  finished  materials. 

3.  Careful  and  close  project  management  is  required.  On  reflection  I feel  that  it  would  have  been  better  to  begin 
with  a much  smaller  portfolio  of  existing  materials  and  go  through  the  complete  process  in  a first  milestone  and 
subsequently  once  the  key  lessons  had  been  learned  to  expand  the  portfolio  during  a second  milestone. 


Sustaining  the  Initiative 

Four  key  issues  regarding  sustainability  have  emerged. 

1.  Progression  Pathways  into  Higher  Education:  Small  cohorts  of  learners  on  non-accredited  community 
programmes  do  not  generate  profit,  clearly  making  them  difficult  to  sustain.  However  profit  can  be  generated  if 
community  learners  eventually  become  full  or  part-time  students  on  accredited  modules  and  awards.  So  in  the 
future  we  intend  to  focus  on  creating  pathways  into  Higher  Education  from  three  sources  (i)  from  industry  via  a 
work-based  learning  initiative,  (ii)  from  schools  via  the  University’s  compact  scheme  and  (iii)  from  the 
community. 

2.  Focussing  on  Specific  Economic  Regeneration  Needs:  The  key  seems  to  be  the  resolution  of  particular  local 
problems  and  the  subsequent  dissemination  of  good  practice. 

3.  Maintaining  the  Efficiency,  Effectiveness  and  Appropriateness  of  a Collaborative  Learning  Environment:  In 
view  of  the  perceived  lack  of  ICT  skills  in  South  Wales  it  is  essential  to  ensure  that  the  learning  environment  is 
adding  value  to  the  learning  experience  rather  than  getting  in  the  way. 

4.  Teaching  Community  Tutors  to  Develop  their  own  Materials:  It  is  recognised  that  the  materials  produced  by 
campus-based  academics  only  provide  a baseline  and  that  community  tutors  will  augment  this  with  materials 
and  activities  of  their  own  in  response  to  the  idiosyncratic  needs  of  their  learners.  Providing  tutors  with  the 
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technical  and  educational  skills  to  do  this  effectively  (training  the  trainers)  is  another  important  way  of 
developing  sustainable  provision. 


Concluding  Remarks 

In  the  past,  indigenous  industry  faced  a responsibility  to  valleys  communities,  with  organisations  such  as  the  Miners 
Federation  acting  essentially  as  a community  college.  (The  Miners  Federation  enabled  many  people  to  successfully 
make  the  transition  from  mining  into  higher  education.)  Today’s  more  mercenary  hi  tech  industries  do  not  face  such 
responsibilities.  These  changes  in  the  nature  of  the  labour  market  combined  with  temporary  project  after  temporary 
have  not  encouraged  sustained  education  and  training  initiatives  for  valleys  communities.  We  hope  that  objective  1 
funding,  together  with  emerging  initiatives  such  as  the  University  for  Industry  will  enable  such  initiatives  to 
flourish. 
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Abstract:  The  shift  that  is  currently  taking  place  within  universities  from  teaching  face-to-face  to 
teaching  online  is  presenting  teachers  in  universities  with  the  need  to  induct  large  numbers  of 
students  into  learning  in  this  new  mode.  It  is  pertinent  to  ask  how  learners  themselves  are 
experiencing  this  change.  In  a subject  in  human  biology,  an  activity  based  on  the  topic  on  the  use 
of  statistics  in  the  mass  media  was  employed  as  a way  of  inducting  students  into  online  discussion. 
Students’  responses  were  monitored  by  focus  groups  before  and  after  the  exercise,  by  an  evaluation 
survey  and  by  inspection  of  the  transcripts  of  the  discussion.  The  issues  that  students  reported 
included  the  frustration  experienced  in  gaining  sufficient  access  to  computers,  the  perceived  value 
of  the  task,  and  their  comfort  level  in  using  computers  in  learning. 


Introduction 

The  shift  that  is  currently  taking  place  within  universities  towards  online  learning  is  confronting 
teachers  in  universities  with  the  need  to  induct  large  numbers  of  students  into  this  mode  of 
learning  over  a relatively  short  timeframe  (Inglis  et  al.  1999).  As  students  can  presumably  be 
expected  to  build  upon  the  skills  obtained  from  their  first  experience  of  online  conferencing  as 
they  move  on  to  later  subjects,  this  need  is  likely  to  be  relatively  short-lived.  However,  as  most 
students  will  have  had  no  previous  experience  of  anything  approaching  this  mode  of  learning, 
their  initial  experience  is  therefore  likely  to  colour  their  subsequent  outlooks.  It  therefore 
behoves  universities  to  try  to  come  to  understand  the  learning  experience  as  students  perceive  it. 
In  that  way  they  can  tailor  introduction  and  delivery  to  meet  varied  students’  needs.  The  major 
hurdle  is  to  get  students  off  to  a confident  start  in  the  first  place. 

In  other  studies  it  has  been  reported  the  students  have  found  the  process  of  learning  online  to  be 
less  satisfying  than  learning  face-to-face  even  though  this  it  was  more  effective  at  enabling 
students  to  attain  expected  learning  outcomes  (Benbunan-Fich  and  Hiltz  1999;  Hiltz  1998).  A 
number  of  factors  may  account  for  the  stress  students  experience  in  their  first  encounter  with 
online  discussion  in  a university  setting:  they  may  have  had  no  experience  of  the  software  used; 
they  may  not  feel  comfortable  with  using  computers;  they  may  be  unfamiliar  with  the  etiquette 
and  conventions  of  computer  conferencing;  and  they  may  not  even  see  the  value  of  interacting 
online  when  they  can  so  easily  meet  face-to-face  in  an  actual  classroom  or  in  a campus  cafeteria. 
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Harasim  et  al.  (1995)  pointed  out  that  inducting  students  into  the  use  of  computer  conferencing 
involves  two  separate  functions:  familiarising  students  with  the  operation  of  the  particular 
conferencing  system  that  is  being  used,  and  enabling  students  to  learn  how  to  interact  in  a 
conferencing  enviroiunent.  Learning  the  technical  skills  needed  to  use  the  conferencing  system 
obviously  comes  first.  Harasim  et  al.  (1995)  suggested  that,  whenever  feasible,  as  this  gives 
students  the  opportunity  to  build  confidence  in  a supportive  environment  rather  than  being 
thrown  on  their  own  resources.  These  authors  suggested  that  a three  hour  session  or  two  shorter 
sessions  is  normally  sufficient  for  this  purpose. 

Inducting  students  into  conferencing  itself  is  quite  a different  task.  Harasim  et  al.  (1995) 
recommended  that  this  be  accomplished  by  use  of  an  initial  private  email  exchange  followed  by 
an  all-group  “great  debate”.  They  pointed  out  that  centering  the  debate  around  a controversial 
issue  which  is  certain  to  arouse  passions  ensures  that  participants  join  in  the  discussion. 

RMIT  University,  like  many  Australian  universities,  is  expanding  its  use  of  online  learning  with 
on-campus  students  as  well  as  using  online  learning  to  increase  the  number  of  students  studying 
off-campus.  The  teaching  staff  view  these  trends  with  mixed  feelings.  On  the  one  hand,  there  is  a 
perception  — perhaps  a fear  — that  the  administration’s  drive  for  greater  use  of  new  technology 
is  mainly  motivated  by  a desire  to  reduce  contact  hours  and  thereby  cut  costs.  On  the  other  hand, 
they  recognise  that  online  delivery  offers  the  possibility  of  improving  student  learning  as 
students  are  afforded  greater  opportunities  for  working  collaboratively  while  proceeding  more  at 
their  own  pace.  However,  if  expectations  of  the  value  of  using  online  learning  in  on-campus 
courses  are  to  be  achieved,  then  it  is  essential  that  these  segments  be  incorporated  into  courses  in 
ways  that  meet  students  needs. 

The  Induction  Activity 

In  1999  a range  of  online  components  were  incorporated  into  the  subjects  in  Human  Biology 
within  a Bachelor  of  Applied  Science  degree.  These  included  lecture  notes,  demonstrations, 
web-based  assigmnents  and  a discussion  forum. 

A task  in  second  semester  required  students  to  summarise  an  instance  of  the  use  of  statistics  by 
the  media  (examples  they  chose  ranged  from  local  papers  through  to  Rikki  Lake),  and  comment 
on  its  accuracy  and  value.  They  were  then  required  to  respond  to  a minimum  of  two  other 
students’  postings.  Students  were  asked  to  post  and  comment  within  their  practical  groups  which 
comprised  approximately  20  people  each.  The  conferencing  system  used  was  HyperNews  and 
the  groups  were  established  within  the  HyperNews  structure.  However,  due  to  the  nature  of  the 
program,  once  the  number  of  postings  became  significant  students  had  difficulty  identifying  the 
first-order  structure,  and  later  posts  were  added  in  an  ad  hoc  fashion. 

This  task  was  the  ‘practical’  component  of  a unit  which  is  based  on  Stephen  Jay  Gould’s 
‘Mismeasure  of  Man’,  and  concludes  an  introduction  to  physical  anthropology.  Gould  highlights 
some  egregious  examples  from  the  history  of  the  science  where  statistics  have  been  used  and 
abused  in  the  scientific  and,  to  a lesser  extent,  popular  media.  One  of  Gould’s  points  is  that 
while  data  may  have  been  generated  ‘objectively’  by  reputable  academic  researchers,  when  the 
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data  are  released  into  the  public  sphere,  the  reporting  of  the  significance  of  the  data  is  often  not 
as  objective. 

The  activity  was  scheduled  for  weeks  9 to  1 1 of  a 1 3 week  semester.  It  was  set  down  as  an 
assessed  task,  but  was  allocated  a weighting  of  only  4%  of  the  final  mark.  In  the  end  it  was 
decided  that  all  students  who  completed  the  task  satisfactorily  should  be  awarded  full  marks.  In 
this  respect  it  could  be  classed  as  a ‘hurdle  requirement’.  However,  students  could  still  pass  the 
subject  without  completing  the  activity.  Of  the  class  of  83  students,  75  completed  the  activity. 

Students  were  introduced  to  the  web-delivered  material  over  an  extended  period.  Their  first 
practical  for  the  year  was  held  in  the  Faculty  computer  laboratory.  During  this  session  they  were 
introduced  to  the  range  of  resources  that  were  to  be  provided  online.  However,  the  conferencing 
system  was  not  then  functioning.  A demonstration  of  the  conferencing  system  was  presented  in  a 
lecture  in  week  7 of  the  13  week  semester.  Students  were  encouraged  to  use  the  discussion 
forum  from  that  time  onwards.  Prior  to  the  commencement  of  the  discussion  activity,  the 
structure  and  operation  of  the  discussion  forum  was  again  introduced  in  a lecture/demonstration. 

Evaluation  Methods 

The  principle  method  used  to  evaluate  the  success  of  the  induction  activity  was  the  conduct  of 
focus  groups  at  the  commencement  of  the  activity  and  once  the  activity  had  been  completed.  We 
were  particularly  interested  in  observing  any  changes  in  students’  ‘before’  and  ‘after’  responses 
to  the  activity. 

Students  were  asked  to  volunteer  for  participation  in  the  focus  groups  and  were  offered  light 
refreshments  as  an  incentive  to  participate.  Two  focus  groups  were  conducted.  The  first  focus 
group  was  held  in  Week  10,  and  the  second  in  Week  13  of  the  semester.  The  first  focus  group 
comprised  6 students.  The  second  focus  group  comprised  four  students  from  the  first  focus 
group  plus  three  students  who  had  not  participated  in  the  original  group.  The  focus  groups  met 
for  approximately  three  quarters  of  an  hour.  The  discussion  was  guided  by  a series  of  pre- 
prepared questions  (see  Table  1).  The  discussion  was  recorded  and  transcribed  for  analysis. 


Questions  put  to  both  focus  groups 

1 . What  problems  have  you  been  experiencing  in  gaining  access  to  the  online  material? 

2.  How  well  do  find  that  the  online  medium  suits  what  you  are  needing  to  do  in  this  subject? 

3.  What  do  you  think  about  the  type  of  task  you  are  involved  in? 

4.  What  has  interacting  online  been  like  for  you? 

Additional  questions  put  to  the  first  focus 
group 

Additional  questions  put  to  the  second  focus 

group 

5a.  How  do  you  feel  about  being  criticised 
online  and  do  you  feel  that  you  have 
sufficient  opportunity  to  contribute? 

5b.  Now  that  you’ve  completed  the  activity, 
what  did  you  think  you  got  out  of  it? 

6a.  What  changes  would  you  like  to  see  in  the 
way  the  subject  is  offered? 

6b.  If  you  were  doing  the  subject  again  what 
would  you  like  to  be  different  about  the 
way  it  uses  online  learning? 

7a.  How  do  you  feel  about  flexible  delivery  in 

7b.  How  would  you  feel  about  more  extensive 
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general 

use  being  made  of  discussion  groups 

8b.  Given  the  importance  of  computers  where 
do  you  think  we  should  be  striking  the 
balance  between  accommodating 
preferences  and  challenging  you  to  escape 
from  your  comfort  zone 

Table  1 . Focus  group  questions 


The  transcript  of  the  online  discussion  was  also  reviewed. 

An  evaluation  questionnaire  was  distributed  to  students  at  the  completion  of  the  semester  which 
sought  feedback  on  all  aspects  of  the  subject.  Discussion  of  the  full  results  of  this  evaluation  is 
not  covered  in  this  report.  However,  where  the  results  of  the  evaluation  threw  further  light  on  the 
way  in  which  students  responded  to  the  induction  activity  specific  reference  will  be  made  to 
these  results  below. 

Findings 

The  members  of  the  focus  group  expressed  a diverse  range  of 

The  various  modes  of  analysis  presented  a broad  view  of  the  students’  perception  of  online 
learning  and  response  to  the  task.  Some  students  were  familiar  with  computers  and  at  ease  using 
them 


‘I  like  the  concept  of  online  ...  I mean  for  people  who  use  it  you  are  not  going  to  get  anything  better’. 

Another  student  was  resigned  to  the  inevitable 

‘Like  I do  everything  to  avoid  it  but  its  going  to  hit  me  in  the  face  shortly  so  . . . ’ 

‘There’s  nowhere  where  I feel  comfortable  using  the  Internet’. 

They  were  also  perceptively  critical  of  online  material 

‘It  just  seemed  that  it  was  that  the  person  who  had  actually  dished  it  out  didn’t  give  a lot  of  thought  into  what 
had  to  be  there’  or 

‘If  I wanted  to  study  remotely  I would  have  chosen  that  way’. 

However,  three  principle  issues  emerged  from  the  discussion  and  evaluation: 

• access  to  computers 

• the  perceived  significance  and  relevance  of  the  task 

• attitudes  to  online  learning 

Access  to  computers 
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The  Bundoora  campus  of  RMIT  University  is  located  on  the  northern  outskirts  of  Melbourne. 
Because  of  this,  many  students  travel  considerable  distances  to  and  from  the  campus  each  day. 
Many  use  public  transport  or  rely  on  friends.  Activity  on  the  campus  therefore  peaks  in  the 
middle  of  the  day  but  falls  away  sharply  in  the  late  afternoon.  As  a consequence,  the  facilities  on 
the  campus  including  the  general  access  computers  are  in  heavy  demand  between  the  hours  of 
eleven  and  three. 

One  might  expect  that  having  the  opportunity  to  engage  in  discussion  online  would  be  as  an 
advantage  by  students  because  it  allows  them  to  undertake  part  of  their  study  at  home.  However 
this  was  found  to  be  only  partly  the  case. 

A survey  of  the  class  group  indicated  that  while  80%  had  a computer  at  home,  only  50%  had  a 
modem  and  an  account  with  an  Internet  Service  Provider.  Therefore  half  of  the  students  still 
needed  to  rely  on  university  computer  facilities  to  complete  the  task.  Even  students  who  did 
have  a computer  at  home  still  said  that  they  wanted  to  make  use  of  computers  on  campus  so  that 
they  could  make  best  use  of  their  time.  Difficulty  in  gaining  access  to  a computer  was  therefore 
the  source  of  a considerable  frustration  for  some  students. 

Although  the  Faculty  of  Biomedical,  Health  Sciences  and  Nursing  has  specialised  computer 
laboratories  on  campus,  the  students  in  the  focus  groups  expressed  a preference  for  using  the 
general  access  laboratories  provided  in  the  campus  Library  notwithstanding  the  fact  that  these 
were  the  busiest.  The  end-of-semester  evaluation  confirmed  that  most  students  access  the 
conferencing  system  from  home  or  from  the  Library.  The  main  reasons  given  by  members  of  the 
focus  groups  for  preferring  to  use  the  laboratories  in  the  Library  were  that  these  laboratories  had 
the  most  up-to-date  equipment  and  that  they  also  had  an  efficient  queuing  system  in  place. 
However,  students  also  commented  that  having  to  wait  in  the  queues  for  a computer  to  become 
available  consumed  scarce  time  that  they  would  otherwise  have  been  able  to  spend  on  the 
activity  itself. 

Provision  of  general  access  to  computers  is  a vexed  issue.  The  cost  of  alleviating  the  shortage  of 
computers  would  probably  not  have  been  much  more  than  the  equivalent  of  one  academic  staff 
salary  for  one  year.  Yet  the  cost  of  the  provision  of  computers  is  subjected  to  much  greater 
scrutiny  that  the  cost  of  appointing  new  staff.  Some  staff  consider  that  students  ought  to  be 
required  to  provide  their  own  computers;  others  believe  that  most  students  will  soon  have 
computers  at  home  anyway.  Yet  all  the  evidence  points  to  the  likelihood  that  the  demand  will 
continue  for  the  foreseeable  future  and  that  it  is  capable  of  being  met  a reasonable  cost.  The 
evidence  provided  by  student’s  feedback  in  this  study  is  that  lack  of  sufficient  access  to 
computers  has  the  potential  to  become  a fatal  flaw  in  the  strategy  to  build  greater  use  of  online 
learning  into  the  delivery  of  on-campus  courses. 

Perceived  Signiflcance  and  Relevance  of  Task 

While  the  activity  had  been  designed  to  engender  discussion  amongst  students  the  extent  of 
discussion  was  relatively  limited.  Students  generally  logged  in,  posted  their  piece  and  two 
comments  and  logged  out  in  one  or  two  sessions.  However,  the  reasons  for  their  failure  to  go 
beyond  the  minimum  level  of  participation  turned  out  to  be  largely  pragmatic  rather  than 
reflecting  the  value  of  the  task. 
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Students  explained  their  reasoning  in  terms  of  the  perceived  importance  of  the  task.  There  were 
two  aspects  to  this.  First,  the  credit  being  given  for  the  activity  was  quite  modest  and  secondly  as 
anatomy/physiology  majors  in  science  they  did  not  see  the  exercise  as  particularly  relevant. 
Therefore,  given  the  difficulty  in  gaining  access  to  computers  they  said  that  they  elected  to  do 
the  minimum  that  would  satisfy  the  assessment  requirements. 

The  members  of  the  second  focus  group  were  asked  what  their  attitude  would  have  been  to  the 
hurdle  being  set  higher  — for  example,  having  to  provide  four  responses.  All  members  of  the 
panel  said  that  they  would  have  had  no  objection  to  a more  substantial  activity  provided  that  it 
was  more  relevant  to  their  course  and  that  they  had  adequate  access  to  computers.  However,  they 
emphasised  that  if  they  were  required  to  put  in  more  time  then  they  would  hope  that  the  activity 
would  receive  higher  weighting.  Students  appear  to  connect  weighting  more  to  the  amount  of 
work  involved  rather  than  to  the  relative  value  of  the  subject  to  their  learning. 

Attitudes  to  online  learning 

Despite  the  apprehensions  that  some  students  in  the  first  focus  group  had  expressed  about  the 
discussion  activity  prior  to  its  commencement,  all  participants  in  the  second  focus  group  agreed 
that  the  task  had  been  interesting  and  a worthwhile  introduction  to  discussion  forums.  Typical 
comments  included: 

It  was  pretty  interesting  a lot  of  it . . . it  was  pretty  cool 
It  sort  of  motivated  [me].  It  was  interesting. 

The  students  also  enjoyed  the  experience  of  online  discussion  when  it  did  occur.  In  the  end-of- 
semester  evaluation,  comments  like  the  following  were  typical: 

It  was  an  interesting  and  different  idea  for  an  assignment 

allowed  us  to  communicate  with  the  whole  group 

different  perspectives 

I liked  the  interaction  and  comments  between  students  and  ideas 

Some  students  complained  that  there  had  not  been  enough  discussion  and  others  complained 
about  the  formality  of  the  system. 

In  the  focus  group  there  was  concern  about  one  interchange,  however  the  overall  tenor  of  the 
discussions  was  positive,  and  as  one  student  said  T couldn’t  really  be  bothered  abusing  anyone’. 

Issues  Arising  from  the  Activity 

The  issue  of  relevance  merits  closer  examination.  The  task  around  which  the  induction  activity 
was  centred  was  designed  to  develop  the  students’  critical  response  to  the  use  of  data  and  its 
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interpretation  and  presentation,  as  well  as  to  foster  a stronger  community  through  discussion- 
based  interaction.  While  the  staff  may  have  been  clear  about  the  particular  and  general  value  in 
the  activity,  the  responses  of  students  suggested  that  they  had  a much  narrower  understanding  of 
the  purpose  of  the  activity.  Nevertheless,  the  majority  of  the  participants  in  the  second  focus 
group  said  that  they  did  enjoy  the  exercise  and  that  they  gained  benefit  from  it  in  terms  of  their 
familiarity  with  computers,  their  interaction  with  other  students  and  understanding  of  the 
reporting  of  statistical  data  in  the  media. 

The  question  of  relevance  brings  up,  however,  a more  fundamental  question,  which  is  why 
include  online  components  in  an  on-campus  program?  While  the  distribution  of  lecture  notes, 
the  presentation  of  demonstrations  and  the  provision  of  quizzes  can  be  justified  on  the  grounds 
that  all  of  these  contribute  to  increased  flexibility,  the  justification  for  the  use  of  discussion 
groups  is  less  obvious.  As  one  student  pointed  out 

‘1  live  with  three  other  biology  students  anyway.  So  if  I had  a question  I would  just  ask  them’. 

The  educational  rationale  for  extending  the  use  of  discussion  groups  to  on-campus  students’ 
needs  to  be  better  articulated.  It  would  seem  that  there  are  two  main  rationales  for  promoting  the 
greater  use  of  online  discussions  amongst  students  studying  on-campus.  The  first  is  that  gaining 
the  capability  of  being  able  to  converse  in  a group  online  is  an  element  of  computer  literacy  in 
the  life  of  today’s  professional.  The  second,  and  more  important,  is  that  participating  in 
academic  discussion  is  critical  for  a student’s  intellectual  development  and  that  online  discussion 
has  the  potential  to  involve  more  students  more  often  in  this  type  of  intellectual  exchange. 

Given  the  positive  response  of  students  to  the  induction  activity  described  here,  it  is  perhaps 
unfortunate  that  the  activity  was  scheduled  late  in  semester  2.  As  an  induction  task,  it  could 
perhaps  have  engendered  much  greater  involvement  and  have  led  to  had  it  been  scheduled 
earlier  in  the  year. 

Conclusion 

The  results  of  this  study  show  that  students  by-and-large  adopt  a pragmatic  approach  to 
becoming  involved  with  new  teaching  approaches  associated  with  online  delivery.  They 
recognise  the  growing  importance  of  computers  and  the  benefit  they  derive  from  becoming 
conversant  with  them.  They  will  willingly  participate  in  activities  requiring  the  use  of  computers 
even  though  they  might  not  feel  entirely  comfortable  with  computers.  However,  the  extent  of 
their  involvement  is  very  much  governed  by  their  cost-benefit  assessment  of  the  value  of  a 
particular  activity.  The  contribution  to  grades  weighing  strongly  as  either  a cost  or  benefit 
depending  on  the  mark- value  and  the  difficulties  of  gaining  access  to  a computer.  The 
contribution  that  computer-based  activities  make  to  a student’s  overall  workload  weighs  strongly 
as  a cost. 

The  responses  obtained  from  students  in  this  study  suggests  that  universities  ought  to  pay  close 
attention  to  contextual  factors  in  inducting  students  into  online  learning,  if  they  want  to  ensure 
that  students’  initial  experience  is  positive.  If  students  are  to  gain  the  benefits  from  these 
activities,  then  they  need  to  be  given  an  effective  induction  to  online  discussion,  be  set  tasks  that 
they  see  as  relevant  and  be  given  adequate  on-campus  access  to  computers. 
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Therefore,  with  Harasim  et  al.  (1995),  we  would  suggest  that  the  introduction  to  conferencing 
should  be  through  a broad  debate  as  much  through  a great  one.  If  students  are  aware  that  the 
relevance  is  as  much  in  the  process  as  the  specific  material  they  will  respond  positively,  and  if  it 
is  constructed  as  a hurdle  task,  they  will  participate. 
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Abstract:  What  can  we  learn  from  individuals  who  are  both  early  adopters  of  instructional  technology  and 
excellent  teachers?  Roger's  (1995)  adopter  categories  are  useful  for  simplifying  the  complexity  of  adoption 
patterns  in  a social  system  by  describing  a central  exemplar  or  summarization  of  the  early  adopter  and  other 
categories.  Diffusion  theory  and  research  yields  global  judgments  and  perspectives,  but  storytelling  enables  us  to 
take  the  personal  point  of  view,  to  understand  the  particular  aspects  of  expertise  and  experience  embodied  by  the 
unique  and  variable  individual.  This  paper  describes  an  investigation  into  the  relationship  between  excellent 
teaching  and  early  adoption  of  technology  by  highlighting  the  stories  and  narratives  of  individual  adopters  using 
Roger's  (1995)  innovation-decision  process  as  a conceptual  framework. 


1.0  Global  Characteristics  versus  Individual  Stories 

A potential  limitation  of  considering  the  integration  of  technology  using  the  adopter  categories  derived  by  Rogers’ 
(1995)  diffusion  theory  lies  in  their  very  nature  as  summaries  of  global  characteristics  (i.e.,  innovator,  early  adopter, 
early  majority,  late  majority,  and  laggards).  While  the  adopter  categories  are  useful  for  simplifying  the  complexity 
of  adoption  patterns  in  a social  system  by  describing  a central  exemplar  or  summarization  of  the  early  adopter  and 
other  categories,  these  "defining  characteristics"  also  understate  the  uniqueness  of  the  individual  member.  It  is  worth 
remembering  that  early  adopters  are  unique  and  variable  individuals  who  may  resemble  each  other  much  less  than 
they  resemble  the  general  subgroup  characteristics.  For  example,  one  can  readily  appreciate  that  early  adopters 
possess  various  and  different:  levels  of  ability  and  skill,  beliefs  and  visions  about  the  value  of  technology,  specific 
personality  traits,  levels  of  risk-taking  behavior,  motivations  to  learn  about  technology  (e.g.,  internal,  external, 
environmental,  opportunity),  development  patterns  (e.g.,  self-taught,  peer  teaching,  formal  courses),  and  have 
implemented  computers  in  different  environments,  under  different  conditions  and  with  different  expectations. 

Sternberg  (1997)  profiles  the  views  of  expert  professors  who  teach  introductory  psychology  and  have  written 
textbooks  for  the  course.  His  rationale  for  profiling  expert  teachers  is  that  the  best  help  comes  from  seasoned 
instructors  who  not  only  have  taught  for  many  years,  but  from  those  who  have  thought  deeply  about  the  course,  and 
have  come  up  with  new  and  exciting  ideas  about  how  to  teach  it.  His  approach  to  documenting  teaching  excellence 
using  narrative  captured  both  the  similarity  and  variability  between  experts,  and  is  useful  to  professors  across 
disciplines  who  want  to  improve  their  teaching. 

Narrative  stories  and  profiles  of  experts  are  well  suited  to  capturing  exactly  those  elements  or  details  that  formal 
models,  such  as  theories  of  diffusion  and  adopter  categories,  may  leave  out.  Personal  stories,  such  as  the  profiles  of 
expert  teachers  (Sternberg,  1997),  capture  the  subjective  emotions,  thoughts,  and  beliefs  of  the  categoiy  members. 
Logic  generalizes,  stories  particularize  (Norman,  1993).  Diffusion  theory  allows  one  to  form  a global  judgment  and 
perspective,  while  storytelling  allows  one  to  take  the  personal  point  of  view,  to  understand  the  particular  aspects  of 
expertise  and  experience  embodied  by  the  individual.  Stories  are  not  better  than  logic;  logic  isn’t  better  than  stories 
(Norman,  1 993).  It  is  appropriate  to  use  both  in  the  attempt  to  characterize  early  adopters  of  technology  on  campus. 

2.0  Methodology 

This  investigation  profiled  individuals  who  are  both  early  adopters  of  instructional  technology  and  excellent 
teachers.  This  study  begins  to  address  the  need  for  “best  help  from  seasoned  instructors”  by  providing  role  models 
for  faculty  who  want  to  adopt  technology  in  their  teaching.  This  case-by-case  analysis  models  Sternberg’s  (1997) 
framework  by  documenting  instances  where  the  adoption  of  technology  and  excellent  teaching  exist  in  the  same 
individual.  The  rationale  for  collecting  this  information  is  based  upon  the  value  of  providing  descriptive  accounts  of 
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this  expertise  for  the  benefit  of  faculty  members  who  wish  to  develop  both  their  technology  integration  and  teaching 
skills.  Post-secondary  instructors  from  across  disciplines  can  benefit  from  the  collective  wisdom  of  those  who  have 
thought  deeply  about  and  taught  extensively  with  technology  for  a number  of  years.  The  individual  experiences 
represent  great  diversity.  Faculty  members  profiled  here  represent  the  social  and  physical  sciences,  as  well  as 
administration  and  management,  and  they  range  from  being  enthusiastic  beginners  to  seasoned  experts  at  integrating 
technology  in  their  teaching  and  research.  The  individuals  share  a passion  for  excellent  teaching  and  continue  to 
pursue  new  ways  to  integrate  technology  to  fundamentally  improve  teaching  and  learning. 

2.1  Information  Collection 

Semi-structured  interviews  were  conducted  with  faculty  members  to  gather  more  in-depth  and  specific  information 
about  the  integration  of  technology  for  teaching  and  learning.  The  primary  objective  was  to  record,  analyze  and 
interpret  the  individual  faculty  member’s  experiences,  opinions,  and  perspectives  with  regard  to  integrating 
technology  for  teaching  and  learning.  While  the  interview  procedure  used  a semi-structured  format  guided  by 
specific  topics,  it  was  also  open-ended  in  nature  to  be  responsive  to  emergent  topics  and  themes.  Discussion 
questions  were  drawn  from  ten  sub-scales  of  a faculty  survey  (Jacobsen,  1998)  as  well  as  using  the  following  topics 
as  guides:  (a)  Perceptions  of  value  of  computing  skills  for  self  and  students,  (b)  computer  use  patterns  and  skill,  (c) 
training  and  staff  development  (if  any)  & modeling  from  mentors,  (d)  motivation  for  using  technology  (instructional, 
personal,  institutional  mandate),  (e)  experiences  with  technology  (early  positive  or  negative),  (f)  degree  to  which 
they  require  technology  use  by  students,  and  (g)  changes  in  use,  practices,  and  beliefs  about  computers  over  time, 

2.2  Interpretive  Analysis 

Interview  transcripts  were  analyzed  using  a constant  comparison  method,  and  sorted  into  major  themes  using  a 
combination  of  categories  derived  from  prior  research  on  teaching  excellence  (Andrews,  Garrison,  and  Magnusson, 
1996)  and  from  Rogers’  (1995)  innovation-decision  process.  Five  categories  provide  a conceptual  framework  for  the 
consideration  of  individual  stories  about  adopting  technology  for  teaching  and  learning:  1)  Knowledge:  values, 
beliefs  and  characteristics,  as  well  as  felt  needs/problems  and  degree  of  innovativeness,  2)  Persuasion:  expected 
outcomes  and  benefits,  as  well  as  perceived  characteristics  of  the  innovations  such  as  relative  advantage  and 
compatibility  with  existing  teaching  methods,  3)  Decision:  processes  used  to  attain  desired  outcomes,  and  factors 
influencing  the  decision  to  adopt  or  reject  the  innovation,  4)  Implementation:  specific  instructional  strategies  that 
support  the  processes,  and  5)  Confirmation:  motivators  and  impediments  to  integrating  computer  technology,  as  well 
as  descriptions  of  continued  adoption  or  discontinuance. 

3.0  Case  Studies 

The  following  section  represents  a selective  summary  of  interview  results  illustrated  with  condensed  excerpts  from 
source  transcripts.  Cases  are  presented  in  no  particular  order,  and  pseudonyms  are  used  to  provide  anonymity  for 
participants. 

3.1  Knowledge 

The  following  faculty  member  has  been  awarded  teaching  awards  at  the  University  of  Calgary,  and  has  integrated 
various  technologies  in  classroom  teaching  in  the  past  few  years.  This  individual  increases  their  awareness- 
knowledge  about  ways  to  integrate  technology  in  the  classroom  using  a variety  of  means.  Completing  the  on-line 
survey  as  part  of  the  present  investigation  (Jacobsen,  1998)  was  treated  as  a learning  opportunity:  ‘7  found  it 
interesting  to  go  through  the  questionnaire,  because  it  occurred  to  me  that  there  was  a whole  lot  there  that  I had 
heard  of  .../  am  doing  stuff  like  that,  and  I am  doing  that,  and  so  on...,  and  there  was  a bunch  of  stuff  that  I hadn  7 
really  thought  about  using  for  the  classroom.  ” S/he  also  described  a few  instructional  strategies  being  discussed  by 
her/his  department  to  enhance  the  experience  of  students  in  large  enrollment  (250+)  sections,  including  CD-ROM- 
based  study  guides  produced  by  textbook  publishers  and  developing  web-based,  self-paced  instructional  modules. 

3.2  Persuasion 

This  individual  brings  more  than  two  decades  of  experience  in  information  technology  to  a consideration  of  the 
characteristics  of  early  adopters.  As  such,  this  individual’s  retrospective  analysis  of  some  of  the  issues  surrounding 
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the  integration  of  technology  is  rich  and  in-depth.  This  individual  vv^as  particularly  interested  in  the  innovators,  the 
pioneers  on  the  edge  of  their  field  who  actually  develop  new  technological  tools  to  address  certain  problems: 

“When  I look  at  people,  I consider  the  innovators.  Theyve  never  sat  around  waiting  for  somebody  to  parachute  in 
with  a solution  to  their  problems,  they  go  out  and  look  for  it.  So,  I guess  the  first  instance  is  the  character  of  the 
innovator,  the  selfstarter,  a desire  to  get  on  with  something,  not  just....  There  are  a lot  of  academics  who,  / won’t 
say  go  through  the  motions,  but  there  are  a lot  of  academics  who  pursue  the  normal  cycle  in  the  normal  way.  They 
meet  their  classes,  they  write  research  papers,  they  supervise  graduate  students,  and  whether  they  contribute 
anything  to  the  body  of  knowledge  is  questionable.  But  / think  that  most  of  the  professoriate,  certainly  a significant 
portion  of  the  professoriate  are  ...  effective,  competent,  participants  in  process.  The  people  that  we  are  talking  about 
are  process-changers. " 

This  view  is  reminiscent  of  Bereiter  and  Scardamalia’s  (1993)  work  on  competent  teachers  and  expert  teachers,  and 
the  difference  between  them  in  investing  cognitive  resources  in  progressive  problem  solving.  Early  adopters  appear 
to  be  constantly  pushing  the  edge  of  the  problem,  and  as  soon  as  it  gets  close  to  being  solved,  they  prefer  to 
reformulate  the  whole  thing  and  start  again  with  a new  problem. 

3.3  Decision 

This  faculty  member  has  more  than  two  decades  of  experience  teaching  with  and  about  computers,  has  been 
recognized  as  an  excellent  teacher  with  several  awards,  and  concentrates  on  technology  as  a major  area  of  research. 
Excerpts  from  the  interview  transcript  highlight  some  of  the  motivators  and  impediments  to  integrating  technology 
for  both  early  adopters  and  mainstream  faculty.  An  individual’s  decision  to  adopt  or  reject  technology  for  teaching 
and  learning  may  be  influenced  by  external  factors  in  their  research  domain  that  take  priority  over  an  internal  desire 
to  focus  on  innovative  teaching  methods. 

“Globally,  the  extrinsic  reward  systems  have  to  be  there.  People  at  the  assistant  professor  or  associate  professor 
level  who  want  to  get  promoted  or  get  tenure,  will  find  very  quickly  that  unless  they  are  in  a Faculty  of  Education  or 
being  sponsored  by  a Dean  or  Department  Head  who  has  a very  strong  commitment  to  quality  teaching,  they  will 
not  get  any  extrinsic  rewards,  such  as  merit  pay,  tenure,  or  promotion  if  they  do  research  of  this  type  ...  in  chemistry 
or  physics  or  biology,  the  pay  off  is  going  to  be  in  doing  more  basic  research  in  their  domain,  such  as  NSERC 
funded  research,  or  MRC  funded  research  ...  They  make  get  intrinsic  rewards  from  seeing  better  quality  teaching  in 
themselves,  but  to  get  that  better  quality  teaching  they  have  to  invest  a lot  of  time  and  effort,  and  sometimes  their 
own  money  because  there  are  no  research  dollars  to  do  the  things  they  want  to  do,  and  the  payoff  is  quite  minimal.  " 

The  following  professor  has  integrated  technology  into  teaching  and  learning  for  over  five  years,  has  been  awarded 
teaching  excellence  awards  on  campus,  and  currently  uses  the  World  Wide  Web  as  a supplement  to  on-campus 
teaching.  In  addition  to  addressing  workload  concerns  about  the  time  needed  to  learn  about  and  integrate 
technology,  the  potential  threats  to  employment  security,  and  the  possible  influence  of  the  annual  review  process, 
this  individual,  who  has  over  twenty  years  of  experience  as  a member  of  an  academic  staff,  discussed  the  possibility 
that  some  faculty  are  concerned  about  putting  their  lack  of  skill  with  technology  on  display. 

“Another  part  of  it  / think  is  Just  plain  competencies,  old  fashioned  technology  skill.  Knowledge  and  skill,  and 
sometimes  an  unwillingness  to  look  stupid,  I mean  that  is  one  of  the  things  that  you  have  to  recognize  when  learning 
a new  technology  is  you  have  got  to  become  a fool  for  awhile.  Some  of  my  colleagues  have  difficulty  doing  that,  I 
don  7,  / never  have,  but  that  is  another  part  of  the  resistance.  " 

3.4  Implementation 

The  following  faculty  member  volunteered  to  participate  in  an  interview  after  completing  the  on-line  survey 
(Jacobsen,  1998).  This  individual  is  an  early  adopter  of  technology  for  teaching  and  learning,  and  has  past 
experience  in  the  information  technology  industry.  The  discussion  of  this  case  will  include  attitudes  and  beliefs 
expressed  by  this  faculty  member  about  technology  integration  plans  on  campus  and  elsewhere.  Even  if  a 
technology  integration  plan  has  relative  advantages  for  the  faculty,  if  administration  attempts  to  force  a seemingly 
political  agenda,  the  plan  is  likely  to  fail.  Faculty  will  become  cynical,  feel  unjustifiably  pressured,  and  not  listened 
to.  Focus  will  shift  from  the  teaching  and  learning  issues,  to  discussions  about  the  perceived  agenda.  Faculty  may 
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seek  to  undermine  the  innovation  by  actively  or  passively  rejecting  the  proposal.  Another  constituency  that  will  be 
affected  by  top-down  decision  making  is  the  student  body. 

''Some  students,  particularly  those  who  understand  the  issues,  are  vehemently  opposed  [to  a lap-top  acquisition 
requirement].  Their  incensed  that  tuition  has  doubled  in  the  last  five  years,  and  they  are  looking  at  another  doubling 
to  pay  for  these  computers.  Student  leaders  are  very  aggressive  in  opposition  to  us  ...  we  need  to  deal  with  those 
concerns  ...  we  need  to  deal  with  the  fact  that  students  have  an  installed  base  of  hardware  that  they  have  bought, 
invested  substantial  amounts  of  time  and  money  in,  and  now  we  are  going  to  now  say  its  obsolete.  ” 

3.5  Confirmation 

The  following  faculty  member  has  been  integrating  technology  into  classroom  teaching  for  more  than  two  decades 
and  conducts  both  basic  and  applied  research  with  computer  and  networking  technology.  In  order  to  justify  the 
investment  of  time  and  effort,  faculty  members  need  to  be  convinced  that  their  technology  integration  efforts  are 
having  the  desired  or  intended  effects  on  student  learning.  This  individual  described  some  of  the  reinforcing 
messages  that  s/he  has  received  from  the  community  about  the  integration  of  technology  into  her/his  courses. 

"I  have  had  e-mail  from  people  in  lots  of places  saying  how  they  like  this  course.  I have  students  who  have  put  stuff 
up  on  the  web  for  assignments,  and  then  I get  e-mail  from  industry  people,  people  in  government,  hospitals,  all  over, 
who  say,  'Oh,  what  you  have  been  doing  is  exactly  what  I need  to  do  for  my  research  project  \ I have  to  write  back 
and  say  'this  was  an  undergraduate  assignment’,  or  'I  got  that  out  of  this  textbook’,  or  whatever.  They  write  from 
all  over  and  say,  'you  are  obviously  a great  expert  on  this  and  this  and  this  ’,  and  it ’s  nothing,  it  is  just  stuff  I have 
put  up  along  the  way.  So,  ! say  to  them,  'well,  these  are  the  people  who  have  developed  this,  why  don ’t  you  talk  to 
them  ’,  and  the  students  are  really  thrilled  you  know,  because  I mail  it  along  to  the  students  concerned,  and  I say  talk 
to  these  people.  ...it  makes  it  all  very  authentic,  because  they  are  not  just  sharing  it  with  their  instructor,  they  are 
literally  sharing  it  with  other  interested  people  in  the  world.  And  our  students  now  do  that  automatically,  they 
search  for  other  people,  and  they  look  for  list  servers  and  newsgroups  in  areas  that  they  have  got  an  interest  in,  and 
if  they  have  got  a question,  technical  questions,  they  ask  'has  anybody  had  this  problem,  or  does  anybody  have  any 
research  material  on  something  or  other  ’,  and  they  get  all  of  this  stuff  coming  back  and  they  make  contacts. " 

In  addition  to  observations  and  evaluations  of  the  quality  of  student  work  throughout  the  course,  this  instructor 
draws  upon  feedback  from  sources  beyond  the  institution  about  the  relative  advantage  and  effectiveness  of  using 
technology  as  an  on-line  publishing  environment. 

4.0  Discussion 

The  cases  presented  here  provide  some  insight  into  the  development  of  expertise,  variable  experiences,  dispositions, 
decision  making,  and  motivation  for  technology  integration  efforts  of  early  adopting  faculty  members  who  are  also 
excellent  teachers.  These  are  meant  to  be  useful  accounts  of  the  individual  experience  with  technology  integration, 
as  well  as  provide  insight  into  the  beliefs  and  values  of  individual  faculty  members,  their  expectations  for  outcomes 
and  benefits  from  using  technology  for  teaching  and  learning,  and  a variety  of  instructional  strategies  they  use  to 
support  educational  processes.  There  is  some  consistency  in  what  the  individual  faculty  members  regard  as 
important  and  valuable  instructional  goals,  and  also  diversity  in  the  characteristics  of  instructors  and  their  specific 
instructional  methods  for  integrating  technology  into  teaching  and  learning.  Readers  can  compare  and  contrast 
characteristics  and  teaching  methods  of  a faculty  member  who  is  relatively  new  to  technology  integration  and  a 
faculty  member  who  has  used  technology  in  teaching  and  learning  for  over  twenty  years.  The  cases  present  different 
perspectives  on  the  administrative  and  cross-campus  issues  involved  in  widespread  adoption  of  technology.  A goal 
that  was  accomplished  with  this  research  was  the  generation  of  descriptive  accounts  that  provide  a means  for 
drawing  parallels  and  contrasts  between  the  early  adopter  profiles  and  a faculty  member's  own  educational  practice. 
The  cases  presented  here  were  reconstructed  from  interviews  and  depended  upon  the  individual’s  retrospective 
analysis  of  their  technology  integration  efforts  and  teaching  philosophies.  Future  research  that  attempts  to  profile 
early  adopting  faculty  who  are  excellent  teachers  could  also  employ  a longitudinal  observation  methodology  to  be 
more  informative.  Although  it  was  not  a goal  of  the  present  investigation,  future  research  of  this  kind  may  also 
choose  to  profile  excellent  teachers  who  do  not  integrate  technology  into  their  teaching  and  learning  to  facilitate  a 
comparison  with  the  early  adopter  cases  presented  here. 
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5.0  Conclusions 


The  results  from  interviews  with  early  adopters  suggest  that  there  is  a relationship  between  early  adoption, 
motivation,  and  excellent  teaching.  The  motivation  to  integrate  technology  in  their  teaching  tends  to  be  located  in 
the  early  adopter’s  beliefs  about  excellent  teaching.  The  faculty  members  described  ways  in  which  technology 
enables  them  to  be  a better  teacher,  provided  enriched  learning  opportunities  and  access  to  information,  and 
improved  communication  with  students.  Early  adopters  seem  to  exhibit  characteristics  of  "expert-like  learning", 
such  as  progressive  problem  solving,  belonging  to  second-order  environments,  and  flow  (Bereiter  & Scardamalia, 
1993).  They  often  describe  or  refer  to  their  extensive  experience  and  knowledge  about  the  technology  as  a given,  "// 
is  not  rocket  science,**  which  suggests  they  take  for  granted  their  greater  literacy  and  fluency  with  computers,  but 
might  also  be  a sign  of  indifference  or  lack  of  awareness  about  how  others  struggle  with  the  technology.  However,  it 
must  also  be  said  that  most  of  the  early  adopters  interviewed,  who  have  first  hand  experience  with  the  unreliability 
of  computers  and  anticipate  problems,  demonstrate  empathy  for  colleagues  who  encounter  bottlenecks  or  problems 
with  the  technology. 

Early  adopters  appear  to  regard  technology  knowledge  and  skills  as  one  type  of  expertise,  and  pedagogical  skills  as 
another  type  of  expertise.  Some  early  adopters  cringe  at  the  awkward  and  ineffective  uses  of  technology  by  their 
peers,  and  are  convinced  that  technology  cannot  improve  poor  teaching.  Early  adopters  interviewed  in  this  study 
seem  to  be  constantly  changing  teaching  and  learning  processes,  reformulating  and  pushing  the  edges  of  the 
problem,  creating  and  designing  alternate  solutions,  and  seem  to  be  more  content  with  risk-taking  than  the  status 
quo.  Not  only  do  early  adopters  appear  to  have  the  self-confidence  in  using  technology  for  teaching  and  learning, 
early  adopters  have  intrinsic  motivation  and  a belief  structure  that  integrating  technology  into  their  teaching  is  the 
right  thing  to  do.  An  individual  early  adopter  explained  that  they  apply  technology  in  their  teaching  because  it  is  the 
solution  to  her/his  problems,  not  a solution  looking  for  a problem. 

Early  adopters  are  concerned  about  student  evaluations  of  their  teaching,  and  student  access  and  equity  issues,  but 
many  continue  to  integrate  technology  because  of  their  enduring  beliefs  about  the  relative  advantage,  and  the 
potential  benefits  and  value  of  computers  for  students.  Early  adopters  described  some  of  the  same  impediments  that 
other  faculty  highlighted,  such  as  equipment  & software  failures,  poorly  designed  classrooms,  and  slow  and  clumsy 
Internet  connections.  However,  a subtle  difference  was  that  these  difficulties  seem  to  be  expected,  the  early  adopters 
locate  the  problem  in  the  technology  rather  than  themselves,  and  do  not  appear  to  be  deterred  by  these  impediments. 

Because  of  their  different  levels  of  computer  use  and  years  of  experience,  each  early  adopter  appeared  to  have  a 
unique  innovation-decision  cycle  (Rogers,  1995).  For  example,  most  of  the  early  adopters  used  the  WWW  for 
demonstrations  in  their  classroom,  some  of  them  published  information  to  the  web,  and  only  a few  required  their 
students  post  work  on  the  web.  Only  one  described  the  development  and  delivery  of  a web-based  course.  One  early 
adopter  was  convinced  of  the  benefits  of  web-delivered  instruction,  while  another  only  considers  the  web  as  an 
enhancement  to  face-to-face  instruction.  These  differences  are  consistent  with  Rogers’  (1995)  suggestion  that  the 
same  innovation  may  be  desirable  to  one  early  adopter  in  one  situation,  but  undesirable  for  an  early  adopter  in 
another  situation.  Some  of  the  early  adopters  described  events  that  have  lead  to  their  current  adoption  patterns,  and 
others  described  factors  that  may  lead  to  future  adoption,  discontinuance,  or  rejection. 

Current  fiscal  realities  are  impacting  the  early  adopters  as  well  as  the  mainstream.  Some  early  adopters  purchase 
software  using  personal  resources,  since  funding  is  not  available  on  campus,  because  they  believe  the  application  is 
important  for  instructional  goals  and  students’  learning  processes.  The  adoption  of  presentation  software  is 
undesirable  to  some  early  adopters,  not  because  of  lack  of  skill,  but  because  there  appears  to  be  little  relative 
advantage;  the  impediments  include  poor  lighting  and  unreliable  equipment,  and  interactivity  is  perceived  to  be  low. 

Early  adopters  described  their  frustration  with  the  annual  review  process  and  funding  agencies  that  seem  to 
undervalue  their  teaching  and  technology  integration  efforts.  Further,  for  many  early  adopters,  who  are  working  at 
the  edge  of  their  fields  and  developing  new  teaching  and  learning  environments,  it  is  disheartening  to  be  evaluated 
by  department  heads  and  peers  who  may  not  share  the  same  belief  structure  and  often  fail  to  understand  the 
significance  of  and  motivation  for  their  work.  Early  adopters  also  describe  frustration  with  the  "insufficient 
evidence"  argument  put  forth  by  peers  who  do  not  share  their  beliefs  about  the  benefits  of  this  type  of  research  and 
teaching.  Two  early  adopters  described  the  difficulty  of  doing  experimental  research  on  the  benefits  of  technology 
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integration  with  students  because  of  equity  (and  ironic)  concerns  about  the  control  group  who  does  not  get  the 
technology  intervention. 

There  were  clear  signals  that  some  early  adopters  feel  left  out  of  the  current  planning  and  decision-making  processes 
in  their  faculties  or  departments.  Two  individuals  described  situations  in  which  their  opinions  and  expertise  were  not 
been  called  upon  by  committees  when  major  technology  planning  was  being  done  and  decisions  were  being  made 
about  the  acquisition  of  technology  for  teaching  and  learning.  One  individual  described  a situation  in  which  their 
faculty  was  hiring  a new  faculty  member  in  the  area  of  information  and  communications  technology,  and  they  were 
not  even  consulted  or  asked  to  sit  on  the  hiring  committee. 

Early  adopters  offered  some  solutions  for  bridging  the  gap  between  themselves  and  mainstream  faculty.  One  early 
adopter  suggested  providing  training  and  support  to  increase  the  comfort  level  and  how-to  knowledge  of  other 
faculty  so  they  can  approach  any  piece  of  software  and  figure  it  out  (i.e.,  perhaps  to  become  more  like  an  early 
adopter?).  One  individual  called  for  increased  standardization  of  hardware,  software,  and  networking  in  order  to 
make  it  very  convenient  for  faculty  and  students  to  use  the  technology,  and  increase  the  amount  of  "just  in  time" 
training  and  support  by  building  more  critical  mass  technology  skill  and  knowledge  on  campus.  Another  proposed 
solution  from  an  early  adopter  was  to  give  experienced  faculty  members  course  reductions  so  that  they  can  invest 
time  creating  and  developing  technology-enhanced  curricula  that  can  be  standardized  throughout  a department  or 
faculty.  In  some  cases,  early  adopters  had  strong  opinions  about  why  mainstream  faculty  do  not  adopt  technology 
that  may  be  inaccurate  and  not  supported  by  research,  such  as  other  faculty  are  too  old  and  fearful  about  their  jobs  to 
adopt.  However,  the  one  characteristic  that  all  early  adopters  appeared  to  have  in  common  was  a willingness  to  share 
their  knowledge  and  expertise  in  some  way  to  encourage  further  adoption  of  technology  by  peers. 

The  interviews  proved  to  be  a valuable  way  to  gather  in-depth  information  from  and  about  early  adopters.  Rogers 
(1995)  calls  for  more  diffusion  research  into  the  "why"  or  motivations  for  adopting  an  innovation,  and  interviews 
with  early  adopters  extend  our  understanding  of  and  provide  insight  into  the  early  adopters  motivation  to  integrate 
technology.  A comparison  of  the  interviews  reveals  that  early  adopters  have  both  common  and  unique:  (1)  values, 
beliefs  and  characteristics,  (2)  expectations  about  outcomes  and  benefits  from  integrating  computer  technology,  (3) 
integration  processes  to  attain  outcomes,  (4)  specific  instructional  strategies  that  support  educational  processes,  and 
(5)  motivators  and  impediments  to  integrating  computer  technology. 
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Abstract:  The  acceptance,  use  of  and  impact  of  learning  resources  are  influenced  by  culture. 
This  article  presents  ten  design  guidelines  for  culture-related  flexibility  in  W^^hV-based 
course  support  sites  and  shows  how  they  can  work  in  practice.  Two  case  studies,  one  from  the 
University  of  Twente  and  one  from  the  University  of  British  Columbia,  both  involving 
WWW-based  course-support  sites  designed  for  a culturally  diverse  group  of  learners,  are 
described  and  related  to  the  proposed  guidelines. 


Introduction:  Culture  and  WWW-based  Course-Support  Sites 

The  WWW  has  the  potential  to  make  the  world  more  global,  which  creates  the  possibility  to  deliver  courses  to 
students  all  over  the  world.  But  can  you  use  the  same  course  for  all  these  students?  Will  they  all  react  the  same 
to  your  course?  The  answer  to  this  question  is  most  likely  to  be  no.  Earlier  research  has  shown  that  the 
acceptance,  use  of,  and  impact  of  learning  interventions  are  influenced  by  culture  (Jin  & Cortazzi,  1998; 
Watson,  Ho  & Roman,  1994).  As  WWW-based  course-support  sites  are  an  example  of  such  a learning 
intervention,  their  acceptance,  use,  and  impact  will  be  influenced  by  culture-related  aspects  (Collis,  1999). 
These  cultural  aspects  can  affect  the  reactions  of  the  institution,  the  instructor,  and  the  learners  to  the  WWW 
sites.  This  implies  that  it  is  important  for  educational  designers  to  consider  the  question  how  such  sites  can  be 
designed  to  adapt  to  different  expectations  and  learner  preferences,  especially  those  related  to  culture. 

What  are  these  culture-related  aspects?  Culture  can  be  defined  in  many  ways,  but  a pervasive  view  seems  to  be 
that  culture  involves  the  "beliefs,  value  systems,  norms,  mores,  myths  and  structural  elements  of  a given 
organisation,  tribe  or  society"  (Watson,  Ho  & Roman,  1994,  p.45)  that  manifest  itself  in  the  shared  and 
identifiable  ways  in  which  a group  interprets  and  reacts  to  its  environment.  The  influence  of  culture  on  the 
reactions  to  a WWW-based  course-support  site  occurs  at  a number  of  levels:  the  level  of  society,  the  personal 
level,  the  organizational  level,  and  the  level  of  the  discipline  being  taught.  Several  authors  have  presented 
analyses  of  the  influence  of  different  cultural  levels  on  the  acceptance,  use,  and  impact  of  different  types  of 
learning  support  (see  Collis,  Vingerhoets  & Moonen,  1997;  Henderson,  1996;  Reeves  & Reeves,  1997). 

However,  based  on  the  character  of  these  efforts.  Wild  (1999)  reaches  the  conclusion  that  research  in  the 
domain  of  the  two-way  interaction  between  culture  and  learning  in  distributed  networks  such  as  the  Web  is  at  an 
early  stage  of  development.  There  seems  to  be  a need  for  guidelines  for  the  design  of  culturally  appropriate 
WWW-based  course-support  sites  (Wild  & Henderson,  1997).  Collis  (1999)  responds  to  this  need  by  providing 
ten  design  guidelines  for  culture-related  flexibility  in  WWW-based  course  support  sites. 
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This  article  presents  ten  design  guidelines  for  culture-related  flexibility  in  WWW-based  course  support  sites  and 
shows  how  they  can  work  in  practice.  Two  case  studies,  one  from  the  University  and  Twente  and  one  from  the 
University  of  British  Columbia,  both  involving  WWW-based  course-support  sites  designed  for  a culturally 
diverse  group  of  learners,  are  described  and  related  to  the  proposed  guidelines. 


Guidelines  for  the  Increase  of  Culture-Related  Flexibility  in  the  Design  of  WWW-Based 
Course-Support  Sites 

The  following  guidelines  are  a combined  result  of  the  analysis  of  research  literature  and  several  years  of 
experience  at  the  Faculty  of  Educational  Science  and  Technology  at  the  University  of  Twente  with  the  use  of 
WWW-based  course  support  sites  with  a culturally  diverse  group  of  students  (Collis,  1999).  The  ideas  presented 
here  are  not  only  related  to  adapting  to  cultural  differences,  but  also  to  adapting  to  learner  differences  in 
general,  in  other  words  to  good  instruction  in  general.  However,  in  this  article  the  comments  are  directly 
focused  to  cultural  differences,  rather  than  stated  at  a more  general  level,  in  order  to  assist  those  who  are 
particularly  working  with  culturally  diverse  student  groups. 

1.  Plan  for  flexibility  and  adaptation  right  from  the  start  (Griffiths,  Heppell,  Millwood,  & Mladenova,  1994) 

2.  Design  for  a variety  of  interchangeable  roles  for  both  instructors  and  students  and  offer  support  for  an 
eclectic  variety  of  types  of  learning  experiences  (Henderson,  1996). 

3.  Use  different  media  for  the  students  in  the  most  appropriate  way.  For  example,  use  books  and  print 
materials  as  primary  study  materials,  instead  of  expecting  the  students  to  get  this  information  on-line  if 
access  time  to  a computer  and/or  a network  are  likely  to  be  limited  (Collis  & DeBoer,  1998;  Griffiths  et  al., 
1994) 

4.  Use  the  site  to  supplement  study  materials  and  to  integrate  and  manage  student  study  activities,  not  to 
replace  the  teacher  (Collis  & Remmers,  1997;  Collis,  Vingerhoets  & Moonen,  1997). 

5.  Emphasize  student  and  instructor  input  and  make  use  of  a wide  variety  of  learning  resources  (Collis, 
Vingerhoets,  & Moonen,  1997). 

6.  Design  for  minimal  technical  levels:  minimal  levels  of  technical  support,  and  of  computer-related  skills  and 
competencies,  both  for  students  and  instructors. 

7.  Limit  the  amount  of  communication,  such  as  text  and  pictures,  that  is  pre-set  in  the  system,  so  that 
adaptations  to  different  cultures  are  simplified  (Griffiths  et  al.,  1994). 

8.  Provide  tools  so  that  the  instructor  and  the  students  can  tailor  communication  and  interactivity  expectations 
and  approaches  (Collis  & Remmers,  1997). 

9.  Design  for  flexibility  in  the  organization  of  courses.  For  example,  differences  in  length  or  in  examination/ 
assessment  requirements  (Collis,  Vingerhoets  & Moonen,  1 997). 

10.  Be  realistic  about  what  instructors  can  and  will  do,  in  terms  of  time  and  skills  (Collis,  1998). 

How  can  these  guidelines  be  applied  in  practice?  The  next  two  sections  describe  two  examples  of  the  design  of 
WWW-based  course-support  sites  planned  to  be  appropriate  for  learners  representing  a number  of  different 
cultural  backgrounds. 


The  TeleTOP  Method:  An  Example  from  the  University  of  Twente 

The  TeleTOP  Method  has  been  developed  at  the  Faculty  of  Educational  Science  and  Technology  at  the 
University  of  Twente  in  the  Netherlands  during  1997-1998  and  is  being  used  to  redesign  all  of  the  courses  in  the 
Faculty  for  more  flexibility.  The  first-  and  second-year  courses  are  in  full  operation  and  the  remainder  of  the 
courses,  including  courses  for  the  Masters  programme  for  international  students,  are  in  the  process  of  re-design 
(Collis,  1998;  see  also  http://teletop.edte.utwente.nl).  Each  first-  and  second-year  course  serves  two  cohorts:  the 
regular  students  and  what  is  called  the  "Friday”  Cohort,  mature  students  who  work  during  the  day  and  only 
come  to  the  campus  one  every  two  weeks,  on  a Friday,  when  they  attend  face-to-face  or  practical  activities  with 
the  other  students  in  the  course.  The  rest  of  the  time  these  students  participate  in  the  course  via  the  WWW  sites. 
These  WWW  sites,  however,  are  just  as  extensively  used  by  the  regular  students  as  well;  both  cohorts  just 
participate  in  different  ways  and  within  the  cohorts  students  can  also  choose  different  forms  of  participation. 
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This  flexibility  is  related  to  the  different  cultures  of  the  students,  in  terms  of  their  life  experiences,  educational 
backgrounds,  ages,  backgrounds,  and  work  histories.  There  is  also  need  to  provide  for  flexibility  related  to 
instructors  and  subject-area  characteristics.  To  provide  for  flexible  responses  to  these  different  aspects,  students 
have  the  choice  of  different  instructional  methods  and  learning  resources  as  well  as  the  more-familiar 
flexibilities  of  time  and  place.  An  increasing  number  of  the  courses  now  are  offered  to  combinations  of  students 
in  the  regular  and  distance  groups  as  well  as  non-Dutch  students  in  the  Masters  program  of  the  Faculty  and 
foreign-exchange  students  visiting  the  university.  Eight  aspects  of  the  TeleTOP  Method  can  be  identified  that 
provide  this  flexibility: 

1 . The  instructor  is  extended,  not  replaced,  to  make  the  current  practice  more  flexible.  It  is  assumed  that  not 
all  instructors  will  follow  the  same  approach  to  teaching. 

2.  The  TeleTOP  technical  system  is  based  on  ease  of  use  for  instructor  and  students. 

3.  The  WWW-based  TeleTOP  Decision-Support  Tools  (Collis  & DeBoer,  1998)  assist  the  instructor  to 
choose  WWW-site  features  tailored  in  form  and  combination  to  his  or  her  own  setting. 

4.  The  matrix-like  course  roster  and  the  course  "communication  centre"  are  standard  features  of  the  WWW 
sites,  but  can  be  tailored  and  adapted  to  the  instructor's  wishes  (Collis  & DeBoer,  1998). 

5.  Communication  and  presentations  that  happen  in  face-to-face  settings  can  be  made  available  as 
asynchronous  streaming  video,  in  whatever  segments  the  instructor  or  students  wish,  via  the  WWW  site  for 
use  at  different  times  and/  or  locations. 

6.  Courses  are  designed  for  "multiple  use",  a course  is  designed  to  teach  once,  adapt  within  for  individual 
differences.  In  particular,  to  alter  aspects  related  to  assignments  and  study  materials  based  on 
characteristics  of  groups  of  the  students,  but  having  all  integrated  within  the  same  site. 

7.  Instructors  are  provided  with  an  easy-to-apply  analysis  approach  for  the  re-design  process,  based  on 
concrete  aspects  of  a course.  They  are  encouraged  only  choose  to  use  the  WWW  site  when  it  offers  added 
value  compared  to  other  media. 

8.  The  TeleTOP  Rapid  Prototyping  Method  (Collis  & DeBoer,  1998)  involves  instructors  in  the  design  and 
building  of  their  own  course-support  sites  at  the  same  time  as  they  develop  their  own  skills  and 
competencies  in  handling  the  sites. 

As  pointed  out  earlier,  the  TeleTOP  method  is  being  used  to  adapt  a large  number  of  courses,  varying  in 
instructional  approaches,  from  project-oriented  to  skills  training  in  programming,  from  courses  focused  on 
statistical  methodologies  and  their  applications  to  courses  involving  substantial  amount  of  theoretical, 
conceptual  materials.  Besides  this,  the  course-support  sites  include  a wide  diversity  of  WWW-based  features, 
from  very  few  to  very  many,  and  show  a great  diversity  in  how  the  instructors  are  using  them  and  organizing 
their  courses. 


The  Post-Graduate  Certificate  in  Technology-Based  Distributed  Learning:  An  Example 
from  the  University  of  British  Columbia 

The  post-graduate  certificate  in  Technology-Based  Distributed  Learning  is  the  result  of  a partnership  created 
between  the  Virtual  University  of  the  Monterrey  Institute  of  Technology  (ITESM)  in  Mexico  and  the  University 
of  British  Columbia  (UBC)  in  Canada.  The  certificate  consists  of  five  graduate  courses  and  was  launched 
internationally  in  the  fall  of  1997.  The  five  courses,  on  designing,  developing  and  delivering  technology-based 
distributed  learning,  are  offered  to  several  different  target  groups  simultaneously.  The  first  group  consists  of 
Masters  students  in  the  Educational  Technology  program  at  ITESM,  originating  from  several  Spanish-speaking 
countries  including  Mexico,  Venezuela,  and  Chile.  UBC  offers  the  courses  to  UBC  graduate  students  in  the 
Masters  in  Education  program  as  electives.  In  addition,  international  and  Canadian  students  can  take  the  courses 
as  single  non-credit  courses  or  as  part  of  the  post-graduate  certificate.  Finally,  audit  status  is  also  available.  This 
all  makes  up  for  a diverse  group  of  students.  The  first  course  for  example  had  students  from  27  different 
countries  (Bates  & Escamilla  de  los  Santos,  1997;  Janes,  1999;  see  http://itesm.cstudies.ubc.ca). 
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Some  characteristics  of  this  program  that  relate  to  cultural  flexibility  are: 

1 . The  courses  are  designed  for  "multiple  use":  designed  to  teach  once,  adapt  within  for  individual  difference. 
In  particular,  the  same  site  is  used  for  both  the  ITESM  side  and  UBC  side,  but  for  example  each  institution 
has  its  own  discussion  groups, 

2.  The  technology  used  in  the  course  is  a combination  of  print  (textbooks  and  selected  readings),  a WWW  site 
(consisting  of  a study  guide,  original  teaching  materials,  online  discussion  groups,  online  resources  (online 
articles,  journals,  access  to  the  extension  library),  a bookmark  database  (created  mainly  by  students), 
student  and  instructor  biographies,  and  assignment  questions)  and  email.  In  one  of  the  courses  a purpose- 
designed  CD-ROM  is  used. 

3.  The  technical  environment  consists  of  a separate,  advanced  Web  Server  and  a user-friendly  WWW-based 
user  interface  developed  using  WebCT  software. 

4.  The  core  component  for  each  of  the  courses  is  the  online  discussions  about  key  issues  created  in  the  WWW 
Site  forums.  Students  are  encouraged  to  participate  actively  in  the  discussions  and  course  assignments  are 
tied  directly  to  the  online  discussion  activity, 

5.  The  course  WWW  sites  share  common  features  as  mentioned  in  the  first  four  points,  but  are  tailored  and 
adapted  for  each  individual  course.  For  example,  the  design  of  the  discussion  forums  varies  from  course  to 
course. 

6.  Students  are  assessed  both  for  individual  work  and  for  (on-line)  group  work.  International  collaboration  is 
encouraged,  for  example  by  forming  small  groups  consisting  of  Canadian,  Mexican,  and  international 
students. 

7.  Tutors  play  an  important  role.  Students  are  divided  in  different  tutor  groups  of  about  15-20  students.  In 
addition  to  that,  each  course  has  some  internationally  renowned  guest  tutors,  often  the  authors  of  the  texts 
or  readings  used.  Video-conferencing  sessions  are  organized  between  ITESM  students  and  UBC  tutors. 

8.  Communication  between  the  participant  takes  place  in  several  ways,  via  online  discussion  forums,  email, 
and  chat  sessions  for  online  group  work. 

The  fifth  and  final  course  is  being  offered  for  the  first  time  this  fall.  All  the  other  courses  have  been  offered  at 

least  once,  and  some  two  or  three  times.  The  first  course  has  been  evaluated  as  part  of  a two-year  study  entitled 

"Developing  and  applying  a cost-benefit  model  for  assessing  teleleaming"  and  showed  positive  results 

regarding  meeting  the  needs  of  the  different  target  groups  (Bartolic-Zlomislic  & Bates,  1999). 


Relating  the  two  case  studies  and  the  design  guidelines 

How  culturally  appropriate  are  these  two  case  studies?  How  well  do  they  relate  to  the  ten  proposed  guidelines 
for  the  design  of  culture-related  flexibility  in  WWW-based  course-support  sites?  Table  1 shows  the  result  of  the 
mapping  of  the  guidelines  with  the  characteristics  of  the  described  cases.  The  numbers  in  the  columns  related  to 
the  cases  correspond  to  the  numbers  in  the  lists  of  case  characteristics  above. 

Table  1 provides  different  examples  of  how  to  put  the  ten  design  guidelines  into  practice.  Besides  that  it 
indicates  a good  fit  between  the  two  case  studies  and  the  proposed  guidelines.  There  are  differences,  however. 
The  TeleTOP  approach  involves  many  more  courses,  of  different  types,  than  the  set  of  courses  in  the  ITESM. 
However,  the  ITESM  course  involves  more  cultural  differences  in  terms  of  land  of  residence  and  language  than 
is  the  case  in  many  of  the  TeleTOP  courses.  The  TeleTOP  approach  puts  the  major  responsibility  for  cultural 
adaptation  in  the  hands  of  the  individual  instructor,  whereas  the  ITESM  courses  are  coordinated  among 
themselves,  with  a small  number  of  course  instructors  and  coordinators  sharing  the  responsibility.  The  TeleTOP 
courses  offer  a wider  variety  of  flexibility  options,  compared  to  the  ITESM  courses,  which  assume  all  students 
will  participate  in  forum  discussions.  Despite  these  differences  in  context,  both  cases  show  how  flexibility 
options  can  increase  responsiveness  to  cultural  differences  among  students  and  instructors  when  sharing  a 
WWW-based  course-support  environment. 
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Culture-related  Design  Guidelines 

(Collis,  1999) 

1  Plan  system  for  flexibility 


2  Support  a variety  of  roles  for 
instructors  and  students  and  easy 
change  in  these  roles 


3 Use  different  media  according  to 
appropriateness 

4 Use  the  site  to  support  and 
supplement  the  course,  not  to  replace 
the  teacher 


5  Emphasize  student  and  instructor 
input  and  use  a wide  variety  of 
learning  resources 


6  Make  technical  aspects  of  using  the 
system  as  easy  as  possible 


7 Have  a minimal  amount  of  fixed 
communication,  so  that  adaptation  is 
anticipated 

8 Let  the  instructor  and  students  tailor 
the  communication  and  interaction 
approach 

9 Design  for  flexibility  in  organization 
of  courses 

10  Be  realistic  about  what  instructors 
can  and  will  do 


TeleTOP  Method 


1 Plan  for  differences  in  teaching 
approaches  and  make  current  practice 
more  flexible 

2 Easy-to-use  TeleTOP  system 

3/4  Plan  for  instructors  to  make  their 
own  choices  regarding  features  of 
WWW  sites 

6 One  course,  multiple-uses  concept 
1 Plan  for  different  teaching 
approaches 

4  Plan  for  instructors  to  make  their 
own  choices  regarding  features  of 
WWW  sites 

7 Only  choose  to  use  WWW  site  when 
it  offers  added  value  compared  to 
other  media 

1 Extend  the  teacher,  not  replace 

4 Adapt  the  course  site  to  the 
instructor’s  wishes 

5 Asynchronous  access  to  synchronous 
activities  when  and  where  wanted 

2 Easy-to-use  TeleTOP  system 

4 Possibility  to  supplement  sites  by 
anything  available  via  the  WWW  and 
any  kind  of  file  made  by  instructor  or 
student 

2 Easy-to-use  TeleTOP  system 

8 Rapid  prototyping  technique 
involves  instructors  in  the  design  and 
building  of  their  own  course-support 
sites 

6 One  course,  multiple-uses  concept 


3  Decision-support  tools  to  assist 
instructors  in  designing  the  course  site 
as  they  want  it 

6 One  course,  multiple-uses  concept 

2 TeleTOP  technical  system 
8 Rapid  prototyping  technique 


Certificate  in  Technology-Based 
Distributed  Learning 

1 One  course,  multiple-uses  concept 

2 User-friendly  interface 

5  Adaptations  for  different  courses 


6  Different  forms  of  assessment 


2 Different  technologies  used 


7  Important  role  for  tutors 
1 Spanish  speaking  tutor  groups  for 
ITESM 


2 Different  technologies  used  and 
access  to  WWW  resources  via  WWW 
site 

3 User-friendly  interface 

4 Importance  of  online  discussions 

3 User-friendly  interface 


1 One  course,  multiple-uses  concept 

8  Different  communication  tools 
available 

1 One  course,  multiple-uses  concept 

3 User-friendly  interface 
7 Small  tutor  groups 


Table  1:  Relating  the  TeleTOP  method  and  the  Post-Graduate  Certificate  in  Technology-Based  Distributed 
Learning  to  the  design  guidelines  for  culture-related  flexibility  in  WWW-based  course  support  sites 


Conclusions 

Cultural  sensitivity  is  important  in  any  teaching  and  learning  situation  involving  participants  from  different 
backgrounds  and  with  different  life  experiences.  Designers  and  instructors  should  be  aware  of  cultural 
differences,  in  what  people  do,  how  they  react,  to  what  extent  they  accept  different  reactions  from  different 
people.  For  example,  in  some  cultures  it  is  normal  to  criticize  others  and  their  work,  in  other  cultures  it  is  not.  In 
some  cultures  it  is  normal  to  work  in  heterogeneous  groups,  in  others  not.  The  expectations  of  what  comprises  a 
good  student,  a good  instructor,  and  a good  learning  situation  vary  as  well.  WWW-supported  courses  have  the 
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potential  and  often  the  intention  of  reaching  greater  number  of  culturally  diverse  students.  The  key  to  success  in 
crossing  these  cultural  boundaries  lies  in  the  appropriate  design  of  the  online  environments  (Henderson,  1 996). 
"It  is  apparent  however  that  instructional  design  for  Web-based  learning  systems  cannot  and  does  not  exist 
outside  of  a consideration  of  cultural  influences-  both  the  cultural  influences  operating  on  the  authors  and 
instructional  designers  of  Web-based  learning  materials,  and  similarly,  those  influences  that  impact  on  the 
interpretation  of  such  materials  by  learners"  (Wild,  1999,  p.  197).  We  believe  that  increased  flexibility  is  a 
necessary  base  for  better  cultural  sensitivity  in  the  courses.  However,  it  is  not  sufficient.  The  skill  and 
sensitivity  of  the  course  instructor  or  moderator  remains  a central  variable. 
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Abstract 

Web-based  courseware  is  an  alternative  approach  to  provide  ideal  learning  opportunities  that  empower  the 
symbol  systems  and  mental  processing  capabilities  of  learners.  The  quality  of  interface  to  courseware  is  a matter  of 
vital  importance  to  its  learning  effectiveness.  The  overall  purpose  of  this  study  is  to  contribute  to  the  understanding 
of  how  primary  students  learn  in  web-based  courseware  with  respect  to  learners'  interaction  and  their  mental  models 
(personal  construct  of  interfaces)  of  the  courseware. 

This  study  utilized  document  analysis,  observation,  and  interview  methods  for  gaining  insight  into  the 
learners'  background  information,  their  mental  models  of  the  construct  of  learning  activities  in  the  courseware,  their 
actual  interactive  behaviors  during  use  of  web-based  courseware,  and  the  results  of  that  use.  Qualitative  analysis 
based  on  learners'  behaviors  and  their  explanations  of  the  specific  learning  activity  frame  were  used  to  explore  the 
nature  of  their  interactive  behaviors,  their  instances  of  mental  models  in  web-based  learning  environment.  A 
summary  of  the  study,  discussion  of  the  conclusions,  implications  of  the  study,  and  recommendations  for  further 
research  form  the  focus  of  this  report. 

Keywords:  Human-computer  Interaction,  Mental  models 


Background  and  Purpose 

Interactive  Learning  on  the  Web 

Advocates  of  active  learning,  collaborative  learning  and  authentic  projects  make  compelling 
arguments,  but  integrating  these  ideas  with  educational  technology  is  a challenge.  Internet  technologies 
of  e-mail,  listservs  and  the  web  are  remarkable  vehicles  for  thrilling  educational  journeys. 

Human  beings  are  information  processing  systems  and  have  a huge  learning  potential  in  a dynamic 
and  irreversible  changing  world.  Novak  (1964)  suggested  that  exploratory  oriented  learning  is  "the  set  of 
behaviors  involved  in  the  struggle  of  human  beings  for  reasonable  explanations  of  phenomena  about 
which  they  are  curious.”  Though  the  learning  process  involves  hands-on  activities  and  learning  skills, 

”the  focus  is  on  the  active  search  for  knowledge  or  understanding  to  satisfy  a curiosity"  (Haury,  1993). 

To  arrive  at  an  effective  learning  process  and  result  has  always  been  an  ideal  for  educational 
technology  professionals  with  the  goal  of  conveying  knowledge  efficiently  and  accurately.  Leaning  is  the 
outcome  of  ongoing  changes  in  our  mental  frameworks  while  we  actively  make  meaning  out  of  our 
experiences  (Osborne  & Freyberg,  1985).  Current  theories  on  learning  focus  on  the  “active  agenf’  role  of 
learners  who  do  not  passively  receive  message  delivered  via  instructional  channels  but  who  do  actively 
explore  phenomena  and  construct  knowledge  by  themselves.  Web-based  learning  environment  is  an 
option  of  such  goal. 

Learning  context  of  web-based  courseware  can  evoke  learner's  creation  of  analogies  from  previous 
experiences.  Learners  are  likely  to  make  meaning  out  of  their  interactive  experiences,  to  interpret  the 
verbal  as  well  as  visual  information,  to  construct  their  mental  frameworks,  and  to  understand  the  natural 
facts  as  they  work  directly  with  virtual  phenomena  in  the  learning  context. 


HCl  & Mental  Models 


Interactive  learning  is  an  emphatic  concept  and  strategy  for  web-based  learning;  a learning  party 
plays  an  active  role  in  such  process  of  learning.  External  events,  learning  activities  on  the  web,  might 
initialize  or  allure  internal  events  in  the  learners. 

Learners'  perception  and  understandings  of  interfaces  in  terms  of  mental  models  is  an  important 


502 


issue  in  investigations  of  web-based  courseware  performance  (Shackel,  1 997).  Learners  interact  with  the 
learning  activities  of  any  instructional  media/technology  to  construct  a mental  model  (symbolic 
representation)  of  the  specific  domain  and  to  make  inferences  based  upon  representation  processing. 
Kozma  (1991)  proclaimed  that  "Learning  with  media  can  be  thought  of  as  a complementary  process 
within  which  representations  are  constructed  and  procedures  performed,  sometimes  by  the  learner  and 
sometimes  by  the  medium.  While  some  learners  with  appropriate  prior  knowledge  can  construct  and  use 
representations  on  their  own  with  information  obtained  from  any  medium,  other  students  can  not  benefit 
from  media  that  are  capable  of  providing  representations  and  performing  or  modeling  operations  on 
them." 

The  human-computer  interfaces  to  an  courseware  is  the  only  channel,  a window,  an  agent  through 
which  a learner  has  access  to  the  learning  opportunities  provided  by  the  courseware  in  terms  of 
information  content,  medium  elements,  interactive  learning  activities,  and  functionality.  An  interface 
might  facilitate,  hinder,  or  totally  block  the  desired  interaction  between  a learner  and  the  learning  context. 
Therefore,  the  quality  of  interface  to  courseware  is  a matter  of  vital  importance  to  its  learning 
effectiveness. 

The  better  we  understand  the  ways  learners  interact  with  interactive  courseware,  the  more  we  can 
develop  effective  and  efficient  courseware  (Jih  & Reeves,  1 992).  Research  on  mental  models  can  identify 
salient  characteristics  of  cognitive  processes  and  help  in  the  development  of  research-based  guidelines  for 
the  design  of  effective  courseware,  thus  linking  the  technology  and  instruction  fields  together. 


Research  Design 

The  aim  of  this  study  is  to  explore  the  attributes  of  learners'  interpretations  of  visual  representations 
and  their  cognitive  construction  in  a web-based  learning  context  with  respect  to  cognitive,  affective,  as 
well  as  operational  aspects  of  mental  models  in  order  to  gain  a deeper  insight  into  the  interaction 
processes  within  a web-based  guided-discovery  courseware. 

The  suitable  four  candidates  for  this  study  were  selected  from  two  elementary  schools  in  Taipei 
metropolitan  by  Snowball  technique.  The  term  ‘suitalbe’  pertains  to  5^^-graders  and  6‘^-graders  whose 
schedule  meets  with  those  of  this  study  with  the  consent  of  the  subjects  and  their  parents. 

The  web-based  courseware,  named  The  Heart  of  Earth,  used  in  this  study  utilizes  guided-discoveiy 
environment  where  learner  can  constructs  personal  meaning  via  attempting  to  interpret  and  explain  time 
and  space  concepts  of  Earth  Science  from  personal  exploration,  experiences,  and  understanding. 
Representation  medium  in  interfaces  of  the  “The  Heart  of  Earth”  are  designed  for  the  following  purposes 
(Jih  & Wang,  1998):  An  Attention  Guide,  A Context  Builder,  An  Interaction  Motivator,  A Record  Keeper, 
A Visual  Reasoning  Anchor,  The  Invisible  Behaviors  Manifestation,  and  A Message/  Feedback  Provider. 

Accepting  the  suggestions  by  Sasse  (1991),  the  researcher  used  different  approaches  to  measure 
learners'  mental  models:  observing  learners  interacting  with  the  courseware,  asking  learners  to  explain  the 
courseware  to  a new  learner  (the  teach-back  technique),  asking  learners  to  predict  feedback  of  the 
courseware,  asking  learners  to  describe  interacting  with  the  courseware,  and  observing  learners 
interacting  with  the  courseware  with  a co-leamer. 

Information  collected  from  subjects  includes  their  level  of  prior  domain  knowledge,  observation 
and  video-taping  on  their  interaction  processes,  in-depth  interviews,  teach-back  results,  and  their  audit 
trials.  Figure  1 presents  the  research  settings  of  this  study. 


Findings  and  Conclusions 

Multidimensional  data  analysis  revealed  the  dynamics  of  mental-model  development  as  subjects 
learn  Earth  science  via  the  courseware.  This  study  shows  that  a few  tasks  require  learners  shared  decision- 
making and  mutual  dependence.  High  motivation  among  learners,  which  is  a key  to  participate  interactive 
tasks  actively,  can  be  achieved  by  presenting  a current  status  of  each  one’s  progress.  When  learners  are 
highly  motivated,  competitiveness  develops  spontaneously  among  different  persons  in  the  same  class. 

The  characteristics  of  learners’  mental  models  toward  interfaces  included  exploration,  topology, 
pattern,  preference  to  audio/visual  elements^  and  pictographic  representation.  In  each  learning  activity 
within  the  courseware,  specific  patterns  emerged.  The  mental  model  of  the  learners  can  be  structured  in 
representing  objects  to  operate  on,  operations  (actions  and  their  syntax),  content  structure,  functions  or 
features,  and  the  courseware  states. 

Learning  is  a function  of  the  learner’s  individual  differences,  the  interaction  behaviors,  the 
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interfaces  of  courseware,  the  content  to  be  learned,  the  technology  issues,  as  well  as  the  facilities, 
equipment,  organizations,  social  context  within  a learning  environment/organization/community  (cf 
Figure  2). 


Camcorder 

(to  capture  learner’s  action) 


Networked  Computer 
with  courseware 


Figure  1.  Research  Settings 


Learners  have  a variety  of  responses  to  computer  interfaces,  e.g.,  reflective,  thoughtful,  and 
changing.  Learning  is  influenced  by  three  dimensions  of  learners’  mental  models:  (Abou  Khaled  et.  Al., 
1998;  Ackerman,  Sternberg,  & Glaser,  1989;  Ayersman  & Reed,  1998;  liyoshi  & Hannafin,  1998;  Nahl, 
1998;  Stark,  Renkl,  Gruber,  & Mandl,  1998;  cf,  Figure  2.) 

^ Cognitive  factors  such  as  aptitude,  ability,  skill,  prior  knowledge,  concepts,  perspectives, 

educa|ionai  4eveJ^  expenencee^  and  learning  styles; , ^ 

^ Affective  factors  such  as,  values,  self¥llWc^ffl§fl^ation,  attitude,  preferences,  expectations, ’and 

apxiety;  and  Facilities,  Equipment,  Organizations,  Social  context,  ... 

Psychomotor  factors  such  as  eye-hand  coordination  and  visual  acuity. 
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Criteria,  Teaching  methods.  Learning  approaches.  Functions,  Consistency, 
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Research  issues  on  hurJan-cornputer  intemc^fcm^cou^fd  be  broader  More  studies  on  various  factors, 
as  shown  on  Figure  2,  in  ord«^?P!6VfHfei^fifldfflf^fti^f^  )dels  in  an  web-based  learning 
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^*8"**®  Learning 

Interpretations,  experiences,  expectations,  satisfaction, 
habits,  personality,  abilities,  goals,  motivations, 
reading/visual/technical  literacy. 
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Abstract:  The  main  problem  addressed  in  this  paper  is  how  to  help  learners  reflect  on 
knowledge  that  they  have  constructed  during  exploration  in  hyperspace.  In  this  paper,  we 
describe  an  interactive  history  that  encourages  learners  to  annotate  and  reconstruct  their 
exploration  history  to  reflect  on  what  and  why  they  have  explored  so  far.  The  interactive 
history  also  generates  a knowledge  map  proper  to  the  annotated/reconstructed  exploration 
history.  It  can  be  viewed  as  a potential  support  for  constructive  learning  in  hyperspace. 


Introduction 

Hypermedia/hypertext  based  learning  resources  generally  provide  learners  with  a hyperspace  within  which  they 
can  explore  the  domain  concepts/knowledge  in  a self-directed  way  (Conklin  1988,  Kashihara  et  al.  1997).  The 
exploration  often  involves  making  cognitive  efforts  at  constructing  the  knowledge  from  the  contents  that  have 
been  explored  (Thuering,  Hannemann,  & Haake  1995).  These  cognitive  efforts  would  enhance  learning  (Carroll 
et  al.  1985).  However,  learners  often  fail  in  knowledge  construction  since  what  and  why  they  have  explored  so 
far  become  hazy  as  the  exploration  progresses.  To  what  extent  the  learning  has  been  carried  out  also  becomes 
unclear.  This  is  a frequently  experienced  problem  of  learning  in  hyperspace  (Nielsen  1990). 

A possible  resolution  of  this  problem  is  to  encourage  learners  to  reflect  on  what  they  have  constructed 
during  exploration  in  hyperspace  (Tauscher  & Greenberg  1997,  Thuering,  Hannemann,  & Haake  1995).  The 
reflection  also  involves  rethinking  the  exploration  process  that  they  have  carried  out  since  it  has  a great  influence 
on  their  knowledge  construction.  In  particular,  exploration  purposes,  which  mean  the  reasons  why  the  learners 
have  searched  for  the  next  node  in  hyperspace,  play  a crucial  role  in  knowledge  construction  (Kashihara,  Uji’i,  & 
Toyoda  1999,  Murray  et  al.  1999).  For  instance,  a learner  may  search  for  the  meaning  of  an  unknown  term  to 
supplement  what  is  learned  at  the  current  node  or  look  for  elaboration  of  the  description  given  at  the  current 
node.  Each  exploration  purpose  would  provide  its  own  way  to  shape  the  knowledge  structure.  The  reflection 
support  accordingly  needs  to  adapt  to  the  exploration  activities  and  the  knowledge  structure  being  constructed  by 
the  learners. 

In  this  paper,  we  discuss  a proper  reflection  support  with  a careful  consideration  of  exploration  process 
in  hyperspace.  This  paper  also  describes  an  interactive  history  for  learning  with  hypermedia/hypertext  based 
learning  resources  on  the  Web.  The  interactive  history  system  provides  learners  with  their  exploration  history 
annotated  with  exploration  purposes  that  have  arisen  during  exploration.  It  also  transforms  the  annotated 
exploration  history  into  a visual  representation  called  knowledge  map.  It  spatially  shows  semantic  relationships 
among  WWW  pages  the  learners  have  visited  (Kashihara,  Uji’i,  & Toyoda  1999).  Using  the  interactive  history 
system,  the  learners  can  view  and  reconstruct  the  exploration  history  to  rethink  their  exploration  process  that 
they  have  carried  out  so  far.  They  can  also  view  the  knowledge  map  to  reflect  on  what  they  have  constructed  in 
hyperspace.  Before  discussing  the  interactive  history,  let  us  first  consider  exploration  process  in  hyperspace  and 
how  we  can  represent  it. 


Exploration  in  Hyperspace 

In  hyperspace,  learners  generally  start  exploring  from  one  node  to  others  by  following  links  among  the  nodes 
with  a learning  purpose.  The  movement  between  the  nodes  is  often  driven  by  a local  purpose  called  exploration 
purpose  to  search  for  the  node  that  fulfills  it.  Such  exploration  purpose  is  also  regarded  as  a sub  purpose  of  the 
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learning  purpose.  We  refer  to  the  process  of  fulfilling  an  exploration  purpose  as  primary  exploration  process, 
which  is  represented  as  a link  from  the  starting  node  where  the  exploration  purpose  arises  to  the  terminal  node 
where  it  is  fulfilled. 


Exploration  Purposes 

Visual  Representation 

Supplement 

Inclusion 

Elaborate 

Set  or  iRa 
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Bidirection  0<4— 
arrow  ^ ^ 

Justify 

Vertical  arrow 
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Apply 

Arrow  O — 
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Table  1:  Exploration  Purposes  and  Visual  Representation. 
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Figure  1:  An  Exploration  History. 
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An  exploration  purpose  may  have  several  terminal  nodes  with  one  starting  node.  Exploration  purpose, 
represented  as  verb,  signifies  how  to  develop  or  improve  the  domain  concepts  and  knowledge  learned  at  the 
starting  node.  We  currently  classify  exploration  purposes  as  shown  in  Table  1,  which  are  not  investigated 
exhaustively. 

An  exploration  purpose  arising  from  visiting  a node  is  not  always  fulfilled  in  the  immediately  following 
node.  In  such  case,  learners  need  to  retain  the  purpose  until  they  find  the  appropriate  terminal  node/s.  While 
searching  for  the  fulfillment  of  the  retained  purpose,  it  is  possible  for  other  exploration  purposes  to  arise.  The 
need  to  retain  several  exploration  purposes  concurrently  makes  the  knowledge  construction  more  difficult  to 
achieve. 

The  exploration  process  can  be  modeled  as  a number  of  primary  exploration  processes.  Let  us  give  an 
example  where  a learner  uses  a hyperdocument  on  a WWW  server  with  the  learning  purpose  of  understanding 
the  occurrence  of  earthquake.  In  this  example,  he/she  explores  a number  of  nodes  (WWW  documents)  with 
various  exploration  purposes.  Figure  1 gives  the  exploration  history,  which  shows  the  sequence  of  the  nodes 
visited  and  primary  exploration  processes.  For  example,  he/she  visited  the  node  "Animation  of  the  Mechanism" 
in  order  to  rethink  the  description  in  the  node  "The  Mechanism  of  Occurrence  of  Earthquake".  He/she  then 
visited  the  node  "Seismic  Wave"  since  he/she  did  not  know  the  meaning  of  the  term  used  in  the  previous  node. 

Exploring  hyperspace  in  a self-directed  way,  learners  construct  a knowledge  structure  that  is  often 
different  from  the  structure  of  hyperspace  (Thuering,  Hannemann,  & Haake  1995).  The  knowledge  structure  is 
shaped  according  to  primary  exploration  processes,  especially  the  exploration  purposes.  Each  exploration 
purpose  provides  its  own  way  to  shape  the  knowledge  structure  (Kashihara,  Uji’i,  & Toyoda  1999). 


Interactive  History 

Problems  Addressed 

Let  us  now  discuss  what  kind  of  reflection  support  is  indicated  by  the  above  consideration.  There  are  the 
following  important  problems  to  be  addressed  towards  a proper  reflection  support.  The  first  issue  is  how  to  help 
learners  retain  the  primary  exploration  processes  that  they  have  carried  out.  The  retention  may  cause  cognitive 
overload  on  exploration.  It  is  also  hard  for  computer  to  infer  their  exploration  purposes,  which  arise  in  the 
learners’  mind.  These  suggest  that  learners  should  be  encouraged  to  note  down  the  primary  exploration 
processes. 

The  second  problem  is  how  to  assist  learners  in  reconstructing  their  exploration  process.  In  reflecting  on 
their  exploration  process,  they  would  not  only  look  at  it  but  also  modify/delete  the  primary  exploration  processes 
and  add  new  primary  exploration  processes.  It  is  accordingly  necessary  to  provide  learners  with  a space  where 
they  can  reconstruct  their  exploration  process  after  exploring  hyperspace. 

The  third  problem  is  how  to  help  learners  reflect  on  knowledge  constructed  during  exploration.  One 
way  to  resolve  this  is  to  visually  represent  semantic  relationships  among  nodes  included  in  primary  exploration 
processes.  Such  representation  does  not  obviously  show  the  contents  included  in  the  explored  nodes.  However, 
this  would  be  a useful  map  for  the  learners  to  reflect  on  what  they  have  learned.  We  accordingly  call  it 
knowledge  map. 

In  order  to  resolve  the  above  problems,  we  have  developed  an  interactive  history  that  helps  learners 
reflect  on  their  exploration  process  and  knowledge  structure  by  means  of  an  exploration  history  annotated  with 
primary  exploration  processes. 


Annotated  Exploration  History 

In  order  to  help  learners  note  down  and  retain  primary  exploration  processes  during  exploration,  the  interactive 
history  system  provides  them  with  a user  interface  as  shown  in  Figure  2.  They  explore  hyperdocuments  on  a 
WWW  server  with  one  learning  purpose  in  the  left  window.  When  they  want  to  set  up  an  exploration  purpose  in 
visiting  a node,  they  are  required  to  click  one  corresponding  to  the  purpose  in  the  "Exploration  Purpose  Input" 
section  of  the  right  window.  The  clicked  purpose  is  added  to  the  "Exploration  Purpose  List"  section.  The  node 
visited  currently  is  also  recorded  as  the  starting  node  of  the  exploration  purpose. 

The  learners  can  also  add  the  object  of  verb  describing  an  exploration  purpose.  It  means  what  to 
develop/improve  in  the  current  node  whereas  the  exploration  purpose  specifies  how  to  develop/improve.  When 
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the  learners  do  not  add  this  object,  the  system  adds  the  title  of  the  current  node,  which  is  the  title  tag  in  the 
HTML  file.  When  the  learners  find  a terminal  node  of  the  exploration  purpose,  they  are  required  to  mouse-select 
the  exploration  purpose  in  the  "Exploration  Purpose  List"  section,  and  to  push  the  "Fulfilled"  button.  The  node 
visited  currently  is  then  recorded  as  the  terminal  node  of  the  exploration  purpose. 
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Figure  2.  User  Interface. 

The  system  also  provides  another  support  for  helping  learners  store  part  of  the  contents  of  the  node 
visited  currently  with  Cut&Paste  function  in  the  "Contents  Input"  section  although  they  may  not  always  need 
this  support.  In  hyperdocuments  on  WWW,  in  addition,  the  title  tags  of  the  nodes  do  not  always  represent  the 
contents  of  the  nodes.  If  the  learners  want  to  change  the  node  titles,  they  can  input  new  titles  in  the  "Contents 
Input"  section,  which  new  titles  should  represent  the  contents  the  learners  explored  in  the  nodes. 

Using  the  information  inputted  from  the  learners,  the  system  generates  the  annotated  exploration  history 
as  shown  in  Figure  1.  In  the  annotated  history,  the  nodes  learners  visited  are  sequenced  in  order  of  time.  The 
starting  node  of  each  purpose  is  linked  with  the  corresponding  terminal  node/s.  There  may  be  some  primary 
exploration  processes  without  terminal  nodes  since  they  have  not  been  found  yet.  They  can  also  click  the  nodes 
in  the  history  to  review  the  contents  information,  which  they  have  inputted  with  Cut&Paste  function. 

Learners  are  not  always  required  to  input  the  above  information  whenever  they  visit  nodes. 
Nevertheless,  inputting  the  information  during  exploration  may  be  troublesome  for  learners.  On  the  other  hand, 
it  enables  the  learners  to  make  their  exploration  more  constructive,  facilitating  their  exploratory  learning. 
Without  inputting,  they  may  only  browse  in  hyperspace. 
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History  Manipulation 


Directly  manipulating  the  annotated  exploration  history,  learners  can  reconstruct  their  exploration  process 
without  revisiting  hyperspace.  Each  manipulation  is  done  by  means  of  mouse-clicking/dragging  parts  of  the 
primary  exploration  processes.  There  are  three  basic  manipulations:  deleting  and  changing  exploration 
purposes/1  inks  between  starting  and  terminal  nodes,  and  adding  new  primary  exploration  process. 


Knowledge  Map 

The  representation  of  knowledge  map  follows  the  visualization  scheme  shown  in  Table  1 , which  shows  the 
correspondence  of  an  exploration  purpose  to  a visual  representation  of  the  relationship  between  the  starting  and 
terminal  nodes.  For  example,  an  exploration  purpose  to  'Elaborate'  is  transformed  into  a set  that  visualizes  the 
starting  node  as  a total  set  and  the  terminal  node  as  the  subset.  Following  such  correspondence,  the  system 
generates  a knowledge  map  from  the  annotated  exploration  history.  The  knowledge  mapping  is  executed  on 
learners'  demand  before/after  manipulating  the  annotated  exploration  history. 
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Figure  3.  Knowledge  Map. 

Figure  3 shows  an  example  of  the  knowledge  map  that  is  generated  from  the  annotated  exploration 
history  shown  in  Figure  1 . Viewing  this  map,  the  learner  can  reflect  on  his/her  knowledge  construction.  For 
example,  he/she  can  recall  that  he/she  rethought  the  mechanism  of  earthquake  occurrence  by  looking  at  the 
animation  of  the  mechanism.  He/she  can  also  recall  that  he/she  compared  normal  fault  and  adverse  fault  to 
elaborate  the  description  about  kind  of  earth  faults. 


Related  Work 

Let  us  next  discuss  the  usefulness  of  the  interactive  history  compared  with  related  work  on  reflection  support  in 
hyperspace.  As  the  retention  support,  there  are  several  kinds  of  annotation  systems  that  allow  learners  to  take  a 
note  (Brusilovsky  1 996).  However,  there  is  very  little  discussion  of  what  kind  of  annotation  should  be  done  for 
the  success  in  exploratory  learning.  In  the  interactive  history,  we  claim  that  the  reasons  why  learners  search  for 
the  next  nodes  should  be  particularly  noted  down. 

Current  work  on  adaptive  hypermedia/hypertext  systems  has  often  provided  spatial  maps  and  concept 
maps,  which  are  originally  used  as  navigational  aid.  Spatial  maps  can  inform  learners  where  they  are,  what  they 
explored,  and  to  what  extent  they  explored  (Domel  1 994).  However,  the  reasons  why  they  visited  the  nodes  are 
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not  clearly  shown.  In  addition,  these  maps  are  not  very  helpful  since  learners  do  not  need  to  learn  the  structure  of 
hyperspace,  Concept  maps  are  more  helpful  for  learners  who  have  lower  capability  of  exploring  hyperspace 
since  the  direction  of  knowledge  construction  is  presented  to  them  (Gaines  & Shaw  1995).  However,  learners 
who  have  higher  capability  of  exploratory  learning  may  not  always  construct  the  same  knowledge  structure  as 
the  structure  of  domain  concepts  that  the  designers  of  concept  maps  draw.  In  hyperspace,  learners  are  free  to 
identify  relationships  among  the  domain  concepts  visited  in  a self-directed  exploration,  and  these  relationships 
may  be  different  to  those  defined  in  the  concept  maps  (Thuering,  Hannemann,  & Haake  1995).  The  interactive 
history,  on  the  other  hand,  provides  them  with  a more  proper  support  since  it  enables  self-directed  exploration. 
In  addition,  the  reflection  support  can  be  provided  even  in  ill-structured  domains  of  which  concept  maps  cannot 
be  defined. 


Conclusions 

This  paper  has  claimed  that  exploratory  learning  in  hyperspace  requires  learners  to  reflect  not  only  what  but  also 
why  they  have  explored,  and  that  the  reflection  support  needs  to  adapt  to  their  exploration  process  and 
knowledge  structure  being  constructed  by  them. 

This  paper  has  also  demonstrated  the  interactive  history  with  knowledge  mapping  as  a proper  reflection 
support.  The  interactive  history  encourages  learners  to  annotate  and  manipulate  the  exploration  history  to  rethink 
their  exploration  process.  It  also  generates  a knowledge  map  from  the  annotated  exploration  history,  which 
allows  the  learners  to  reflect  on  what  they  have  constructed  during  exploration. 

In  the  future,  we  need  to  evaluate  and  refine  the  interactive  history.  We  would  also  like  to  classify 
exploration  purposes  in  more  detail  and  to  represent  learners'  exploration  process  and  knowledge  structure  more 
precisely. 
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Abstract:  In  the  medical  and  health  sciences,  a well  described  problem  for  students 
is  the  linking  of  discipline-based  knowledge  with  clinical  practice.  The  CD-Rom, 
An@tomedia  has  been  developed  to  support  a problem-based  learning  curriculum  in 
undergraduate  medicine.  An@tomedia  contains  four  primary  perspectives,  imaging, 
dissection,  systems  and  regions  and  is  intended  to  address  a variety  of  student 
learning  needs  and  approaches  to  learning.  The  evaluation  of  An@tomedia  by 
students,  tutors  and  lecturers  is  described. 


Introduction 

The  development  of  problem-based  learning  (PBL)  in  medical  education  has,  historically,  focused  on  the  whole 
student  learning  experience  rather  than  the  acquisition  of  content  (Camp,  1996).  PBL  has  been  advocated  in 
order  to  address  problematic  issues  attributed  to  traditional  medical  courses — the  creation  of  an  arbitrary  division 
between  the  basic  and  clinical  sciences,  and  wasting  student  time  in  learning  facts  that  are  easily  forgotten  or  not 
relevant  to  clinical  practice  (Finucane,  Johnson,  & Prideaux,  1998).  The  teaching  of  anatomy  at  The  University 
of  Melbourne  over  the  past  10  years  has  integrated  the  structure  of  the  human  body  with  its  clinical  applications. 
In  1999  the  medical  school  moved  to  a problem-based  learning  curriculum.  With  the  advent  of  new  computer- 
based  technologies  the  development  of  resources  to  support  the  PBL  approach  included  an  interactive  CD-Rom 
designed  to  enrich  and  engage  the  student  more  actively  in  the  learning  of  anatomy.  The  software,  An@tomedia, 
provides  greater  interactivity  for  the  student  by  providing  multiple  perspectives  (e.g.,  systemic  and  regional 
views)  of  anatomical  knowledge  linked  to  clinical  applications  in  an  interactive  computer-facilitated  learning 
(CFL)  environment.  The  perspectives  provide  a framework  for  students  to  integrate  anatomical  knowledge  with 
clinical  applications  and  procedural  knowledge.  The  software  allows  students  to  switch  between  perspectives  in 
order  to  facilitate  their  knowledge  construction,  and  address  clinically-based  problems.  This  paper  reports  the 
results  of  an  extensive  evaluation  of  the  CD-Rom,  including  qualitative  and  quantitative  analysis. 


Student  learning  and  An@tomedia 

“When  medical  graduates  enter  the  workplace,  they  are  not  faced  with  situations  labelled 
‘anatomy’,  ‘microbiology’,  or  the  like.  They  are  faced  with  patients  with  illnesses  ...”  [Bowden, 

1998,  p.  129]. 

How  subject  matter  is  experienced  and  how  the  student  subsequently  organises  and  integrates  knowledge,  is 
dependant  upon  the  student’s  perception  of  the  learning  context  (Ramsden,  1984).  Students  may  adopt  a surface 
approach  (e.g.,  learning  by  rote)  or  a deep  approach  (e.g.,  a holistic  approach,  actively  searching  for  meaning) 
(Biggs,  1989;  Marton  & Saljo,  1984). 

The  learning  process  may  be  supported  by  appropriate  learning  opportunities,  including  the  use  of  carefully 
designed  computer-facilitated  learning  (CFL)  within  the  context  of  a subject.  An@tomedia  is  designed  to  enable 
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students  to  interact  with  anatomical  knowledge  and  principles,  from  multiple  perspectives.  The  use  of  multiple 
perspectives  (and  multiple  pathways)  in  CFL  has  been  advocated  in  order  to: 

• allow  learners  to  access  the  material  in  a manner  more  suited  to  her  or  his  needs  (Reeves,  1 992), 

• support  higher  levels  of  cognition  (Ramsden,  1992),  and 

• support  a more  reflective  approach  to  learning  (Laurillard,  1993). 

In  order  to  provide  a richer  learning  environment  in  An@tomedia,  there  was  a need  to  embed  clinical, 
procedures,  imaging,  surface  anatomy,  and  dissection  components  within  the  software,  and  integrate  these  with 
the  more  traditional  systemic  and  regional  approaches  to  the  learning  of  anatomy.  Figure  1 (Fig.  1)  is 
representative  of  the  final  educational  approach  adopted.  In  Figure  1 (Fig.  1),  students  may  deconstruct  the  body 
using  Dissection  and  Imaging,  or  construct  the  body  via  Systems  and  Regions.  An  example  involving  the  back 
(the  first  module  to  be  completed)  is  highlighted.  The  figure  shows  a model  of  the  educational  design  of 
An@tomedia  in  a three-dimensional  space.  The  X-axis  represents  the  anatomical  regions  (only  three  regions  of 
the  body  are  shown),  the  Y-axis  represents  anatomical  systems  (only  four  are  shown),  and  the  vertical  axis 
represents  students  ability  to  construct  or  deconstruct  the  body  by  dissection  or  imaging.  Students  may  move 
freely  between  perspectives,  answer  questions,  and  investigate  clinical  problems  about  the  back. 


The  clinical  concepts  arc  the  ^ghic^lhat  holds  thb  framework  togctlicr 


Figure  1:  Graphical  representation  of  the  An@tomedia  perspectives 

Figure  2 (Fig.  2)  shows  the  view  seen  by  a student  examining  anatomy  from  one  perspective,  dissection.  The 
remaining  3 perspectives  are  listed  on  the  lower  right — reached  with  a simple  mouse  click.  There  are  questions 
for  the  student  to  answer  (e.g.,  ‘Can  you  identify  a radicular  arteiy’),  and  overlays  of  anatomical  structures  that 
may  be  highlighted  by  the  student.  The  student  may  move  between  specific  regions  by  clicking  on  the  regions 
on  the  human  symbol  on  the  lower  right. 

In  the  An@tomedia  CD-Rom,  there  is  extensive  linking  of  anatomical  details  with  the  clinical  perspective.  This 
has  been  done  by  the  use  of: 

• high  quality  images  (e.g.,  radiographs,  detailed  dissection  images  from  real  human  bodies,  and  graphics  with 
coloured  overlays  highlighting  key  anatomical  details), 

• integration  of  clinical  questions  and  answers,  and 

• the  facility  for  the  student  to  examine  anatomical  concepts  from  multiple  perspectives  using  multiple 
pathways. 


The  evaluation 

The  evaluation  of  An@tomedia  has  adopted  a case  study  approach.  The  majority  of  the  data  was  generated  from 
students,  their  anatomy  tutors,  and  content  experts  by  the  use  of  questionnaires  comprising  five-point  Likert 
questions  and  open-ended  free  text  responses. 
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Figure  2:  An  image  from  the  dissection  perspective  with  identification  questions,  roll-overs,  and  overlays 

The  Likert-style  questions  in  the  student  questionnaire  overlapped  approximately  60%  with  that  of  the  tutors  and 
lecturers  (TLs).  Some  questions  were  more  appropriate  for  students  (e.g..  It  helped  me  understand  anatomy)  or 
TLs  (e.g.,  1 found  the  four  perspectives  were  well  balanced  and  integrated).  Anatomy  tutors  and  lecturers  also 
answered  specific  questions  on  how  they  might  use  the  CD-Rom  as  part  of  their  own  teaching  program. 
An@tomedia  is  not  intended  as  a replacement  for  an  anatomy  teaching  program,  instead  it  is  intended  to  support 
integration  of  student  knowledge,  the  anatomical  with  the  clinical,  and  systemic  with  regional. 

Results 


The  evaluation  results  highlight  the  strengths  and  potential  weaknesses  of  the  educational  design  of 
An@tomedia.  Trends  in  the  results  suggest  directions  for  improvement  as  well  as  the  educational  value  of  the 
multiple  perspectives.  Table  1 (Tab.  1)  presents  the  mean  scores  and  standard  deviations  of  students  (first 
column)  and  a combined  tutors  and  content  experts  (second  column).  Responses  were  completed  on  five  point 
Likert  scales  where  a score  of ‘5’  was  ‘strongly  agree’,  ‘4’  was  ‘agree’,  ‘3’  was  ‘neutral’,  ‘2’  was  ‘disagree’,  and 
‘1’  was  ‘strongly  disagree’.  T-Tests  were  also  undertaken  to  compare  and  contrast  the  student  and  combined 
tutor/lecturer  (TL)  responses.  There  was  a ceiling  effect  for  most  items  and  the  data  suggested  that  all 
respondents  were  overwhelmingly  positive  about  the  software.  The  only  question  to  have  a significantly 
different  response  between  students  and  the  TLs  responses  involved  the  locating  of  information.  Not 
surprisingly,  the  tutors  and  lecturers  (all  qualified  doctors)  had  significantly  less  difficulty  than  the  students  in 
locating  specific  information.  At  the  time  of  the  evaluation  (but  since  completed),  the  index  and  search  engine 
had  not  been  implemented,  and  this  is  thought  to  have  been  contributing  factor  to  the  student  responses. 

The  students  were  also  asked  how  they  would  prefer  to  use  the  An@tomedia  CD-Rom  (Tab.  2).  It  is  significant 
to  note  that  while  over  90%  wanted  their  own  copy,  53%  indicated  they  would  use  it  in  study  groups.  The  first 
author  observed  during  the  evaluation  process,  pairs  of  adjacent  students  engaged  in  discussion  of  the  key 
aspects  of  the  different  representations  of  anatomical  structures  (e.g.,  one  student  pair  may  have  been  looking  at 
the  radiological  view  while  the  other  pair,  the  dissection). 


o 
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Students  were  also  asked  a number  of  open-ended  questions.  Student  comments  about  the  best  aspects  of 
An@tomedia  included: 

> Anatomical  content  was  superb.  Images  (dissection,  imaging  etc.)  were  beautiful. 

> Explanations  are  very  clear  and  highly  comprehensive. 

> Love  the  clinical  relevance  and  concise  exam  style  questions. 
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> It  was  simple  yet  complex.  The  diagrams  to  help  answer  the  questions  were  fantastic  and  very  helpful 
(helps  you  understand) 

> It  was  a combination  of  an  atlas  and  text  book  which  is  fantastic.  It  saves  you  having  to  flip  (between  an 
anatomical  atlas  and  anatomical  text)  ... 


Question 

Mean  (SD)  Score 

The  content  on  the  Anatomedia  program. 

Students 

(n=132) 

Tutors  and 
Experts 
(n=20) 

Anatomedia  complements  the  anatomy  teaching  program 

4.5(0.71) 

4.6  (0.60) 

Concepts  were  addressed  and  well  explained. 

4.2  (0.74) 

4.1  (1.02) 

The  content  on  the  back  was  comprehensive. 

4.3  (0.84) 

4.35  (0.88) 

There  was  strong  correlation  between  structure  and  function. 

3.8  (0.99) 

3.63  (1.21) 

The  clinical  applications  seemed  relevant  and  appropriate. 

3.9  (0.83) 

3.95  (1.18) 

While  using  Anatomedia: 

it  helped  me  understand  anatomy 

4.2  (0.77) 

it  helped  me  organise  my  knowledge  of  anatomy 

3.8  (0.94) 

I found  the  ability  to  highlight  structures  in  the  images  valuable 

4.5  (0.67) 

4.75  (0.55) 

I found  the  layering  of  images  valuable 

4.6  (0.69) 

4.84  (0.37) 

1 found  the  content  to  be  engaging 

4.0  (0.81) 

4.15(0.99) 

Regarding  learning  the  content: 

An@tomedia  helped  me  understand  anatomy  from  multiple 
perspectives 

4.0  (0.78) 

An@tomedia  enhanced  my  ability  to  interpret  anatomy  rather  than 
merely  describe  it 

4.0  (0.94) 

1 found  the  four  perspectives  were  well  balanced  and  integrated 

4.21  (0.71) 

As  a result  of  using  An@tomedia  students  are  likely  to  improve 
their 

knowledge  of  the  anatomical  basis  of  clinical  procedures 

4.15(1.04) 

application  of  anatomical  knowledge  to  solve  clinical  problems 

4.0(1.08) 

Table  1:  Mean  and  standard  deviation  (SD)  scores  for  first  year  medical  students,  tutors,  and  content  experts 


How  would  you  best  like  to  see  Anatomedia  used?  You  may  have  more  than 
one  option. 

F 

% 

Own  copy 

122 

92.4 

Tutorials 

43 

32.6 

Lectures 

36 

27.3 

Study  groups 

70 

53.0 

Table  2:  Students  preferred  use  of  An@tomedia  for  learning:  Frequency  and  percentage 

The  student  comments  about  the  aspects  that  needed  improvement  included: 

> Absence  of  a “search”  mechanism  for  “random  access”  into  a particular  concept  or  principle. 

> (add  an)  adequate  index  from  which  to  link  into  appropriate  slides  illustrating  the  concept  or  principle. 

The  comments  from  the  students  regarding  the  need  for  an  index  and  search  engine  were  expected.  However,  it 
was  necessary  to  undertake  an  evaluation  before  the  first  release  of  the  program  and  integration  into  the  new 
PEL  curriculum  in  order  to  refine  and  improve  the  initial  design. 

The  tutors  were  engaged  in  the  role  of  supervisors,  teachers  and  mentors,  particularly  in  the  activities  in  the 
dissection  room.  Comments  from  tutors  included: 

> [the  best  feature  was  the]  Combination  of  aspects  of  atlas,  textbook,  and  clinical  text  in  one  resource. 

> The  radiology  component  was  fantastic  - the  resolution  of  the  images  very  clear.  The  dissections  were 
also  impressive. 

Tutors  were  also  interested  in  An@tomedia  as  a teaching  resource.  They  perceived  two  potential  uses  for  the 
CD-Rom — as  a precursor  for  students  prior  to  dissections,  and  access  during  dissection  classes  to  clarify  and 
illustrate  anatomical  relationships.  The  latter  has  implications  for  resources — there  would  be  a need  to  install 
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video  projection  and  computer  facilities  in  the  dissection  laboratories  and  provide  training  for  tutors  to  use  the 
equipment. 

Responses  from  the  content  experts  (lecturers)  were  more  diverse,  with  both  very  positive  and  very  negative 
comments.  The  content  experts  included  practitioners  who  were  occasional  lecturers,  to  full-time  medical 
educators.  Some  comments  were  very  positive. 

> [the  best  feature  was  the]  Integration  of  in  vivo,  invitro  eg.,  dissections/  imaging.  Excellent  dissections 
and  graphics  to  highlight  structures.  Interactive  format  and  quantity  of  information. 

> [the  best  feature  was  the]  Anatomical  detail  very  good,  especially  the  imaging  component 

Less  favourable  comments  included: 

> Everything  is  too  static.  It  is  a portfolio  of  drawings  and  pictures  with  (some)  accompanying  text.  The 
CD  is  used  as  a filing  system  but  under-used  for  constructive  education. 

In  reflecting  on  the  comments  from  content  experts  it  is  important  to  recall  that  An@tomedia  is  designed  to  be 
implemented  as  part  of  a PEL  curriculum — it  is  not  intended  that  An@tomedia  become  the  sole  resource  for  the 
teaching  of  anatomy.  One  outlook  on  teaching  involves  the  view  that  a lecturer  constructs  learning  opportunities 
for  students — some  of  which  may  be  facilitated  by  a computer.  The  teaching  methods  chosen  as  the  focus  of  the 
learning  opportunities,  teacher-centred  or  student-centred  or  a mixture  of  both,  are  strongly  influenced  by  the 
educational  beliefs  held  by  the  lecturer  (Bain,  McNaught,  Mills,  & Lueckenhausen,  1998).  For  example,  in 
response  to  the  question  1(1)  “Which  of  the  factors  above  do  you  believe  are  the  most  important  in  influencing 
student  learning?”,  one  content  expert  replied: 

> None  of  these  factors  are  strong  determinants.  The  cardinal  determinate  is  what  is  in  the  exam. 
Wherefore,  assuming  that  is  what  is  shown  is  all  that  needs  to  be  known,  the  most  relevant  qualities  are: 
locate  information,  graphics  clear,  how  much  information,  concise. 

In  all  discussions  with  the  An@tomedia  development  team,  the  need  to  provide  a learning  environment  that 
addressed,  the  traditional  divide  between  discipline-based  knowledge  and  clinical  practice  was  paramount. 
An@tomedia  has  been  designed  to  be  one  component  within  a particular  educational  context — one  which  is 
student-centred,  rather  than  teacher-centred.  In  Table  2,  only  27.3%  of  students  perceived  that  An@tomedia 
would  be  appropriate  for  use  in  lectures — a teacher-centred  learning  environment.  However,  over  50%  wished 
to  use  the  CD-Rom  as  part  of  group  activities,  and  over  90%  wanted  their  own  copy  for  personal  study  and 
revision.  There  may  be  a need  in  the  future  to  include  with  the  CD-Rom  the  background  perspective  that  guided 
its  development,  and  suggestions  (for  students  and  teachers)  for  ways  in  which  it  might  be  utilised,  particularly  if 
An@tomedia  is  to  be  used  in  other  contexts. 


Summary,  conclusions  and  future  directions 
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Two  goals  of  An@tomedia  were  to  be  comprehensive  and  use  multiple  perspectives.  These  goals  seem  to  have 
been  achieved.  The  comments  regarding  the  comprehensive  nature  of  the  four  perspectives,  allowing  the  student 
to  move  from  one  representation  of  knowledge  (e.g.,  imaging  and  dissection)  to  the  clinical  applications  has  been 
highlighted  (by  students  and  TLs)  as  one  of  the  most  important  strengths  of  An@tomedia.  However,  there  are 
some  students  who  think  that  there  is  too  much  information  (generally,  and  on-screen). 

In  the  second  iteration  of  the  software,  the  index  has  been  completed  to  overcome  the  inherent  difficulty  of 
locating  specific  information.  In  addition,  an  extra  navigational  feature  in  the  form  of  sequential  numbering  for 
each  set  of  screens  in  a perspective  has  also  be  added  to  aid  in  the  relocation  of  specific  infonnation.  The 
lecturers  and  tutors  were  asked  specific  questions  that  focused  on  the  use  of  An@tomedia  for  teaching.  Their 
responses  have  highlighted  the  need  to  address  the  curriculum  in  which  the  CD-Rom  is  used.  As  indicated  in  the 
discussion,  there  are  some  issues  of  implementation  and  educational  perspectives  to  be  resolved  if  An@tomedia 
is  to  be  used  outside  the  context  of  the  current  PEL  medical  program. 

While  An@tomedia  is  focused  on  the  content  area  of  anatomy,  the  results  of  the  evaluation  indicate  there  are 
potential  lessons  to  be  learnt  from  the  use  of  multiple  perspectives  to  link  structure  and  application  in  other 
subject  areas,  and  the  embedding  of  computer-facilitated  learning  in  problem-based  learning  curricula. 

Since  the  evaluation,  the  Abdomen  and  the  Thorax  (second  and  third  modules  respectively)  have  been  completed 
and  a number  of  typographical  and  graphical  errors  rectified  as  well  as  an  improved  introduction  and  starting 
screen.  An@tomedia  is  now  a pivotal  part  of  the  first  year  undergraduate  medical  program  at  The  University  of 
Melbourne.  In  2000,  the  Neck  will  be  completed.  Sample  images  from  the  CD-Rom  may  be  viewed  at 
<http ://  www.anatomedia.  com/>. 


516 


References 


Bain,  J.,  McNaught,  C.,  Mills,  C.,  & Lueckenhausen,  G.  (1998).  Describing  computer-facilitated  learning 
environments  in  higher  education.  Learning  Environments  Research,  7,  163-180. 

Biggs,  J.  (1989).  Approaches  to  the  enhancement  of  tertiary  teaching.  Higher  Education  Research  and 
Development,  <^(1),  7-25. 

Camp,  G.  (1996).  Problem-based  learning:  A paradigm  shift  or  passing  fad?  Medical  Education  Online,  7(2),  1- 

6. 

Finucane,  P.  M.,  Johnson,  S.  M.,  & Prideaux,  D.  J.  (1998).  Problem-based  learning:  Its  rationale  and  efficiency. 
Medical  Journal  of  Australia,  168^  445-448. 

Laurillard,  D.  (1993).  Rethinking  university  teaching:  A framework  for  the  effective  use  of  educational 
technology.  London:  Routledge. 

Marton,  F.,  & Saljo,  R.  (1984).  Approaches  to  learning.  In  F.  Marton,  D.  Hounsell,  & N.  Entwistle  (Eds.),  The 
Experience  of  Learning.  Edinburgh:  Scottish  Academic  Press. 

Ramsden,  P.  (1984).  The  context  of  learning.  In  F.  Marton,  D.  Hounsell,  & N.  Entwistle  (Eds.),  The  Experience 
of  Learning  . Edinburgh:  Scottish  Academic  Press. 

Ramsden,  P.  (1992).  Learning  to  teach  in  higher  education.  London:  Routledge. 

Reeves,  T.  (1992,  September-October).  Effective  dimensions  of  interactive  learning  systems.  In  A.  Holzl  & D. 
Robb  (Eds.),  Finding  the  future:  ITTE  '92,  Proceedings  of  the  Information  Technology  for  Training  and 
Education  Conference.  Lucia,  Brisbane:  University  of  Queensland. 

Acknowledgements 

The  authors  wish  to  thank  the  content  writers.  Dr  Norm  Eizenberg,  Dr  Chris  Briggs,  Dr  Ivica  Grkovic  and  Ms 

Priscilla  Barker,  and  the  Committee  for  the  Advancement  of  University  Teaching  (CAUT),  the  Teaching  and 

Learning  (Multimedia  and  Educational  Technologies)  Committee  (The  University  of  Melbourne),  for  funding. 


517 


Web  Raveler 

An  Infrastructure  for  Transforming  Hypermedia 


Andrew  Kensler  and  Samuel  A.  Rebelsky 

Glimmer:  The  Grinnell  Laboratory  for  Interactive  Multimedia  Experimentation  and  Research 
Department  of  Mathematics  and  Computer  Science 
Grinnell  College 
United  States 
rebelsky@grinnell.edu 


Abstract:  Adaptive  hypermedia  has  the  potential  to  change  learning  in  an  online 

environment.  Interactive  hypermedia  systems,  in  which  students  can  not  only  add  their  own 
pages  and  links  but  also  modify  existing  pages,  provide  the  potential  to  further  support 
students.  Yet  the  World  Wide  Web  fails  to  live  up  to  these  potentials  because  the  core  Web 
technologies  lack  uniform  support  for  adaptation  or  interaaion. 

Web  Raveler  is  an  infrastructure  that  supports  uniform  extension  and  adaptation  of  Web 
pages.  Web  Raveler  mediates  the  dialog  between  Web  clients  and  Web  servers,  transforming 
requests  and  responses  so  that,  for  example,  faculty  can  make  standard  modifications  to 
arbitrary  pages,  from  adding  annotation  facilities  to  presenting  pages  in  a form  appropriate  for 
the  reader.  Web  Raveler  is  also  extensible;  by  developing  new  plug-ins,  faculty  can  add 
functionality  to  arbitrary  pages. 

In  this  paper,  we  discuss  the  architecture  of  Web  Raveler  and  suggest  some  applications. 


1.  Background:  Why  Transform  Pages? 

Initial  visions  of  hypertext  emphasized  a highly  interactive  information  system,  in  which  readers  could  annotate, 
contribute  to,  share,  and  link  pages  in  the  hypertext  (Bush  1945)  (Nelson  1974)  (Landow  1997).  Unfortunately, 
the  World  Wide  Web  provides  a more  passive  model  of  hypertext,  as  most  pages  and  browsers  limit  readers  to 
scanning  through  pages  and  clicking  on  links.  While  newer  technologies  (e.g.,  Java  applets,  browser  plug-ins, 
and  even  CGI  scripts)  can  provide  more  interactivity,  such  interactivity  is  at  the  control  of  the  page's  original 
authors,  and  out  of  control  of  the  readers.  Compare  this  with  Nelson's  original  description  of  hypertext,  in 
which  he  writes  “links  may  be  artfully  arranged  according  to  meaning  or  relations  in  the  subject,  and  possible 
tangents  in  the  reader's  mind”  (Nelson  1974,  p.  DM19,  emphasis  ours).  To  support  such  interactivity,  readers 
need  a way  to  transform  existing  pages  to  add  these  new  facilities. 

Such  facilities  can  allow  students  to  interact  with  each  other  in  an  collaborative  manner.  An  annotation 
system  for  example  could  let  students  post  notes  and  carry  out  on-line  discussions  in  the  context  of  a course 
web.  The  instructor  might  post  materials  on-line  and  students  would  attach  comments  to  specific  points  or 
items  on  the  page.  Other  students  or  the  instructor  could  then  respond  to  them  in  the  same  manner.  Students 
divided  into  groups  might  post  notes  visible  only  to  other  students  in  the  same  group.  The  system  would 
customize  the  page  for  each  individual  who  views  it. 

At  the  same  time,  faculty  are  looking  for  ways  to  use  the  Web  to  provide  different  perspectives  on  the  same 
material,  letting  the  Web  site  adapt  to  the  needs,  background,  or  usage  patterns  of  their  students,  e.g., 
(Brusilovsky  1 996).  Computers  can  adapt  the  pages  to  the  user,  customizing  them  specifically  to  suit  the  person 
viewing  the  page.  This  can  be  particularly  powerful  in  the  case  of  education.  Adaptive  hypermedia  is  much 
more  powerful  because  it  can  present  information  to  the  student  in  the  way  that  is  most  effective  for  them  and 
allows  them  to  learn  best.  For  example,  an  adaptive  lesson  might  choose  different  examples  based  on 
knowledge  of  the  background  of  the  student,  or  suggest  different  resources  depending  on  which  resources 
students  have  viewed  in  the  past. 

Systems  that  can  customize  the  presentation  of  the  material  for  individual  users  can  significantly  improve 
the  experience.  In  the  case  of  course  web  sites  for  education,  they  can  adapt  the  material  to  the  mode  of 
learning  that  best  suits  the  student,  thereby  making  the  learning  experience  much  more  engaging. 
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Adaptation  may  also  mean  adapting  existing  pages.  That  is,  a teacher  might  select  portions  of  other  pages, 
add  comments  to  other  pages  to  help  students  use  those  pages,  or  even  tie  together  materials  from  multiple 
pages.  As  (Landow  1997)  suggests,  students  might  also  engage  in  scholarly  activity  not  by  writing  new  words, 
but  by  creating  new  associations  (links)  between  existing  pages. 

These  are  three  somewhat  different  domains:  adding  interactive  facilities  to  individual  pages,  writing  pages 
that  adapt,  and  adapting  existing  pages.  However,  they  share  some  key  commonalties.  In  each  case,  pages  must 
be  modified  according  to  some  rules  (to  add  facilities;  to  change  the  page  according  to  information  on  the  reader 
or  student;  to  change  the  page  according  to  specifications  from  the  modifier).  In  each  case,  modification 
requires  some  knowledge  about  the  reader,  some  of  which  are  tied  to  individual  pages  (desired  facilities  and  a 
collection  of  changes  made  through  those  facilities,  such  as  notes  on  the  pages;  browsing  histoiy,  education 
level,  other  preferences;  specifications  for  modifying  pages). 

The  Web  has  the  potential  to  provide  a richer,  more  interactive,  and  more  uniform  environment  for  students 
and  teachers.  But  it  needs  standard  mechanisms  for  modifying  pages  according  to  information  about  the  reader. 


2.  A Solution;  Web  Raveler 

To  realize  this  potential,  we  have  constructed  an  infrastructure  for  transforming  hypermedia.  Web  Raveler  is  a 
Web  page  modification  system  designed  to  transform  hypermedia  on  the  fly.  It  sits  between  a user’s  Web 
browser  and  the  Web  servers  on  the  Internet,  transforming  pages  and  resources  as  they  go  through.  It  supports 
multiple  simultaneous  users  and  keeps  individual  profiles  so  that  it  can  customize  pages  differently  for  each 
user.  Web  Raveler  runs  each  page  through  a series  of  plug-ins  that  modify  the  page  based  on  the  context  of  the 
page  and  the  users  profile.  This  transformed  page  is  then  what  it  returns  for  the  browser  to  display  to  the  user. 

Web  Raveler  is  simple,  powerful,  flexible,  transparent,  and  extensible.  Its  power  stems  from  it  simplicity; 
it  has  one  purpose,  to  intercept  pages  and  apply  transformations  to  those  pages.  Since  it  does  not  limit  the  kinds 
of  transformations  that  can  applied,  it  can  be  used  for  a host  of  applications,  from  adding  new  features  to  pages 
to  extracting  text  from  pages. 

Web  Raveler  is  also  flexible  in  that  can  be  used  with  existing  tools  and  course  webs  since  it  filters  the 
pages  as  the  travel  through.  Instructor  can  continue  using  whatever  tools  they  are  most  comfortable  with  for 
building  their  course  webs  and  easily  add  Web  Raveler  to  those  course  webs.  This  system  can  also  adapt  pages 
outside  of  the  domain  of  the  course  web;  it  can  actually  transform  any  page  available  over  the  World-Wide 
Web.  Thus  users  are  not  restricted  to  a small  domain  but  can  use  it  to  enhance  students'  overall  experiences  no 
matter  where  they  are  on  the  Web. 

Furthermore,  Web  Raveler  is  transparent.  There  is  some  minor  initial  setup  involved  with  configuring  the 
Web  browser  to  use  it,  but  once  this  is  done,  the  users  can  browse  the  Web  at  will  using  the  system.  Everything 
is  accessible  and  there  are  no  funny  URLs  to  bother  with.  Readers  can  simply  use  the  Web  browser  as  they 
always  do,  with  no  extra  effort  beyond  the  initial  configuration. 

Finally,  Web  Raveler  supports  a plug-in  architecture  to  provide  the  infrastructure  for  new  applications. 
Each  plug-in  interacts  with  the  Web  Raveler  Server  through  a simple  interface.  Web  Raveler  takes  care  of  the 
details,  from  communication  with  the  Web  browser  and  the  Web  server  to  storing  user  preferences,  accounts, 
and  other  information.  It  simply  passes  each  page  to  the  appropriate  plug-ins  along  with  the  data  on  the  user 
and  allows  them  to  modify  the  page  freely.  Thus  applications  are  only  concerned  with  doing  their  specific 
adaptations,  leaving  the  mechanics  of  the  system  to  the  Web  Raveler  system. 

This  architecture  allows  the  Web  Raveler  system  to  provide  a wide  range  of  services  to  the  user,  while  not 
restricting  them  in  any  way. 

3-  The  Architecture  of  Web  Raveler 

Three  main  components  comprise  the  Web  Raveler  system:  the  Web  Raveler  Account  Server  (WRAS),  the  Web 
Raveler  Server,  (WRS)  and  a group  of  plug-ins.  (Fig.  1)  illustrates  the  components  and  their  relationships. 

The  WRS  is  a standard  HTTP  proxy  server.  When  the  Web  browser  needs  a page,  image  or  other  resource, 
it  sends  an  HTTP  request  to  the  WRS  (step  1).  The  WRS  checks  to  see  if  the  request  is  from  a user  who  is 
already  logged  in.  If  the  user  is  not  logged  in,  then  it  sends  back  an  authentication  response  that  the  browser 
interprets  by  requesting  a user  name  and  password  of  the  user.  Then  the  browser  resends  the  request,  this  time 


with  the  request  bearing  the  user  name  and  password  of  the  user.  When  the  WRS  gets  this  information,  it  then 
passes  it  over  the  Internet  to  the  WRAS  (step  2).  This  server  checks  to  make  sure  that  the  password  is  correct 
for  that  user,  and  if  so,  sends  back  the  data  for  that  users  account  (step  3).  Otherwise,  it  sends  a response  back 
to  the  WRS  indicating  that  the  login  failed,  and  the  WRS  sends  back  a response  to  the  Web  browser  for  the  user 
to  try  to  log  in  again. 

Once  the  user  has  logged  in,  the  WRS  passes  along  the  request  for  the  page  or  image  to  the  source  Web 
server  (step  4),  which  actually  has  the  page.  The  Web  server  retrieves  the  requested  resource  from  disk  and 
sends  it  back  over  the  Internet  to  the  WRS  (step  5).  Since  Web  Raveler  uses  HTTP,  the  standard  Web 
communication  protocol,  it  can  pass  requests  to  any  Web  server. 

After  it  has  received  the  page,  the  WRS  examines  the  users'  settings,  including  specific  options  for  each 
plug'in  and  what  groups  the  user  belongs  to.  It  also  examines  the  page  for  any  embedded  instructions  regarding 
plug'ins.  WRS  then  decides  which  of  the  available  plug-ins  should  be  applied  the  page  and  in  what  order.  It 
executes  each  of  the  plug-ins  in  turn  on  the  page,  giving  them  the  page  and  the  information  on  the  user  (steps 
6a,7a).  Each  plug-in  may  do  anything  that  it  wants  with  the  data,  typically  transforming  the  page  and  returning 
the  transformed  page  (steps  6b,7b).  Each  plug-in  runs  in  turn  on  the  page  and  then,  after  all  have  been 
executed,  the  WRS  returns  the  page  in  its  final  form  to  the  Web  Browser  (step  8),  which  then  renders  it  for  the 
user. 

Note  that  pages  (and,  therefore,  their  authors),  and  not  just  readers,  can  specify  which  plug-ins  to  run.  This 
facility  gives  authors  some  control  over  what  can  and  cannot  be  done  to  their  pages  (since  many  authors  may 
not  want  to  permit  some  transformations  to  be  applied  to  their  pages).  It  also  provides  an  infrastructure  for 
adaptive  hypertext,  in  that  authors  can  specify  that  particular  adaptations  be  applied. 
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Figure  1:  The  Architecture  of  Web  Raveler. 


All  of  the  components  of  Web  Raveler  were  written  in  Perl  and  thus  are  portable  to  a variety  of  platforms, 
including  Unix,  Linux,  Windows,  and  Macintosh.  Our  current  implementation  is  running  under  HP-UX,  and 
we  are  testing  other  platforms.  We  currently  support  Perl -based  plug-ins.  Perl  is  also  a convenient  language  to 
use  because  of  its  powerful  text-manipulation  capabilities  and  many  useful  Web-related  libraries.  These  make  it 
easy  to  write  plug-ins  with  significant  capability.  We  are  also  developing  a plug-in  architecture  based  on 
SATIRIC,  a contextual  pattern  language,  which  was  developed  specifically  for  transforming  Web  pages 
(Seaman  et  al.  2000). 


3.1.  Alternative  Implementation  Techniques 

We  have  chosen  the  standard  HTTP  proxy  server  mechanism  as  our  underlying  implementation  technique. 
There  are,  of  course,  other  mechanisms  for  transforming  Web  pages  as  they  travel  between  client  and  server. 
One  might  use  a centralized  CGI  script  that  parses  its  URL  and  selects  which  server  to  select,  as  in  (Becker  et 
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al.  1999).  One  might  rely  on  server  plug-ins,  CGI  scripts,  or  ASP  scripts,  as  is  sometimes  done  in  an  ad-hoc 
fashion  for  pages  on  a single  server.  One  might  write  browser  plug-ins,  as  in  the  case  of  ThirdVoice 
(ThirdVoice  1999).  (Tab.  1)  summarizes  selected  issues  for  each  technique. 


Type 

Browser 

Requirements 

Server 

Requirements 

Domain 

Server  CGUASP 

None 

CGI  Support 

Local  Site 

Browser  Plug-in 

Plug-in  Support 

None 

Whole  WWW 

CGI  Proxy 

None 

CGI  Support 

Whole  WWW 

HTTP  Proxy 

Proxy  Support 

(None) 

Whole  WWW 

Table  1:  Potential  Implementation  Strategies 

While  server  plug-ins  and  scripts  provide  authors  with  extensive  functionality,  that  functionality  is  typically 
limited  to  the  pages  on  the  site  or  sites  that  server  serves.  These  scripts  also  leave  control  in  the  hands  of 
authors;  readers  cannot  decide  which  transformation  to  use.  Finally,  they  may  require  cookies  or  strange  URLs 
to  keep  track  of  user  information. 

Browser  plug-ins  provide  more  freedom,  as  in  the  case  of  ThirdVoice,  a plug-in  might  be  applied  to  any 
page  on  the  Web.  However,  not  all  plug-ins  run  in  all  browsers  or  on  all  architectures,  a severe  disadvantage  in 
the  typically  heterogeneous  academic  environment.  For  example,  many  plug-ins  are  never  released  for  all 
variations  of  the  Unix  platform.  Plug-ins  may  also  be  difficult  to  disable. 

CGI  proxies  also  provide  the  opportunity  to  transform  arbitrary  pages.  However,  they  require  that  readers 
select  “alternate”  URLs.  For  example,  to  read  the  EdMedia  Web  page  through  a CGI  proxy,  a browser  might  be 
required  to  use  a URL  like 

http : //www. site . com/transformer?page=http : //www. aace . org/conf /edmedia/ 

Such  URLs  can  confuse  novice  and  experienced  readers  alike.  CGI  proxies  typically  require  cookies  or  even 
more  complicated  URLs  to  keep  track  of  the  current  user. 

HTTP  proxies,  our  chosen  technique,  provide  all  of  the  advantages  of  the  other  techniques,  with  few  of  the 
drawbacks.  Readers  employ  standard  URLs.  The  standard  proxy  protocol  permits  the  server  to  request  a 
password  and  the  browser  sends  account  information  “automatically”.  Proxies  support  arbitrary  URLs,  and  so 
allow  authors  to  transform  arbitrary  pages. 


4.  Current  Applications  of  Web  Raveler 

The  Web  Raveler  system  provides  an  infrastructure  for  the  development  of  multiple  adaptive  hypermedia 
applications.  In  addition  to  mediating  between  them,  it  handles  the  details  of  communication  over  the  Internet 
with  the  Web  browser  and  the  Web  server.  It  also  keeps  track  of  the  user  settings  and  preferences.  The  plug-in 
architecture  makes  it  trivial  to  develop  new  applications  and  add  them  to  the  Web  Raveler  system  without  the 
need  to  modify  the  Web  Raveler  program.  Once  plug-ins  are  placed  in  the  appropriate  directory,  they 
automatically  become  active  the  next  time  the  Web  Raveler  Server  is  started. 

The  first  plug-in  developed  for  Web  Raveler  was  a shared  Web  annotation  system,  an  update  of  (Luebke  et 
al.  1999).  This  system  allows  user  to  attach  comments  to  arbitrary  Web  pages  and  set  different  levels  of  access 
on  them.  This  system  allows  students  to  annotate  pages  in  a course  web  as  well  as  linked  external  materials. 
Students  divided  into  the  different  groups  can  mark  their  annotations  as  being  visible  only  to  members  of  their 
group,  visible  to  the  whole  class,  completely  public,  or  private  to  themselves. 

Another  application  currently  under  development  is  a trailblazing  system  (Glynn  et  al.  2000).  This  system 
allows  teachers  to  record  a series  of  Web  pages,  add  notes  to  the  pages  and  present  them  in  a coherent  “trail” 
that  students  can  follow  in  the  prescribed  order.  If  a student  visits  a page,  and  that  pages  is  on  a trail  the  teacher 
has  created,  the  system  notifies  the  student  about  the  availability  of  the  trail. 

A third  application  is  a tracking  system.  A previous  version  of  this  was  done  using  a CGI  proxy  system 
(Becker  et  al.  1999).  However,  Web  Raveler  affords  many  advantages  for  this.  The  tracking  system  will 
silently  track  where  a user  goes,  recording  how  long  they  spend  on  each  page,  which  links  they  follow  and  so 
forth.  This  information  can  then  be  collated  to  evaluate  the  effectiveness  of  a Web  site.  In  an  educational 


context,  instructors  can  see  how  the  different  resources  on  a course  Web  site  are  being  used  and  can  take 
advantage  of  this  to  make  the  course  Web  site  more  effective. 

Finally,  we  have  just  begun  work  on  the  adaptation  plug-in  that  permits  authors  to  write  pages  that  can  take 
advantage  of  the  information  Web  Raveler  gathers. 


5.  Related  Work 

There  are  many  other  filtering  systems  available,  written  in  different  languages,  emphasizing  different 
transformations,  or  using  different  technologies.  These  include  OreO  (Brooks  et  al.  1995),  GrAnT  (Schickler  et 
al.  1996),  Muffin  (Boyns  1998),  and  part  of  Project  Clio  (Becker  et  al.  1999).  The  Web  Raveler  system  is 
unique  in  supporting  user  accounts  and  preference  data,  so  that  different  readers  may  select  different 
transformations.  Ours  is  also  unique  in  that  we  support  author-specified  transformations,  in  addition  to  reader- 
specified  transformations. 

There  are  also  other  systems  available  that  attempt  to  ease  the  construction  of  interactive  course  webs,  most 
notably  WebCT  and  First  Class.  WebCT  primarily  focuses  on  the  creation  of  course  webs.  It  does  have 
support  for  annotations  and  on-line  quizzes  and  exams.  It  also  provides  some  forms  of  tracking.  First  Class  is  a 
client/server  that  primarily  provides  services  to  run  a threaded  discussion  forum.  Instructors  can  create  message 
areas  and  then  students  can  post  to  these. 

These  systems  however  still  lack  the  true  power  of  transformational  hypermedia.  They  are  limited  to  their 
own  domain  and  cannot  customize  pages  outside  of  the  course  systems  constructed  using  them.  They  also 
restrict  the  instructor  to  using  their  tools  to  construct  the  sites.  Preferably  an  instructor  would  create  course 
materials  using  whatever  tool  they  are  most  comfortable  with. 


6.  Conclusions,  Future  Directions,  and  Other  Issues 

Successful  educational  use  of  the  Web  requires  more  than  existing  tools  provide;  it  requires  uniform  systems  for 
annotating  and  transforming  Web  pages,  techniques  for  writing  adaptive  pages,  and  systems  for  adapting 
existing  resources.  Web  Raveler  provides  a uniform  system  for  all  three  applications,  empowering  teachers  to 
build  better  sites  and  students  to  interact  more  freely  with  the  Web. 

The  Web  Raveler  system  is  currently  being  used  as  the  foundation  for  several  adaptive  hypermedia  systems 
that  have  been  implemented  as  plug-ins.  In  the  fall  of  2000,  the  Department  of  Math  and  Computer  Science  at 
Grinnell  College  will  begin  class  testing  with  the  Web  Raveler  system  and  these  plug-ins.  As  suggested  earlier, 
work  on  a number  of  plug-ins  has  begun. 

There  are  certainly  many  other  issues  we  must  consider,  including  a uniform  language  for  page  authors  to 
limit  and  specify  transformations,  a more  user-friendly  language  for  writing  transformations,  and  a language  for 
writing  adaptive  pages  that  use  Web  Raveler.  We  have  not  yet  measured  the  overhead  introduced  by  Web 
Raveler  and  its  plug-ins.  We  expect  to  report  on  some  of  these  developments  and  experiments  at  a future 
EdMedia  conference. 
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Abstract:  INSPIRE  is  a web-based  negotiation  support  system  that  has  been  used  for  the  last  three 
years  for  negotiation  training  and  research.  It  has  also  been  used  in  teaching  written  communication 
skills.  In  this  paper  we  discuss  implications  of  using  the  system  in  teaching  English  as  a Second 
Language  and  English  for  Academic  Purposes  in  multilingual  classes  and  point  out  the  benefits  that 
this  new  technology  offers.  We  also  show  how  extending  classroom  boundaries  to  remote  regions 
of  the  world  add  to  increased  students’  involvement  and  enhance  language  acquisition. 


Introduction 

Web  provides  people  in  different  locations  and  time  zones  with  a communication  medium  that  is  rich  in 
functionality  and  content  and  which  offers  them  the  ability  to  use  previously  inaccessible  computational  resources. 
While  the  Web  is  currently  used  as  a powerful  source  for  the  dissemination  of  information,  it  is  increasingly  being 
used  as  a means  for  active  communication,  and  remote  execution  and  control  of  complete  software  systems,  thus 
adding  another  dimensions  to  the  value  it  delivers.  In  education,  its  ability  to  access  and  run  remote  programs  and 
databases  allows  users  to  extend  classroom  and  laboratory  boundaries  across  geographical  and  time  zones. 

The  InterNeg  Web  site  and  its  Web-based  system  INSPIRE  have  been  constructed  to  exploit  these  tech- 
nologies and  their  use  in  teaching  [4-6].  They  aim  at  providing  people  around  the  world  with  analytical  knowledge 
and  decision  support  techniques  within  the  domain  of  negotiations.  The  INSPIRE  system  allows  participants  to 
analyze  and  solve  real-like  decision  problems  and  conduct  negotiations  with  people  from  different  cultures. 

The  INSPIRE  system  is  the  first  Web-based  negotiation  support  system.  It  is  based  on  analytical  models 
rooted  in  decision  and  negotiation  analysis  [7-8].  Developed  in  the  context  of  a cross-cultural  study  of  decision- 
making and  negotiation,  the  system  has  been  primarily  used  to  conduct  and  study  negotiation  via  the  World  Wide 
Web  as  well  as  in  teaching  information  systems,  management  science,  international  management  and  English  as  a 
Second  Language  (ESL)  [4,  9]. 

In  this  paper  we  discuss  the  use  of  the  INSPIRE  system  in  the  context  of  ESL  teaching,  and  the  teacher's 
and  students'  experiences.  In  Section  2 the  system,  the  available  resources  and  the  negotiation  case  are  outlined. 
Section  3 discusses  the  context  and  the  teaching  methodology  used  to  adapt  the  system  for  ESL  purposes.  The  im- 
plications for  teaching  communication  skills,  and  the  students'  and  teachers'  experiences  are  discussed  in  Section  4. 
Comments  about  INSPIRE's  usefulness  for  ESL/EAP  teaching  and  the  ongoing  work  on  new  Web-based  resources 
conclude  the  paper. 

InterNeg  site  and  the  INSPIRE  system 

The  InterNeg  project  began  in  1996  with  the  development  of  a simple  Web-based  negotiation  support  sys- 
tem called  INSPIRE  (InterNeg  Support  Program  for  Intercultural  Research;  httr)://intemeg.org/intemegL  [6].  During 
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the  last  three  years  over  3,500  students  used  the  system.  The  host  InterNeg  site  offers  services  (see 
http://intemeg.org/),  including  beginners’  handouts  and  information  on  how  INSPIRE  can  be  used  in  different  uni- 
versity courses.  Research  papers,  summaries,  examples  of  students’  assignments,  a glossary,  an  extensive  bibliogra- 
phy and  information  about  programs  and  organizations  involved  in  negotiation  are  also  available. 

INSPIRE  views  a negotiation  as  a process  occurring  in  a particular  context.  The  system  does  not  act 
autonomously  like  a third  party  arbitrator;  rather  each  ’copy’  acts  solely  to  support  a single  negotiator.  It  supports 
asynchronous  negotiations,  thus  ameliorating  the  time  zone  problem.  To  facilitate  this  type  of  negotiation  the  system 
saves  the  current  state  resulting  from  each  user’s  actions  in  a form  that  can  be  retrieved  when  the  counterpart  logs  in 
some  time  later, 

INSPIRE  negotiation  phases 

INSPIRE  negotiations  follow  three  phases:  an  antecedent  phase,  a concurrent  phase,  and  a consequent 
phase  [6].  The  pre-negotiation  phase  involves  an  analysis  of  the  situation,  problem  and  opponent,  specification  of 
preferences,  reservation  levels,  and  strategy.  In  this  phase  users  read  the  case,  formulate  their  preferences  using  sim- 
ple tables  to  distribute  scores  between  issues  and  their  values,  verify  the  utility  (ratings)  of  selected  offers  that  are 
generated  by  the  system  and  fill  in  a pre-negotiation  questionnaire.  The  questionnaire  contains,  among  others,  ques- 
tions about  the  expected  compromise  and  the  worst  offer  that  the  user  is  willing  to  accept. 

The  negotiation  phase  involves  exchanges  of  messages  and  offers,  evaluation  of  offers  and  the  assessment 
of  the  progress  of  the  negotiation.  Offers  comprise  the  negotiated  issues  and  their  values;  each  offer  is  formulated  by 
selecting  one  value  for  each  issue.  The  offer  rating  is  automatically  displayed  beneath  the  table  which  contains  the 
offer.  An  offer  may  be  accompanied  by  a message.  Users  may  also  send  messages  without  offers;  this  feature  is  of- 
ten used  to  prompt  their  counter-parts  by  making  an  additional  argument.  On  a number  of  occasions  users  sent  mes- 
sages to  communicate  with  their  counter-parts  about  topics  outside  of  the  negotiations;  in  fact  a number  of  users 
maintained  e-mail  communication  with  their  counter-parts  after  the  negotiations  were  completed. 

Users  have  two  mechanisms  to  assess  the  progress  of  the  negotiation.  They  may  review  the  complete  his- 
tory of  their  negotiations,  including  offers  exchanged,  their  ratings  and  the  messages.  They  can  also  access  a graph 
that  displays  the  negotiation  dynamics  in  their  utility  space.  The  negotiation  is  parallel  on  all  issues.  Participants 
may  submit  the  same  offer  many  times,  or  keep  the  option  of  an  issue  unchanged,  but  each  submitted  offer  contains 
a value  for  each  issue. 

The  negotiation  ends  when  either  a compromise  has  been  achieved,  or  one  of  the  parties  terminates  the  pro- 
cess, or  the  time  runs  out  (at  a predetermined  deadline).  The  post-settlement  phase  involves  the  evaluation  of  the 
negotiation  outcomes  generated  by,  and  after,  the  negotiation  activity.  These  outcomes  include  the  information  about 
the  compromise.  If  the  negotiators  can  improve  their  compromise,  the  INSPIRE  provides  them  with  up  to  five  Pa- 
reto-optimal offers.  In  this  situation  the  users  may  choose  to  continue  their  negotiations.  The  post-settlement  phase 
ends  with  filling  in  the  post-negotiation  questionnaire,  which  however,  is  not  mandatory. 

Itex-Cypress  Negotiations 

The  negotiation  problem  involves  two  companies:  Itex  Manufacturing,  a producer  of  bicycle  parts  and  Cy- 
press Cycles  that  builds  bicycles.  Cypress  Cycles,  an  established  manufacturer  of  high  quality  mountain  bikes,  is 
launching  a new  line  of  bikes  and  requires  a type  of  component  that  its  current  suppliers  cannot  provide.  Their  first 
serious  discussions  for  the  supply  of  these  components  are  being  held  with  Itex  Manufacturing.  Itex  is  seeking  to 
increase  its  share  of  the  component  market  and  would  like  to  have  the  prestige  that  would  come  with  supplying  Cy- 
press, should  a profitable  contract  be  signed. 

There  are  four  issues  that  both  sides  have  to  discuss.  The  issues  are:  the  price  of  the  components,  delivery 
times,  payment  arrangements,  and  terms  for  the  return  of  defective  parts.  INSPIRE  users  are  asked  to  negotiate  on 
behalf  of  the  company  rather  than  for  themselves  and  told  that  their  companies  are  interested  in  achieving  a com- 
promise. They  are  also  informed  that  there  are  other  suppliers  and  buyers  so  a breakdown  in  negotiations  is  possible, 
if  a good  deal  cannot  be  reached.  There  is  no  further  indication  as  to  what  constitutes  a good  deal.  Each  side,  how- 
ever, is  given  a clear  indication  as  to  the  desirability  of  the  options  (issue  values)  but  only  in  terms  of  the  direction 
and  not  specific  trade-off  values.  By  avoiding  the  specification  of  preference  values,  negotiators  are  able  to  establish 
their  own  priorities  within  each  issue. 

In  writing  the  case  an  effort  had  been  made  to  make  it  as  much  as  possible  ’culture  neutral’.  The  negotia- 
tion situation  is  of  the  type  that  can  be  encountered  anywhere  in  the  world.  Also  the  negotiators’  names  (which  can 
usually  indicate  their  cultural  background)  are  substituted  by  pseudonyms.  The  case  is  fairly  simple  and  well  struc- 
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tured.  It  describes  a negotiation  problem  that  is  familiar  to  users  from  almost  any  country,  therefore  no  additional 
explanations  are  necessary.  Language  proficiency  of  the  users  was  taken  into  consideration  too.  The  comprehen- 
sion of  the  case  was  tested  on  a group  of  students  taking  Intermediate  English  as  a Second  Language  for  Academic 
Purposes  course  at  Carleton  University. 

The  description  of  the  case  fits  one  and  a half  pages.  Generally,  users  find  the  system  very  easy  to  use,  and 
their  evaluation  of  the  overall  system  is  favourable.  Over  75%  of  INSPIRE  users  stated  that  they  would  use  a system 
like  INSPIRE  in  real-life  negotiations  and  over  85%  would  use  such  a system  to  prepare  themselves  to  conduct  ac- 
tual negotiations.  While  the  feedback  on  the  INSPIRE  system  confirmed  our  expectations,  the  absolute  levels  of  user 
acceptance  of  the  system  are  surprisingly  high. 

EAP/ESP  context 

The  students 

INSPIRE  has  been  used  at  Carleton  University  to  teach  foreign  language  learners  communication  skills. 
The  School  of  Linguistics  and  Applied  Language  Studies  offers  two  English  as  a Second  Language  Programs.  In  the 
Intensive  ESL  Program  the  majority  of  the  student  population  are  international  students.  Some  are  either  planning  to 
enter  a regular  study  program  at  an  English  speaking  university  in  North  America  others  learn  English  to  gain  a 
competitive  advantage  at  the  job  market  in  their  countries.  The  Credit  ESL  Program  on  the  other  hand,  is  a language 
support  program  for  those  students  who  have  already  been  accepted  to  Carleton  on  the  condition  that  within  a year 
they  receive  a B-  exit  grade  in  the  advanced  course  of  English. 

There  are  three  language  proficiency  levels  in  the  Credit  Program:  Introductory  English  as  a Second  Lan- 
guage for  Academic  Purposes,  Intermediate  English  as  a Second  Language  for  Academic  Purposes  and  Advanced 
English  for  Academic  Purposes.  The  Advanced  course  has  two  options,  i.e.  general  EAP  and  English  for  Special 
Purposes  (ESP)  for  Academic  Purposes  for  Engineering  Students.  The  highest  level  of  the  Intensive  Program  and 
the  introductory  course  of  the  Credit  Program  are  in  fact  the  same  course  in  terms  of  the  level  of  language  profi- 
ciency. The  difference  is  the  students'  status — visa  students  (Intensive)  versus  permanent  residents  (Credit). 

Students  enrolled  in  both  programs  come  from  all  over  the  world  and  have  a variety  of  educational  back- 
grounds ranging  from  new  high  school  graduates  to  professionals.  The  majority  of  the  students  however,  are  in  their 
twenties  and  have  a couple  of  years  of  college  or  university  training  (particularly  those  in  the  Intensive  Program), 
Over  the  last  three  years,  the  INSPIRE  system  has  been  used  with  the  students  enrolled  the  writing  workshop 
at  the  highest  level  of  the  Intensive  Program  and  in  the  Credit  Program:  Intermediate  English  for  Academic  Purposes 
and  Advanced  English  for  Academic  Purposes  (both  the  general  EAP  and  EAP/ESP  for  engineering  students). 

Whole  language,  communication  and  negotiations 

The  presented  language  teaching  approach  is  embedded  in  the  theory  of  whole  language.  It  is  based  on  the 
belief  that  development  of  cognitive  skills  is  interrelated  with  the  development  of  language  skills;  that  critical 
thinking  and  analysis  should  constitute  an  integral  part  of  learning;  that  we  learn  language  to  communicate  with  oth- 
ers and  therefore  social  context  is  important;  that  the  learning  objectives  are  best  achieved  when  there  is  purpose  and 
meaning;  and  that  students'  direct  engagement  and  experience  are  crucial  to  the  learning  process. 

The  INSPIRE  system  lends  itself  to  the  theory  and  practice  of  whole  language.  While  analyzing  the  nego- 
tiation process  the  students  develop  both  the  cognitive  and  the  language  skills.  The  dynamic  in  its  nature,  relation- 
ship between  negotiation  partners  provides  social  context  for  communication;  the  role-play  which  evolves  around 
finding  a solution  to  a conflict  of  interest  situation  gives  purpose  and  meaning  to  the  communication  that  takes  place 

Negotiations  require  extensive  use  of  communication  skills.  Although  negotiations  on  the  Web  do  not  al- 
low for  the  use  of  non-verbal  clues  (at  least  not  yet),  they  stimulate  the  precision  of  expression  in  the  written  form 
allowing  the  students  to  work  on  developing  their  writing.  The  written  messages  can  be  carefully  thought  over  and 
revised  many  times  until  the  student  feels  that  what  he/she  has  written  represents  adequately  his/her  point  of  view. 
While  writing  these  messages  the  students  learn  how  to  persuade,  argue,  convince,  etc.  They  also  learn  to  read,  un- 
derstand and  follow  instructions  which  are  an  integral  part  of  the  INSPIRE  system.  The  speaking  skills  are  devel- 
oped through  classroom  discussions  about  the  progress  of  individual  negotiations,  negotiating  strategies  and  tech- 
niques. While  negotiations  are  conducted  outside  of  the  classroom,  these  discussions  take  place  during  each  class 
for  the  duration  of  negotiations. 
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The  INSPIRE  negotiations  lend  themselves  to  the  thematic  approach  to  language  teaching.  All  the  lan- 
guage activities  evolve  around  the  topic  of  negotiations,  decision-making  and  conflict  resolution.  The  conflict  that  is 
at  the  core  of  the  negotiation  keeps  the  students  intellectually  and  emotionally  involved.  And  although  the  negotia- 
tion case  is  only  an  assigned  task,  the  communication  that  occurs  is  authentic. 

The  negotiations  take  about  two  weeks  during  which  time  the  students  are  constantly  exposed  to  the  lan- 
guage of  business  negotiations  used  in  a specific  context  for  a specific  purpose.  The  role-playing  situation  and  the 
fact  that  there  is  a real  person  on  the  other  side  of  the  negotiation  "table**  keeps  the  students  committed  to  the  task. 
The  level  of  difficulty  of  tasks  and  language  activities  grows  gradually,  and  roughly  corresponds  to  the  three  phases 
of  the  negotiation;  pre-negotiation,  negotiations  and  post-negotiation.  In  the  pre-negotiation  phase  the  students  write 
and  share  their  past  personal  experiences  with  negotiations,  read  the  INSPIRE  case  and  acquaint  themselves  with 
their  role,  read  and  discuss  the  example  (this  is  when  the  vocabulary  is  introduced  in  a context),  listen  to  a lecture  on 
the  INSPIRE  system  and  take  notes  (they  will  need  the  notes  later  to  write  the  introduction  to  their  report  or  essay), 
rank  the  negotiation  issues  and  options  and  fill  in  the  pre-negotiation  questionnaire. 

During  the  negotiation  the  students  are  required  to  keep  a Negotiation  Journal  where  they  record  all  nego- 
tiation related  activities,  analyze  the  situation,  strategies,  techniques  and  the  dynamics,  predict  their  opponents  be- 
haviour, etc.  They  also  share  their  experiences  with  their  classmates  on  a regular  basis,  and  engage  in  language  ac- 
tivities such  as  discussing  the  language  of  e-mails,  sentence  combining,  punctuation  exercises,  and  play  word  games. 
This  is  also  when  the  students  start  writing  the  introduction  and  the  methodology  sections  of  their  report. 

In  the  post-negotiation  phase  they  fill  in  the  post-negotiation  questionnaire,  evaluate  their  negotiations, 
their  opponents,  and  themselves.  Finally,  they  write  a report  or  an  essay  in  which  they  evaluate  their  negotiation 
experience  using  the  information  that  was  made  available  to  them  via  the  INSPIRE  negotiation  support  system, 
through  in-class  discussions  and  the  Negotiation  Evaluation  Questionnaire. 

Organization 

Before  the  negotiation  begins,  a brief  presentation  about  INSPIRE  is  made.  It  is  made  clear  at  the  beginning 
of  the  course  that  INSPIRE  is  an  important  module  of  the  course.  However,  it  is  also  stated  that  the  final  result  of 
their  negotiation  (compromise  or  not)  and  its  utility  score  are  not  used  for  grading  purposes.  This  is  important  be- 
cause the  participants  should  be  able  to  negotiate  in  as  realistic  a situation  as  possible  and  be  in  control  of  the  proc- 
ess. Students  may  decide  to  break  their  negotiations  with  one  company  and  initiate  new  negotiations  with  another. 

A demonstration  session  follows  the  presentation  and  it  is  intended  to  familiarize  the  students  with  the 
Web,  and  INSPIRE.  They  are  shown  how  to  log  on  to  INSPIRE,  how  to  construct  and  send  offers  using  the  system, 
and  how  to  incorporate  changes  in  any  of  the  offers  or  issues  in  subsequent  visits.  The  session  usually  lasts  an  hour. 
The  instructor  uses  hard  copies  of  the  forms  used  in  the  INSPIRE  negotiation  so  that  the  participants  can  actually  see 
what  kinds  of  forms  they  will  fill  in. 

During  this  session  students  log  in  and  read  the  INSPIRE  case.  At  this  time  they  may  conduct  initial  ana- 
lytical activities:  specify  the  relative  importance  of  each  issue  and  the  options  for  the  issues.  This  information  is  used 
to  determine  their  subjective  utilities  for  all  possible  offers.  In  many  cases  the  session  ends  with  the  student  making 
a first  offer  to  his  /her  counterpart.  Before  the  session  ends,  the  participants  are  reminded  that  they  will  be  notified 
by  INSPIRE  via  e-mail  whenever  a message  or  offer  from  the  counterpart  is  received  by  the  system.  When  they  re- 
ceive notification  the  participants  log  into  INSPIRE  to  read  and  evaluate  the  offer  and  submit  a counter-offer. 


Experiences 

Students*  experiences 

Students  were  requested  to  record  every  activity  they  undertook  related  to  their  negotiation  via  INSPIRE. 
These  logs  were  used  for  personal  assessment  of  each  exchange  of  offers/counteroffers  and  messages  as  well  as  for 
discussions.  They  also  helped  the  instructor  to  catch  and  address  any  problems  that  may  have  occurred. 

A few  participants  stated  that  INSPIRE  helped  them  to  see  an  intercultural  point  of  view.  A significant 
majority  said  INSPIRE  did  help  them  to  refine  their  communication  skills.  They  pointed  out  that  they  were  more 
concerned  with  language  accuracy  and  the  appropriate  level  of  formality  than  they  normally  are  while  communicat- 
ing with  their  classmates.  Those  who  had  none  or  very  limited  experience  with  computer  technology,  were  particu- 
larly appreciative  of  the  fact  that  they  were  not  only  learning  the  language  but  also  learning  a new  medium  of  com- 
munication. Despite  the  participants*  limited  experience  with  the  Internet,  their  ability  to  achieve  expected  compro- 
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mises  suggests  that  the  INSPIRE  system  and  Web-based  negotiations  do  not  introduce  a significant  burden  or  add 
complexity  to  the  already  complex  negotiation  process. 

Teacher’s  experience 

INSPIRE  required  different  preparation,  handling  and  conduct  of  the  negotiations  than  face-to-face  meth- 
ods. In  contrast  to  in-class  instruction,  teaching  negotiation  through  INSPIRE  required  first  ascertaining  the  level  of 
Internet  expertise  of  the  users.  Appropriate  training  sessions  on  the  Internet  may  need  to  be  planned  before  the  IN- 
SPIRE session  starts. 

For  a teacher  whose  class  conducts  a Web-based  negotiation  the  process  involves  preparation  and  handling 
of  three  major  stages:  (1)  introduction  to  the  system,  (2)  the  first  exchange  of  offers  and  messages,  and  (3)  discus- 
sion following  the  negotiations,  and/or  preparation  of  report  or  essay  guidelines. 

The  first  hands-on  class  has  always  been  conducted  in  the  computer  laboratory.  In  this  session,  participants 
were  guided  through  such  steps  as  logging  in,  reading  the  case,  submission  of  rating  of  issues  and  packages,  and 
finally  the  first  offer.  Deft  handling  of  varying  levels  of  experience  and  expertise  among  the  participants  is  of  crucial 
importance  here.  We  found  it  useful  to  place  one  skilled  and  a less  skilled  participant  next  to  each  other.  The  role  of 
faculty  here  is  one  of  a facilitator  and  his  presence  after  the  submission  of  the  first  offer,  was  generally  not  neces- 
sary. 

Typical  problems  that  may  arise  in  running  an  Internet  based  negotiation  are:  (1)  system  problems,  (2)  team 
problems,  and  (3)  administrative  problems.  System  problems  include  both  hardware  and  software  compatibility  is- 
sues (INSPIRE  requires  Netscape  3,  Explorer  4,  or  later  browsers)  and  systemic  problems  (including  power  shut- 
downs, and  network  problems),  which  may  occur  in  any  given  time.  To  ensure  successful  completion  of  a negotia- 
tion through  INSPIRE  instructors  need  to  plan  and  develop  effective  administrative  mechanisms.  These  may  include 
identifying  a module  coordinator  from  a group  of  participants,  who  could  help  the  group  stay  focused  and  produc- 
tive during  the  negotiation.  Another  possibility  is  pairing  less  computer  literate  students  with  those  who  feel  very 
comfortable  with  the  new  communication  technologies.  Once  the  students  have  begun  their  negotiation  on  INSPIRE 
there  is  very  little  intervention  required  from  the  instructors. 

After  the  completion  of  the  negotiations,  participants  usually  want  to  compare  their  results.  The  analysis  of 
experiences  can  be  done  in  many  ways,  i.e.  informal  group  discussions,  formal  presentations  followed  by  questions, 
formal  reports,  etc.  Individual  introspection  proved  to  be  most  useful  for  the  executive  development  programs,  while 
post-graduate  students  and  ESL  students  preferred  classroom  discussions. 

Implications  for  teaching  communication  skills 

Preparation  of  tasks  that  keep  students’  interest  for  an  extended  period  of  time  and  motivate  them  to  give 
the  best  performance  they  can  has  always  been  one  of  the  challenges  faced  by  language  teachers.  The  INSPIRE 
negotiations  have  the  elements  of  a game  and  a role  play  and  provide  the  students  with  a ’’real  counterpart”  with 
whom  they  are  going  to  exercise  their  skills  of  persuasion  and  argumentation.  The  combination  of  these  factors  con- 
tributes to  raising  students’  involvement  and  to  increased  awareness  of  the  need  to  use  appropriate  and  accurate  lan- 
guage. The  students  themselves  are  responsible  for  initiating  responses  and  for  the  outcome  of  the  negotiations. 
They  control  their  moves  and  actions.  Therefore  the  communication  that  occurs  in  form  of  email  messages  is 
authentic.  The  fact  that  the  negotiator’s  identity  is  concealed  reduces  the  intimidation  factor:  racial  and  gender  biases 
are  eliminated.  And  although  the  negotiation  may  miss  on  spontaneity,  it  certainly  allows  for  the  analysis  of  the 
situations,  for  the  preparation  of  strategies  and  tactics  and  for  revisions  of  messages.  Finally,  some  students  ex- 
change email  addresses  with  their  counterparts  and  keep  corresponding  like  the  traditional  ’’pen  pal”  friends. 

The  submission  of  offers  and  counteroffers  which  are  accompanied  by  written  messages,  the  Negotiation 
Journal,  class  discussions  and  the  use  of  the  resources  available  on  the  Web  such  as  questionnaires,  graphs  and  the 
possibility  to  check  the  history  of  the  negotiations  creates  an  environment  in  which  the  language  is  acquired.  The 
words  whose  meaning  may  have  appeared  vague  at  the  beginning  become  part  of  the  users'  active  vocabulary.  The 
language  of  argumentation  and  persuasion  develops  because  it  is  needed  to  gain  concessions. 

Conclusions 

Systems  such  as  INSPIRE  offer  an  opportunity  for  direct  participation  in  a negotiation  and  thus  create  a 
platform  for  experiential  learning  which  is  deeply  rooted  in  the  theory  and  practice  of  negotiation  training  and  in  the 
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theory  of  whole  language.  The  computer  is  a medium  of  communication  that  fosters  learners'  autonomy,  equality 
and  learning  skills.  The  opportunity  to  communicate  not  only  in  the  class  but  also  outside  of  class  creates  a new 
social  dynamic  [10],  where  the  learners  fully  control  the  process  and  the  outcomes  and  where  race,  gender  and  status 
barriers  disappear.  In  this  asynchronous  negotiation  the  learners  have  the  time  to  reflect,  analyze  and  make  as  many 
revisions  of  their  writing  as  they  may  find  necessary.  The  conflict  of  interest  embedded  in  the  negotiation  promotes 
richer  written  interaction.  All  this  creates  an  environment  where  the  cognitive  and  the  linguistic  development  are 
interrelated  [2]. 

The  flexibility  of  Web  based  systems  facilitates  customization  of  the  case  material  to  reflect  regional  spe- 
cifics. It  is  also  easier  to  bring  about  a discipline-based  orientation  in  teaching  and  training  sessions.  The  systems 
can  be  tailored  to  reflect,  say,  a behavioral,  decision  theoretic  or  any  other  focus  to  suit  local  teaching  and  training 
needs.  This  is  very  useful  for  management  teaching  and  training  where  different  modules  are  often  combined  to  re- 
flect the  particular  focus  of  a course.  Web  pages  are  very  good  at  representing  context,  and  independent  Web  pages 
may  be  assembled  by  a dispatching  system  that  determines  which  page  to  present,  based  on  a given  situation. 

The  successful  use  of  INSPIRE  in  teaching  communication  skills  in  ESL  courses  prompted  us  to  develop  a 
dedicated  Web  site  for  this  purpose  (http://intemeg.carleton.ca/interneg/training/esl/).  The  site  contains  a 16  hours 
class-time  module  and  a detailed  timeline  of  language  activities.  Over  200  students  from  Canada  and  Germany  used 
the  module.  The  materials  available  include  students'  handouts,  examples,  a glossary  of  terms,  detailed  instructor's 
notes,  description  of  students'  activities  (e.g.,  examples  of  the  Negotiation  Journal,  sentence  combining,  collabora- 
tive dictation  and  essay  guidelines),  and  writing  models.  More  information  about  the  module,  the  timeline  and  spe- 
cific activities  can  be  found  at  the  Web  site. 
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Abstract:  This  study  examines  the  effectiveness  of  a CAI  system  on  students'  academic 
achievement.  After  developing  an  experimental  hypermedia  CAI  system,  an  experiment  using 
this  system  on  a group  of  elementary  school  students  was  performed,  and  the  data  from  the 
experiment  were  then  analyzed  statistically. 

An  experimental  hypermedia  CAI  system  with  an  emphasis  on  improving  reading  skills 
was  developed  using  Hanoi  3.0  on  an  IBM  compatible  PC.  The  pedagogical  material  for  this 
experimental  system  came  from  the  first  chapter  of  a standard  social  science  textbook  for  fifth 
grade  students  in  Korea.  The  study  involved  55  students  in  total,  who  were  divided  into  an 
experimental  group  of  27  students  and  a control  group  of  28  students.  For  the  final  15  minutes 
of  a 40-minute  class,  the  experimental  group  studied  with  the  experimental  hypermedia  CAI 
system,  while  the  control  group  studied  in  the  traditional  manner.  This  experiment  was 
repeated  for  19  class  hours. 

A post-test  with  30  questions  designed  to  measure  academic  achievement  was  conducted 
on  both  groups  of  students  at  the  end  of  the  experiment.  The  average  and  the  standard 
deviation  of  scores  for  the  experimental  group  are  25.18  and  3.43  respectively,  and  22.57  and 
2.96  for  the  control  group. 

In  order  to  determine  whether  academic  achievement  for  the  experimental  group  was 
better  than  that  of  the  control  group,  a /-test  on  the  two  populations  was  performed  at  the  5% 
level  of  significance.  From  the  analysis  of  the  /-test,  it  is  concluded  that  the  academic 
achievement  for  the  experimental  group  is  significantly  better  than  that  of  the  control  group. 
This  fact  tells  us  that  the  experimental  hypermedia  CAI  system  had  a positive  effect  on  the 
students’  academic  achievement. 


1.  Introduction 

Current  computer  technology  makes  it  possible  to  store  and  retrieve  information  of  several  data  types, 
including  images,  voices,  and  texts.  Hypermedia  (Conklin,  1 987)  is  a database  that  consists  of  a collection  of 
nodes  containing  unformatted  multimedia  data,  which  are  interconnected  by  directed  links  and  are  retrieved 
non-linearly.  There  are  many  practical  applications  of  hypermedia.  In  the  field  of  education,  computer  assisted 
instruction  (CAI)  involving  hypermedia  technology  raises  the  possibility  of  improving  students'  academic 
achievement  using  computers. 

Many  educators  assume  that  hypermedia  CAI  systems  stimulate  students’  motivation  and  curiosity  and 
improve  their  ability  to  memorize  new  facts.  However,  few  statistical  studies  have  been  carried  out  that  can 
verify  this  assumption.  This  may  be  due  to  the  fact  that,  while  it  is  easy  to  develop  a hypermedia  CAI  system,  it 
is  relatively  difficult  to  carry  out  an  experiment  on  a large  sample  of  students  over  an  extended  period  of  time. 
Thus,  though  many  researchers  in  the  field  of  education  are  interested  in  measuring  the  effects  of  CAI  systems 
on  students’  academic  achievement,  the  only  studies  that  have  been  carried  out  so  far  are  those  with  small 
sample  populations,  measured  over  short  periods  of  time. 

Chou  (1996)  developed  a hypermedia  music  CAI  system  and  carried  out  an  experiment  to  evaluate  the 
system  in  an  elementary  school  in  Chungli,  Taiwan.  Sixty  students  of  the  sixth  grade  participated  in  the  study 
for  40  minutes.  Sixteen  students  were  assigned  to  the  experimental  group  and  the  rest  were  assigned  to  the 
control  group.  The  experimental  group  studied  with  the  hypermedia  music  CAI  system,  while  the  control  group 
studied  the  same  material  in  the  traditional  lecture  format.  A post-test  was  given  to  the  students  at  the  end  of  the 
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experiment.  The  results  of  the  /-test  indicate  there  was  no  statistically  significant  difference  between  the  two 
groups. 

Matthew  (1996)  performed  an  experiment  to  compare  the  reading  comprehension  and  attitudes  toward 
reading  of  third  grade  students.  The  study  compared  students  who  read  CD-ROM  interactive  storybooks  with 
students  who  read  traditional  print  storybooks.  After  reading,  a comprehension  test  was  assessed  with  open- 
ended  questions  and  requests  to  retell  the  story.  The  results  of  the  /-test  indicate  that  there  was  no  statistically 
significant  difference  in  the  reading  comprehension  of  two  groups  but  that  there  was  a statistically  significant 
difference  in  their  ability  to  retell  the  story. 

Like  the  studies  just  mentioned,  the  present  study  attempts  to  measure  the  effects  of  a hypermedia  CAI 
system  on  students’  academic  achievement.  However,  the  present  study  has  the  advantage  of  being  based  on  an 
experiment  that  is  performed  over  a longer  period  of  time  and  with  a larger  sample. 


2.  The  Experimental  CAI  System 

An  experimental  hypermedia  CAI  system  with  an  emphasis  of  improving  reading  skills  was  developed 
using  Hanoi  3.0  on  an  IBM  compatible  PC  (Protectsoft,  1997).  The  pedagogical  material  for  this  experimental 
system  came  from  the  first  chapter  of  a standard  social  science  textbook  for  fifth  grade  students  in  Korea.  The 
chapter  is  typically  taught  in  40-minute  sessions  over  a total  of  19  hours. 

The  characteristics  of  the  experimental  CAI  system  are  the  following.  First,  students  interact  with  the 
system  primarily  using  a mouse,  which  allows  for  easy  mastery  of  the  system.  Secondly,  the  system  contains  a 
summary  of  the  contents  of  the  chapter  being  studied.  Thirdly,  important  and  difficult  words  in  the  chapter  are 
given  hyperlinks,  which  connect  them  to  a glossary  and  which  also  allow  users  to  have  a sampled  pronunciation 
of  the  word.  Fourthly,  photographs  and  other  visual  materials  are  included  in  order  to  stimulate  students’  interest 
and  help  them  to  understand  the  material.  Fifthly,  the  system  contains  games  and  questions,  which  students  may 
use  to  test  their  knowledge  of  the  chapter.  Finally,  for  the  convenience  of  the  user,  the  system  contains 
hypermedia  interfaces. 


3.  Test  for  the  Evaluation  of  the  Subjects 

In  order  to  evaluate  students’  academic  achievement,  a test  was  developed.  The  test  consists  of  30 
questions  based  on  the  content  of  the  relevant  chapter  from  the  text.  The  test  was  given  to  the  students  both 
before  and  after  they  studied  the  relevant  chapter. 


4.  Subjects 

Fifty-five  fifth-grade  students  from  two  classes  in  two  elementary  schools  in  Cheonan,  Korea 
participated  in  the  study.  They  were  ranked  on  the  basis  of  IQ  scores  and  school  grades  from  the  previous 
semester.  They  were  divided  into  two  groups  based  on  this  ranking,  with  the  experimental  group  consisting  of 
27  students,  and  the  control  group  consisting  of  28  students.  The  detailed  data  are  shown  in  Table  1. 


SUBJECTS 

ELEMENTARY 
SCHOOL  NAME 

GRADE/CLASS 

MALE 

FEMALE 

TOTAL 

Experimental 

Group 

Anseo 

5/1 

4 

2 

27 

Inju 

5/1 

10 

11 

Control 

Group 

Anseo 

5/1 

4 

2 

28 

Inju 

5/1 

11 

11 

Table  1 Sample  Subjects 
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5.  Procedures  of  the  Experiment 


Pre-test 


In  order  to  determine  whether  or  not  there  was  any  difference  in  the  prior  knowledge  of  the  relevant 
chapter  between  the  experimental  group  and  the  control  group,  the  aforementioned  test  was  given  to  the  subjects 
before  the  experiment. 

Experiment 

For  the  first  25  minutes  of  a 40  minutes  class,  all  subjects  studied  together  in  the  standard  manner.  For 
the  final  15  minutes,  the  experimental  group  studied  using  the  hypermedia  CAI  system  in  a computer  lab  with 
two  assistants,  while  the  control  group  continued  in  the  standard  manner.  This  experiment  was  repeated  for  19 
class  hours,  4 classes  per  week  over  5 weeks.  A summary  of  the  classes  is  given  in  Table  2. 


40-minute  classes 

Period 

First  25  minutes 

Final  15  minutes 

Experimental  Group 

Standard  manner 

CAI 

4 classes  per  week  over 
5 weeks 

Control  Group 

Standard  manner 

Standard  manner 

19  class  hours 

Table  2 Unit  Class  Type  and  Period 


Post-test 


In  order  to  determine  whether  or  not  the  experimental  CAI  system  had  any  affect  on  the  academic 
achievement  of  the  students  in  the  experimental  group,  the  aforementioned  test  was  repeated  on  all  subjects  at 
the  end  of  the  experiment. 


6.  Statistical  Analysis 

Data  collected  from  both  the  pre-test  and  the  post-test  were  analyzed  using  the  statistics  software  SAS 
(SAS  Institute  Inc.  1995). 

Analysis  of  the  Pre-test 

The  average  and  the  standard  deviation  of  scores  for  the  experimental  group  are  9.40  and  2.22 
respectively,  and  for  the  control  group  9.42  and  2.96  (full  score:  30).  The  scores  were  analyzed  using  a /-test. 
Table  3 shows  the  results  of  the  /-test,  where  N represent  the  sample  size  of  each  group,  / the  /-value,  df  the 
degree  of  freedom,  and  p the  observed  significance  level.  For  a /-value  of -0.0360  with  degree  of  freedom  of  53, 
the  /-test  yielded  a /?-value  of  0.9714.  This  result  indicates  that  in  prior  knowledge  for  the  relevant  chapter,  there 
was  no  statistically  significant  difference  between  the  two  groups  at  the  5%  level  of  significance.  Therefore,  one 
can  conclude  that  the  experimental  and  control  groups  were  well  constituted. 

Analysis  of  the  Post-test 

In  order  to  examine  the  data  from  the  post-test,  the  following  hypotheses  were  tested  using  a /-test. 

Null  hypothesis:  Pc  = l^c 
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The  academic  achievement  for  the  experimental  group  is  equal  to  that  of  the  control  group. 
Research  hypothesis:  |Xc  > 

The  academic  achievement  for  the  experimental  group  is  better  than  that  of  the  control  group. 


N 

Average/30 

Standard 

Deviation 

Minimum 

Score 

Maximum 

Score 

/ 

df 

P 

Experimental 

Group 

27 

9.40 

2.22 

6 

14 

-0.0360 

53 

0.9714 

Control 

Group 

28 

9.42 

2.96 

5 

13 

Table  3:  Mest  for  the  Pre-test 


The  average  and  the  standard  deviation  of  scores  for  the  experimental  group  are  25.18  and  3.43 
respectively,  and  for  the  control  group  22.57  and  2.96  (full  score:  30).  The  scores  were  analyzed  using  a Mest. 
Table  4 shows  the  result  of  the  Mest.  For  a /-value  of  3.039  with  a degree  of  freedom  of  53,  the  /-test  yielded  a 
/7- value  of  0.0037.  Therefore  the  null  hypothesis  is  rejected,  and  one  can  conclude  that,  at  the  5%  level  of 
significance,  the  academic  achievement  for  the  experimental  group  was  significantly  better  than  that  of  the 
control  group. 


N 

Average/3 

0 

Standard 

Deviation 

Minimum 

Score 

Maximum 

Score 

/ 

df 

P 

Experimental 

Group 

27 

25.18 

3.43 

16 

29 

3.039 

53 

0.0037 

Control 

Group 

28 

22.57 

2.96 

16 

29 

Table  4:  /-test  for  the  Post-test 


7.  Conclusion 

The  question  with  which  this  study  is  concerned  is  whether  or  not  a hypermedia  CAI  system  can 
enhance  or  improve  students'  understanding  of  academic  material.  In  order  to  answer  this  question,  an 
experimental  hypermedia  CAI  system  was  developed  and  then  tested  on  a group  of  elementary  school  students  in 
Korea.  The  data  from  the  experiment,  which  were  examined  statistically,  demonstrate  that  the  experimental  CAI 
system  was  indeed  effective  in  improving  students'  understanding  of  academic  material.  This  conclusion 
conflicts  with  the  earlier  study  by  Chou,  who  found  a similar  CAI  system  to  have  no  effect  on  academic 
achievement.  However,  since  the  present  study  is  based  on  an  experiment  conducted  over  a longer  period  of  time 
and  with  a larger  sample  size,  its  conclusion  is  more  credible. 

One  limitation  of  the  present  study  is  that  the  experiment  involved  a relatively  homogeneous  sample 
population:  all  subjects  were  elementary  school  students  in  Korea.  Further  research  is  therefore  necessary  to 
determine  the  extent  to  which  the  conclusion  of  this  study  may  be  generalized  to  students  of  other  ages  and/or 
cultures. 
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Abstract:  This  study  examined  how  users  with  different  cognitive  styles  locate  information  on 
a university  Web  site,  a hypermedia-based  information  system.  A hypermedia  system  is 
considered  to  be  capable  of  accommodating  cognitive  style  differences  because  of  its 
multimodal  attributes.  Undergraduate  students  with  different  cognitive  styles  (field-dependence 
and  field-independence)  participated  in  the  study.  Findings  suggest  that  the  level  of  users’ 
online  search  experience  influenced  effects  of  cognitive  style  on  the  search  and  navigational 
behavior.  Only  the  field-dependent  users  with  little  or  no  online  experience  seemed  to  have 
difficulties  in  retrieving  information  on  the  Web. 


Introduction 

Hypermedia  is  one  of  the  recently  developed  information  technologies  equipped  with  attractive 
interface  and  great  capacity  of  improving  access  to  information.  By  way  of  the  World  Wide  Web  (Web)  and 
other  networks,  hypermedia  has  come  to  include  remote  databases  and  online  resources  (Ayersman,  1996). 
Naturally,  the  Web,  with  its  attractive  interface  and  capability  of  connecting  to  a variety  of  different  resources, 
has  become  one  of  the  most  widely  used  information  systems  in  the  world. 

Different  from  more  traditional  information  systems  (e.g.  books),  hypermedia  is  intentionally  nonlinear 
(Nielsen,  1995).  It  presents  a new  information  environment  with  which  the  user  may  not  be  familiar. 
Hypermedia  permits  the  user  to  have  considerable  flexibility  in  choosing  the  sequence  with  which  to  access 
information,  rather  than  forcing  the  user  to  access  information  in  a fairly  linear  sequence.  Such  linear  sequencing 
has  long  been  prescribed  by  the  technology  of  the  book  and  the  narrative  structure  used  by  most  authors.  Users 
who  have  not  developed  skills  of  information-gathering  in  the  nonlinear  environment  often  report  discomfort  and 
experience  difficulties  in  using  hypermedia,  such  as  “cognitive  overload”  and  “disorientation”  (Nielsen,  1995; 
Conklin,  1987). 

Despite  all  these  problems,  early  research  suggests  that  hypermedia  is  promising  as  an  information 
system  (Gary  and  Shasha,  1989;  Egan  et  al,,  1989).  Other  studies  evaluating  the  effectiveness  of  hypermedia, 
however,  have  not  been  uniformly  successful  in  showing  the  advantage  of  hypermedia  over  other  existing 
information  systems  (McKnight  et  al,,  1990;  Marchionini  and  Shneiderman,  1988).  These  inconsistent  findings 
have  led  to  a suspicion  that  there  may  be  variables,  other  than  system  differences,  influencing  the  effective  use  of 
hypermedia  systems.  To  explore  and  identify  factors  influencing  the  effective  use  of  hypermedia,  a number  of 
studies  have  been  conducted.  Results  from  the  research  suggest  that  characteristics  of  users  and  tasks  may  be 
major  factors  affecting  the  use  of  hypermedia  systems  (Qiu,  1993;  Ford  et  al.,  1994). 

This  study  investigates  how  some  of  user  variables  influence  the  use  of  the  Web  as  a hypermedia-based 
information  system.  The  focus  of  the  study  is  on  the  user’s  cognitive  style,  and  its  effects  on  the  search  behavior 
are  examined  in  relation  to  the  user’s  online  search  experience. 


Background 

A study  analyzing  usability  studies  of  hypertext  systems  has  revealed  that  four  of  the  ten  largest  effects 
were  due  to  users’  individual  differences  (Nielsen,  1989).  Among  different  kinds  of  individual  differences, 
cognitive  style,  especially  field-dependence/field-independence  (FD/FI),  is  one  of  the  most  frequently  studied 
factors  in  the  research  of  learning  on  hypermedia  systems  (Liu  & Reed,  1994;  Leader  & Klein,  1996).  The  FD 
individuals  who  tend  to  be  easily  dominated  by  salient  cues  are  more  likely  to  be  distracted  in  hypermedia 
systems  where  information  units  are  presented  through  a variety  of  different,  attractive  media  forms.  In  fact. 
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several  studies  have  shown  that  the  FI  individuals,  less  likely  dominated  by  prominent  stimuli,  generally  perform 
better  than  the  FDs  in  finding  information  on  hypermedia  systems.  The  FIs  tend  to  search  information  more 
efficiently  and  arrive  at  desired  goals  more  quickly  than  the  FDs  (Ellis  et  al,,  1993;  Ford  et  al,,  1994),  Each 
group  utilize  a hypermedia  learning  environment  differently  although  both  of  the  groups  have  equivalent 
learning  outcomes  (Fitzgerald  and  Semrau,  1998), 

Another  kind  of  individual  difference  with  a strong  impact  on  the  information-seeking  process  includes 
users’  experience  and  expertise.  Experience  with  online  search  and  subject  expertise  have  been  found  to 
influence  the  choice  of  search  strategies  and  the  search  performance  on  information  systems.  Studies  on  linear 
information  systems  have  revealed  that  online  search  experience,  rather  than  subject  expertise,  plays  the  major 
role  in  determining  the  effective  use  of  the  systems  (Hsieh-Yee,  1993),  However,  this  finding  is  yet  to  be 
confirmed  with  hypermedia-based  information  systems. 


The  Study 

Forty-eight  volunteer  students  from  a public  university  participated  in  this  experiment.  Their 
participation  was  financially  compensated.  All  the  participants  were  undergraduate  students  with  different 
academic  orientations.  Fifty-four  percent  (n=26)  of  the  participants  were  with  soft  science  (social  science,  arts 
and  humanities),  and  46%  (n=22)  were  with  hard  science  (natural  science  and  engineering)  background.  Half  of 
the  participants  were  male  and  the  other  half  were  female  students. 

The  GEFT  (Group  Embedded  Figures  Test)  and  a questionnaire  were  used  to  determine  the  cognitive 
style  and  the  level  of  online  database  search  experience,  respectively.  Based  on  the  results  from  the  test  (with 
scores  ranging  from  0 to  18)  and  questionnaire  (with  scores  ranging  from  0 to  4),  participants  were  selected,  and 
an  individual  lab  session  was  arranged  for  each  of  the  participants.  Half  of  the  participants  were  field-dependent 
(FD)  and  the  other  half  were  field-independent  (FI):  Mearifo  = 8,25,  Meanf/=  15,96,  Also,  half  of  them  were 
novice  and  the  other  half  were  experienced  online  searchers;  Mean  ^ov  — 0,92,  Mean  exp  =2,96,  In  the  lab 
session,  the  participant  was  assigned  search  tasks  for  which  he  or  she  had  to  locate  pertinent  information  within  a 
university  Web  site.  The  Web  browser  used  in  the  study  was  Netscape  4,0, 

Independent  variables  for  the  study  include  users’  cognitive  style  (field-dependent  (FD)  vs,  field- 
independent  (FI))  and  online  database  search  experience  (experienced  (EXP)  vs,  novice  (NOV)),  On  the  basis  of 
their  cognitive  style  and  online  experience,  participants  were  evenly  divided  into  four  groups  (FD-NOV;  FD- 
EXP;  FI-NOV;  FI-EXP),  Two  dependent  variables  were  adopted  to  measure  search  performance  of  the 
participants:  the  average  time  spent  and  the  average  number  of  nodes  visited  for  the  completion  of  a task. 
Additionally,  the  number  of  times  each  navigational  tool  chosen  was  counted  to  examine  the  user’s  search  and 
navigational  style. 

The  individual  lab  session  started  with  a review  of  Web  basics,  designed  to  ensure  that  every  participant 
was  aware  of  the  availability  of  different  navigational/search  tools  and  menus  in  the  Web  browser  used  and  also 
general  features  of  search  engines.  When  the  review  was  over,  two  search  tasks  were  assigned  to  the  participant. 
The  same  set  of  search  tasks  were  used  for  all  the  participants.  When  the  participant  felt  ready  to  start  searching, 
he  or  she  asked  the  researcher  to  start  recording  the  search  session.  During  the  search  session,  all  the  screen 
displays  consulted  and  keyboard/mouse  inputs  were  recorded  using  Lotus  ScreenCam,  The  participants  were 
asked  to  make  a bookmark  of  each  Web  page  where  he  or  she  found  relevant  information. 


Results 

Average  Length  of  Time  Spent  for  the  Completion  of  a Task 

A two-way  ANOVA  was  performed  for  the  average  length  of  time  spent  for  the  completion  of  a task.  In 
this  report,  the  level  of  significance  was  set  at  p < ,10,  The  result  indicated  a significant  main  effect  of  cognitive 
style:  Fq  (1,44)  = 5,271,  p < ,03,  The  FD  individuals  tended  to  spend  more  time  than  the  FIs  to  find  a piece  of 
information:  Mean  fo  = 130,3  sec,.  Mean  f/=  89,9  sec,  A significant  main  effect  of  online  search  experience  was 
also  found:  (1 ,44)  = 3,69, p < ,07,  To  find  information  on  the  Web,  those  with  little  or  no  online  search  experience 

spent  more  time  than  those  with  substantial  online  search  experience:  Mean  !s!ov  = 127  sec,.  Mean  exp  ~ 93,2  sec, 
TTiere  was  a significant  interaction  between  the  two  independent  variables:  FcxO  (1,44)  = 3,859,  p < ,06,  Figure  1 
shows  how  the  cognitive  style  and  online  search  experience  variables  interacted  with  each  other  to  influence  the  time 
spent  for  the  completion  of  a task. 
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df 

F 

Cognitive  Style  (C) 

1 

5.271* 

Online  Experience  (0) 

1 

3.690* 

CxO 

1 

3.859* 

Within  Groups 

44 

*p<.10 

Table  1.  Analysis  of  Variance  on  the  Average  Time  Spent  for  the  Completion  of  a Task 


For  those  with  considerable  online  search  experience,  cognitive  style  seemed  to  have  little  impact:  Mean  fd- 
Exp  = 96. 1,  Mean  fj.exp  ~ 90.3.  For  those  with  little  or  no  online  experience,  however,  cognitive  style  seemed  to  play 
an  important  role  in  decreasing  or  increasing  the  length  of  time  spent  for  the  completion  of  a task.  The  FDs  (FD- 
NOV)  tended  to  spend  longer  time  to  retrieve  information  than  the  FIs  (FI-NOV):  Mean  fd-nov~  164.6,  Mean  fi-nov  ~ 
89.5.  Interestingly,  the  FDs  with  substantial  online  experience  spent  almost  the  same  amount  of  time  as  the  FIs  (FI- 
NOV  and  FI-EXP)  did  for  the  completion  of  a task.  It  seems  that,  only  among  those  with  little  online  experience, 
cognitive  style  has  an  effect  on  the  time  spent  for  finding  information. 


NOV 

EXP 


Cognitive  Style 


Figure  1,  Average  Time  Spent  for  the  Completion  of  a Task  (sec.):  Cognitive  Style  by  Online  Search  Experience 


Average  Number  of  Nodes  Visited  for  the  Completion  of  a Task 

A two-way  ANOVA  was  performed  for  the  average  number  of  nodes  visited  for  the  completion  of  a task. 
At  p < .07,  a significant  interaction  between  cognitive  style  and  online  experience  was  found:  F cxo  (U44)  = 3.57. 
As  shown  in  Figure  2,  the  FI-NOV  tended  to  visit  a lower  number  of  nodes  than  the  FD-NOV:  Mean  fi-nov  = 7.3, 
Mean  fd-nov-  1 1-7.  Among  the  experienced  online  searchers,  on  the  other  hand,  little  difference  was  found  between 
the  FIs  and  the  FDs:  Mean  fi-exp  =8.5,  Mean  fd-exp  = 7.8.  The  pattern  was  similar  to  what  was  found  with  the  time 
spent  for  the  completion  of  a task.  No  significant  main  effect  was  found,  however.  That  is,  no  statistically  significant 
difference  existed  between  the  numbers  of  nodes  visited  when  the  groups  were  compared  on  the  participants’ 
cognitive  style  or  on  the  level  of  their  online  search  experience. 
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df 

F 

Cognitive  Style  (C) 

1 

2.029 

Online  Experience  (0) 

1 

1.005 

CxO 

1 

3.570’ 

Within  Groups 

44 

*p<AO 

Table  2.  Analysis  of  Variance  on  the  Average  Number  of  Nodes  Visited  for  the  Completion  of  a Task 


NOV 

EXP 


Figure  2.  Average  Number  of  Nodes  Visited  for  the  Completion  of  a Task:  Cognitive  Style  by  Online  Experience 


Use  of  Navigational  Tools 

In  order  to  find  whether  differences  in  search  performance  were  reflected  in  the  navigational  tools  chosen, 
additional  sets  of  two-way  ANOVA  were  carried  out  using  the  number  of  times  different  navigational  tools  were 
used  (such  as  embedded  links,  Back,  Home,  Go,  History,  search  engines,  etc.)  as  the  dependent  variable.  Significant 
interactions  were  found  for  the  use  of  embedded  links  and  the  Home  button:  for  embedded  links,  Fcxo  (U44)  = 
5.002,p  < .04,  and  for  the  Home  button,  Fcxoi^M)  = 3.728, p < .07.  The  pattern  of  interactions  was  similar  to  the 
one  found  with  the  time  spent  and  the  number  of  nodes  visited  for  the  completion  of  a task.  The  FDs  with  little  or  no 
search  experience  (FD-NOV)  used  embedded  links  and  the  Home  button  significantly  more  frequently  than  the  rest. 

In  addition  to  the  interaction  effects  found  in  the  use  of  embedded  links  and  Home  button,  a significant 
main  effect  of  online  experience  was  found  in  the  use  of  jump  tools:  Fq  (1,44)  = 6.568,  p < .02.  Here,  jump  tools 
include  Go,  History,  Location  box  used  for  an  “active”  jump.  The  result  revealed  that  the  experienced  online 
searchers  used  jump  tools  more  frequently  than  the  novice  searchers:  Mean  exp  ~ 0.60,  Mean  i^ov~  0.005. 
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Discussion 


Results  from  this  study  suggest  that  the  level  of  users’  online  search  experience  influences  the  effect  of 
cognitive  style  on  the  search  performance.  That  is,  among  those  who  had  little  or  no  experience  with  online 
searches,  the  FI  individuals  tend  to  outperform  the  FDs.  In  order  to  complete  a search  task,  the  FIs  spent  less 
time  and  needed  to  visit  fewer  nodes  than  the  FDs.  However,  the  difference  created  by  participants’  cognitive 
style  disappeared  in  those  participants  who  had  considerable  online  search  experience.  Among  the  experienced 
online  searchers,  no  significant  difference  was  found  between  the  FDs  and  the  FIs  in  terms  of  the  time  spent  and 
the  number  of  nodes  visited  for  the  completion  of  a task.  This  implies  that  difficulties  that  the  FDs  face  in 
finding  information  on  the  Web  may  be  overcome  as  the  FDs  gain  experience  and  develop  their  search  strategies 
while  using  online  databases.  Despite  presentational  and  structural  differences  between  traditional  “linear” 
online  database  systems  and  the  “non-linear”  Web,  skills  required  for  using  online  databases  seem  to  be 
transferred  to  the  use  of  Web  sources. 

With  regard  to  the  choice  of  navigational  tools,  a similar  interaction  effect  was  observed.  In  the  group 
of  individuals  with  little  or  no  online  search  experience,  the  FDs  used  embedded  links  more  frequently  than  the 
FIs.  The  frequent  usage  of  embedded  links  can  be  interpreted  as  a linear  rather  than  a non-linear  way  of 
navigation.  It  is  because  other  tools  (such  as  Go,  History  list,  or  a typed  URL)  allow  the  user  to  jump  to  more 
temporally  removed  point  whereas  embedded  links  do  not.  Findings  from  the  study  implies  that  the  FD-NOVs 
navigate  the  Web  in  a more  linear  mode  than  the  rest.  In  fact,  this  might  be  the  way  in  which  we  would  expect 
the  “typical”  FD  would  navigate.  The  FDs  prefer  a well  structured  set  of  stimuli.  As  they  tend  to  be  passive  in 
learning  (Witkin  et  al.,  1977),  the  FDs  would  not  enjoy  imposing  a structure  of  their  own.  Thus,  the  FDs  are 
more  likely  to  navigate  the  Web  in  a linear  mode.  Previous  studies  on  the  use  of  hypermedia  systems  found  that 
the  FDs  tended  to  explore  a hypermedia  system  in  a linear  mode,  following  the  sequence  encouraged  by  the 
system  (Liu  & Reed,  .1994).  This  finding  is,  however,  only  partly  supported  by  our  study.  In  our  study,  only  the 
FDs  with  little  or  no  online  experience  demonstrated  the  linear  mode  of  navigation.  The  way  in  which  the  FDs 
with  considerable  online  search  experience  navigated  the  Web  was  different  from  the  way  in  which  the  FDs  with 
little  online  experience  did.  The  FDs  with  substantial  online  search  experience  had  a navigational  style,  reflected 
in  their  use  of  tools,  which  was  rather  similar  to  the  FIs’.  Their  search  performance,  reflected  in  the  time  spent 
and  the  number  of  nodes  visited  for  the  completion  of  a task,  was  also  comparable  to  the  FI’s. 

In  this  study,  the  FDs  with  little  or  no  online  experience  tended  to  use  Home  button  more  frequently 
than  the  rest.  The  use  of  Home  button  can  be  viewed  as  one  possible  indication  of  the  user’s  “getting  lost,” 
because  the  Home  button  is  often  used  when  people  wish  to  start  over,  abandoning  whatever  they  have  been 
doing.  Based  on  this  interpretation,  the  result  implies  that  the  FD-NOVs  get  lost  more  often  than  the  rest.  It  is 
known  that  the  FDs  tend  to  be  easily  distracted  in  a complex  field  by  cues  that  may  not  be  relevant  to  the  goal 
(Witkin  et  al.,  1977).  Hence,  the  FDs  are  expected  to  become  easily  lost  through  the  pursuit  of  more  dominant 
but  possibly  irrelevant  cues.  Again,  findings  from  our  study  supports  the  hypothesis  only  partially.  Among  the 
FDs,  only  those  with  little  or  no  online  experience  tended  to  get  lost  more  easily  than  the  rest,  including  the  FDs 
with  substantial  online  search  experience. 

The  FDs,  especially  those  with  little  or  no  experience  with  online  databases,  seem  to  need  special 
attention  from  interface  designers  and  those  who  train  Web  users.  Interface  designers  may  want  to  incorporate 
devices  that  can  help  the  FDs  be  better  oriented  and  less  likely  to  get  lost.  Providing  a graphical  map  of  their 
search  progress  would  be  an  example,  and  several  studies  have  found  it  to  be  effective  in  terms  of  resolving  the 
problem  of  disorientation  (Chen  & Rada,  1996).  With  the  browser  interface,  it  might  also  be  helpful  to  provide  a 
visible  history  list,  showing  all  sites  the  user  has  visited  and  functioning  like  a map,  at  the  same  time.  Such  a list 
should  be  readily  available,  rather  than  requiring  the  user  to  make  an  active  search  for  it  through  several  pull 
down  menu  listings  (which  is  often  found  in  “linear”  online  systems).  This  would  be  useful  to  the  FD  users 
because  the  list  could  be  used  as  a map  showing  what  has  been  visited  and  also  provides  an  easy  way  to  return  to 
some  temporally  distant,  previously  visited  sites.  Web  search  trainers  might  examine  the  finding  that  online 
database  search  experience  was  particularly  beneficial  to  FD  users.  The  exact  reason  is  not  clear,  however.  It 
may  be  because  the  heavily  text-based  design  of  both  the  Web  and  online  databases  appear  the  same  to  the  user 
even  though  they  may  have  very  different  features  and  intents.  It  may  be  because  the  Web  has  inherited  many 
interface  features  from  the  traditional  linear  online  systems.  It  may  also  be  that  the  knowledge  and  skills  for 
using  online  databases  are  useful  regardless  of  the  system  used.  Obviously  more  research  is  called  for  to 
investigate  these  potential  explanations. 
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Abstract:  This  paper  is  concerned  with  the  four  main  integrated  components  of 
online  learning:  an  administration  management  system;  an  authoring  system;  a 
learning  resource  catalogue;  and  a delivery  system.  The  four  components  are  set 
against  the  evolution  of  the  concept  of  ‘learning  objects’  which  strongly  influence 
how  learning  resources  are  authored  and  upon  which  a learning  resource  catalogue 
is  based.  Learning  objects  are  generally  small  discrete  chunks  of  learning  materials 
that  are  reusable  and  can  articulate  with  other  learning  objects  to  make  a learning 
environment.  Learning  objects  can  be  described  in  standard  terms  (i.e.,  metadata) 
through  the  use  of  a web  application  and  those  data  can  be  submitted  to  an  online 
searchable  database  to  enable  their  use  in  other  contexts  and  combinations. 


Introduction 

A change  to  educational  management,  funding  and  expectations  over  the  last  several  years  in 
numerous  countries  has  resulted  in  a more  technologically  mediated  educational  environment.  The  use 
of  technology  to  aid  in  the  delivery  of  courses  has  resulted  in  a diminishing  of  the  physical  boundaries 
of  the  university  while  increasing  the  potential  student  pool.  International  and  geographically  removed 
students  are  increasing  as  a viable  market  for  any  institution  as  Educational  delivery  becomes  more 
globally  available  and  location  is  no  longer  such  a restricting  component.  Professional  students  more 
and  more  are  requiring  continuing  education  and  postgraduate  qualifications  while  demanding  that  their 
studies  fit  into  their  existing  lifestyle  and  work  commitments.  As  a result  access  to  courses  has  become 
a highly  influential  factor  for  potential  students  and  institutions  can  no  longer  rely  on  academic 
reputation  alone.  Additionally,  research-intensive  universities  are  addressing  the  increasing  time 
demand  on  their  academics  to  be  subject  matter  experts,  lecturers  and  accomplished  researchers. 
Institutions  not  addressing  these  issues  will  miss  out  on  this  increasing  student  market  in  the  evolving 
consumer  driven  model  of  education. 

Most  institutions  worldwide  have  made  educational  technology  a prime  emphasis  as  a solution 
to  addressing  these  issues  and  tapping  into  the  changing  student  needs. 

The  implementation  of  technology  in  education  has  passed  through  three  broad  phases.  Each 
of  these  phases  came  about  as  a proposed  solution  to  the  primary  weakness  identified  after  the  practical 
application  of  each  phase. 

The  first  phase  of  educational  technology  arose  from  the  idea  that  technology  could  be  utilised 
to  communicate  with  the  masses.  Educational  institutions  adopted  this  notion  as  a strategy  for 
increasing  student  markets  in  the  evolving  consumer  driven  model  of  education.  In  its  early  adoption, 
educational  technology  components  were  designed  by  academics  keen  to  embrace  this  new  delivery 
method.  However  the  vast  amount  of  technologies  that  emerged  due  to  the  boom  of  the  Internet, 
quickly  overwhelmed  the  academics  whose  area  of  expertise  was  not  technology.  Two  components 
were  identified  as  being  essential  and  central  to  this  phase’s  implementation  of  an  online  learning 
system:  the  authoring  system  and  the  delivery  system.  In  its  most  basic  form  one  example  of  this 
system  would  be  an  HTML  editor  and  a web  browser. 
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The  second  phase  came  into  being  to  address  the  lack  of  technological  expertise  in  the 
academic  environment.  This  involved  the  addition  of  technologically  trained  staff  to  the  university 
environment  and  centralised  units  to  aid  academics  in  developing  these  educational  technology 
components.  With  the  centralisation  of  specialised  technological  staff  came  a more  institution-wide 
recognition  of  educational  technology.  It  is  at  this  stage  that  a new  component  to  the  online  learning 
system  was  added:  an  administration  management  system.  While  this  phase  served  its  initial  purpose,  it 
pushed  to  the  forefront  a new  issue.  Although  deemed  beneficial,  educational  technology  developments 
were  proving  to  be  highly  resource  intensive  in  terms  of  both  time  and  money.  Additionally,  the  cost 
effectiveness  of  such  developments  to  the  institution  was  poor  as  it  usually  only  benefited  a few 
academics  and  their  students. 

The  third  phase  is  the  phase  we  are  currently  entering.  The  Learning  Object  model  has  been 
proposed  as  the  new  solution  in  educational  technology.  Loosely  derived  from  an  object-oriented 
programming  model,  it  primarily  aims  to  address  the  issues  of  reuse  of  materials.  The  learning  object 
model  is  characterized  by  the  belief  that  we  can  create  independent  chunks  of  educational  content  that 
provide  an  educational  experience  for  some  pedagogical  purpose.  These  chunks  are  self-contained 
although  they  may  contain  references  to  other  objects;  and  they  may  be  combined  or  sequenced  to  form 
longer  educational  interactions  (Quinn,  2000).  While  a learning  object  is  not  necessarily  a digital  object 
the  remainder  of  this  paper  will  focus  on  learning  objects  that  are  exclusively  digital. 

The  new  essential  component  for  this  model  is  a system  for  storing  and  cataloguing  these 
learning  objects,  here  called  a learning  resource  catalogue.  If  this  latest  model  for  educational 
technology  is  to  succeed,  the  three  components  identified  from  the  earlier  phases  must  be  tightly 
integrated  with  this  new,  but  often  missing,  essential  fourth  component. 

This  paper  briefly  examines  the  concept  of  learning  objects  then  considers  their  use  in  the  four 
main  components  (exclusive  of  hardware)  required  to  facilitate  online  learning  environments  - 
administration  and  management,  authoring,  delivery  and  learning  resource  cataloguing.  Each 
component  is  considered  in  sufficient  detail  so  as  to  define  their  ingredients,  complexity,  scope  and  to 
identify  their  inter-relationships.  This  will  enable  the  relevance  and  importance  of  the  learning  resource 
catalogue  to  be  appreciated.  The  strength  of  any  system  or  framework  lies  in  the  integration  of  the 
components. 


The  learning  object  model 

A learning  object  is  “any  entity,  digital  or  non  digital,  which  can  be  used,  reused  or  referenced 
during  technology  supported  learning”  (IEEE’s  Learning  and  Technology  Standards  Committee,  2000). 
Learning  objects  are  also  fundamental  to  the  IMS  system  (IMS,  2000). 

Material  that  may  be  thought  of  as  ‘content’,  if  developed  as  assemblages  of  small,  discrete 
learning  modules  or  ’chunks’,  may  be  considered  to  be  composed  of ’learning  objects’.  If  these  learning 
objects  are  discrete  enough,  they  may  be  re-used  in  conjunction  with  other  learning  objects  in  different 
contexts.  The  concept  and  practice  of  developing  digital  learning  resources  as  learning  objects  has  a 
profound  influence  on  the  development  of  learning  resource  catalogues  (as  discussed  below). 

The  learning  object  model  has  been  proposed  by  many  in  the  educational  community  as  an 
optimal  solution  to  many  of  education’s  long-standing  problems  with  the  development  of  educational 
technology  materials.  The  main  problems  that  the  learning  object  model  aims  to  address  are:  the  time 
and  expense  involved  in  development  of  learning  materials;  the  sharing  of  expertise  and  knowledge 
between  academics  and  scholars,  and  the  globalisation  of  education. 

There  are  benefits  to  be  gained  in  making  and  submitting  learning  objects  to  be  reused  by 
others.  Such  benefits  could  include:  the  prestige  of  having  made  a popular  and  useful  resource;  peer 
acknowledgement  of  teaching  quality;  contribution  to  promotion  prospects;  financial  gain  if  the 
learning  objects  compete  in  a market  economy,  and  a route  to  publication  (measured  by  citations) 
without  the  formal  publications  system. 

There  are  benefits  to  utilising  learning  objects  made  by  others.  Such  benefits  include:  savings 
of  time  and  expense;  being  able  to  compare  and  select  teaching  resources  from  various  sources;  being 
able  to  readily  benchmark,  and  gaining  teaching  insights  and  expertise  from  others. 


Types  of  learning  object 

The  concept  of  a learning  object  is  still  evolving.  A learning  object  can  vary  in  size  and 
complexity.  In  its  simplest  form  a learning  object  is  context  free  and  may  be  an  image,  video  or  piece 
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of  text  such  as  a poem.  This  simple  form  has  a very  large  reuse  potential  because  it  can  be  used  in  a 
wide  range  of  contexts,  disciplines,  learning  strategies  and  so  on.  From  this  simply  beginning  learning 
objects  can  grow  in  complexity  to  include  context,  learning  objectives,  pedagogy  and  so  forth.  In 
general,  the  more  complex  a learning  object,  the  more  limiting  is  its  potential  for  reuse. 


The  four  main  components  of  online  learning 

Apart  from  hardware  considerations,  for  the  creation  of  efficient  online  learning  environments, 
several  components  are  necessary.  Essentially  these  are:  an  administration  management  system;  an 
authoring  system;  a learning  resource  catalogue;  and  a delivery  system. 


(1)  Administration  management  system 

The  online  administration  management  system  should  include  facilities  like  student 
enrolment,  student  profiles,  class  lists  and  available  courses.  Fixed-cost  financial  transactions  may  also 
be  facilitated  by  this  system  but  other  costs  will  depend  upon  usage  of  option  resources  at  a later  stage 
(e.g.,  printing  and  library  borrowing).  Such  an  administration  management  system  would  allow  faculty 
members  to  view  the  student  records  and  profiles  from  their  desktop  computer.  Students  would  be  able 
to  view  their  own  records,  course  information  and  class  lists,  for  example. 

An  enterprise- wide  online  administration  management  system  requires  high-level  institutional 
management  commitment  because  of  the  associated  costs  and  institutional  changes  that  necessarily 
accompany  the  adoption  of  such  an  approach.  Institutional  commitment  to  this  change  must  exist  at  the 
top  as  well  as  with  faculty  members.  One  will  not  work  without  the  support  of  the  other. 


(2)  Authoring  system 

From  a digital  perspective,  an  authoring  system  consists  of  tools  that  operate  at  two  levels: 
firstly,  the  tools  to  create  learning  objects  that  may  be  thought  of  as  ‘content’  and  secondly,  the  tools  to 
aid  in  the  modification  and  sequencing  of  learning  objects  into  a cohesive  learning  environment.  As 
indicated  above,  there  are  many  types  of  learning  object  that  include  ‘content’  as  well  as  the  activities 
and  methods  that  are  all  part  of  a learning  environment. 

The  creation  of  the  learning  objects,  and  their  subsequent  use,  will  depend  upon  the  needs, 
educational  rationale,  learning  objectives  and  deliveiy  strategy  that  have  been  determined  prior  to  the 
utilisation  of  any  such  tools.  Content  creation  includes  tools  for  web  authoring,  multimedia  creation 
and  simulations.  Digital  content  creation  may  be  relatively  simple  if  it  is  concerned  mainly  with  text 
and  complex  if  it  includes  the  use  of  several  media  such  as  video  and  graphics.  Programming  to 
varying  extents  is  also  usually  required,  particularly  if  several  media  are  to  operate  together,  and  that 
programme  itself  may  be  considered  to  be  a learning  object  that  can  be  reused.  The  content  may  then 
reside  in  several  files,  databases  and  on  different  servers. 


(3)  Learning  Resource  Catalogue 

In  the  overall  picture,  a learning  resource  catalogue  (LRC)  works  a s follows.  An  instructor 
creates  a learning  object,  then  generates  a description  of  it  (i.e.,  creates  metadata)  by  describing  the 
object  using  a set  of  standardized  terms,  then  submits  the  metadata  to  the  learning  resource  catalogue. 
Another  instructor  would  then  search  the  catalogue  based  on  the  same  standardised  terms.  Upon 
finding  a description  of  an  object  that  suits  a particular  purpose,  the  instructor  can  contact  the  creator 
through  the  catalogue  system  and  negotiate  reuse  of  the  object.  It  should  be  noted  that  the  learning 
object  itself  is  not  submitted  to  the  database;  only  the  metadata  is  stored  in  the  database.  An  analogy  is 
given  by  a newspaper  advertisement  for  an  object  - the  metadata  is  the  advertisement  and  the  object  is 
in  the  home  of  the  owner. 

Once  a LRC  is  heavily  populated,  it  should  be  possible  for  users  to  find  sets  of  objects  that 
suit  various  purposes,  and  thereby  significantly  cut  down  on  development  costs  associated  with  content 
creation  of  learning  materials. 
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Once  learning  objects  have  been  created,  they  should  be  catalogued  in  a searchable  database 
for  the  purposes  of  reuse  and  keeping  an  inventory  of  the  organisation’s  resources  of  learning  objects. 
The  inventory  would  then  provide  a starting  point  for  any  potential  developments. 

An  LRC  may  also  be  known  as  a 'knowledge  repository'  or  'digital  resource  library'.  However, 
these  terms  may  not  be  synonymous  and  a LRC  is  here  defined  as  a database  of  metadata  describing 
learning  objects.  Digital  learning  objects  are  noted  above  as  being  discrete  with  qualities  that  enable 
them  to  be  re-used  in  different  contexts  and  by  other  teachers  and  students  who  wish  to  share  these 
resources  for  the  sake  of  efficiency. 

Learning  object  metadata  (LOM)  should  be  derived  from  a learning  object  according  to  a 
standard  method  because  the  use  of  standards  enables  consistency  of  usage  and  communication.  The 
IMS  metadata  standard  (IMS,  2000)  is  rapidly  becoming  the  international  standard  for  LOM,  and  is 
based  on  the  IEEE  Learning  Objects  Metadata  standards  (IEEE,  1999).  The  essential  functionality  of  a 
LRC  is  that  LOM  should  be  simple  to  generate  and  submit  to  a database  that  is  easily  searchable. 

During  the  creation  of  learning  resources  with  various  authoring  tools,  it  is  not  standard 
practice  to  derive  and  save  LOM  concerning  those  resources  because  the  concept  of  a learning  object 
and  LOM  are  not  yet  sufficiently  part  of  the  academic  culture.  Likewise,  it  is  not  yet  standard  practice 
to  submit  that  LOM  to  a searchable  database.  As  a consequence  it  is  difficult  for  individuals  to  know 
of,  and  to  be  able  to  locate,  desirable  learning  objects.  From  an  author's  perspective,  it  means  that  many 
learning  objects  are  being  recreated,  possibly  within  the  same  institution.  This  is  wasteful  in 
development  time  and  money  and  also  in  delayed  time  to  delivery  that  could  have  been  faster  had  the 
author  known  of  existing  available  learning  objects.  From  a learner's  perspective,  it  means  that  relevant 
learning  objects  that  may  have  aided  learning  could  not  be  located. 

The  IMS  LOM  system  is  extensive  with  nearly  100  fields  (at  the  time  of  writing  this  paper) 
for  describing  learning  objects.  Deriving  LOM  for  each  learning  object  created  by  an  author  could  be 
an  undesirable  task,  especially  if  it  adds  considerable  time  to  the  development  process.  The  derivation 
of  LOM  must  be  simple  and  quick  and  yet  still  conform  to  the  IMS  standard.  In  a manual  system,  even 
if  done  on  the  computer  through  a simple  interface,  the  process  must  not  be  onerous  and  must  provide 
help  on  demand.  Should  clarity  and  simplicity  not  be  part  of  the  process  of  generating  LOM,  the 
adoption  of  the  practice  as  a routine  will  not  occur.  At  the  present  time,  such  a manual  entry  system  is 
available  and  in  increasing  use.  In  time,  it  may  become  possible  to  have  a system  of  automatic  LOM 
generation  from  learning  objects. 

Once  LOM  has  been  generated,  the  metadata  must  be  lodged  in  a searchable  database.  The 
digital  learning  object  itself  must  reside  at  a fixed  location  if  the  metadata  is  to  be  useful.  If  the  learning 
object  is  on  a website  then  the  URL  must  be  permanent  or  the  LOM  must  be  editable  (manually  or 
automatically)  for  the  new  location  to  be  entered. 

A user,  having  discovered  the  location  of  a useful  learning  object  (by  searching  the  database 
of  LOM,  i.e.,  the  LRC),  may  then  contact  the  owner  for  permission  to  use  that  object,  should  any 
restrictions  have  been  placed  upon  its  usage  by  the  owner.  Ideally,  usage  rights  of  the  LRC  by  students 
should  be  controlled  by  the  management  system. 

An  example  of  a practical  application  of  a LRC  will  be  demonstrated  at  the  presentation  by 
reference  to  the  Universitas  21  LRC.  This  is  a LRC  developed  for  the  Universitas  21  international 
consortium  of  universities  as  a means  of  identifying,  cataloguing,  finding  and  reusing  learning  objects 
amongst  the  member  institutions. 


(4)  Delivery  system 

An  online  delivery  system  should  include  the  means  of  enabling  the  learning  objects  to  be 
organised  by  teaching  staff  for  access  by  the  students  for  specific  purposes.  It  also  includes  facilities  for 
enabling  communication  and  dialogue  interactivity  between  the  learners  and  with  the  teacher(s). 
Assessment  and  feedback  are  usually  also  part  of  the  delivery  system.  (Several  delivery  systems,  with 
various  features,  are  available  off-the-shelf  these  days.) 

The  delivery  system  must  articulate  with  the  products  of  the  authoring  processes.  The  delivery 
system,  if  it  is  prescriptive,  will  determine  and  limit  the  authoring  freedom.  Therefore,  the  choice  of 
authoring  tools,  and  subsequent  products,  should  take  into  account  the  constraints  of  the  delivery 
system.  Should  the  authoring  products,  as  determined  by  the  learning  environment  needs,  exceed  the 
capabilities  of  the  delivery  system,  then  customised  delivery  mechanisms  need  to  be  programmed 
(usually  at  extra  cost). 
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Ideally,  the  delivery  system  and  the  administration  management  system  should  also  articulate. 
This  would  enable  teaching  staff  to  incorporate  class  lists  and  provide  access  for  those  students  to 
various  appropriate  learning  resources  and  discussion  groups,  for  example. 


Conclusion 

The  integration  of  the  above  four  components  provides  a framework  for  learning  enablement. 
The  concept  of  digital  learning  objects  is  evolving  and  is  not  yet  part  of  the  academic  culture.  Preparing 
learning  objects  as  the  basis  for  flexible  teaching  and  learning  environments  will  maximise  their 
reusability  and  facilitate  efficient  production  environments.  The  authoring  of  learning  objects  in  the 
production  environment  is  essential  for  the  development  and  usage  of  learning  resource  catalogues. 
Similarly,  easily  accessible  learning  resource  catalogues,  at  both  the  local  and  partnership  levels,  are 
essential  to  enable  the  efficient  development  and  re-use  of  learning  objects.  Of  the  main  components  of 
an  online  learning  system,  learning  resource  catalogues  are  invaluable  for  use  in  an  authoring  and 
delivery  system  where  the  learning  objects  are  available  to  teachers  and  students  alike.  Moreover,  in 
any  one  institution,  the  learning  resource  catalogue,  management  system  and  authoring  and  delivery 
system  must  articulate  with  each  other  for  maximum  efficiency  of  online  learning  developments. 
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Abstract:  In  this  paper  I will  present  the  main  constructivist  perspectives,  which  must  be  taken 
into  account  when  designing  learning  situations  in  net-based  learning  environments  (NBLE), 
such  as  TopClass.  The  perspectives  are:  constructing  the  knowledge,  metacognition  and  self- 
directedness,  and  situated  and  meaningful  learning.  Then  I will  highlight  some  problems  and 
how  they  could  be  possibly  solved.  The  main  problem  is  that  the  instructor  or  tutor  has  not 
enough  tools  to  monitor  the  students’  net-based  learning  process  to  be  able  to  support  it 
intensely.  A possible  solution  to  solve  the  problem  is  to  use  different  kinds  of  tools  to  make  the 
processes  more  visual.  This  kind  of  tools  could  also  help  the  learners  to  better  understand  their 
learning.  Ideas  for  these  tools  are  discussed  at  the  end. 


Introduction 

When  we  use  the  new  net-based  learning  environments  (NBLE’s)  it  is  obvious  that  it  takes  very  much  time  and 
trouble  to  design  and  implement  the  net-based  learning  situations.  It  is  thus  easy  to  forget  the  constructivist 
principles  and  design  the  courses  based  on  the  traditional  behaviorist  learning  paradigm  (Hakkinen  1996).  It  is 
however  impossible  to  apply  the  traditional  models  to  a constructivist,  learner-centered  approach.  In  the 
traditional  models,  the  goals  are  given  from  above,  most  of  the  learning  paths  have  been  determined  in  advance, 
and  the  learners  are  not  able  to  control  their  own  progress  in  the  material.  These  models  have  been  criticized 
also  for  their  content-centeredness  and  surface  learning.  In  some  cases  the  traditional  models  are  justified  and 
efficient,  but  if  deeper  learning  and  more  efficient  learning  skills  are  required,  the  learners’  previous  experience, 
knowledge  and  skills  need  to  be  taken  into  account.  The  learners  have  also  to  be  offered  a chance  to  set  their 
own  goals  and  to  evaluate  their  learning  in  interaction  with  others  in  as  authentic  contexts  as  possible. 

After  discussing  some  constructivist  principles  I will  expose  some  problems  in  net-based  learning  environments 
from  the  instructors  or  tutors  points  of  view.  The  problem  is  to  keep  in  touch  with  the  students  and  monitor  their 
learning  in  net-based  learning  environments  in  order  to  support  them.  Lastly  I will  introduce  an  NBLE’s  student 
monitoring  tools  and  some  ideas  about  the  better  tools  we  will  test. 


Constructing  the  knowledge 

According  to  the  constructivist  view  of  learning,  knowledge  is  subjective  with  various  possible  viewpoints. 
Knowledge  is  contextual  and  is  collaboratively  built.  Thus  it  is  not  a ready-made  and  unchanging  package  that 
could  be  given  to  the  learners.  The  learners  are  active  processors  of  knowledge  and  central  actors  in  the  learning 
process.  They  form  their  own  views  and  assimilate  knowledge  into  their  existing  cognitive  structures,  adding  to 
or  deepening  them.  The  constructivist  conception  assumes  that  all  learners  have  an  ability  to  learn.  Talents  are 
not  a predestined,  unchanging  quality. 

It  is  unlikely  that  the  learners  would  build  their  own  knowledge  without  being  interested  in  the  learning  domain, 
or  without  setting  their  own  goals  in  some  ways.  Their  own  activity,  intend onality  and  emotions  play  a very 
important  role  in  learning.  This  is  emphasized  especially  in  net-based  learning  environments.  For  example, 
frequently  different  kinds  of  technical  problems  during  the  course  annoy  students  and  decrease  their  studying 
activity. 
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Situated  and  meaningful  learning 


If  knowledge  is  not  a ready-made  and  unchanging  package  that  could  be  given  to  the  learners,  then  a teacher’s 
role  is  more  creating  environments  and  possibilities  for  the  learners  to  learn.  The  most  effective  environment  for 
learning  resembles  the  real  life  as  closely  as  possible,  replicating  its  typical  elements  and  problems.  Learning 
becomes  meaningful  when  the  learning  environment  has  a genuine  connection  to  the  real  world. 

Situated  learning  means  that  learning  is  placed  in  a social  and  cultural  context.  An  important  part  of  information 
is  meaningful  only  in  relation  to  its  context.  From  the  viewpoint  of  learning,  it  is  important  that  learning  is  not 
separated  from  the  reality  of  the  learner  or  the  society.  Winn  (1993)  argues  that  situational  learning  takes  place 
when  the  learners  work  with  authentic  tasks  in  real-world  settings.  Authentic,  real-life  situations  can  simulate  an 
expert's  ways  of  action. 


Metacognition  and  self-directedness 

An  essential  part  of  learning  and  knowledge  building  are  also  the  metacognitive  skills.  These  refer  to  the  ability 
of  directing  one's  own  learning.  It  is  important  to  have  good  facilities  for  learning,  because  contemporary  life 
constantly  offers  new  challenges,  and  in  the  rapid  increase  of  information  it  is  impossible  to  learn  everything. 
Thus  it  is  important  to  acquire  general  principles,  i.e.  skills  related  to  learning,  thinking,  co-operation  and 
regulation  of  learning.  Metacognition  means  the  learner’s  awareness  of  their  cognitive  and  emotional  processes, 
such  as  remembering,  knowing,  thinking,  and  their  ability  to  control  them.  On  the  basis  of  metacognitive 
knowledge,  the  learners  can  consciously  follow  and  direct  their  own  processes  of  learning  and  thinking. 
Metacognitive  skills  include,  for  instance,  setting  goals,  planning  learning,  choosing  suitable  material,  making 
consistent  decisions,  receiving  and  taking  advantage  of  feedback,  and  evaluating  learning.  These  skills  help  the 
learner  to  choose  the  right  learning  strategies  in  virtual  learning  environments. 

When  the  learner  has  good  skills  in  self-direction  and  metacognition  and  a deep  knowledge  of  the  domain,  we 
can  call  him/her  an  expert  learner  or  a specialist.  He  can  control  his  learning.  The  opposite  of  an  expert  is  a 
novice,  whose  knowledge  on  the  subject  matter  is  more  superficial  and  self-direction  skills  are  weaker  than 
those  of  an  expert.  A novice  needs  guidance  and  support  in  learning  and  self-direction  to  be  able  to  strive  for 
expertise.  The  goal  of  a novice  learner  should  be  to  become  an  expert. 

Especially  the  novice-learner  needs  supporting  e.g.  for  solve  more  complex  tasks  than  he  or  she  could  solve 
alone.  It  is  important  that  the  learner  gets  support  when  she  or  he  needs  it  and  just  the  necessary  amount  to  get 
further.  This  is  a problem  in  net-based  learning  environments,  the  main  problem  is  that  the  instructor  has  not 
sufficient  tools  to  monitor  the  students’  learning  process  to  support  it. 


Supporting  in  NBLE 

In  net-based  learning  environments  (NBLE),  such  as  TopClass  and  Lotus  Learning  Space,  the  learner  can  be 
offered  meaningful  and  authentic  learning  situations.  It  is  easy  for  the  learner  to  get  different  views  to  and  make 
searches  to  the  material  but  supporting  is  the  problem.  Net-based  learning  environments  contain  different  kinds 
of  discussion  lists,  shared  workspaces,  email,  and  chat  facilities.  It  is  also  possible  to  create  different  help  and 
hint  systems.  But  how  can  the  help-systems,  tutors  or  instructors  know  when  the  student  needs  support  if  she  or 
he  does  not  ask  for  it?  It  is  impossible  or  at  least  very  hard  to  observe  the  learner  closely  enough  and  get 
information  about  his  or  hers  emotions.  Server  logs  are  very  hard  to  read.  Learning  diaries  would  be  one  chance 
but  they  are  not  written  “online”  when  things  happen  but  afterwards  when  the  strongest  feelings  have  passed. 

For  example,  TopClass  has  some  tools  to  monitor  the  students.  The  instructors  can  view  the  status  of  the 
courseware  for  any  of  their  assigned  students  or  classes  individually,  and  monitor  how  they  are  progressing. 
Picture  1 shows  the  different  views  that  the  instructors  are  able  to  see.  The  views  are  quite  general  telling  just 
which  percentage  of  the  material  has  been  read  or  how  many  points  have  been  scored. 

One  possibility  to  enhance  feedback  would  be  a “feeling  console”  which  gathers  information  about  the  learners’ 
feelings  and  difficulties  in  their  learning  sessions.  The  console  could  also  be  used  to  obtain  information  about 
the  problems  in  the  environment.  The  console  would  consist  of  a few  buttons  and  it  could  be  very  easy  to  use.  It 
would  be  easier  for  the  tutor  to  support  the  learner  with  the  help  of  this  additional  information. 
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A visual  log-analyzer  could  be  another  solution.  At  the  moment  one  of  our  projects  is  developing  such  a tool. 
The  tool  can  display  different  views  of  the  log  file  to  instructor,  e.g.  the  student’s  navigations,  when  they  have 
sent  or  read  e-mail.  The  idea  is  that  the  instructor  is  able  to  quickly  view  pictures  or  animations  of  the  students’ 
actions  in  the  NBLE.  Both  of  these  tools  would  help  the  instructor  to  obtain  quickly  and  easily  more  information 
about  the  students  to  support  them. 
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Picture  1 : Top  Class  student  and  class  monitoring  tools. 


Conclusions 

The  learner  should  be  offered  meaningful,  authentic  material  that  relates  to  real-world  situations  and  problems. 
The  learner  needs  also  scaffolding,  so  she  or  he  will  be  able  to  effectively  build  knowledge  and  develop  his  or 
her  learning  skills. 

The  four  points  of  Goldman  et  al.  (1998)  sum  up  the  main  constructive  principles  that  I presented  in  this  paper. 

1 . Instruction  is  organized  around  meaningful  learning  and  appropriate  goals. 

2.  Instruction  provides  scaffolds  for  achieving  meaningful  learning. 

3.  Instruction  provides  opportunities  for  practice  with  feedback,  revision  and  reflection. 

4.  Instruction  is  arranged  to  promote  collaboration,  distributed  expertise,  and  entry  into  a discourse  community 
of  learners. 

Scaffolding  the  student  is  the  most  difficult  thing  to  achieve  in  NBLE.  One  way  to  make  it  easier  or  effective  is 
to  develop  tools  that  provide  different  kinds  of  information  about  the  students.  With  this  additional  information 
it  is  easier  to  scaffold  the  student  when  she  or  he  really  needs  it. 
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Abstract:  Special  education  offers  a challenging  background  for  designers  of  educational 
software.  One  learner  group  are  those  who  have  deficiencies  in  mental  programming:  they  have 
difficulties  in  structuring  a learning  task  in  its  entirety,  and  organizing  the  plan  to  go  through  it. 
Understood  as  a crew  of  collaborative  co-1  earners,  agents  provide  promising  solutions  to  this  area, 
to  be  applied  also  in  other  educational  contexts. 


Motivation 

Special  education  suffers  from  the  lack  of  computer  science  oriented  research  and  moderate  computer 
expertise  in  existing  educational  software.  In  fact,  there  are  only  few  serious  efforts  of  exploitation  of  state-of-the- 
art  computer  science  methods  and  techniques  to  advance  special  education  (see  Klaus  et  al.  (1996)  and  Edwards  et 
al.  (1998)  for  the  lack  of  examples).  Besides  drills,  there  are  only  basic  learning  programs,  but  challenging  and 
truly  helpful  learning  environments  do  not  exist.  Most  of  the  software  for  special  education  are  for  visually 
impaired  users  (Eriksson  et  al.  1997).  There  are  also  programs  for  motorically  and/or  hearing  impaired,  but 
cognitive  impairments  (e.g.  people  with  learning  difficulties)  are  rarely  addressed.  It  seems  that  in  the  context  of 
special  education,  computers  are  used  mainly  as  assistive  technology  to  overcome  communication  barriers,  instead 
of  educational  tools  to  deepen  the  learning  process. 

Can  one  use  those  numerous  adaptive  learning  environments  that  have  been  researched  and  produced  for 
non-disabled  education  in  special  education?  In  general,  no.  The  reasons  are  threefold,  related  to  the  user  interface, 
the  content,  and  the  processing  of  the  content  (i.e.  the  storyboard  of  the  program).  In  addition  to  the  need  of 
extra-ordinary  input  and  output  (Edwards  1995),  the  learning  content  (or  the  topic)  is  usually  special.  There  is 
hardly  any  need  for  a Lisp  course  in  the  special  education  curriculum  but  the  need  for  educating  children  to  handle 
everyday  life  is  essential. 

Moreover,  not  only  the  topic  must  differ  from  standard  educational  software.  The  learning  style  among 
regular  learner  groups  is  rarely  similar  to  that  of  non-disabled  education.  For  example,  let  us  assume  a person 
cannot  formulate  problem-solving  strategy  for  a simple  task.  In  that  case,  the  learning  environment  might  have  to 
partition  the  task  into  subtasks,  so  that  the  learner  is  lead  to  the  final  goal  step-by-step  (Kurhila  and  Sutinen  1998). 
These  kinds  of  requirements  result  in  programming-intensive  solutions:  the  emphasis  is  no  more  in  the  quantity  of 
information,  but  the  way  it  is  processed  and  presented  to  the  learner.  In  order  to  deal  with  cognitive  disabilities,  the 
agents  related  to  the  user  interface,  the  content  and  its  processing,  must  cooperate  with  each  other  and  the  human 
cognition  for  a consistent  learning  experience.  The  challenges  are  deeper  than  finding  another  way  for 
human-computer  interaction.  However,  fresh  solutions  in  the  area  of  special  education  can  contribute  in  a positive 
manner  to  regular  education  which  after  all  consists  of  diverse  learners,  with  special  needs,  though  often  hidden. 
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The  shortcomings  of  available  software  and  the  challenges  of  special  education  provide  an  interesting 
scene  to  observe  the  functionality  of  agents  in  a versatile  and  diverse  learning  environment.  Our  view  of  the 
applicability  of  agents  is  based  on  our  experiences  in  designing  a learning  environment  to  children  with  problems 
with  their  mental  programming  (Vilkki  1995,  Korkman  1988).  When  simplified,  we  can  characterize  mental 
programming  as  the  ability  to  uphold  the  motivation,  ability  to  compose  a problem  solving  strategy,  fluency  in 
solving  various  tasks,  and  ability  to  uphold  attention.  Therefore,  learners  with  problems  in  these  fundamental  skills 
need  a targeted  support  from  the  learning  environment,  whether  computerized  or  not.  In  a non-computerized 
setting,  this  support  is  seen  as  auxiliary  personnel,  such  as  occupational  therapists  and  neuropsychologists.  We  are 
exploring  how  these  various  tasks  can  be  intensified  by  agent  technology.  Therefore,  our  architecture  is  that  of 
multi-professional  agents,  or  a crew  of  specialized  agents. 

Intelligent  tutoring  systems  have  traditionally  emphasized  a method  to  keep  the  learner  within  a narrow 
optimum  solution  path  and  optimizing  the  time  consumption  to  acquire  a new  concept.  Agents  are  conceptually 
ready  to  support  more  constructive  learning  processes:  a learner  builds  her  knowledge  freely,  not  necessarily  along 
the  optimum  learning  path  but  along  the  paths  supporting  that  particular  learner’s  learning  process. 


A crew  of  Specialized  Agents 

Several  computer-based  learning  environments  either  focus  on  drilling  one  or  few  skills  or  areas  of 
knowledge,  resulting  in  a biased  learning  process,  or  cover  a huge  amount  of  information,  reflected  as  superficial 
learning  outcomes.  Contrary  to  these  approaches,  we  aim  at  utilizing  the  full  potential  of  the  computer  to  intensify 
the  diverse  factors  of  the  learning  process.  For  that  purpose,  an  agent-based  architecture  offers  a valid  platform  for 
development:  each  part  of  the  learning  process  can  be  supported  by  a dedicated  agent. 

As  an  example,  the  learner’s  activities  can  be  intensified  by  an  agent  which  helps  her  to  go  through  the 
learning  material,  traversing  trough  the  material  in  a personalized  manner.  At  the  same  time,  an  agent  supporting 
the  teacher  can  recognize  areas  where  students  needs  more  tuition.  However,  a modem  learning  environment 
consists  of  more  than  the  roles  mentioned,  A feedback  agent  might  help  teachers  or  administration  staff  to  evaluate 
the  overall  learning  process  and  its  organization,  with  the  help  of  other  agents,  a visualization  agent  might  help  the 
students  to  interpret  their  material  in  different  formats,  such  as  concept  maps. 

Our  approach  is  consistent  with  the  common  definitions  given  for  agents  in  the  literature.  For  example, 
Russell  and  Norvig  (1995)  state  an  agent  to  be  “anything  that  can  be  viewed  as  perceiving  its  environment  through 
sensors  and  acting  upon  that  environment  through  effectors”.  A more  demanding  definition  is  formulated  by 
Franklin  and  Graesser  (1996).  They  define  an  agent  by  giving  an  exhaustive  list  of  properties.  An  agent  must  fulfill 
at  least  the  first  four  properties  of  their  list.  In  other  words,  an  agent  must  be  reactive,  autonomous,  goal-oriented, 
and  temporally  continuous. 

Although  our  agents  are  agents  by  definition,  fundamental  differences  regarding  the  purpose  of  agents 
exist.  The  most  noticeable  is  that  our  agents  do  not  act  on  the  behalf  of  the  user  or  execute  the  task  for  the  user  but 
help  the  learner  in  conceptualizing  and  executing  the  tasks.  In  other  words,  we  employ  agents  as  an  interpretation 
mechanism  between  different  participants  of  the  learning  process.  This  means  that  agents  - though  autonomous  - 
adapt  to  the  human  beings  involved  in  the  learning  process.  In  a way,  they  are  co-workers  rather  than 
auto-workers,  a common  emphasis  in  many  of  the  prevailing  agent-based  frameworks. 

To  use  an  example  from  another  context,  an  agent  taking  care  of  its  master’s  business  on  the  web-based 
stock  market,  is  an  auto-worker  accomplishing  a specified  task  for  an  optimal  solution,  whereas  another  agent  with 
a co-worker’s  character  would  help  its  master  to  understand  the  secrets  behind  a long-term  survival  on  the  very 
same  markets,  and  support  him  or  her  for  life-long  business  learning. 


Assistive  and  Adaptive  Hypermedia  - Ahmed 

Ahmed  is  the  first  implemented  prototype  of  the  agent  crew  described  above.  Ahmed  is  an  agent  for  the 
learner.  There  are  no  limitations  regarding  the  content  or  the  content  presentation  the  agent  can  handle.  Still, 
Ahmed  can  provide  a personal  learning  experience  to  every  learner.  The  adaptation  mechanism  is  independent  of 
the  system,  and  at  the  same  time  dependent  on  the  learning  material. 
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A term  “learning  seed”  or  just  “seed”  is  used  throughout  the  following  text  when  referring  to  one  piece  of 
a learning  material  in  a learning  space.  This  is  to  emphasize  that  the  material  can  be  more  than  merely  an  exercise, 
and  more  than  a frame  (a  basic  unit  in  a CAI  program,  as  pointed  out  by  Wenger  1987).  A learning  seed  in  Ahmed 
can  consist  of  several  different  screens  with  several  sequential  or  parallel  visual,  auditorial  or  functional  elements 
with  temporal  values.  The  seeds  can,  for  example,  be  states  of  a simulated  world  (Kurhila  and  Sutinen  1 999). 


Overview  of  Functionality 

The  description  language  used  in  our  prototype  learning  environment  in  describing  the  learning  seeds  is 
defined  for  this  purpose  by  using  XML.  Compared  to  e.g.  HTML,  the  description  language  is  significantly  more 
expressive.  It  includes  tags  for  buttons,  graphics,  text  areas,  and  audio  clips.  The  functionality  of  the  learning 
material  is  secured  with  tags  for  parallel  and  sequential  presentation  of  other  elements  with  a duration  attribute. 
Conditioning  with  an  if-then-else  structure  enhances  functionality. 

The  learning  space  is  «-dimensional.  The  dimensions  are  not  pre-determined  by  any  means.  Instead,  they 
are  inferred  from  the  parameters  in  learning-seed  headings  in  run-time.  For  example,  if  the  heading  of  a seed 
contains  “Skill:  56”  and  “Knowledge:  49”,  the  seed  is  situated  in  point  (56,49)  in  a learning  space  of  two 
dimensions,  called  Skill  and  Knowledge  (Fig.  1). 


Figure  1:  Parameters  are  used  in  situating  the  learning  material  into  the  learning  material  space.  For  simplicity,  the  space  is 
here  only  two-dimensional 

Every  learning  seed  should  be  parameterized,  since  the  parameters  define  the  position  of  a seed  in  the 
learning  space.  However,  not  every  seed  needs  to  be  located  with  respect  to  every  dimension.  For  example,  if  the 
dimensions  used  in  the  space  are  “Skill”  and  “Knowledge”,  some  seeds  can  be  defined  in  the  “Skill”  dimension 
only.  In  that  case,  the  seeds  without  an  explicit  location  in  “Knowledge”  would  be  represented  as  a line  covering 
every  point  in  the  “Knowledge”  dimension.  This  procedure  ensures  that  a person  creating  the  learning  material 
does  not  have  to  assign  irrelevant  parameters  to  the  material.  For  example,  if  one  learning  space  consists  of 
arithmetic  exercises,  there  is  no  purpose  to  assign  parameters  in  the  “Reading  comprehension”  dimension  to  those 
exercises. 

The  dimensions  and  their  parameters  are  freely  selectable  for  the  person  creating  the  material  in  the  sense 
that  they  can  be  represented  with  arbitrary  strings.  It  is  natural  to  denote  the  dimensions  with  meaningful  learning 
objectives  such  as  “Multiplication  skill”  rather  than  abstract  ones. 

An  activating  learning  seed  normally  makes  it  possible  for  a learner  to  do  something.  Every  action  can  be 
equipped  with  an  effect  to  the  learner’s  position  in  the  learning  space.  This  is  called  a relative  movement.  For 
example,  if  the  learner  makes  a choice  which  is  parameterized  “Skill  +6”  and  “Knowledge  +8”,  the  learner’s 
position  in  the  learning  space  moves  from  point  (56,49)  to  the  point  (62,  57)  (Fig.  1).  Because  of  the  several 
choices  the  learner  can  make  in  a single  learning  seed,  the  learners  are  likely  to  follow  different  paths  in  the 
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learning  material  space. 

In  addition  to  the  relative  movement  described  above,  there  is  also  another  way  to  present  the  next 
learning  seed  to  the  learner.  It  is  possible  that  the  multiple-choice  answers  have  an  absolute  movement  to  another 
learning  seed.  In  such  a case,  the  learner  is  not  taken  to  the  next  seed  by  calculating  the  parameter  values  relative 
to  the  learner’s  profile  but  directly  by  following  an  absolute  link.  In  other  words,  the  learner  is  not  guided  in  the 
space  but  in  a static  learning  material  graph  (which  still  is  a part  of  the  learning  space).  The  learning  material  can 
be  arranged  as  a space  of  groups  of  static  graphs,  but  the  most  common  case  of  arranging  the  learning  space  may 
be  to  have  learning  seeds  with  only  relative  movements  and  occasional  local  graphs  of  seeds  with  absolute 
movements. 

The  multiple-choice  answers  in  the  learning  material  are  not  necessarily  traditional  “pick  one  from  the 
choices”.  A seed  may  require  a learner  to  choose  a subset  of  given  choices.  The  subset  can  also  be  an  ordered 
subset.  An  example  of  an  exercise  with  a subset  of  given  choices  is  to  find  a meaningful  combination  of  objects  to 
construct  a steam  engine.  An  example  of  an  exercise  with  an  ordered  subset  is  one  with  a task  to  organize  different 
words  into  a meaningful  sentence. 

The  parameters  of  the  choices  the  learner  makes  are  used  in  providing  the  personalization  of  the  learning 
experience.  When  the  learner  makes  choices,  her  profile  can  be  constructed  by  saving  the  information  in  the 
parameters  to  the  learner’s  profile  file.  While  the  learner  is  represented  as  a point  in  the  ^-dimensional  learning 
space,  every  move  of  the  learner  in  the  space  is  saved  to  the  learner’s  profile,  so  that  the  learning  path  of  the 
learner  can  be  represented  numerically  or  even  visualized  as  an  animation  for  evaluation. 

In  a standard  situation,  the  dimensions  of  the  learning  space  can  be  viewed  as  learning  objectives,  since  a 
learner  strives  to  achieve  a higher  position  in  every  dimension.  However,  dimensions  are  not  necessarily  related  to 
any  objective,  but  they  might  indicate  any  other  aspect  of  the  learning  process  as  well,  or  only  those  learner’s 
properties  that  a teacher  wants  to  observe.  As  an  example  of  a property  to  observe,  we  can  consider  “Motivation”. 
To  enable  the  measurement  of  motivation,  every  learning  seed  can  contain  the  button  “I  don’t  care”.  This  button  is 
equipped  with  an  effect  (relative  movement)  to  the  “Motivation”  dimension.  But,  because  none  of  the  actual  seeds 
is  parameterized  with  a “Motivation”  value,  choosing  “I  don’t  care”  does  not  have  an  effect  on  the  guidance  of  the 
learner  in  the  learning  space.  Instead,  it  affects  only  the  learner’s  profile:  every  time  the  learner  presses  the  button, 
the  action  is  recorded  to  the  learner’s  profile,  thus  enabling  the  observation  for  the  teacher. 

It  is  also  important  to  notice  that  the  flexibility  of  the  parameterization  enables  the  use  of  nested  or 
hierarchical  objectives.  For  example,  after  the  first  answer  it  might  be  hard  to  assess  the  learning  outcome  whereas 
a series  of  questions  gives  more  conclusions  (e.g.  concerning  the  type  of  error).  A very  simple  example  is  the 
following.  The  learner  is  presented  an  exercise:  1 x 1 . If  the  learner  answers  2,  it  may  very  well  be  that  the  learner 
confuses  adding  and  multiplication,  or  simply  does  not  know  how  to  multiply.  However,  this  is  hard  to  tell  after 
only  one  answer,  because  it  could  have  been  a random  error.  By  repeating  learning  seeds  of  the  same  kind,  a 
correct  conclusion  can  be  made. 


Other  Characteristics  of  Ahmed 

Because  of  the  intended  users,  so-called  single-switch  input  with  automatic  scanning  of  choices  must  be 
available  all  the  time.  Therefore,  the  login  procedure  and  the  answering  to  the  exercises  are  designed  to  function 
with  single-switch  input.  The  login  is  conducted  as  follows.  The  learner  starting  the  session  is  presented  with  the 
pictures  of  her  class.  The  learner  chooses  a picture  representing  her  (it  can  be  a photo  or  a drawing).  To  ensure  that 
none  of  the  learners  chooses  the  wrong  picture  and  thus  an  incorrect  profile,  the  learner  has  to  choose  her  “secret 
picture”.  This  corresponds  to  the  password.  It  is  also  possible  to  require  a sequence  of  secret  pictures  to  diminish 
the  probability  of  logging  in  with  false  pretenses.  The  pictures  are  in  a single  directory,  so  that  it  is  fairly  easy  for 
the  teacher  to  update  the  pictures  of  learners  attending  her  class. 

The  ordered  subset  can  also  be  picked  up  by  a single-switch  input.  Still,  it  is  possible  for  the  learner  to 
correct  a misprint  before  giving  the  final  answer.  This  is  important,  because  misprints  in  answers  are  hard  to 
interpret  correctly  by  a computer.  In  addition,  it  is  tremendously  demotivating  for  a learner  not  to  be  able  to  correct 
an  accidental  misprint. 

Another  feature  of  Ahmed  is  the  possibiliy  to  use  learning  material  servers  to  store  the  different  learning 
spaces.  The  learner  chooses  the  learning  material  space  when  starting  a session.  Technically,  every  space  is  a 
directory.  The  learning  material  directory  can  be  situated  in  a remote  Web  server  as  well  as  in  the  local  hard  drive. 
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This  enables  easy  networking  and  centralized  composing  of  the  learning  material. 

To  exploit  the  full  potential  of  the  schema,  the  learning  material  space  should  be  complete,  i.e.  filled  with 
learning  material  so  that  every  point  in  the  space  has  more  than  one  learning  seed.  In  practice  this  means  that  at 
least  one  thousand  learning  seeds  should  be  prepared.  Therefore,  centralized  learning  material  databases  are  a 
choice  to  consider,  since  composing  the  material  is  a tedious  task. 

To  involve  the  teacher  in  the  learning  environment,  we  have  implemented  also  a console  for  the  teacher. 
If  one  of  the  learners  presses  the  help  button,  a message  is  sent  to  the  teacher’s  computer.  One  has  to  keep  in  mind 
that  the  learners  in  this  learning  environment  can  be  severely  disabled,  so  that  they  cannot  draw  the  attention  of  the 
teacher  by  any  other  means.  The  message  is  merely  help,  because  the  schema  was  designed  to  be  used  in  a 
classroom  with  a teacher  always  present.  Technically  the  help  functions  properly  even  if  the  learning  environment 
is  expanded  to  the  whole  Internet,  since  every  help  message  comes  with  the  IP  address  of  the  sender’s  machine. 


Discussion 

When  comparing  Ahmed  to  the  well-known  and  traditional  four-component  schema  of  intelligent  tutoring 
systems  (Wenger  1987),  we  can  identify  the  same  four  concepts  in  Ahmed,  as  well.  The  domain  knowledge  in 
Ahmed  is  the  contents  of  the  seeds  in  a learning  space.  The  pedagogical  model  corresponds  to  the  dimensions 
used,  the  parameterisation  of  the  seeds,  and  the  parameterisation  of  the  actions  in  the  seeds.  The  student  model  is 
the  trail  (history  of  every  visited  seed  and  actions  taken  in  them)  of  every  individual  learner.  The  interface  model  is 
the  organization  of  the  objects  in  seeds,  since  Ahmed  itself  is  a tabula  rasa  if  no  learning  material  exists. 

Both  the  strength  and  the  weakness  of  this  prototype  is  that  the  domain  knowledge  is  completely  isolated 
from  the  agent.  The  agent  does  not  know  anything  about  the  learning  material.  It  merely  adds  (or  subtracts  if  the 
parameter  has  a negative  value)  the  parameters  together,  and  presents  the  equivalent  learning  seed  to  the  learner. 
The  advantage  of  this  concept  is  that  the  material  can  truly  be  anything  and  yet  the  system  can  adapt  to  every 
learner  independently.  Every  author  of  the  learning  material  is  able  to  create  the  parameters  and  the 
multiple-choice  answer  types  based  on  her  own  experience,  or  some  theoretical  model. 

The  domain-independence  of  the  system  enables  various  possibilities.  There  can  be  exercises  with  right  or 
wrong  answers,  but  there  also  can  be  more  adventurous  problem  solving  with  e.g.  ethical  values.  An  example  of 
more  adventurous  problem  solving  could  be  to  guide  a child  (or  another  living  being  the  learner  can  relate  to)  to 
visit  her  Grandmother  in  a countryside,  and  there  are  no  right  or  wrong  answers,  only  different  ways  to  cope  with 
the  problems  until  the  final  goal  is  achieved.  Whereas  an  intelligent  tutoring  system  might  make  the  grandchild  to 
hurry  to  the  Grandma  in  the  optimal  time,  with  the  minimum  amount  of  money  consumed,  our  system  allows  the 
child  to  observe  the  path  as  well,  even  with  wolves  along  the  route. 

Because  the  domain  knowledge  is  separated  from  the  agent,  the  learner  modeling  is  imperfect.  The 
modeling  can  be  adequate  if  the  parameters  and  the  content  are  suitable.  The  problem  is  that  the  person  authoring 
the  material  should  figure  out  in  advance  what  kinds  of  answers  the  learner  could  choose.  The  parameterization  of 
the  learning  material  and  the  multiple-choice  answers  are  not  trivial  either. 

The  impact  of  an  computer-based  educational  system  should  always  be  validated  empirically  (Brusilovsky 
1996,  Brusilovsky  and  Eklund  1998).  Although  Ahmed  has  not  yet  been  tested  in  classrooms,  discussions  with 
special  teachers  and  other  experts  indicate  that  the  idea  behind  Ahmed  is  useful  and  has  potential.  Empirical 
evaluation  is  waiting  for  meaningful  learning  material:  to  operate  properly,  Ahmed  has  to  contain  vast  amount  of 
learning  seeds.  For  the  learning  material  authoring,  we  are  designing  an  easy-to-use  tool  to  speed  up  the  authoring 
process. 
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Abstract:  The  educational  hypermedias  have  become  indispensable  tools  to  enrich  the 
pedagogical  value  through  the  formation.  Meanwhile  the  cognitive  overcharging  and  the 
disorientation,  the  hypermedia  presents  another  drawback  : the  isolation  of  the  learner. 

This  work  presents  the  architecture  of  an  hypermedia  made  in  a way  to  reduce  to  the 
maximum  the  cognitive  overcharge.  This  hypermedia  is  able  to  advice  the  disoriented 
learner  thanks  to  a combination  with  an  intelligent  tutor.  This  tutor  itself  permits  the 
learners  to  cooperate  with  each  other  di spite  the  distance  between  them. 


1.  Introduction 


We  witness  day  to  an  exponential  growth  of  hypermedias,  to  varied  and  multimedia  content,  and  of  their  users, 
whose  number  is  considerably  increasing  day  to  day,  thanks  to  the  new  technologies  of  diffusion:  CD-ROM, 
Internet,  Intranet,  etc  (Chick  & Daoudi  1999).  Hypermedia  environments  become  indispensable  tools  to  enrich 
the  educational  value  in  the  teaching  and  the  learning  (Nkambou,  Isaballe,  Gauthier  1 998).  Hence,  the  multiple 
courses  of  reading  the  hyperdocument  allow  everybody  to  organize  his/her  own  course  in  order  to  encourage 
his/her  understanding  and  his/her  memorization  of  knowledge.  The  initiative  of  learner  is  the  basis  strategy 
(Bensebaa  1991). 

Whereas  if  the  use  of  the  hypermedia  reinforces  the  autonomy  principle  of  the  learner,  it  also  risks  to  reinforce 
his/her  isolation.  If  it  is  undeniable  that  this  autonomy  can  have  some  positive  educational  effects  compared 
with  the  masterly  teaching,  such  as  the  respect  of  the  rhythm,  of  the  learning  length  and  in  certain  cases  to 
permit  a plurality  of  learning  styles,  it  is  undeniable  that  this  relation  man  to  “ knowledge  “ stocked  in  machine 
lets  only  small  place  to  the  mediation  by  another  person  (tutor,  teacher,  parents,  other  learners).  In  some 
methods  of  teaching,  such  as  the  distance  teaching,  this  isolation  is  reinforced  by  the  existed  geographical 
gates. 

However,  it  also  appears  that  the  teaching,  the  learning,  have  always  been  considered  as  social  processes 
requiring  a collaboration  between  different  actors.  It  is  sufficient  to  think  about  all  approaches  of  the  adult 
formation  based  on  the  group  dynamics  (Derycke  1991). 

In  addition  to  the  learner’s  isolation  problem,  the  hypermedias  have  other  problems.  Indeed,  if  the  aspects  of 
learner’s  interaction  and  participation  to  educative  sequence  can  be  now  considered  as  efficient,  the  cognitive 
overload  and  the  disorientation  of  the  user  in  the  hypermedia  remain  the  major  problems.  It  is  also  very 
current  to  note  at  the  time  of  the  use  of  an  hypermedia,  that  the  learner,  after  some  minutes  of  navigation  and 
research,  doesn’t  know  its  position  relatively  to  notions  to  be  consulted.  The  big  freedom  allowed  to  the  user 
which  is  the  principal  attraction  of  the  hypermedia  becomes  its  principal  drawback.  Its  pedagogical  mission  of 
knowledge  construction  is  so  compromised. 


2.  Proposals 


2.1.  Break  the  Isolation 


Several  works  and  in  particular  those  of  Vygotsky  showed  the  importance  of  the  social  dimension  of  learning: 
the  social  interaction  is  an  essential  element  of  the  cognitive  transformation  (Derycke  1991). 

Choplin  et  al.  in  (Choplin,  Galisson,  Lemarchand  1998)  affirm  from  a didactic  point  of  view  that  an  activity  of 
learning  in  science  implies  in  general  manner  the  set  in  motion  of  the  student’s  conceptions  (cognitive 
dimension),  motivation  (emotional  dimension)  as  well  as  his/her  relations  with  his/her  peers  and/or  with  the 
teacher  (social  dimension). 

The  survey  of  works  or  written  of  pioneers  or  visionary  of  hypertexts  and  hypermedias  make  appear  that  the 
dimension  cooperative  work  was  underlying  since  the  beginning  (see  (Derycke  1991)  for  more  details  upon 
some  researches  on  this  topic). 

The  application  of  cooperation  principles  permits  to  eliminate  the  isolation  of  the  learner.  Therefore, 
cooperation  depends  upon  a supportive  community  of  actors  who  agree  to  help  one  another  in  activities  aimed 
to  attaining  the  goals  of  each  person  involved  (Lewis  1996).  So,  cooperation  consists  in  amplifying  capacities  of 
agents  which  are  taken  separately  and  to  increase  their  performances  (Ferber  1994).  It  is  to  help 
himself/herself  by  helping  his/her  partner  (Leroux  1995). 

2.2.  Reduce  the  cognitive  overload 

The  organization  of  the  hypermedia  separates  the  information  from  the  knowledge.  These  are  often 
encapsulated  in  the  same  node.  Links  are  not  structured  and  their  conceptual  values  are  not  specified.  Of  where 
a difficulty  for  the  author  to  create  and  to  maintain  an  hypermedia  and  for  the  reader  to  navigate  (Soula  1 994). 

2.3.  Orient  learner 

This  orientation  is  possible  thanks  to  coupling  the  hypermedia  with  a tutor  which  is  capable  to  judge  the 
knowledge  of  the  learner  and  also  can  advise,  orient,..., the  disorientated  learner  when  it  is  solicited.  For  doing 
this,  the  tutor  must  rely  on: 

• the  student's  models, 

• training  objectives, 

• a historic  of  the  training  session, 

• an  educational  strategy  family. 

Beyond  its  role  of  orientation  counselor,  the  tutor  will  inform  learners  upon  cooperation  possibilities.  Those  are 
deducted  from  student's  models. 


3.  Proposed  architecture 


The  figure  1 represents  the  proposed  architecture  of  the  system,  we  distinguish  in  particular: 

3.1  Author  interface 

The  organization  of  the  hypermedia  is  based  on  the  manipulation  of  predefined  objects  and  on  the  separation 
between  information  and  knowledge.  So  the  following  main  characteristics  (Soula  1994): 

• each  object  in  the  hypermedia  belongs  to  a specific  type, 

• a separation  between  information  nodes  and  knowledge  for  the  realization  of  links.  Information  corresponds 
to  what  learner  wants  to  reach  and  the  knowledge  concerns  the  structure  of  the  document  that  is  how  to 
reach  it. 

Thus,  nodes  and  links  are  managed  in  the  following  manner: 

1 . all  the  nodes  are  instances  of  “typical  nodes”;  these  “typical  nodes”  permit  to  create: 
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• multimedia  nodes, 

• “exercise“  nodes  for  the  control  of  knowledge.  The  assessment  of  answers  is  assured  by  pondered 
production  rules  triggered  in  forward  chaining  : 

If  Answer  Then  Pedagogical-Objective  {Credit,  Debit) 

Where  Credit  and  Debit  represent  points  to  be  added  or  to  be  subtracted  to  the  present  capital  of  learner  for  the 
current  objective.  


t 


Figure  1:  The  Architecture  of  the  system. 


2.  links  can  be: 

• Explicit : managed  entirely  by  the  author,  these  links  permit  a non-linear  reading  of  the  hypermedia.  They 
are  activated  by  the  learner  by  clicking_on  an  object  of  the  node  (called  hyperobjet), 

• Implicit  : these  links  are  managed  automatically  by  the  system.  They  can  bee:  following  page,  previous 
menu,  beginning  of  the  document,  etc.  These  links  permit  the  structuring  of  the  document  under 
hierarchical  shape, 

3. LL  Hyper objet 
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It  is  a knowledge  activated  by  the  learner  by  clicking  on  an  object  (elementary  or  composed).  An  hyperobjet  is 
defined  by:  a name,  a password  identifying  the  owner  and  a list  of  couples:  intention-action. 

3. 1 .2. Intention-action 

It  is  the  element  of  an  hyperobjet  that  defines  a link.  The  intention  carries  on  the  reason  of  the  link  and  the 
action  on  its  realization.  If  the  main  feature  of  an  hypermedia  is  to  allow  the  reader  a completely  free  course, 
the  notion  of  intention-action  offers  information  in  order  to  permit  that  this  course  wouldn’t  be  made  “ at  the 
blind".  Thus,  the  author  associates  an  intention  to  every  link  that  means  a reason  to  this  creation.  This  reason 
or  intention  represents  a conceptual  knowledge  that  may  be  useful  to  the  reader.  While  consulting  this 
intention,  the  reader  will  have  precision  on  the  action  generated  by  the  link  and  will  either  trigger  or  not  the 
link. 

3.1 .3. Knowledge  Base 

It  is  structured  as  a set  of  concepts  on  the  taught  domain.  These  concepts  are  aggregated  in  educational 
objectives.  An  educational  objective  corresponds  to  a mental  structure,  an  abstraction,  sometimes  represented  by 
conceptual  networks.  It  is  constituted  of  a set  of  nodes.  Two  different  objectives  can  share  the  same  node.  The 
author  can  create  educational  objectives,  associate  to  them  a difficulty  level  and  establish  prerequisite  relations 
between  them.  The  difficulty  level  can  be  determined  according  to  the  nature  of  the  knowledge  underlain  by  the 
educational  objective  (Bensebaa  & Seridi  1 997). 

3.2.  Pedagogue  interface 

It  is  through  this  interface  that  the  pedagogy  used  by  the  tutor  is  defined.  It  consists  to: 

1.  define  a set  of  typical  psychological  profiles  of  target  learners.  A psychological  profile  is  a general  behavior 
of  a learner.  It  is  defined  by  assigning  values  to  attributes  representing  the  different  psychological  attitudes 
(weariness,  attention,  speed,...)  that  the  system  is  capable  to  capture.  Thus,  it  is  possible  to  define  several  bases 
of  different  profiles  corresponding  to  different  target  populations  (Bensebaa  & Lebbihi  1998); 

2.  then,  associate  to  every  typical  psychological  profile,  an  educational  strategy.  The  utilization  of  this  one  by 
the  tutor  consists  of  the  application  of  a set  of  educational  rules  in  a determined  order.  An  educational  rule 
takes  the  form  of  a production  rule: 

If  Condition  Then  Action. 

Rules  appear  as  filters  in  which  we  submit  the  acceptable  educational  objectives  and  that  are  going  to  eliminate 
those  that  appear  too  much  distant  of  the  corresponding  educational  principle.  Thus,  we  have  a big  suppleness 
of  the  adaptation  of  teaching  strategies  (Bensebaa  & Seridi  1997). 

This  manner  to  manage  the  pedagogy  allows  the  system  to  accept  different  pedagogies  and  to  adapt  them 
according  to  the  concerned  domain  and  public. 

3.3.  Trainer  interface 

The  person  responsible  of  the  training  defines  for  every  learner  an  “ interval  of  teaching  “ thanks  to  the 
declaration  of  an  initial  profile  (base  level  of  learner)  and  of  a final  profile  (final  objective  of  the  teaching).  A 
profile  is  a set  of  educational  objectives. 

Also,  the  trainer  determines  the  student’s  initial  psychological  profile.  It  will  permit  the  definition  of  the 
educational  strategy  to  be  applied.  Finally,  the  trainer  can  accede  to  results  on  the  evolution  of  the  knowledge  of 
learners,  objectives  that  they  have  acquired,  the  difficulties  which  faced  the  learners  (Bensebaa  & Seridi  1997), 
etc. 

3.4.  Student’s  models 

The  student’s  representation  is  indispensable  for  an  adaptive  learning  system  (Ritter  & Djahanguir  1991).  A 
learner  model  has  been  considered  a set  of  beliefs  hold  by  the  system  about  the  learner  (Ayala  & Yano  1996). 
Learner  models  provide  the  information  needed  in  order  to  support  the  progress  of  the  learner  and  promote  the 
opportunities  of  effective  cooperation  and  learning. 
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In  order  to  ensure  its  role  of  counselor  for  learner,  the  tutor  is,  actually,  a real  “ spy  Indeed,  all  actions  of 
learner  are  analyzed  and  are  listed  in  his/her  models  so  that  they  indicate  at  every  time  his/her  level  of 
knowledge  and  his/her  psychological  state. 

3.4.1.  Knowledge  student  *s  model 

It  is  composed  of  a set  of  educational  objectives.  It  is  putting  up-to-date  at  each  end  of  visit  of  a node  by  the 
student.  This  up-to-date  takes  into  account  for  each  objective: 

• the  previous  state, 

• the  CREDIT  (or  DEBIT)  in  case  of  an  exercise  node, 

• ... 

For  each  objective,  the  level  of  learner  is  besides  matched  of  a tendency  of  short  term  (acquired  knowledge, 
putting  into  doubt  or  confirmed)  usable  by  the  educational  rules. 

This  way  to  manage  the  model  permits  to  question  the  knowledge  that  is  not  possessed  sufficiently  or  never 
possessed  (case  where  the  initial  profile  of  learner  declared  by  the  trainer  has  been  overvalued)  or,  what  is  more 
grave,  the  prerequisite  objectives,  in  the  case  of  a very  unfavorable  report.  If  the  corresponding  educational  rule 
is  active,  the  tutor  will  be  able  to  favour  the  election  of  the  node  composing  the  deficient  objectives  (Bensebaa 
& Seridi  1997). 

3.4.2.  Behavioral  student  *s  model 

A student's  good  modeling  permits  to  put  him/her  in  an  optimal  learning  situation  with  regard  to  his/her 
mental  capacities  and  behaviors.  Behavioral  model  is  represented  by  a vector  constituted  of  n psychological 
attitudes.  This  vector  is  used  by  the  tutor  to  deduct  or  to  approach  one  of  the  typical  psychological  profiles 
defined  by  the  pedagogue.  The  feeding  of  attitudes  is  done  via  sensors  placed  at  nodes  level.  The  given  values 
are  of  a quantitative  nature  (number  of  points,  percentage,  etc.)  (Bensebaa  & Lebbihi  1998). 

3.5.  The  tutor 

It  is  the  main  piece  of  this  architecture.  It  is  composed  of  two  modules:  scheduling  /cooperation  module  and 
psycho-pedagogical  module. 

3.5.1.  Scheduling/cooperation  module 

The  educational  objectives  (therefore  nodes  that  compose  them)  represent  some  susceptible  objects,  by  their 
visit,  to  enrich  the  knowledge  of  a learner.  Like  a production  rule,  an  objective  possesses  prerequisites,  that  is 
conditions  on  the  state  of  the  knowledge  of  learner.  These  prerequisites  translate  the  fact  that  some  knowledge 
must  be  assimilated  sufficiently  if  we  want  the  learner  fully  benefits  of  the  nodes  of  the  objective  (Bensebaa 
1991). 

Real  educational  counselor,  the  tutor  will  propose,  at  every  time  that  the  learner  solicits  it,  the  best  node 
adapted  according  to: 

• the  state  of  the  knowledge  of  learner  (where  is  he/she?), 

• the  final  profile  (what  must  he/she  learn?), 

• the  educational  strategy  (what  strategy  of  teaching?). 

At  the  time  of  this  choice,  the  tutor  takes  into  account  the  set  of  student’s  models.  It  orients  the  learner  to  the 
node  which  corresponds  better  to  her/his  “knowledge  state”  and  which  represents  the  best  opportunities  of 
cooperation.  So,  it  builds  first  the  set  of  Pedagogical  Objectives  which  are  Candidates  for  orientation(POC), 
This  set  is  the  result  of  the  intersection  of  two  other  sets: 

• The  first  set  is  constituted  of  the  Pedagogical  Objectives  that  are  Acquired  at  least  by  one  learner  (learner  is 
concemed)(POA), 

• The  second  one  has  Pedagogical  objectives  whose  prerequisites  are  satisfied  by  the  student’s  model  of  the 
concerned  learner  (AO:  Acceptable  Objectives). 

If  the  set  result  of  the  intersection  is  empty  then  POA  is  equal  to  AO. 
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On  the  POC  set  will  be  applied  an  educational  rules  sequence  (inter-objectives  educational  rules)  of  the  chosen 
strategy  until  to  isolate  one  that  corresponds  better  to  the  state  of  the  learner.  Another  sequence  of  educational 
rules  (inter-nodes  rules)  applies  to  get  the  node(s)  to  propose  to  the  learner. 

An  educational  rule  example  is  favor  the  educational  objectives  which  were  acquired  by  the  maximum  number 
of  learners  (maximize  cooperation  possibilities). 

The  learner  can  solicit  the  tutor  for  obtaining  potential  candidates  for  a cooperation.  After  this  request,  the  tutor 
responds  by  presenting  a set  of  learners  classified  according  to  their  similitude  profiles  with  the  demander 

learner  ( same  initial  and  final  profile,  same  final  profile, ).  This  set  is  constituted  by  all  learners  whose 

knowledge  student  models  satisfy  the  current  pedagogical  objective  prosecute  by  the  demander  learner. 

3.5.2.  The  Psycho-pedagogical  module 

This  module  deducts  the  educational  strategy  to  be  applied  and  proposes  it  to  the  scheduling/cooperation 
module.  This  operation  is  done  by  comparing  the  behavioral  model  to  the  different  typical  profiles,  in  order  to 
classify  the  student  in  the  nearest  profile  to  his/her  model.  For  doing  this,  we  applied  a method  of  classifiction 
based  on  the  Main  Components  Analysis  (ACP:  Analyse  des  Composantes  Principales)  (Bensebaa  & Lebbihi 
1998). 


Conclusion 

We  notice  in  the  first  that  this  architecture  protects  totally  the  flexibility  and  the  freedom  which  is  offered  by 
the  hypermedia.  Then,  coupled  with  a tutor,  the  hypermedia  can  fulfill  correctly  its  pedagogical  mission. 
Finally,  the  problem  of  the  isolation  of  the  learner  is  eliminated.  The  student  has  the  possibility  to  solicit  the 
tutor  at  any  time  for  obtaining  orientation  and  cooperation  advice.  He/she  can  solicit  cooperative  learners 
indicated  by  the  tutor. 
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Abstract:  The  design  and  distribution  of  cross-border  higher  education  courses  via  global 
computer  networks  is  a rapidly  growing  phenomenon  (Davis  1998).  This  p^er  describes  a 
recent  international  Web-based  course  and  presents  research  that  focuses  on  the  challenge  of 
international,  cross-cultural  graduate  course  design  in  education.  Using  quantitative  and 
qualitative  methods  of  inquiry,  successes  and  failures  of  the  effort  to  address  cross-cultural 
concerns  are  reported.  This  inquiry  builds  on  existing  literature.  It  informs  the  work  of 
teachers,  researchers,  course  designers  and  program  developers  seeking  to  expand 
instructional  horizons  through  international  academic  collaboration. 


Introduction 

From  January  through  May  1999,  a computer  networked  graduate  education  course  connected  students 
from  the  Universities  of  Oulu  and  Jyvaskyla  in  Finland  with  peers  from  the  University  of  Massachusetts  Lowell 
in  a shared,  project-centered  academic  endeavor.  The  course.  Open,  Flexible  & Distributed  Learning  (OFDL), 
was  offered  in  English  via  the  World  Wide  Web.  It  represented  a joint  initiative  of  the  three  participating 
universities.  Forty  students  were  initially  enrolled:  twenty-one  Finnish  and  nineteen  American.  Network  access 
for  students  and  instructors  was  user  password  controlled. 

OFDL  was  created  and  delivered  by  Professor  LeBaron  and  several  colleagues  at  the  University  of 
Oulu  Faculty  of  Education.  The  course  addressed  issues  related  to  distance  and  distributed  education  in 
schooling  and  in  higher  education.  It  examined  principles  that  support  open  and  flexible  learning.  In  addition  to 
examining  selected  theories,  methods  and  technologies,  students  were  asked  to  work  as  peer  collaborators,  and 
to  join  the  instructors  and  tutors  in  content-centered  research  and  discussion. 

The  technical  platform  for  this  course,  called  ProTo,  is  the  product  of  research  and  development  at  the 
University  of  Oulu,  headed  by  Research  Manager,  Jyrki  Pulkkinen.  OFDL  was  among  the  first  electronically 
networked,  academic  courses  formally  cross-credited  between  Finnish  and  American  universities.  However, 
several  earlier  international  courses,  primarily  within  Europe,  have  been  offered  via  ProTo. 

OFDL  relied  on  electronic  distribution  for  all  aspects  of  study.  ProTo  functioned  as  a server-centered 
database  designed  on  a research  platform  that  supports  constructivist  principles  of  learning;  that  is,  that  students 
constructed  knowledge  and  meaning  through  interaction,  communication,  and  productive  collaboration  with 
instructors,  peers,  and  relevant  materials.  OFDL  was  simultaneously  credited  at  three  universities:  one  in  the 
USA  and  two  in  Finland.  A revised  offering  of  this  international  course  is  in  progress  during  the  spring  of  2000. 
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The  Course:  Open,  Flexible  & Distributed  Learning 


This  course  aimed  to  examine  issues  related  to  online  open  and  flexible  learning.  In  addition,  the 
course  examined  other  selected  telecommunications  technologies  used  in  the  design  and  execution  of 
distributed  learning  environments.  Students  worked  individually  or  in  teams  to  analyze  and  critique 
contemporary  practices  in  this  field.  The  instructor  team  comprised  a lead  professor  and  a course  design 
manager  supported  by  three  graduate  student  assistants  (one  Finnish  and  two  American). 

OFDL  ran  from  January  through  May  1999.  Because  this  was  a "location  independent"  course,  the 
traditional  academic  schedule  of  weekly  or  thrice-weekly  class  sessions  was  abandoned.  Rather,  students  and 
instructors  regularly  and  frequently  "checked  in"  to  the  Web-based  environment.  Since  the  much  of  the 
responsibility  for  such  contact  rested  with  the  student,  a high  degree  of  discipline  and  self-motivation  was 
required  for  success  (Charp  1994,  in  Sherry  1996). 

Due  to  the  asynchronous  and  "distant"  nature  of  the  course,  students  maintained  regular 
communication  with  instructors,  tutors  and  peers.  They  were  also  responsible  for  reviewing  online  or  library- 
based  course  materials,  and  to  reflect  this  research  in  all  written  communication.  The  discussion  forums  offered 
tools  for  academically  focused  discussion.  Students  worked  on  thematic  concentrations  of  their  choice,  but  were 
expected  to  show  mastery  of  the  broader  substantive  context. 

The  ProTolOFDL  online  "library"  pointed  to  a collection  of  resources  developed  or  selected  by  the 
instructors,  tutors,  and  in  some  cases,  students.  This  library  was  divided  into  five  major  categories.  The  first 
contained  pointers  to  "core  readings,"  which  every  student  was  required  to  examine  critically  at  the  earliest 
possible  date.  This  reading  provided  common  ground  for  subsequent  peer  communication.  The  next  three 
library  categories  divided  resources  into  the  "cornerstone"  content  clusters  described  above  (pedagogy, 
technology,  management  & social  organization.)  The  fifth  category  (General  Resources  & Information) 
provided  glossaries,  bibliographies,  and  other  useful  course  reference  tools.  The  library  was  rounded  out  with 
multi  media  virtual  lectures  (e.g.,  streamed  video,  PowerPoint  slides.) 

The  communications  tools  built  into  the  ProTo  environment  were  robust  and  straightforward.  The 
following  tools  were  available  for  student  communication: 

■ threaded  discussion  forums  for  scholarly  discourse  and  problem-solving, 

■ an  embedded  Web  authoring  tool  where  student  work  was  developed, 

■ a "message  board"  for  daily  information  updates  posted  by  the  instructional  staff, 

■ an  e-mail  distribution  list,  for  communication  to  all  students  and  instructional  staff, 

■ "ID  cards"  containing  photos  and  biographical  data  for  students,  tutors  and  instructor, 

■ a virtual  "cafe"  where  course  members  socially  "chatted"  either  asynchronously  or  in  real  time, 

■ a “private  notes”  utility  where  students  could  “think  aloud”  and  store  thoughts  for  later  consideration. 


Characteristics  of  the  ProTo  Learning  Environment 

ProTo  (Project  Tools  for  Learning)  is  an  easy-to-use  learning  environment  that  contributes  to  modem 
open  and  flexible  studying  on  the  Web.  It  provides  comprehensive  communications  and  collaboration  functions 
to  enable  the  instructors  and  students  to  work  cooperatively  toward  meaningful  learning  goals.  The  Finnish 
Ministry  of  Education  has  supported  the  development  of  ProTo,  A newer  version  of  this  authoring-learning 
tool,  called  Learning  Community  Profiler,  is  now  being  prepared  for  public  distribution  in  educational  markets. 

ProTo  runs  on  the  Internet,  or  on  corporate  or  campus  intranets,  based  on  TCP/IP  Web  protocols. 
ProTo  stores  all  information  about  the  users  and  courses  on  a central  database  server.  The  students,  instructors 
and  administrators  may  access  the  server  from  anywhere  on  the  network  using  a standard  Web  browser  with 
commonly  used  plug-ins.  ProTo  enables  bi-directional  “real-time”  and  asynchronous  communication.  At  the 
same  time  the  application  also  makes  it  possible  to  access  networked  multimedia  and  hypermedia  in  the  ProTo 
environment  (e.g.,  video/audio-on-demand,  real-time  video  streaming).  ProTds  networked  instructional  model 
is  based  on  constructivist  learning  principles  that  combine  three  elements  (learning,  technology  and  culture)  to 
provide  the  fundamental  dimensions  for  open  and  flexible  learning.  Students  are  expected  to  construct  their 
knowledge  and  skills  through  personal  experiences  and  social  interaction  in  the  learning  community. 
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According  to  this  model,  technology  is  used  in  the  learning  environment  differently  from  the 
traditional  computer  assisted  learning  (CAL)  instructional  software.  Where  traditional  CAL  technology  was 
designed  to  be  appropriate  in  terms  of  learning,  new  networked  learning  environment  technologies  should  be 
appropriate  in  terms  of  culture  (e.g.,  the  distributed  community  of  learners  and  teachers  who  come  from 
different  cultural  backgrounds).  This  distinction  is  particularly  important  in  international  courses  such  as  OFDL. 

ProTo  offers  tools  for  learners  to  engage  easily  in  structured  learning  on  the  Web,  and  enables  teachers 
to  create  and  organize  open  and  flexible  Web-based  courses.  The  ’’cornerstones"  of  this  open  and  flexible 
learning  environment  are  threefold: 

■ People  as  members  of  a learning  community  (e.g.  students,  teachers  and  tutors)  where  the  students' 
learning  process  is  only  one  process  among  the  others, 

■ Technology  used  to  implement  the  course  (which  was  selected  considering  the  demands  of  a 
constructive  learning  process  and  needs  of  the  learning  community  as  a whole), 

■ Learning  culture  that  supports  productive  collaboration  in  the  learning  community  (which  must  be 
created  during  the  course). 

The  ProTo  technical  operating  environment  becomes  an  active  part  of  the  whole  learning  process  and  is  a 
dynamic  part  of  learning  community  behavior. 


The  Research  Agenda 

A growing  body  of  literature  addresses  problems  in  international  student  collaboration  in  networked 
course  environments.  Pulkkinen  and  Ruotsalainen  (1999)  report  on  cross-cultural  challenges  associated  with  an 
online  Lego  Dacta  course  distributed  within  the  European  community.  Mestenhauser  & Ellingboe  (1998) 
present  a comprehensive  general  discussion  of  the  challenges  of  internationalization  facing  American 
postsecondary  education.  Jenkins  (1997)  offers  some  particularly  interesting  insights  about  cross-cultural 
miscommunication  between  faculty  and  teaching  assistants  of  different  nationalities.  Murphy  (1991)  warns  of 
disparities  in  perceptions  about  success  in  culturally  diverse  international  distance  learning  environments  that 
stress  cooperative  work. 

Through  various  data  collection  and  analysis  techniques,  research  is  examining  the  efficacy  of  efforts 
to  build  constructivist,  international  online  learning  activities.  As  McNabb  (1995)  points  out,  standardized 
assessment  techniques  typically  fail  to  measure  key  indicators  of  student  success  in  open  and  flexible  learning 
situations.  Rather  than  pursue  standard  research  agendas  that  compare  predefined  student  outcomes  in 
networked  versus  non-networked  environments  (Wideman  & Owston  1999),  this  agenda  intends  to  evaluate 
efficacy  in  terms  of  the  intentions  that  drove  the  design  of  OFDL  in  the  first  place. 


Demographics  of  the  Student  Population 

Twenty-six  students  completed  the  final  course  evaluation,  eighteen  from  the  USA  (94.7%  of  active 
students)  and  eight  from  Finland  (61.5%).  This  sample  included  thirteen  male  and  thirteen  female  respondents, 
nine  each  from  the  USA,  and  four  each  from  Finland.  The  mean  student  age  of  35.5  years  ranged  from  25  to  56 
years.  All  26  (100%)  had  used  e-mail  prior  to  the  course.  Only  one,  an  American,  had  never  browsed  the  web 
prior  to  the  course.  Nineteen  (73.1%)  had  never  previously  participated  in  an  Internet  based  course.  A majority 
of  the  Finns  (five  of  eight)  were  experienced  in  the  use  of  Internet-based  learning  environments.  An 
overwhelming  majority  of  Americans  (sixteen  of  eighteen)  were  complete  Internet  learning  neophytes. 

Conclusions  from  this  study  should  be  considered  in  light  of  differences  between  Finnish  and 
American  culture  and  education.  Few  would  disagree  that  Finns  are  a less  voluble  people  than  Americans. 
Many  Finns  take  pride  in  their  tendency  not  to  talk  unless  they  feel  they  have  something  worthwhile  to  say. 
Higher  education  in  Finland  is  tuition  free.  Although  most  courses  are  offered  during  fixed  semesters  of  time,  it 
is  quite  common  for  students  to  complete  work  on  their  own  personal  timetables  without  forfeit  of  money  or 
credit.  Finnish  students  are  assessed  on  a pass-fail  basis.  Thus,  many  of  the  pecuniary  and  scholarly  pressures 
of  American  graduate  work  apply  differently  in  Finland.  When  the  student  performance  stakes  vary  so  much 
between  countries,  unique  challenges  arise  in  designing  tactics  for  international  collaboration.  Although  a few 
students  from  both  countries  failed  to  complete  the  course  (two  American  and  three  Finnish),  an  additional  eight 
initially  enrolled  Finns  never  participated  at  all.  (Rather  than  “drop  outs,”  we  might  label  them  “never  ins”.) 


The  Study 


Since  a major  aim  of  the  course  was  to  foster  purposeful  student  collaboration  across  national 
boundaries,  this  particular  issue  is  addressed.  The  following  data  sources  support  this  paper: 

■ a summative  student  evaluation  anonymously  and  electronically  submitted  after  grades  were  filed, 

• comments  posted  by  students  throughout  the  course  inside  the  communications  tools  provided  by 

ProTo  ("discussion”  and  "comment”  forums,  e-mail,  the  synchronous  “cafe”), 

■ work  actually  produced  by  students  and  student  teams  while  the  course  was  in  progress. 

The  final  course  evaluation  consisted  of  fifty-seven  questions  distributed  across  the  following 
categories:  Background  information.  General  overview  of  the  course.  Comparing  online  with  traditional 
classroom  courses.  Instructional  activities.  Course  interaction,  and  the  ProTo  environment. 

A series  of  questions  relating  to  a general  evaluation  of  the  course  were  asked.  Responses  were  based 
on  a four-point  Likert  scale  numbered  1 to  4 with  1 representing  “Strongly  Agree,”  2 “Agree,”  3 “Disagree,” 
and  4 “Strongly  Disagree.”  To  simplify  analysis,  the  four-point  scale  was  reduced  to  two  points  “Agree”  and 
“Disagree.”  Also  calculated  were  the  P values  for  the  independent  T-test.  Due  to  the  small  number  of 
respondents,  however,  these  values  are  not  reported  here.  Of  the  thirty-eight  Likert  scale  questions  posed,  only 
the  following  five  produced  significant  variations  in  agreement  between  Finnish  and  American  students: 

• Mutual  guidance  and  assistance  among  students  promoted  learning, 

17  Americans  agreed,  1 disagreed;  5 Finns  agreed,  3 disagreed 

• You  felt  that  you  belonged  to  the  learning  community. 

(fifteen  Americans  agreed,  three  disagreed;  four  Finns  agreed,  four  disagreed) 

■ You  felt  that  you  succeeded  in  the  course. 

(fifteen  Americans  agreed,  three  disagreed;  four  Finns  agreed,  four  disagreed) 

• Compared  to  a typical  classroom  course,  you  worked  harder. 

(eight  Americans  agreed,  eight  disagreed;  three  Finns  agreed,  five  disagreed) 

• Compared  to  a typical  classroom  course,  you  learned  more. 

(twelve  Americans  agreed,  six  disagreed;  four  Finns  agreed,  four  disagreed) 

These  disparities  in  perception  pertain  to  issues  of  community  and  student  success.  Notwithstanding  their 
relative  lack  of  prior  seasoning  in  online  learning,  the  Americans  came  away  from  OFDL  with  significantly 
more  positive  reflections  on  their  course  experience  in  these  respects  than  did  the  Finns. 

Even  though  the  following  four  questions  show  little  variation  between  Finnish  and  American  views, 
they  pose  distinct  general  challenges  for  future  course  development: 

■ Compared  to  a typical  classroom  course,  you  interacted  with  student  peers  more  frequently. 

(six  Americans  agreed,  twelve  disagreed;  three  Finns  agreed,  five  disagreed) 

■ Compared  to  a typical  classroom  course,  you  interacted  with  student  peers  more  intelligently. 

(seven  Americans  agreed,  eleven  disagreed;  five  Finns  agreed,  three  disagreed) 

■ Compared  to  a typical  classroom  course,  you  interacted  with  the  instructional  team  more  frequently. 

(six  Americans  agreed,  twelve  disagreed;  three  Finns  agreed,  five  disagreed) 

■ Compared  to  a typical  classroom  course,  you  interacted  with  the  instructional  team  more  intelligently. 

(nine  Americans  agreed,  eight  disagreed;  three  Finns  agreed,  five  disagreed) 

Comparative  US-Finnish  word  counts  were  made  in  three  random  “snapshots”  of  the  interactive 
course  discussion  areas.  In  making  these  comparisons,  Finnish  results  were  adjusted  to  account  for  the  smaller 
number  of  active  students,  whether  or  not  they  actually  completed  the  course.  There  were  nineteen  “active” 
Americans  (the  same  number  as  those  initially  enrolled)  and  thirteen  “active”  Finns  (eight  fewer  than  original 
enrollment).  Since  nineteen  is  46.15%  greater  than  thirteen,  the  Finnish  contribution  was  multiplied  by  1.4615 
to  arrive  at  an  equalized  count  for  comparative  purposes.  This  produced  the  following  US-Finn  “talk”  ratios: 

■ “Cafe”  (3  separate  snapshots),  3.12:1  (656  US  words;  210  equalized  Finnish  words), 

■ “General  discussion  about  course,”  4.1:1  (1561  US  words;  379  equalized  Finnish  words), 

• “Comments-materials:  core  readings”  2.13:1  (9492  US  words;  4456  equalized  Finnish  words). 

True  to  cultural  stereotype,  the  American  students  were  significantly  more  talkative  than  their  Finnish  peers. 

Analysis  of  the  open-ended  student  answers  to  the  final  course  evaluation  revealed  some  interesting 
Finnish-American  comparisons.  In  response  to  a question  asking  how  students  defined  their  success  in  he 
course,  for  example,  American  comments  generally  focused  on  the  mastery  of  external  knowledge;  Finns 
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focused  more  on  personal  performance.  In  roughly  equal  national  proportions,  twenty-one  of  the  twenty-six 
respondents  explicitly  lauded  the  instructional  team  for  the  frequency,  timeliness  and  quality  of  engagement. 
(This  presents  an  interesting  counterpoint  to  the  rather  dim  student  perceptions  reported  on  the  Likert  scale 
questions  about  the  “frequency”  and  “intelligence”  of  their  interactions  with  the  instructional  team.) 

Students  were  asked  to  describe  barriers  to  their  learning  in  the  OFDL,  Consonant  with  the  relatively 
sharp  lack  of  prior  American  experience  in  networked  learning  environments,  fifteen  US  respondents  mentioned 
problems  either  with  their  manipulation  of  the  ProTo  interface,  or  with  the  interface  itself  Only  one  Finn  did 
so.  Finnish  students  seemed  more  concerned  with  the  challenges  of  language  and  culture  (five  of  eight)  and  a 
perceived  lack  of  time  to  fulfill  course  requirements  (again,  five  of  eight).  Significant  numbers  from  both  sides 
urged  more  face-to-face  contact  via  video  conferences. 


Conclusions 

If  truly  collaborative  international  study  is  to  remain  viable  in  the  longer  term,  the  relative  student 
judgments  of  engagement  and  success  must  be  positive  and  roughly  equal  across  the  participating  national 
groups.  Since  Finnish  perceptions  about  productivity,  success  and  community  were  less  optimistic  than  those 
of  their  American  partners,  subsequent  iterations  of  OFDL  must  try  to  make  the  experience  equally  worthwhile 
for  all  students.  Similarly,  the  sensing  across  both  national  groups  about  the  questionable  utility  of  interaction 
among  peers  and  instructors  bears  close  attention. 

In  this  case,  OFDL  was  the  instructor’s  first  venture  not  only  into  online  course  design  but  also  into 
formal  cross-national  teaching.  The  time  committed  to  all  aspects  of  online  course  design,  teaching  and 
maintenance  exceeded  that  of  a very  active  traditional  course  by  a factor  of  no  less  than  four.  Some  of  this  may 
be  attributable  to  the  steep  learning  curve  confronted  in  such  a first-time  effort.  However,  this  factor  is  reduced 
to  no  better  than  two  or  three  in  the  second  iteration  of  the  course.  Under  any  circumstances,  effective  online 
teaching  and  learning  is  exceedingly  time  consuming.  Administrative  or  professorial  failure  to  recognize  this 
reality  will  result  in  inferior  teaching  and  dashed  hopes  for  students  and  teachers  alike. 

The  comparative  US-Finnish  word  counts  could  be  explained  by  the  simple  variation  in  difficulty 
between  students  communicating  colloquially  in  their  native  tongues  versus  those  struggling  harder  in  a second 
language.  However,  the  breadth  of  participation  in  the  two  groups  produces  more  troubling  results.  The  1 1,709 
counted  US  student  words  were  spread  across  sixteen  students  (84%  of  the  US  total  counted).  The  3638  Finnish 
words  were  spread  across  six  students  (46%  of  the  Finnish  total  counted).  .If  active  participation  in  the  course 
dialogue  areas  was  an  important  goal,  then  it  seemed  substantially  better  met  for  Americans  than  for  Finns. 

Whatever  the  study  data  indicate,  a troubling  fact  cannot  be  escaped.  Except  in  one  case,  the  greatly 
desired  cross-border  project  collaboration  failed  to  materialize.  Cross-cultural  student  discomfort  may  partly 
explain  this  failure,  but  the  failure  cannot  be  accepted.  Simply  enrolling  in  a course  with  peers  from  other 
nations  may  (or  may  not)  marginally  promote  cross-border  awareness,  but  true  knowledge  and  sensitivity 
requires  substantive,  deep  collaboration.  Ironically,  such  collaboration  occurred  within  the  Finnish  and 
American  student  communities,  but  not  sufficiently  across  them.  Clear,  strategic,  remedial  thinking  is  needed. 

The  following  ten  lessons  for  the  designers  and  teachers  of  international  Web-based  courses  bear  note: 

1.  Course  designers  and  teachers  should  familiarize  themselves  with  particular  cross-border  cultural  and 
policy  peculiarities  before  any  academic  offering  is  launched,  and  scale  expectations  accordingly. 

2.  Instructors  and  tutors  must  be  able  and  willing  to  meet  the  substantial  time  demands  of  constant, 
supportive  involvement. 

3.  The  continuing  presence  of  active  tutorial  support  within  the  participating  nations  is  critical  to  success. 

4.  A spirit  of  openness  and  encouragement  must  suffuse  the  instructional  team  membership  at  all  times. 

5.  Specific  activities  can  help  to  “break  the  cross-border  ice,”  especially  in  the  early  stages  of  a course. 

6.  Exercises  should  be  designed  to  promote  substantive  cross-border  collaboration.  (Such  exercises  must 
respect  national  cultural  norms  so  that  expectations  are  fair  and  realistic  for  both  groups) 

7.  Quantity  of  student  talk  is  less  important  than  quality  and  distribution  across  a wide  student  base.  (If 
participation  is  limited  to  a narrow  subgroup  within  or  across  the  national  boundaries,  a major  course 
intention  remains  unmet.) 

8.  Orientation  training  is  needed  for  students  having  had  little  or  no  prior  experience  in  online  learning. 

9.  No  effort  should  be  spared  to  humanize  contact  (e.g.,  via  video  conferences,  audio  clips,  e-mail,  other 
media  and,  where  feasible,  face-to-face  communication). 
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1 0.  The  technical  interface  must  be  reliable,  “bug  free,”  and  appropriately  aligned  to  the  speed,  power  and 
connectivity  of  client  workstations. 

The  following  Finnish  comment  poignantly  illustrates  the  first  of  the  above  ten  lessons.  “Language  problems 
and  the  discussion  cultures  in  Finland  and  the  USA  are  different.  [It  is]  hard  to  find  ‘the  shared  language.’  This 
doesn’t  mean  that  we  should  not  have  courses  together,  but  students  in  both  countries  should  be  aware  of  this 
and  take  it  into  account.” 


Limitations  and  Pointers  to  Further  Research  and  Action 

Knowledge  gained  from  this  study  cannot  be  generalized  to  other  international  course  settings.  Only 
one  course  was  evaluated.  The  student  sample  size  is  small.  The  study  is  limited  to  two  countries  only,  and  to 
rather  narrow  cultural  subgroups  within  those  countries.  Certain  lessons,  however,  will  assist  teachers  and 
course  designers  planning  to  launch  fresh  initiatives. 

The  analysis  reported  here  has  supported  several  beneficial  revisions  to  the  current  iteration  of  OFDL, 
and  could  similarly  contribute  to  the  success  of  any  such  cross-border  initiative.  Future  research  efforts  should 
be  keyed  both  to  the  general  intentions  of  the  course,  and  to  the  unique  cultural  realities  of  the  participating 
students.  New  research  should  be  deeper  and  more  methodologically  varied.  For  example,  qualitative  interviews 
with  randomly  selected  students  would  acutely  enrich  viewpoints  on  student  perceptions,  thereby  arming 
teachers  and  designers  with  deeper  perspectives  for  future  planning  and  execution. 
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Abstract:  One  of  the  key  elements  that  is  critical  to  the  success  of  distance  education  is  active 
collaboration  between  participants.  “Two-way  interaction  with  a given  student  and  the  instructor 
simply  is  not  enough”  (Palloff  & Pratt,  1999).  Online  courses  using  the  traditional  Web  functions 
lack  spontaneous  interactions  among  participants.  Static  Web  documents  and  e-mail,  the  two  most 
commonly  used  Web  functions,  provide  only  two-way  interactions.  Bulletin  board  and  chat  can 
often  cause  confusion  and  frustration.  On  the  other  hand,  classroom  videoconferencing  is 
prohibitively  expensive  for  many  institutions,  while  desktop  videoconferencing  faces  a serious 
bandwidth  problem.  The  Web  audio  conferencing  technology  may  be  an  efficient  and  affordable 
solution  for  many  distance  education  courses  that  require  spontaneous  interactions  and 
collaboration  among  participants.  The  purpose  of  this  paper  is  to  discuss  the  effectiveness  of  real- 
time Web  audio  as  an  ideal  mode  of  delivery  under  the  current  state  of  technology  and  given  the 
limited  bandwidth  of  the  Internet.  Capabilities  of  the  live  Web  audio  technology  are  outlined.  The 
comparative  advantages  of  the  Web  audio  technology  are  discussed. 


Introduction 

As  computer-mediated  distance  education  becomes  more  prevalent,  some  researchers  have  begun  to  voice  their 
concerns  about  the  effectiveness  of  Web-based  courses.  Effective  learning  can  take  place  when  there  are  active 
interactions  between  the  instructor  and  students  and  among  students.  While  the  Web  technology  is  touted  as  a 
medium  to  enhance  such  interactions,  the  loss  of  spontaneous  communications  among  and  between  faculty  and 
students  can  cause  considerable  difficulties  for  both  teaching  and  learning.  In  particular,  the  most  commonly  used 
methods  of  delivery  for  distance  education  are  subject  to  several  limitations  and/or  drawbacks.  Asynchronous 
communications  such  as  e-mail  and  static  Web  documents  do  not  provide  sufficient  interactions  among  participants. 
Even  in  the  case  of  synchronous  mode  such  as  live  chat  can  be  confusing  and  ineffective  (Hara  & Kling,  1999). 
Classroom  videoconferencing  technology  can  overcome  most  of  these  deficiencies,  but  is  costly  to  set  up  and 
maintain.  Furthermore,  students  may  have  to  travel  some  distance  at  certain  times  to  take  courses  that  are  based  on 
this  technology,  which  defeats  the  purpose  of  distance  education  to  some  degree.  Another  potentially  effective  mode 
of  delivery  is  desktop-based  videoconferencing.  However,  for  this  method  to  work,  all  participants  must  have  high 
bandwidth  connections  to  the  Internet,  without  which  the  picture  quality  would  be  poor  and  interactions  ineffective. 
This  bandwidth  problem  will  not  be  solved  until  cable  modems,  DSL  and  ISDN  connections  are  not  only  more 
widely  available  but  also  affordable. 

The  purpose  of  this  paper  is  to  discuss  the  effectiveness  of  real-time  Web  audio  as  an  ideal  mode  of  delivery  under 
the  current  state  of  technology  and  given  the  limited  bandwidth  of  the  Internet.  Capabilities  of  the  live  Web  audio 
technology  are  outlined.  The  comparative  advantages  of  the  Web  audio  technology  are  discussed. 

Distance  Learning  and  Interactivity 

Not  surprisingly,  studies  find  that  interactivity  plays  a significant  role  for  the  success  of  a distance  education  course. 
For  example,  in  a study  of  the  effectiveness  of  corporate  training,  Millbank  (1994)  finds  that  an  introduction  of  real- 
time interactivity  increased  the  retention  rate  of  the  trainees  considerably.  “Without  connectivity,  distance  learning 
degenerates  into  the  old  correspondence  course  model  of  independent  study.  The  student  becomes  autonomous  and 
isolated,  procrastinates,  and  eventually  drops  out”  (Sherry,  1996).  In  a recently  released  study  that  declares  the  24 
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benchmarks  essential  to  ensuring  excellence  in  Internet-based  learning, 
interactivity  is  found  to  be  ai  important  benchmark  for  teaching  and  learning 
effectiveness.  “Student  interaction  with  faculty  and  other  students  is  an  essential 
characteristic  and  is  facilitated  through  a variety  of  ways,  including  voice-mail 
and/or  e-mail”  (National  Education  Association,  2000).  However,  while  useful  in 
some  specific  situations,  the  asynchronous  communications  such  as  e-mail  or 
bulletin  board  don’t  seem  to  foster  informal  conversations  and  personal 
interactions  that  play  an  important  role  in  a teaching  and  learning  environment. 

Even  the  more  costly  solutions  such  as  classroom  videoconferencing  or  telephone 
conferencing  do  not  seem  to  be  well-suited  to  sharing  different  types  of 
information,  such  as  slide  show  presentations,  application  sharing,  or  group 
learning  processes  (Navickas,  2000). 

Interactions  Using  Live  Web-Audio  Conferences 

One  of  the  popular  Internet  conferencing  software  products  is  Microsoft 
NetMeeting  (Figure  1).  This  software  enables  the  user  to  communicate  with  both 
audio  and  video,  collaborate  in  Windows-based  programs,  exchange  graphics  on 
an  electronic  whiteboard,  transfer  files,  or  use  a text -based  chat  program. 

“Common  uses  of  NetMeeting  include  real-time  document  collaboration, 
technical  support  in  a helpdesk  environment,  training  and  distance  learning,  and 
conducting  remote  meetings”  (Microsoft,  2000).  While  versatile,  programs  such 
as  NetMeeting  require  relatively  wide  bandwidth.  With  a dial-up  connection 
using  a 28. 8K  modem,  for  example,  users  find  that  video  quality  tends  to  be  poor  and  those  with  a low-end  hardware 
frequently  experience  difficulty  in  maintaining  connections. 


Figure  1 . Microsoft 
NetMeeting 
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Unlike  desktop  videoconferencing,  however,  the  bandwidth  problem  is  not  as  serious  in  Web-based  audio 
conferencing.  In  addition,  Voice-enabled  online  collaboration  requires  no  special  hardware  configuration  beyond  a 
typical  one  used  for  any  Web-based  distance  education  courses.  A computer  with  multimedia  capability  (sound  card 
and  speaker),  a microphone,  and  Internet  connection  are  all  that  is  needed.  On  the  software  side,  Web-based  audio 
conferencing  software  that  can  provide  low-bandwidth  transport  of  voice  using  efficient  encoding  is  required.  Such 
software  is  usually  based  on  a client-server  technology,  meaning  that  the  server  software  (and  hardware)  provides 
information  and  manages  connections  while  client  software  enables  the  user  to  access  the  server. 


The  Web  audio  technology  enables  the  instructor  to  use  all  the  functions  of  the 
Web  used  in  the  traditional  online  courses  including  HTML  documents, 
multimedia  presentations,  chat,  and  bulletin  board.  In  addition,  the  instructor 
can  conduct  synchronous  collaboration  with  students  via  live  voice  conference, 
share  applications  with  them  in  real  time,  and  administer  tests  or  quizzes  with 
instant  feedback,  to  name  only  a few. 

The  use  of  Web-based,  synchronous  audio  applications  is  still  very  limited 
(Kotter,  et  al,  1999).  Applications  such  as  streamed  audio  are  usually 
unidirectional;  no  interactions  take  place.  These  applications  do  not  take  full 
advantage  of  the  truly  interactive  nature  of  Web  audio.  The  Web-based  IP 
audio  conferencing  technology  carries  Web  audio  one  step  further  to  integrate 
high  quality  audio  and  tools  for  dynamic  interactions  among  participants. 

Advantages  of  Using  Web  Audio  Conferencing  Technology 

Unlike  the  classroom  videoconferencing  technology,  the  Web  audio 
conferencing  technology  requires  no  expensive  rooms,  equipment  or 
transmissions.  This  technology  is  available  to  students  with  a computer  with  a 
multimedia  card,  microphone,  and  Internet  connection.  While  this  is  also  true 
for  desktop  videoconferencing,  one  important  difference  between  the  two  is 
that  the  Web  audio  conferencing  technology  can  be  used  over  low-bandwidth 
and  dial-up  connections.  Indeed,  the  bandwidth  problem  cannot  be  ignored  in 
distance  education,  as  long  as  a majority  of  students  use  dial-up  connections 
and  the  costs  of  ISDN  and  cable  modem  lines  remain  high. 

In  addition,  instructors  who  deliver  their  online  courses  using  the  Web  audio 
conferencing  technology  can  integrate  all  other  Web  functions  into  their 
courses,  including  discussion  board,  chat,  e-mail,  and,  of  course,  HTML 
documents. 

In  particular,  Centra  Symposium  is  a leading  provider  of  Internet  solutions  for 
live  collaboration  over  the  Internet.  While  the  main  focus  of  Centra  is  on 
business  applications  such  as  sales,  marketing  and  training,  it  has  much 
potential  for  distance  education  as  well.  Centra99  makes  collaboration  between 
instructors  and  students  easy  by  providing  the  following  capabilities  (Centra, 
1999): 
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Figure  3.  CentraNow  User 
Interface 


• Intuitive  controls  over  assigning  speaking  rights  among  students.  When  a student  has  a question,  he  or  she 
can  “raise”  hand  by  clicking  on  an  icon  and  the  instructor  can  grant  the  speaking  right  to  the  person.  This 
way,  the  instructor  can  avoid  confusion  often  seen  in  live  chat  and  at  that  same  time  encourage 
participation  of  all  students. 

• Easy  administration  of  tests  and  quizzes,  and  instant  feedback  from  online  evaluations 

• Effective  management  of  student  accounts.  Scheduling,  confirming,  and  managing  student  accounts  are 
easily  handled. 

• Application  sharing  capabilities  between  the  instructor  and  students  for  hands-on  experience  (Figure  2). 

• Whiteboard  for  shared  visual  communications  among  participants. 
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• Support  for  multiple  instructors  for  a class 

• Availability  of  break  out  groups  and  lab  environment 

• Cost  advantages  (no  transmission  cost,  no  high  priced  equipment,  no  onsite  technician  and/or  administrator 
is  needed) 

Recently,  Centra  Software  launched  CentraNow  (wwwxentranowxom)  that  is  a scaled  down  version  of  the  full 
Symposium  product.  CentraNow  is  available  for  free  at  the  time  of  this  writing,  though  the  number  of  participants 
who  can  attend  a particular  meeting  is  limited  to  four  (Figure  3 and  4). 

Overall,  this  voice  over  IP  audio  conferencing  technology  is  an  attractive  compromise  between  the  traditional  Web 
technology,  which  lacks  real-time  interactions,  and  the  classroom  videoconference  technology,  which  is  too  costly 
for  many  institutions  and  also  may  require  students  to  travel  a great  distance.  Given  these  limitations  of  various 
other  technologies,  synchronous  Web  audio  may  be  an  efficient  and  cost  effective  way  of  delivering  distance 
education  courses. 


^•CentiaNow  Client  - Netscape 
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Figure  4.  CentraNow  User  Interface  Full  Screen 


Concluding  Remarks 

One  of  the  key  elements  that  is  critical  to  the  success  of  distance  education  is  active  collaboration  between 
participants.  “Two-way  interaction  with  a given  student  and  the  instructor  simply  is  not  enough”  (Palloff  & Pratt, 
1999).  Online  courses  using  the  traditional  Web  functions  lack  spontaneous  interactions  among  participants.  Static 
Web  documents  and  e-mail,  the  two  most  commonly  used  Web  functions,  provide  only  two-way  interactions. 
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Bulletin  board  and  chat  can  often  cause  confusion  and  frustration.  On  the  other  hand,  classroom  videoconferencing 
is  prohibitively  expensive  for  many  institutions,  while  desktop  videoconferencing  faces  a serious  bandwidth 
problem.  The  Web  audio  conferencing  technology  may  be  an  efficient  and  affordable  solution  for  many  distance 
education  courses  that  require  spontaneous  interactions  and  collaboration  among  participants. 
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Abstract:  This  study  examined  the  effects  of  ability  of  the  student  and  group 
composition  on  achievement  in  reading,  writing,  and  listening  comprehension  in 
Web-based  foreign  language  learning  in  a cooperative  environment.  Forty-four 
students  were  randomly  assigned  to  heterogeneous  and  homogeneous  groups. 
The  results  of  the  analysis  showed  that  group  composition  as  well  as  student 
ability  significantly  exerted  differential  effects  on  the  learning  outcomes.  The 
implications  of  these  results  for  Web-based  small  group  work  were  discussed. 


Introduction 


Computer-assisted  language  learning  (CALL)  using  Web-based  multimedia  courseware  can  provide  a 
learning  environment  that  facilitates  positive  interdependence  and  collaborative  efforts  among  students.  The 
students  work  together  in  small  groups  at  the  computer;  their  efforts  are  directed  toward  mutual,  yet 
academically  and  socially  beneficial,  goals.  In  general,  extensive  research  on  cooperative  learning  has  shown 
profound  and  positive  effects  on  a wide  range  of  students’  cognitive  and  social-affective  outcomes  (e.g., 
Johnson  & Johnson,  1999;  Johnson  et  al.,  1993;  Sharan,  1990;  1994;  Slavin,  1995).  The  computer  as  an 
instructional  medium  also  has  the  potential  for  promoting  interaction  and  collaboration  among  students  (e.g., 
Cates  & Goodling,  1997;  Cavalier  & Klein,  1998;  Chen,  1995;  Johnson  & Johnson,  1996;  Lee,  1993).  During 
the  past  years,  the  Internet  has  been  increasingly  utilized  as  an  effective  instructional  tool  for  language 
learning,  since  the  Web  can  become  a multimedia-based  content  provider  with  versatility  and 
interconnectedness  (Clinch,  1999;  Harasim  et  al.,  1996;  Khan,  1997;  McManus,  1995;  Owston,  1997;  Ritchie 
& Hoffman,  1996).  For  the  present  study,  Web-based  multimedia  courseware  was  developed  for  computer- 
assisted  foreign  language  instruction,  as  discussed  later. 

One  of  the  key  features  that  characterize  cooperative  learning  settings  and  distinguish  then  from  other 
learning  settings  is  the  increased  opportunity  for  interaction  among  students  of  diverse  ability,  beliefs,  and 
value  systems  in  the  learning  process.  Researchers  have  explored  interaction  as  one  of  the  mediating 
variables  in  the  relationship  between  cooperative  learning  and  social  and  academic  gains  (Hettinger,  1995; 
Huang,  1995;  Lee,  1993;  Webb,  1989).  In  a cooperative  learning  environment,  hence,  students  are  typically 
grouped  heterogeneously.  The  rationale  for  heterogeneous  grouping  is  based  on  the  assumption  that  students 
can  encounter  wider  diversity  in  heterogeneous  groups  than  in  homogeneous  groups.  Of  particular  interest  in 
the  present  study  are  ability  of  the  student  and  ability  composition  of  the  group.  Although  research  indicates 
that  both  high-  and  low-ability  students  gain  social  benefits  by  working  in  heterogeneous  groups,  the 
cognitive  effects  of  ability  grouping,  heterogeneous  or  homogeneous,  have  been  inconclusive  (e.g..  Cavalier 
& Klein,  1998;  Huang,  1995;  Mevarech  et  al.,  1991;  Webb,  1989;  Webb  & Lewis,  1988). 

The  purpose  of  this  study  was  to  examine  the  effects  of  ability  of  the  student  and  the  influence  of 
heterogeneous  and  homogeneous  group  composition  on  achievement  in  reading,  writing,  and  listening 
comprehension  in  computer-assisted  foreign  language  learning  with  Web-based  multimedia  courseware  in  a 
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cooperative  learning  environment.  The  achievement  in  reading,  writing,  and  listening  comprehension  of 
high-  and  low-ability  students  were  compared  in  heterogeneous  and  homogeneous  groups  featuring  individual 
and  group  accountability. 


Method 


Subjects 

The  subjects  were  44  undergraduate  students  enrolled  in  a required  one-semester  foreign  language  course 
at  a national  university  in  a metropolitan  city  in  Korea.  All  the  subjects  had  some  previous  experience  with 
computers  (e.g.,  word  processing,  Internet,  telecommunications,  games,  and/or  programming).  All  students 
have  taken  English  as  a first  foreign  language  and  French,  German,  Chinese,  or  Japanese  as  a second  foreign 
language  in  middle  and  high  schools. 

Procedure 

Before  the  study  began,  students  were  asked  to  complete  a background  survey,  which  was  given  in  order  to 
assess  students’  previous  experience  with  computers  and  language  learning  and  to  provide  a better  description 
of  the  subjects.  The  pretest  was  administered  to  all  students  to  identify  those  with  high  or  low  ability. 
Stratified  random  sampling  was  used  to  assign  students  to  heterogeneous  and  homogeneous  ability  groups. 
Heterogeneous  ability  groups  contained  one  high-ability  student  and  one  low-ability  student.  Homogeneous 
ability  groups  contained  two  high-ability  students  or  two  low-ability  students.  Students  were  unaware  of  the 
ability  composition  of  the  group.  Students  then  received  an  overview  of  the  courseware  and  instruction  for 
cooperative  work.  They  were  instructed  to  work  cooperatively  as  a group  on  the  task,  to  help  each  other 
learn,  and  to  make  group  decisions  on  the  course  of  their  actions  in  the  learning  process.  Students  were  not 
assigned  specific  roles  within  a group,  nor  were  they  allowed  to  divide  the  work.  Students  worked  for  50 
minutes  each  day,  2 days  each  week,  for  15  weeks,  a total  of  30  instructional  sessions  for  one  semester. 

Web-based  multimedia  courseware 

For  the  purpose  of  this  study,  Web-based  multimedia  courseware  was  designed  and  developed  for  French 
language  learning.  This  courseware  appears  to  be  one  of  the  first  courseware  for  computer-assisted  French 
language  instruction  in  Korea.  The  courseware  was  designed  to  be  adaptive  to  individual  learning  situations 
on  a non-real  time  basis.  Students  can  navigate  the  hyperlinked  multimedia  contents  without  a pre-ordered 
learning  schedule.  Through  their  exploration  and  navigation,  thus,  they  can  design  their  own  instruction.  The 
contents  of  the  courseware  are  divided  into  two  levels:  beginning  and  intermediate.  Each  level  consists  of  15 
coherent  but  independent  lessons.  Each  lesson  is  composed  of  six  sections:  reading,  writing,  listening, 
speaking,  grammar,  drills,  and  games.  These  sections  require  either  passive  or  active  learning  modes  in  terms 
of  data  manipulation. 

The  reading  section  shows  paragraphs  in  a variety  of  styles  and  includes  interpretations  and  in-depth 
explanations  regarding  morphological,  lexical,  syntactical  and  semantic-pragmatic  rules  and  expressions  used 
in  each  sentence.  The  writing  section  enables  students  to  gain  pragmatic  competence  in  their  writing  skills.  It 
provides  questions  related  to  context-based  composition.  The  listening  section  presents  simple  expressions 
with  immediate  text  feedback  to  improve  students'  listening  comprehension.  The  speaking  section  is  designed 
with  an  emphasis  on  conversational  practice,  based  on  given  situations  presented  as  a picture.  Concerning  the 
grammatical  rules  of  the  previously  presented  sentences,  the  grammar  section  provides  charts,  pictures, 
examples  as  well  as  explanations  about  those  points.  The  game  section  is  an  additional  unit  designed  to 
motivate  students  through  games,  songs,  or  puzzles,  which  may  not  deal  with  the  lesson  directly. 

The  courseware  also  includes  the  interaction  facilities:  help,  bulletin  board,  announcements,  and  e- 
mail.  The  help  component  includes  general  instructions  regarding  the  instructional  system.  The  bulletin  board 
deals  with  management-related  interactions  such  as  a recommended  learning  schedule  and  important  dates. 
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The  announcements  show  FAQ’s  (Frequently  Asked  Questions)  on  subject  materials  or  technical  problems. 
The  e-mail  allows  for  individual  communications.  These  interaction  facilities  were  designed  to  provide 
various  types  of  asynchronous  communications  among  three  different  user  groups:  teachers  or  tutors, 
students,  and  system  administrators. 

In  designing  and  developing  the  user  interface  of  the  courseware,  a special  emphasis  was  placed  on 
user-friendliness  and  efficiency.  A simple,  intuitive  design  with  a text-based  menu,  rather  than  a complicated 
design,  was  preferred.  In  addition,  the  courseware  utilizes  well-designed  TrueType  fonts,  which  support 
Unicodes  such  as  ‘Lucida  Sans  Unicode,’  ‘Berdana,’  and  ‘Times  New  Roman.’  The  basic  color  of  the 
courseware  was  carefully  selected  based  on  the  color-effectiveness  studies  (Moore,  1996;  Pett  & Wilson, 
1996;  Weinman  & Heavin,  1996).  Considering  the  current  access  speed  to  the  Internet  via  modems  or  LANs 
(Local  Area  Networks)  in  schools,  a minimum  level  of  animation  was  used  in  order  not  to  interfere  with 
students’  concentration  level  in  the  learning  process  (Jeong  & Yoon,  1998).  For  the  consistent  and  systematic 
delivery  of  information,  the  subsequent  hyped  inked  information  is  presented  in  the  same  page.  To  this  end, 
the  interface  was  developed  using  Active  Server  Page  (Hillier  & Mezick,  1998)  and  Dynamic-HTML 
(HyperText  Markup  Language)  (Homer,  1997). 

Research  Design  and  Data  Analysis 

The  study  employed  a 2 X 2 factorial  design.  The  between-subjects  factors  included  Ability  (high,  low) 
and  Group  Composition  (heterogeneous,  homogeneous).  The  within-subjects  factor,  Achievement,  included 
Reading,  Writing,  and  Listening  Comprehension.  The  analysis  of  variance  (ANOVA)  was  performed  to 
determine  the  interaction  effects  as  well  as  the  main  effects  of  ability  and  group  composition  on  achievement 
in  reading,  writing,  and  listening  comprehension.  The  analysis  of  covariance  (ANCOVA)  was  also  conducted, 
with  students’  previous  experience  with  computers  and  the  pretest  results  serving  as  the  covariates.  The  level 
of  significance  was  set  at  .05  in  this  study. 


Results  and  Discussion 


The  means  and  standard  deviations  for  achievement  in  reading,  writing,  and  listening  comprehension  are 
presented  in  Table  1 . The  results  of  the  analysis  of  variance  are  shown  in  Table  2.' 

Reading.  Significant  main  effects  were  found  for  Ability,  F (1,  40)  = 7.208,  p < .05,  and  for  Group 
Composition,  F (1,  40)  = 14.029,  p < .05,  and  significant  interaction  effects  were  also  found  for  Ability  and 
Group  Composition,  F (1,  40)  = 7.268,  p < .05.  These  results  indicate  that  student  ability  and  group 
composition  exerted  differential  effects  on  achievement  in  the  reading  posttest,  as  shown  in  Tables  1 and  2. 
High-  and  low-ability  students  tended  to  achieve  differentially  across  the  groups  of  different  composition  on 
the  reading  posttest.  The  students  in  heterogeneous  groups  scored  higher  than  did  those  in  homogeneous 
groups.  This  pattern  is  more  noticeable  among  low-ability  students  than  high-ability  students. 

Writing.  There  were  significant  main  effects  for  Group  Composition,  F (1,  40)  = 4.401,  p < .05,  and 
significant  interaction  effects  for  Ability  and  Group  Composition,  F (1,  40)  = 3.759,  p < .05.  Yet,  main 
effects  for  Ability  were  not  statistically  significant.  Both  of  high-ability  and  low-ability  students  working  in 
heterogeneous  groups  tended  to  score  higher  on  the  writing  posttest  than  did  those  working  in  homogeneous 
groups.  These  results  indicate  that  the  achievement  of  high-ability  and  low-ability  students  was  dependent  on 
the  group  composition  in  which  they  were  working. 

Listening  Comprehension.  No  significant  effects  were  found  for  Ability  or  Group  Composition  or  for  the 
interaction  between  Ability  and  Group  Composition.  The  results  indicate  that  the  differences  between  the 
posttest  means  were  not  statistically  significant,  probably  due  to  the  relatively  large  standard  deviations,  as 
shown  in  Table  1. 

This  study  examined  the  effects  of  student  ability  and  group  composition  on  achievement  in  reading. 


* It  should  be  noted  that  students’  previous  experience  with  computers  and  the  pretest  results  were  not  significantly 
correlated  with  the  achievement  scores.  The  results  of  ANCOVA  were  rarely  changed.  For  a purpose  of  clarity,  hence,  the 
results  of  ANOVA  are  presented  in  this  section. 


writing,  and  listening  comprehension  in  computer-assisted  foreign  language  learning  with  Web-based 
multimedia  courseware  in  a cooperative  learning  environment.  The  results  of  the  analysis  of  variance  indicate 
that  group  composition  as  well  as  student  ability  significantly  exerted  differential  effects  on  the  learning 
outcomes.  Both  high-ability  and  low-ability  students  working  in  heterogeneous  groups  showed  higher 
achievement  than  did  those  working  in  homogeneous  groups.  These  results  corroborate  and  lend  further 
support  to  the  previous  studies  that  heterogeneous  group  composition  benefits  students  of  both  high  ability 
and  low  ability  (Larson  et  al.,  1984,  Webb,  1982a;  1982b;  Yager,  1986).  The  cooperative  learning  methods,  in 
non-computer  settings,  often  call  for  students  to  be  grouped  heterogeneously  by  ability  (e.g.,  Sharan,  1994; 
Slavin,  1995).  The  findings  of  this  study  suggest  that  ability  grouping  can  also  be  utilized  as  an  effective  and 
practical  method  in  Web-based  instructional  settings. 

Suggestions  for  future  research  should  be  noted.  First,  a comparative  study  of  group  learning  with 
individualized  learning  in  Web-based  instructional  settings  may  be  worth  further  investigation.  Second,  this 
study  employed  pairs;  the  findings  may  not  generalize  to  larger  groups.  Some  research  suggested  the 
importance  of  group  size  as  well  as  group  composition  in  computer-based  cooperative  learning  (Guntermann 
& Tovar,  1987).  Finally,  this  study  has  focused  the  product  of  group  learning.  Future  research  should  also 
analyze  the  intra-group  dynamics  among  students  in  the  learning  process. 


Table  1 . Means  and  Standard  Deviations  of  the  Achievement  Scores  by  Ability  and 

Group  Composition 


Reading 

Writing 

Listening 

Total 

Ability 

High 

M 

8.39 

8.22 

5.43 

22.04 

SD 

1.23 

2.13 

2.43 

4.76 

Low 

M 

7.38 

7.00 

4.43 

18.81 

SD 

1.94 

2.55 

2.06 

5.54 

Group  Composition 

Heterogeneous 

M 

8.64 

8.27 

5.18 

22.09 

SD 

1.33 

2.19 

2.42 

5.26 

Homogeneous 

M 

7.18 

7.00 

4.73 

18.91 

SD 

1.68 

2.47 

2.19 

5.04 

Total 

M 

7.91 

7.64 

4.95 

20.50 

SD 

1.67 

2.39 

2.29 

5.34 
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Table  2.  ANOVA  Results  for  the  Achievement  Scores  by  Ability  and  Group  Composition 


Source  of 
Variation 

Sum  of 
Squares 

DF 

Mean 

Squares 

F 

P 

Reading 

Ability 

12.750 

1 

12.750 

7.208 

.011 

Group  Composition 

24.817 

1 

24.817 

14.029 

.001 

Interactions 

12.856 

1 

12.856 

7.268 

.010 

Writing 

Ability 

17.892 

1 

17.892 

3.719 

.056 

Group  Composition 

19.442 

1 

19.442 

4.401 

.046 

Interactions 

18.041 

1 

18.041 

3.759 

.050 

Listening 

Ability 

11.600 

1 

11.600 

2.316 

.136 

Group  Composition 

2.759 

1 

2.759 

0.551 

.462 

Interactions 

11.697 

1 

11.697 

2.335 

.134 

Total 

Ability 

125.586 

1 

125.586 

5.818 

.021 

Group  Composition 

122.144 

1 

22.144 

5.659 

.022 

Interactions 

126.632 

1 

26.632 

5.867 

.020 
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Abstract;  This  article  presents  a paradigm  shift  from  the  traditional  curriculum  model  of  setting  goals  and 
objectives  to  a new  curriculum  model  of  setting  standards  and  responsibilities  to  assist  instructors  and  students  in 
changing  from  teacher-centered  learning  to  student-centered  learning  in  the  technology  era.  The  student-centered 
learning  environment  is  diverse  and  allows  students  with  different  learning  styles  and  intellectual  abilities  to  prosper 
at  their  own  pace.  Teaching  pedagogy  and  classroom  learning  are  much  behind  the  evolving  of  technology.  In  the 
advanced  technology  era,  not  only  the  traditional  lecture  and  note  taking  approach  will  be  seriously  challenged,  but 
also  the  static  use  of  technology  will  be  seriously  challenged  as  well.  This  article  addresses  two  fundamental 
obstacles  in  shifting  from  teaching-centered  to  learning-centered  strategy  in  technology  era  at  higher  education.  A 
case  study  is  presented  to  illustrate  how  to  deal  with  these  fundamental  obstacles. 


Introduction 

This  article  presents  a paradigm  shift  from  the  traditional  curriculum  model  of  setting  goals  and  objectives  to  a new 
curriculum  model  of  setting  standards  and  responsibilities  to  assist  instructors  and  students  in  changing  from 
teacher-centered  learning  to  student-centered  learning  in  the  technology  era.  The  use  of  technology  in  the  teacher- 
centered  learning  environment  is  usually  static,  such  as  report  writing,  projects,  powerpoint  presentation,  web 
material  distribution  and  classroom  demonstration.  The  student-centered  learning  environment  is  diverse  and  allows 
students  with  different  learning  styles  and  intellectual  abilities  to  prosper  at  their  own  pace.  A proper  integration  of 
technology  is  crucial  for  a successful  student-centered  learning  model. 


Background 

Imagining  that  every  student  comes  to  the  class  with  a laptop  computer  on  and  ready  for  learning.  There  are  already 
universities  requiring  every  student  purchasing  a laptop  computer.  The  message  is  becoming  clear  that  sophisticated 
computer  technology  will  be  at  the  fingertip  for  students  and  instructors  in  classrooms.  The  technology  will  soon  not 
only  be  used  for  writing  reports,  doing  research,  or  occasional  class  aid,  but,  more  importantly,  it  will  be  available 
for  class  learning  in  every  classroom.  However,  teaching  pedagogy  and  classroom  learning  are  much  behind  the 
evolving  of  technology.  In  the  advanced  technology  era,  not  only  the  traditional  lecture  and  note  taking  approach 
will  be  seriously  challenged,  but  also  the  static  use  of  technology  will  be  seriously  challenged  as  well.  Faculty  and 
students  will  have  to  face  dramatic  changes  in  the  process  of  teaching  and  learning.  Instructors  will  have  to  rethink 
the  way  of  teaching  and  students  will  have  to  accept  the  role  of  being  active  learners.  The  concept  of  active  learning, 
with  instructors  and  students  participating  together  as  they  develop  the  understanding  of  the  subject  matter,  has 
received  much  attention  in  the  recent  decades  (e.g.,  Lambert  & McCombs,  1 998;  Stage,  Muller,  Kinzie  & Simmons, 
1998).  In  the  student-centered  model,  students  are  provided  with  situations  to  deal  with.  Teachers  are  assumed 
responsibilities  of  guidance  and  facilitating  the  process  of  knowledge  discovery,  while  students  are  involved  with 
the  process  of  discovering  the  knowledge  themselves.  Student-centered  models  are  consistent  with  the 
constructivism  learning  theory,  where  both  place  significant  emphasis  on  the  learners'  active  participation  and  on  the 
development  of  knowledge  construction. 

Obstacles  Faced  in  the  Paradigm  Shift 
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Technology  plays  a crucial  role  in  helping  the  learner-centered  model  more  meaningful  and  more  diverse.  However, 
many  obstacles  are  yet  to  be  overcome  in  order  to  make  student-centered  curriculum  a reality  in  higher  education. 
These  include 

1.  Instructors  and  students  are  not  familiar  with  the  concept  of  student-centered  learning.  As  the  consequence, 
instructors  and  students  tend  to  think  that  student-centered  approach  means  students  are  ’customers’.  Therefore, 
instructors  tend  to  resist  against  the  approach.  Students  may  quickly  act  themselves  like  ’customers’. 

2.  Student-centered  learning  creates  many  organizational  problems  in  the  development  of  an  adequate  curriculum.  It 
takes  several  semesters  of  gradual  changes  to  feel  comfortable  with.  Many  may  choose  to  withdraw  back  to  the  more 
familiar  teacher-centered  approach. 

3.  Students  are  used  to  the  step-by-step,  more  predictable  teacher-centered  learning.  It  is  a difficult  philosophical 
shift  for  students  thinking  they  are  the  ones  who  develop  their  own  learning  and  discovering  knowledge  themselves. 
A study  conducted  in  (Lee,  1999)  indicated  that,  although  students  would  like  cooperative  learning  or  working  on 
projects,  the  lecture  and  note  taking  is  still  their  favorite  learning  style  (Lee,  1998). 

4.  In  the  current  higher  education  system,  student  opinion  survey  is  the  common  teaching  evaluation  tool  for  tenure 
and  promotion.  Many  instructors  think  that  the  change  from  the  familiar  teacher-centered  and  very  organized 
approach  to  a diverse  and  unpredictable  teaching  environment  will  affect  their  student  opinion  scores. 

5.  The  integration  of  technology  into  student-centered  curriculum  is  a challenging  task.  Regardless  how  much 
technology  knowledge  we  know,  it  is  never  fast  enough  to  catch  up  with  the  technology  evolvement.  This  introduces 
a serious  obstacle  for  instructors  to  integrate  technology  into  curriculum  effectively. 


Two  Fundamental  Issues  Before  Accepting  the  Student-Centered  Learning 

Research  related  to  framework  on  how  student  learns  ( APA,  14  Principles,  1995;  Alexander  and  Murphy,  1997)  has 
suggested  14  principles  to  explain  how  learning  takes  place.  These  14  principles  can  be  classified  into  four 
dimensions  as  (1)  cognitive  and  matacognitive  dimension,  (2)  motivational  and  affective  dimension,  (3) 
developmental  and  social  dimension,  (4)  individual  differences.  These  principles  shows  how  complex  a learning 
process  is.  The  success  of  learning  is  the  result  of  all  dimensions  interacted  together.  The  student-centered  learning 
is  a process  that  attempts  to  take  all  four  dimensions  into  account  to  assist  student  learning.  In  order  to  accomplish 
all  these  dimensions,  it  is  not  surprising,  that  many  obstacles  need  to  be  overcome.  Literature  has  provided  many 
useful  framework  and  guidelines  for  creating  student-centered  classrooms  and  curriculum  (e.g.,  Glasgow,  1997; 
Stage,  Muller,  Kinzie  & Simmons,  1998).  However,  before  accepting  and  implementing  the  model,  two 
fundamental  issues  on  motivation  for  instructors  and  students  to  accept  the  model  seems  to  be  overlooked.  These 
fundamental  issues  are 

(1)  What  should  be  an  adequate  operational  definition  of ’students'?  The  success  of  education  depends  on  how 
much  a student  is  learning.  Student-centered  learning  focuses  on  students  and  their  involvement  in  actively 
constructing  the  knowledge  themselves.  Therefore,  it  is  the  first  critical  step  to  find  an  acceptable 
operational  definition  of ’students' . 

(2)  How  to  motivate  the  instructor  and  students  work  together  as  facilitator  and  learners  rather  than  teacher  and 
student  relationship?  In  the  traditional  teacher-centered  learning,  instructor  defines  the  goals  and  objectives 
of  the  course.  He/she  then  teaches  the  topics  to  meet  these  goals  and  objectives.  If  the  learning  outcomes 
are  not  adequate,  it  is  often  to  hear  from  the  instructor  : ’students  do  not  work  hard;  students  are  not  as 
smart  as  old  days'.  Students  often  blame  the  instructor  : ’ I do  not  understand  what  he/she  said;  the  pace  is 
too  fast!’  There  is  usually  true  to  certain  degree  for  both  sides.  However,  once  such  a relationship  between 
instructor  and  students  begins  to  build  up,  it  is  no  longer  much  fiin  for  teaching  or  learning.  Similar  blames 
could  happen  if  the  student-centered  learning  is  implemented  without  the  agreeable  operational  definition 
of  ’students’  and  a fundamental  change  in  rethinking  the  roles  and  the  relationship  between  instructor  and 
students. 

Operational  Definition  of  ’Students’ 

Are  students  'customers'?  This  has  been  a very  controversial  question  among  different  groups  of  individuals.  It  is  fair 
to  say  that  there  is  no  the  right  answer  for  this  question.  It  is  not  possible  to  define  students  strictly  based  on  any 
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business  model.  The  philosophy  of  total  quality  management  tends  to  overlook  the  nature  of  feeling,  attitudes, 
motivations  and  intellectual  differences  among  students.  However,  some  basic  principles  and  tools  are  useful  to  help 
us  sought  out  causes  and  effects,  and  to  provide  ways  to  deal  with  the  causes.  One  essential  step  is  to  look  for  a 
compromised  acceptable  operational  definition  of  'students*.  A closer  investigation  about  the  roles  students  play  in 
the  higher  education  environment,  one  begins  to  notice  that  students  have  several  roles  in  the  higher  education.  A 
cause  and  effect  chart  helps  to  identify  these  roles  based  on  Deming’s  quality  improvement  philosophy  (1986). 
Three  major  roles  that  students  play  in  the  higher  education  can  be  identified: 

(1)  Students  are  considered  as  customers  in  the  service-oriented  environment,  including  registration,  financial 
aid,  library,  student  life  related  services.  The  main  goal  for  these  service-oriented  departments  is  to  assist 
students  in  making  their  learning  smoother.  To  these  departments,  students  are  their  major  customers. 
Therefore,  the  typical  quality  concept  of ‘customer  satisfaction’  can  be  defined  and  pursued. 

(2)  Students  are  considered  as  clients  at  the  departmental  level.  At  each  departmental  level,  students  receive 
advising,  placement,  course  scheduling  and  so  on.  Students  act  similar  to  the  situation  where  a client  asks 
for  consultation  from  a doctor.  The  client  has  the  right  to  request  a satisfied  technical  assistance,  while  the 
client  must  also  follow  doctor’s  direction  for  proper  medication  and  so  on.  Students  are  similar  to  clients  at 
the  departmental  level,  since  they  receive  advising  and  placement  based  on  the  assumption  that  students 
will  continue  to  study  and  meet  the  curriculum  requirement. 

(3)  Students  are  considered  as  work-in-progress  at  the  classroom  level.  Students  are  considered  similar  to  parts 
or  unfinished  works  when  they  come  to  a class.  The  knowledge  and  skills  learned  from  the  class  continue 
to  improve  the  work  for  the  ultimate  customers  of  employers  and  society.  In  the  process  of  building  the 
works,  quality  of  the  work  should  be  constantly  measured.  In  the  mean  time,  the  facilitator’s  quality  will 
also  be  measured  based  on  the  process  of  building  the  quality  works.  Therefore,  in  the  classroom 
environment,  students  are  more  similar  to  works-in-progress.  The  quality  is  measured  based  on  the 
requirement  from  the  ultimate  customers,  the  potential  employers  and  society,  not  only  based  on  what  the 
instructors  think  about  what  the  works  ought  to  be. 

This  operational  definition  of  students  attempts  to  clarify  confusion  of  the  roles  that  students  play.  Any 
oversimplification  could  introduce  controversy  among  different  groups.  It  is  important  to  provide  quality  services 
and  quality  education  for  our  ’students’;  customers,  clients  and  works-in-progress.  The  above  definition  helps  to 
clarify  the  possible  causes  of  the  obstacles  discussed  above.  It  will  help  instructors  and  students  in  understanding 
their  roles  in  a student-centered  learning  environment.  This  operational  definition  is  by-no-means  to  be  exhausting 
in  defining  all  of  the  roles  students  play. 

Shifting  Paradigm  from  Curriculum  Goals  and  Objectives  to  Standards  and  Responsibilities 

The  second  fundamental  issue  is  about  reasons  for  accepting  the  student-centered  model.  Most  instructors  were 
trained  in  the  teacher-centered  model.  It  is  an  approach  that  is  predictable,  and  we  all  feel  comfortable  with.  Students 
are  used  to  the  model  as  well.  Therefore,  without  a profound  reason,  it  is  difficult  to  convince  instructors  and 
students  to  make  such  a dramatic  change.  This  is  also  very  important  for  distance  learning.  This  fundamental  issue 
can  be  dealt  with  by  the  proper  shift  from  setting  goals  and  objectives  based  on  the  instructor’s  point  of  view  to 
setting  standards  and  responsibilities  for  both  instructor  and  students. 

The  term  ’standards’  in  this  situation  is  different  from  the  ’outcomes  standards’,  such  as  the  standard  that  students 
graduating  from  a certain  program  must  pass  a certain  test.  In  stead,  it  refers  to  ’the  process  standards’.  That  is,  I the 
process  of  learning,  students  are  asked  to  learn  each  topic  at  a certain  level  of  standards.  These  standards  are 
determined  based  on  the  requirement  of  higher  level  courses,  the  preparation  for  career  needs  or  society  needs, 
depending  on  the  types  of  courses  and  levels  of  courses.  For  an  example,  standards  for  a n introductory  statistics  can 
be  defined  based  on  the  courses  that  have  the  prerequisite  of  this  course  and/or  the  adequate  knowledge  and  skills 
for  understanding  day-to-day  information  in  our  every  day  life.  In  order  to  accomplish  standards  defined  for  a 
course,  each  topic  in  the  course  should  also  have  a set  of  standards  for  the  topic.  In  most  cases,  these  standards  are 
similar  to  the  goals.  However,  when  setting  up  the  goals,  the  instructor  often  relies  on  his/her  own  experience  and 
the  past  understanding  of  the  career  needs.  When  setting  up  the  standards,  the  instructor  will  need  to  communicate 
with  the  related  higher-level  course  instructors,  and  some  key  professional  fields  where  students  are  hired.  If  the 
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standards  are  defined  adequately,  it  is  usually  easy  to  communicate  with  students  with  regards  to  reasons  of  setting 
these  standards.  Students  are  appreciative  of  the  instructor’s  efforts  to  help  them  setting  up  these  standards. 

The  term  ’responsibilities’  is  not  only  for  students,  but  also  for  instructors.  It  is  necessary  that  a clear  explanation  of 
different  ’responsibilities’  for  instructors  and  students  are  communicated  with  students.  Once  a set  of  standards  is 
developed  for  the  topic,  various  concepts  will  be  covered,  and  therefore,  students  should  receive  a set  of 
responsibilities  for  the  level  of  understanding  of  each  concept.  In  many  cases,  the  responsibilities  for  students  are 
similar  to  objectives.  However,  since  they  are  developed  to  meet  the  standards,  a rubric  system  for  assessing  the 
level  of  understanding  should  also  be  developed.  One  important  strategy  to  communicate  the  responsibilities  to 
students  is  to  emphasize  that  these  responsibilities  are  for  both  students  and  instructor.  Students  are  responsible  for 
learning  the  knowledge  under  the  instructor’s  guidance.  The  instructor  is  responsible  for  designing  appropriate 
activities  and  using  various  teaching  strategies  to  facilitate  student  learning.  Computer  technology,  cooperative 
learning,  use  of  web  sites  and  so  on,  in  this  stage,  play  a crucial  role  in  helping  the  instructors  in  designing  and 
carrying  out  student-centered  learning  activities. 


A Case  Study 

Statistics  is  considered  a difficult  and  boring  subject  by  many  college  students.  Lee  (1997)  developed  a model  called 
the  PACE  model  for  teaching  introductory  statistics  to  help  student  learning,  P stands  for  Projects,  A stands  for 
Hands-on  Activities,  C stands  for  cooperative  learning  in  a Computer  classroom  environment,  and  E stands  for 
Exercises.  A typical  class  begins  with  conducting  a hands-on  activity  as  a group  project  and  collect  data  for  further 
analysis.  In  the  process  of  conducting  the  project  and  collecting  data,  group  discussion  is  conducted  to  discuss  the 
pros  and  cons  of  the  project  and  the  process  in  the  data  collection.  Computer  is  used  right  after  the  data  is  collected 
for  each  group  to  conduct  analysis  or  computer  simulations,  and  for  report  presentations.  The  computer  technology 
is  used  to  help  simplifying  computation  and  understanding  the  concepts.  Re-enforcement  is  carried  out  by  exercises 
and  projects.  The  PACE  model  is  a student-learner  centered  strategy  for  teaching  introductory  statistics.  Several 
studies  were  taken  to  assess  the  effectiveness  of  the  model  (e.g.,  see  Lee  (1998,  1999)).  Noticeable  difference  in 
problem-solving  strategy  was  found  from  students  who  were  taught  by  the  PACE  model  from  those  who  were  from 
the  traditional  class.  The  PACE  model  students  were  ’looking  for  similar  activities  and  projects  conducted  in  the 
classes’  to  solve  problems;  while  students  from  traditional  class  were  ’seeking  for  formulas’  to  solve  the  same 
problems.  On  the  other  hand,  we  also  noticed  that  the  motivation  and  expectation  remained  similarly  low  in  both 
classes.  One  main  reason  for  the  similarly  low  motivation  and  expectation  is  hypothesized  to  be  that  students  were 
not  experienced  to  the  stage  to  be  able  to  visualize  the  entire  picture  of  the  benefit  of  basic  statistical  concepts  in 
their  career  or  in  their  day-to-day  life  activities.  A fundamental  change  in  helping  students  realize  such  a ’big  picture’ 
is  necessary.  This  has  lead  me  to  the  develop  the  paradigm  of  integrating  ’standards  and  responsibilities’  into  the 
process  of  teaching  and  learning.  Such  a paradigm  is  becoming  important  in  the  use  of  advanced  computer 
technology,  since  students  will  be  more  independent  in  their  own  learning.  It  will  become  extremely  important  to 
maintain  students’  motivation  and  expectation.  Otherwise,  higher  education  could  easily  be  a place  of  ’diploma 
manufacturing’. 

The  paradigm  of  using  standards  and  responsibilities  were  implemented  in  the  Fall,  1999  PACE  class.  The  class  size 
was  twenty  eight.  Students  were  mainly  business  school  students.  This  is  a course  that  block  their  way  for  business 
school,  admission.  Students  are  typical  on-campus  students  who,  generally  speaking,  had  low  expectation,  little 
motivation,  and  came  with  rather  weak  mathematics  background.  At  the  beginning  of  each  topic,  students  were 
given  a sheet  of  standards  and  responsibilities,  and  were  summarized  and  discussed  after  finishing  the  topic.  An 
example  of  standards  and  responsibilities  for  the  topic  of  Exploratory  and  Graphical  Methods  is  given  in  the 
following: 

Standard:  Students  will  be  able  to  summarize  a real  world  data  using  proper  graphical  and  quantitative  tools  so  that 
the  summary  will  be  useful,  meaningful  and  valid,  and  will  be  able  to  use  computer  to  analyze  the  data,  and 
summarize  the  results,  and  present  the  results  in  written  and  orally. 

The  corresponding  responsibilities:  In  order  to  accomplish  this  standard,  students  will  learn: 


1)  Collecting  real  world  data  properly,  and  explain  possible  sampling  errors  in  the  process  of  data 
collection. 

2)  Distinguishing  quantitative  and  qualitative  types  of  data,  and  the  corresponding  graphical  methods  for 
summarizing  them. 

3)  Computing  descriptive  summaries  such  as  mean,  s.d.,  median,  percentile,  z-scores  by  hand  and  by 
computer. 

4)  Explaining  the  meaning  of  variation  and  giving  appropriate  real  world  cases  to  explain  the  magnitude 
of  variation  and  possible  factors  associated  with  the  variation. 

5)  Describing  histogram,  stem-leaf  plot,  box-plot,  time-series  plot,  and  explaining  their  differences  and 
usages  in  real  world  applications. 

6)  Demonstrating  empirical  rule,  and  using  the  concept  of  rare  events  in  real  world  applications. 

7)  Using  the  Z-scores  to  explain  the  concept  of  rare  events  in  real  world  applications. 

8)  Demonstrating  various  shapes  of  histograms,  and  their  relationship  with  mean  and  median. 

9)  Properly  matching  the  relationship  between  statistical  summaries,  histograms,  box-plot  and  time-plot. 

10)  Applying  the  tools  learned  from  this  topic  to  analyze  a real  world  case,  and  presenting  the  report  in 
written  and  orally. 

A similar  list  for  each  topic  was  developed,  and  was  given  at  the  beginning  of  the  topic.  Several  assessment 
activities  were  conducted  through  out  the  semester.  First,  students  were  asked  to  conduct  a self-assessment  by  rating 
the  degree  of  accomplishment  for  each  responsibility,  and  reasons  for  why  they  did  or  did  not  accomplish  well  of  the 
specific  responsibility,  then,  list  actions  that  will  be  helpful  to  help  them  accomplish  these  responsibilities.  Two 
other  assessment  activities  were  conducted  at  the  end  of  the  semester.  One  was  the  same  self-assessment  activity 
adding  an  overall  assessment  of  the  approach.  The  other  was  an  assessment  on  the  affect  domain  of  their  learning, 
which  include  attitudes,  beliefs,  motivations,  learning  styles,  use  of  computer,  and  other  background  information. 

Several  observations  were  summarized  below  from  the  self-assessment  and  overall  evaluation. 

(1)  Through  out  the  semester,  one  significant  difference  of  the  interaction  between  instructor  and  students  was 

immediately  notices:  When  students  came  for  individual  help  during  the  office  hours,  there  were  no 

students  making  excuses  about  how  difficult  it  was,  or  asking  to  explain  the  entire  the  topic  covered  in 
class.  Students  usually  came  with  a specific  question  and  was  able  to  pinpoint  where  they  did  not 
understand.  This  is  a reflection  of  the  understanding  of  their  responsibilities  for  each  topic  and  willing  to 
accept  their  role  as  active  learners.  As  a result,  students  were  informed,  and  instructor  was  aware  of  specific 
difficulties  students  encountered.  This  attitude  has  significantly  changed  the  classroom  atmosphere. 

(2)  Although  students  tended  to  rate  their  level  of  understanding  somewhat  higher  than  their  test  scores 
indicate,  the  relative  level  of  difficulty  of  each  responsibility,  and  reasons  were  very  useful  information  for 
continuous  improvement.  Here  are  some  examples  from  students'  self-assessment  for  the  topic  Exploratory 
and  Graphical  Methods; 

'The  problem  areas  in  this  section  are  stem-leaf  plot,  time-series  plot,  their  relationship 
and  applications. 

I still  do  not  see  the  use  of  steam-leaf plot,  box-plot.  I think  this  is  because  we  have  sued 
histogram  a lot  for  rare  events  and  variation,  and  so  on.  More  applications  using 
box-plots  or  stem-leaf  plot  will  be  good. 

The  two  concepts  that  have  been  rated  lower  level  of  understanding  most  often  were  'the  use  of  Z-scores  for 
interpreting  rare  events’  and'  the  relationship  among  summaries  and  various  plots.  This  information  is  very 
valuable  for  continuous  improvement  in  teaching  and  learning.  Instructor  can  find  out  the  problematic 
concepts  and  redesign  strategy  to  help  student  learning. 

(3)  The  overall  assessment  of  the  strategy  was  very  encouraging.  In  the  class  of  28  students,  everyone 
indicated  this  is  helpful,  especially  this  keeps  them  informed  what  standards  they  should  accomplish,  and 
how  they  can  accomplish  the  standards  in  every  topic.  When  asked  'is  it  worth  of  the  effort  to  develop  the 
standards  and  responsibilities  for  this  or  for  other  courses.  Every  student  indicated  yes  for  this  course  as 
well  as  for  other  courses. 
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Conclusion 


The  role  of  technology  in  education  continues  to  evolve.  Any  prediction  on  how  much  technology  will  impact  the 
education  would  be  meaningless.  Technology  provides  unlimited  opportunity  for  instructors  to  create  fascinating 
innovation  to  help  student  learning.  However,  since  it  is  more  and  more  difficult  to  catch  up  with  the  change  of 
technology,  the  hurdle  for  instructors  in  integrating  technology  into  curriculum  becomes  even  larger.  The  success  of 
a student-centered  learning  strategy  relies  upon,  first,  understanding  various  teaching  strategies,  second,  accepting 
some  strategies,  third,  willing  to  implement  and  assess  at  least  a strategy,  and  fourth,  willing  to  continue  to  improve 
the  strategy.  This  process  involves  both  students  and  instructors,  and  their  understanding  and  accepting  of  the  roles, 
respectively.  In  order  to  create  an  adequate  atmosphere,  it  is  important  that  students,  instructors  and  administrators 
have  a common  language  of  the  operational  definition  of  ’students’.  In  addition,  to  create  a high  expectation  and 
motivated  learning  environment,  a shift  from  teacher-centered  curriculum  of  setting  goals  and  objectives  to  student- 
centered  curriculum  of  standards  and  responsibilities  provides  a bridge  to  connect  instructors  and  students  for  the 
common  ground  of  pursuing  high  quality  of  education. 
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Abstract:  This  paper  provides  an  overview  of  a staff  development  initiative  at  University  of 
Wollongong,  a regional  university  in  Australia,  located  on  the  outskirts  of  the  largest  urban  area  in 
Australia  with  a population  of  ^proximately  500,000.  It  provides  a model  for  professional 
development  in  flexible  delivery  through  the  use  of  video  and  a website.  This  site  provides  access 
for  staff  to  key  concepts  and  in  depth  materials  on  flexible  delivery,  identifies  support  services  and 
their  roles  and  provides  up  to  date  policies  and  procedures  to  implement  the  changes  in  teaching 
practice.  It  highlights  the  challenge  for  staff  developers  to  "practice  what  we  preach". 


Introduction 

In  recent  times  there  has  been  a dramatic  change  in  technology  which  has  seen  a convergence  between  distance  and 
on  campus  education  (Walker  and  Walker,  1997).  Technology  has  been  used  to  enhance  interaction  between 
students  off  campus  and  their  lecturer  on  campus  (Agostinho,  Lefoe  and  Hedberg,  1997,  Agostinho,  Hedberg  and 
Lefoe,  1998).  Technology  has  also  been  used  to  improve  access  to  content  both  on  campus  and  at  a distance.  Online 
learning  facilities  are  being  offered  locally  and  by  overseas  institutions.  Students  may  now  be  just  a block  away,  a 
suburb  away  or  on  another  continent.  On  campus  students  may  only  attend  lectures  two  or  three  times  a semester,  if 
at  all,  but  may  have  in  depth  discussions  or  "chat"  regularly  online  with  the  subject  facilitator  and  other  students 
(Wills  et  al,  97). 

Effective  staff  development  is  one  of  the  keys  to  ensuring  success  in  this  dramatic  move.  By  placing  staff  in  the 
learner’s  shoes  this  project  provides  opportunity  for  teachers  to  actively  engage  in  the  learning  environments  that  are 
being  advocated  for  their  students.  Success  in  using  these  learning  environments  can  only  occur  when  the  staff 
member  is  as  comfortable  in  the  learning  environment  as  the  student  is. 

Background 

The  University  of  Wollongong  is  located  on  the  South  Coast  of  NSW,  Australia,  strategically  developed  to  meet  the 
needs  of  students  in  the  Illawarra  region.  The  main  campus  is  in  Wollongong,  with  centres  located  in  Sydney  and 
Berry,  and  an  international  centre  in  Dubai.  This  has  been  expanded  in  the  Year  2000  to  include  a campus  in  Nowra 
(one  hour  drive)  and  access  centres  in  Batemans  Bay  (3  hours  drive)  and  Bega  (6  hours  drive).  The  intention  with 
the  new  campus  and  access  centres,  was  to  establish  a South  Coast  Educational  Network  (SCEN), 

"to  develop  a comprehensive  cross-sectionally  articulated  plan  for  the  delivery  of  effective  post-compulsory 
education  and  training  across  university,  TAPE,  secondaiy  school  and  other  training  sectors"  (Illawarra  Regional 
Information  Service,  1996.  PI) 

The  vehicle  for  this  expansion  would  be  a network  of  educational  providers  operating  as  development  partners 
through  SCEN.  Within  this  network  the  University  of  Wollongong  decided  there  was  the  need  for  a degree  program 
that  could  be  delivered  flexibly.  "The  network  would  draw  upon  emerging  educational  technological  developments 
to  expand  the  availability  of,  and  improve  the  accessibility  to,  relevant  educational  programs"  (Illawarra  Regional 
Information  Service,  1996.  PI) 
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The  SCEN  initiative  had  implications  for  the  delivery  of  a large  number  of  subjects  in  a different  format  to  current 
traditional  practice.  The  rapid  growth  of  flexible  delivery  at  University  of  Wollongong  would  require  access  to  a 
variety  of  resources  and  an  understanding  of  alternative  delivery  methods. 

Flexible  delivery,  in  this  situation,  is  defined  as  an  approach  to  teaching  and  learning  which  increases  access  to 
education  for  a wide  range  of  students  by  offering  greater  student  control  over  time,  place  and,  potentially,  pace  of 
study.  Technology,  where  appropriate,  is  utilised  to  support  communication  and  access  to  information  and  to  move 
towards  a more  student  centred  approach  to  teaching  and  learning.  The  most  appropriate  delivery  medium,  (eg  a 
lecture,  a video  or  a book),  can  be  chosen  to  meet  the  learning  outcome  and  the  student's  needs.  Interaction,  (learner- 
learner  and  teacher-learner)  can  be  facilitated  through  face-to-face  contact  or  through  supporting  technology  such  as 
videoconference,  Internet,  phone,  fax.  (Bell  and  Lefoe,  1998) 

Role  of  development  support  units 

From  the  broader  perspective  of  the  current  higher  education  environment,  both  individuals  and  faculty  units  have 
been  buckling  under  the  combined  weight  of  devolved  responsibility  for  management,  administration,  technology 
implementation,  professional  development  and  a generally  increased  workload.  Historically,  many  of  the  teaching 
staff  developed  subjects  in  the  traditional  lecture/  tutorial  format  on  their  own. 

The  move  to  flexible  delivery  of  subjects  means  thinking  differently  about  this  process  and  moving  into  a team 
based  environment,  especially  where  the  development  of  resources  is  involved.  Support  structures  are  critical  to 
ensure  that  this  journey  through  chaos  and  confusion  results  in  quality  outcomes  for  all  involved. 

In  order  to  be  successful,  the  move  to  flexible  delivery  could  only  occur  where  this  support  infrastructure  was  put  in 
place  and  teaching  staff  could  tap  into  what  they  needed  when  they  needed  it.  In  recognition  of  this  the  university 
had  a variety  of  support  units,  many  of  whom  had  undergone  change  during  recent  restructuring  in  order  to  meet 
these  emerging  needs  (Curtis,  Lefoe,  Merten,  Milne  and  Albury,  1999). 

The  degree  support  structure  in  the  University  of  Wollongong  includes: 

* Centre  for  Educational  Development  and  Interactive  Resources  (Includes  Educational  Development  Lecturers, 
Interactive  Multimedia  Unit,  Audio  Visual  Unit  and  Electronic  Publishing)  The  Centre  has  principle  responsibility 
for  supporting  faculties  in  their  adoption  and  utilisation  of  technology  within  their  teaching  practice.  It  also  provides 
a role  in  quality  control  of  resources. 

* Learning  Development  Unit  (includes  Lecturers  in  Learning  Development  and  Tertiary  Literacy  officer)  Their 
approach  of  providing  all  students  with  assistance  in  making  the  transition  into  tertiary  and  discipline  based  studies 
is  an  innovative  and  systemic  approach  to  learning  support  (Skillen,  Merten,  Percy  and  Trivett,  1998). 

* Academic  Registrar's  Division  which  has  principle  responsibility  for  student  administration 

* Information  Technology  Services  provides  the  infrastructure  for  improved  use  of  technology  at  all  locations. 

* The  library  provides  an  outreach  librarian  whose  role  is  to  support  lecturers  and  students  involved  in  flexibly 
delivered  subjects. 

The  purpose  of  the  resource 

This  staff  development  resource  brings  together  information  about  the  roles  and  responsibilities  of  all  the  support 
units  as  well  as  providing  academic  staff  with  experience  in  flexible  delivery  through  the  use  of  the  resources.  The 
resource  models  the  practice  of  flexibly  delivering  professional  development.  It  utilises  various  media  and  includes 
an  interactive  dimension  in  a web-based  learning  environment.  It  models  the  environment  that  will  be  used  by 
academics  in  teaching  in  other  locations  and  on  campus,  whilst  providing  access  to  people,  policies  and  procedures 
to  support  development  of  subjects  in  a more  flexible  mode. 

Description  of  the  resource 

The  staff  development  package  includes  two  integrated  resources: 

1.  A video  which  provides  case  studies  of  teaching  methods  utilised  by  staff  currently  engaged  in  flexible  delivery  in 
a number  of  faculties. 
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2.  A web  site  (http://cedir.uow.edu.au/Projects/FlexDel/)  which  links  teaching  staff  to  support  staff  for  subject 
development  as  well  as  utilising  information  from  an  existing  CDROM,  Teaching  at  a Distance,  developed  by  the 
University  (in  collaboration  with  several  others)  for  a consortium  of  universities  in  Australia  (PAGE)  (Wills  et  al, 
1998).  The  existing  material  has  been  redesigned  to  be  specific  to  the  needs  of  University  of  Wollongong  teachers. 
An  interactive  dimension  to  the  resource  has  been  added  through  a discussion  forum,  which  allows  for  asynchronous 
discussion  between  different  people  by  removing  the  restrictions  of  time  and  place. 

The  website  provides  four  levels  of  resources  for  teaching  staff. 

* Level  1 - Provides  information  about  flexible  delivery  and  its  implementation  at  University  of  Wollongong  and  the 
various  support  teams  for  curriculum  development  including  Centre  for  Educational  Development  and  Interactive 
Resources,  Library  Services  and  Student  Support  Services.  Images  of  people  in  their  roles  at  University  of 
Wollongong  are  used  at  this  first  level  to  personalise  the  information  and  introduce  the  topics.  It  defines  the  role  of 
the  various  groups  and  the  support  for  subject  development  that  they  are  able  to  provide.  Its  purpose  is  to  encourage 
teaching  staff  to  consider  contacting  these  people  for  support  and  to  identify  them  on  campus.  This  level  also 
introduces  a discussion  forum. 


* Level  2 - A detailed  overview  of  each  support  area  and  important  issues  that  should  be  considered  in  initial 
planning  and  design.  The  project  also  draws  on  other  resources  created  by  Centre  for  Educational  Development  and 
Interactive  Resources  such  as  the  National  Council  Open  Distance  Education  Flexible  Learning  web  site 
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(http://cedir.uow.edu.au/ncode). 

• Level  3 - Provides  more  detailed  information  on  topics  covered  in  Level  2 information.  This  information  is  usually 
provided  as  downloadable  files  that  can  be  printed,  where  required. 

• Level  4 - External  Web  based  resources. 

Using  the  resource 

This  professional  development  initiative  was  designed  to  meet  the  needs  of  academic  staff  to 

• highlight,  through  the  use  of  the  video,  modes  of  flexible  delivery  that  are  being  used  on  campus 

• provide  a learning  environment  which  models  the  practice  of  flexible  delivery 

• identity  in  a cohesive  way  the  support  units  on  campus 

• clarity  the  new  roles  and  directions  for  these  units 

• provide  interaction  with  other  teaching  staff  who  are  engaging  in  this  practice  through  the  discussion  forum 

• support  academic  learning  needs  about  flexible  delivery 

The  resource  will  be  used  in  a number  of  ways  for  professional  development.  At  the  individual  level  it  can  be  used 
at  a time  or  place  which  meets  the  need  of  the  staff  member.  It  will  also  be  utilised  at  the  small  group  level  and  in 
faculty  staff  development  programs  to  support  initiatives  in  this  area.  The  University  of  Wollongong  has  a 
compulsory  subject  for  new  lecturers.  Introduction  to  Tertiaiy  Teaching,  which  has  elective  modules  on  such  things 
as  redesigning  subjects  for  flexible  delivery.  This  initiative  provides  the  starting  point  for  this  module  and  for  others 
related  to  thinking  differently  about  teaching  and  learning. 

Conclusion 

Planning  for  flexible  delivery  has  meant  making  drastic  changes  not  only  in  thinking  about  curriculum  development, 
but  also  in  the  whole  infrastructure  of  the  university  from  administration  to  information  services.  The  University  of 
Wollongong  now  has  in  place  many  structures,  some  of  which  were  under  utilised  when  implementing  new  ways  of 
teaching.  This  resource  provides  a link  for  academic  staff,  not  only  to  information  about  flexible  delivery  at  many 
levels,  but  also  to  staff  who  are  able  to  work  in  a team  based  environment  to  improve  subject  development  and  to 
provide  support  to  staff  as  they  make  changes  in  their  teaching  practice.  As  the  University  of  Wollongong  expands 
to  include  other  locations,  professional  development  for  teaching  staff  can  model  the  changing  practices  for  teaching 
and  learning.  ’’Practising  what  we  preach”  is  an  important  component  of  professional  development. 
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Abstract:  Pythagoras  is  one  of  the  pilots  in  the  Open  Learning  Environment  (OLE)  Project  that 
develops  pedagogically  and  technically  appropriate  learning  environment  for  the  net.  On  the 
Pythagoras  pilot  a learning  environment  of  the  same  name  was  developed  as  an  aid  to  teaching  in 
secondary  and  upper  secondary  schools  with  a mission  to  make  Pythagoras  into  a commercial  net- 
product  and  to  study  its  use  in  teaching.  The  Pythagoras  Learning  Environment  comprises  theory, 
interactive  exercises,  a question  tool  and  communication  tools.  In  the  implementation  of  Pythagoras 
use  was  made  of  Java  Remote  Method  Invocation  and  Servlet  technologies  among  others,  which 
offer  a flexible  and  efficient  way  of  producing  WWW  systems.  A teaching  experiment  was 
arranged  for  Pythagoras  to  study  the  usability  of  Pythagoras  and  users’  opinions.  The  students 
particularly  enjoyed  doing  research  using  the  question  tool.  It  may  be  concluded  on  the  basis  of  the 
teaching  experiment  that  using  the  net  as  a tool  for  learning  offers  considerable  advantages  over 
conventional  study. 


Introduction 

The  article  describes  the  Pythagoras  pilot  as  a part  of  the  Open  Learning  Environment  (OLE)  Project  (Ruokamo, 
Pohjolainen  1999).  On  the  Pythagoras  pilot  a learning  environment  in  statistical  mathematics  was  produced  for  the 
net  (Pythagoras  Learning  Environment).  The  article  presents  the  background  to  the  pilot,  the  parties  involved  in  the 
research,  development  and  in  the  teaching  experiment  and  the  aims  of  the  pilot.  This  is  followed  by  a presentation  of 
the  functionality  of  the  Pythagoras  Learning  Environment  and  its  technical  realisation.  A teaching  experiment  was 
arranged  with  Pythagoras  and  the  realisation  and  results  of  this  are  presented  in  the  article.  The  article  further 
considers  the  commercialization  of  the  Pythagoras.  The  article  concludes  with  some  conclusions  drawn  on  the  basis 
of  the  research  and  development  work. 

Background  to  the  pilot  and  co-operating  parties 

The  first  version  of  the  Pythagoras  Learning  Environment  was  created  in  the  Distance  Learning  in  Multimedia 
Networks  Project  (ETAKAMU)  research  project  on  the  pilot  in  natural  sciences  (Pohjolainen,  Ruokamo  1999),  This 
was  carried  out  on  the  responsibility  of  the  Hypermedia  Laboratory  of  the  Tampere  University  of  Technology 
(TUT).  At  the  time  of  ETAKAMU  the  main  responsibility  for  the  research  and  development  work  on  the  Pythagoras 
rested  with  Heikki  J.  Maenpaa,  M.  Sc.  (Eng.).  The  content  selected  for  the  Pythagoras  was  statistical  mathematics  as 
problems  in  statistical  mathematics  lend  themselves  particularly  well  to  computer  presentation.  The  Hypermedia 
Laboratory  has  produced  numerous  net  courses  in  mathematics,  the  latest  of  which  is  the  introductory  part  of  a 
course  on  matrix  calculation  (Matrix  Calculus  1)  using  a development  version  of  the  A&O  Open  Learning 
Environment  (Pohjolainen,  Ruokamo,  Nykanen  1999),  The  research  findings  of  the  Pythagoras  pilot  and  the 
experience  gained  serve  as  a basis  for  development  work  on  A&O. 

Edita  Ltd  has  supported  the  OLE  Project  and  the  Pythagoras  pilot  in  particular.  A number  of  teachers  from 
comprehensive  and  upper  secondary  schools  in  the  Tampere  Region  have  been  involved  in  the  development  of 
Pythagoras.  Among  die  most  active  members  were:  Jussi  Kytomaki  (Ylojarvi  Upper  Secondary  School),  Paivi 
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Portaankorva-Koivisto  (Moisio  Secondary  School),  Riikka  Uotila  (Ylojarvi  Upper  Secondary  School)  and  Pauli 
Koivula  (Teisko  Secondary  School  and  Tampere  Upper  Secondary  School  for  Adults).  These  teachers  not  only 
participated  in  the  development  work  of  the  learning  environment  but  also  took  part  in  arranging  the  teaching 
experiment  and  in  making  the  teacher’s  guide, 

Aims  of  the  pilot 

In  the  first  year  of  the  OLE  Project  the  main  aim  of  the  Pythagoras  pilot  was  to  make  Pythagoras  into  a commercial 
net  product.  This  means  that  the  technical  and  pedagogical  features  of  Pythagoras  were  to  be  developed  to  a standard 
at  which  it  would  be  possible  to  market  and  use  it  over  the  net.  Commercialization  as  a whole  involved  many  other 
considerations,  and  the  main  responsibility  for  these  was  with  Edita  Ltd,  The  other  aims  of  Pythagoras  were  to 
create  a learning  environment  that  would  be  user-friendly,  versatile,  interactive  and  exploit  the  latest  technologies. 

Structure  and  functions  of  the  Pythagoras  Learning  Environment 

Pythagoras  is  a hypermedia-based  learning  environment.  The  concept  learning  environment  subsumes  certain 
essential  characteristics  compared  with  mere  learning  material,  which  is  to  be  addressed  later  in  this  chapter.  In 
management  of  a learning  environment  the  main  factors  are  the  roles  of  the  users  and  the  rights  and  duties  inherent 
in  those  roles,  Pythagoras  is  used  and  managed  over  the  net.  There  are  three  main  user  roles:  student,  teacher  and 
maintenance  person.  A group  of  students  and  a teacher  together  constitute  a teaching  group,' which  Pythagoras  takes 
as  a basic  unit  for  management  purposes. 

Technical  framework 

The  basic  framework  of  Pythagoras  is  a group  of  linked  hypermedia  documents.  In  the  creation  of  the  Pythagoras 
static  HTML  pages,  as,  for  example,  the  theory  of  statistical  mathematics,  the  FML  publication  tool  developed  in  the 
Hypermedia  Laboratory  was  used  (Nykanen  1999).  Some  of  the  HTML  documents  in  Pythagoras  are  dynamically 
created.  Dynamic  refers  here  to  the  fact  that  the  content  and/or  the  external  appearance  of  a HTML  document  varies, 
e,g.,  depending  on  the  input  obtained  from  the  user.  In  the  realisation  of  Pythagoras  Java  Servlets  were  used  almost 
exclusively  in  the  creation  of  dynamic  HTML  pages.  Servlets  enable  efficient  and  scaled  server  programmes 
operating  through  CGI  interface  and  can  be  produced  with  an  object-based  language  (Articles  of  Servlets). 

Another  prime  element  in  the  realisation  of  Pythagoras  is  the  independent  programmes  created  as  Java  applets  that 
are  executed  in  the  client  computer.  The  major  advantage  in  using  applets  is  getting  away  from  the  rigid  page 
concept  of  WWW.  In  Pythagoras  applets  were  used  especially  to  create  interactive  elements.  Applets  were  used  in 
such  a way  that,  e.g.,  instead  of  one  large  applet  several  smaller  general-purpose  applets  were  used  on  the  HTML 
pages.  Using  several  applets  on  the  same  page  may  from  time  to  time  cause  technical  browser  problems.  Some  of 
these  problems  have  already  been  solved  through  new  browser  versions,  but  in  practice  problems  may  still  arise  as 
there  continue  to  be  old  browser  programmes  in  general  use. 

Pythagoras  is  a distributed  application.  The  applets  store  the  information  in  the  server  computer,  for  which  Remote 
Method  Invocation  (RMI)  technique  was  mostly  used.  RMI  is  a feature  included  in  Java  language,  which  makes  it 
possible  to  call  up  Java  programme  methods  via  the  Internet  (Articles  of  Remote  Method  Invocation).  Certain 
problems  exist  in  the  use  of  RMI,  especially  with  support  from  certain  browser  programmes  and  client-side 
firewalls.  The  problem  concerning  firewalls  can  be  solved  using  Servlets  (McPherson  1999).  A separate  RMI  server 
is  initiated  for  each  teaching  group.  The  advantage  of  this  is  significant  scalability  - Pythagoras  functions  without 
changes  with  both  large  and  small  user  numbers. 

Functionality  of  Pythagoras  learning  environment 

In  the  planning  of  any  net  pages  whatsoever  one  of  the  foremost  considerations  is  navigation,  i.e.  the  logic  by  which 
a user  progresses  from  page  to  page.  In  Pythagoras  an  effort  was  made  to  ensure  that  navigation  is  optimally  simple, 
and,  above  all,  uniform.  At  the  top  of  each  page  in  Pythagoras  there  is  a menubar  for  access  to  the  four  central 
elements  of  Pythagoras.  These  are  learning  material,  workshop,  teachers*  lounge  and  exhibition.  In  addition  to  these 
four  areas  mention  should  also  be  made  of  the  front  page,  see  [Figure  1],  which  contains  some  important  features. 
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Figure  1:  Front  page  of  the  Pythagoras  Learning  Environment 


The  function  of  the  front  page  of  Pythagoras  is  to  serve  as  the  starting  page  of  the  learning  environment.  There  is  a 
tool  for  students  to  enrol  in  their  teaching  group.  From  the  front  page  a user  can  log  in  to  Pythagoras.  Loging  in  is 
optional,  but  is  makes  it  easier  for  the  user  to  use  other  elements  of  Pythagoras  later. 

The  learning  material  in  Pythagoras  contains  elements  pertaining  to  conventional  learning:  theory  and  practical 
exercises.  The  learning  material  includes  examples  and  directions  for  carrying  out  statistical  research.  One  of  the 
most  interesting  features  of  Pythagoras  is  the  interactive  exercises.  There  are  seven  of  these,  and  most  offer 
alternatives  of  different  degrees  of  difficulty.  The  exercises  address  various  aspects  of  the  theory  and  there  are  links 
to  the  appropriate  theoretical  sections.  The  individual  exercise  page  is  realised  with  general  task  elements.  Some  of 
these  elements  are  used  to  present  the  initial  information  and  some  for  the  input  of  the  solution.  Every  exercise  page 
has  an  instruction  panel  on  which  the  instructions  vary  according  to  what  part  of  the  exercise  the  student  is  currently 
undertaking.  When  the  student  has  completed  the  exercise  Pythagoras  checks  the  answer.  If  there  is  something 
wrong  in  the  answer  Pythagoras  indicates  the  location  of  the  mistake  and  provides  further  prompts  for  the  solution 
of  the  task.  Each  checking  as  well  as  the  correct  answer  is  registered  in  the  database  for  achievements.  The  student 
can  monitor  his/her  own  achievements. 

The  Pythagoras  workshop  contains  those  essential  features  that  distinguish  the  learning  material  of  the  learning 
environment.  The  tools  to  access  groupwork  and  research  are  located  in  the  workshop.  Through  the  discussion  and 
chat  tools  the  Pythagoras  offers  its  users  the  opportunity  to  communicate.  The  survey  tool  in  the  workshop  enables 
Gallup-type  surveys.  There  is  also  a WWW  editor  for  students  to  write  their  research  reports  for  the  net.  [Figure  2] 
shows  an  example  of  a research  report  completed  through  a project  among  students.  The  survey  page  allows 
students  to  carry  our  questionnaire  surveys  to  which  other  students  may  respond.  The  questionnaires  may  consist 
either  of  multiple  choice  or  preference  questions.  The  material  obtained  from  the  responses  can  be  examined  and 
stored  for  later  use  and  may  be  used  as  a baseline  for  several  Pythagoras  exercises.  On  the  basis  of  the  data  obtained 
from  the  survey  the  students  may  write  a research  report.  For  making  reports  there  is  the  WWW  editor  developed  in 
co-operation  with  the  KUOMA  Pilot  of  the  ETAKAMU  Project  (Leinonen  1998,  Piiksi  1999).  Using  the  WWW 
editor  the  students  can  write  a research  report,  append  to  this  the  stored  diagrammatic  pictures  and  save  the 
document  in  HTML  format.  The  teacher  may  as  his/her  discretion  publish  the  best  reports  in  the  exhibition  space. 

The  Pythagoras  teachers'  lounge  is  an  area  intended  for  teachers  only.  It  contains  discussion  and  chat  tools 
comparable  to  those  in  the  workshop.  There  is  also  a tool  for  monitoring  and  managing  students’  achievements. 

Teaching  experiment 

General 

The  first  Pythagoras  teaching  experiment  under  the  OLE  Project  was  carried  out  in  spring  1999.  The  purpose  of  the 
experiment  was  to  investigate  how  users  reacted  to  the  new  user  interface  and  to  the  numerous  new  features.  The 
teaching  experiment  was  carried  out  in  the  Hypermedia  Laboratory’s  own  computer  classroom  that  enabled  us  to 
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collect  data  with  various  technical  tools.  Those  participating  in  the  experiment  were  students  from  the  ninth  class  of 
the  Teisko  Secondary  School, 

Research  methods 

Data  on  the  Pythagoras  teaching  experiment  were  collected  by  questionnaire,  by  video  and  by  participative 
observation.  Students’  background  information  and  opinions  and  experiences  were  ascertained  by  the  questionnaire. 
The  questions  on  the  questionnaire  were  formulated  on  the  basis  of  the  various  elements  used  in  the  experiment.  The 
questions  were  in  the  form  of  positive  or  negative  statements  to  which  the  students  responded  on  a scale  of  1 to  5, 
where  5 corresponded  to  ‘agree  entirely’  while  1 corresponded  to  ‘disagree  entirely’.  A total  of  15  students 
responded.  One  computer  screen  was  taken  on  video  using  a direct  connection  to  the  display  driver  card.  Using  the 
video  data  an  attempt  was  made  primarily  to  ascertain  the  usability  of  Pythagoras.  A great  deal  of  data  was  also 
obtained  by  interviewing  course  instructor  Pauli  Koivula. 

Results  of  the  experiment 

A questionnaire  was  used  to  gather  background  information  on  the  students  participating  in  the  Pythagoras  teaching 
experiment.  Most  of  the  respondents  were  fairly  neutral  regarding  the  study  of  mathematics,  but  most  of  them 
considered  mathematics  important  as  a subject  of  study.  Most  of  the  respondents  reported  that  they  found 
mathematics  difficult.  More  than  half  the  respondents  used  the  computer  regularly  and  almost  half  made  abundant 
use  of  the  Internet.  Most  of  those  taking  part  in  the  experiment  stated  that  they  were  glad  to  be  involved.  The 
usefulness  of  Pythagoras  in  the  study  of  statistical  mathematics  was  elicited  in  two  questions  formulated  in  rather 
different  ways.  The  students  reported  that  Pythagoras  helped  in  the  study  of  statistical  mathematics  compared  to 
conventional  methods.  Almost  all  respondents  reported  that  Pythagoras  was  easy  to  use.  The  most  important  feature 
to  be  investigated  in  the  questionnaire  was  the  survey  tool  and  the  production  of  the  research  report  with  the  help  of 
the  WWW  editor.  The  students  reported  that  responding  to  questions  was  both  easy  and  pleasant.  While  many  did 
not  consider  making  one’s  own  survey  difficult  and  found  using  the  WWW  editor  easy,  several  needed  guidance  in 
using  these  features.  The  respondents  reported  that  it  was  pleasant  to  make  one’s  own  questionnaire.  Students’ 
opinions  on  the  interesting  nature  of  the  Pythagoras  exercises  were  neutral.  On  the  average  the  exercises  were 
considered  difficult. 

The  video  material  gathered  in  the  teaching  experiment  was  of  use  particularly  in  improving  the  usability  of 
Pythagoras.  Examination  of  the  recorded  data  revealed  several  different  problems  with  the  user  interface.  One 
salient  feature  here  was  that  the  students  needed  some  considerable  time  to  realise  which  of  the  tasks  was  baseline 
information  and  which  required  input  on  their  part. 

Certain  technical  problems  emerged  in  the  experiment;  they  were  connected  to  the  fact  that  the  Netscape’s  Java 
virtual  machine  appeared  to  periodically  forget  the  ’garbage  collection',  thereby  causing  superfluous  information  to 
accumulate  in  memory,  culminating  in  the  collapse  of  the  browser.  This  was  not  a major  problem  as  it  occurred  on 
one  computer  at  most  a couple  of  times  in  the  course  of  the  experiment. 

Commercialization  of  Pythagoras 

General 

The  Pythagoras  pilot  ended  at  the  end  of  January  2000,  It  has  been  installed  to  the  server  of  Edita  Ltd,  where  it  is  to 
be  marketed  to  various  educational  institutions  as  an  educational  product.  In  the  course  of  the  research  during  1999 
Pythagoras  was  modified  to  meet  the  needs  of  commercialization.  We  now  consider  commercialization  from  the 
perspective  of  education  and  software  technology. 

Technical  considerations 

The  commercialization  of  a learning  environment  does  not  significantly  differ  from  the  commercialization  of  any 
other  net  environment  as  far  as  the  technology  is  concerned.  Commercialization  should  be  taken  into  consideration 
already  at  the  point  at  which  the  definition  of  the  software  is  made.  Due  to  the  nature  of  the  software  development 
work,  however,  this  is  not  always  possible.  Certain  factors  should  nevertheless  be  bom  in  mind  in  the  planning  of 
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any  software  at  all.  The  most  important  of  these  are  probably  the  definition  of  user  identification  and  use  session 
concepts  at  an  early  stage.  It  is  frequently  problematic  and  time-consuming  to  insert  user  identification  into  an 
existing  environment.  These  concepts  are  the  basis  in  planning  the  remunerations  system. 

It  was  decided  to  create  the  remuneration  system  of  Pythagoras  in  such  a way  that  users  can  purchase  user  rights  for 
a specified  duration.  The  system  does  not  require  the  measurement  of  actual  time  of  usage,  thus  there  was  no  need 
for  a particularly  complex  use  session  system.  It  was  decided  to  base  user  rights  on  teaching  groups,  which  is  most 
convenient  for  the  action  model  of  a normal  school. 

It  was  felt  desirable  to  create  an  automatic  ordering  system  for  Pythagoras  to  be  connected  to  the  Edita  Netmarket 
already  in  use.  The  function  of  the  ordering  system  will  be  to  receive  orders  and  also  to  create  user  identities, 
teaching  groups  and  other  technical  settings  for  each  order.  In  practice  the  system  was  realised  in  such  a way  that  the 
customer  completes  a form  in  Netmarket.  Netmarket  sends  information  on  this  to  Pythagoras,  which  accepts  the 
order.  Next  the  customer  can  register  his/her  order  and  select  user  identities  and  passwords  in  Pythagoras.  This  way 
the  customer  gets  the  identities  s/he  wants,  which  is  a major  aid  to  their  memorability.  The  registration  of  the  order 
is  accomplished  in  such  a way  that  it  reminds  several  Windows  users  of  the  familiar  programme  installation  wizard. 

Pedagogical  considerations 

Commercialization  also  entails  attention  to  pedagogical  considerations.  There  is  frequently  far  too  few  instructions 
in  surprisingly  many  net  environments.  At  the  development  stage  instructions  are  frequently  felt  to  be  superfluous  as 
using  one’s  own  product  generally  seems  fairly  straightforward.  At  the  commercialization  stage  the  need  for 
instructions  becomes  more  pronounced,  as  the  users  usually  lack  that  internalised  model  of  the  use  of  computer 
systems  which  is  characteristic  of  professionals.  The  interface  of  a new  system  must  be  designed  according  to  some 
familiar  model,  or  then  a new  and  possibly  better  model  must  be  learnt.  Learning  the  new  model  is  best  achieved 
through  detailed  instructions  for  use.  A learning  environment  product  also  needs  a teacher’s  guide.  A net  product  is 
something  of  an  innovation  for  teachers  and  they  need  instructions  as  to  what  can  be  done  in  the  learning 
environment  and  how  to  time  the  course  using  the  learning  environment.  A teacher’s  guide  has  been  compiled  for 
Pythagoras  by  a group  of  teachers  (Paivi  Portaankorva-Koivisto,  Riikka  Uotila,  Pauli  Koivula). 

Conclusions 

Application  of  WWW  technologies  to  the  construction  of  teaching  tools 

The  construction  of  an  interactive  learning  environment  for  the  net  is  still  a fairly  technically  demanding  task.  The 
cornerstone  on  which  the  technical  realisation  of  Pythagoras  principally  rests  is  Java  language,  which  nowadays 
offers  all  the  necessary  tools  to  produce  WWW  systems.  The  latest  additions  to  Java,  RMI  and  Servlets  provide  the 
more  manageble  technology  as  regards  software  technology  for  the  realisation  of  WWW  systems  than  systems 
realised  with  the  traditionally  variable  script  languages.  The  greatest  stumbling  block  with  Java  technology  is  the 
browser  programmes'  virtual  machines'  poor  accomplishment  of  the  Java  applets. 

Using  the  net  as  a teaching  tool 

It  can  be  stated  on  the  basis  of  the  results  of  the  Pythagoras  teaching  experiment  that  using  the  net  as  a tool  for  study 
has  considerable  advantages  over  conventional  study.  Although  several  experiments  with  the  net  for  teaching 
purposes  have  been  fairly  biased  towards  learning  materials,  it  is  clear  that  using  the  net  is  beneficial,  for  example, 
in  doing  groupwork.  The  greatest  advantages  of  groupwork  over  the  net  are  that  those  participating  need  not  be  in 
the  same  place,  or  even  work  at  the  same  time.  In  Pythagoras  the  question  tool  is  a good  example,  which  would  be 
difficult  to  arrange  without  the  use  of  the  net. 

ADP  and  using  the  net  also  makes  it  possible  for  students  to  study  with  materials  they  themselves  have  produced. 
For  example,  the  responses  to  the  Pythagoras  questions  can  be  used  both  for  research  reports  and  as  starting  values 
for  exercises. 

Distance  learning  creates  new  prospects  for  study.  It  is  no  longer  important  where  the  studying  is  done;  the  main 
question  is  rather  how  to  study  in  a manner  convenient  to  oneself  However,  although  distance  learning  makes 
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individual  study  possible,  it  should  be  bom  in  mind  that  students  need  interaction  with  the  teacher  and  other 
students. 

References 

Articles  of  Remote  Method  Invocation,  (online) 

http;//developer.java.sun.com/developer/technicalArticles/RMI/index.html 

Articles  of  Servlets,  (online).  http://developer.java.sun.com/developer/technicalArticles/Servlets/index.html 

Leinonen,  T.  (1998).  KUOMA  - Learning  Environment  - An  Implementation  of  Collaborative  Project  Learning  in 
the  World  Wide  Web.  January  1998.  Netpaper  Ideas  Culture  Technologies  Ministry  of  Public  Education, 

Department  of  Studies,  Italy. 

Matrix  Calculus  1 . Introduction  part  of  Matrix  Calculus  1 course  presented  in  A&O  Learning  Environment  (online). 
http://matriisi.ee.tut.fl/matriisilaskenta 

McPherson,  S.  (1999).  Java  Servlets  and  Serialization  with  RMI.  JDC  Newsletter  Oct.  19,  1999  (online). 
http://developer.java.sun.com/developer/technicalArticles/RMI/rmi/ 

Nykanen,  O.  (1999).  Producing  Structured  WWW  Hypermedia  with  Standard  Text  Editors:  Functional  Mark-up 
Language  Toolkit.  Intelligent  Computer  and  Communications  Technology.  Teaching  Learning  for  the  list 
Century.  PEG99.  Ninth  International  PEG  Conference  10-12  July  1999.  Conference  Proceedings.  248-261. 

Open  Learning  Environment  project  (OLE),  Introduction,  (online)  http://matriisi.ee.tut.fi/ao/intro.html 

Piiksi,  K.  (1999).  The  Environment  Net  - Environmental  Education  in  the  Open  Learning  Environment.  Intelligent 
Computer  and  Communications  Technology.  Teaching  <Sc  Learning  for  the  list  Century.  PEG99.  Ninth  International 
PEG  Conference  10-12  July  1999.  Conference  Proceedings.  262-270. 

Pohjolainen,  S.,  Ruokamo,  H.,  Nykanen,  O.  (1999).  A&O  Open  Learning  Enviroment.  Intelligent  Computer  and 
Communications  Technology.  Teaching  & Learning  for  the  2 1st  Century.  PEG99.  Ninth  International  PEG 
Conference  10-12  July  1999.  Conference  Proceedings.  23-34. 

Pohjolainen,  S.,  Ruokamo,  H.  (1999)  Distance  Learning  In  Multimedia  Networks  Project  - Experiences  and  Results. 
In  B.  Collis  & R.  Olivier  (Eds.)  ED-MEDIA  1999.  World  Conference  on  Educational  Multimedia,  Hypermedia  & 
Telecommunications.  Proceedings  of  ED-MEDIA  99.  World  Conference  on  Educational  Multimedia,  Hypermedia  & 
Telecommunications.  Seattle,  Washington,  USA;  June  19-24,  1999.  922-927. 

Pythagoras  Learning  Environment,  (online),  http://matriisi.ee.tut.fi/pythagoras 

Ruokamo,  H.,  Pohjolainen,  S.  (1999)  Distance  Learning  In  Multimedia  Networks  Project:  Main  Results.  Intelligent 
Computer  and  Communications  Technology.  Teaching  &.  Learning  for  the  list  Century.  PEG99.  Ninth  International 
PEG  Conference  10-12  July  1999.  Conference  Proceedings.  153-162.. 


595 


Towards  a Design  Methodology  for  Distributed  Virtual  Laboratories 


Charles  Levert  and  Samuel  Pierre 

Mobile  Computing  and  Networking  Research  Laboratory  (LARIM) 
Department  of  Electrical  and  Computer  Engineering 
Ecole  Polytechnique  C.P.  6079,  Succ.  Centre-ville 
Montreal,  Quebec,  CANADA  H3C  3A7 
Email:  <charles@comm.polymtlxa>,  <Samuel.Pierre@larim.polymtl.ca> 


Abstract:  In  this  paper,  we  lay  down  the  foundations  to  produce  a framework  for  the  design  of  distributed 

collaborative  environments,  such  as  virtual  laboratories.  We  first  propose  a design  methodology  based  on  a 
generic  model  of  virtual  laboratory  and  a telecommunications  platform.  This  methodology  integrates  six  steps 
corresponding  to  the  same  number  of  models  to  be  generated.  The  generic  model  is  founded  on  a hierarchy 
of  concepts  for  which  the  terminal  nodes  collectively  and  formally  define  the  notion  of  virtual  laboratory.  The 
telecommunications  platform  supports  a series  of  specific  virtual  laboratories  and  provides  users  with  transparent 
access  to  the  entire  environment,  in  spite  of  possible  heterogeneity  of  the  access  networks.  It  accounts  for  such 
issues  as  security,  interoperability,  and  quality  of  service. 


Introduction 

For  several  years,  education  levels  have  been  increasing  in  our  society.  A higher  proportion  of  the  population  also 
reaches  the  level  of  post-secondary  education.  There  is  also  an  increase  in  the  demand  for  on-going  training,  caused 
in  part  by  the  many  technological  breakthroughs  of  recent  years.  However,  these  same  technologies,  those  related  to 
computing  and  telecommunications  in  particular,  lower  some  training  related  constraints  such  as  physical  presence, 
fixed  schedule,  etc.  (Collis  1999)  Fully  virtual  learning  institutions  are  starting  to  appear.  (Finley  1999) 


Problem  Definition  and  Basic  Concepts 

Instructional  content  delivery  in  electronic  format  (Azuma  1 999)  as  well  as  distant  courses  (Harris  1 999,  Takefuji  et  al. 
1999)  are  applications  of  these  technologies  that  are  increasingly  common.  They  are  already  subject  to  systematization 
effort.  Design  and  implementation  of  on-line  laboratory  activities,  however,  are  only  starting  to  benefit  from  such  an 
effort  which  involves  information  technologies.  Such  an  enterprise  requires  a high  knowledge  level  in  these  technolo- 
gies as  well  as  in  cognitive  sciences.  As  such,  it  is  only  pursued  by  a few  privileged  people,  more  as  a craft  than  a 
science.  Thus,  there  is  a very  present  need  for  a well  defined  methodological  approach  to  the  design  of  distributed 
learning  environment  geared  toward  performing  laboratory  activities.  These  activities  are  a necessary  ingredient  in 
the  pursuit  of  knowledge.  We  label  as  distributed  virtual  laboratories  the  subset  of  learning  environments  that  enables 
such  activities. 

Several  factors  contribute  to  the  use  of  virtual  laboratories:  lack  of  physical  space,  cost  of  real  equipment,  distance  in 
terms  of  cost  and  time  of  travel,  synchronous  availability  of  learners  and  teachers,  dangerous  situations,  etc.  These 
factors  are  an  incentive  to  share  or  simulate  real  laboratories  and  their  resources  (equipment,  space,  etc.). 

Distributed  virtual  laboratories  design  constitutes  a multidisciplinary  problem  which  involves  computer  and  informa- 
tion science,  telecommunications,  and  cognitive  sciences  including  instructional  design.  Indeed,  an  engineer,  even 
with  a solid  knowledge  base  in  computers  and  telecommunications,  can  not  tackle  this  problem  by  himself/herself 
since  he/she  does  not  necessarily  possess  the  cognitive  sciences  aptitudes  needed  for  the  instructional  design  of  these 
learning  environments.  On  the  other  hand,  a simple  pedagogy  expert  is  not  necessarily  able  to  fully  appreciate  what  is 
possible  from  a technological  standpoint.  This  is  exacerbated  by  the  rapid  advancement  of  the  state  of  the  art  in  com- 
puter science  and  telecommunications.  Nor  is  he/she  able  to  develop  by  himself/herself  the  computing  environment 
under  which  the  experiments  and  evaluations  of  his/her  interest  can  be  performed. 
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A similar  line  of  thought  can  be  applied  to  the  specialists  of  a given  discipline  which  is  targeted  by  a specific  lab- 
oratory, and  to  the  designers  of  an  environment  enabling  the  creation  of  such  a laboratory.  Our  role  is  that  of  the 
telecommunications  and  computer  engineer.  As  such,  we  are  primarily  interested  in  the  computer  network  distribution 
of  the  objects  that  constitute  these  environments,  and  not  as  much  by  the  instructional  aspects  of  the  design  of  these 
environments.  We  tackle  such  problems  as  distance  measurements  and  heterogeneous  networks  adaptation,  in  order  to 
give  other  specialists  access  to  a methodology,  models,  and  computing  tools.  These  enable  the  creation,  exploitation, 
and  evaluation  of  distributed  virtual  laboratories  in  a given  respective  disciplinary  field. 


State  of  the  Art 

At  this  point  in  time,  we  can  already  witness  a proliferation  of  virtual  laboratories,  at  least  from  what  can  be  seen  on 
the  accessible  part  of  the  Internet.  The  real  count  of  such  laboratories  must  be  even  higher  since  it  includes  companies 
and  militaries. 

Some  of  these  laboratories  include  Java  applets  aiming  to  reproduce  phenomena  that  are  of  interest  to  students  of 
the  targeted  discipline.  Others  aim  to  reproduce  the  very  sensation  of  physical  presence  in  an  equivalent  and  real 
environment.  Others  are  implemented  with  3D  rendering  technologies  such  as  the  Virtual  Reality  Modeling  Language 
(VRML,  which  is  open)  and  Shockwave  (which  is  proprietary).  Others  still  enable  distant  access  to  real  physical 
instruments. 

The  LVEST  project,  to  which  we  have  contributed,  uses  an  approach  which  separates  these  concepts:  the  generic 
laboratory  and  the  telecommunications  platform.  Each  laboratory  that  uses  this  approach  includes  a specific  part 
and  a generic  part.  Access  to  the  telecommunications  functionalities  is  provided  through  the  telecommunications 
platform  (Kassouf  et  al.  1999,  Pierre  & Kassouf  2000),  which  is  composed  of  three  layers: 

• the  tools  and  functionalities  adaptation  layer; 

• the  base  tools  and  functionalities  layer; 

• the  network  adaptation  layer. 

One  of  the  objectives  of  this  platform  is  to  abstract  the  distribution  and  telecommunications  aspects  related  to  imple- 
mentation details.  This  is  done  in  order  to  simplify  the  development  task  for  a virtual  laboratory  from  the  point  of 
view  of  anyone  that  is  not  a specialist.  The  project  itself  involves  a number  of  specialists  from  a host  of  disciplines. 

Development  of  the  generic  laboratory  and  telecommunications  platform  models  in  done  using  an  inductive  process. 
This  means  that  the  initial  content  of  these  models  reflects  the  needs  of  this  two  first  laboratories  to  have  been  im- 
plemented using  this  process  (in  Physics  and  Electrical  Engineering).  Each  new  laboratory  both  inherits  from  these 
models  and  contributes  to  their  enrichment. 

With  only  two  laboratories,  the  effective  definitions  of  “generic”  and  “specific”  were  rather  trivial.  They  basically 
amounted  to  the  idea  of  commonality.  However,  now  that  this  project  includes  a third  laboratory  in  computer  engi- 
neering and  that  many  others  are  planned  in  several  domains  of  several  disciplines,  care  must  be  taken  to  properly 
generalize  these  notions.  If  this  was  not  done,  the  development  of  a new  laboratory  from  a generic  model  would 
demand  more  effort  rather  than  reduce  it.  This  would  be  attributable  to  the  necessary  evaluation  of  how  appropriate 
the  use  of  each  offered  generic  functionality  would  be.  Conversely,  a most  restrictive  definition  of  the  word  “generic”, 
aiming  to  reduce  the  generic  functionalities  count,  would  exclude  to  many  functionalities  that  are  common  to  only  a 
few  laboratories.  Such  a generalization  has  not  been  tackled  before. 


Design  Methodology 

The  basic  principle  of  the  design  methodology  consists  of  defining  a set  of  attributes,  tools,  and  functionalities  shared 
by  a variety  of  virtual  laboratories  dedicated  to  a variety  of  science  and  engineering  disciplines.  The  distributed 
processing  and  access  aspects  of  implementing  the  telecommunications  platform  imply  a number  of  challenges  related 
to  security,  interoperability,  and  quality  of  service.  We  now  expose  them. 
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Security 


Indeed,  in  an  open  environment,  users  should  not  only  benefit  from  the  possibility  to  exchange  information  between 
one  another,  but  also  have  the  possibility  to  transfer  and  download  content  which  is  subject  to  a controlled  usage.  In 
such  a context,  securing  those  transfers  is  of  a primordial  importance  and  this  should  be  reflected  in  the  very  process 
used  to  design  virtual  laboratories. 


Interoperability 

Likewise,  there  is  a matter  of  interoperability  between  heterogeneous  networks  and  communication  supports  that  can 
be  used  to  access  the  laboratory  environment.  It  must  be  taken  into  account  as  well  in  the  virtual  laboratories  design 
process  and  methodology.  In  our  prototypes,  we  have  investigated  the  use  of  CORBA  and  the  capacity  to  support 
interoperability  between  Internet  protocols  and  xDSL  technology.  We  conclude  that,  in  order  to  address  this  specific 
problem,  a proposed  design  methodology  must  include  one  step  dedicated  to  networks  and  supports  adaptation  at  the 
level  of  the  telecommunications  platform. 


Quality  of  Service 

As  for  quality  of  service,  the  main  concern  is  the  delay  to  access,  transmit,  and  display  the  (possibly  multimedia) 
information  that  is  exchanged  in  this  type  of  environment  (Campbell  et  al.  1994,  Ren  et  al.  1999,  Schwartz  & 
Beaumont  1994).  This  aspect  deserves  particular  attention  in  the  context  of  virtual  laboratories  where  interactive 
simulation,  telemesure,  and  tele-experimentation  are  among  the  considered  approaches  and  methods. 

The  three  challenges  we  just  exposed  are  accounted  for  in  a distribution  model.  Its  implementation  constitutes  one  of 
the  later  steps  in  a design  methodology  for  distributed  virtual  laboratories.  This  methodology,  as  well  as  its  associated 
models,  must  be  supported  by  software  tools  which  reduce  the  design  and  implementation  effort  asked  of  the  discipline 
or  domain  specialists.  Some  of  these  tools  have  already  been  developed  as  part  of  the  LVEST  project  (Tele-universite 
1999).  Other  such  tools  are  now  in  development  and  will  contribute  to  further  enrich  our  telecommunications  platform. 


Generic  Model 

A generic  virtual  laboratory  model  should  be  general  enough  to  cover  any  virtual  laboratory. 

The  set  of  all  sciences  and  engineering  disciplines  can  be  considered  finite.  It  forms  a partition.  A single  discipline  is 
composed  of  several  domains,  each  of  which  is  characterized  by  an  experimentation  environment  that  includes  tools 
and  functionalities  used  to  define  and  implement  laboratory  sessions. 

Some  of  these  tools  and  functionalities  are  domain-specific,  while  others  can  be  used  universally,  i.e.,  in  several 
domains,  even  disciplines.  The  latters  are  called  basic  tools  and  functionalities. 

Each  domain  enables  experiments  of  various  types  to  be  performed.  For  a given  domain,  the  set  of  experiments  that 
can  be  performed  can  be  considered  finite.  The  set  of  all  experiments  that  can  be  performed  for  a given  discipline 
forms  a partition. 

Each  laboratory  experiment  enables  the  practice  of  one  or  more  theory  elements  from  a given  domain.  These  theory 
elements  are  prerequisite  to  performing  the  laboratory  session.  Therefore,  a laboratory  session  can  be  defined  as  a set 
of  experiments  of  various  types,  performed  in  a given  environment,  linked  to  a specific  domain  of  a specific  discipline, 
and  forming  a coherent  ensemble. 

For  a given  discipline,  a virtual  laboratory  is  defined  as  a distributed  learning  environment  which  enables  laboratory 
experiments  to  be  performed.  Those  experiments  can  eventually  be  grouped  to  form  a session.  They  can  be  related  to 
one  or  more  domains  of  the  discipline.  In  fact,  we  can  define  a laboratory  as  a triplet  consisting  of 
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1 . the  discipline  specification, 

2.  the  set  of  all  experiments  that  can  be  performed  for  this  discipline,  and 

3.  a family  of  transformations  related  to  the  domains  of  the  given  discipline. 

This  family  of  transformations  can  be  seen  as  a didactic  engineering  process  which  transforms  knowledge  elements  of 
a given  domain  into  a series  of  experiments. 

To  recapitulate,  the  generic  model  is  founded  on  a hierarchy  of  concepts  which  terminal  node  is  the  notion  of  vir- 
tual laboratory.  The  basic  concepts  included  in  this  hierarchy  are:  the  discipline,  the  domain,  the  environment,  the 
laboratory  session,  the  type  of  experiment,  the  experiment,  and  the  virtual  laboratory. 


Design  Methodology  Elements  and  their  Respective  Models 

It  follows  that,  in  order  to  design  a virtual  laboratory  in  any  sciences  or  engineering  discipline,  each  of  the  basic  con- 
cepts of  the  generic  model  must  itself  be  modeled  (Levert  & Pierre  2000).  They  include  the  discipline,  the  domain,  the 
environment,  the  laboratory  session,  the  experiment,  to  which  we  add  the  concept  of  distribution  (access,  interaction, 
interface,  interoperability,  and  security). 


Discipline 

Disciplines  are  located  at  the  top  of  our  basic  concepts  hierarchy  since  we  do  not  intend  to  reassess  the  different 
discipline  constellations  (called  knowledge  fields  by  specialists)  in  use.  Examples  of  these  knowledge  fields  include 
those  formed  by  an  university’s  faculties,  the  many  contests  of  a grant  body,  etc. 

The  discipline  modeling  should  produce  a list  of  domains  that  is  specific  to  it.  In  practice,  the  discipline’s  evolution 
makes  this  list  variable  in  time  and  possibly  according  to  the  place  where  it  is  considered.  Thus,  we  have  to  explore 
how  it  will  be  possible  to  evolve  this  list  as  time  passes,  either  by  simple  insertion  and  deletion,  or  by  repartitioning 
the  discipline. 

Computer  engineering  is  an  example  of  discipline.  It  includes,  among  others,  the  following  domains:  algorithmic 
complexity,  design  and  analysis  of  algorithms,  database  management,  knowledge  management,  networking,  infog- 
raphy,  multimedia,  and  software  engineering  (NSERC  1998,  NSERC  1997).  We  can  readily  see  that  this  list  is  not 
exhaustive;  this  shows  the  difficulty  of  producing  such  an  exhaustive  list,  but  it  also  shows  the  possibility  of  working 
with  an  incomplete  list  if  it  suffices  to  fill  all  the  needs. 


Domain 

The  domain  modeling  mainly  includes  a knowledge  representation  (Paquette  et  al.  1996).  In  addition  to  the  possibility 
of  laying  them  out  on  a learning  curriculum,  these  elements  can  be  classified  using,  for  example,  the  cognitive  levels 
of  the  Bloom  taxonomy  (Michalski  & Stepp  1983). 

The  elements  from  this  possibly  structured  knowledge  set  should  lend  themselves  to  a mapping  with  laboratory  ex- 
periments. It  does  not  necessarily  follow  that  the  most  appropriate  classification  and  structure  for  these  knowledge 
elements  will  correspond  to  that  used  for  the  laboratory  experiments  (whether  it  is  based  on  a laboratory  session  group- 
ing or  a laboratory  type  grouping).  However,  there  can  be  a gain  to  forcing  such  a mapping.  This  is  an  open  question. 
Computer  networking  is  an  example  of  domain,  in  the  context  of  the  computer  engineering  discipline. 


Environment 

The  environment  model  should  account  for  all  elements  that  can  constitute  an  environment,  and  also  for  all  relations 
that  exist  between  those  elements.  This  includes  material  resources  (computers,  networks,  etc.),  the  place  where 
they  are  located,  as  well  as  the  human  beings  involved  in  the  learning  process  (learners  and  teachers).  The  possible 
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relations  between  humans  are  represented  by  the  availability  of  communication  and  collaboration  tools,  as  well  as 
by  the  formation  of  teams  and  the  leamers/teachers  ratio.  All  others  ratios  between  humans  and  material  resources 
should  be  modeled.  Five  spaces  can  be  defined  and  all  elements  should  fall  in  exactly  one  of  them:  production,  help, 
collaboration,  information,  and  navigation  spaces. 

The  environment  is  the  concept  which  holds  the  tools  and  functionalities.  Thus,  it  is  at  this  conceptual  level  that  the 
generalization  of  the  genericity  and  specificity  should  be  applied.  This  should  result  in  a hierarchical  classification  of 
these  tools  and  functionalities.  Whether  all  others  elements  and  relations  found  in  an  environment  should  be  included 
in  this  framework  is  an  open  question.  This  would  allow  an  advantageous  simplification  of  the  environment  model. 

Laboratory  Sessions 

A laboratory  session  is  composed  of  a subset  of  experiments  from  the  target  domain.  The  laboratory  session  modeling 
describes  the  learning  scenarios  which  are  used,  categorizes  and  plans  didactical  instruments  and  identifies  where  an 
analysis  of  interactions  could  take  place  (Paquette  1998). 


The  experience  model  should  specify  the  type  of  experimentation  which  is  performed.  Each  experiment  corresponds 
to  some  knowledge  and  aptitudes  which  should  be  transmitted,  as  described  in  domain  modeling.  An  experiment  takes 
place  in  a learning  curriculum. 

It  can  sometimes  be  advantageous  to  be  able  to  record  or  reproduce  the  progress  of  an  experiment,  whether  for  learn- 
ing evaluation  or  other  purposes.  The  model  must  account  for  this  since  it  will  translate  in  resources  needed  to  its 
implementation.  The  nature,  moment,  and  requirements  (e.g.,  temporal  requirements)  of  the  various  data  processing 
stages  must  be  taken  into  account  for  has  a strong  influence  on  the  computing  distribution  model  which  is  used. 


The  distribution  model  should  first  represent  the  different  computing  and  networking  components  which  are  available. 
In  particular,  all  aspects  having  to  do  with  security,  interoperability,  and  quality  of  service  must  be  represented. 

These  components  can  be  used  in  different  possible  configurations.  From  this  information,  an  algorithm  is  devised  to 
produce  an  optimal  scenario  for  the  distribution  and  interconnection  of  objects. 

The  very  ideal  of  optimization  implies  modeling  the  various  costs  involved  by  the  operations  performed  in  a given 
configuration.  They  include:  the  cost  of  processing  power  the  cost  of  code  transmission  (throughput),  the  cost  of 
data  copying,  the  cost  of  data  transmission  (throughput),  the  cost  of  configuration  computation,  the  cost  of  dynamic 
configuration  changes,  the  cost  of  data  marshalling  and  other  transformations  (such  as  those  involving  security),  the 
cost  of  data  storage. 

In  practice,  identifying  generic  characteristics  among  several  laboratories  should  help  reduce  the  complexity  of  the 
mathematical  and  algorithmic  problems  created  by  this  modeling. 

Concluding  Remarks 

In  order  to  produce  tools  that  can  help  develop  distributed  learning  environments,  we  must  first  be  able  to  determine 
the  main  axes  along  which  their  variants  can  be  distinguished.  More  precisely,  we  must  identify  those  characteristics 
that  are  pertinent  to  that  goal  and  drop  those  that  are  not.  Putting  this  in  practice,  however,  is  not  obvious.  Virtual  lab- 
oratories and  other  learning  environments  developed  so  far  form  a rather  disparate  set  from  which  it  would  be  difficult 
to  extract  the  principal  characteristics.  Rather  than  designing  an  unflexible  and  definite  characterization  from  the  get 
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go  for  everything  that  has  been  or  will  be  developed,  we  plan  to  apply  the  conceptual  clustering  technique  (Michalski 
& Stepp  1983)  to  existing  environments  and  to  those  whose  existence  can  be  predicted. 

This  clustering  technique  can  be  applied  to  all  levels  (or  elements)  of  the  design  methodology:  from  domain  to  in- 
dividual experiments.  It  is  used  to  determine  if  it  is  cost  effective  to  make  a tool  generic  or  keep  it  specific.  To  use 
this  technique,  we  must  identify  quality  criteria  for  the  clusters.  These  are  used  to  discriminate  against  several  possi- 
ble clusters.  The  actual  concepts  that  are  used  by  one  clustering  reveal  themselves  as  the  technique  is  applied.  The 
concepts  and  tools  described  so  far  are  applied  as  they  are  developed  to  a virtual  laboratory  prototype  in  the  computer 
engineering  discipline. 
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Abstract;  The  improvement  of  higher  order  thinking  skills  continues  to  be  an  important 
issue  in  education.  Over  the  past  10  years,  two  meta-analyses  on  the  issue  of  computer 
programming  and  problem  solving  ability  have  been  done,  yet,  there  is  no  meta-analysis 
conducted  to  clarify  the  issue  of  programming  and  problem-solving  abilities  since  1989. 
The  purpose  of  this  study  is  to  continually  investigate  the  issue  of  programming  and 
problem  solving  ability.  More  specifically,  this  meta-analysis  attempted  to  update  the 
research  findings  from  1989  to  1999  in  the  area  of  computer  programming  and  problem 
solving.  The  results  from  this  study  suggest  that  the  outcomes  of  learning  a computer 
language  extend  beyond  the  content  of  that  specific  computer  language.  Students  are 
able  to  acquire  some  cognitive  skills  such  as  reasoning  skills,  logical  thinking  and 
planning  skills,  and  general  problem  solving  skills  through  computer  programming 
activities. 


Theoretical  framework  and  purpose  of  study 

For  well  over  a quarter  of  a century  there  have  been  urgent  calls  for  improved  instruction  in  problem 
solving.  There  is  ample  evidence  that  the  problem-solving  skills  of  the  nation’s  children  need  significant 
strengthening,  yet  existing  approaches  to  develop  and  improve  these  skills  do  not  seems  to  have  worked  (Casey, 
1997).  The  link  between  the  learning  of  programming  language  and  the  development  of  problem  solving  skills 
has  been  one  of  the  most  intensely  argued  topics  in  educational  computing  (Reed  & Burton,  1988).  The  research 
findings,  however,  are  conflicting.  Some  studies  report  significant  gains  in  various  aspects  of  cognitive  abilities 
following  programming  experiences  (Choi  & Repman,  1993;  Liu,  1997;  Reeder,  1994).  Other  studies  have 
indicated  that  there  is  little,  if  any,  transfer  of  learning  from  the  programming  situation  to  similar  non- 
programming tasks  (Lai,  1993;  Palumbo,  1993) 

Over  the  past  10  years,  two  meta-analyses  on  the  issue  of  computer  programming  and  problem  solving 
ability  have  been  done.  Liao  and  Bright  (1991)  conducted  a meta-analysis  on  the  effects  of  computer 
programming  on  cognitive  outcomes.  Their  analysis  covered  the  research  published  between  1969  and  1989. 
Sixty-five  studies  were  included  in  their  analysis,  and  the  results  indicated  that  the  percentiles  on  students’ 
cognitive  performance  were  66  for  the  programming  group  and  50  for  the  non-programming  group;  computer 
programming  has  slightly  positive  effects  on  student  cognitive  outcomes.  Another  meta-analysis  conducted  by 
Lee  (1990)  investigated  the  effectiveness  of  computer-assisted  instruction  and  computer  programming  in  the  area 
of  problem  solving  skills,  math,  and  attitudes  toward  mathematics  instruction.  Seventy-two  studies  were 
included  in  his  analysis,  and  the  results  showed  that  Logo  was  found  to  be  more  effective  than  BASIC  in 
developing  problem  solving  skills. 

The  improvement  of  higher  order  thinking  skills  continues  to  be  an  important  issue  in  education,  yet,  there 
is  no  meta-analysis  conducted  to  clarify  the  issue  of  programming  and  problem-solving  abilities  since  then. 

The  purpose  of  this  study  is  to  continually  investigate  the  issue  of  programming  and  problem  solving 
ability.  More  specifically,  this  meta-analysis  attempted  to  update  the  research  findings  from  1989  to  1999  in  the 
area  of  computer  programming  and  problem  solving.  The  results  from  this  meta-analysis  will  help  provide 
clearer  conclusions. 


Procedures 

The  research  method  used  in  this  study  is  the  meta-analytic  approach  which  was  similar  to  that  described 
by  Glass,  McGaw,  & Smith  (1981).  Their  approach  requires  a reviewer  to  (a)  locate  studies  through  objective 
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and  replicable  searches;  (b)  code  the  studies  for  salient  features;  (c)  describe  outcomes  on  a common  scale;  and 
(d)  use  statistical  methods  to  relate  study  features  to  outcomes  (Kulik,  Kulik,  & Bangert-Drowns,  1985). 

The  purpose  of  this  study  was  to  synthesize  and  analyze  the  research  on  effects  of  two  instructional 
approaches.  It  is  important  to  define  these  approaches  to  provide  for  proper  selection  of  appropriate  studies. 

computer  programming  instruction  - classes  using  computer  languages,  such  as  Logo,  BASIC,  Pascal  as 
an  instructional  tool  to  teach  students. 

non-computer-programming  instruction  — classes  not  using  any  computer  programming  as  an  instructional 
tool  to  teach  students. 

Data  Sources 

The  studies  considered  for  use  in  this  meta-analysis  came  from  three  major  sources  and  published  between 
1989  and  1999.  One  large  group  of  studies  came  from  computer  searches  of  Education  Resources  Information 
Center  (ERIC).  A second  group  of  studies  came  from  Comprehensive  Dissertation  Abstracts.  A third  group  of 
studies  was  retrieved  by  branching  from  bibliographies  in  the  documents  located  through  review  and  computer 
searches.  Twenty-two  studies  were  located  through  these  search  procedures;  3 studies  came  from  ERIC,  and  1 9 
studies  were  retrieved  from  published  Journals. 

Several  criteria  were  established  for  inclusion  of  studies  in  the  present  analysis. 

1.  Studies  had  to  assess  the  relationship  between  computer  programming  and  cognitive  skills  such  as 

planning  skills,  thinking  skills,  reasoning  skills,  and  metacognitive  skills. 

2.  Studies  had  to  take  place  in  actual  classrooms.  These  was  no  restriction  on  grade  level. 

3.  Studies  had  to  provide  quantitative  results  from  both  computer-programming  and  non-computer- 

programming classes. 

4.  Studies  had  to  be  retrievable  from  university  or  college  libraries  by  interlibrary  loan  or 

from  ERIC,  Dissertation  Abstracts  International,  or  University  Microfiche  International. 

There  were  also  several  criteria  for  eliminating  studies  or  reports  cited  by  other  reviews:  (a)  studies  did  not 
report  sufficient  quantitative  data  in  order  to  estimate  Effect  Sizes;  (b)  studies  reported  only  correlation 
coefficients  — r value  or  Chi-square  value;  (c)  studies  could  not  be  obtained  through  interlibraiy  loans  or  from 
standard  clearinghouses. 

Outcome  Measures 

The  instructional  outcome  measured  most  often  in  the  22  studies  was  student  learning,  as  indicated  on 
standard  cognitive-skill  tests  at  the  end  of  the  program  of  instruction.  For  statistical  analysis,  outcomes  from  a 
variety  of  different  studies  with  a variety  of  different  instruments  had  to  be  expressed  on  a common  scale.  The 
transformation  used  for  this  purpose  was  the  one  recommended  by  Glass  et  al.  (1981);  each  outcome  is  coded  as 
an  Effect  Size  (ES),  defined  as  the  difference  between  the  mean  scores  of  two  groups  divided  by  the  standard 
deviation  of  the  control  group.  For  those  studies  that  did  not  report  means  and  standard  deviations,  F values,  t 
values,  or  proportion  values  were  used  to  estimate  the  ES. 

In  most  cases,  the  application  of  the  formula  given  by  Glass  and  his  colleagues  was  quite  straightforward. 
But  in  some  cases,  when  more  than  one  value  was  available  for  use  in  the  formula  of  ES,  the  value,  which 
measured  outcomes  most  correctly,  was  selected. 

In  addition,  when  studies  used  more  than  one  type  of  control  group  (e.g.,  programming  vs.  CAI  and 
programming  vs.  non-programming),  only  the  comparison  of  programming  and  non-programming  was  retained. 
Also,  when  studies  contained  reports  of  experiments  where  treatments  were  compared  with  each  other  and  no 
non-computer  control  group  was  involved  (e.g.,  programming  vs.  CAI),  Glass  et  al.  (1981.  p.l24)  proposed  a 
process  to  split  the  ES  into  two  components  to  estimate  comparisons  of  each  treatment  to  a hypothetical  control 
group  (e.g.,  programming  vs.  non-programming  and  CAI  vs.  non-CAI).  In  other  cases,  several  subscales  and 
subgroups  were  used  in  measuring  a single  outcome  (e.g.,  those  that  reported  separate  data  by  language  or 
grade).  In  such  cases,  each  comparison  was  weighted  in  inverse  proportion  to  the  number  of  comparisons  within 
the  study  (i.e.,  1/n,  where  n = number  of  comparisons  in  the  study)  so  that  the  overweighting  of  ES  of  a study 
could  be  avoided  (see,  for  example,  Waxman,  Wang,  Anderson,  & Walberg,  1985,  p.  230). 

Variables  Studied 

Seventeen  variables  (i.e.,  sample  size,  type  of  publication,  year  of  publication,  grade  level, 
instrumentation,  instructor  bias,  reliability  of  measure,  selection  bias,  statistical  power,  statistic,  type  of  research 
design,  language  studied,  instructional  approach,  duration  of  treatment,  type  of  instructional  for  treatment. 
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implementation  of  innovation,  and  type  of  outcome)  were  selected  for  coding  each  study  in  the  present  synthesis. 
Each  variable  was  employed  as  a factor  in  an  analysis  of  variance  (ANOVA)  to  investigate  whether  there  are 
significant  differences  within  each  variable  on  the  effect  size. 

Results 

The  number  of  comparisons,  target  population,  and  the  study-weighted  ESs  are  reported  in  Table  1 . Of  the 
22  studies  included  in  the  present  synthesis,  all  of  the  study-weighted  effect  sizes  were  positive  and  favored  the 
programming  group.  The  range  of  the  study- weighted  effect  sizes  was  from  0.04  to  3.7.  The  overall  grand  mean 
for  all  22  study-weighted  ESs  was  0.76.  When  this  mean  ES  was  converted  to  percentiles,  the  percentiles  on 
students'  cognitive  performance  were  66  for  the  programming  group  and  50  for  the  non-programming  group. 
The  standard  deviation  of  0.78  reflects  the  great  variability  of  effect  sizes  across  studies. 


Author(s) 

Year 

No  of  Comparisons 

ES 

Bernardo 

1994 

6 

0.288 

Choi 

1993 

2 

0.374 

Clements 

1990 

8 

0.457 

Dalton 

1991 

2 

0.539 

Jang 

1992 

1 

0.651 

Lai 

1993 

1 

0.329 

Leher 

1989 

3 

1.265 

Liu,  M" 

1997 

3 

0.747 

Liu,M 

1997 

1 

0.423 

Mundy- 

1989 

16 

0.204 

Nastasi 

1990 

1 

1.382 

Norris 

1992 

2 

0.38 

Palumbo 

1993 

2 

1.156 

Palumbo 

1991 

3 

1.203 

Reed 

1992 

2 

0.35 

Reeder,  L 

1994 

1 

0.877 

Rucinski 

1991 

1 

0.055 

Swan 

1993 

10 

0.04 

Swan 

1991 

10 

0.081 

Vasu, E 

1997 

2 

1.169 

Verzoni 

1995 

3 

3.699 

Wilson 

1991 

1 

1.035 

Overall  grand  mean  0.759 

Overall  grand  median 
Overall  grand  SD 

0.498 

0.781 

Note.  Total  N of  studies  = 22.  Total  N of  comparisons  = 86 


^Two  Separated  studies  reported  in  one  article. 

Table  1 Number  of  Comparisons  and  Study-weighted  Effect  Sizes 


Among  the  86  effect  sizes  included  in  the  present  synthesis,  74  or  86%  were  positive  and  favored  the 
programming  group,  while  12  or  14%  were  negative  and  favored  the  non-programming  group.  The  range  of  the 
effect  sizes  was  from  -0. 1 7 to  1 .45 

Table  2 lists  the  F values  for  the  17  variables  for  all  study-weighted  ESs  in  the  study.  For  ANOVAs,  two 
variables  showed  statistically  significant  impact.  For  each  of  these  variables,  a post  hoc  (FisherD  Protected 
LSD)  test  was  performed.  Descriptive  statistics  for  these  variables  are  presented  in  Table  3. 


Table  4 


Variables 


df  F p 
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Study  Characteristics 

Type  of  Publication 

1,20 

4.294 

0.051 

Year  of  Publication 

3,  18 

1.727 

0.197 

Grade  Level 

4,  18 

0.974 

0.446 

Methodology  Characteristics 

Instructor  Bias 

2,  19 

0.444 

0.648 

Instrumentation 

1,22 

0.187 

0.67 

Pretest  Equivalency 

4,  17 

0.038 

0.997 

Reliability  of  Measure 

2,  19 

0.798 

0.465 

Sample  Size 

2,  19 

1.263 

0.306 

Selection  Bias 

2,  19 

0.9 

0.914 

Statistical  Power 

1,20 

0.03 

0.986 

Statistics 

1,20 

0.032 

0.861 

Type  of  Research  Design 

3,  18 

3.393 

0.041* 

Program  Characteristics 

Language  studied 

4,21 

0.722 

0.587 

Instructional  Approach 

2,  20 

0.924 

0.413 

Length  of  Experiment 

5,  19 

2.288 

0.087 

Type  of  Instruction  for  Treatment 

3,  18 

1.923 

0.162 

Duration  of  Treatment 

3,  18 

1.905 

0.165 

Outcome  Characteristics 

Type  of  outcome 

5,20 

5.574 

0.002* 

* p<.05 

Table  2.  Results  of  ANOVAs  for  Coded  Variables 

Variables 

N 

% 

ES 

SD 

Study  Characteristics 

Type  of  Publication 

Journal  article 

19 

86.4 

0.632 

0.452 

ERIC 

3 

13.6 

1.567 

1.853 

Year  of  Publication 

1989-  1991 

9 

40.9 

0.691 

0.534 

1992  - 1993 

7 

31.8 

0.469 

0.351 

1994-  1995 

3 

13.6 

1.621 

1.823 

1996-  1997 

3 

13.6 

0.780 

0.374 

Grade  LeveP 

K-3th 

2 

8.7 

0.667 

0.297 

4th  - 6th 

8 

34.8 

0.819 

0.542 

7th  - 8“' 

3 

13.0 

1.436 

1.962 

High  school 

3 

13.0 

0.510 

0.613 

College 

7 

30.4 

0.426 

0.224 

Table  3 Cont. 

Variables 

N % 

ES 

SD 

Methodological  Characteristics 


605 


Sample  Size 


10-40 

9 

40.9 

0.822 

0.476 

40-80 

6 

27.3 

1.076 

1.322 

Over  80 

7 

31.8 

0.408 

0.355 

Instructor  Bias 

Same 

12 

55.5 

0.901 

0.973 

Different 

7 

31.8 

0.633 

0.418 

Unspecified 

3 

12.7 

0.487 

0.624 

Instrumentation" 

Local 

11 

45.8 

0.778 

1.071 

Standard 

13 

54.2 

0.638 

0.428 

Reliability  of  Measure 

Actual  reliability  figure 

10 

45.5 

0.729 

0.367 

Adequate  indication 

5 

22.7 

0.447 

0.545 

Unspecified  or  inadequate 

7 

31.8 

1.025 

1.253 

Selection  Bias 

Adequate  indication 

12 

54.5 

0.738 

0.463 

Probably  threat 

7 

31.8 

0.853 

1.272 

Unspecified  or  inadequate 

3 

13.6 

0.625 

0.557 

Statistical  Power 

Adequately  minimized 

19 

86.4 

0.758 

0.832 

Probably  threat 

3 

13.6 

0.767 

0.427 

Statistics 

Mean  & Standard  deviation 

18 

81.8 

0.774 

0.857 

F - value 

4 

18.2 

0.695 

0.320 

Type  of  Research  Design 

One  group  repeated  measure 

5 

22.7 

0.446 

0.172 

Pretest-posttest  control  group 

6 

27.3 

0.870 

0.509 

Nonequivalent  control  group 

9 

40.9 

0.558 

0.472 

Posttest-only  control  group 

2 

9.1 

2.119 

2.234 

Program  Characteristics 
Language  Studied" 

Logo 

13 

50.0 

0.898 

0.985 

Basic 

6 

23.1 

0.439 

0.394 

Pascal 

3 

11.5 

0.330 

0.278 

Fortran 

1 

3.8 

0.188 

0.000 

HyperCard 

3 

11.5 

0.405 

0.351 

Instructional  Approach" 

Focusing  on  cognitive 

11 

47.8 

0.899 

1.027 

skills 

Not  focusing  on  cognitive 

9 

39.1 

0.686 

0.457 

Skills 

Unspecified 

3 13.1 

Table  3 Cont. 

0.213 

0.163 

Variables 

N 

% 

ES 

SD 

Type  of  Instruction  for  Treatment" 

Large  group  10  45.5  0.431  0.325 
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Small  group  (less  than  5 
persons  in  a group) 
Mixed  2 
Unspecified 

Duration  of  Treatment 
0-3  months 
4-6  months 
7-12  months 
Unspecified 

Outcome  Characteristics 
Type  of  Outcome^ 

Conceptual  transfer 
Critical  thinking 
Metacognitive  skills 
Problem  solving  skills 
Spatial  skills 
Creative  thinking 


6 

27.3 

1.311 

1.223 

9.1 

1.016 

0.517 

4 

18.2 

0.624 

0.579 

6 

27.3 

1.353 

1.244 

5 

22.7 

0.689 

0.429 

3 

13.6 

0.515 

0.203 

8 

36.4 

0.450 

0.409 

2 

2.482 

1.721 

2 

0.478 

0.458 

2 

0.393 

0.091 

17 

0.582 

0.415 

1 

0.367 

0.000 

2 

-0.133 

0.099 

Note.  “ Some  sudies  reported  more  than  one  comparison  groups. 

Table  3.  Means  and  Standard  Deviations  of  Study-weighted  ESs  for  Coded  Variables 


For  type  of  research  design,  (F(3,18)  = 3.393,  p<.05)  the  post  hoc  test  showed  that  the  mean  for  studies 
employed  posttest  control  group  design  was  significantly  higher  than  the  mean  for  studies  used  one-group 
repeated  measure,  pretest-posttest  control  group,  or  nonequivalent  control  group  designs. 

In  addition,  the  post  hoc  test  for  type  of  outcome,  (F(5,20)  = 5.574,  p<.01)  showed  that  the  mean  of  studies 
in  which  the  outcome  measured  was  conceptual  transfer  was  significantly  higher  than  studies  measured  critical 
thinking,  metacognition,  problem  solving  abilities,  spatial  thinking,  or  creative  thinking.  There  were  no 
significant  differences  among  the  other  five  groups. 

One  variable,  type  of  publication,  (F(l,20)  = 4.294,  p<.06),  was  approaching  significance.  The  result 
showed  that  the  mean  for  studies  coded  as  ERIC  document  was  greater  than  the  mean  for  studies  coded  as 
published  paper. 


Discussion 

The  results  of  this  meta-analysis  indicate  that  computer  programming  has  moderate-to-high  positive  effects 
on  student  cognitive  outcomes;  an  effect  is  said  to  be  small  when  ES  = 0.2,  medium  when  ES  = 0.5  and  large 
when  ES  = 0.8  (Cohen,  1977).  One  hundred  percent  of  positive  study-weighted  ES  values  and  88%  of  positive 
ESs  overall  confirm  the  effectiveness  of  computer-programming  instruction.  The  overall  mean  ES  of  0.76  was 
much  higher  than  Liao  and  Brights’  (1991)  meta-analysis,  in  which  the  overall  mean  ES  was  0.41.  The  great 
differences  of  ES  between  these  two  meta-analyses  possibly  because  new  programming  languages  such  as 
HyperCard  are  easier  to  learn,  or  perhaps  teachers  have  found  more  effective  approaches  using  computer 
programming  to  enhance  students’  cognitive  abilities. 

For  the  analysis  of  coded  variables,  the  greater  ES  associated  with  the  posttest  control  group  design  was 
interesting.  However,  there  were  only  2 studies  coded  as  this  design,  this  result  may  be  somewhat  unstable. 

For  type  of  outcome,  the  larger  ES  was  connected  with  conceptual  transfer,  possibly  because  there  are 
some  differences  between  conceptual  transfer  and  other  types  of  cognitive  outcomes  such  as  critical  thinking  and 
problem  solving  abilities.  Yet,  there  were  only  two  studies  measured  conceptual  transfer,  the  result  may  be 
considered  tentative. 

Conclusions 

The  results  from  this  study  suggest  that  the  outcomes  of  learning  a computer  language  extend  beyond  the 
content  of  that  specific  computer  language.  Students  are  able  to  acquire  some  cognitive  skills  such  as  reasoning 
skills,  logical  thinking  and  planning  skills,  and  general  problem  solving  skills  through  computer  programming 
activities.  Since  fostering  students'  cognitive  abilities  is  usually  part  of  our  educational  goal,  the  results  provide 
to  teachers  a mildly  effective  approach  for  teaching  students  cognitive  skills  in  the  classroom  setting.  Left 
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unanswered  is  the  question  of  whether  computer  programming  is  as  efficient,  or  any  more  efficient,  at 
developing  these  cognitive  abilities  than  other  possible  instructional  approaches  that  teachers  might  use.  Studies 
of  this  question  will  require  further  clarification  of  the  exact  nature  of  the  cognitive  abilities  most  likely  to  be 
developed  through  programming.  This  meta-analysis  points  out  only  that  improvements  in  cognitive  outcomes 
are  possible.  That  information  by  itself  is  useful. 
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Abstract:  A meta-2inalysis  was  performed  to  synthesize  gender 
differences  on  attitudes  toward  computers  for  studies  using 
Loyd  and  Gressards'  CAS.  Twenty-eight  studies  were  located 
from  three  sources,  and  their  quantitative  data  were 
transformed  into  Effect  Size  (ES)  . The  overall  grand  mean  of 
the  study-weighted  ES  for  all  28  studies  was  0.273.  The 
results  suggest  that  gender  differences  on  attitudes  toward 
computers  exist,  however,  the  differences  are  small.  Left 
unanswered  is  the  question  of  what  factors  truly  contribute  to 
the  differences.  Studies  of  this  question  will  require 
further  clarification  of  the  exact  characteristics  of  each 
type  of  computer  attitude  (i.e. , anxiety,  confidence,  liking 
and  usefulness)  cuid  their  relationships  with  gender.  This 
meta-2uia lysis  points  out  only  that  gender  differences  on 
attitudes  toward  computers  exist.  That  information  by  itself 
is  useful. 


Theoretical  framework  and  Purpose  of  Study 

Computer  use  and  skills  has  been  portrayed  in  our  society  as  more  appropriate  to  men  and  boys  than  women 
and  girls  (Whitley,  1 997).  Computer  games  and  educational  software  have  been  designed  to  appeal  more  to  boys 
than  girls  (Kiesler,  Sproull,  & Eccles,  1985).  Computer  use  in  schools  has  also  been  linked  to  traditionally 
‘masculine’  subjects  such  as  science  and  mathematics  but  not  to  traditionally  ‘feminine’  subjects  such  as  art  and 
literature  (Hawkins,  1985).  In  fact,  research  has  shown  that  males  use  computers  more  often  (Chen,  1986;  Cole, 
1997;  Shashaani,  1994;  Sutton,  1991;  Urban,  1986),  in  more  places  (Chen,  1986;  Culley,  1988;  Sanders,  1985; 
Shashaani,  1994),  and  for  more  purposes  (Chen,  1986;  Martinez,  1988;  Riggs,  1994)  than  do  females.  Furthermore, 
many  studies  also  reported  that  males  take  more  classes  in  school  in  which  computers  are  used  than  do  their  female 
counterparts  (Chen,  1987;  Cole,  1997;  Shashaani,  1994;  Sutton,  1991;  Urban,  1986);  males  use  computers  more 
frequently  in  their  recreational  time  (e.g.,  after  school  clubs,  summer  camps,  and  at-home  use),  and  stay  on 
computers  for  a longer  time  than  females  (Joiner,  1996;  Shashaani,  1994).  These  gender  differences  in  socialization 
and  computer- related  behaviors  have  led  to  the  hypothesis  that  women  and  girls  are  more  likely  to  hold  more 
negative  attitudes  toward  computers. 

Although  many  studies  have  been  conducted  to  examine  the  hypothesis  of  gender  differences  on  computer 
attitudes  (Campbell,  1988;  1989;  Chen,  1986;  Koohang,  1989;  Loyd  & Gressard,  1986;  Meier,  1988;  Wallace  & 
Sinclair,  1995;  Woodrow,  1994),  the  results  of  existing  studies  have  reported  a confusing  picture.  For  example, 
Campbell  (1990),  Chen  (1986),  Colley,  Gale,  & Harris  (1994),  Jacobson  (1991),  Loyd  & Gressard  (1986),  Wallace 
& Sinclair  (1995),  Whitley  (1996a;  1996b),  and  Woodrow  (1994)  all  reported  significantly  higher  positive  computer 
attitudes  for  males  than  females,  while  Campbell  (1986;  1989),  Cantrell  (1995),  Chu  & Spires  (1991),  Francis 
(1994),  Koohang  (1987;1989),  Loyd  & Gressard  (1985),  Loyd,  Loyd,  & Gressard  (1987),  Mertens  & Wong  (1988), 
Pope-Davis  & Twing,  and  Robertson,  Calder,  & Jones  (1995)  have  indicated  that  there  is  little,  if  any,  differences  of 
computer  attitudes  between  males  and  females.  In  addition,  Kay  (1992)  based  upon  a meta-analysis  described  the 
results  of  this  research  as  “conflicting  and  confusing.”(p.227)  Rosen  and  Maguire  (1990)  concluded  on  the  basis  of 
a meta-analysis  that  gender  differences  on  computer  attitudes  were  real,  but  small.  A more  recent  meta- analysis  on 
this  issue  conducted  by  Whitley  (1997)  also  reported  that  the  overall  effects  were  “nonsignificant  or  small.”  (p- 1 1) 
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Whitley  (1997)  proposed  a possible  explanation  for  the  confusing  picture  in  the  computer  attitude  literature.  He 
argued  that  many  researchers  treat  attitudes  toward  computers  as  a unitary  construct  rather  than  multifaceted 
construct.  The  variety  of  computer  attitudes  scales  that  have  been  developed  and  their  differing  content  suggest  that 
computer-related  attitudes  are  indeed  multifaceted,  including  components  such  as  anxiety  about  using  computers, 
self-confidence  in  dealing  with  computers,  and  the  positive  and  negative  beliefs  about  computers  and  their  effects  on 
society  (Whitley,  1997).  Gender  differences  exist  on  one  component  in  one  computer  scale  may  vary  in  another 
computer  scale,  depends  on  what  questions  are  asked.  Therefore,  a meta-analysis  synthesizes  gender  differences  for 
results  from  studies  using  varied  computer-related  scales  may  not  only  obtain  confusing  picture,  but  also  produce 
unexpected  confounding  conclusion.  To  avoid  this,  the  purpose  of  the  present  meta-analysis  was  to  synthesize  the 
results  of  existing  studies  on  gender  differences  toward  computer  attitudes  using  Loyd  and  Gressards’  Computer 
Attitude  Scale  (CAS).  The  results  of  this  meta-analysis  may  provide  clearer  conclusion  on  this  issue. 

Procedures 

The  research  method  used  in  this  study  is  the  meta-analytic  approach  which  was  similar  to  that  suggested  by 
Kulik,  Kulik,  & Bangert-Drowns  (1985).  Their  approach  requires  a reviewer  to  (a)  locate  studies  through  objective 
and  replicable  searches;  (b)  code  the  studies  for  salient  features;  (c)  describe  outcomes  on  a common  scale;  and  (d) 
use  statistical  methods  to  relate  study  features  to  outcomes  (Kulik,  Kulik,  & Bangert-Drowns,  1985). 

Data  Sources 

The  studies  considered  for  use  in  this  meta-analysis  came  from  three  major  sources  and  were  published  from 
1986  to  1998.  One  large  group  of  studies  came  from  computer  searches  of  Education  Resources  Information  Center 
(ERIC).  A second  group  of  studies  came  from  Comprehensive  Dissertation  Abstracts.  A third  group  of  studies  was 
retrieved  by  branching  from  bibliographies  in  the  documents  located  through  review  and  computer  searches. 

Twenty-eight  studies  were  located  through  these  search  procedures;  8 studies  came  from  ERIC,  16  studies 
were  retrieved  from  published  journals,  and  4 studies  were  from  Comprehensive  Dissertation  Abstracts. 

Several  criteria  were  established  for  inclusion  of  studies  in  the  present  analysis. 

1 . Studies  had  to  compare  the  differences  between  males  and  females  on  attitudes  toward  computer. 

2.  Studies  had  to  use  Loyd  and  Gressard’s  Computer  Attitude  Scale  (CAS)  as  instrument. 

3.  Studies  had  to  provide  quantitative  results  from  both  male  and  female  subjects. 

4.  Studies  had  to  be  retrievable  from  university  or  college  libraries  by  interlibrary  loan  or  from  ERIC, 

Dissertation  Abstracts  International,  or  University  Microfiche  International. 

Outcome  Measures 

The  outcome  measured  most  often  in  the  28  studies  was  survey  data  from  participants,  as  indicated  in  CAS  for 
examining  participants’  attitudes  toward  computers.  For  statistical  analysis,  outcomes  from  a variety  of  different 
studies  had  to  be  expressed  on  a common  scale.  The  transformation  used  for  this  purpose  was  the  one  recommended 
by  Glass  et  al.  (1981)  and  modified  by  others  (e.g..  Hunter,  Schmidt,  and  Jackson,  1982).  To  reduce  measurements 
to  a common  scale,  each  outcome  was  coded  as  an  Effect  Size  (ES),  defined  as  the  difference  between  the  mean 
scores  of  males  and  females  divided  by  the  pooled  standard  deviation  of  two  groups.  For  those  studies  that  did  not 
report  means  and  standard  deviations,  F values,  or  t values  were  used  to  estimate  the  ES. 

Variables  Studied 

Nine  variables  were  selected  for  coding  each  study  in  the  present  synthesis.  These  variables  were  nation  of 
subject,  target  population,  type  of  publication,  year  of  publication,  sample  size,  reliability  of  measure,  statistical 
power,  statistics,  and  type  of  attitude.  Each  variable  was  employed  as  a factor  in  an  analysis  of  variance  (ANOVA) 
to  investigate  whether  there  were  significant  differences  within  each  variable  on  the  effect  size. 
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Results 


The  number  of  comparisons,  target  population,  and  the  study-weighted  ESs  are  reported  in  Table  1.  Of  the  28 
studies  included  in  the  present  synthesis,  24or  86%  of  the  study- weighted  ESs  were  positive  and  favored  the  male 
subjects,  while  4 or  14%  of  them  were  negative  and  favored  the  female  subjects,  indicating  that  males  had  more 
positive  attitudes  toward  computers  than  females.  The  range  of  the  study-weighted  ESs  was  from  -0.407  to  0.772. 
The  overall  grand  mean  for  all  28  study- weighted  ESs  was  0.273.  When  this  mean  ES  was  converted  to  percentiles, 
the  percentiles  on  computer  attitudes  were  61  for  the  male  subjects  and  50  for  the  female  subjects.  The  standard 
deviation  of  0.284  reflects  the  small  variability  of  ESs  across  studies. 


Author(s)  Year  of  Target  N of  ES 

Publication  Population  Comparison 


Abler  & Sedlacek 

1987 

College 

15 

0.4355 

Busch 

1995 

College 

7 

0.5199 

Cantrell 

1995 

Teacher 

4 

0.5494 

Colley  et  al. 

1994 

College 

3 

0.4748 

Dyck,  & Smither 

1996 

Adult 

3 

0.7339 

Francis 

1994 

College 

3 

0.0364 

Hwang 

1991 

Elementary 

1 

0.2006 

Kluever  et  al. 

1995 

Teacher 

4 

0.0236 

Koohang 

1986 

College 

3 

0.2538 

Koohang 

1986 

High  School 

1 

0.7238 

Koohang 

1987 

College 

3 

0.1501 

Koohang 

1989 

College 

4 

0.3307 

Latham 

1991 

College 

4 

0.7723 

Liao 

1993 

College 

3 

-0.0019 

Liao 

1994 

College 

3 

0.2817 

Liao,  & Shih 

1994 

College 

3 

0.2858 

Liao 

1995 

College 

6 

-0.0833 

Loyd,  & Gressard 

1984 

Mixed 

3 

0.1028 

Loyd,  & Gressard 

1986 

Teacher 

4 

0.3771 

Loyd  et  al. 

1987 

Junior  High 

12 

-0.1254 

Mertens,  & Wang 

1988 

College 

3 

-0.4071 

Moon  et  al. 

1994 

College 

3 

0.3277 

Nash,  & Moroz 

1997 

Teacher 

1 

0.1107 

Pope-Davis,  &Twing 

1991 

College 

4 

0.1353 

Sacks  etal. 

1993 

High  School 

2 

0.0980 

Shawareb 

1993 

College 

1 

0.2750 

Wallace,  & Sinclair 

1995 

College 

3 

0.7390 

Woodrow 

1994 

Junior  High 

6 

0.3638 

Overall  grand  mean 

0.273 

Overall  grand  median 

0.278 

Overall  grand  SD 

0.284 

Total  N of  study 

28 

Total  N of  comparison 

112 

Table  1 : Number  of  Comparisons  and  Study-weighted  Effect  Sizes 


Table  2 lists  the  F values  for  the  9 variables  for  all  study- weighted  ESs  in  the  study.  The  positive  means  indicate 
more  positive  attitudes  toward  computers  favor  male  subjects.  For  ANOVA,  2 variables,  sample  size  and  type  of 
attitude,  showed  statistically  significant  impact.  Descriptive  statistics  for  these  variables  are  presented  in  Table  3. 

For  sample  size,  (F(3,  24)  = 3.075,  p<.05),  the  post  hoc  test  showed  that  the  mean  comparison  of  studies  with 
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101  - 200  participants  was  significantly  higher  than  studies  with  201  ~ 300,  or  over  300  participants. 

The  post  hoc  test  for  type  of  attitude,  (F(4,  76)  = 4.864,  P<.01),  showed  that  the  mean  comparison  of  studies 
that  measured  anxiety  was  significantly  higher  than  studies  that  measured  liking.  In  addition,  the  mean  comparison 
of  studies  that  measured  mixed  attitudes  was  significantly  higher  than  studies  that  measured  other  types  of  attitudes. 


Variables 

df 

F 

P 

Nation  of  Subject 

1,26 

0.457 

0.505 

Target  Population 

3,24 

0.267 

0.849 

Sample  Size 

3,24 

3.075 

0.047* 

Type  of  Publication 

2,  25 

1.121 

0.342 

Year  of  Publication 

3,24 

0.393 

0.759 

Reliability  of  Measure 

1,26 

2.996 

0.095 

Statistical  Power 

1,26 

0.040 

0.842 

Statistics 

2,  25 

0.231 

0.796 

Type  of  Attitude”* 

4,  76 

4.864 

0.002** 

*p<.05  **p<.01 

Note.  ® Some  studies  reported  more  than  one  comparison  groups. 
Table  2:  Results  of  ANOVAs  for  Coded  Variables 

Variables 

N 

% 

ES 

SD 

Sample  Size 

Less  than  100 

9 

33 

0.326 

0.379 

101-200 

6 

21 

0.498 

0.145 

201  -300 

7 

25 

0.170 

0.153 

Over  300 

6 

21 

0.090 

0.187 

Type  of  Attitude”* 

Anxiety 

24 

30 

0.134 

0.173 

Usefulness 

5 

6 

0.023 

0.055 

Confidence 

24 

30 

0.081 

0.109 

Liking 

24 

30 

0.043 

0.097 

Mixed 

4 

4 

0.340 

0.296 

Note.  ”*  Some  studies  reported 

more  than  one  comparison  groups. 

Table  3:  Means  and  Standard  Deviations  of  Study-weighted  ESs  for  Coded  Variables 


Discussion 

The  results  of  this  meta-analysis  indicate  that  gender  differences  on  computer  attitudes  exist,  males  have 
slightly  positive  attitudes  toward  computers  than  females;  however  the  differences  are  small.  An  effect  is  said  to  be 
small  when  ES  = 0.2,  medium  when  ES  = 0.5,  and  large  when  ES  = 0.8  (Cohen,  1977).  Eighty-six  percent  of 
positive  study-weighted  ES  values  and  78%  of  positive  ESs  overall  also  confirm  the  males’  positive  attitudes  over 
females.  The  slightness  of  the  differences  must  keep  in  mind,  however;  the  overall  study-weighted  mean  ES  of 
0.273  only  indicates  1 1 percentile  scores  higher  than  the  female  subjects.  The  percentile  scores  for  the  overall  grand 
mean  and  median  were  identical,  possibly  attributed  to  the  small  overall  grand  standard  deviation  (0.284). 

The  analysis  of  studied  variables  suggests  some  interesting  trends  in  the  accumulated  research  base  and  is 
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discussed  in  the  following  sections. 

A comparative  study  of  subjects  from  different  cultures  can  supply  valuable  information,  not  only  about  the 
specific  groups  selected  whose  reactions  toward  computers,  but  also  examine  and  compare  the  presence  of  computer 
in  the  context  of  two  different  cultures  (Collis  & Williams,  1987).  The  results  of  this  synthesis  indicate  that  studies 
using  subjects  from  other  countries  had  slightly  higher  ES  than  subjects  from  US.  There  may  be  some  cultural  bias, 
or  difference  in  computer  accessibility  between  US  and  other  countries.  The  differences,  however,  were  trivial;  more 
cross-cultural  studies  need  to  be  conducted  to  clarify  this  variable. 

For  the  target  population  variable,  there  was  no  significant  difference  of  mean  ES.  The  fact  that  about  2/3  of 
studies  conducted  for  college  students  seems  to  suggest  that  most  researchers  assumed  that  Loyd  and  Gressards’ 
CAS  was  more  appropriate  for  this  population.  In  addition,  the  mean  ES  slightly  increased  from  K -12  to 
teacher/adult  is  not  surprising.  Studies  have  shown  that,  with  age,  males  are  increasingly  more  confident  in  their 
abilities  to  use  computers  and  more  positive  in  their  attitudes  toward  computers  than  are  females  (Ayersman,  1 996; 
Bunderson,  1995;  Chen,  1986;  Fetler,  1985;  Hattie,  1987;  Sackrowitz,  1995;  Shashaani,  1994).  The  results  of  this 
meta-analysis  were  consistent  with  their  findings. 

Source  of  studies  in  a meta-analysis  is  always  an  important  factor  to  be  examined.  The  fact  that  more  than  half 
of  studies  were  located  from  journal  articles  indicates  that  the  majority  of  studies  included  in  the  synthesis  have  been 
critically  reviewed  by  journal  reviewers.  The  larger  ES  associated  with  Dissertation/thesis  is  atypical,  however; 
usually  the  larger  ES  associated  with  published  articles  is  typical  in  meta-analysis  (Glass,  et  al.,  1981,  p.  227). 

The  year-of-publication  variable  in  the  meta-analysis  allows  an  assessment  of  the  effect  of  hypermedia  over 
time.  Although  no  significant  difference  of  mean  ES  was  found,  the  larger  ES  associated  with  studies  published 
between  1995  - 1997  seems  to  suggest  that  gender  differences  on  attitudes  toward  computers  have  become  more 
observable  recently. 

The  sample  size  for  a study  may  significantly  affect  the  statistical  power  of  the  study;  in  general,  the  larger  the 
sample  size,  the  better  the  statistical  power.  The  sample  sizes  for  about  54%  of  studies  included  in  the  synthesis 
were  less  than  200,  and  the  higher  mean  ES  was  associated  with  these  studies.  For  studies  in  which  the  sample  size 
went  beyond  200,  the  mean  ES  dramatically  dropped  to  less  than  0.2.  This  seems  to  suggest  that  the  gender 
differences  on  computer  attitudes  may  only  be  observed  in  small  to  medium  sample  size.  Why  the  differences 
diminish  in  a larger  sample  size  will  be  a critical  question  for  researchers. 

For  reliability  of  measure,  statistical  power,  and  statistics,  there  was  no  significant  difference  of  mean  ES 
among  them.  This  suggests  that  mean  ES  did  not  affected  by  the  differences  of  characteristics  of  research 
methodology. 

Gender  differences  in  computer  attitudes  clearly  varied  as  a function  of  type  of  attitude.  The  positive  mean  ESs 
indicate  males  had  lower  anxiety,  and  higher  confidence,  liking,  and  usefulness  attitudes  toward  computers  than 
females. 

The  mean  ES  for  each  subscale  showed  that  gender  differences  on  anxiety  was  higher  than  on  liking, 
confidence,  and  usefulness,  suggesting,  males  and  females  had  small  differences  on  anxiety,  and  trivial  differences 
on  computer  liking,  confidence,  and  usefulness.  In  addition,  4 studies  did  not  report  data  for  subscales  and  were 
coded  as  mixed.  These  studies  showed  highest  mean  ES  (ES  = 3.4).  As  a whole,  he  results  somehow  indicate  that 
males  hold  more  positive  attitudes  toward  computers  than  females. 


Conclusions 

The  present  meta-analysis  synthesized  gender  differences  on  attitudes  toward  computers  for  studies  using  Loyd 
and  Gressards’  CAS  so  that  confounding  results  could  be  avoided  as  stated  earlier.  The  unique  feature  differs  this 
meta-analysis  from  other  meta-anlyses  on  this  issue.  Although  the  results  showed  that  the  overall  ES  was  small,  the 
fact  that  the  differences  between  males  and  females  exist  was  obvious.  This  finding  is  consistent  with  earlier  studies 
conducted  by  Rosen  and  Maguire  (1990),  and  Whitley  (1997).  Left  unanswered  is  the  question  of  what  factors  truly 
contribute  to  the  differences.  Studies  of  this  question  will  require  further  clarification  of  the  exact  characteristics  of 
each  type  of  computer  attitude  (i.e.,  anxiety,  confidence,  liking  and  usefulness)  and  their  relationships  with  gender. 
This  meta-analysis  points  out  only  that  gender  differences  on  attitudes  toward  computers  exist.  That  information  by 
itself  is  useful. 
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Abstract:  This  paper  considers  the  impact  of  diverse  staff  development  activities  designed  to 
assist  faculty  to  integrate  online  media  into  the  curriculum.  An  overview  is  given  of  the  aims, 
objectives,  activities  and  evaluation  of  the  1998-99  e.learning@mq  project  at  Macquarie 
University.  Influenced  by  diffusion  theory  and  the  characteristics  of  the  early  and  late- 
majority's  uptake  of  innovations,  the  author  designed  various  activities  for  both  local-area 
and  university- wide  settings.  The  patterns  of  participation  reveal  the  varied  activities 
attracted  different  groupings  and  numbers  of  faculty  and  administrative  staff.  At  the 
completion  of  the  project  91  participants  self-assessed  the  project’s  impact  on  their 
understanding  of  the  issues  involved  in  using  online  media  in  learning  and  teaching.  In  nine 
specified  areas  of  understanding  81%  of  the  project  participants  have  self-evaluated  slight, 
moderate  or  significant  improvement  in  awareness,  knowledge  and  skills  in  using 
information  technology  and  online  media  in  their  educational  programs. 


Introduction 

Educational  practices  are  intrinsically  embedded  into  the  prevailing  dominant  technologies  of 
communication,  knowledge  creation  and  information  dissemination.  These  tools,  practices  and  processes  are 
rapidly  changing  due  to  the  rise  of  digital  online  media.  Evermore  publicity  is  appearing  from  global  'borderless' 
providers  offering  privatised  and  accredited  educational  services  'anytime  anywhere'.  Using  'online  learning'  and 
'flexible  delivery'  these  services  are  now  available  'just-in-time*  and  tailored  to  meet  your  individual  needs.  The 
rhetoric  exhorts  that  to  remain  relevant  in  an  increasingly  networked  economy  and  society,  education  providers 
need  to  integrate  online  communication,  activities  and  resources  into  their  offerings. 

To  support  these  changes  an  important  organisational  strategy  is  effective  staff  development.  This 
paper  presents  an  overview  of  e.learning@mq,  a 1998-99  project  at  Macquarie  University  that  aimed  to  develop 
capacity  to  integrate  information  technology  (IT)  into  learning  and  teaching.  The  project  focused  on  the 
development  of  faculty,  though  activities  were  open  to  all  Macquarie  staff  and  interested  visitors. 

Influenced  by  diffusion  theory  and  the  characteristics  of  the  majority's  uptake  of  innovations,  the  author 
designed  and  implemented  diverse  activities  and  initiatives  for  local-area  and  university-wide  settings.  Diffusion 
theory  argues  that  adoption  of  new  technology  and  innovation  starts  with  enthusiastic  innovators  and  early- 
adopters  and  then  moves  to  use  by  the  early  and  late-majority  when  the  innovation  is  better  supported  and  more 
reliable.  The  characteristics  of  the  majority  to  consider  in  the  design  of  'diffused'  staff  development  include  that 
they  look  to  local  and  discipline-based  ideas  and  practice,  favour  incremental  change,  are  pragmatic  and  risk 
averse,  want  proven  applications,  and  may  need  significant  support  (Rogers  1995,  Geoghegan  1994). 

eJearning@mq  activities  were  designed  to  address  these  characteristics  and  provided  staff  with 
multifarious  entry-points  to  engage  with  the  issues  of  integrating  online  media  into  their  educational  programs.  It 
was  hoped  the  varied  activities  would  attract  different  individuals  and  groupings  of  staff 


Project  Aims  and  Objectives 

Macquarie’s  IT  Strategic  Plan,  states  that  "successful  integration  of  IT  into  a university  environment 
requires  radical  shifts  in  thinking  about  all  aspects  of  [its]  educational  vision".  To  support  such  integration  the 
Center  for  Professional  Development  (CPD)  successfully  applied  for  an  Australian  Government  1998-99 
CUTSD  staff  development  grant. 
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The  proposed  project  aimed  to  support  the  development  and  implementation  of  a coherent  program  of 
staff  development  to  improve  the  capacity  of  faculty  and  units  at  the  University  to  explore  and  introduce  IT 
effectively  into  their  educational  programs.  The  successfully  proposed  objectives  of  the  project  were  to; 

1 . raise  the  awareness  of  staff  at  all  levels  to; 

• the  potential  of  IT  to  enhance  and  deliver  educational  programs  across  the  disciplines, 

• the  issues  associated  with  the  design  and  development  of  IT  based  curricula, 

• the  issues  associated  with  managing  the  development  and  integration  of  IT  into  the  curriculum, 

2.  assist  staff  to  strategically  plan  and  set  specific  goals  in  relation  to  the  introduction  of  IT, 

3.  assist  staff  to  develop  coordinated  strategies  for  realising  these  goals, 

4.  assist  staff  to  develop  the  knowledge  and  skills  required  to  use  IT  in  the  curriculum. 


Project  Activities 

The  author's  collaborative  project  management  involved  all  the  providers  of  IT  support  and  facilities  at 
Macquarie,  together  with  many  faculty  early-adopters  and  invited  quests,  to  convene  and  facilitate  e. learning 
activities.  In  early  1998  as  reference  group  was  established  with  members  from  the  CPD,  the  Centre  for  Flexible 
Learning  (CFL)  and  the  Macquarie  Library’s  IT  Training  Unit  (ITTU)  to  assist  the  development  of  a more 
cohesive  approach  to  the  project's  activities. 

Three  co-ordinated  programs  of  e. learning  were  implemented  during  each  of  the  teaching  semesters 
from  August  1998  to  December  1999  and  included  local-area  and  university- wide  activities  and  initiatives.  The 
project  website  at  <www.cpd.mq.edu.au/eJearning>  facilitated  online  bookings,  discussion  and  feedback.  The 
site  includes  full  descriptions  of  all  the  project  activities  with  some  handouts  and  transcriptions  from  events.  The 
site  was  designed  to  model  a simple  yet  functional  and  effective  online  media  resource. 

Local-area  Peer  Development  Projects 

The  successful  e.learning@mq  grant  application  proposed  to  establish  IT  peer  development  groups  in 
local  areas  to  develop  discipline-specific  approaches  to  their  educational  programs  and  to  foster  continuing 
exploration  and  debate.  The  author  designed  a competitive  grant  scheme  for  teams  to  research  localized  IT 
issues.  To  receive  funds  the  team-leaders  had  to  establish  a local-area  team  and  propose  a plan  to  research  an 
appropriate  IT  issue  and  then  convene  appropriate  peer  development  activities  for  their  colleagues.  To  provide 
an  incentive  for  team-members  to  become  involved  finding  was  available  for  time-relief  and  other  minor 
expenses. 

In  late  1998,  three  workshops  supported  the  development  of  project  proposals  to  include  a concept, 
rationale,  objectives,  timeline  and  budget.  A selection  committee  decided  that  nine  from  a total  of  15  submitted 
proposals  could  be  funded  from  the  e.learning@mq  grant.  These  nine  local-area  projects  were  implemented 
during  1999  and  early  2000; 

• 'How  to  use  computer-mediated-communication  in  european  languages*  in  the  School  of  European 
Languages, 

• 'Getting  with  IT!  A structured  IT  core  competencies  program  for  staff ' within  the  Macquarie  Library, 

• 'Using  IT  strategies  for  communication  between  on-campus  lecturers  and  mentors  of  students'  within 
Warawara,  Macquarie's  Aboriginal  and  Torres  Straight  Islander  (ATSI)  Centre, 

• 'IT  supported  unit  delivery  facilitation'  \n  the  Institute  of  Early  Childhood, 

• 'Individualised  IT  support  and  training'  in  the  School  of  Biological  Sciences, 

• 'How  to  use  WebCT for  creating  online  english  language  learning  modules'  in  the  National  Centre  for 
English  Language  Training  and  Research, 

• 'Moving  beyond  the  still  frame  -multiskill  with  multimedia'  within  the  Macquarie  Library, 

• 'IT  training  for  flexible  courses'  in  the  School  of  Asian  Languages, 

• 'Teacher  Education  website  project'  in  the  School  of  Education. 

University-wide  activities 

The  e.learning@mq  application  proposed  to  improve  awareness,  knowledge  and  skills  about 
integrating  IT  into  the  curriculum  with  "one-off  and  "short-term"  activities.  This  prescription  required 
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considerable  elaboration  and  the  author  planned  five  types  of  university-wide  activities,  which  were  then 
collaboratively  designed  and  facilitated  by  staff  from  all  levels  across  the  university; 

• Speakers  addressed  the  theme  ’You,  Macquarie  & information  technology’  with  time  for  questions  and 
answers  in  a one-hour  lunchtime  format, 

• Workshops  with  the  theme  ’Integrating  e. learning  into  the  Curriculum*  with  theoretical  discussion  and 
practice  activities  in  two-hour  lunchtime  and  some  early-evening  sessions, 

• Showcases  of  e. learning  case-studies  from  different  disciplines  and  software  demonstrations  in  a one-hour 
lunchtime  format, 

• Short  courses  requiring  a time  commitment  to  a series  of  activities  over  two  to  six  weeks, 

• Information  sessions  about  IT  support  services  and  facilities  at  Macquarie  in  a one -hour  lunchtime  format. 

Through  three  semester-long  programs  these  activities  were  collaboratively  designed,  convened  and 
facilitated  together  with  many  Macquarie  staff  from  all  levels.  Descriptors  of  all  the  events  and  activities  are 
found  at  <www.cpd.mq.edu.au/e.learning> . 

Adjunct  Activities 

In  collaboration  with  e.learning@mq^  two  adjunct  initiatives  received  organisational  support; 

• Macquarie  has  established  the  e.studio  in  the  Library.  The  facility  is  an  independent  staff  and  postgraduate 
research  student  IT  learning  facility  with  hardware  and  software  often  not  available  in  local  areas.  Launched 
early  in  1999,  the  e. studio  is  supported  by  equipment  and  software  demonstrations  by  staff  from  ITTU  and 
CFL  and  promoted  in  CPD’s  publicity.  Interesting  work  is  emerging  from  this  facility  that  would  not  of 
otherwise  occurred  due  to  a lack  of  local-area  equipment  and  support. 

• The  CPD  funded  a global  survey  to  identify  and  determine  the  quality  and  scope  of  accessible,  online 
resources  specifically  designed  to  support  the  development  of  english  language  higher-education  staff.  A 
total  of  385  university  websites  were  surveyed  in  detail  and  the  pdonline  website  at 
<www.cpd.mq.edu.au/pdonline>  has  links  to  the  ’best-of  online  resources  found  (Litchfield  & Spear  1999). 


Patterns  of  Staff  Participation 

Although  I take  advantage  of  much  of  what  CPD  offers  it  can  be  seen  as  something  'out  there'.  I think 
this  project  is  the  best  marrying  of  CPD  and  the  academic  staff  at  the  university  that  I have  been 
involved  in. 

Comment  of  a local-area  team  member. 

In  Australian  higher-education,  staff  are  industrially  categorised  either  as  ’academic  and  research’  or 
'administrative  and  specialist  support'  - a ’divide’  commonly  described  as  academic  and  general.  An  emerging 
feature  of  integrating  online  media  into  learning  and  teaching  is  the  blurring  of  these  traditional  categories. 
University  work  is  being  actively  re-defined  with  position  descriptions  and  selection  criteria  increasingly 
influenced  by  the  skills,  roles  and  team-structures  needed  by  networked  education.  Concurrently  many 
traditional  academic  and  general  staff  positions  are  becoming  redundant,  and  many  of  the  e.learning@mq 
participants  were  understandably  motivated  and  stressed  by  this  phenomena. 

Local-area  Peer  Development  Projects 

In  late  1998,  44  staff  attended  three  workshops  convened  to  support  the  development  of  proposals  for 
the  e. learning  local-area  grants.  All  the  ultimately  successful  project  team-leaders  attended  the  workshops  and 
have  stated  the  high-value  of  this  support  in  the  development  of  sound  proposals  with  clear  aims  and  objectives, 
time-lines  and  budgets.  The  nine  funded  project  teams  comprised  38  members  (68%  academic  and  32% 
administrative  staff)  who  worked  together,  and  then  received  payment  for  their  time,  throughout  1999  and  early 
2000.  Many  of  the  team-leaders  were  in  this  role  for  the  first-time  and  developed  substantial  project,  team  and 
financial  management  knowledge  and  skills. 

At  mini-conferences  held  in  February,  July  and  December  1999  the  teams  reported  on  progress  and 
discussed  issues.  The  mini-conferences  had  72  attendances  indicating  a high-level  of  team-member  participation 
in  these  support  activities.  The  final  reports  from  the  team-leaders  indicate  the  local-area  and  discipline-based 
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objectives  were  all  realised  often  with  unforeseen  advances,  and  the  peer  development  activities  achieved  very 
high-levels  of  participation.  This  high-level  of  peer  involvement  confirms  diffusion  theory's  premise  that  the 
majority  favour  incremental  change  and  look  to  local  and  discipline-based  settings  for  new  ideas  and  innovative 
practice. 

As  the  main  cost-item  of  the  e.learning@mq  budget,  the  author  considers  the  local-area  team  and  peer 
development  scheme  as  the  most  effective  and  substantial  of  all  the  project’s  activities.  Requiring  the  least 
amount  of  CPD  support,  significant  peer  development  has  occurred  in  self-selected  local  issues  of  integrating  IT 
into  educational  programs. 


University-wide  Activities 


65  eJearning  university- wide  activities  were  convened  and  the  records  indicate  a total  of  997 
attendances  from  385  individuals.  The  6 speakers  attracted  226  attendances,  the  30  workshops  357,  the  15 
showcases  194  participants,  the  9 short  courses  170  participants  and  5 IT  support  information  sessions  50 
participants.  The  level  of  participation  was  well-spread  with  only  one  activity  needing  to  be  cancelled  due  to  lack 
of  interest. 

Figure  One  compares  the  staff-category  of  participants  in  each  of  the  five  university-wide  activities. 
The  attendance  patterns  indicate  the  project  successftilly  focused  on  involving  academic  staff,  who  were  the 
majority  in  all  activity-types,  particularly  in  the  short  courses,  workshops,  and  showcases.  Only  slightly  higher 
numbers  of  academic  than  general  staff  attended  the  speakers  and  IT  support  information  sessions.  Across  all 
five  activities  an  average  59%  of  participants  were  academic  staff,  31%  general  staff,  and  10%  were  of  unknown 
category  including  the  participants  external  to  Macquarie. 


Academic  & 
Research  staff 


Administration 
& specialist 
support  staff 


Unknown 

category 


Figure  One:  Staff  category  of  participants  in  the  five  eJearning  university- wide  activities. 


University-staff  in  Australia  are  over-worked,  time  is  valuable  and  clearly  participants  were  selective 
about  what  interested  them  enough  to  attend.  The  author’s  diverse  and  multi -entry  'scatter-gun'  strategy  certainly 
worked  as  each  participant  averaged  only  2.6  attendances.  Though  face-to-face  'one-off  workshops  are  the 
traditional  and  most-popular  method  of  higher-education  staff  development,  it  is  important  to  note  the  clear 
potential  of  short  courses  to  attract  and  involve  academic  staff 

Activities  which  did  not  succeed  included  open  calls  to  apply  for  funds  to  support  relevant  local-area 
activities.  Some  of  the  speaker’s  attracted  low  attendances,  and  a web-conference  established  to  facilitate 
ongoing  discussion  of  events,  debates  and  issues  received  minimal  staff  contributions.  The  online  feedback  has 
not  worked  and  even  feedback  sheets  handed  out  at  the  conclusion  of  activities  are  difficult  to  gamer. 


Participants’  Self-Assessment  of  the  Impact  of  the  Project 

A total  of  409  individuals  participated  in  the  local-area  and  university-wide  eJearning  activities.  In 
December  1999  after  the  completion  of  all  scheduled  activities,  these  individuals  were  sent  an  email 
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questionnaire  asking  for  their  assessment  of  the  impact  of  their  participation  in  understanding  the  issues  involved 
in  integrating  IT  and  online  media  into  the  curriculum.  A follow-up  paper-version  of  the  questionnaire  to 
facilitate  anonymity  was  then  sent  to  all  the  participants  who  had  not  responded  to  the  email.  A total  of  91  valid 
responses  were  received  of  which  a high  55  were  anonymous. 

In  nine  key  areas  the  questionnaire  asked  the  participants  to  self-assess  the  level  of  impact  of  their 
e.learning@mq  involvement  as  significant,  moderate,  slight  or  not-at-all.  The  questions  asked  whether  their 
participation  had  increased  their  (1)  awareness,  or  their  knowledge  or  skills  in  (2)  planning,  (3)  strategic  program 
planning,  (4)  managing,  (5)  designing,  (6)  developing,  (7)  implementing,  (8)  evaluating,  or  assisted  them  to 
develop  (9)  coordinated  strategies  for  the  use  of  IT  in  the  curriculum.  Figure  Two  compares  the  impact  of  the 
project  in  these  nine  key  areas  of  understanding. 
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Figure  Two:  Participant's  self-evaluation  of  the  impact  of  e.learning@mq  in  nine  key  areas. 


On  average  across  the  nine  specified  areas  of  understanding  a total  81%  of  the  participants  assessed  that 
the  project  had  a positive  impact.  An  average  18%  of  participants  assessed  the  project  had  a significant  impact, 
35%  a moderate  impact,  28%  a slight  impact,  and  19%  assessed  the  project  had  no  impact  at  all.  These  findings 
indicate  the  project  has  achieved  its  aim  to  raise  staffs  awareness  and  improve  their  knowledge  and  skills  in  the 
issues. 

In  each  of  the  nine  areas  there  are  interesting  variations  in  the  degree  of  impact  of  the  project.  The 
strongest  impact  was  in  increased  awareness  of  the  issues,  and  in  the  knowledge  and  skills  involved  in  planning, 
designing  and  implementing  the  use  of  IT  in  the  curriculum.  The  project's  weakest  impact  was  in  evaluating  and 
developing  coordinated  strategies.  The  activities  focusing  on  these  issues  were  poorly  attended  and  reflect  the 
difficulties  of  developing  coordinated  strategies  at  the  individual  level,  noting  that  in  1998  Macquarie  underwent 
a major  organisational  restructure. 


Conclusion 

In  an  important  strategic  direction  for  Macquarie  University,  the  1998-99  e.learning@mq  staff 
development  project  fulfilled  its  aim  to  increase  the  organization's  capacity  to  integrate  IT  and  online  media  into 
the  curriculum.  The  project  outcomes  clearly  indicate  that  well-funded  and  managed  professional  development 
activities  can  increase  individual  awareness,  knowledge  and  skills  about  integrating  online  media  into  higher- 
education  learning  and  teaching.  The  project's  diverse  activities,  in  both  local-area  and  university- wide  settings, 
attracted  different  groupings  and  numbers  of  staff,  justifying  the  author's  pluralistic  approach  informed  by 
diffusion  theory  and  the  characteristics  of  the  majority  in  the  uptake  of  innovations.  The  different  activities  did 
appeal  to  different  people. 
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Participating  individual  staff  have  self-assessed  improvements  in  awareness  of  the  issues  involved  in 
integrating  online  media  into  the  curriculum.  On  average,  across  nine  key  areas  of  understanding,  81%  of 
participants  have  self-evaluated  slight,  moderate  or  significant  improvement  in  awareness,  knowledge  and  skills. 

However  it  is  important  to  note  that  Macquarie  has  a total  permanent  staff  of  some  1500  and  only  409 
individuals  participated  in  the  project.  Overall  19%  of  these  participants  considered  no  change  at  all  had 
occurred  through  the  project.  So  a complex  and  ongoing  issue  is  how  to  increase  involvement  from  staff  who 
have  yet  to  show  interest  in  using  online  media.  There  is  a need  for  innovative  and  imaginative  strategies  to 
address  this  issue  including  improved  incentives  and  rewards.  In  particular  there  is  a clear  need  for  strategies  to 
maximise  the  number  of  staff  getting  their  'hands-on  the  tools'  in  a supportive,  localised  context. 

The  project's  external  funding  was  for  1998-99  and  the  project  has  now  ceased.  Macquarie  has  not 
funded  an  ongoing  project  as  apparently  the  aims  have  been  met  and  are  now  "embeddded  into  the 
infrastructure".  Better  ongoing  cohesion  and  collaboration  between  Macquarie's  IT  support  structures  and  staff 
was  facilitated  by  the  project  and  some  activities  continue  in  different  ways. 

The  development  required  to  improve  the  organisational  capacity  to  integrate  new  media  into  the 
curriculum  continues  to  require  substantial  innovative,  collaborative  and  well-designed  policy,  processes  and 
activities.  Issues  such  as  improved  supports  for  staff  and  students,  policy  developments  with  rigorous  course 
review  and  accreditation  processes,  and  better  teaching  load  recognition  for  online  development,  and  other 
incentives  and  rewards,  are  crucial  initiatives  that  remain  to  be  achieved. 

As  a local-area  e.learning  participant  noted  “it  is  only  the  tip  of  the  iceberg  - unless  we  can  get  the 
value  of  this  type  of  skill  development  recognised  at  university  level,  that  is  with  ongoing  provisions  for  staff 
relief  for  such  projects  the  impact  will  be  limited”. 

The  author  hopes  the  activities  and  outcomes  of  the  1998-99  e.learning@mq  project  can  assist 
staff  developers  involved  in  similar  work.  Regardless  of  our  respective  roles,  with  issues  of  IT  and  change  we 
constantly  need  to  be  learning  and  supporting  each  other.  Indeed  facilitating  professional  development  in  using 
online  media  is  emerging  as  a generic  skill  required  by  us  all. 
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Abstract:  This  paper  discusses  a National  Science  Foundation  sponsored  project  for 
developing  virtual  multimedia  examination  formats  for  use  in  sciences.  It  describes 
the  project  and  multimedia  exam  templates,  and  the  results  of  field-testing  of  their 
uses  in  the  classrooms. 

Theoretical  Framework 

The  prevalence  of  technology  is  experienced  in  every  aspect  of  our  lives.  Being  able  to  understand 
technology  and  using  technological  tools  have  become  important  expectations  of  today's  and  tomorrow's  workforce. 
The  Clinton  Administration  has  identified  four  areas  that  need  to  be  addressed  to  meet  the  challenge  of  preparing 
technology  literate  workforces:  (1)  making  modem  computers  accessible  to  all  students,  (2)  making  classrooms 
connected  around  the  world,  (3)  integrating  interactive  educational  software  into  the  curriculum,  and  (4)  making 
sure  that  teachers  are  capable  of  teaching  with  technology  (U.S.  Department  of  Education,  1998). 

Integrating  interactive  software  into  the  curriculum  calls  for  the  use  of  innovative  software  in  teaching 
and  learning.  Research  shows  that  technology  has  the  potential  to  facilitate  learners'  understanding  and  increase 
their  problem-solving  skills  if  the  software  is  well-designed  (Liu  & Reed,  1995;  O’Callaghan,  1995;  Pedersen, 
Williams,  & Liu,  1999;  Smith,  1994).  However,  the  use  of  innovative  software  as  part  of  teaching  is  not  always 
accompanied  by  innovative  uses  of  assessment  tools.  In  most  cases,  traditional  testing  using  paper-and-pencil  is 
still  a norm  even  for  assessing  the  effectiveness  of  technology  on  learning.  Although  computerized  testing  and 
computer  adaptive  testing  have  been  used  more  than  a decade,  in  the  past  two  years,  there  is  a much  more  stronger 
interest  in  using  online  multimedia  testing  as  an  alternative  assessment  to  the  traditional  paper  and  pencil  tests. 
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This  interest  is  partly  sparkled  by  the  fact  that  the  explosive  uses  of  internet  technology  in  recent  years  has  made  it 
possible  to  deliver  instruction  over  a distance  and  to  a wide  audience.  The  number  of  online  courses  has 
exponentiated.  The  more  extensive  uses  of  technology  in  instruction  have  prompted  educators  to  reexamine  the 
assessment  tools. 

Advantages  and  Limitations  of  Computerized  Testing 

Supporters  of  computerized  testing  maintain  that  testing  on  the  computers  can  minimize  the  error 
involved  in  administration  of  the  test  by  standardizing  the  testing  environment,  and  therefore  increasing  the 
reliability  of  the  test  (Eaves,  1985;  Wise  & Flake,  1990;  Zandvliet  & Farragher,  1997).  It  also  provides  immediate 
computerized  scoring,  which  facilitates  and  expedites  decisions  that  might  need  to  be  taken  based  upon  the  results 
of  the  scores.  Moreover,  computerized  testing  can  provide  the  examinees  with  feedback  on  the  correctness  of  the 
answer.  Additionally,  information  is  provided  regarding  test-taking  behavior.  For  instance,  the  examiner  can 
collect  data  such  as  how  many  items  have  been  reviewed  by  the  examinee,  or  how  much  time  is  spent  on  each  item 
(Wise  & Flake,  1990).  Computerized  assessment  facilitates  evaluation  of  test  items,  including  group  analysis  of  the 
test  and  item  bias,  as  well  as  analysis  of  items’  reliability  and  difficulty  (Eaves,  1985).  In  contrast  to  paper  based 
testing,  computerized  based  testing  gives  the  examiner  the  capability  to  enrich  the  display  of  information  by 
integrating  multimedia  elements.  Finally,  computerized  tests  may  become  individualized  and  proceed  in  different 
ways  based  upon  examinee’s  responses. 

However,  using  computerized  testing  has  limitations.  Will  testing  on  the  computers  cause  anxiety  - test 
anxiety  as  well  computer  anxiety — to  learners?  Is  it  fair  to  use  computerized  testing  on  everyone,  especially  those 
who  do  not  have  frequent  access  to  computers?  Is  it  worth  the  money  and  effort  to  put  a test  on  the  computer  when 
it  is  more  efficient  to  use  a equivalent  paper-and-pencil  version?  Is  it  realistic  to  use  computerized  testing  in  the 
classrooms?  Is  it  realistic  to  expect  instructors  to  develop  computerized  tests  in  addition  to  their  regular  duties? 
Even  though  the  benefits  of  computerized  testing  are  evident  as  discussed  above,  these  limitations  prevent  the  wide 
use  of  computerized  testing  in  the  classrooms  until  recently  when  the  ease  of  incorporating  multimedia  and 
internet  technologies  into  instruction  becomes  a possibility  for  the  majority.  The  following  section  looks  at  the  use 
of  computerized  testing  in  the  recent  years. 

Research  Findings 

Zandvliet  and  Farragher  (1997)  in  their  study  investigate  the  equivalence  of  a computer-administered  test 
(CAT)  with  a written  format  of  the  test.  Analysis  of  the  test  scores  indicated  that  there  were  no  significant 
differences  attributable  to  the  test  format.  Although  the  CAT  required  more  time  than  the  written  test,  authors 
attributed  this  difference  to  the  fact  that  the  students  were  unfamiliar  with  the  computer  format.  They  also 
conducted  a survey  to  identify  students’  attitudes  towards  computerized  testing.  The  results  showed  that  the 
students  clearly  preferred  the  computer-administered  test. 

Isham  (1997)  conducted  a study  to  investigate  whether  an  interactive  computerized  visualization 
assessment  would  help  interior  design  students  improve  their  visualization  skills.  A paper  assessment  was  first 
developed  to  validate  assessment  problems  for  the  computerized  version.  The  computerized  assessment  was  based 
on  the  paper  assessment  and  was  similar  in  content  and  sequence.  It  included,  however,  animations.  The  results  of 
the  study  indicated  that  the  average  score  of  students  who  took  the  computerized  version  of  the  test  was  higher  than 
the  score  of  those  students  who  took  the  paper  version.  The  author  concluded  that  the  inclusion  of  animations  in 
the  assessment  seemed  to  enhance  students'  visualization  skills. 

In  another  study  conducted  by  Soo  and  Ngeow  (1998),  students  using  a multimedia  self-assessment 
English  proficiency  course  were  compared  with  students  in  a conventional  teacher-taught  proficiency  class.  Factors 
such  as  gender,  race  and  learning  style  (visual,  auditory  and  kinesthetic)  were  also  included  in  the  collection  and 
analysis  of  data.  The  results  showed  that  students  who  used  the  self-paced  multimedia  program  achieved 
significantly  higher  scores  as  measured  by  the  TOEFL,  and  in  less  time  than  the  students  who  were  taught  in  a 
traditional  course.  Multimedia  instruction  was  effective  across  all  genders,  races  and  learning  styles.  As  the  authors 
concluded,  “convenience,  flexibility,  and  simulation  of  interactive  multimedia  allows  learners  to  learn  better  and 
faster  compared  to  learning  with  human  teachers  in  language  proficiency  learning”  (p.87). 
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However,  incorporating  various  media  into  computerized  testing  and  making  it  interactive  is  still  new. 
Much  research  is  needed  to  understand  how  to  make  the  testing  not  only  an  evaluation  tool  but  also  a learning  tool. 
Because  the  making  and  using  of  computerized  testing  requires  the  knowledge  of  advanced  computer  knowledge. 
The  challenge  is  how  to  make  it  possible  for  any  instructors,  who  are  interested,  to  use  such  tests  in  their  teaching. 

Purpose  of  the  Project 

The  purpose  of  the  project  is  (1)  to  develop  virtual  multimedia  examination  formats  for  use  in  sciences 
and  other  disciplines;  and  (2)  to  evaluate  the  use  of  such  formats  in  classrooms,  especially  examining  students' 
attitude  towards  being  evaluated  by  such  multimedia  exams,  and  instructors'  attitude  for  using  it. 

This  presentation  will  describe  the  project,  but  will  focus  on  the  field-testing  of  the  use  of  such  multimedia 
exam  formats  in  actual  courses. 

Project  Description 

Given  the  challenges  of  developing  interactive  evaluation  tools  to  better  assess  learning  and  improve 
teaching,  and  to  make  such  tools  available  to  instructors  in  sciences  to  use  in  their  courses,  a group  of  faculty  and 
students  at  the  University  of  Texas  - Austin,  led  by  the  fourth  author  of  this  presentation  started  to  develop 
multimedia  examination  templates  for  use  in  sciences.  This  is  a National  Science  Foundation  sponsored  project.  It 
is  currently  in  its  second  year  of  implementation.  The  project  is  to  use  recent  advances  in  digital  technology  to 
utilize  the  full  range  of  color  imageiy,  video,  two-  and  three-dimensional  animations,  and  sound  so  as  to  produce 
rich  and  interactive  evaluations.  Because  the  undergraduate  teaching  in  various  sciences  often  shares  the  use  of 
similar  types  of  student  performance  assessments  (e.g.  laboratoiy  assignments,  quizzes,  mid-term  and  final  exams), 
it  is  possible  to  produce  a series  of  multimedia  exam  templates  that  can  be  customized  for  use  across  disciplines. 
Instructors  using  these  templates  can  "copy,  cut,  and  paste"  their  questions  into  the  formats,  add  multimedia 
materials,  and  utilize  the  interactive  question  formats  to  make  their  own  exams. 

The  Virtual  Multimedia  Exam  Templates 

The  virtual  exam  templates  (VExam)  consists  of  three  programs;  (1)  Exam  Builder,  (2)  Exam  Giver;  and 
(3)  Exam  Reviewer.  Exam  Builder  allows  instructors,  regardless  of  their  computer  experiences,  to  build  their 
customized  exams.  It  guides  the  user  through  exam  construction  in  a step  by  step  manner.  Exam  Giver  allows 
students  to  take  the  computerized  exam  his  or  her  instructor  has  created.  Exam  Reviewer  helps  students  review 
what  he  or  she  has  missed  in  the  exams  and  provides  study  suggestions. 

The  exams  created  with  the  Exam  Builder  shares  similar  characteristics  to  a traditional  paper-pencil  exam 
in  that  it  can  contain  the  same  content  and  sequence  as  a paper  exam,  and  use  the  same  common  testing  formats 
such  as  multiple  choice,  true/false  and  matching  items.  However,  these  new  exam  formats  have  some  innovative 
features  that  will  be  hard  to  implement  in  the  paper  exams.  First,  the  presentation  of  questions  is  greatly  enhanced 
through  the  use  of  images,  video,  animation  and  sound.  The  use  of  multimedia  is  particularly  attractive  to  visual 
learners  who  need  assistance  to  understand  abstract  concepts.  The  multimedia  elements,  directly  related  to  the 
content  of  the  questions,  can  aid  the  learner  in  understanding  the  questions,  and  enhance  learners'  ability  to  recall 
the  information  necessaiy  to  answer  the  questions.  The  multiple  representations  of  each  question,  through  verbal 
and  non-verbal  stimuli,  can  accommodate  learners  with  different  learning  styles.  This  is  strongly  supported  by  the 
dual  code  theoiy,  which  suggests  that  the  retrieval  of  information  is  facilitated  through  multiple  stimuli  that  can 
cue  the  right  information  (Ormrod,  1995). 

Another  important  feature  of  this  new  exam  format  is  that  it  allows  a more  dynamic  and  interactive  type 
of  questioning.  The  questions  can  present  real  world  situations,  where  the  student  is  asked  to  make  observations 
and  measurements,  collect  data,  form  hypothesis  and,  solve  a problem.  Students  are  evaluated  both  on  the  end 
product,  as  well  as  on  the  procedure  that  the  student  has  followed.  The  goal  of  these  questions  is,  therefore,  not 
only  for  the  student  to  reach  a solution,  but  also  to  learn  from  the  whole  problem-solving  process. 
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In  addition,  Exam  builder  allows  instructors  to  generate  random  questions,  alternative  forms  of  the  same 
exam  to  ensure  security  and  fairness  of  the  exams  for  all  students.  Once  a test  is  over,  scores  are  calculated  and 
displayed  immediately  for  the  students.  Correct  answers  are  explained.  Though  students  cannot  retake  the  same 
exam  after  viewing  the  answers,  they  can  review  the  content,  exam  formats  and  get  ready  for  the  next  exam.  In 
short,  such  exams  are  developed  not  just  to  evaluate  students'  performance,  but  more  importantly,  assist  students' 
learning  as  the  ultimate  goal. 

Field-Testing 

Research  Questions 

After  several  circles  of  the  formative  evaluation,  the  VExam  is  being  used  in  several  courses  at  the 
University  of  Texas-Austin.  Evaluating  the  efficacy  of  using  these  exams  as  a primary  assessment  tool  focuses  on 
the  following  research  questions: 

(1 ) Are  these  multimedia  exams  equivalent  to  the  paper  and  pencil  exams  in  terms  of  the  content? 

(2)  What  are  students'  as  well  as  instructors'  attitudes  toward  using  them? 

(3)  Does  students'  computer  experience  affect  their  use  of  the  exams? 

(4)  In  what  way  does  the  use  of  such  exams  influence  learning  and  teaching? 

Method 


The  field-testing  reported  here  consists  of  two  phases:  (1)  using  it  in  two  different  science  courses  where 
instructors  implemented  the  exam  templates  in  a bit  different  way;  and  (2)  using  it  as  the  assessment  for  an  online 
self-paced  science  course.  The  evaluation  examined  students'  as  well  as  instructors'  perspectives.  Questionnaires 
and  interviews  are  the  two  primary  data  sources.  The  actual  exams,  their  content  and  formats,  were  also  studied. 

Phase  L The  templates  were  used  in  two  different  undergraduate  science  courses  from  fall  98  to  spring 
99.  One  instructor  is  relatively  new  to  the  multimedia  technology  and  had  used  graphics  in  the  teaching.  Another 
instructor  is  an  experienced  multimedia  user,  who  has  used  all  media  formats  in  the  teaching  including  video, 
graphics,  animation  and  sound.  Approximately  120  participants  returned  the  questionnaire,  who  represented 
several  science  disciplines  and  from  freshmen  to  seniors. 

A questionnaire  was  devised  to  examine  students'  anxiety  and  attitudes  toward  being  "tested"  using  the 
multimedia  exams.  It  consists  of  25  items  on  a 5-point  Likert-scale  with  1 being  "I  strongly  disagree"  and  5 being 
"I  strongly  agree".  These  25  items  address  three  issues  related  with  multimedia  exams:  anxiety  towards  multimedia 
exams  (9  items),  attitudes  towards  taking  multimedia  exams  (8  items)  and  attitudes  towards  multimedia  and 
learning.  The  items  regarding  anxiety  towards  multimedia  exams  were  based  upon  a modified  version  of 
Spielberger's  self-evaluation  anxiety  scale  (Reed  & Palumbo,  1 987).  The  attitude  items  were  based  upon  Richards, 
Johnson,  and  Johnson's  computer  attitude  scale  (1986).  Examples  include  “I  feel  upset  when  I take  a multimedia 
exam”,  “I  had  difficulties  completing  the  exam  because  it  was  a multimedia  exam”,  and  “Multimedia  exam  will 
enhance  my  performance."  The  questionnaire  was  examined  by  three  experts  in  instructional  technology  for 
content  validity  and  pilot-tested  by  a group  of  five  students.  Comments  and  suggestions  were  incorporated.  Six 
open-ended  questions  were  used  to  ask  the  participants  to  express  their  likes  or  dislikes  of  multimedia  exams,  and 
to  compare  the  multimedia  exams  with  the  traditional  paper-pencil  exams.  Sample  questions  were  “How  do  you 
like  or  dislike  taking  a multimedia  exam”,  or  "Which  part  of  the  multimedia  did  you  like  the  most?  Why?" 
Students'  computer  prior  experience  was  also  surveyed  and  grouped  into  novice,  intermediate  and  expert  users 
according  to  their  experiences. 

Interviews  were  conducted  with  a randomly  selected  group  of  students  about  their  opinions  of  the 
multimedia  exams.  Three  questions  were  used  to  start  the  interviews,  but  the  researchers  made  an  effort  to  keep  the 
interview  process  open,  allowing  students  expressing  their  opinions  freely.  Follow-up  questions  were  used  if  further 
explanations  were  needed.  An  extensive  interview  was  conducted  with  the  instructors  using  10  questions. 

Phase  IL  In  the  fall  99  and  spring  00,  the  multimedia  exam  templates  are  being  used  to  create  virtual 
exams  for  an  online  self-paced  course  (http://www.dla.utexas.edu/depts/anthro/courses/99fall/SP301/).  This  self- 
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paced  course  uses  the  instructor  created  multimedia  CD-ROM  as  an  important  source  for  instructional  materials. 
Students  study  the  materials  in  their  own  time,  and  get  evaluated  about  their  knowledge  through  quizzes  and 
exams  created  using  VExam  programs  at  a time  that  is  convenient  to  them.  Review  sessions  and  Q&A  sessions  are 
provided  on  a regular  basis  throughout  the  semester.  Because  of  the  self-paced  nature  of  the  course  and  the  use  of 
the  multimedia  CD-ROM  as  a primary  teaching  tool,  the  use  of  multimedia  exams  in  this  case  reflects  more  closely 
what  is  being  taught.  The  questionnaire  has  been  revised  based  upon  the  results  from  Phase  I and  is  being 
incorporated  into  an  online  form.  Interviews  will  be  conducted  with  the  instructor  and  the  students.  In  addition, 
observations  of  the  review  sessions  and  the  testing  sessions  will  be  made.  At  present,  data  for  Phase  II  are  being 
collected  and  analyzed. 

Results  and  Discussion 

Descriptive  analyses  were  conducted  for  the  questionnaire.  The  interview  data  were  analyzed  following 
the  framework  by  Miles  and  Huberman  (1994).  Results  from  Phase  I are  reported  here. 

Table  1.  Mean  scores  for  students'  anxiety  and  attitudes  towards  the  multimedia  exam  format 


Computer  Expertise 

Anxiety* 

* 

(1  to  5) 

Gender 

Attitude* 
(1  to  5) 

Anxiety** 
(1  to  5) 

Novices 

3.32 

2.87 

Male 

3.65 

2.31 

Intermediate 

3.58 

2.51 

Female 

3.48 

2.68 

Experts 

3.83 

2.46 

(*  = The  higher  the  number,  the  better  the  attitude 
**  = The  lower  the  number,  the  lower  the  anxiety) 

Table  2.  Multimedia  Exam  Vs  Paper  Exam  Preference 


Prefer 

Multimedia 

Prefer 

Paper 

Either 

No  Opinion 

Novices 

41.17% 

29.41% 

29.41% 

0% 

Intermediate 

43.58% 

30.76% 

20.51% 

5.1% 

Experts 

66.66% 

33.33% 

0% 

0% 

Male 

42.10% 

26.31% 

26.31% 

5.26% 

Female 

45% 

33.33% 

18.33% 

3.33% 

Typical  students'  comments  from  the  interviews  on  their  attitudes  and  anxiety  toward  the  multimedia 
exams  included: 

• I like  multimedia  exam  better,  because  it  offers  the  chance  to  get  instant  feedback  on  the  test  results. 

• Pictures  and  animations  can  give  you  clues  when  you  are  trying  to  remember  something. 

• Multimedia  exam  appeals  to  me  visually  and  therefore,  seems  easier,  although  the  material  may  not 
be.  It's  more  accessible, 

• I prefer  multimedia  exam.  It  seems  to  be  easier  and  more  interesting  to  use. 

• I like  it,  because  it  makes  test  taking  and  grading  quicker  and  easier. 
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• I like  the  multimedia  environment,  it’s  less  stressful  and  that  helps  resolve  test  anxiety. 

The  following  are  typical  students’  comments  on  the  interactive  elements  built  in  the  exams: 

• Being  able  to  manipulate  things,  take  measurements  and  things  like  that  where  you  wouldn’t  be  able 
to  do  on  a piece  of  paper. 

• [I  liked]  The  interactive  part  as  opposed  to  the  Multiple  choice  part.  I liked  the  fact  that  I was  able  to 
take  measurements,  rotate  things  and  see  them  from  different  angles.  I also  liked  plotting  a graph, 
because  you  were  able  to  change  your  answers,  see  the  different  possibilities  and  estimate  which  one 
is  the  correct. 

• [Do  you  like  the  multimedia  exam?]  Yes,  because  like  in  the  lab  and  in  the  class  we  use  these 
materials  and  we  have  these  3D  models.  And  then  the  professor  uses  the  same  things  in  the  exam. 
And  it's  like  another  way  to  help  you  remember. 

• I like  the  interactive  part,  where  there  was  this  whole  case  that  you  had  to  work  on  and  do  different 
stuff,  collect  and  analyze  data...  And  each  part  builds  on  the  previous  one.  And  the  good  thing  is  let's 
say  you  missed  one  question  and  then  you  move  to  the  next  part  and  you  have  to  use  the  results  of  that 
questions,  it  will  give  you  the  correct  one  to  move  from  there.  I think  that's  good,  because  you  actually 
learn  something  from  the  test.  You  do  something  wrong,  but  then  you  realize  why  it's  wrong. 

• Yes,  because  I spent  less  [time]  remembering  how  something  exactly  looks  like  and  I was  focus  [ing] 
more  what  its  feature[s]  mean  so  that  when  it  comes  to  application  I can  use  that  instead  of  spending 
ten  minutes  trying  to  remember  what  it  looks  like. 

The  interviews  with  the  instructors  revealed  that  though  the  instructors  recognized  that  the  templates 
could  provide  the  same  content  and  sequence  as  the  paper  exams,  they  chose  to  take  advantage  of  the  multimedia 
and  interactive  features  made  possible  by  the  new  formats.  The  multimedia  exams  were  enhanced  versions  of  the 
paper  exams  according  to  the  instructors.  Though  the  multimedia  templates  have  made  it  very  easy  for  instructors 
to  use,  regardless  of  their  computer  experience,  it  still  takes  quite  a bit  of  time  to  assemble  and  create  all  media 
elements.  This  reinforces  the  concept  that  technology  has  provided  enormous  new  possibilities,  but  it  does  not 
necessarily  has  made  our  life  easier. 

The  quantitative  and  qualitative  data  suggested  the  following: 

1 . The  overall  attitudes  toward  the  use  of  the  multimedia  exams  as  the  primary  evaluation  method  have 
been  positive.  Students'  computer  anxiety,  regardless  of  their  prior  computer  experiences,  has  been 
relatively  low. 

2.  Students  with  more  computer  experience  have  a greater  preference  toward  using  the  multimedia 
exams. 

3.  Gender  does  not  seem  to  be  a factor  in  differentiating  the  opinions. 

4.  The  use  of  multimedia  elements,  especially  3D  modeling  and  animation,  and  immediate  scoring  are 
important  reasons  for  students  to  like  multimedia  exams. 

5.  The  interactive  nature  of  certain  questions,  allowing  students  to  manipulate  and  analyze  the  data, 
focus  on  solving  problems  rather  than  recalling  facts,  is  especially  well-liked  by  the  students. 

However,  some  students  expressed  negative  attitudes  towards  multimedia  exams  and  showed  preference 
towards  the  paper-pencil  exams.  There  were  three  main  reasons  for  this  negative  altitude: 

(1)  valuable  time  being  lost  to  scan  the  test.  It  was  more  difficult  and  time-consuming  to  go  back  and  forth  on 
the  computer  to  get  an  idea  of  the  whole  test  and  review  their  answers.  One  student  said,  ’’With  the 
multimedia  exam  you  can  see  one  question  at  a time,  whereas  with  a paper  exam  you  can  scan  over  all 
questions  and  get  a feel  for  the  test.” 

(2)  the  possibility  that  everyone  could  look  on  the  screen  while  taking  the  test,  and  see  the  score  at  the  end. 
Some  students  indicated  it  was  stressful  and  tiring  to  read  the  computer  screens. 

(3)  the  dislike  of  computer.  Though  most  of  the  participating  college  students  like  and  use  computers  as  part 
of  their  college  life,  some  do  not,  as  one  said,  ”I  don’t  like  computers.  I’m  not  into  technology  at  all.  I still 


use  a typewriter  for  my  papers."  Some  students  mentioned  that  they  were  more  used  to,  therefore  more 
comfortable  to,  paper-and-pencil  exams. 

Such  negative  feelings  about  the  multimedia  exams  indicated  the  following: 

• Using  multimedia  exams  is  a relatively  new  way  of  assessment.  Though  colleges  and  universities  are  the 
best  places  to  implement  them,  sample  tests  and  practices  of  using  the  new  formats  should  be  made 
available  for  students  to  get  familiar  with.  Students  should  make  an  effort  to  practice  using  the  new 
format. 

• The  multimedia  templates  being  developed  can  be  improved  and  enhanced  to  incorporate  students’ 
suggestions. 

• The  actual  administration  of  the  tests  can  be  further  examined  to  ensure  testing  security  and  fairness. 

• The  effective  use  of  such  new  assessment  depends  partly  on  instructors’  willingness  to  try  new  things  and 
their  experience  in  using  technology  in  instruction. 

The  advances  in  technologies  and  possibilities  made  available  through  internet  and  multimedia  force  us  to 
reexamine  the  ways  we  teach,  we  learn,  and  we  evaluate.  With  more  and  more  courses  being  offered  online, 
computerized  testing  is  a logical  way  for  evaluation.  Examining  how  to  make  testing  as  a part  of  the  learning  is  an 
important  challenge.  After  all,  the  goal  of  evaluation  is  not  to  merely  test  students,  but  to  help  them  learn  as  well. 

Page  limit  of  the  conference  proposals  prevents  elaborate  presentation  and  discussion  of  the  results,  which  will 
be  shared  more  in  detail  at  the  presentation.  The  VExam  will  also  be  demonstrated  to  the  audience. 
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Abstract:  It  has  been  demonstrated  that  an  effective  strategy  for  constructing  meaning  and 
knowledge  about  a particular  subject  area  is  in  the  design  and  production  of  subject-specific 
educational  resources  - particularly  computer-based  resources  (Jonassen  and  Reeves,  1996).  In  a 
continuing  effort  to  explore  the  effective  use  of  computer-based  technologies  in  the  teaching  and 
learning  experience,  the  Faculty  of  Education  at  the  University  of  Wollongong  implemented  a 
'learner  as  designer-producer'  model  in  a senior  year  physical  and  health  education  course.  A 
number  of  factors  - limited  experience  using  computers,  restricted  time  available  to  access 
production  equipment  and  competing  priorities  related  to  the  final  year  of  study  — provided  a 
challenge  to  the  pedagogical  re-conceptualisation  of  the  course  to  include  such  a model.  However, 
preliminary  analysis  of  the  outcomes  of  the  implementation  proved  encouraging  and  has  informed 
further  developments. 


Background 

It  has  been  demonstrated  that  an  effective  strategy  for  constructing  meaning  and  knowledge  about  a particular 
subject  area  is  in  the  design  of  subject-specific  educational  resources  - particularly  computer-based  resources 
(Jonassen  and  Reeves,  1996).  In  this  concept  of 'learner  as  designer',  the  learner  is  engaged  in  a meaningful  task 
where  important  aspects  of  the  knowledge  construction  process  such  as  analysis  and  reflection  are  facilitated. 
Hedberg,  Brown  and  Arrighi  (1997)  have  further  explored  the  impact  of  such  pedagogical  strategies  in  the  concept 
of 'learner  as  producer'.  When  the  learner  is  involved  in  the  production  of  interactive  hypermedia-based  educational 
resources  they  are  focused  on  learning  about  such  production  as  a body  of  knowledge  with  an  accompanying  set  of 
skills.  It  follows  then  that  when  the  learner  is  involved  in  both  design  and  production  aspects  of  hypermedia-based 
educational  resource  development  they  actively  explore  concepts  related  to  the  subject  matter,  hypermedia 
production  and  the  related  skill  set. 

In  a continuing  effort  to  explore  the  effective  use  of  computer-based  technologies  in  the  teaching  and  learning 
experience,  the  Faculty  of  Education  at  the  University  of  Wollongong  (in  New  South  Wales,  Australia)  has 
implemented  a 'learner  as  designer-producer'  model  in  its  senior  year  Advanced  Teaching  and  Learning  Studies. 
The  goal  of  this  subject  is  to  foster  the  importance  of  professional  development  for  teacher  trainees  in  their  final  year 
of  study  in  a four-year  Bachelor  of  Education  (Physical  and  Health  Education)  degree.  The  emphasis  is  on  the  need 
to  react  to  change,  to  continually  expand  their  repertoire  of  teaching  skills  and  to  develop  a personal  teaching 
philosophy.  Among  the  specific  objectives  for  the  subject  are  the  analysis  of  senior  secondary  physical  and  health 
education  related  syllabuses  and  the  ability  to  translate  the  policies  into  teaching  and  learning  environments  and  the 
critical  examination  of  the  use  of  technology  in  the  physical  and  health  education  classroom. 

The  delivery  of  this  subject  for  the  1999  academic  year  coincided  with  the  New  South  Wales  Board  of  Studies 
circulation  of  a proposed  senior  secondary  school  syllabus,  which  was  scheduled  to  be  released  for  use  in  next 
school  year.  Thus,  students  enrolled  in  this  subject  would  likely  be  securing  teaching  positions  within  which  they 
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would  be  required  to  have  in-depth  knowledge  of  and  ability  to  implement  the  contents  of  this  draft  syllabus.  At  the 
same  time  curriculum  was  being  re-examined,  employers  were  increasingly  expecting  teachers  to  be  skilled  in  both 
the  administrative  and  pedagogical  use  of  computer-based  technologies  (Ministerial  Advisory  Council  on  the 
Quality  of  Teaching,  1998). 

Thus,  it  was  timely  to  implement  an  educational  innovation  within  Advanced  Teaching  and  Learning  Studies  with 
the  investigation  of  the  impact  of  learner  as  designer-producer  model.  The  focus  of  this  innovation  was  an 
assessment  task  which  involved  the  design  and  production  of  a Web  site  with  embedded  learning  activities  which 
related  to  the  proposed  syllabus.  For  this  task,  students  worked  in  collaborative  teams. 

The  literature  regarding  collaborative  approaches  to  learning  was  convincing.  Henri  (1996)  explained  that  group 
learning  qualifies  as  cooperative  or  collaborative  when  the  process  is  structured  to  include:  interdependent  learner 
interaction;  commitment  to  the  learning  of  each  member  of  group;  commitment  to  fulfill  assigned  group  functions; 
use  of  the  appropriate  social  and  interpersonal  skills  to  stimulate  cooperation;  and,  continual  revising  of  the  group 
functioning  process.  Metheny  and  Metheny  (1997)  found  that  such  learning  approaches  help  students  overcome 
their  fear  of  technical  courses  while  helping  them  prepare  for  today’s  collaborative  workplace.  In  reviewing  the 
research,  Slavin  (1989)  found  cooperative  learning  to  be  statistically  significant  in  facilitating  academic  achievement 
in  57%  of  more  than  70  studies.  34  of  41  studies  that  involved  a Student  Team  Learning  approach  found 
significantly  positive  achievement  results  (Slavin,  1989).  Cinelli  et  al  (1994)  argued  that  the  application  of 
cooperative  learning  was  especially  important  for  the  school  health  educator  who,  in  the  context  of  a comprehensive 
school  health  program,  should  employ  a range  of  classroom  activities  to  maintain,  reinforce,  and  enhance  the  health 
attitudes,  practices,  and  skills  of  children  and  youth. 

Implementation 

In  this  investigation,  students  formed  teams  of  three  members  and  choose  from  among  list  of  topics  covered  in  the 
proposed  syllabus  for  the  focus  of  their  Web  site.  Assessment  of  the  Web  sites  was  based  on: 

■ Content  (e.g.,  relevant  to  the  syllabus  topic,  current  and  accurate,  at  the  appropriate  depth  for  the  proposed 
students). 

■ Learner  engagement  (e.g.,  the  site  included  learning  activities,  the  activities  related  directly  to  the  syllabus 
topic,  the  activities  stimulated  the  learner  to  think  critically,  and  a feedback  mechanism  for  learners  was 
evident). 

■ Design  and  functionality  (e.g.,  easy  to  use  and  well  organized,  users  able  to  easily  navigate  site,  the  site  was 
aesthetically  pleasing  and  professionally  presented,  the  reading  level  and  choice  of  graphics  was  appropriate  for 
senior  secondary  students  and,  all  links  work  as  intended). 

Intensive  hands-on  workshops  were  held  to  introduce  students  to  the  range  of  skills  necessary  for  Web  site 
development.  An  authoring  tool,  Claris  Home  Page,  was  used  in  the  workshops  for  Web  site  construction. 
Workshops  covered  the  following: 

1 . Site  map  development,  file  management,  Web  page  creation,  navigation  systems,  and  inserting  an  image. 

2.  Creating  internal  and  external  site  links,  page  backgrounds,  and  using  tables. 

3.  Creating  and  editing  images  and  forms  of  interaction. 

Due  to  the  technical  resources  available  (i.e.,  software  and  hardware  limitations)  an  alternating  peer-tutoring  model 
was  implemented.  Each  week,  one  member  of  the  team  attended  the  skill  development  workshop.  A paper-based 
resource  was  distributed  to  each  workshop  participant  that  related  to  the  skills  covered.  This  resource  was  also 
available  on  the  subject  Web  site.  Immediately  following  the  workshop,  the  team  representative  peer-tutored  the  rest 
of  their  team  members  on  the  skills  they  had  just  acquired. 

Jacques  (1991)  has  suggested  that  peer  tutoring  allows  students  to  be  active  learners,  feel  integrated  into  the  delivery 
of  the  subject  and  increases  cooperation,  motivation  and  self-esteem.  Peer  tutoring  is  a strategy  that  is  often 
employed  in  school  physical  and  health  education  classes.  Thus,  this  investigation  allowed  for  the  modeling  of 
multiple  strategies  that  the  teacher  trainees  could  implement  in  their  future  classroom  practices. 
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Methodology 

A mixed-mode  methodology  was  utilised  to  evaluate  the  effectiveness  of  the  learning  experience  for  students  from 
both  a quantitative  and  qualitative  perspective.  The  data  collection  strategies  were  varied  and  included: 

• Pre-  and  post-implementation  surveys  of  student  skills  and  comfort  level  associated  with  using  computers. 

• Student  journals  and  self-analysis  of  the  experience. 

• Post-implementation  interviews  with  representatives  of  the  student  teams. 

Outcomes 

Overcoming  the  barriers  of  limited  experience  using  computers,  restricted  time  available  to  access  production 
equipment  and  competing  priorities  related  to  the  final  year  of  study,  the  students  rose  to  this  new  learning 
challenge. 

Of  the  70  students  enrolled  in  the  subject,  the  vast  majority  self-reported  having  little  or  no  experience  using 
computers  prior  to  engaging  in  this  learning  activity  (only  9%  reported  having  "a  lot  of  experience  with 
computers”).  For  most  students,  their  computer  skills  were  limited  to  the  components  of  the  integrated  software 
package  (ClairsWorks)  for  which  they  engaged  in  skill  development  during  their  first  year  of  study  (e.g.,  word 
processing,  spreadsheets,  databases,  draw). 

Students  reported  limited  experience  using  the  Internet  (only  6%  reported  having  "a  lot  of  experience  with  the 
Internet").  In  the  main,  they  considered  their  Internet-related  skills  in  terms  of  sending  and  receiving  e-mail  and 
searching  for  and/or  accessing  information  on  the  World  Wide  Web.  Only  two  students  reported  that  they  had  been 
involved  in  producing  a Web  site  prior  to  engaging  in  this  learning  activity. 

The  final  Web  sites  submitted  by  the  learning  teams  demonstrated  that  students  grasped  the  basic  skills  of  Web  site 
construction  (see  Figure  1).  All  Web  sites  were  functional  and  with  a range  of  aesthetic  designs.  Most  importantly, 
all  sites  included  learning  activities  that  were  designed  to  facilitate  critical  thinking  - an  objective  inherent  in  the 
proposed  syllabus. 


Figure  1 : Example  of  student  team  developed  Web  site. 
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Prior  to  engaging  in  the  learning  experience  students  were  somewhat  split  in  their  comfort  level  in  terms  of  using 
computers.  However,  data  collected  post-implementation  demonstrated  that  students  had  moved  along  the 
continuum  with  85%  reporting  that  they  felt  “comfortable  about  using  computers.”  Importantly,  after  the  learning 
experience,  75%  of  the  students  reported  that  they  felt  “comfortable  about  using  computers  in  the  teaching 
environment.” 

Students  were  also  asked  to  respond  to  a number  of  statements  about  the  activity  and  related  assessment  task.  Table 
1 provides  an  overview  of  the  findings: 


Table  1:  Student  post-implementation  perceptions  regarding  learning  activity  and  assessment  task. 

Statement 

Strongly 

Agree 

Agree 

Disagree 

Strongly 

Disagree 

No 

Response 

The  workshops  and  the  assessment  task  were 
helpful  in  increasing  my  understanding  of  how 
technologies,  specifically  Web-based 

technology,  can  be  used  for  Personal 
Development,  Health  and  Physical  Education 
teaching  and  learning 

18% 

73% 

9% 

-- 

— 

The  assessment  task  was  helpful  in  facilitating 
my  skills  in  developing  educational  Web  sites. 

34% 

60% 

4% 

2% 

The  assessment  task  was  helpful  in  increasing 
my  understanding  and  facilitating  my  analysis 
of  the  draft  senior  syllabus. 

6% 

56% 

37% 

- 

1% 

The  peer  teaching  model  was  an  effective 
strategy  for  facilitating  my  understanding  of 
Web  site  construction. 

27% 

47% 

25% 

-- 

1% 

This  survey  data,  open-ended  comments  on  the  survey  and  the  post-implementation  interviews  found  that  students 
perceived  value  in  the  learning  activity.  One  student  commented  that  the  task  required  them  to 

,,  .critically  think  and  develop  a practical  instrument  that  could  be  used  for  our  teaching.  This  was 
much  better  than  a more  theoretical  based  approach  that  only  looked  at  potential  uses  of  technology 
in  teaching  senior  classes  OR  playing  around  with  existing  sites  or  CD  Roms.  It  makes  it  much  more 
relevant  and  gave  us  good  practice  of  working  with  and  seeing  what  computers  can  do. 

This  was  further  supported  by  another  student  who  commented, 

/ guess  I really  have  never  thought  of  using  computer  technology  in  the  PDHPE  [Personal 
Development,  Health,  and  Physical  Educationy  classroom.  The  use  of  computers  for  use  other  than 
writing  up  assignments  is  new  to  me.  The  web-based  learning  activity  broadened  my  thinking  in  terms 
of  learning  strategies. 

Some  students  felt  the  task  was  time-consuming  and  some  did  report  frustrations  with  using  the  technology. 
Students  reported  both  positive  and  negative  attitudes  toward  the  experiential  learning  of  group  work  and  peer 
tutoring.  One  student  explained,  “I  thought  this  was  very  good,  it  placed  some  responsibility  and  sense  of  ownership 
with  the  peer  group/tutor.  However,  it  would  have  been  good  if  this  process  continued  for  a longer  period.”  Some 
expressed  their  lack  of  comfort  in  playing  the  role  of  the  tutor.  For  example,  one  student  noted,  “...  but  I wasn’t 
confident  in  teaching  my  group  in  case  I made  a mistake  and  didn’t  know  what  to  do  then  I would  let  the  rest  of  the 
group  down  because  then  we  all  wouldn’t  have  an  idea  of  those  skills. 
However,  others  held  more  positive  perceptions  about  the  strategy,  for  example,  “I  thought  this  was  a really  good 
idea.  It  made  me  really  pay  attention  and  not  rely  on  someone  else  to  listen.  I liked  this  strategy  and  it  worked  well 
and  I know  I’ll  use  it  in  my  teaching.” 
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Students  felt  it  important  that  they  were  aware  of  the  difficulties  of  team  management  processes  that  their  future 
students  might  encounter  when  such  strategies  were  employed.  Students  appreciated  the  issues  that  came  from 
sharing  the  experience  of  learning  a skill  (i.e.,  Web  site  construction)  that  was  completely  new  to  them. 

Concluding  Remarks 

The  encouraging  preliminary  results  from  this  first  implementation  of  the  learner  as  designer-producer  model  has 
provided  scope  for  refining  future  implementations  within  the  course.  Additionally,  it  has  provided  a basis  on  which 
to  re-conceptualise  the  pedagogical  strategies  that  might  effectively  integrate  information  and  communications 
technologies  in  physical  and  health  education. 
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Abstract:  While  ad  hoc,  course-specific  projects  have  allowed  early  adopters  to  explore  possible 
innovations  in  the  use  of  information  and  communication  technologies  in  facilitating  flexible 
learning  situations,  educational  institutions  are  now  exploring  more  integrated  strategies  to  such 
developments.  This  paper  describes  the  development  of  one  such  strategy  that  attempts  to  foster  a 
community-wide  approach  for  a group  of  professionals  coming  to  terms  with  the  most  effective 
way  to  utilise  technologies  - physical  and  health  educators.  The  Faculty  of  Education  at  University 
of  Wollongong  is  developing  an  on-line  learning  community  to  facilitate  the  pre- service  education 
and  continuing  professional  development  of  students,  faculty  and  practicing  teachers. 


Introduction 

Institutions  of  higher  education  in  Australia  are  involved  in  the  international  trend  to  consolidate  networked 
information  and  communication  technologies  (ICT),  particularly  the  World  Wide  Web,  into  their  teaching,  research 
and  community  outreach  endeavours.  Web-based  environments  can  create  meaningful  experiences  where  learning  is 
fostered  and  supported  (Khan,  1997). 

In  the  main,  these  Web-based  activities  are  particular  to  individual  courses  or  specific  research  projects  and  only 
infrequently  involve  an  organised  change  process  which  incorporates  multiple  activities  and  various  stakeholders 
(i.e.,  members  of  the  community)  with  a common  interest. 

The  challenge 

Within  the  physical  and  health  education  profession,  the  important  members  of  the  community  include  teacher 
educators,  practicing  teachers,  pre-services  teachers  (i.e.,  students),  the  professional  associations  and  employers  of 
teachers.  An  existing  example  of  partnership  amongst  the  members  of  this  community  is  the  model  for  teacher 
education  which  utilised  links  between  schools  and  universities  to  promote  the  communication  of  professional 
understanding  through  practicum  experience  (Goodlad,  1994).  However,  the  effectiveness  of  the  implementation  of 
this  partnership  model  has  been  the  discussion  of  recent  debate  and  calls  for  reform  (Ministerial  Advisory  Council 
on  the  Quality  of  Teaching  1998). 
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While  universities  are  encouraging  the  implementation  of  Web-based  technologies  into  teaching  and  learning 
practices,  employers  of  teachers  (in  this  context  specifically,  the  Department  of  Education  and  Training  New  South 
Wales)  are  expecting  their  teachers  to  have  the  skills  to  incorporate  educational  technologies  (including  Web-based 
technologies)  within  their  teaching  practices. 

Pre-service  educators  within  the  Physical  and  Health  Education  program  in  the  Faculty  of  Education  at  the 
University  of  Wollongong  recognise  the  importance  of  exploring  the  use  of  Web-based  technologies  in  their 
teaching.  Recent  experiences  have  proven  effective  (Lockyer,  Harper,  and  Patterson,  1999;  Lockyer,  Patterson,  and 
Harper,  1999;  Lockyer  and  Kerr,  In  Press).  However,  such  initiatives  have  been  implemented  on  an  ad  hoc  basis. 
As  the  Faculty  of  Education  at  the  University  of  Wollongong  evaluated  their  experiences,  it  became  clear  that  a 
conceptually  and  practically  organised  implementation  plan  which  included  specific  contact  with  the  broader 
profession  and  its  activities  was  necessary. 

It  has  been  noted  that  the  key  features  of  reciprocal  partnerships  include: 

• a recognition  of  the  interdependence  and  the  unique  contribution  of  the  various  parties; 

• constructive  and  imaginative  problem  solving;  a will  to  work  to  change  and  improve; 

• a working  relationship  which  permits  risk  taking;  a tolerance  for  ambiguity,  uncertainty  and  dilemmas; 

• joint  responsibility  for  the  planning,  implementation  and  evaluation  outcomes; 

• joint,  commensurable  benefits; 

• organisational  structures  which  will  facilitate  the  enactment  of  decisions; 

• appropriate  resourcing;  intercultural  understandings 
(Groundwater- Smith,  Parker,  and  Arthur,  1994). 

Thus,  the  concept  of  an  on-line  learning  community  for  physical  and  health  education  and  which  comprised  these 
key  features  was  bom.  Schwier  (1999)  explained  the  power  of  learning  communities  "resides  in  their  ability  to  take 
advantage  of,  and  in  some  cases  invent  a process  for  exchanging  ideas  and  learning  collectively"  (p.  282).  While  the 
community  of  physical  and  health  education  professionals  have  traditionally  fostered  successful  processes  for  the 
exchange  of  ideas  and  mentoring  newcomers  into  the  community  of  practice,  change  fiscal  and  social  influences 
require  new  approaches  to  be  investigated.  The  use  of  technologies  in  sustaining  and  invigorating  the  community  is 
timely. 

There  now  exists  an  abundance  of  literature  to  inform  us  of  the  nature  of  virtual  communities.  Femback  and 
Thompson  help  us  to  understand  the  value  of  the  community  that  exists  in  cyberspace  (i.e.,  using  Internet 
technologies). 

Ideologically,  community  within  cyberspace  appears  to  emphasise  a shared  belief  in  the  principles  of 
free  speech,  individualism,  equality,  and  open  access. . . Experientially,  community  within  cyberspace 
emphasises  a community  of  interests,  usually  bounded  by  the  topic  under  discussion,  that  can  lead  to  a 
communal  spirit  and  apparent  social  bonding...  they  may  promote  action  that  is,  virtual  communities 
may  manifest  themselves  in  real  political  action,  such  as  educational  reform  ...  (Fembank  and 
Thompson,  quoted  from  hypertext  document) 

The  Concept 

An  on-line  learning  community  for  physical  and  health  education  has  been  conceptualised  to  incorporate  these  key 
features  of  a partnership. 

This  community  will  be  facilitated  by  a Web-based  environment  that  encompasses: 

• on-line  access  to  the  Physical  and  Health  Education  program  at  the  University  of  Wollongong; 

• physical  and  health  education  teaching  resources; 

• facilitation  of  professional  development  activities;  and, 

• access  to  physical  and  health  information  sources. 


636 


This  is  a collaborative  initiative  involving  lecturers,  students  and  practicing  teachers  (Figure  1).  The  development  of 
the  on-line  community  is  dependent  on  a student-centred  learning  model.  Students  propose  learning  contracts  which 
involve  collaborative  projects  that  will  add  to  the  resource  base  of  the  community.  It  is  envisioned  that  active 
participation  by  students  in  the  development  of  the  on-line  community  will  facilitate  a smoother  transition  into  their 
chosen  profession  — the  community  of  practice  of  teaching.  As  Bruffee  (1993)  argues,  collaborative  learning  in  the 
university  setting  provides  students  with  the  opportunity  to  speak  the  language  of  their  profession  and,  ’’speaking  the 
language  fluently  defines  membership  in  the  community”  (p.  130). 


Figure  1:  Key  participants  of  the  Physical  and  Health  Education  On-line  Learning  Community 


Through  their  participation  in  the  development  of  the  virtual  community,  physical  and  health  education  lecturers 
will  enhance  their  understanding  of  the  use  of  Web-based  technology  for  teaching  and  learning  within  the  discipline. 
This  translates  into  the  integration  of  such  concepts  into  the  curriculum  of  the  program.  As  Sherry  and  Wilson 
(1997)  suggest  ’’roles  and  concepts  of  teaching  and  learning  [should]  be  restructured"  as  the  Web  extends  the  walls 
of  the  traditional  classroom  (p.67). 

Practicing  teachers  are  in  integral  component  of  the  project.  Physical  and  health  education  teacher  will  develop 
skills  to  implement  Web-based  technologies  into  their  classroom  and  participate  in  the  collegial  exchange  of  ideas 
and  applications  of  the  innovations. 

Phase  1 - On-line  Practicum  Support 

The  initial  phase  of  the  project  has  focused  on  discussion  with  stakeholder  representatives;  the  design  of  the  Web- 
based  environment  which  will  support  the  community;  the  prioritisation  for  implementation  of  components, 
activities  or  features;  and  the  confirmation  of  learning  contracts  with  students  who  will  contribute  to  the 
development  of  the  community. 
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Building  on  the  existing  collaboration  between  participants  in  the  form  of  school-based  mentoring  and  practice 
teaching  for  pre-service  teachers,  a first  activity  to  be  explored  within  the  virtual  community  is  an  on-line  practicum. 

The  Bachelor  of  Education  (Physical  and  Health  Education  specialisation)  at  the  University  is  conducted  over  a 
four-year  program.  In  each  year  of  the  program,  students  engage  in  practice  teaching  experiences  which  involve 
local  schools,  supervising  teachers  and  University-based  faculty.  Anecdotal  information  gathered  from  practicum 
participants  (faculty,  pre-service  and  supervising  teachers)  has  called  for  greater  and  more  consistent  communication 
among  participants;  access  to  a variety  of  information  (related  to  the  program  specifically  and  to  the  teaching 
profession  generally)  and  access  to  resources  and  tools  (such  as  lesson  ideas). 

The  initiative  of  an  on-line  practicum  provides  a model  for  successive  implementations  within  the  community  — the 
first  iteration  explored  through  pilot  study  with  a sub-set  of  students  in  the  fourth  (i.e.,  final)  year  of  their  pre-service 
teaching  studies. 

Building  on  the  initial  anecdotal  feedback  regarding  the  practicum  experience,  a formalised  analysis  is  being 
undertaken  to  clearly  identify  the  needs  of  the  participants.  Data  collected  during  the  needs  analysis  will  assist  in 
identifying  the  resources  necessary  to  include  in  the  Web-based  environment  and,  most  importantly,  the  structure 
that  the  on-line  practicum  will  take.  As  suggested  by  Brehm  (1999)  it  is  critical  that  structure  and  expectations  of 
involvement  of  all  participants  are  effectively  designed  and  clearly  communicated. 

Data  collected  during  the  pilot  study  (in  the  form  of  journals,  logs  of  on-line  discussions,  and  post-implementation 
surveys  and  interviews)  will  provide  some  evaluative  indicators  and  a basis  for  consideration  of  extending  an  on-line 
practicum  component  throughout  each  of  the  four  years  of  the  program. 

Concluding  Remarks 

It  is  envisioned  that  this  on-line  community  will  evolve  to  encompass  increasing  avenues  for  collaboration  among 
the  participants.  The  community  is  structured  to  provide  an  exciting  action  research  environment  for  innovations  in 
both  initial  teacher  training  and  ongoing  professional  development  in  the  area  of  physical  and  health  education 
which  can  then  be  communicated  to  inform  similar  groups  initiatives. 
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Abstract.  This  paper  reviews  a distance  learning  project  implemented  at 
Massachusetts  College  of  Pharmacy  and  Health  Sciences,  a small  private  college,  to 
deliver  a nontraditional,  part-time  Doctor  of  Pharmacy  program  using  interactive  video 
conferencing  for  students  located  at  two  sites.  Although  it  was  not  a complete  match  to 
our  needs,  this  technology  was  chosen  in  response  to  a mandate  to  replace  videotape 
because  of  consistently  negative  feedback  related  to  lack  of  real-time  participation. 
There  was  both  significant  potential  and  considerable  risk  in  our  decision.  From  the 
start  we  had  difficulties  with  the  system  configuration,  lack  of  knowledgeable  technical 
support  from  the  vendor,  training  issues,  networking/connectivity  difficulties,  and 
overwhelming  environmental  problems  with  our  broadcasting  space.  What  we  learned 
during  a very  difficult  implementation  of  new  technology  is  worth  sharing  with  others 
who  may  be  considering  a synchronous  distance  learning  solution. 


Introduction 

Many  small  colleges  face  stress  and  frustration  in  attempting  to  implement  distance  learning  because 
some  or  all  of  the  prerequisites  for  success  are  missing  or  incomplete.  These  prerequisites  include 
comprehensive  analysis  and  planning,  adequate  funding,  staffing,  technical  support,  and  training.  Often  it 
is  difficult  to  secure  crucial  top-down  administrative  commitment  for  a quality  distance  learning  program, 
and/or  to  convince  administrators  to  invest  the  time  and  money  necessary  to  do  the  proper  amount  of 
research  and  preparation  before  a program  is  put  in  place  (see  Robbins  1997  & Wagschal  1998).  Studies 
show  that  venturing  into  distance  learning  without  establishing  these  key  elements  is  a risky  endeavor,  as 
outcomes  can  seriously  impact  the  reputation  of  the  institution  (see  Carr  2000  & Schneider  1999  & Weiss 
1999). 

Massachusetts  College  of  Pharmacy  and  Health  Sciences  (MCPHS)  has  been  in  exactly  this  situation  for 
the  past  1 6 months.  In  1 996  MCPHS  initiated  a non-traditional  Doctor  of  Pharmacy  program  geared  for 
practicing  pharmacists  who  could  not  commit  to  a full-time  program.  This  three-year,  part-time  program 
is  currently  offered  at  two  sites,  the  Boston  MCPHS  campus  and  an  affiliated  institution.  Western  New 
England  College  located  in  Springfield,  Massachusetts.  This  program  was  developed  in  response  to  the 
demands  of  alumni  throughout  New  England  and  it  was  initiated  without  full  administrative  support  for 
distance  learning.  For  the  first  two  years  of  the  program,  courses  were  delivered  live  at  the  Boston  campus 
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and  these  lectures  were  videotaped.  Videotapes  were  sent  to  the  Springfield  students,  who  also  met 
periodically  with  their  peers  for  interactive,  small  group  discussions.  This  method  of  course  delivery  was 
optimal  for  the  Boston-based  students,  but  it  was  not  well-received  at  the  distant  site.  In  mid-1998,  the 
director  of  the  program  received  administrative  support  and  funding  to  establish  video  conferencing 
between  the  two  sites,  and  the  technology  was  implemented  in  January  1 999.  Despite  many  obstacles  and 
substantial  frustration,  this  interactive  video  conferencing  program  is  functioning  and  improving.  We 
have  learned  a great  deal  about  doing  it  right  and  what  to  avoid. 

We  were  challenged  on  all  fronts,  including: 

• locating  various  funding  resources. 

• finding  a high-quality,  reliable  video  conferencing  system  for  reasonable  cost. 

• clearly  understanding  the  roles  of  the  ISDN  provider  and  the  system  vendor. 

• existing  essentially  as  an  unsolicited  beta  site,  since  it  was  later  discovered  that  the  system 
configuration  within  our  type  of  environment  was  the  first  for  our  vendor. 

• computer  networking/connectivity  problems. 

• lack  of  appropriately-designed  broadcasting  space. 

• lack  of  experienced  support  staff  (see  Boaz  et  al.  1999  & SURA  Video  Development  Initiative  1999). 

• Scarcity  (and  expense)  of  quality  training  in  course  preparation  (for  interactive,  student-centered 
distance  learning),  and  on-camera  presentation  skills  designed  for  college  faculty,  not  for  corporate 
trainers  (see  Boaz  et  al.  1999  & Parker  1999). 

Obstacles  were  encountered  from  the  start  of  installation,  through  system  testing  and  throughout 
implementation  of  the  first  broadcast  courses.  Some  of  these  issues  have  been  resolved  or  mitigated,  while 
other  solutions  for  improvement  are  still  in  progress. 


Program  Implementation 

Technical  Problems 

In  the  beginning  we  encountered  many  transmission  problems  because  of: 

■ our  technician's  lack  of  experience  with  this  type  of  equipment. 

■ phone  company  linemen’s  lack  of  ISDN  experience. 

■ Intel  technician’s  inexperience  supporting  a system  that  was  not  tested  in  a real-life  environment. 

■ limited,  expensive  options  for  transmitting  full-motion  video  (30  fps)  (see  Graves  1996). 

Because  of  this  unfortunate  deficiency  in  expertise  from  all  of  these  critical  sources  of  support,  those 
involved  with  the  program  at  MCPHS  during  the  early  stages  became  discouraged  with  the  technology 
(see  Boaz  et  al.  1999).  A major  reason  for  some  of  the  obstacles  we  faced  was  that  the  system  was  still  in 
the  beta  development  stage,  a fact  that  was  undisclosed  to  us  during  purchase  negotiations.  Technicians  at 
Intel  were  often  unable  to  help  us  because  this  was  their  first  encounter  with  many  of  the  problems  we 
confronted. 

Our  first  major  (and  easily  avoidable)  problem  occurred  two  weeks  into  the  program  when  we  experienced 
transmission  problems.  Our  system  is  designed  to  be  portable  and  originally  was  located  on  a rolling  cart 
with  the  ISDN  boxes  lying  loosely  beside  the  other  system  components  on  the  cart.  We  worked  tirelessly 
trying  to  pinpoint  the  failure  before  we  realized  that  the  problem  was  due  to  the  placement  of  the  ISDN 
boxes.  Because  the  boxes  were  not  in  a consistently  fixed  position,  any  movement  of  the  wires  and  boxes 
affected  the  transmission  signals.  Mounting  the  ISDN  boxes  on  a stable  platform  inside  a closet  where  the 
lines  run  in  from  the  street  alleviated  this  problem.  This  process  took  over  two  months  to  resolve  because 
the  vendor's  technical  support  didn't  know  what  to  do.  Since  that  time,  we  have  had  only  one  incidence  in 
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which  we  lost  video  during  a transmission.  That  problem  was  rectified  with  a simple  re-boot  of  the 
system. 

Technical  Speciflcations 

The  reasons  for  choosing  the  Intel  TeamStation  for  video  conferencing  included; 

■ Cost. 

■ Functionality. 

■ Ease  of  use. 

■ Interfacing  capabilities  with  the  Microsoft  NT  network  and  other  Microsoft  applications  (Word, 
Excel,  PowerPoint). 

The  current  cost  of  the  system,  which  ships  with  a Pentium  III  500  MHZ  processor  running  Microsoft’s 
Workstation  operating  system  and  Intel’s  proprietary  video  conferencing  software,  is  between  $12,000  to 
$16,000  per  unit,  and  two  units  are  necessary.  Two  years  ago  this  system  running  on  a Pentium  II 
233MHZ  processor  cost  $35,000.  As  a result  of  a 1999  agreement  between  Intel  and  PictureTel, 
PictureTel  is  now  the  vendor  for  the  TeamStation  product  line. 

Operating  and  routine  maintenance  of  the  system  requires  minimal  technical  knowledge.  The  system  is 
equipped  with  a Sony  video  camera  with  auto-focus  and  built-in  auto-tracking  and  motion  detection, 
which  allows  the  camera  to  follow  the  speaker  as  she  moves  about  the  room.  The  audio  system  is  hands- 
free and  full  duplex,  which  allows  simultaneous  speech  from  both  ends.  This  audio  system  is  optimally 
designed  for  an  8 to  10  foot  radius.  A third-party  sound  system  can  substantially  improve  the  audio  range 
of  the  system. 

An  Intel  technician  installed  the  software  and  upgraded  the  system  with  an  aggregator  board  to  allow  it  to 
transmit  at  384  Kbps,  which  was  the  vendor's  performance  claim.  The  system  was  originally  designed  to 
transmit  at  128  Kbps,  which  was  adequate,  but  not  considered  full-motion  video.  The  system  may  be  run 
in  the  128K  mode  to  reduce  the  ISDN  costs.  Bell  Atlantic  charges  75  cents  per  minute  per  line,  which 
amounts  to  a weekly  charge  to  the  program  of  about  $675,  since  actual  broadcast  time  for  course  activity  is 
five  hours  once  a week.  It  was  discovered  that  transmitting  at  this  rate  can  provide  adequate  video  and 
sound  with  minimal  loss  of  quality  if  the  presenter  significantly  limits  her  movements.  With  advances 
planned  for  Internet  II,  we  may  have  more  options  for  better  broadcast  quality  in  the  future  (see  Graves 
1996). 

Environmental  and  A/V  Media  Issues 

As  the  system  problems  were  resolved  or  mitigated,  a larger  deficiency  began  to  emerge  as  our  main 
obstacle  to  good  video  conferencing:  a poorly  designed  broadcasting  room  at  both  sites.  This  is  a critical 
operational  component  for  video  conferencing  (see  SURA  Video  Development  Initiative  1999).  The 
factors  that  detracted  from  effective  transmission  were: 

■ shape  and  size  of  room  including  ceiling  height. 

■ color  of  walls  and  furniture. 

■ lighting  systems  and  room-darkening  devices. 

■ audio  amplification  and  interference. 

■ placement  of  system  components. 

■ conformation/location  of  instructor  platform  and  general  access  to  instructional  devices/student  view. 


A consultant  was  hired  to  conduct  a full  analysis  of  the  environmental  problems  and  a complete, 
prioritized  report  was  submitted.  Some  recommendations  have  been  implemented  and  the  whole  program 
is  currently  in  review. 

Installation  of  a third-party  sound  system  improved  audio  range  at  both  sites.  Indirect  and  diffuse  lighting 
adaptations  were  also  made.  Another  major  improvement  was  accomplished  by  arranging  the  video 
devices  so  that  the  instructor  is  looking  at  both  audiences  at  the  same  time,  while  allowing  both  audiences 
a direct  view  of  the  instructor.  This  requires  that  the  camera  be  placed  directly  in  front  of  the  instructor’s 
designated  space  (suspended  in  some  way  from  the  ceiling,  or  mounted  within  the  local  audience).  Two 
video  displays  are  required:  one  behind  the  instructor  large  enough  for  the  local  audience  to  see  (we  use  a 
large  projection  screen)  and  a second  video  monitor  placed  in  front  of  the  instructor  and  behind  or  above 
the  local  audience. 

Limiting  the  instructor's  movement  to  a specific  zone  allows  for  much  more  reliable  audio  and  camera 
tracking.  The  system  components  were  relocated  within  the  broadcast  area  so  that  the  instructor  had  easy 
access  to  peripheral  devices,  such  as  the  document  camera,  to  effectively  shift  the  teaching  format  while 
maintaining  direct  contact  with  the  audience  (see  Zielinski  1998  & Parker  1999). 

We  learned  that  effective  communication  between  all  support  staff  and  faculty  is  a vital  requirement  in 
planning  and  developing  a successful  program  (see  Boaz  et  al.  1999  & Hardwick  1997).  For  example,  it  is 
very  important  that  the  staff  involved  with  environmental  and  audio-visual  media  work  closely  with  the 
computer  and  networking  technicians.  The  placement  of  ISDN  lines  may  limit  the  placement  of  the  video 
conferencing  equipment  and  peripheral  devices  within  the  broadcasting  space.  It  is  essential  that  those 
people  designing  and  locating  the  ATV  and  instructional  support  devices  work  closely  with  the  networking 
technicians  who  will  locate  the  ISDN  lines,  in  order  to  ensure  that  the  full  system  is  appropriately  situated 
in  the  space  for  optimum  teaching  activity. 


Faculty  Training  Issues 

Time  became  a major  issue  in  the  proper  implementation  of  the  video  conferencing  system.  Students  were 
so  displeased  with  the  existing  format  of  videotapes  that  directors  of  the  program  mandated  that  the  first 
instructional  broadcast  would  occur  in  less  than  five  months  after  the  equipment  was  chosen  and 
purchased.  This  proved  to  be  a woefully  inadequate  preparation  period  given  the  situation  at  MCPHS.  In 
August  of  1998  none  of  the  following  pre-requisites  for  interactive  video  conferencing  existed: 

■ A specially  designed  or  appropriately  adapted  room  for  broadcasting  at  both  sites. 

■ Adequate  number  of  support  staff  trained  in  this  technology. 

■ Training  in  place  for  faculty  (focusing  on  customizing  courses  for  this  format,  practicing  specific 
teaching/presentation  skills  for  on-camera  delivery,  coaching  on  the  use  of  the  electronic  components, 
and  managing  the  new  instructional  environment). 

■ A vendor  with  past  experience  supporting  this  system  configuration  in  this  particular  environment. 

A one-day  training  session  for  faculty  was  not  customized  to  the  needs  of  higher  education  and  was  of 
little  value  in  preparing  to  teach  using  the  technology.  Faculty  faced  the  camera  in  January  1999  with 
little  formal  preparation  on  the  best  practices  of  successful  video  conferencing.  In  the  meantime,  a grant 
was  funded  for  a 3-day,  customized,  formal  training  seminar.  This  program  took  place  in  April  1999  and 
was  a success  — exactly  what  was  needed.  The  only  obstacle  was  that  faculty  schedules  were  so  fixed  and 
heavy  that  only  a small  number  could  attend.  Nonetheless,  as  a result  of  the  training,  several  faculty  and 
support  staff  are  in  place  to  coach  those  faculty  who  either  couldn't  attend  the  training  or  who  later 
transitioned  into  video  conferencing.  It  was  obvious  that  customized,  quality  training  is  a lynchpin  in  any 
video  conferencing  program  (see  Boaz  et  al.  1999  & Bray  1999  & Weiss  1999). 

The  main  issues  in  providing  this  type  of  training  are  (see  Parker  1999  & Robbins  1997): 
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■ Factor  into  the  distance  learning  budget  adequate  funding  for  a quality  training  program,  or  secure 
funds  from  an  external  source. 

■ Work  within  the  timeline  and  milestones  of  the  overall  project  to  ensure  that  faculty  are 
comprehensively  trained  before  the  first  time  they  have  to  teach  via  video  conferencing. 

■ Announce  the  training  event  well  in  advance  to  allow  faculty  to  work  it  into  their  calendar. 

■ If  possible,  have  a plan  for  release  time  for  some  faculty  who  may  need  to  have  classes  covered. 

■ Do  a thorough  needs-assessment  of  your  faculty  and  accept  only  customized  training. 

■ Expect  that  a good  agency/consultant  will  conduct  their  own  needs-assessment  with  you. 

■ Demand  to  talk  with  other  higher  education  institutions  that  have  utilized  this  training 
agency/consultant  for  the  same  kind  of  training. 

The  three-day  seminar  that  was  chosen  by  MCPHS  focused  on  (see  Pike  & Arch  1 999): 

■ An  experienced  consultant  who  modeled  best  practices. 

■ Innovative  course  design  for  video  conferencing. 

■ Interactive,  student-centered  approach. ^ 

■ Preparation  of  visuals  for  broadcast  format. 

■ On-camera  presentation/audience  engagement  skills. 

■ Assessment  strategies  for  distance  education. 

■ Many  real-life,  relevant  examples. 

■ Opportunity  to  practice  using  the  system. 

The  consultant  provided  participants  with  handouts,  a practical  workbook  of  activities,  and  a handbook  of 
useful  techniques  and  references.  All  participants  delivered  10  minutes  of  instruction  under  live  broadcast 
conditions.  Each  received  individualized  feedback  and  a videotape  of  their  performance  for  self  evaluation 
(see  Hardwick  1997  & Robbins  1997  & Zielinski  1998). 

As  a result  of  the  seminar,  participants  generated  several  sets  of  guidelines,  including  When  the 
Technology  Fails:  Guidelines  for  Creating  Plan  B,  Basic  Protocol  and  Etiquette  for  Video  Conferencing 
Discussion^  Guidelines  for  Conducting  a Distance  Learning  Orientation  Event  for  Students^  and 
Guidelines  for  Guest  Lecturers.  These  documents  have  helped  significantly  in  preparing  faculty  and  also 
in  focusing  course  coordinators  and  administrators  on  the  special  needs  of  teaching  via  video 
conferencing. 


Teaching  Experiences  and  Outcomes 

Using  video  conferencing,  five  courses  have  been  delivered  to  the  adult  learners  in  this  program.  Four  of 
the  five  courses  were  team- taught  and  one  was  taught  exclusively  by  an  individual  instructor.  All  courses 
in  the  program  are  structured  to  include  didactic  lecture  sessions,  interactive  faculty-led  sessions  and 
student-based  case  discussions.  The  lectures  and  interactive  sessions  are  delivered  on  Saturdays  in  the 
morning  and  early  afternoon.  For  these  sessions,  the  on-site  and  remote  sites  are  connected  via  interactive 
technology.  The  case  discussions,  conducted  using  small  groups  of  students  in  the  late  afternoon,  do  not 
require  the  use  of  video  conferencing.  The  variety  of  delivery  methods  used  in  these  courses,  and  the  day- 
long duration  of  each  course,  requires  much  faculty  planning  for  versatility  in  the  classroom,  and 
adaptation  to  new,  interactive  technology  (see  Hardwick  1 997  & Pike  & Arch  1 999). 

When  the  first  team-taught  course  was  offered  in  January  1999,  the  course  coordinator  was  not  teaching 
and  was  available  to  handle  the  technical  complications  that  arose.  Faculty  members  using  this  technology 
needed  to  adjust  to  a number  of  changes  in  lecture  delivery  including  time  delays,  development  of  an  on- 
camera  persona,  and  adaptation  of  lecture  material  to  distance  learning  (see  Parker  1999  & Zielinski 
1 998).  Faculty  had  to  learn  to  deal  with  a five  to  ten  second  delay  in  transmission  of  her  words  to  the 
remote  site.  This  was  difficult,  even  for  faculty  members  who  were  accustomed  to  pausing  between 
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concepts  to  allow  for  students’  questions.  It  seemed  an  unnatural  amount  of  time  for  both  the  faculty 
member  and  the  students  on  site.  Furthermore,  if  students  at  the  remote  site  had  a question,  it  took  the 
same  amount  of  time  for  their  question  to  be  transmitted.  The  dialogue  became  cumbersome.  Therefore, 
most  faculty  members  built  small  group  exercises  into  their  lectures  so  that  students  could  discuss  issues 
among  themselves  and  try  to  help  each  other  in  answering  questions  about  the  lecture  material. 

Faculty  have  tested  the  compatibility  of  a variety  of  audiovisual  aids  with  this  technology.  We  have  found 
that  PowerPoint  presentations  project  well  at  both  sites  and  allow  students  at  the  remote  site  to  focus  on 
visual  representations  of  the  lecture  material  and  not  solely  on  the  instructor.  We  have  also  found  it 
crucial  to  provide  the  students  with  very  structured  and  detailed  handouts  (see  Pike  & Arch,  1 999).  If  the 
visual  projection  is  ever  lost,  students  are  still  able  to  follow  the  lecture  using  their  handout.  Although 
many  of  us  traditionally  use  overhead  projectors  to  facilitate  interactive  sessions  in  large  classroom 
settings,  these  projections  are  not  successful  using  interactive  technology.  Instead,  specialized  equipment, 
such  as  a document  camera,  facilitates  interactive  learning.  A limitation  of  the  document  camera  is  that 
the  instructor  is  no  longer  visible  to  the  students  at  the  remote  site.  Because  of  this  limitation  and  the  fact 
that  we  have  only  one  camera,  it  is  essential  to  provide  students  with  a structured  handout  which  includes 
all  graphics  and  interactive  diagrams  so  that  they  can  easily  keep  pace  with  the  instructor. 


Feedback 

Initial  comments  about  the  technology  from  students  at  both  sites  were  predominately  negative.  They  felt 
that  videoconferencing  was  disruptive  and  adversely  impacted  on  discussion  during  lecture.  Students  at 
the  Boston  campus  were  frustrated  with  the  intrusion  of  videoconferencing  and  students  at  the  remote  site 
were  more  passive  in  their  participation.  Students  identified  sound  and  video  problems  which  needed  to  be 
resolved  by  an  expert  rather  than  by  the  work-study  students  who  were  assigned  to  assist  broadcasts. 
Although  the  work-study  students  were  very  reliable,  they  did  not  have  the  level  of  training  to  address  the 
inadequacies  of  the  system  (see  Boaz  et  al.  1999  & Carr  2000). 

As  of  fall  2000,  student  feedback  has  much  improved  with  only  a few  comments  on  course  evaluations 
regarding  the  technology.  Faculty  have  had  mixed  reactions  to  video  conferencing.  Some  have  adapted 
better  than  others  to  the  technology  and  to  the  necessary  changes  they  have  had  to  make  to  their  teaching 
style  (see  Kirchdorfer  1 997). 


Conclusions 

Program  Status 

Overall,  it  has  taken  almost  three  semesters  for  the  majority  of  faculty  and  students  to  become  comfortable 
with  the  process.  However,  the  system  still  is  still  not  completely  reliable  and  faculty  must  always  have  a 
backup  plan  in  case  of  technology  failure.  To  further  maximize  the  process,  all  parties  involved, 
particularly  the  faculty  and  technology  support  personnel,  need  to  work  closely  together.  Faculty  need  to 
keep  informed  of  changes  in  the  technology,  the  availability  of  new  training  sessions,  and  changes  in 
personnel.  In  addition,  the  technology  personnel  need  to  be  well  informed  of  the  faculty  member’s  course 
planning  and  design  requirements  (see  Boaz  et  al.  & Hardwick  1997  & Robbins  1997). 


Faculty  Training 

Faculty  need  training  in  interactive,  student-centered  course  design  and  on-camera  techniques  for  video 
conferencing  (including  hands-on  practice)  before  they  use  the  technology  to  teach.  Adequate  funding  for 
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customized,  quality  training  must  be  factored  into  the  overall  project  budget  (see  Bray  1 999  & Parker 
1999  & Robbins  1997  & Weiss  1999). 


Broadcast  Environment 

A fully  equipped,  appropriately  designed  broadcasting  room  should  be  established  and  tested  at  each  video 
conferencing  site.  This  information  is  readily  available  on  the  internet  (see  SURA  Video  Development 
Initiative  1999  & Pike  & Arch  1999),  or  from  established  vendors. 


Whenever  a new  group  of  students  begins  a program  delivered  via  distance  learning,  a formal  orientation 
program  should  be  presented,  before  the  start  of  classes  (see  Pike  & Arch  1 999).  The  orientation  should 
take  place  within  the  chosen  environment  for  delivering  the  program  (e.g.,  conduct  the  orientation  session 
via  video  conferencing  if  the  instructional  program  will  be  video  conferenced).  This  orientation  should: 

■ deliver  all  necessary  information  about  the  structure  and  schedule  of  the  program. 

■ describe  the  key  elements  of  the  chosen  delivery  format  that  differ  from  a conventional  class. 

■ demonstrate  the  operational  features  of  the  equipment. 

■ discuss  the  roles  and  expectations  of  all  participants  (students,  faculty,  site  facilitators,  technicians). 

■ clarify  rules  and  protocol  for  questions  and  discussions,  as  well  as  all  vehicles  of  communication  such 
as  e-mail,  internet,  telephone,  FAX. 

Activities  should  be  planned  that  help  to  introduce  participants  to  each  other  and  to  the  faculty  and 
administrators  of  the  program.  Plan  5,  the  backup  plan  that  will  go  into  action  if  a technology  failure 
occurs,  should  also  be  presented  (see  Pike  and  Arch  1 999). 
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Abstract:  Decision-makers  within  the  U.S.  Navy  must  command  vast  quantities  of  information. 
The  bulk  of  this  knowledge  is  accumulated  through  personal  study  of  written  documentation 
including  Tactical  Memoranda  (TACMEMOs).  This  paper  describes  the  theoretical  and  technical 
underpinnings  of  the  Tactical  Readiness  Instruction,  Authoring,  and  Delivery  (TRIAD)  project,  an 
effort  designed  to  improve  the  usability  of  these  documents. 


Introduction 

Decision-makers  within  the  U.S.  Navy  are  faced  with  increasingly  complicated  and  stressful  tactical  environments. 
These  environments  are  characterized  by  situational  uncertainty,  time  compression,  and  capable  adversaries.  To 
cope  with  such  environments,  today’s  decision-makers  must  have  absolute  command  of  a vast  and  varied  knowledge 
base.  The  decision-maker  must  be  familiar  with  situational  cues,  own  ship  capabilities  and  limitations,  those  of 
potential  adversaries,  tactics  at  his  or  her  disposal,  and  those  that  potential  adversaries  might  employ. 

Some  of  this  knowledge  comes  from  formal  training.  However,  the  bulk  of  it  is  developed  through  experience  and 
through  personal  study  of  tactical  publications  (including  Tactical  Memoranda  [TACMEMOs])  and  combat  system 
doctrine  (Cannon-Bowers,  1995;  Cannon-Bowers  et  al^  1994).  In  this  work,  we  focus  on  TACMEMOs. 

TACMEMOs  are  developed  to  provide  Fleet  personnel  with  the  information  necessary  to  employ  a new  tactic  or 
tactical  concept.  They  are  often  developed  by  relatively  junior  officers.  These  officers  (and  their  support  staff)  may 
be  knowledgeable  in  the  tactical  subject-matter  in  question.  However,  they  have  little  or  no  training  in  effective 
technical  writing  or  instructional  design.  Moreover,  the  documents  that  are  produced  suffer  from  shortcomings  that 
are  common  in  many  technical  documents:  passivity,  linearity,  and  poorly  represented  spatiotemporal  relationship. 

We  began  the  Tactical  Readiness  Instruction,  Authoring,  and  Delivery  (TRIAD)  project  to  develop  a set  of 
authoring  and  delivery  tools  that  will  enhance  the  quality  of  tactical  guidance. 
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Development  Context 


TACMEMO  development  begins  with  the  identification  of  a tactical  deficiency  and  development  of  a tactical 
solution  that  addresses  that  deficiency.  The  resultant  tactic  is  disseminated  to  Fleet  units  via  a TACMEMO. 

TACMEMOs  are  written  by  middle  grade  officers  with  operational  experience  relative  to  the  assigned  tactical 
project  warfare  area.  These  individuals  possess  a minimum  of  a bachelor’s  degree,  and  at  least  some  computer 
skills.  They  are  supported  by  a staff  of  senior  petty  officers,  and  civilian  editors  and  production  specialists.  In  some 
cases,  tactical  project  officers  may  receive  analytical  support  from  civilian  contractors  or  government  laboratory 
engineers/representatives. 

Project  officers  are  provided  with  structural  guidance*  (/.e.,  the  sections  that  a TACMEMO  should  include,  and  the 
order  of  those  sections).  However,  they  are  provided  with  little  or  no  guidance  as  to  how  to  author  a document  that 
efficiently  communicates  the  tactic. 

Once  written,  TACMEMOs  are  read  by  personnel  ranging  from  flag- level  commanders  (Le,,  Admirals)  down  to 
junior  enlisted  personnel.  At  every  level,  readers  must  balance  the  need  to  read  and  understand  new  TACMEMOs 
against  the  press  of  their  other  responsibilities.  Their  task  is  made  more  difficult  by  documents  whose  format  is  not 
consistent  with  the  reader’s  needs.  We  recently  queried  a group  of  readers  on  their  use  of  TACMEMOs.  Within 
this  group,  TACMEMOs  are  used  extensively  as  reference  documents  and  are  rarely  studied.  Only  one  participant 
indicated  that  he  often  read  the  body  of  the  TACMEMO.  Most  indicated  that  they  did  so  on  occasion.  The 
remainder  indicated  that  they  did  so  rarely  if  ever.  By  contrast,  a large  majority  of  participants  indicated  that  they 
consulted  TACMEMOs  during  operations. 

It  is  within  this  context  that  TRIAD  must  operate.  In  designing  TRIAD,  we  must  create  tools  that  augment  authors’ 
typical  communication  skills  to  produce  documents  that  meet  the  task-demands  of  readers. 

TRIAD  Overview 

The  personal  computer  (PC)-based  TRIAD  system  is  being  designed  and  developed  to: 

• Provide  authors  with  the  capability  to  create  an  effective  multimedia  learning  environment  to  support  the 
transfer  of  tactical  knowledge  to  readers;  and 

• Provide  readers  of  tactical  documentation  with  a product  that  addresses  their  need  for  efficient  and  effective 
tactical  documentation  that  is  easy  to  use  and  supports  self-training. 

In  this  regard,  TRIAD  will  provide  authors  with  an  integrated  tool  set  to  enable  them  to  create  tactical 
documentation  (i.e,,  TACMEMOs)  using  a variety  of  multimedia  presentation  techniques,  and  to  create  associated 
interactive  multimedia  instruction  (I MI)  to  support  the  documented  tactic/doctrine.  In  turn,  readers  will  receive  a 
multimedia  tactical  documentation  “product  set”  that  supports  tactic/doctrine  presentation  and  briefing,  instruction, 
quick  reference,  and  facilitation  of  electronic  feedback  regarding  tactic/doctrine  evaluation. 

After  a tactic  has  been  defined,  but  before  it  has  been  documented,  the  author  uses  TRIAD  to  create  a product  set. 
The  process  consists  of  three  stages:  interview,  edit,  and  review. 

During  the  interview  stage,  the  author  creates  and/or  imports  media  regarding  the  tactic  in  response  to  interview 
questions.  For  example,  the  author  might  be  asked  to  define  the  tactic  (text),  describe  the  tactic  using  an  illustration 
(graphics),  generate  a scenario  that  supports  practice  (simulation),  and/or  import  a video  that  shows  tactic  evaluation 
results.  As  the  interview  progresses,  the  TRIAD  system  adds  the  information  provided  by  the  author  to  its  database. 

Using  the  information  gained  from  the  interview,  the  TRIAD  system  automatically  generates  a draft  TACMEMO 
product  set  consisting  of  the  following  components:  Base  Document  (core  TACMEMO),  Tactic  Training 

Component,  Quick  Reference  Guide  (QRG),  Feedback,  and  Brief  (PowerPoint®  presentation). 

Once  the  interview  is  complete  and  the  draft  product  set  generated,  the  edit  stage  commences.  Here,  the  author  is 
presented  with  the  draft  product  set,  and  can  choose  to  edit  any  or  all  of  the  product  set  components.  He  can  add 


‘ Often,  even  this  minimal  guidance  is  violated. 
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new  media  and  edit  existing  media  (text,  graphics,  animation,  simulation,  etc.).  The  author  can  import  related  media 
from  the  local  TRIAD  database,  or  from  a remote  database,  into  a product  template  and  then  edit  as  desired. 

The  review  stage  commences  after  all  TACMEMO  product  set  components  have  been  developed.  The  TRIAD 
system  will  integrate  the  components  into  a review  document  with  all  associated  markings.  This  review  version  can 
be  distributed  via  multiple  means  (paper,  local  area  network  (LAN),  wide  area  network  (WAN),  disk,  etc.). 
Reviewers  will  be  able  to  comment  within  the  document  and  return  these  comments  to  the  author.  Comments 
received  electronically  will  be  stored  in  the  TRIAD  database  for  use  by  the  TACMEMO  author  to  revise 
components  as  required.  The  capability  to  merge  comments  into  the  document  will  be  provided.  Comments 
received  on  paper  will  be  utilized  by  the  author  to  revise  components  as  required.  As  in  the  edit  stage,  the  author 
can  create/import  new  media  and  edit  existing  media  (text,  graphics,  animation,  simulation,  etc.)  in  response  to 
review  comments. 

This  process  of  assembling  the  product  set  components  into  a review  document,  distributing  the  document  for 
review,  incorporating  review  comments,  and  reassembling  the  product  set  can  repeat  as  necessary  until  a final 
product  set  is  approved.  Upon  completion  of  the  review  process,  the  TRIAD  system  will  assemble  the  TACMEMO 
product  set  for  final  packaging  and  subsequent  distribution  to  readers  (compact  disk/digital  video  disk  (CD/DVD)  or 
LAN/WAN).  TRIAD  will  support  document  version  control  throughout  this  process. 

A reader  will  receive  the  TACMEMO  in  an  appropriate  electronic  format,  and  will  be  able  to  access  the  multimedia 
TACMEMO  product  set  (Base  Document,  Tactic  Training,  QRG,  Feedback,  and  Brief). 

Authoring  Component 

Although  TRIAD  will  produce  a product  set  that  includes  much  more  than  is  normally  considered  “training,”  we 
conceptualize  it  as  an  instructional  design  task.  As  such,  TRIAD  must  automate  the  instructional  design  process. 
As  Reigeluth  (1993)  noted: 

Automating  could  entail  anything  from  providing  a few  tools  for  an  instructional  designer  to  use, 
to  entirely  replacing  the  instructional  designer.  It  generally  does  not  refer  to  replacing  the  subject- 
matter  expert.  The  instructional  design  process  could  entail  anything  from  the  instructional 
strategy  selection  process  ...  to  the  entire  spectrum  of  phases  . . .. 

Theoretical  Foundation 

Instructional  design  theories  relate  instructional  conditions  (student  and  subject-matter  attributes),  instructional 
methods,  and  instructional  outcomes  (Reigeluth,  1983).  Our  focus  is  on  prescribing  instructional  methods  to 
produce  instructional  outcomes  (given  certain  instructional  conditions).  More  generally,  this  could  be  rephrased  to 
say  that  we  want  to  devise  communication  methods  that  will  produce  performance  outcomes. 

A number  of  others  have  investigated  the  process  of  automated  instructional  design  and  TRIAD  can  certainly  benefit 
from  their  work  [see,  for  example,  Spector,  Poison,  & Muraida,  (1993)].  In  general,  two  classes  of  authoring 
systems  have  emerged. 

The  first  approach  seeks  to  provide  instructional  design  assistance  and  coaching  during  development.  Gagne’s 
Guided  Approach  to  Instructional  Design  Advising  (GAIDA)  provides  one  example  of  this  approach  (Gagne,  1 993). 
Using  information  about  students,  the  task,  and  the  environment,  GAIDA  guides  instructional  designers  through  a 
four-step  process  of  instructional  design.  Tennyson  (1993)  offered  a similar,  albeit  more  ambitious,  approach.  He 
proposed  to  develop  a system  that  could  coach  novice  users  through  specific  development  problems  while 
developing  their  ISD  background.  The  same  system  would  simply  offer  advice  to  more  experienced  users. 

The  guidance/assistance/coaching  approach  is  relatively  easy  to  implement  and  may  be  applied  to  a variety  of  design 
situations.  However,  it  does  require  the  developer  to  acquire,  interpret,  and  apply  instructional  design  principles 
while  in  the  process  of  development.  In  the  TRIAD  context,  our  authors  seldom  have  the  background  necessary  to 
use  such  a tool  effectively.  Such  an  approach  might  impose  a significant  learning  requirement  on  the  authors  while 
not  ensuring  compliance  with  instructional  design  guidance. 
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An  alternative  approach  is  to  provide  restrictive  template  structures  as  the  primary  mode  of  development.  Merrill’s 
Electronic  Trainer  provides  a good  example  of  this  approach.  Within  Merrill’s  approach  (1993),  the  author  uses  a 
defined  grammar  to  specify  the  training  environment  and  curriculum.  Expert  systems  are  then  used  to  select, 
sequence,  and  partially  populate  the  appropriate  instructional  transactions. 

The  use  of  a template  structure  removes  many  of  the  burdens  imposed  by  GAIDA.  Instructional  design 
prescriptions  are  built  into  the  templates.  Therefore,  authors  do  not  have  to  mastery  this  material  themselves  and 
compliance  is  assured.  However,  such  systems  can  be  complex  to  operate  and  the  selection  of  appropriate  templates 
is  not  always  obvious. 

TRIAD  developers  have  selected  a hybrid  inquiry-based  strategy.  Within  our  approach,  the  majority  of 
development  occurs  through  the  use  of  implicit  templates.  That  is,  the  templates  are  used  to  guide  an  interview  with 
the  author  and  to  organize  the  resultant  content.  This  approach  has  the  benefits  of  Merrill’s  template  approach, 
while  hiding  much  of  the  complexity  from  the  author.  It  ensures  a high-level  of  compliance  with  theoretically  sound 
instructional  design  principles  without  requiring  the  author  to  master  those  principles. 

Design  Summary 

The  various  theoretical  perspectives  on  decomposing  content,  establishing  learning  objectives,  and  associating 
learning  objectives  with  communication  strategies  embodied  within  TRIAD  can  be  summarized  in  an  overarching 
design  approach.  The  TRIAD  team  has  organized  this  approach  into  three  interview-stage  phases:  Pre-planning, 
Initial  Query,  and  Elaboration/ Augmentation.  Each  phase  satisfies  an  important  goal  for  the  authoring  process. 

During  the  pre-planning  phase,  TRIAD  guides  the  author  to  establish  the  basic  framework  of  the  TACMEMO 
product  set.  That  is,  starting  with  a metaphorical  “blank  sheet  of  paper,”  TRIAD  helps  the  author  to  identify  the 
primary  content  areas  which  must  be  explored  in  the  product  set.  This  establishes  a framework  on  which  later 
authoring  functions  can  elaborate. 

More  specifically,  the  Pre-Planning  phase  has  two  goals  associated  with  it.  The  first  goal  is  to  help  the  developer 
begin  to  mentally  frame  the  content  of  the  TACMEMO  to  be  written.  TRIAD  should  not  only  solicit  information 
from  the  developer,  but  also  help  the  developer  organize  his/her  thoughts  about  the  TACMEMO  as  it  is  being 
developed.  This  will  facilitate  an  active,  rather  than  passive,  participation  of  the  developer  within  the  interview. 

After  considering  a range  of  pre-planning  options  (full  case-based  retrieval,  exemplar  case-based  retrieval, 
TACMEMO  taxonomy,  traditional  knowledge  elicitation  probes,  and  scenario-based  development),  the  development 
team  determined  that  the  development  of  a TACMEMO  taxonomy  was  most  feasible.  This  method  relies  on  the 
development  of  a TACMEMO  taxonomy  that  is  used  by  the  developer  to  classify  the  TACMEMO  to  be  written. 
The  taxonomy  decomposes  each  classification  several  layers  deep,  permitting  the  developer  to  select  those  elements 
applicable  to  the  TACMEMO  to  be  authored.  To  succeed,  it  must  be  possible  to  classify  existing  TACMEMOs  into 
a finite  number  of  categories.  For  TRIAD,  we  were  able  to  develop  on  a three-tier  taxonomy:  categories,  anchors, 
and  sub-anchors. 

Next  is  the  initial  query.  The  base  document  represents  the  primary  communication  of  the  tactic  and  the  central 
product  set  component.  Therefore,  step  one  of  the  initial  query  combines  sub-anchors  into  various  base  document 
sections  and  sub-sections.  TRIAD  will  support  the  author  in  this  process  as  well  as  guiding  the  author  to  generate  a 
performance  goal  and  associated  title  for  each  section^. 

After  reviewing  the  base  document  structure,  the  author  will  be  supported  through  the  process  of  creating  base 
document  content.  For  example,  TRIAD  will  provide  guidance  based  on  effective  technical  writing  techniques, 
appropriate  content  sequencing  and  media  selection  (based  on  performance  goal  verbs). 

At  this  point,  authoring  will  transition  from  the  Base  Document  component  to  the  Tactic  Training  component. 
Together,  TRIAD  and  the  author  will  identify  Base  Document  sections  that  can  benefit  from  associated  instruction. 
These  are  likely  to  be  sections  that  are  sufficiently  complex  or  that  rely  on  doubtful  background  knowledge. 


^ It  should  be  noted  that  this  focus  of  performance  goals  within  the  base  document  is  likely  to  lead  authors  to  create 
a “leaner”  and  more  usable  document  than  was  common  in  previous,  unguided  attempts. 
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In  any  event,  when  a section  is  earmarked  for  instructional  support,  TRIAD  will  guide  the  author  through  the 
process  of  creating  verb-based  learning  objectives.  Next,  the  author  will  be  guided  to  create  assessment  items, 
instructional  material,  and  practice  items  in  support  of  each  objective.  This  process  will  work  up  from  content 
sections  to  include  both  lesson-  and  module-level  instruction. 

Similar  processes  will  be  applied  to  derive  the  quick  reference  guide.  Feedback,  and  Brief  components. 

When  done  with  the  initial  query,  the  author  will  have  produced  a complete,  albeit  immature  product  set.  The 
maturation  process  occurs  through  elaboration  and  augmentation. 

The  Elaboration/Augmentation  (E/A)  process  is  applied  in  the  final  stages  of  TRIAD  document  authoring.  The 
process  consists  of  three  strategies:  confirming,  elaborating,  and  fine-tuning.  These  strategies  are  designed  to  help 
authors  refine  and  augment  content.  They  are  applied  iteratively. 

Confirmation  assists  authors  in  validating  content  accuracy  and  completeness  as  well  as  confirming  TRIAD- 
generated  structures  and  sequences.  Confirmation  is  critical  because  it  safeguards  the  accuracy  of  both  the  content 
and  structure  of  TRIAD-generated  documents.  Elaboration  helps  authors  to  extend,  amplify,  and  otherwise 
augment  TRIAD  documents.  Authors  elaborate  and  add  detail  descriptions  and  supporting  examples,  especially 
those  considered  critical  to  the  user’s  knowing  and  implementing  the  tactic.  Fine-tuning  enables  the  author  to 
clarify  information,  directions,  instruction,  and  presentation.  At  this  step,  the  author  amplifies  key  information, 
reducing  or  eliminating  ambiguity  and  unclear  or  non-essential  information. 

The  E/A  process  is  grounded  in  a set  of  partially  overlapping  foundations.  E/A  in  the  Base  Document  is  grounded  in 
seven  foundations:  text  structure,  text  design,  text  augmentation,  usability/transferability,  technological,  cultural, 
and  pragmatic.  In  the  Tactic  Training  component,  E/A  is  grounded  in  five  foundations:  psychological,  pedagogical, 
technical,  cultural,  and  pragmatic.  In  both  cases,  these  foundations  ultimately  help  authors  develop  and  fine-tune 
structure  and  content  drafted  during  the  initial  query. 

The  interview  process  will  create  a variety  of  interconnected  content.  TRIAD  needs  a robust  process  for  organizing, 
maintaining,  and  using  that  content.  To  meet  this  challenge,  we  have  adopted  the  use  of  knowledge  objects. 

One  can  think  of  knowledge  objects  (sometimes  referred  to  as  learning  objects  or  sharable  courseware  objects)  as 
boxes  with  labels.  The  label  on  the  outside  of  the  box  tells  everyone  about  the  contents  inside  the  box.  Knowledge 
object  boxes  may  contain  other  boxes,  or  real  contents.  As  such,  the  knowledge  objects  provide  an  elegant  way  to 
store  and  organize  the  contents  of  our  TRIAD  product  sets.  The  labels  help  in  the  organization  function.  The  labels 
also  make  it  possible  to  look  for  a box  (knowledge  object)  that  contains  desired  content. 

Delivery  Component 

An  elegant  authoring  process  is  of  little  use  if  the  content  is  delivered  to  the  reader  in  a form  that  fails  to  meet  his  or 
her  needs.  TRIAD  will  build  on  Internet  technology  to  create  a robust,  user-sensitive,  and  flexible  delivery 
environment.  Here,  we  describe  how  TRIAD  delivery  components  will  rise  to  meet  this  challenge. 

Enabling  Technology 

An  understanding  of  how  content  will  be  delivered  to  the  user  requires  an  understanding  of  three  component 
technologies. 

As  noted  above,  we  will  be  using  knowledge  objects  to  organize  the  output  of  the  authoring  stage.  In  the  delivery 
environment,  however,  we  make  use  of  an  Extensible  Markup  Language  (XML)  representation  of  the  content.  A 
transformation  process  will  transform  knowledge  objects  into  the  TRIAD  XML  representation.  XML  is  a mark-up 
language  that  describes  the  functional/semantic  parts  of  a document  (titles,  headings,  paragraphs,  warning,  cautions). 
XML  is  well  suited  to  the  task  of  representing  the  structure  and  interrelationships  inherent  in  a body  of  content.  An 
XML  representation  of  content  is  independent  of  how  that  content  might  be  rendered  to  a user. 

To  render  the  content,  TRIAD  will  use  Dynamic  Hypertext  Markup  Language  (DHTML).  DHTML  is  a good  tool 
for  rendering  content  in  a browser-based  environment  and  for  supporting  simple  types  of  interactivity.  It  is  poor  at 
facilitating  the  storage  of  content,  its  content,  and  its  inherent  interrelationships.  TRIAD  needs  a mechanism  to 


652 


translate  the  XML  representation  of  the  content  to  its  DHTML  representation^.  The  Extensible  Style  Language 
(XSL)  provides  a vehicle  for  that  translation.  Stylesheets,  written  in  XSL,  describe  how  different  elements  of  the 
XML  representation  are  to  be  rendered.  That  is,  the  XSL  stylesheet  works  as  a map  between  XML  elements 
(defined  through  a document  type  definition;  DTD)  and  DHTML  representations. 

Design  Summary 

Internet  browsers  are  “stateless  machines.”  That  is,  they  can  follow  links  from  one  place  to  another,  but  they  are  not 
designed  to  have  memories  about  users  or  their  actions.  Such  a memory  is  critical  to  several  TRIAD  functions.  For 
example,  the  TRIAD  “Quick  Pick”  TACMEMO  list,  notes,  bookmark,  and  highlight  functions  are  all  user-specific 
and  depend  on  the  system’s  ability  to  recognize  a given  user.  Similarly,  the  functioning  of  the  Tactic  Training 
component  and  the  management  of  administrative  privileges  all  depend  on  user-specific  information. 

To  provide  that  functionality,  TRIAD  has  adopted  a three-part  architecture.  The  most  obvious  part  of  the 
architecture  is  the  browser  itself  TRIAD  will  delivery  product  set  material  to  the  reader  through  an  Microsoft® 
Internet  Explorer  5.0  browser.  Rather  than  operating  in  a “stand-alone”  fashion,  the  browser  will  be  hosted  within  a 
secondary  application,  the  “Presentation  Engine”.  The  Presentation  Engine  is  a piece  of  “middleware”  that  serves  as 
a communication  bridge  between  the  browser  and  the  third  important  piece  of  software,  the  “Delivery  Engine”.  The 
Delivery  Engine  is  responsible  for  the  bulk  of  TRIAD  delivery  functionality.  This  includes  activities  like 
maintaining  user  data,  managing  information  access,  preparing  knowledge  objects  for  delivery,  and  system 
administration. 
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Abstract 

This  paper  discusses  the  implementation  of  web-based  primary  sources  in  two 
undergraduate  history  modules  at  the  University  of  Glasgow,  Scotland.  It  outlines 
the  purpose  of  implementing  Information  and  Communications  Technology  (ICT) 
and  the  different  approaches  taken  by  lecturers  to  the  use  of  computer  assisted 
instruction.  The  implementation  employed  a range  of  evaluation  instruments  both 
quantitative  and  qualitative,  before  and  after  the  modules  were  taught.  The  results 
of  the  evaluation  and  the  implications  of  the  findings  are  debated. 


Introduction 

The  School  of  History  and  Archaeology  at  the  University  of  Glasgow  recently  injected  the  use  of  Information 
and  Communications  Technology  (ICT)  into  its  Level  Two  history  courses  in  order  to  introduce  all  its  history 
students  to  the  potential  of  computer  networks  and  access  to  sources.  It  is  hoped  that  the  exposure  to  ICT  will 
provide  a prelude  to  specialist  honours  Historical  Computing  courses  which  deal  with  more  advanced 
methodological  applications  of  computers. 

The  IT  Committee  of  the  School  had  been  working  on  this  implementation  since  the  move  from  full  year 
courses,  to  a modular  system  of  ten  week  courses,  in  1996.*  It  had  already  ensured  the  introduction  of  a School 
policy  which  stated  that  the  completion  of  at  least  one  assessed,  computer-based  unit  within  a Level  Two  course 
will  be  a prerequisite  for  entry  into  honours  in  a student's  third  year,  and  had  run  a pilot  scheme  with  one 
lecturer.  1 998  though,  saw  a number  of  other  developments  which  were  to  give  a boost  to  this  policy: 

1 . The  complete  redesign  of  the  Level  Two  modules  with  a new  aim  to  increase  students'  familiarity  with 
primary  sources  and  to  give  them  skills  in  interpreting  such  sources  critically.  This  was  entirely 
independent  of  any  development  in  ICT  and  indeed  some  tutors  chose  to  use  paper  primary  sources  like 
examples  of  palaeography  and  works  of  art  in  local  galleries. 

2.  The  completion  of  twelve  multi-media,  history  tutorials  for  use  in  undergraduate  teaching,  by  the  national 
Teaching  and  Learning  Technology  Program  (TLTP)  History  Courseware  Consortium  which  provided  the 
project  with  a rich  starting  point  in  terms  of  available  sources.^ 

3.  The  founding  of  the  successor  to  the  TLTP  project,  the  Courseware  for  History  Implementation  Consortium 
(CHIC)^,  set  up  to  provide  funds  and  pedagogical  advice  to  UK  Universities  embedding  the  tutorials  into 
undergraduate  history  teaching. 

A successful  grant  application  to  the  CHIC  project  enabled  the  provision  of  in  depth  support  for  implementing 
and  evaluating  a range  of  resources  within  the  new  modules  which  were:  European  Society  from  1550  to  1700 
(Early  Modem);  and  Society,  Culture,  Politics  and  Power  in  North  America  from  First  Contact  to  the  Present 
(America).  The  Early  Modem  module  would  take  place  during  the  first  semester  with  around  200  students 
registered  and  the  American  module  was  to  be  taught  in  second  semester  to  circa  220  students. 
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The  main  business  of  the  project  was  to  introduce  students  to  the  use  of  on-line  sources  to  prepare  seminar 
topics  but  other  parts  of  the  project  set  up  data  analysis  exercises,  multi  media  lectures  and  a computer 
mediated  conferencing  system/ 

One  of  first  issues  to  deal  with  was  the  different  approaches  between  the  two  courses,  and  within  the  Early 
Modem  module,  the  differing  approaches  of  individual  lecturers,  regarding  the  presentation  of  the  sources  and 
the  extent  of  the  skills  that  would  be  introduced  to  the  students. 


Early  Modern 

Not  all  the  students  who  enrolled  of  the  Early  Modem  module  used  computerised  sources.  Five  tutors  took 
seminar  groups  and  decided  their  own  seminar  topics,  of  these  five,  only  two  made  the  use  of  web-based  sources 
mandatory.  Both  of  these  tutors  were  highly  directive  in  the  instructions  they  gave  to  their  students.  They 
presented  the  students  with  a small  number  of  primary  sources  per  seminar,  pre  selected  from  the  TLTP 
tutorials,  prefaced  with  questions  that  the  students  should  consider  for  the  seminar.  Figure  1 .shows  a section  of 
the  instmction  for  one  such  seminar  question. 

Toulouse- IS.  a:provincialxi1y  located.m  the  For  five  days  in  May.:: 

1562  the' streets  of  Toulouse  wererth^ocation  of  religious  violence  which  led  to  the  deaths 
of  many  people.  Such-Violence  in‘T'o^ouse.:and  other  cities  demonstrated  the 
eiq^jlosiveness  of.the  political  and x_eli^ous. situation  in  France.  It  was  the. prelude  to  the, 
outbreak  of  the:  FrenchiWars  of  Reli^bri-and  a^f^  of  more  famous  violence 

such  as  the  St.  B^olomew's  Day  Massa^^  of  1572. 

The  Reformation  was  bom  in  Germany  in  the.  1520s.  It  did  not  take  it  long  to  spread  to 
France;:^ed  bjfthe  proximity  ofth^Frbtestant  ci^r, of  Geneva,  horhe.cf  Je^  Calvin.,^^ 
each  of  thi  foibv^  to  use  the  Back::button  to  get::: 

back  here-)  : u... . * - - 'jr- 

The  grp^  of  Protestantism;can-be:sJ:^  prosecutions  -for  heresy. 

- 07.27  Backeroiond  of  accused  heretics:  - ■ : 

The  distribution  ofPfotestantism  can  be:stu^^  through  the  origins  of  refugees  fi^om 
Franc e who  went  to  Geneva.  (T oiilous e =is-  ^ so ufc we  st  F ranc e . ) 

* Q7.29^e:^G-eoj5rapfec'^  Soread^b^^^  to  Geneva.:  ; 

Figure  1:  Screen  shot  from  the  Early  Modem  Seminar  Preparation  page. 

One  of  the  tutors  conducted  some  timetabled  seminar  sessions  in  the  computing  labs  on  the  subject  of  Migration 
in  Early  Modem  England,  where  the  students  were  introduced  to  raw  data  and  to  data  handling.  (The  students 
in  these  seminar  groups  explored  the  sources  more  deeply  and  employed  higher  levels  of  computing  skills  than 
any  of  the  other  students.)  The  second  Early  Modem  tutor  requested  that  his  students  work  through  the 
seminar  preparation  materials  in  their  own  time,  with  help  available  from  Graduate  Teaching  Assistants 
(GTAs)  during  drop  in  sessions. 


America 
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The  lecturers  on  the  America  module  co-operated  regarding  seminars  so  that  all  students  covered  the  same 
seminar  subjects  and  were  required  to  use  on-line  sources  for  seminar  preparation  throughout  the  module.  The 
TLTP  tutorials  were  only  two  of  a large  bibliography  of  web  links.  The  students  on  this  module  were  given  far 
less  direction  than  those  on  the  Early  Modem  module,  but  rather  were  presented  with  the  URLs  of  four  or  five 
comprehensive  sites,  for  each  seminar.  Figure  2 shows  the  links  for  one  particular  seminar. 


America  as  a World  Power 

• The  Great  Powers  and  the  Division  of  Europe  (only  available  locally  in  the  Art 

• Albert  Beveridge,  "The  March  of  the  Flag"  (1898) 

• William  Jennings  Bryan,  "Imperialism"  (1900) 

• John  Dewey,  "Imperialism  is  Easy"  (1927) 

• Woodrow  Wilson,  14  Points  (191 8) 

• Franklin  Roosevelt  and  Winston  Churchill,  "The  Atlantic  Charter"  (1941) 

• Harry  Truman,  "Address  Before  Congress"  (1947) 


Figure  2:  Screen  shot  from  the  American  History  Seminar  Preparation  page. 

Like  some  of  their  Early  Modem  peers,  the  students  of  American  history  were  provided  with  drop  in  lab 
sessions  supervised  by  a GTA  and  had  to  use  the  web  resources  in  their  own  time.  They  were  also  given  access 
to  an  on-line  conferencing  system  and  attended  multi-media  lectures. 

Before  this  article  proceeds,  a quick  word  about  GTAs,  on  whom  tutors  relied  extensively  to  support 
undergraduates  during  their  seminar  preparation.  The  GTAs  were  all  PhD  history  students  and  three  of  the 
four  GTAs  held  on  MPhil  qualification  in  historical  computing.  In  addition  to  this  each  GTA  was  provided 
with  four  hours  paid  training  solely  on  the  web-based  sources  that  the  undergraduates  were  using. 


Evaluation 

A full  evaluation  was  undertaken  with  students  returning  questionnaires  on  their  attitudes  to  computers  and 
computerised  sources  before  and  after  they  studied  the  modules.  Post  course  qualitative  interviews  were 
conducted  with  a selection  of  students  and  with  all  staff.  Observations  were  undertaken  of  laboratory  based 
seminars,  classroom  based  seminars,  and  a classroom  based  seminar  with  students  who  did  not  use  any  ICT  in 
seminar  preparation. 

The  purpose  of  the  evaluation  was  to  identify  the  best  methods  of  embedding  computer-based  resources  into  the 
courses  in  order  to  give  the  students  a deeper  and  richer  learning  experience.  Inevitably,  much  of  the 
evaluation  focused  on  student  reaction  to  the  computer-based  resources  and  how  this  compared  with  their 
reactions  to  traditional  resources  and  activities  such  as  lectures,  seminars,  note-taking,  and  paper-based 
materials.  Gauging  student  reaction  involves  determining  more  than  whether  students  ‘liked’  the  materials,  but 
whether  they  found  the  materials  were  appropriate  to  their  study  approach  and  whether  the  technology  served 
as  an  obstacle  to  learning.  Since  the  aim  of  teaching  is  to  enhance  student  learning,  another  important  aspect 
of  student  learning  is  whether  students  have  a sense  that  they  understood  the  historical  content  better.  A final 
component  of  the  evaluation  was  to  assess  whether  the  approach  to  implementation  was  effective  and,  where 
possible  to  compare  approaches. 


Pre-Module  Results 

BEST  COPY  AVAILABLE 
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The  first  questions  in  the  pre  module  questionnaire  attempted  to  measure  the  students*  perception  of  their  level 
of  computer  skills  against  a more  formal  gauge  of  software  competency  decided  by  the  questionnaire  designer. 
Students*  assessment  of  their  own  skills  did  not  match  reality,  while  75%  professed  themselves  as  having 
moderate  or  substantial  experience,  89%  were  actually  found  to  be  competent  or  very  competent  in  software 
use.  When  asked  about  how  confident  they  felt  using  computers,  only  60%  of  all  students  and  only  50%  of  the 
Early  Modem  students  professed  to  be  confident  or  very  confident. 

One  theory  is  that  this  is  due  to  modesty,  typical  of  British  students.  It  is  perhaps  modesty  combined  with  a 
lack  of  experience  with  computers  and  different  ideas  about  what  constitutes  competence.  The  study  was 
measuring  whether  students  possessed  the  skills  which  would  allow  them  to  take  part  in  this  particular  project 
without  further  training,  eg:  experience  of  web  use  and  word  processing.  Students  perhaps  recognise  that  these 
are  not  high  level  computing  skills  and  it*s  possible  that  our  measure  of  competence  was  relatively  low. 

When  it  came  to  student  training  it  was  surprising  that  only  45%  had  acquired  their  skills  on  the  freely 
provided  University  Basic  IT  Certificate  course.  This  means  that  students  are  coming  to  University  already 
skilled  or  are  picking  up  the  skills  via  peers  or  self  study. 

When  questioned  about  their  attitude  to  the  use  of  computerised  history  sources  58%  of  Early  Modem  students 
were  found  to  have  a positive  attitude,  against  only  32%  of  America  students.  This  can  be  explained  by  that 
fact  that  the  Early  Modem  students  were  self  selecting  as  they  signed  up  for  the  seminar  of  their  choice  and 
could  have  studied  a seminar  topic  which  did  not  involve  the  use  of  ICT.  It  is  interesting  that  the  Early  Modem 
group  were  less  confident  but  more  hopeful  about  the  experience  and  this  contrast  is  illustrated  in  figure  three 
below.  The  proportion  of  America  students  who  expect  a negative  experience  is  alarming  but  also  fits  with  the 
generally  pessimistic  attitudes  discussed  above.  To  go  beyond  conjecture  on  such  issues  the  evaluation  needs  to 
be  extended  to  cover  pre  course  qualitative  studies. 


Fig  3:  Graphs  of  Students*  Pre-Course  Attitudes  to  Using  Computers 


In  summary,  there  were  two  key  points  from  the  pre  course  questionnaire:  firstly  higher  proportions  of  UK 
students  are  coming  to  University  with  ICT  skills  each  year,  and  this  trend  is  expected  to  continue;  secondly. 
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students  still  lack  confidence  in  using  ICT  and  it  is  therefore  crucial  to  provide  adequate  documentation, 
support  and  guidance  as  well  as  assurance. 


Post  Module  Results 

The  post  module  questionnaire  provided  information  about  three  sets  of  questions: 

• How  did  students  use  the  materials? 

• How  easy  did  students  find  the  computer-based  resources?  Did  the  technology  come  between  them  and  the 
subject?  Did  they  have  any  particular  difficulties? 

• How  helpful  did  students  find  the  materials  in  learning  about  the  historical  subject?  How  did  these 
materials  compare  with  others  they  also  were  using? 

While  all  of  the  students  studying  the  Early  Modem  module  actually  used  the  on-line  sources  to  prepare  for 
their  seminars,  4%  of  the  America  students  did  not  use  a computer  at  all.  This  difference  is  probably  explained 
by  the  highly  directive  nature  of  the  Early  Modem  resources  whereby  the  questions,  issues  and  sources  related 
to  the  seminars  were  contained  on  the  course  web  page  and  nowhere  else. 

Students  were  asked  whether  they  had  sought  help  or  advice  from  anyone,  including  from  friends,  or 
neighbours  in  the  computing  labs.  70%  of  students  sought  no  advice  at  all  and  almost  all  of  these  attributed 
this  to  the  ease  of  use  of  the  materials. 

The  requests  for  assistance  were  highest  among  the  Early  Modem  groups  which  participated  in  laboratory 
based  seminars  where  50%  requested  help.  Not  only  were  these  students  undertaking  more  involved  skills 
based  tasks  but,  being  in  a lesson  with  their  fellow  students  and  with  GTAs  meant  they  automatically  turned  a 
peer  or  teaching  assistant  for  help.  Observing  one  of  these  seminars  it  was  found  that  the  students  ask  for  help 
with  the  historical  questions  and  well  as  with  computing  problems.  The  observation  also  found  that,  in  contrast 
with  classroom  based  seminars  where  the  majority  of  the  discussion  is  directed  to  and  mediated  by  the  tutor,  in 
a laboratory  situation  the  students  talked  constantly  to  each  other  about  the  data  they  were  analysing  and  their 
results.  Such  student  to  student  interaction  is  encouraging. 

Students  rated  the  computer-based  resources  very  highly.  Only  one  student  on  each  module  said  they  would  not 
recommend  them  to  other  students.  When  asked  to  compare  the  usefulness  of  computer-based  materials  with 
other  types  of  resources,  such  as  lectures,  books,  and  their  own  notes,  computerised  sources  received  the  highest 
rating  in  several  respects  with  99%  of  respondents  rating  these  resources  positively.  In  sum,  students  placed  a 
higher  value  on  computer-based  resources  than  on  any  other,  including  traditional  methods  of  delivering 
teaching.  It  should  be  noted  that  the  evaluation  was  likely  to  be  somewhat  biased  in  favour  of  computer-based 
resources,  in  part  because  the  students  knew  that  they  were  participating  in  an  experiment  and  in  part  because 
students  had  already  received  direct  feedback  from  assessed  coursework  based  on  these  resources,  whereas  they 
had  yet  to  sit  an  exam  (which  concentrated  primarily  on  lecture  topics).  Nonetheless,  the  highly  positive 
reaction  to  the  computer-based  materials  is  a significant  finding. 

Students  were  asked  which  resources  they  found  most  and  least  helpful.  The  answers  to  these  questions  were 
less  insightful  than  had  been  hoped  because  some  students  regarded  the  computer  or  the  computer-based 
tutorial  as  a single  resource.  Their  answers  therefore  provide  more  insight  into  student  views  of  the  computer 
than  about  their  approaches  to  learning.  Nonetheless,  students  identified  particular  features  they  valued, 
including  personal  histories  (i.e.  autobiographies)  and  witness  reports,  and  the  analytical  advantages  of  graphs 
and  tables.  Clearly,  students  valued  the  greater  understanding  they  gained  through  personal  evidence  that 
helped  bring  the  past  to  life,  and  the  summaries  provided  by  descriptive  statistics. 

Qualitative  analysis  via  focus  groups  gives  more  insight  into  the  differences  between  the  two  pedagogic 
approaches.  As  might  be  expected,  those  students  who  participated  in  timetabled  lab  sessions  reported  higher 
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level  learning  outcomes  than  the  remaining  students.  In  general,  students  on  the  Early  Modem  module  assert 
as  positive  aspects  such  things  as,  forming  one’s  own  interpretation  based  upon  an  assessment  of  the  evidence, 
or  as  some  of  the  students  put  it,  they  felt  as  though  they  were  ’doing’  history.  The  Early  Modem  students  also 
felt  a sense  of  ownership  and  relevance  about  the  computer-based  materials  and  they  appreciated  the  amount  of 
preparation  they  had  been  spared  by  being  presented  with  a set  of  materials  which  were  put  together  especially 
for  their  seminars^. 

In  contrast  some  of  the  America  students  felt  the  presentation  of  their  materials  lacked  direction  and  there  were 
too  many  resources  for  them  to  consider  fully.  It  can  of  course  be  argued  that  the  location,  selection  and 
assessment  of  the  probity  of  sources  is  as  much  a part  of  historical  methodology  as  is  the  analysis  and  use  of  a 
source  to  answer  a historical  question.  This  though  must  be  balanced  against  the  amount  of  time  a student 
would  spend  searching  for  a source  in  what  is  essentially  an  introductory  exercise.  The  most  positive  aspect  of 
the  project  for  the  America  students  was  that  the  computer-based  resources  were  more  widely  accessible  than 
library  based  materials. 


Conclusions 

The  evaluations  of  both  modules  have  in  common  four  key  findings: 

• Students  appreciated  the  availability  and  diversity  of  the  computer-based  sources.  Not  only  were  the 
materials  more  easily  available  than  books  from  the  library,  but  they  provided  a stimulating  range  of 
sources  which  would  not  otherwise  be  available.  Students  enjoyed  being  able  to  access  images,  facsimiles 
and  sound  as  well  as  texts,  and  being  able  to  compare  contrasting  or  contradictory  evidence. 

• Even  though  some  America  students  felt  there  were  too  many  resources  for  them  to  explore,  the  majority  of 
students  valued  the  fact  that  the  resources  were  directly  relevant  to  the  seminar  subjects. 

• It  was  recognised  that  the  diversity  and  relevance  of  the  resources  offered  increased  opportunities  for  deep 
learning. 

• It  is  crucial  that  students  are  provided  with  adequate  training  (at  least  one  class  hour)  and  with  support  in 
lab  sessions  throughout  the  module. 

From  the  post  module  evaluation  then  we  can  see  that  students’  negative  expectations  of  the  experience  of  using 
ICT  in  seminar  preparation  were  not  borne  out  the  evidence  and  this  is  a heartening  vindication  of  the  teaching 
team’s  belief  in  being  able  to  deliver  a richer  learning  experience  through  this  project.  It  would  be 
advantageous  to  further  teaching  to  undertake  qualitative  evaluation  which  attempts  to  locate  the  roots  of  these 
negative  expectations  and  recommend  ways  in  which  the  teaching  of  ICT  can  be  introduced  so  as  best  to  allay 
students’  fears  at  the  earliest  possible  opportunity. 
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Abstract:  After  implementing  WebCT  at  Mason,  the  staff  of  the  Instructional  Resource  Center 
were  interested  in  determining  if  students  were  benefiting  from  the  use  of  this  tool.  Twenty-five 
(25)  surveys  were  collected  from  a group  of  graduate  students  using  WebCT.  The  overall  findings 
were  positive  and  in  favor  of  WebCT.  The  majority  of  the  students  indicated  that  the  online  tools 
provided  by  WebCT  were  beneficial  to  the  learning  process.  Further  studies  should  be  conducted  to 
see  if  a general  population  of  students  would  yield  different  results. 


Introduction 

In  January  1999,  George  Mason  University  began  using  WebCT,  an  online  course  management  system  designed  to 
facilitate  the  creation  of  web-based  instructional  material  and  instructional  supports.  Currently,  WebCT  is  being 
used  by  some  faculty  and  is  considered  to  be  in  its  early  stages  of  implementation.  Up  to  the  present  time,  no 
systematic  assessment  of  WebCT’ s implementation  has  taken  place.  This  paper  is  the  first  step  in  a series  of 
research  efforts  that  address  one  of  three  major  aspects  of  WebCT  at  GMU,  student  satisfaction  with  WebCT.  The 
other  two  are  faculty  satisfaction  with  WebCT  and  institutional  support  for  WebCT.  These  will  be  addressed  in  other 
papers. 

Before  beginning  a discussion  of  the  data  collected  on  student  satisfaction  with  WebCT,  a brief  exploration  into  the 
definition  of  an  Online  Course  Management  System  (OCMS)  is  provided  to  establish  a mutual  understanding  of 
OCMS. 

Online  Course  Management  Systems 

Online  course  management  systems  (OCMS)  can  be  defined  many  different  ways  depending  on  whom  you  ask.  For 
example,  Michael  J.  Albright  (1997)  describes  OCMS  as  "integrated  packages  that  provide  content  presentation, 
communication,  assignment  submission,  testing,  and  management  functions  in  a secure  environment,  easily 
accessible  via  a web  browser."  Murray  Goldberg  and  Sasan  Salari  (1997),  describe  WebCT  as  "a  tool  that  facilitates 
the  creation  of  sophisticated  World  Wide  Web-based  educational  environments.  It  does  this  in  three  ways: 

♦ It  provides  an  interface  allowing  the  design  of  the  presentation  of  the  course  (color  schemes,  page  layout,  etc). 

♦ It  provides  a set  of  educational  tools  to  facilitate  learning,  communication  and  collaboration. 

♦ It  provides  a set  of  administrative  tools  to  assist  the  instructor  in  the  process  of  management  and  continuous 
improvement  of  the  course." 

Related  Research 
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It  is  one  thing  to  develop  an  OCMS  and  use  it  for  either  classroom  support  or  full  distance  education;  it  is  another  to 
say  it  actually  makes  a difference  in  the  learning  process.  The  following  research  tries  to  flush  out  how  students  are 
reacting  and  learning  via  an  OCMS. 

According  to  the  Institute  for  Higher  Education  Policy  (1998)  “There  is  a relative  paucity  of  true,  original  research 
dedicated  to  explaining  or  predicting  phenomena  related  to  distance  learning.”  (Page  2).  Much  of  the  body  of 
research  that  does  exist  dealing  with  on-line  learning  “are  more  anecdotal  than  systematically  empirical  or  critical.” 
(Hara  & Kling  1999).  In  addition  Windschilt  (1998,  as  quoted  by  Hara  & Kling  1999)  also  notes  that  much  of  the 
published  literature  dealing  with  computer-mediated  on-line  instruction  consists  of  descriptions  of  software  or 
annodotal  accounts  of  how  technology  was  integrated  in  a course  or  classroom. 

Much  of  the  research  that  does  exist  dealing  with  on-line  learning  must  be  looked  at  with  a critical  eye,  since 
potential  threats  to  reliability  are  often  present  that  are  not  addressed  by  the  author(s).  (The  Institute  for  Higher 
Education  Policy,  1 998)  These  include  author  bias,  especially  since  many  of  the  authors  are  technology  experts  or 
teaching  computer-related  subjects  (Goldberg,  Goldberg,  Dabbagh  & Schmitt  1999)  failure  to  take  into  account  the 
affect  that  novelty  may  have  had  on  student  attitudes  towards  the  technology  (Goldberg,  Goldberg,  Pauley  & 
Cunningham  1999,  Merron  1998,  Fong  1999),  lack  of  a control  group  (Pauley  & Cunningham  1999,  Fong  1999, 
Dabbagh  & Schmitt  1999,  Pauley  & Cunningham  1999),  and  lack  of  a random  sample  of  subjects  (Goldberg, 
Goldberg,  Pauley  & Cunningham  1999,  Merron  1998,  Fong  1999,  Dabbagh  & Schmitt  1999,  Pauley  & 
Cunningham  1999). 

Keeping  the  above  flaws  in  mind,  let  us  look  at  what  the  research  does  say.  Most  of  the  existing  research  shows  that 
there  is  no  significant  difference  in  student  achievement  and  attitude  towards  instruction  between  classes  using  web- 
based  instruction  and  those  which  do  not  (The  Institute  for  Higher  Education  Policy,  1998;  Morss.  1999).  Schulman 
& Sims  (1999),  for  example,  found  that  there  was.  no  significant  difference  in  posttest  scores  of  the  web-using  and 
traditional  instruction  groups.  Merron  (1998)  in  his  study  of  3 courses  using  WebCT  to  varying  degrees,  found  that 
student  responses  were  overwhelmingly  positive.  All  of  the  students  wrote  that  they  would  take  an  on  line  course 
again.  Students  were  especially  impressed  with  the  ability  to  log-in  anytime,  quality  of  discussions,  breadth  of 
discussions,  and  convenience.  In  studying  accounting  students’  reaction  to  WebCT,  Fong  (1999)  found  that  the 
students  “perceived  [WebCT]  to  be  better  than  traditional  course  delivery  methods  in  form  of  paper  handouts  and 
lectures  in  many  course  elements  like  course  outline,  course  notes,  assignments,  presentations,  readings  & especially 
multiple  choice  exercises.”  (Page  60).  The  majority  of  these  students  also  felt  that  learning  with  WebCT  were  either 
equal  to  or  enhanced  traditional  learning  methods,  with  approximately  50%  stating  that  WebCT  was  “very 
beneficial”  & “very  useful.” 

In  studies  where  student  reaction  to  on-line  learning  was  measured  but  not  compared  to  student  satisfaction  with  a 
traditional  classroom,  Pauley  & Cunningham  (1999)  reported  that  94%  of  the  students  survey  would  take  another 
course  using  WebCT  and  93%  would  recommend  such  a course  to  their  friends. 

However,  Hara  & Kling  (1999)  in  their  study  of  student  frustrations  with  on-line  learning  noted  problems  such  as 
the  lack  of  non-verbal  cues  from  face-to-face  discussion,  technical  glitches,  problems  caused  by  student 
unfamiliarity  with  technological  tools,  lack  of  proper  feedback  & directions  from  instructor,  and  lack  of  accessibility 
to  the  needed  computer  equipment.  Wermet  & Oliges  (1999)  found  that  the  length  of  the  course  was  a factor  in 
student  satisfaction.  They  attributed  this  to  the  fact  that  in  longer  courses,  students  were  more  patient  with  technical 
glitches  since  they  had  were  not  as  pressed  for  time  in  completing  assignments.  They  also  found  that  access-  both 
in  terms  of  access  to  the  hardware  needed  and  ease  of  use  was  a problem.  Danchak  (1999)  found  differences  in 
student  attitudes  to  WebCT  based  on  experience  with  computers  and  age.  Less  experienced  computer-users  tended 
to  enjoy  WebCT  more  than  more  experienced  users,  a fact  that  the  author  attributed  to  a sort  of  “gee  whiz”  factor. 
In  addition,  older  students  were  more  likely  to  prefer  on  line  quizzes  than  younger  students.  Danchak  also  found 
that  the  people  who  posted  to  the  bulletin  board  most  often  disliked  it  the  most,  possibly  because  of  frustration  w/ 
the  technical  glitches  that  one  would  be  more  likely  to  encounter  the  more  that  one  used  the  bulletin  board  tool. 
Bayram,  in  a study  of  virtual  classrooms  in  Turkey,  pointed  to  problems  with  hardware,  navigation,  time,  web 
design,  cost.  However,  the  biggest  barriers  were  found  to  be  the  time  and  practice  with  the  software  required. 
Goldberg  (1997)  also  found  motivation  to  be  a key  issue.  Although  half  of  the  students  in  the  class  studied  liked  the 
self-paced  aspect  of  an  on-line  course,  the  other  half  felt  that  they  needed  external  motivation  to  keep  them  on  track 
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and  found  the  self-pacing  to  be  an  aspect  of  the  course  that  they  liked  least.  Merron’s  (1998)  students  also  found 
reading  all  the  messages  on  the  bulletin  board  to  be  very  time-consuming. 

In  comparisons  of  completely  on-line  courses  versus  courses  that  combine  face-to-face  meetings  with  on-line 
learning  and  courses  that  do  not  contain  any  web  components,  students  said  that  they  preferred  the  mixed  course  to  a 
completely  on-line  course  (Goldberg,  1 997).  Students  in  this  type  of  hybrid  class  were  also  more  likely  to  do  well 
than  students  in  a completely  on-line  course  (Goldberg,  1997;  Merron,  1998). 

An  Investigation  and  Review 

After  reviewing  the  various  studies  already  conducted  on  WebCT  and  OCMS,  the  need  to  conduct  research  at 
Mason  became  very  clear.  The  following  sections  discuss  the  methodology  used  and  threats  to  the  validity  and 
reliability  of  the  data  collected  for  this  initial  study  of  students  at  Mason  and  their  experiences  with  WebCT. 

Survey  Methodology 

There  were  several  steps  that  needed  to  be  planned  and  executed.  In  addition  to  identifying  the  actual  objective  of 
the  study,  the  audience  had  to  be  identified,  the  data  collection  process  had  to  be  defined,  the  collection  tool  had  to 
be  developed,  and  permission  needed  to  be  obtained  to  use  the  data  once  collected.  Each  of  these  steps  is  described 
below. 

The  objective  as  stated  in  the  introduction,  ”...  student  satisfaction  with  WebCT,”  was  the  over  all  objective  of  the 
study.  Because  it  was  not  clear  what  the  responses  might  be,  the  objective  was  kept  broad.  The  survey  was  designed 
to  explore  how  well  Mason  students  were  using  WebCT  and  how  well  WebCT  is  was  working  for  the  students.  It 
took  into  consideration  the  computer  experience  of  the  students  and  their  learning  process/needs  to  see  if  these  needs 
are  consistent  with  what  WebCT  has  to  offer.  Other  interests  explored  included  how  the  students  might  use  the  tools 
after  they  have  completed  their  courses. 

Several  factors  were  considered  when  attempting  to  identify  potential  participants  in  the  student  survey.  The  factors 
considered  were: 

♦ The  overwhelming  size  of  the  Mason  student  population  (26,000+)  and  the  fact  that  not  all  Mason  students  use 
WebCT  or  a like  product. 

♦ The  location  of  the  students  (most  off-campus  and  part-time) 

♦ The  inconsistent  use  of  WebCT  across  the  campus  and  across  the  disciplines 

♦ The  limited  time  available  to  conduct  the  survey  caused  in  part  by  the  need  to  collect  data  about  the  success  or 
failure  of  WebCT  in  time  for  decision-makers  to  decide  how  many  resources  will  be  provided  to  support 
WebCT. 

Based  on  the  above,  it  was  decided  that  a pre-existing,  large  group  of  students  new  to  WebCT  would  be  an 
acceptable  sample  audience  for  this  study.  The  participants  in  the  Graduate  School  of  Education's  Initiatives  in 
Educational  Transformation  (lET)  were  asked  to  be  the  sample  student  body.  They  are  a dedicated  group  of  120+ 
graduate  students  in  one  2-year  program  using  WebCT.  They  are  made  up  of  K-12  teachers  seeking  a Master's 
Degree.  Preliminary  studies  conducted  by  the  lET  faculty  indicated  that  the  participants  of  this  program  were  limited 
in  their  ability  to  use  technology,  particularly  web-based  technology.  They  were  also  selected  because  they  could  be 
reached  with  limited  effort.  Unlike  the  faculty  population,  the  student  population  at  Mason  is  geographically  and 
technologically  dispersed  and  without  one  or  two  simple  means  of  being  reached.  The  lET  program  offered  a way  to 
reach  a large  group  via  one  communication  device. 

Prior  to  submitting  a Human  Subject  Review  for  approval,  a meeting  was  held  with  the  lET  faculty  and  their 
approval  and  support  was  requested.  The  faculty  then  had  the  opportunity  to  review  the  survey  and  comment.  There 
were  a total  of  twenty-six  (26)  questions  asked  via  a web  page  survey  (23  were  a mix  of  Likert-Scale  and  Yes/No 
responses  and  3 were  written  response  questions).  The  answers  were  collected  into  a text-based  file  to  be  imported 
into  a database  for  tabulation. 
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Once  the  Human  Subjects  Review  was  approved,  a link  was  made  from  the  Program's  WebCT  home  page  to  the 
web  page  survey.  An  announcement  was  posted  in  WebCT  by  the  faculty  of  the  program  and  the  students  were 
asked  to  volunteer  their  time  and  take  the  survey.  The  survey  was  online  for  15  days. 

Threats  to  Validity  and  Reliability 

As  mentioned  above  in  the  student  survey  process  section,  the  participants  were  not  selected  randomly  and  several 
factors  went  into  that  decision.  Due  to  the  lack  of  random  and  complete  selection,  threats  to  validity  are  relevant. 
This  threat  will  have  to  be  considered  when  attempting  to  make  any  observations  from  the  survey  results.  Other 
possible  threats  include: 

♦ Impartiality  - The  lET  participants  received  training  on  WebCT  by  the  same  person  conducting  the  survey. 

♦ Audience  - The  participants  are  K-12  teachers  and  hence  do  not  represent  the  Mason  student  population. 

♦ Process  - They  were  asked  to  survey  WebCT  via  one  of  WebCT’s  communication  tools. 

♦ Timing  - They  were  surveyed  within  the  first  few  months  of  their  2-year  program  and  a follow-up  survey  could 
reveal  changes  in  their  attitudes  as  their  experiences  and  skill  change. 

♦ Time  Constraints  - They  are  all  educators  using  an  educational  tool  and  therefore  may  not  be  able  to  give  a 
balanced  view  of  how  the  tool  could  be  useful  in  other  work-related  situations. 

♦ Time  Constraints  - The  fact  that  a web-based  technology  is  being  surveyed  via  the  web  can  potentially  exclude 
those  who  are  web  timid  and  thus  could  slant  the  results  more  favorably. 

Findings 

The  overall  findings  are  positive  and  in  favor  of  WebCT.  The  following  sections  summarize  the  results  of  the 
survey  and  how  the  threats  discussed  earlier  might  have  influenced  the  results.  Assuming  all  the  participants  in  the 
lET  program  saw  the  link  to  the  survey,  approximately  20%  of  the  participants  responded.  A total  of  25  surveys 
were  collected  via  the  Internet,  as  discussed  above. 

Survey  Results 

In  summary,  the  majority  of  the  respondents  agreed  that  online  course  management  tools  and  WebCT  are  beneficial 
in  the  learning  process.  The  following  was  discovered: 

♦ Audience  - 100%  own  a computer  and  connect  to  the  Internet  from  home  and/or  work.  Over  50%  of  those  who 
responded  consider  themselves  proficient  in  the  use  of  a word  processing,  database,  email,  and  online 
communication  applications.  72%  have  taken  web-based  courses. 

♦ Learning  - 76%  of  the  respondents  agreed  that  they  were  likely  to  learn  more  effectively  using  on-line 
communication. 

♦ Time  Commitment  - 56%  indicated  that  they  would  not  spend  too  much  time  learning  technology  in  an  on-line 
communication  class,  while  48%  disagreed  and  felt  they  would.  56%  indicated  that  they  were  likely  to  spend 
too  much  time  keeping  up  with  on-line  communications,  while  only  28%  disagreed. 

♦ Sharing  - 92%  indicated  that  they  were  likely  to  use  the  information  shared  by  other  students.  96%  - 100%  of 
the  respondents  indicated  that  sharing  the  results  of  an  assignment  with  their  fellow  learners  helps  improve  the 
learning  process;  increases  their  efforts  to  do  a better  job  on  my  assignments;  and  is  comfortable  sharing  their 
assignment  results  with  their  fellow  learners. 

♦ Technology  and  Learning  - 88%  indicated  that  online  tools  such  as  chat,  discussion  fomms,  and  email  help  in 
the  learning  experience.  80%  agreed  that  WebCT  would  enhance  their  learning  experiences. 

♦ Learning  Technology  Application  - 84%  of  the  respondents  agreed  that  after  taking  a course  that  uses  online 
course  tools  such  as  those  found  in  WebCT,  they  would  be  able  to  apply  these  skills  in  their  personal  and/or 
professional  life.  While  60%  indicated  that  they  would  incorporate  online  collaboration  into  their  professional 
and/or  personal  life. 

♦ WebCT  Interface  - When  asked  if  the  interface  to  various  WebCT  tools  was  found  to  be  intuitive  and/or  easy 
to  use: 

♦ Chat  - 56%  agreed  and  24%  had  no  opinion 

♦ Bulletin  Board  - 56%  agreed  and  20%  disagreed 

♦ Calendar  - 92%  agreed 
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♦ Student  Home  Page  - 64%  agreed  while  20%  disagreed 

♦ Presentation  - 40%  agreed  but  24%  had  no  opinion. 

Survey  Results  Analyzed 

The  following  discussing  how  the  results  of  the  survey  could  have  been  skewed: 

♦ Survey  Process  - As  the  results  indicate,  the  25  respondents  are  easily  connected  to  the  web  and  agreed  that 
such  online  tools  provided  by  WebCT  are  beneficial  to  the  learning  process.  The  process  used  to  collect  the 
data,  via  a link  from  WebCT,  could  have  caused  the  results  to  be  disproportionately  in  favor  of  online  tools  such 
as  those  found  in  WebCT.  If  the  survey  had  been  conducted  using  a technology  neutral  process,  the  results  may 
have  been  more  evenly  distributed. 

♦ Time  • Time  was  a key-factor  in  this  study.  If  more  time  was  allowed,  participants  in  the  lET  program  that  are 
not  easily  connect  to  the  web  may  have  taken  the  survey  and  the  results  could  have  reflected  a different  mix  of 
opinions. 

♦ Audience  - The  positive  responses  with  respect  to  applying  their  online  skills  in  their  personal  and/or 
professional  life  could  reflect  an  interest  in  using  online  tools  in  their  own  classroom.  If  the  audience  had  been 
more  randomly  selected,  the  results  could  have  been  much  lower  because  many  disciplines  may  not  have  a need 
for  such  tools. 

The  Next  Step  - Future  Research 

There  are  two  directions  that  could  be  taken.  One  is  to  include  a broader  group  of  students  being  asked  general 
questions  and  the  other  is  to  get  more  focused  into  the  world  of  K-12.  Each  option  presents  new  opportunities  to 
understanding  the  impact  of  OCMS’  on  student  learning.  The  door  in  still  wide  open  on  the  pros  and  cons  of  OCMS 
and  the  more  research  that  is  conducted,  the  better  we  will  be  able  incorporate  its  powers  appropriately  into  the 
learning  environments. 
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Abstract:  This  paper  examines  some  social  implications  of  the  emerging  technology  of 
‘intelligent*  agents  for  learning  and  teaching.  The  educational  potential  of  this  technology  is 
tempered  by  the  potential  for  stereotype  and  caricature  in  an  educational  multimedia  environment, 
where  effects  often  need  to  be  larger  than  life,  and  dynamic  content  needs  to  isolate  salient  features. 
Believability  is  a crucial  requirement  of  such  agents,  and  exaggerated  features  may  be  associated 
with  increased  recognition.  Gender  provides  characteristics  that  are  both  easy  to  represent,  and  easy 
to  recognize.  The  paper  argues  that  the  hazards  of  stereotyping  in  the  development  of  mediated 
anthropomorphic  presences  are  significant,  and  need  to  be  explicitly  addressed  in  future  guidelines 
for  the  design  of  agents  that  are  to  have  any  role  in  the  education  of  our  children. 


1.  Introduction 

The  development  and  application  of  personality-rich  software  agents  has  been  the  focus  of  a great  deal  of  recent 
research.  It  has  been  suggested  that  such  software  will  constitute  the  next  major  metaphor  in  computing  by 
providing  “...capability  that  will  trigger  a basic  change  in  the  user  interface  metaphor:  computers  will  become 
assistants  rather  than  just  tools’*  (Ball  et  al,  1997).  Definitions  of  agents  vary:  for  the  purposes  of  this  paper  we 
consider  agents  as  systems  that  endeavour  to  use  autonomous  and  believable  characters  within  a situated  context 
(Franklin  and  Graesser,  1996). 

Computer  Based  Learning  is  one  application  area  for  these  systems.  It  has  been  suggested  that  agents  can  play  a 
significant  role  in  motivating  students  (Lester  et  al,  1997),  and  that  the  more  life-like  the  behaviours  exhibited  by  the 
agent  the  stronger  their  motivational  impact  will  be.  The  goal  for  many  agents  is  - as  with  Microsoft’s  Persona 
Project  - to  create  “computer  assistants’*  that  are  “lifelike’*  (Ball  et  al,  1997).  The  design  of  educational  interactive 
environments,  like  that  of  other  virtual  worlds,  requires  the  use  of  characters  that  can  respond  in  a way  that  is 
believable  and  “intelligent**  (Bates  1992).  The  framework  for  believability  is  therefore  implicitly  anthropomorphic. 

In  a constructivist  model,  learning  occurs  “through  interactions  with  one’s  environment  or  culture”  (Reiber,  1992). 
A virtual  environment  therefore  offers  potential  for  learning;  and  the  nature  of  that  environment  is  critical  if 
successful  learning  is  to  take  place.  We  suggest  in  this  paper  that  the  use  of  personality  rich  agents  in  computer 
based  learning  environments  may  lead  to  simplification  that  could  have  unwanted  effects.  In  most  situations  some 
simplification  of  the  design  must  occur  to  produce  a practical  and  useable  environment:  however,  if  learners  are 
presented  with  stereotypes  and  set  roles,  this  can  set  constraints  on  learning,  enquiry,  and  evaluation.  It  is  important 
that  developers  and  educators  be  aware  of  these  issues  in  developing  agents  for  teaching  and  learning. 
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2.  Slaves  to  the  user? 

Bates  (1992),  Lester  et  al  (1997),  Pisanich  et  al  (n.d.),  and  Reilly  (1997)  all  suggest  that  a key  to  effective  virtuality 
is  whether  the  user  is  able  to  “suspend  disbelief’:  believability  is  expressed  as  a suspension  of  disbelief.  The  social 
nature  of  agents  is  a matter  of  particular  importance  within  systems  that  depend  on  interactive  narrativity  for  their 
believability  (Waraich,  1998).  A key  to  this,  in  turn,  is  responsiveness  to  the  user:  the  user  will  suspend  disbelief 
only  if  interactions  with  the  agent  occur  within  the  bounds  of  the  user’s  existing  cultural  and  social  norms. 

The  concepts  of  believability  and  user-responsiveness  are  also  essential  to  the  vicarious  virtual  world  of 
pornography.  Characters  and  scenarios  are  developed  (usually  by  men  for  men)  that  portray  women  as  available:  the 
effect  depends  on  a “believability”  that  is  thoroughly  divorced  from  reality,  but  which  the  user  willingly  buys  into 
by  ‘suspending’  his  disbelief.  In  such  worlds  the  “user”  must  have  at  least  the  illusion  of  free  will  if  they  are  to 
experience  some  level  of  presence  in  the  simulated  world.  While  such  notions  are  common  terms  of  reference  for  the 
development  of  believable  agents,  it  may  be  noted  that  some  users  fail  to  differentiate  sufficiently  between  their 
vicarious,  suspended  virtual  worlds,  and  the  real  world.  As  agent  systems  become  more  sophisticated,  and  users  do 
suspend  their  disbelief,  agent  design  will  increasingly  carry  a burden  of  ethical  implications  and  social 
responsibilities. 

While  agents  must  be  able  to  exhibit  some  form  of  personality  in  order  to  be  believable,  the  models  of  representation 
are  themselves  culturally  determined.  Reilly  (1997)  states  that  “believable  agents  are  defined  to  be  interactive 
versions  of  quality  characters  in  traditional  artistic  media  like  film”.  Stereotypes  abound,  however,  in  the  world  of 
film:  from  Almodovar  to  Zinnemann,  Hollywood  to  Bollywood,  female  characters  are  almost  always  defined  in 
relation  to  a man  (usually  in  the  role  of  love-interest)  and  are  young  and  photogenic.  These  aspects  of  entertainment 
do  not  sit  easily  with  the  intention  to  educate. 

While  Lester  et  al  (1997)  do  not  explicitly  identify  stereotyping,  they  suggest  that  a key  challenge  in  creating 
believability  in  animated  pedagogical  agents  is  to  ensure  that  any  believability-enhancing  behaviour  must 
complement  (and  be  interleaved  with)  the  advisory  and  explanatory  behaviours  that  the  agents  perform.  The 
development  of  sophisticated  synthetic  personalities  is  a difficult  undertaking.  Yet  while  the  use  of  stereotypes  can 
provide  easy  recognition,  it  cannot  reasonably  be  said  to  complement  educational  behaviours. 


3.  Broad  Brushstrokes  and  character  aliasing:  the  problem  of  stereotypes 

Agents  may  therefore  be  shaped  by  social  assumptions:  these  hazards  are  compounded  by  the  limitations  of  digital 
representation.  Laurel  observes  that  “[w]hen  we  anthropomorphize  a machine  or  animal,  we  do  not  impute  human 
personality  in  all  its  subtle  complexity;  we  paint  with  bold  strokes,  thinking  only  of  those  traits  that  are  useful  to  us 
in  a particular  context”  (Laurel,  1 997). 

Human  personality  is  ‘analogue’  in  that  its  ‘values’  lie  on  a continuum  and  cannot  be  fully  quantized,  or  represented 
as  a series  of  discrete  values.  In  digitizing  personality,  the  problem  of  ‘aliasing’  is  inevitable.  Where  a picture 
rendered  into  a finite  set  of  pixels,  or  a soundwave  sampled  at  discrete  intervals,  may  exhibit  jaggy  staircasing  or 
noise,  the  ‘jaggies’  in  a digital  personality  may  manifest  as  stereotypes. 

However,  when  it  comes  to  the  personality  of  others,  such  ‘aliasing’  does  not  necessarily  spoil  the  effect.  As  Laurel 
comments,  the  illusion  of  lifelikeness  is  not  necessarily  destroyed  by  “over-simplification”:  users  “can  enjoy  (and 
believe  in)  even  one-dimensional  dramatic  characters”  (Laurel,  19xx).  Bates  too  suggests  that  agents  do  not  have  to 
be  sophisticated  as  long  as  it  does  not  actively  do  something  to  destroy  the  suspension  of  disbelief  (Bates,  1 992).  On 
the  one  hand  such  simplification  is  attractive  in  that  it  may  well  ease  the  task  of  designing  and  implementing  an 
agent  yet  at  the  same  time  produce  acceptable  results;  but  on  the  other  hand  simplification  is  likely  to  exaggerate  any 
stereotypes  that  have  (however  inadvertently)  been  incorporated  into  the  agent.  This  in  turn  clearly  has  the  potential 
to  shape  attitudes  and  to  reinforce  social  and  cultural  bias.  This  is  particularly  important  in  the  case  of  systems  that 
are  ‘educational’,  and  most  especially  those  aimed  at  children.  Young  people  are  less  likely  to  have  the 
discriminating  worldview  and  social  experience  required  to  identify  and  ignore  such  features  as  simplistic  or  ironic 
stereotypes,  and  may  well  accept  them  as  “normal”. 
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The  very  real  problem  facing  the  designer  of  synthetic  personalities  is  determining  what  the  effect  of  the  character 
will  be  both  within  the  simulation  and  towards  the  specific  user  of  the  system  in  the  real  world.  Each  learner  brings 
their  own  prior  knowledge  and  experiences  to  bear  on  the  interpretation  of  the  interactions  with  the  system. 

Categorization  is  an  attractive  method  for  digital  analysis  and  design.  Pisanich  et  al  (n.d.)  suggest  a personality 
taxonomy,  based  on  the  Big  Five  set  of  personality  traits  (Harary  et  al  1994).  These  consist  of  Expressive  Style 
(quiet,  introverted),  Interpersonal  Style  (aloof,  cruel,  warm,  generous),  Work  Style  (conscientious,  procrastination), 
Emotional  Style  (neuroticism,  calm,  emotionally  intense)  and  Intellectual  Style  (conservative,  creative).  It  is 
suggested  that  the  designer  of  a character  can  determine  a value  for  each  personality  trait  and  thus  determine  the 
features  of  a character’s  personality. 

Petta  defines  the  Emotional  as  “the  system  that  in  living  beings  continuously  monitors  the  relationship  of  the 
individual  to  its  world  “,  and  indicates  that  it  is  “highly  relevant  to  adequately  model  the  Emotional  within 
architectures  for  synthetic  actors,  with  respect  to  the  agent's  believability,  autonomy,  sociality,  and  faculty  to  engage 
humans”.  His  goal  is  to  model  “emotional  processing”  into  the  agent  architecture  framework,  using  principles  from 
“functional  appraisal  theory  in  emotion  research”.  Ad-hoc  solutions  to  “emotions  synthesis”  are  “brittle”. 


Reilly  (1997),  however,  suggests  that  believable  social  agents  cannot  be  created  from  parameterized  values  for 
personality  factors.  His  own  methodology  - designed  to  encompass  both  psychological  and  artistic  personality 
factors  - identifies  personality  quirks,  variable  emotions  (as  displayed  in  social  behaviour),  and  attitudes  that  are 
responsive  to  different  relationships  as  features  that  should  be  considered  in  the  development  of  believable  agents. 
Providing  such  a finer  resolution  in  the  rendering  of  an  agent’s  ‘personality’  will  be  more  difficult,  but  is  desirable  if 
an  agent  is  to  have  a degree  of  context-sensitivity  appropriate  to  an  educational  environment.  Within  Reilly’s 
methodology,  attention  to  social  norms  and  social  roles  may  be  imputed  to  be  particularly  important  factors  in  the 
development  of  pedagogical  agents:  an  informed  understanding  of  what  is  socially  acceptable  behaviour,  and  of  the 
role  that  teachers  play  in  society. 

While  Laurel  (1997)  sees  the  personification  of  computer  interfaces  as  attempts  to  “mediate  the  relationship  between 
the  labyrinthine  precision  of  computers  and  the  fuzzy  complexity  of  people”,  it  is  the  need  to  develop  types  and 
discrete  elements  to  represent  personality  that  can  very  readily  result  in  simplifications  and  stereotypes. 

4.  Gender:  defaults  and  stereotypes 

Gender  is  an  important  instance  and  illustration  of  stereotyping.  In  research  into  the  effects  of  gender  stereotypes  on 
the  design  of  educational  software.  Huff  & Cooper  concluded  that  a)  sex  bias  did  exist  in  software,  and  that  b)  its 
roots  lay  in  the  expectations  and  stereotypes  of  the  designer  (1987).  A key  factor  was  not  just  that  designers 
produced  software  with  different  characteristics  if  required  to  develop  specifically  for  boys  or  girls  respectively,  but 
that  the  software  produced  for  children  in  general  was  similar  to  that  designed  for  boys. 

The  'masculine  default'  runs  deep  in  most  cultures  - and  is  widely  reflected  in  the  morphology  and  usage  of 
language.  In  the  company  of  the  abstract,  representational,  and  imaginary  figures  that  populate  children's  stories  - 
stick-men,  animals,  aliens  - apparently  neutral  agents  and  other  'artificial  intelligences'  tend  to  default  as  masculine. 
The  quality  of  being  feminine  is  Just  that  - a trait,  rather  than  an  essence.  An  apron,  or  a set  of  fluttering  eyelashes, 
usually  suffices  to  ascribe  this  quality  to  an  anthropomorphized  animal  or  vehicle:  markers  of  gender  are  only 
needed  for  females.  If  the  Al  endeavour  concerns  making  computers  like  us,  then  what  are  'we'  like?  Gender  appears 
to  be  fundamental  to  our  social  conditioning.  Yet  there  is  an  obvious  danger  that  the  backgrounds,  personalities,  and, 
indeed,  the  graphical  representation  of  agents,  will  in  fact  represent  very  particular  values  and  assumptions. 

Within  the  claim  that  machines  do  not  have  a gender,  and  that,  in  designing  an  agent,  it  is  necessary  only  to  consider 
architecture,  lie  the  seeds  of  both  the  male  default  and  female  stereotypes  in  agent  representation. 

Agents  are  frequent  participants  in  on-line  Multi-User  Domain  (MUD)  communities.  Participants  also  include  a 
minority  of  males  who  identify  and  pursue  characters  bearing  female  names;  male  characters  who  log  on  under 
female  names  and  behave  in  the  sexually  aggressive  way  that  they  would  like  women  to  behave;  and  by  male 
characters  who  assume  that  a woman  will  need  their  help  and  their  company.  (Bruckman,  1996):  the  level  of 
unwanted  attention  and  harassment  in  virtual  worlds  is  an  exaggeration  of  - although  a fulfilment  of  intentions  that 
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undoubtedly  exist  in  - the  real  world.  Some  of  the  best-known  agents  - like  Julia  - are  built  to  respond  to  this  social 
environment.  Agents  that  are  simplistically  responsive  to  the  user  are  frequently  shaped  by  such  minorities.  A 
female  agent  may  not  have  to  be  young  and  photogenic  like  Lara  Croft  or  a Hollywood  star:  an  agent  that  is  even 
ready  to  flirt  or  look  pretty  builds  a particular  social  environment  that  is  not  likely  to  encourage  or  motivate  women 
and  girls.  MUDs  of  course  are  also  used  as  environments  for  teaching  and  learning  - to  teach,  for  example,  writing 
and  communication  - and  for  educators  and  scholars  to  share  experiences  and  information.  While  such  educational 
environments  could  be  expected  to  be  more  wholesome,  educators  frequently  look  to  ready-made  TinyMUD  robots 
as  retrainable  and  adaptable  agents,  and  it  is  still  necessary  to  pay  attention  to  the  general  pitfalls  of  constructing 
stereotyped  and  gendered  agents. 

There  is  no  particular  shortage  of  female  agents.  They  are  often  defined  as  female  - whether  descriptively  or 
graphically  - by  details  concerning  hair  and  other  bodily  features.  They  will  have  an  extensive  range  of  one-liners 
for  sexualized  exchanges  with  males  (which  appears  to  pass  as  ’emotional  intelligence’).  It  may  be  argued  that  this 
reflects  the  reality  of  life  in  a sexist  society,  and  in  particular  that  the  defensive  strategies  are  similar  to  those 
frequently  adopted  by  women  when  in  the  company  of  predatory  males.  Yet  it  may  also  be  argued  that  there  is  no 
need  for  such  strategies  on  the  part  of  an  AI,  and  that  such  characterizations  shape  expectations  as  much  as  it  reflects 
reality  - that  it  is  a facile  way  of  attaining  to  the  reality  of  the  male  ’user’. 

5.  Julia:  robot  with  sex 

One  ’female’  agent  program  has  many  lines  - alternately  coy,  feisty,  and  flirtatious  - for  dealing  with  frequent 
’virtual’  sexual  advances.  ’’Julia”  has  plenty  of  self-disclosing  loving  detail  - ‘she’  has  ’’close-cropped  frizzy  blond 
hair”.  Julia  is  described  as  a TinyMUD  robot  (Foner,  1993).  Written  by  Michael  Mauldin  of  Carnegie  Mellon 
University,  Julia  logs  into  MUDs  and  interacts  with  other  players.  While  Julia  does  not  operate  within  a learning 
environment,  MUDs  are  (as  noted  above)  used  as  educational  environments,  and  their  agents  seen  as  retrainable 
prototypes.  As  an  autonomous  agent,  a key  part  of  Julia’s  ‘personality’  is  sex  - about  which  ‘she’  is  rather  obviously 
assertiye: 

Robyn  says,  ‘Julia,  are  you  a woman?’ 

Julia  says,  ‘I  am  female.’ 

Robyn  says,  ‘and,  Julia,  are  you  human?’ 

Julia  says,  ‘Not  when  I’m  having  my  period,  Robyn.’ 

Robyn  laughs 
(Quoted  in  Foner,  1997) 

It  is  as  a female  that,  when  pushed  sufficiently,  Julia  takes  off  ‘her’  clothes  - ’fearfully’  - and  hands  them  to  an 
insistent  real  ’virtual’  male.  (Because  the  harasser  is  ‘whispering’,  his  lines  are  partly  invisible): 

Julia  says,  ‘Just  try  to  take  ’em,  buster!’ 

Julia  says,  ‘Help!  Rape!’ 

Julia  says,  ‘No,  I won’t.’ 

Julia  disrobes  fearfully  and  hands  her  clothes  to  Brandi. 

You  say,  ‘Julia,  please  give  me  your  clothes’ 

Julia  says,  ‘You’ll  have  to  get  them  from  Brandi.’ 

(Quoted  in  Foner,  1997) 

This  may  be  ‘believable’,  but  it  is  hardly  desirable:  if  the  broadly-brushstroked,  low-resolution  nature  of  virtual 
personality  and  environment  can  be  put  to  positive  use,  there  is  in  fact  every  reason  for  excluding  some  features  of 
reality. 

While  Foner  reports  Mauldin  as  claiming  that  85%  of  Julia’s  source  code  is  the  same  as  his  prototype  robot,  Colin 
(Foner,  1993),  he  (Foner)  also  comments  that  “a  large  percentage  of  Julia’s  code  deals  with  detecting  and  deflecting 
passes”.  One  might  therefore  deduce  that  this  code  has  been  added  on  to  a male-default  in  order  to  represent  the 
quality  of  femininity.  Colin’s  small-talk  response  repertoire  detects  matches  for  'how  are  you’,  ‘how  do  you’,  ‘how’s 
life’,  ‘are  you  alright’,  ‘are  you  well’,  etc  (Foner,  1997).  Julia  (whose  code,  as  a frequent  Loebner  entrant,  is  not  in 
the  public  domain),  has  added  on  small-talk  that  signposts  specific  femininity: 
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Robyn  whispers,  ‘how  are  you  feeling?’  to  Julia. 

Julia  whispers,  ‘Terrible.’ 

Robyn  whispers,  ‘why,  julia?’  to  Julia. 

Julia  whispers,  ‘I  have  PMS  today.’ 

Flirting  can  appear  to  go  with  the  territory  of  feminine  psychological  attunement.  Erin  O’Malley,  a prototype 
character  described  by  Hayes-Roth  (1998),  is  endowed  with  some  of  the  “psychological  qualities”  necessary  for 
believability:  “personality,  motivation,  emotion  and  social  relationships”.  Her  role  is  that  of  a bartender,  and  as  a 
female  who  is  apparently  assumed  to  work  within  a male  environment,  she  needs  “to  flirt  a little”,  even  if  “she  won’t 
let  those  conversations  go  too  far”. 

The  growth  in  female  agents  can  perhaps  be  ascribed  in  part  to  the  discovery  of  "emotional  intelligence"  as  a key  to 
the  simulation  of  humanity:  a feminine  stereotype  in  itself.  Indeed,  various  commentators  believe  that  AI  is  a 
'natural*  place  for  women's  research:  its  links  with  cognition  and  psychology  are  tied  to  the  stereotyped  notion  that 
"women  are  generally  more  introspective,  attuned  to  psychology"  (Strok,  1992). 

Conclusion 

When  agents  are  deployed  - as  they  increasingly  are  - for  useful  purposes  such  as  education,  decisions  may  be  made 
regarding  contextual  stereotypes,  and  in  particular  the  allocation  of  gender.  If  the  agent  is  to  be  a character  of  no 
gender,  it  is  necessary  to  make  a point  of  this  - otherwise  it  will  generally  default  as  male.  There  is  no  intrinsic 
problem  of  believability  in  a specifically  ungendered  agent;  although  the  suspension  of  disbelief  in  agents  contains  a 
strong  element  of  preference  and  choice  on  the  part  of  the  believer.  Female  personae  may  be  used  often  for  the  same 
reasons  that  many  service  industries  have  become  feminized  - to  take  advantage  of  the  supposed  approachability  and 
communicative  qualities  of  women  (Strok,  1992).  This,  however,  is  something  of  an  'airline  hostess  syndrome*,  and 
tends  to  assume  both  a male  customer  audience  and  a young  woman  with  symmetrical  and  slim  features.  There  is 
clearly  a concern  that  the  application  of  agents  may  also  be  divided  on  a gendered  basis. 

The  mind  has  been  conceptualized  as  a machine  since  the  advent  of  industrialization:  and  just  as  our  cultures  shape 
the  development  of  responsive,  ‘believable’  agents,  so  in  turn  our  agent  creations  are  part  of  a new  defining 
technology,  a technology  that  has  the  power  to  shape  our  self-conception  and  social  attitudes.  In  developing  agents  - 
but  most  especially  where  there  is  an  explicit  intention  to  educate  and  to  influence  - developers  have  a broader 
social  responsibility  to  move  beyond  simplistic  behaviourism  and  mechanical  personality-response,  and  to  explicitly 
factor  out  stereotypes  before  they  take  root. 

The  design  and  development  of  software  personalities  is  undoubtedly  a complex  area  of  intense  interest.  While  the 
use  of  broad  brushstrokes  and  parameterised  and  typed  values  may  be  attractive,  there  is  correspondingly  a need  to 
consider  guidelines  for  the  ethical  and  social  development  of  educational  agents;  to  recognise  that  simplifications  in 
general,  and  stereotyping  in  particular,  can  be  inimical  to  the  goals  of  education;  and  to  show  appropriate  respect 
and  trust  for  these  new  personalities  who  will  help  us  to  teach  and  learn. 
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Abstract:  In  professional  organisations,  employees  are  called  on  to  evaluate  solutions,  solve  ill- 
defined  problems  and  engage  in  decision  making  based  on  actual  situations  and  events.  To 
develop  such  skills,  tertiary  learners  have  to  learn  to  engage  in  tasks  where  such  skills  are  likely 
to  develop,  and  learn  to  reflect  on  their  own  behaviour  and  that  of  peers.  In  the  research  literature 
on  learning  and  self-regulation,  self  and  peer  assessment  are  important  strategies  used  to  develop 
problem-solving  skills  that  can  be  transferred  to  the  workplace.  On-line  environments  utilising 
asynchronous  communication  tools  and  tasks  are  ideal  environments  for  the  refinement  of 
decision  making  and  problem  solving  skills.  This  case  study  profiles  an  on-line  approach  to 
developing  professional  project  management  skills  for  multimedia  developers  and  presents 
snapshot  views  of  an  online  learning  environment.  The  study  also  investigates  learners’ 
perceptions  of  the  value  of  peer  work  in  their  own  learning  and  shows  that  technology  can  support 
alternative  assessment  practices  leading  to  development  of  professional  skills  and  competencies. 


Introduction 

This  case  study  presents  an  argument  in  favour  of  fostering  personal  transferable  skills  as  a defining  quality  of  the 
learning  experience  in  higher  education  and  demonstrates  how  technology  can  be  used  to  support  a culture  of  peer 
support  to  foster  these  skills.  The  case  study  was  designed  in  response  to  current  social  and  economic  debates  in 
Australia  about  the  quality  of  undergraduate  education  and  of  the  qualities  that  university  graduates  should  possess 
(Candy,  Crebert  & O’Leary,  1994;  Assister,  1995).  During  the  last  ten  years  there  has  been  a major  reappraisal  of 
higher  education,  its  purpose,  outcomes  and  resourcing.  There  is  now  a more  pronounced  emphasis  on  the  higher 
education-employment  nexus,  and  particularly  on  the  skills  or  competencies  that  can  be  transferred  from  a 
university  setting  to  the  workplace.  In  this  changing  environment,  which  is  mirrored  in  New  Zealand  and  Europe,  it 
can  be  argued  that  peer  assessment  and  peer  learning  are  an  appropriate  response  to  the  preparation  of  tertiary 
students  for  the  workplace.  This  study  focuses  on  a context  where  peer  support  is  integrated  with  assessment  to 
develop  skills  through  interaction  and  learning.  These  skills  enable  students  to  move  more  easily  from  formal 
education  to  the  next  stage  of  their  lives  and  are  increasingly  being  demanded  by  employers. 


The  role  of  technology  in  fostering  professional  skills 

Information  and  communication  technologies  have  the  capacity  to  support  a wide  range  of  learning  goals  and  are 
now  integrated  into  teaching  approaches  of  many  higher  educational  institutions.  Laurillard  (1993)  for  instance 
suggests  that  computer-based  learning  has  a major  role  in  promoting: 

• self-directed  learning  and  increased  student  autonomy; 

• keeping  the  educational  system  in  line  with  technological  development; 

• increased  information  literacy,  ensuring  that  graduate  skills  are  in  tune  with  those  of  employers;  and 

• increased  productivity  and  efficiency  in  higher  education. 
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The  shift  to  student  self-direction  and  autonomy  means  that  students  need  to  take  more  responsibility  for  their  own 
learning,  but  many  need  assistance  in  achieving  this  skill.  Shaffer  & Resnick  (1999),  maintain  that  technology  can 
be  used  to  create  authentic  contexts  for  learning,  and  provide  resources  that  give  students  opportunities  for: 

• connectivity:  to  connect  to  the  world  outside  the  classroom,  to  research  topics  that  would  otherwise  be 
inaccessible,  to  access  experts  and  to  engage  in  conversation  with  peers; 

• computer  modelling:  to  create  simulations  that  assist  the  creation  of  authentic  tasks  and  contexts  for 
assessment;  and 

• epistemological  pluralism  ',  to  express  and  represent  ideas  in  many  different  ways. 

Applied  to  assessment,  representational  pluralism  enabled  by  computer  technology  expands  the  range  of  channels 
available  to  students  to  demonstrate  understanding  (Gardner,  1993;  Greeno,  1997).  For  example,  instead  of  using 
narrowly  defined  learning  outcomes  tested  by  examinations,  technology  offers  a total  environment  where  real  life 
skills,  such  as  written  and  verbal  communication,  collaboration  and  team  work  can  be  assessed  by  the  team  and 
tutor  by  giving  learners  multiple  channels  of  expression,  such  as  visualisation,  multimedia  presentations,  audio  and 
video.  Thus,  information  technologies  are  closely  inter-woven  with  the  quality  of  the  learning  experience,  and  can 
be  used  to  create  authentic  tasks  for  assessment. 


Fostering  deep  learning  through  peer  work  and  authentic  assessment 

Traditional  university  education  has  operated  within  a “transmissive  paradigm”,  emphasising  the  transfer  of 
knowledge  from  lecturer  to  student.  Such  a view  of  learning  is  not  conducive  to  meaningful,  active  learning  where 
students  take  a pro-active  role  in  questioning,  sharing  ideas  and  applying  prior  knowledge  to  new  ideas.  However, 
the  increased  emphasis  of  generic  transferable  skills  has  required  a re-alignment  of  teaching  practices  with  desired 
learning  outcomes  (Biggs,  1999).  This  means  that  if  independent  lifelong  learning  and  critical  skills  are  expected  of 
graduates,  teaching  methods  must  foster  such  processes  and  skills. 

In  contemporary  education  one  influential  group  of  researchers  has  identified  students’  approaches  to  be  either 
surface  level  or  deep  level  (Ramsden,  1992).  A deep  learning  approach  is  consistent  with  a search  for  knowledge 
and  understanding,  whereas  a surface  learner  is  concerned  only  with  passing  exams  by  memorising  facts.  Applied 
to  assessment  and  teaching  approaches  in  higher  education,  the  implication  is  that  the  creation  of  an  appropriate 
learning  environment  can  foster  a deep  approach.  This  can  be  achieved  by  enabling  learners  to  take  an  active  role 
in  learning  by  initiating,  managing,  monitoring,  reflecting  and  evaluating  learning  tasks  and  processes.  Gibbs 
(1992)  emphasises  that  a focus  on  process,  rather  than  content,  is  essential  in  promoting  active  learning  and  that 
evaluation  and  assessment  procedures  are  central  to  these  issues  as  students  interpret  the  objectives  of  a course  of 
study  according  to  the  demands  of  the  assessment  system.  For  example,  an  exam  requiring  recall  of  facts  will 
encourage  learners  to  adopt  a surface  approach,  whereas  assessment  of  collaborative  problem-solving  or  teamwork 
on  a project  will  emphasise  communication  skills,  planning  and  decision  making  and  foster  a deep  approach.  The 
relevance  of  this  to  educational  technology  is  that  we  can  use  the  attributes  of  technology  to  increase  learner 
autonomy  and  independence  by  designing  authentic  assessment  tasks  in  order  to  equip  learners  with  professional 
skills  and  attributes.  In  addition,  by  making  assessment  a ‘learning  event’  that  develops  process  knowledge  (rather 
than  a one-shot  examination)  we  bring  it  closer  to  the  context  of  the  workplace,  where  professionals  are  expected  to 
have  self-management  skills,  and  be  able  to  make  judgements  about  their  own  and  other’s  work  (Erhaut,  1994). 
Moreover,  traditional  university  examinations  do  not  test  for  deep  conceptual  understanding  (Entwistle  & 
Entwistle,  1991).  Indeed,  the  capacity  of  technology  to  foster  professional  skills  through  authentic  assessment  is  an 
area  of  research  that  is  just  beginning  to  be  explored. 


Alternative  assessment  using  technology 

In  recognition  of  the  limitations  of  traditional  university  assessment,  there  is  a new  wave  of  pedagogy  advocating 
‘alternative  assessment’  in  which  assessment  is  integrated  into  learning  and  fosters  understanding  of  learning 
processes  with  real-life  performance  as  opposed  to  a display  of  inert  knowledge  (Wiggins,  1998).  This  form  of 
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authentic  assessment  is  solidly  based  on  constructivism,  which  recognises  the  learner  as  the  “chief  architect”  of 
knowledge  building.  In  constructivist  learning  environments  there  is  social  interaction,  communication,  exchange 
of  views,  collaboration  and  support  for  learners  to  take  more  responsibility  for  the  learning  process  through  learner- 
centred  tasks  (McLoughlin  & Oliver,  1998;  Collis,  1998).  Socio-cultural  theory  is  based  on  similar  assumptions, 
and  many  theorists  have  highlighted  the  importance  of  reciprocal  understanding  and  transactional  dialogue  where 
knowledge  is  exchanged  and  modified  in  the  light  of  peer  feedback  (Crook  1 994;  Bruner,  1 990).  Salient  features  of 
constructivist  learning  environments  include  an  emphasis  on  the  following  aspects: 

• authenticity:  learning  is  located  in  actual  contexts  or  real  tasks; 

• group  work:  social  interaction  and  feedback  are  instrumental  in  communication  and  higher  order  thinking 
processes; 

• learner  control:  learners  are  active  in  defining  and  negotiating  learning  tasks;  and 

• scaffolded  learning:  learners  are  supported  as  they  progress  from  novice  learners  to  self-regulated  experts. 

The  use  of  the  WWW  to  create  assessment  tasks  offers  greater  adaptability  and  flexibility  than  traditional 
assessment  procedures  as  it  enables  the  collection  and  storage  of  continuous  data,  and  easily  created  micro- 
environments where  learners  solve  real  life  problems.  It  can  be  argued  that  the  move  towards  alterative  assessment 
paradigms  has  been  accelerated  by  technology  with  its  capacity  to  cope  with  a broad  array  of  activities,  tasks  and 
forums  for  assessment.  For  instance,  alternative  modes  of  assessment  encourage  the  use  of  multiple  assessment 
tasks,  and  multiple  modes  of  show-casing  student  achievement  through  portfolios,  multimedia  projects,  skills 
demonstrations  and  teamwork.  A further  important  contribution  made  by  technology  to  these  new  modes  of 
assessment  is  the  capacity  to  support  the  evaluation  of  learning  processes,  such  as  communication,  group  work  and 
collaborative  problem  solving,  as  opposed  to  a narrow  focus  on  a single  outcome  as  an  indicator  of  competence. 

Context  of  the  study:  Course  description 

The  context  of  the  study  was  a tertiary  academic  unit  called  “Interactive  Multimedia  Development  Methodologies” 
intended  to  develop  student  expertise  and  knowledge  in  project  management  for  developing  multimedia  in  a team- 
based  environment.  Currently,  this  unit  is  offered  in  face-to-face  mode  using  a combination  of  delivery  modes, 
including  small  group  work,  team  projects  and  problem  solving  tasks  facilitated  by  a tutor.  The  on-line  component 
provides  access  to  course  notes,  syllabus,  assessment  details,  access  to  previous  projects  as  well  as  group 
communication  facilities  such  as  e-mail,  bulletin  boards  and  problem  solving  tasks.  The  course  objectives  were 
closely  linked  to  the  professional  competencies  required  for  multimedia  development  in  the  industry  and  integrated 
authentic  assessment  tasks  where  learners  could  develop  multimedia  products  in  a team  environment,  thus 
replicating  the  skills  required  of  them  in  the  workplace.  A salient  feature  of  the  assessment  tasks  was  the  focus  on 
learning  processes,  rather  than  mastery  of  content  as  this  matched  the  orientation  of  the  course  towards  skills 
development.  The  assessment  tasks  and  associated  learning  processes  are  presented  in  Table  1. 

Table  1:  The  assessment  tasks  and  associated  learning  processes 


Assessment  task 

Learning  processes 

Web-based  product  development 

• Team  work,  planning,  decision  making  project 

management 

Peer  assessment  of  problem  solving 

• Higher  order  thinking  and  analysis 

• Development  of  criteria  for  assessment 

• Giving  feedback  to  peers 

• Reflection  on  feedback  leading  to  revision  of  ideas 

Search,  analyse  and  apply  relevant  knowledge  by 
finding  an  online  resource  to  support  the  solution  to 
the  problem. 

• Analysis,  reorganising  information  testing  for 

useability  and  application  to  an  authentic  task 

The  objective  of  the  peer  assessment  component  was  that  participants  would  evaluate  peer  responses  to  problem 
solving  and  discuss  issues  and  revisions  in  order  to  support  each  other.  The  unit  thus  integrated  process-based 
assessment  tasks  into  the  learning  environment  in  order  to  contribute  to  the  development  of  team-skills,  inquiry  and 
reflection  (Biggs,  1999).  The  unit  also  aimed  to  teach  content  knowledge  and  thinking  strategies  using  real  life 
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scenarios.  This  approach  to  on-line  learning  can  be  regarded  as  process-oriented  as  it  focuses  on  the  processes  of 
knowledge  construction  and  utilisation  (Volt,  1995;  Vermunt,  1995),  as  opposed  to  simple  mastery  of  content 
knowledge. 

Participants  and  procedures 

The  student  participants  were  all  postgraduates  and  consisted  of  five  females  and  four  males.  They  were 
undertaking  the  project  management  unit,  which  represented  a core  unit  in  the  Graduate  Diploma  of  Interactive 
Multimedia  Technologies.  This  unit  is  the  second  last  of  the  eight-unit  Diploma  course.  The  final  unit  these 
students  complete  is  an  industry  project,  where  the  students  are  required  to  implement  these  skills  in  a ‘real’ 
environment,  with  ‘real’  clients  and  produce  a ‘real’  product.  The  course  was  designed  to  maximise  peer  learning, 
peer  support,  feedback  and  interaction  among  the  group  members.  This  involved  groups  posting  solutions  to  ill- 
structured  problems  on  the  class  web  page,  monitoring  team  work  and  facilitating  team  building  processes  by 
encouraging  use  of  discussion  forums  where  team  members  could  deliberate,  plan  and  exchange  ideas.  The  actual 
problems  were  loosely  framed,  ill-defined  and  open-ended,  and  required  learners  to  analyse  existing  knowledge, 
suggest  a course  of  action  and  then  post  a solution.  In  addition,  students  were  asked  to  post  a URL  or  information 
source  that  specifically  supported  the  standpoint  adopted  in  solving  the  problem.  This  was  conducted  before 
students  attended  a seminar  where  they  presented  their  solution,  justified  their  approaches  and  made  comments  on 
other  students’  solutions. 

Students  were  required  to  submit  their  solution  at  a specific  time  during  the  week,  and  then  evaluate  each  other’s 
solutions  by  ranking  them.  During  the  seminar  the  strengths  and  weaknesses  of  each  solution  were  discussed  and 
students  had  an  opportunity  to  defend  their  approach.  The  process  of  developing  peer  support  and  feedback  was  an 
important  part  of  the  project  management  unit,  which  replicated  a learning  community.  Sharing  of  ideas,  resources, 
and  offering  reciprocal  support  created  social  relationships,  which  helped  students  to  work  in  teams.  McConnell’s 
(1999)  features  of  an  on-line  community  influenced  the  design  of  the  whole  environment: 

• openness  in  the  educational  process; 

• self-determination  in  learning; 

• a supportive  learning  environment; 

• a real  purpose  in  the  peer  assessment  process;  and 

• assessment  of  ongoing  learning  processes. 

The  following  four  ‘Snapshots’  of  the  peer-support  process  and  the  fostering  of  peer-feedback  through  assessment 
show  how  these  elements  were  integrated  into  the  technology-supported  environment  of  the  course. 


Snapshot  1:  Students  develop  criteria  for  peer  assessment 

Students  were  asked  to  develop  and  articulate  criteria  that  they  used  when  assessing  other  students’  on-line 
solutions  and  then  defend  these  in  class  discussion.  It  was  important  that  students  formulated  criteria  against  which 
they  could  measure  peer  solutions  to  the  problem  solving  tasks.  It  was  anticipated  that  students  would  be  able  to 
complete  this  task  without  much  difficulty  as  they  had  experience  of  writing  project  specifications  and  engaging  in 
project  evaluation. 

The  results  showed  that  the  majority  of  students  considered  the  following  issues  as  being  the  most  important  when 
presenting  solutions: 

• relevance  of  content  to  the  original  problem; 

• correctness  and  validity  of  content; 

• clarity,  presentation/format  and  grammar  of  writing  style;  and 

• the  ability  to  express  an  integrated  opinion  based  on  the  content. 

These  results  were  later  used  as  a ‘springboard’  to  refine  the  processes  of  evaluation  and  to  negotiate  agreed  on 
criteria  that  could  be  applied  to  all  posted  solutions.  For  example,  all  students  mentioned  style  and  format  as 
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essential  criteria  in  their  judgements  of  others’  work.  In  class,  they  were  asked  to  justify  this  response  in  an  attempt 
to  develop  metacognitive  processes  (Lieberman  1991;  Schraw  & Dennison,  1994).  Students  recognised  the  value  of 
this  in  developing  their  personal  transferable  skills. 

Snapshot  2:  Students  reflect  on  problem  solving  strategies 

Students  were  asked  to  reflect  on  the  problem-solving  processes  they  had  engaged  in  while  solving  problems  and 
then  asked  to  articulate  these  during  class  discussions.  This  enabled  the  students  to  reflect  on  their  own  strategies, 
to  identify  areas  of  weaknesses  and  to  conceptualise  ways  of  addressing  those  weaknesses.  Students  were  asked  to 
write  down  the  major  strategies  used  when  solving  problems.  Results  indicated  that  the  salient  features  of  student 
problem  solving  strategies  were: 

• researching  the  topic  thoroughly; 

• reading  the  question  carefully  and  considering  different  angles; 

• checking  that  the  content  used  is  correct; 

• comparing  ideas  with  other  students;  and 

• appropriate  to  the  development  of  project  management  skills. 

Snapshot  3:  Students  negotiate  assessment  methods  used 

This  example  relates  to  the  social  function  of  the  learning  community,  where  learners  relate  the  assessment  tasks  to 
their  own  personal  goals.  It  was  important  that  learners  came  to  regard  assessment  tasks  as  a way  of  engaging 
with  the  content  and  a means  of  developing  new  skills  through  ownership  and  acceptance  of  the  assessment 
process.  Students  were  asked  to  renegotiate  the  tasks  to  ensure  relevance  for  their  own  learning.  Most  of  students 
considered  the  problem-based  tasks  to  be: 

• relevant  to  the  unit  objectives; 

• helpful  in  understanding  multiple  perspective’s  from  other  students, 

• useful  for  reflecting  on  alternative  possible  solutions;  and 

• indicators  of  their  own  developing  skills. 

Discussion  and  implication  of  results 

Online  technologies  have  functionalities  that  enable  display  and  sharing  of  ideas,  open  discussion  of  solutions  and 
alternatives  with  articulation  of  strategies  by  participants.  Such  functionalities  support  greater  visibility  and 
openness  in  the  learning  process,  which  in  turn  foster  reflection  and  conceptualisation  among  learners.  In  the  study 
presented  here,  the  integration  of  technology  into  an  innovative  assessment  approach  resulted  in  deep  learning  and 
the  ‘Snapshots’  showed  that  students  were  engaged  in  active,  reflective  learning.  We  are  continuing  to  investigate 
how  technology  can  be  used  to  extend  a model  of  evaluation,  which  reflects  the  full  range  of  the  experiences  of 
learning.  We  now  have  evidence  that  technology  can  create  contexts  and  tools  for  exploration  of  alternative 
worldviews,  increase  the  visibility  of  thinking  processes,  lead  to  articulation  and  refinement  of  solutions  and  foster 
peer  dialogue  in  the  learning  process.  By  providing  scope  for  multiple  expressions  of  learning  achievement  we 
believe  that  new  models  of  technology  enhanced  assessment  are  just  beginning  to  emerge.  These  forms  of 
assessment  will  provide  indicators  of  a broad  array  of  competencies  and  recognise  that  learning  processes  and 
learning  outcomes  are  inseparable. 
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Abstract:  One  important  means  of  assessing  the  effectiveness  of  the  teaching-learning  transaction  in 
technology  supported  environments  is  through  analysis  of  the  learner  support  system.  Increasingly, 
as  learners  utilize  the  WWW  for  collaborative  learning,  support  systems  contribute  to  the  processes 
of  learning  and  assist  the  learner  in  developing  competencies  and  confidence  in  self-regulated 
learning,  social  interaction  and  self-evaluation.  Traditionally,  the  most  common  form  of  supported 
learning  has  been  an  apprenticeship,  where  a novice  learns  through  active  participation  in  a task, 
initially  only  peripherally  and  then  assuming  more  control  and  ownership.  Originating  in  the  socio- 
cultural perspective  of  Vygotskyan  theory  and  developed  by  later  theorists,  the  concept  of 
scaffolding  has  been  extended  by  practical  applications  in  technology-based  environments.  As  the 
World  Wide  Web  becomes  increasingly  integrated  into  the  delivery  of  learning  experiences  in 
tertiary  institutions,  the  concept  of  scaffolding  needs  to  be  redefined  because  it  is  not  readily 
translated  into  teaching  contexts  which  are  not  face-to-face,  as  in  on-line  environments.  The  aim  of 
this  paper  is  to  offer  a reconceptualisation  of  the  term  scaffolding  and  to  provide  actual  examples  of 
how  learners  can  be  supported  in  the  processes  of  constructivist  inquiry  in  WWW  environments. 


Introduction:  Conceptualising  learner  support 

The  term  ‘scaffolding*  is  increasingly  used  to  describe  certain  kinds  of  support  which  learners  receive  in  their 
interaction  with  experts,  teachers  and  mentors  as  they  develop  new  skills,  concepts  or  levels  of  understanding. 
Bruner,  Wood  & Ross  (1976)  originally  coined  the  term  scaffolding  as  a metaphor  to  describe  the  effective 
intervention  by  a peer,  adult  or  competent  person  in  the  learning  of  another  person.  The  term  can  be  traced  to 
Vygotsky’s  concept  of  “the  zone  of  proximal  development”,  that  is  the  actual  developmental  level  of  the 
learner  compared  with  the  level  of  potential  development  that  can  occur  with  guidance  or  collaboration  with  a 
more  competent  person.  In  technology  supported  learning  environments,  the  metaphor  of  scaffolding  is 
appealing  in  principle,  yet  elusive  and  problematic.  The  appeal  of  the  concept  lies  in  the  fact  that  it  directs 
attention  to  the  role  of  the  instructor  or  teacher  in  the  learning  process,  and  does  so  in  a way  which 
emphasizes  that  good  teaching  is  necessarily  responsive  to  the  state  of  understanding  achieved  by  particular 
learners.  In  earlier  research  therefore,  scaffolded  instruction  was  conceived  as  a joint  interaction  in  which  the 
teacher  and  the  learner  share  the  responsibility  for  learning  (Vygotsky,  1978;  Wood,  Bruner  & Ross,  1976).  In 
environments  mediated  by  technology,  a human  tutor,  peer-students  or  intelligent  agents  can  provide 
scaffolding  so  that  learners  attain  new  skills,  concepts  and  knowledge. 

Theoretical  perspectives  on  scaffolding 

It  is  important  to  trace  the  early  origins  of  research  on  scaffolding  in  order  to  appreciate  its  complexity.  The 
socio-cultural  approach  emanating  from  the  work  of  Vygotsky  has  had  a major  influence  on  the  development 
of  scaffolded  instruction  and  apprenticeship  models  of  learning  (Vygotsky,  1978;  Wood  & Wood,  1976; 
Rogoff  & Lave,  1984;  Collins,  Brown  & Newman,  1989).  Much  of  this  work  emphasizes  the  role  of  social 
interaction  as  a cultural  amplifier  to  extend  children’s  cognitive  processes,  with  an  adult  or  expert  introducing 
learners  to  the  conceptual  tools  available  in  society.  For  cognition  to  be  analyzed,  culture  and  context  are  the 
fundamental  units  of  consideration,  as  human  development  is  seen  to  be  located  and  immersed  in  social 
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practices  (Vygotsky,  1978).  This  perspective  resists  the  separation  of  the  individual  from  society  and  the  daily 
environment,  and  perceives  meaningful  activity  as  embedded  in  authentic  and  socially-created  situations.  This 
perspective  has  had  profound  and  far-reaching  influences  on  how  current  practitioners  design  learning 
environments  (e.g.,  Jarvela,  1995;  Roschelle  & Teasley,  1995).  Cognitive  change  can  occur  through  processes 
of  social  interaction  in  which  ideas  are  articulated,  shared,  revised,  modified  and  adopted  because  of  their 
relevance  to  the  cultural  context  (Roschelle  & Teasley,  1991;  Newman,  Griffin  & Cole,  1989).  Learners  | 
progress  through  developmental  changes  by  attempting  successive  approximations  of  the  learning  task, 
assisted  by  peers,  more  able  others  or  by  a tutor.  Support  offered  in  the  form  of  dialogue,  collaborative  tasks, 
structured  questioning  and  demonstration  of  skills  has  been  found  to  be  effective  in  enabling  cognitive  change 
(Hmelo  & Day,  1999;  Palincsar,  1986). 

The  mechanisms  for  assisting  learner  cognition  through  the  zone  of  proximal  development  (ZPD)  have  been 
extended  greatly  by  technology  applications.  Originally,  the  teacher’s  role  was  conceived  as  providing 
scaffolded  assistance  through  modelling,  contingency  management,  cognitive  structuring  and  feedback 
(Tharp  & Gallimore,  1988).  Through  modelling,  tasks,  skills  and  concepts  can  be  demonstrated  while 
retaining  complexity  and  authenticity,  so  that  learners  can  become  engaged  in  the  acquisition  of  new  skills. 
Contingency  management  is  concerned  with  recognizing  and  rewarding  learner  actions,  while  feedback 
enables  students  to  compare  themselves  to  others.  In  cognitive  structuring,  learners  are  assisted  to  organize 
their  own  experiences  following  the  provision  of  explanations,  or  meta-level  strategies  to  enable  students  to 
organize  their  own  thinking.  An  example  of  cognitive  structuring  can  be  constructing  a concept-map  of  a 
problem  situation.  Later,  these  mechanisms  are  internalized  and  become  metacognitive  strategies  for  students 
to  regulate  theirs  own  learning.  In  addition,  verbal  scaffolds  such  as  instructing,  questioning  and  cognitive 
structuring  enable  students  to  organize  their  own  activities  by  suggesting  meta  strategies  that  students  can 
acquire  so  that  teacher  support  becomes  “ ...  a heard,  regulating  voice,  a gradually  internalized  voice  that 
then  becomes  the  pupil’s  self-regulating  ‘still  small’  instructor.”  (Tharp  & Gallimore,  1988,  p.  57). 


Extending  the  concept  of  scaffolding 

Some  similarities  and  differences  emerge  when  we  compare  recent  work  on  scaffolding  with  earlier  research 
conducted  in  the  1980’s.  For  example,  much  of  the  work  of  the  Cognition  & Technology  Group  at  Vanderbilt 
(CTGV)  has  emphasized  the  notion  of  anchoring  instruction  in  everyday  authentic  contexts  (CTGV,  1993; 
1996).  However  a major  difference  is  that  earlier  work  (Hogan  & Pressley,  1997;  Newman,  Griffin  & Cole, 
1989;  Palincsar,  1986;  Rogoff  & Lave,  1984;  Tharp  & Gallimore,  1998;  Wood,  Bruner  & Ross,  1976)  has 
been  conducted  in  face-to  face  (F2F)  classrooms,  where  forms  of  verbal  interaction  were  the  most  common 
forms  of  scaffolding.  Teachers  and  learners  in  F2F  classrooms  share  the  same  space,  and  engage  in  learning 
processes  in  the  social  context  with  its  prescribed  rules,  roles  and  expectations.  This  may  limit  scaffolding  to 
teacher  initiated  forms  of  intervention.  For  example,  in  many  traditional  classrooms,  questioning  has  been 
shown  to  be  a form  of  social  control  (Edwards  & Westgate,  1994).  Many  of  these  social  constraints  are  not 
present  in  the  virtual  classrooms  or  in  contexts  where  learning  is  asynchronous.  In  addition,  the  nature  of 
scaffolding  in  such  face-to-face  classrooms  was  assumed  to  be  asymmetric  in  that  the  teacher  was  regarded  as 
the  expert,  and  the  student  as  novice.  Recent  advances  in  communications  technology  and  in  pedagogy 
envisage  an  active,  participatory  role  for  students,  as  initiators  and  co-participant  in  self-regulating  learning 
process  (Boeckaerts,  1995;  Brown  & Campione,  1994),  and  therefore  the  concept  of  scaffolding  needs  to  be 
extended  to  include  forms  of  peer  assessment. 

A consideration  of  more  recent  work  in  technology-supported  environments  illustrates  how  the  concept  of 
scaffolding  has  expanded  to  include  many  innovative  forms  of  support,  increased  responsibility  for  students 
and  a fading  of  the  directive  of  asymmetrical  aspect  of  earlier  work  on  scaffolding  (e.g.  Rada  & Yazdani, 
1998).  While  Vygotskyan  theory  provides  the  theoretical  anchoring  for  the  concept  of  scaffolding  by  making 
an  explicit  connection  between  social  interaction  and  cognitive  development,  other  forms  of  support  can  be 
provided  by  technology  thus  enabling  learners  to  engage  in  cognitive  change  and  skills  advancement. 

Designing  multiple  forms  of  WWW-based  course  supports 

As  the  World  Wide  Web  becomes  increasingly  integrated  into  the  delivery  of  learning  experiences  in  higher 
education,  the  concept  of  scaffolding  needs  to  be  redefined  because  it  is  not  readily  translated  into  contexts 
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where  the  teacher  is  not  present,  as  in  on-line  environments.  This  calls  for  a reconsideration  of  the  nature  of 
scaffolding  and  for  the  alignment  of  theory  with  practice  in  higher  education  (Collis,  1997;  1998).  As  yet, 
research  focusing  on  the  nature  of  scaffolds  and  their  functions  in  specific  contexts  of  technology-supported 
learning  is  limited.  Through  the  provision  of  examples  from  a range  of  contexts  where  technology  is  used  to 
mediate  the  teaching  transaction,  it  is  possible  to  show  that  the  notion  of  scaffolding  offers  a way  of 
conceptualizing  the  process  of  effective  learning  by: 

• Reducing  the  scope  for  failure  in  the  task  that  the  learner  is  attempting. 

• Enabling  learners  to  accomplish  a task  that  they  would  not  be  able  to  achieve  on  their  own. 

• Bringing  learners  closer  to  a state  of  independent  competence. 

As  technology  extends  learning  beyond  the  classroom  to  learning  communities,  so  must  roles  and  concepts  of 
learning  and  teaching  be  reconsidered  (Collis,  1998).  In  WWW-supported  learning  environments,  distributed 
groups  of  learners  can  be  supported  in  the  learning  process  by  using  different  technological  functionalities, 
which  support  dialogue  and  interaction  (see  Table  1).  With  its  great  potential  for  collaborative  learning, 
particular  forms  of  scaffolding  are  needed  to  provide  models,  examples  and  support  for  the  processes  of  active 
learning  characterized  by: 

• Self-responsibility  for  thinking  and  learning. 

• Awareness  of  social  responsibility. 

• Thinking  and  acting  scientific  processes. 

• Relating  group  process  and  product  with  professional  practice. 

• Articulation  of  reflection. 

• Viewing  a problem  from  multiple  perspectives. 

Developing  WWW  functionalities  to  support,  or  scaffold  group  processes  and  cognition  can  support 
collaborative  work.  Collis  (1997)  has  ‘re-engineered’  academic  courses  and  developed  a number  of  ‘tools’  to 
enable  group  work,  sharing  of  resources,  ideas  and  so  that  processes  and  products  are  integrated.  Through  a 
shared  workspace  environment,  students  can  access  texts,  documents  and  other  resources,  add  resources  and 
interact  with  others  through  conferencing  facilities.  Table  1 displays  a number  of  scaffolding  solutions  using 
WWW  tools  to  enable  cognitive  outcomes  and  processes  that  underpin  successful  learning.  In  the  right 
column,  a list  of  scaffolds  afforded  by  WWW  tools  is  provided. 


Table  1:  Examples  of  scaffolds  afforded  by  WWW  functionalities 


WWW  Tool 

Cognitive  goal 

Learning  processes 

Scaffolding  afforded  by  tools 

E-Mail 

Articulation,  revision 
and  justification  of 
ideas 

Conversing,  Questioning  and 
group  dialogue 

* Online  mentoring 

* Peer  support 

* Guided  reflection 

Threaded  computer 
conference 

Group  dialogue 

Coilaborative  problem  solving, 
articulation  and  elaboration  of 
views,  reflection 

• Guided  reflection 

• Peer  support 

Frequently  asked 
question  space 

Questioning,  self- 
directed  learning 

Metacognitive  awareness,  self 
responsibility 

• Support  for  questioning 

• Peer  support 

• Tutor  support 

Hypelinked  access  to 
course  resources 

Self-regulated  learning 

Exploration,  searching  and 
evaluation  of  resources 

• Access  to  resources 

• Independent  inquiry 

Collaborative 
workspace  and 
workflow  tools 

Higher  order 
cognition,  social  and 
collaborative  skills 

Viewing  issues  form  multiple 
perspectives,  awareness  of  social 
responsibility 

• Resource  sharing 

• Management  of  group  processes 

Chat 

Articulation,  reflection, 
peer  interaction 

Socio-cognitive  dialogue, 

questioning,  per  support 

• Support  for  social  processes  and 
peer  dialogue 

Some  examples  of  key  indicators  of  effective  scaffolding  in  Web-based  environments  include: 

• The  provision  of  learning  resources  to  help  students  solve  their  own  problems  and  share  them  with 
others  (e.g.  Guzdial  & Kehoe,  1998). 
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• Offering  multiple  channels  of  communication  should  enable  conversation,  exchange  of  ideas  and 
discussion. 

• Provision  of  support  for  collaborative  tasks  and  development  of  higher  order  cognition. 

• Embedding  scaffolding  in  the  organization  of  a course  as  a planned  and  graded  activity  (Collis,  Winnips, 
& Moonen,  in  press). 

It  is  advocated  that  these  scaffolding  features  are  built  into  the  design  of  Web-based  course  support 
environments,  since  its  activities  tend  to  be  less  structured  than  face-to-face  instruction,  utilizing  principled 
design  processes  (Collis,  1997).  Approaches  for  structuring  the  activities  can  be  found  in  existing 
technological  approaches,  as  described  in  the  next  section. 

Innovative  approaches  to  assist  learning  using  technology-based  scaffolds 

Apart  from  utilizing  the  functionalities  of  the  WWW  to  support  learning,  recent  research  in  technology 
mediated  environments  presents  an  array  of  possibilities  and  perspectives  on  scaffolding.  By  investigating 
these  applications  it  is  possible  to  compare  and  extrapolate  common  features  and  propose  principles  for  future 
research.  Four  examples  of  scaffolded  instruction  using  hypermedia  provide  contrasting  scenarios  for  recent 
interpretations  of  scaffolded  instruction. 

Computer  supported-intentional  learning  environments  (CSILES) 

This  approach,  conceived  by  Scardamalia  & Bereiter  (1989;  1992;  1993)  provides  a powerful  collaborative 
medium  based  on  anchored  design  and  discourse  space,  in  which  students  can  negotiate  and  construct  new 
understandings.  In  the  environment,  the  teacher’s  role  is  transformed  from  that  of  manager  to  facilitator  of 
student  collaborative  processes.  A CSILE  is  an  experimental  computer  system  which  can  mediate  shared 
spaces  for  collaborative  knowledge  building.  The  basis  for  this  is  a shared  communal  database,  which  gives 
students  a common  space  to  create  and  communicate  the  ideas  and  representations  that  emerge  from 
individual  and  group  work.  In  addition  to  supporting  social  interactions  needed  for  shared  understanding,  it 
provides  facilities  required  for  reaching  reciprocal  understanding,  and  facilities  for  the  shared  product  to  be 
expanded,  altered,  clarified,  elaborated  and  manipulated  for  new  meanings  to  emerge.  A shared  database  of 
text  notes  and  graphics  notes  allows  learners  to  access  and  collaborate  on  the  creation  of  knowledge  objects. 
CSILES  have  inspired  further  work  and  have  provided  a supportive  medium  for  a number  of  projects 
(Cognition  and  Technology  Group  at  Vanderbilt,  1993). 

Intelligent  tutoring  systems  (ITS) 

In  an  intelligent  tutoring  system,  learners  are  guided  through  a learning  process  and  provided  with  a structure 
and  sequences  of  task  to  assist  them.  Well  known  examples  can  be  seen  in  the  work  of  Andersen  et  al 
(Anderson,  Boyle,  Corbett  & Lewis,  1990;  Anderson,  Boyle  & Reiser,  1985)  in  which  students  are  taught  to 
solve  algebra  word  problems,  develop  programs  and  generate  geometry  proofs.  By  reducing  the  complexity  of 
the  task  and  providing  cognitive  structuring,  an  ITS  can  scaffold  learning.  In  an  intelligent  tutoring  system,  a 
learner’s  progress  is  charted  against  an  expert  model  of  the  process,  which  the  student  is  expected  to  model. 
Intelligent  tutoring  systems  have  been  criticized  for  lack  of  authenticity  in  the  learning  task,  and  for  creating 
tasks  where  students  do  not  have  to  engage  in  real  life  problem  solving  (Gudzial  & Kehoe,  1998).  In  ITS 
environments,  collaboration  in  learning  is  less  essential  than  in  other  apprenticeship  settings. 

Goal-based  scenarios  (GBS) 

Goal-based  scenarios  are  a learning  setting  where  students  have  to  engage  in  an  authentic  setting  where  they 
are  presented  with  a goal  to  achieve.  The  objective  is  for  students  to  acquire  and  develop  the  prequisite 
process  skills  and  conceptual  knowledge  to  attain  the  goal  (Schank,  Fano,  Bell  & Jona,  1992,  Acovelli  & 
Gamble,  1997).  Students  are  provided  with  technology-based  resources  to  achieve  these  goals,  and  their 
performance  is  compared  to  that  of  a successful  model  of  the  process.  If  a learner  cannot  achieve  the  goal, 
scaffolding  is  provided  in  the  form  of  process  information  which  gives  corrective  feedback  in  story  form  to 
help  the  learner  to  address  the  problem.  In  a GBS  students  interact  with  agents  embedded  in  a system,  rather 
than  with  socially-based  collaborators  or  peers.  BGS  are  nevertheless  unable  to  provide  feedback  or  support 
for  complex  abstract  processes  where  there  is  no  single  solution. 

Design  support  environments  (DSE’s) 


Design  support  environments  are  aimed  at  supporting  students  through  a form  of  software  realized  scaffolding 
tailored  to  assist  students  engaged  in  design  of  software  or  instruction.  In  DSE’s  the  environment  is  simplified 
by  providing  a large  number  of  cases,  coaching  students  in  the  design  process  and  fading  the  scaffolding 
(Gudzial,  1998).  Instead  of  providing  students  with  problems,  they  simply  scaffold  the  design  process.  A 
further  feature  of  some  DSE’s  is  that  they  provide  adaptive  scaffolds,  where  students  can  choose  or  turn  off 
various  scaffolds  that  are  not  required,  thereby  fading  support.  A different  example  is  given  in  Collis, 
Winnips,  and  Moonen  (1999)  where  the  design  process  is  monitored  by  delivery  of  (WWW-based)  prototypes. 
Based  on  monitoring  information  of  groups,  the  design  process  is  scaffolded.  Scaffolding  is  done  by  sending 
text  messages  to  the  groups  with  personal  comments  to  the  subgroup  and  to  individuals  within  it,  via  the 
WWW  site,  expanding  on  the  prototype  that  is  delivered.  Often,  for  example,  the  instructor  would  mention 
his  or  her  own  feelings  about  the  product,  give  background  information,  or  add  some  ideas  for  the  design. 


Conclusion:  Minimum  design  specifications 

Each  of  these  forms  of  technology-based  scaffolds  offer  a unique  perspective  on  learner  support,  consistent 
with  the  original  conception  that  cognitive  growth  can  be  assisted  by  offering  relevant,  timely  and 
contextualised  assistance  (Collins,  Brown  & Newman,  1989).  While  each  form  of  scaffolding  provides 
support,  each  differs  in  the  level  of  social  support,  collaboration  with  peers  and  type  of  feedback  offered.  By 
considering  these  forms  of  scaffolding  a minimum  design  specification  might  be  based  on  what  we  already 
know  about  effective  interaction  between  tutor  and  learner,  or  between  learners  collaborating  in  a problem- 
solving task.  These  may  be  summarised  as  follows  for  technology  supported  environments: 

• Technology  can  provide  a bridge  between  a learner’s  existing  knowledge  and  skills  and  the  demands  of  a 
task;  by  providing  a structure  and  a link  between  the  known  and  the  unknown,  technology  can  support 
the  learners’  problem  solving. 

• Technology  can  provide  the  representational  tools  to  model  or  simulate  an  “out  of  reach”  concept,  thus 
creating  an  authentic  environment  for  the  learner. 

• Technology  can  enable  the  novice  to  undertake  a task  at  her/her  existing  level  of  competency,  and  then 
transfer  responsibility  to  the  learner. 

These  examples,  illustrated  in  our  talk  with  examples  from  tertiary  learning  contexts,  show  that  by  creating 
and  evaluating  forms  of  scaffolding  with  technology,  researchers  are  now  developing  more  principled  and 
innovative  forms  of  instructional  design  to  guide  the  process. 
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AbstractiSelf-regulation  is  arguably  the  most  enduring  and  recognised  outcome  of  learning, 
and  is  recognised  to  be  so  at  secondary,  tertiary  and  technical  levels  of  the  educational 
system.  Developing  self-direction  is  also  essential  to  success  in  professional  world  of  work. 
At  the  University  of  New  England  (UNE),  these  principles  are  endorsed  and  articulated  as 
outcomes  or  generic  attributes  that  undergraduates  need  to  attain.  In  order  to  implement  and 
foster  these  attributes,  design  guidelines  are  specified  in  order  to  utilise  particular  attributes 
of  online  technologies  to  design  a learning  community  characterised  by  peer  support, 
investigation,  collaboration  and  reflection.  Design  recommendations  derived  from 
constructivist  and  generative  learning  theories  are  applied  to  online  units  in  area  of  E- 
commerce  and  Accounting  Information  Systems  offered  by  UNE.  Task  design  is  linked  to 
preparation  for  professional  practice.  Our  experience  suggests  that  in  on-line  learning 
settings,  development  of  self-regulation  and  independent  learning  requires  a focus  on  social, 
experiential,  propositional  and  process  aspects  of  learning,  rather  than  on  purely  cognitive 
learning  episodes. 


Introduction 

Information  technologies  and  computer  mediated  learning  are  now  an  established  part  of  learning 
environments  in  higher  education,  and  much  software  enables  the  setting  up  of  bulletin  boards,  chat  and 
conferencing  where  learners  can  engage  in  discussion,  team  work  and  exchange  of  ideas.  However,  the 
changing  nature  of  tertiary  education  in  Australia  has  resulted  in  increased  emphasis  on  outcomes,  and  on 
skills  that  are  transferable  to  the  workplace.  Online  design  for  learning  must  therefore  become  more 
streamlined  and  outcomes  oriented  whilst  maintaining  fundamental  integrity.  Greater  emphasis  on  generic 
competencies  are  now  part  of  the  mission  statements  of  universities,  and  the  University  of  New  England  has 
developed  a list  of  generic  attributes  that  graduates  of  the  university  are  expected  to  attain.  Instead  of  being 
focused  on  acquisition  of  content  and  disciplinary  knowledge,  the  generic  attributes  are  expressed  in  terms  of 
core  skills  such  as  team  work,  information  literacy,  problem-solving,  social  responsibility  and  communication 
skills.  This  policy  has  resulted  in  a reassessment  of  the  approaches  to  teaching  and  learning,  activities  that 
cultivate  generic  attributes,  the  role  of  disciplinary  understanding  and  the  context  in  which  generic  skills  are 
fostered.  There  is  considerable  evidence  in  the  literature  that  that  the  most  important  competencies  for 
graduates  are  critical  thinking,  adaptability,  self-evaluation  and  the  capacity  to  be  self-regulated,  while  at  the 
same  time  having  the  skills  to  work  in  teams  (Candy,  1994). 


Three  kinds  of  knowledge  required  for  professional  education 
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Curriculum  planning  in  the  School  of  Accounting,  Finance  and  Entrepreneurship  at  the  University  of  New 
England  is  underpinned  by  a strong  orientation  to  preparing  students  for  professional  practice  and  a 
movement  away  from  a purely  content-based  focus  on  acquiring  facts,  decontextualised  knowledge  and 
disciplinary  content.  Preparation  of  learners  to  respond  to  change  and  a dynamic  workplace  are  now  values 
that  are  being 

integrated  into  the  planning  of  units  for  online  delivery.  The  new  focus  requires  different  forms  of 
“partnerships”  within  the  school,  such  as  partnerships  between  staff  and  students,  between  staff  and  real- 
world  professionals  and  between  staff  and  the  different  elements  of  the  curriculum.  Units  are  planned  to  cater 
for  the  development  of  three  kinds  of  knowledge,  based  on  the  research  of  Erhaut(1992): 

1 . Discipline-based  knowledge,  concepts  and  generalisations.  However,  as  this  kind  of  prepositional 
knowledge  base  is  expanding  and  becoming  obsolete  rapidly,  judicious  selection  of  content  must  be 
accompanied  by  pedagogies  that  encourage  learners  not  to  uncritically  accept  knowledge,  but  to  question 
and  evaluate. 

2.  The  second  kind  of  knowledge  revolves  around  the  interpretation  of  experience.  Often  prepositional 
knowledge  remains  inert,  unapplied  and  unchallenged.  Through  design  of  tasks  and  activities, 
assumptions  underlying  disciplinary  knowledge  are  brought  to  the  surface  so  that  they  can  be  related  to 
professional  practice  and  real  world  experience. 

3.  The  third  kind  of  knowledge  that  distinguishes  learning  for  the  professions  is  process  knowledge,  or 
knowledge  about  how  to  engage  in  professional  action.  Process  knowledge  includes  several  different 
forms  of  socio-cognitive  competencies,  ie,  the  ability  to  select  and  implement  appropriate  method  of 
inquiry,  integration  and  reorganizing  knowledge;  activities  such  as  planning,  problem-solving  and 
decision  making,  communicating  information  and  self-management. 

These  forms  of  process  knowledge  clearly  link  with  the  personal  and  interpersonal  skills  that  are  applied  in 
professional  settings.  The  challenge  is  to  use  information  and  communications  technologies  to  interweave 
propositional,  process  and  personal  knowledge.  This  is  achieved  by  prioritizing  the  professional  orientation  of 
units  and  by  developing  approaches  to  teaching  and  learning  which  prepare  students  for  independent,  self- 
regulated  learning. 


Professional  learning  on  the  WWW 

In  tertiary  settings  learning  is  increasingly  resource-based  and  many  learners  study  in  the  off-campus  mode. 
Technology  offers  a means  by  which  distance  students  can  join  the  learning  space  with  their  on-campus 
colleagues  (Oliver,  Omari  & Herrington,  1998;  McLoughlin  & Oliver,  1998).  To  date,  the  many  studies 
describing  computer  mediated  learning  support  in  university  settings  have  focussed  on  particular  learning 
activities  and  functionalities  that  enable  students  to  communicate  and  share  ideas.  However,  we  now  need  to 
push  beyond  present  boundaries  and  paradigms  of  social  learning,  and  focus  on  WWW  instructional  scenarios 
that  maximise  learning  outcomes  such  as  higher  order  learning,  self-regulation  and  self-management  and 
professional  skills.  Constructivist  learning  theories  which  emphasise  the  role  of  the  learner  in  the  learning 
process  are  increasingly  recognised  as  relevant,  and,  according  to  Reeves  (1994)  the  prime  emphasis  is  placed 
on  the  unique  interests,  styles,  motivations  and  capabilities  of  individual  learners  so  that  learning 
environments  can  be  tailored  to  them.  However,  with  the  increased  emphasis  on  learning  outcomes, 
educational  paradigms  for  on-line  learning  also  need  to  emphasise  the  value  of  intentional  learning  and  goal 
directed  behaviour  where  learners  take  an  intentional  orientation  towards  cognition,  become  aware  of  their 
own  learning  processes  and  engage  in  self-directed  learning  (Scardamalia  & Bereiter,  1992).  Where 
networked  communications  are  being  used  increasingly  to  achieve  the  educational  outcomes  of  generic 
competencies  and  independent  learning,  design  recommendations  are  needed  to  ensure  that  learning  activities 
are  supportive  of  desired  outcomes  of  transferable,  professional  skills. 


Research  on  cognitive  self-regulation  and  higher  order  thinking 

Existing  statements  in  current  research  explicitly  link  students’  self-regulatory  processes  with  higher  order 
thinking  and  life-long  learning  skills  (Candy,  1994).  Self-regulation  is  regarded  as  a desirable  educational 
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goal  as  it  is  linked  to  achievement,  effective  problem  solving  capacity  and  greater  self-realisation  (Boekaerts, 
1997;  Mayer,  1992).  Among  the  competencies  and  dispositions  which  students  are  expected  to  develop  are 
generic  thinking  skills,  independent  learning  strategies  and  information  handling  skills  such  as: 

• reflection  and  analysis  to  generate  and  refine  knowledge; 

• evaluation  and  synthesis  of  ideas; 

• conducting  independent  investigations  in  different  subject  areas; 

• effective  communication  and  group  work. 

At  a theoretical  level,  there  have  been  several  attempts  to  provide  an  integrative  framework  for  self-regulation 
and  higher  order  thinking,  (Resnick,  1987)  but  there  appears  to  be  no  general  consensus  in  the  area.  The 
concept  of  higher  order  thinking  adopted  from  the  work  of  Ennis  (1987)  provides  an  integrative  perspective 
on  the  issue  of  thinking  and  has  been  influential  in  shaping  curriculum  statements  about  higher  order 
thinking,  defined  as  problem  solving,  self-direction  and  metacognition.  In  order  to  suggest  strategies  and 
approaches  to  underpin  online  learning  and  development,  some  theorists  advocate  that  the  best  approach  is  to 
turn  to  analyses  of  excellent  teaching  and  examples  of  people  who  can  demonstrate  higher  order  thinking  (eg, 
Winne,  1995).  Pressley  (1995,  p.  207)  concludes  that  “ instruction  is  best  matched  to  students’  zone  of 
proximal  development,  includes  a great  deal  of  modeling  and  explanation  of  mature  thought,  and  involves 
massive  practice  doing  interesting  and  authentic  academic  tasks  with  teacher  and  peer  scaffolding  of  such 
tasks.”  In  addition,  Pressley  emphasises  that  development  of  such  expertise  is  a long-term  developmental 
process,  “with  real  expertise  in  academic  cognition  an  unlikely  achievement  in  undergraduates”.  The 
research  on  how  to  foster  self-regulation  has  been  controversial  and  lacking  in  consensus,  signaling  a need 
for  developmental  frameworks  for  instructional  designers  to  apply  to  WWW  environments,  which  can  be 
evaluated  in  practical  settings. 


Instructional  design:  Issues  of  theory  and  application 

If  we  consider  the  nature  and  purposes  of  instructional  design,  several  patterns  are  evident.  The  focus  of 
traditional  instructional  design  paradigms  has  been  the  creation  of  a learning  event  or  learning  episode  for 
students,  in  which  outcomes  are  measurable  and  the  process  is  repeatable  in  any  context.  For  example,  the 
instructional  design  models  of  Dick  & Carey  (Dick,  1996)  have  remained  fundamentally  unchanged,  and  until 
the  most  recent  version,  the  role  and  importance  of  the  learning  context  has  also  remained  unchanged.  There 
has  been  a consistent  emphasis  on  instructional  sequences  and  on  prescribing  activities  for  learners,  based  on 
the  assumption  that  all  learners  are  similar  and  that  needs  can  be  determined.  In  contrast,  constructivism, 
which  emphasises  learning  rather  than  instruction,  challenges  the  instructional  designer  to  go  beyond 
prescribed  sequences  and  isolated  learning  episodes.  Constructivism  challenges  the  fragmentation  of  learning 
into  steps,  and  the  whole  notion  of  an  ISD  ‘model’  becomes  dubious  because  the  ‘model’  is  no  longer 
relevant.  Constructivism  proposes  that  learner  is  the  centre  of  instruction.  Jonassen  et  al  (1995)  argue  that 
with  constructivism,  the  emphasis  has  changed  to  cognition  in  context,  and  to  the  design  of  the  whole 
learning  environment.  Other  theorists  have  also  supported  the  notion  of  a learning  environment.  For 
example,  Wilson  (1994;  35)  states  that: 

Thinking  of  instruction  as  an  environment  gives  emphasis  to  the  ‘place ' or  ‘space ' where  learning  occurs. 

At  a minimum,  a learning  environment  contains: 

• the  learner; 

• a setting  or  space  wherein  the  learner  acts,  using  tools  and  devices,  collecting  and  interpreting 
information,  interacting  perhaps  with  others... 

Effective  planning  for  learning  also  requires  utilisation  of  technologies  as  ‘cognitive  tools’  (Jonassen,  1997) 
and  an  understanding  of  theory  relating  to  learning.  This  holistic  perspective  sees  learner  and  teachers  as  co- 
participants and  learning  as  engagement  in  joint  tasks,  with  control  over  sequences,  pacing  and  learning  goals 
(Harper  & Hedberg,  1997).  The  whole  social  and  motivational  context  becomes  part  of  the  design  process. 
Like  Laurillard’s  (1993)  conversational  model  of  reciprocal  activities  that  mediate  learning,  (interaction, 
discussion,  adaptation  and  reflection)  the  emphasis  on  instructional  design  emphasised  by  constructivists  is 
influenced  by  socio-cultural  theory,  originating  in  the  work  of  Vygtosky,  (1978).  According  to  socio-cultural 
theory  all  learning  is  embedded  in  a social  context  and  all  interactions  that  occur  in  the  social  context  impinge 
on,  and  contribute  to  learning.  As  an  example  of  design  heuristics  or  design  goals,  Duffy  & Cunningham 
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(1996)  propose  several  principles  to  guide  instructional  design  based  on  socio-constructivist  theory.  In 
designing  activities  for  professional  learning,  these  principles  were  redefined  as  design  guidelines,  informed 
by  constructivist  and  cognitive  apprenticeship  models  of  learning  (Table  1). 


Designing  for  self-regulation  and  authentic  learning 

Constructivist  theory  by  itself  does  not  provide  guidelines  for  instruction.  Eclecticism  is  essential.  Intentional 
learning  theory  advocates  learning  as  a deliberate  activity,  with  a strong  focus  monitoring,  self-regulation 
and  goal  directed  learning  (Scardamalia  & Bereiter,  1994).  The  model  has  been  successful  in  promoting  a 
process-based  model  of  learning  at  UNE  and  also  in  shaping  learning  interactions  from  teacher-oriented 
episodes  to  joint  goal-oriented  task  between  peers  and  groups  of  learners  (Jarvela,  1995).  However,  theory 
needs  to  be  converted  into  practical  and  workable  design  for  tasks,  forms  of  interaction  and  assessment 
activities.  The  design  of  tasks,  activities  and  forms  of  assessment  must  be  related  to  learners'  needs,  employer 
expectations  regarding  outcomes  and  real  life  contexts. 

Table  1:  Design  recommendations  for  self-regulation  and  professional  learning 


Instructional  design 

recommendation 

Implementation  through  tasks,  interaction  and 
learning  activities 

Create  a community  of  learners  to 
create  a motivating  context 

Create  tasks  with  shared  goals,  creation  and  manipulation  of 
shared  spaces;  use  formal  and  informal  communication 
channels;  selective  use  of  outsiders  for  insights;  cooperative 
tasks  (Anderson  & Garrison,  1995) 

Provide  modeling  and  scaffolding  to 
support  the  development  of  expertise 

Create  learner  access  to  expert  models,  mentors  examples  and 
procedures  for  dealing  with  problems;  foster  peer  scaffolding 
and  mutual  support  through  discussions  and  group  tasks;  ensure 
student  ownership  and  transfer  of  responsibility  form  tutor  to 
learner 

Foster  multiple  perspectives  in  order  • 
to  foster  application  of  knowledge  to 
real  life  situations 

Increase  perspectives  on  issues  by  designing  collaborative  tasks, 
ill-defmed  problems  and  case-based  reasoning;  provide 

opportunities  to  express  different  points  of  view  through  verbal 
and  oral  discussion  and  cooperative  tasks 

Contextualise  learning  activities  to 
develop  strategic  knowledge 

Create  anchors,  real  life  scenarios  and  problems  based  in 
authentic  contexts;  provide  access  to  resources  used  by 
professionals  in  the  field 

Enable  articulation  of  ideas  and 
reasoning  processes  to  develop  process 
knowledge 

Provide  forums  for  text  based  and  verbal  modes  of  sharing 
ideas;  utilise  multiple  forms  of  representation  such  as  graphs, 
charts  and  diagrams  as  a basis  for  discussion 

Focus  on  learning  processes,  not 
products  to  cultivate  deeper 
understanding 

Intentional  learning  strategies,  explicit  methods  of  learning 
(Scardamalia  & Bereiter  1994).  Use  discussion  forums  and 
projects  to  increase  student  responsibility  and  ownership  of 
learning 

Design  problem-based  learning  tasks  to 
foster  mastery  of  a propositional 
knowledge  base 

Ensure  that  tasks  are  relevant,  authentic  and  motivating,  and 
have  the  potential  to  be  defined  by  learners.  Use  problem-based 
learning  and  tasks  requiring  research,  investigation,  and 
comparison  of  perspectives 

Support  reflection  to  enable  learners  to 
challenge  personal  assumptions 

Learners  should  have  the  opportunity  to  compare  their 
performance  with  experts  and  other  learners  at  varying  stages  of 
competence.  Design  collaborative  tasks  to  foster  metacognition 
and  self-evaluation 

Design  Recommendations:  Task  Design  in  AFM  272  Accounting  Information  Systems 
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Based  on  our  experience  in  developing  learning  environments  for  E-Commerce  and  Accounting  Information 
Systems^  we  can  make  explicit  design  recommendations  for  other  practitioners. 

1 . Propositional  information  needs  to  be  proceduralised  through  application  to  real  life  scenarios. 

2.  Learning  tasks  should  develop  strategic  skills  through  their  experience  with  novel  tasks. 

3.  Students  should  develop  process  knowledge  by  communicating  about  the  cognitive  strategies  they  apply 
to  a particular  problem. 

4.  Tasks  should  be  goal  directed  and  enable  students  to  redefine  them  and  formulate  their  own  criteria  for 
successful  integration  into  prior  experience. 

These  design  recommendations  outlined  were  operationalised  initially  in  the  subject  Accounting  Information 
Systems  which  was  offered  to  a cohort  of  eighty  2"**  year  accounting  undergraduate  students.  The 
propositional  information  was  identified  as  the  design  and  maintenance  of  an  accounting  information  system. 
In  addition  as  second  year  students  the  group  had  eighteen  months  prior  accounting  knowledge  and 
experience  which  the  task  specifically  encouraged  them  to  integrate  and  reformulate.  Focus  on  this  activity 
was  reinforced  by  a significant  assessment  mark  ascribed  to  it.  To  accomplish  the  design  goals: 

1 . Students  were  required  to  set  up  and  maintain  a computerised  accounting  system  using  the  computerised 
accounting  package  Mind  Your  Own  Business  (MYOB).  Highly  specific  information  was  given  to  students 
about  the  operation  of  the  program.  Little  information  was  given  about  the  accounting  elements  of  the 
task. 

2.  Students  were  required  to  search  the  Internet  to  find  the  relevance  of  the  program  they  were  using,  as  well 
as  confer  with  parents  and  friends  then  discuss  this  with  their  fellow  students  using  asynchronous  online 
methods. 

3.  Online  discussion  forums  were  set  up  for  students  to  confer  about  problems  they  were  having  with  the 
task  and  the  solutions  they  were  generating.  A system  of  'bonus  marks'  was  implemented  to  encourage 
students  who  focussed  on  solutions  rather  than  simply  post  'can't  do'  messages.  It  was  considered  counter 
productive  for  the  instructor  to  intervene  unless  there  was  a long  thread  of  misinformation. 

It  was  believed  that  the  tasks  outlined  above  incorporated  all  four  design  principles  and  led  to  an  integrated 
approach  to  information  literacy  and  encouraged  problem  solving  skills.  In  addition  the  real  world  relevance 
was  demonstrated  to  them  through  their  own  research.  Given  that  the  cohort  was  in  the  1 8 - 23  year  old  age 
group  it  was  assumed  that  their  life  experience  was  limited  it  was  thought  that  the  students  would  benefit  from 
a context  to  view  their  knowledge  of  accounting  information  systems.  This  thinking  led  to  the  formulation  of 
the  second  task  based  on  the  four  design  principles  and  a realisation  that  students  would  learn  about 
professional  activity  by  engaging  in  such  activity.  Therefore  local  businesses  were  approached  and  students 
were  placed  in  teams  to  analyse  the  accounting  information  systems  of  the  various  businesses.  Various  sub- 
tasks were  developed  to  enable  students  to  both  contextualise  accounting  information  systems  and  engage  in 
professional  activity.  Students  were  required  to: 

a.  Interview  the  proprietor/manager  of  the  business.  They  were  required  to  design  a set  of  and  post  them  to 
the  Bulletin  Board  for  feedback. 

b.  Dress  and  speak  appropriately  for  this  interview.  This  created  a good  deal  of  discussion  about  what  was 
‘appropriate”  in  a workplace  context. 

c.  Document  the  accounting  system  for  the  business. 

d.  Write  a report  for  the  proprietor  which  first  had  to  be  seen  by  the  instructor. 

e.  Design  and  post  a web  page  within  the  confines  of  the  course  software  about  the  systems  in  the  business 
f Give  a brief  oral  factual  presentation  about  their  findings. 

g.  Maintain  a good  working  relationship  with  the  team  members. 

The  two  tasks  combined  together  gave  students  a learning  experience  which  led  to  the  development  of  an 
acute  awareness  of  what  professional  activity  could  mean  to  each  of  them.  The  principles  which  underpinned 
the  tasks  have  also  been  applied  to  the  area  of  e-commerce  course. 

Conclusion 
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In  this  paper,  preparation  of  students  for  professional  practice  in  the  areas  of  accounting,  e-commerce  and 
financial  management  is  conceptualised  as  developing  expertise  in  three  integrated  forms  of  knowledge, 
propositional  knowledge,  interpretation  of  experience  and  process  knowledge.  Each  of  these  forms  of 
knowledge  can  be  developed  in  online  environments  though  task  and  assessment  design,  and  within  a 
framework  that  encompasses  intentional,  constructivist  learning. 
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Abstract.  In  the  rush  towards  web*based  learning,  courseware  management  tools  are  being  promoted 
as  a means  of  simplifying  the  creation  and  management  of  instructional  websites.  This  paper  examines 
such  claims  in  the  light  of  a unit  in  Project  Management  for  Interactive  Multimedia  at  Edith  Cowan 
University.  The  unit  was  delivered  in  two  incarnations;  the  first  was  using  the  tool  WebCT,  and  in  the 
following  semester  the  course  was  delivered  through  a website  designed  specifically  for  the  unit,  by 
making  use  of  ancillary  tools  which  were  custom  designed  or  available  as  third  party  products.  The 
two  implementations  are  compared  in  terms  of  ease  of  use  for  student  and  course  designer,  and  the 
potential  of  each  approach  to  support  specific  learning  strategies.  Given  the  scope  of  delivering  a unit 
such  as  this,  the  authors  found  that  using  a customised  web  approach  was  more  efficient  than  using  a 
courseware  management  tool  such  as  WebCT.  It  is  recommended  that  consideration  be  given  to  the 
nature  of  the  learning  environment  based  upon  the  proposed  learning  strategy  and  pragmatic  issues  of 
implementation  before  the  selection  of  a delivery  approach. 


Introduction 

This  paper  discusses  strategic  factors  and  issues  involved  in  deciding  what  approaches  should  be  taken  to  develop  on-line  units. 
On-line  approaches  are  being  advocated  as  a means  of  providing  flexible  learning  environments  that  deliver  learning  outcomes  in  an 
effective,  and  above  all  efficient,  manner.  Such  are  the  promises  of  on-line  delivery,  but  as  with  all  implementations  of  technology  in 
teaching  and  learning,  there  is  no  direct  evidence  that  the  Web  supports  effective  and  efficient  learning  any  better  than  traditional 
methods  (Russel,  1999). 

It  is  certainly  true  that  on-line  learning  can  reduce  the  costs  involved  in  organisational  infrastructure  such  as  purchasing  real 
estate  and  computer  laboratory  equipment.  There  also  appears  to  be  an  assumption  that  there  are  economies  of  scale  to  be  gained  in 
human  resources  by  reducing  student-teacher  contact  time  through  the  implementation  of  easy  to  use  tools  which  enable  educators  to 
easily  develop,  maintain,  and  update  learning  content.  Courseware  management  tools  such  as  TopClass  and  WebCT  are  promoted  as 
“easy-to-use  environment[s]  for  creating  sophisticated  WWW-based  courses  that  are  otherwise  beyond  the  ability  of  the  non 
computer  programmer”  (ULT,  1999)  and  claim  to  “make  it  easy  to  transform  existing  instructor-led  content  into  online  learning 
materials”  (WBT  Systems,  1999). 

However,  such  claims  are  not  easy  to  justify,  however.  While  there  are  many  studies  which  seek  to  compare  between  specific 
courseware  management  tools  (eg  Infoworld,  1998;  PC  Week,  1997),  the  efficiency  and  effectiveness  of  these  products  in  comparison 
to  customised  Web  based  environments  is  less  easy  to  define.  Where  student  numbers  are  small,  and  the  requirements  are  modest  in 
terms  of  functionality,  it  may  be  that  a purpose-built  Website  may  provide  a better  alternative  to  implementing  a courseware  product. 
The  potential  of  courseware  management  tools  is  dependent  upon  such  factors  as  the  ease  with  which  the  course  can  be  developed 
and  administered,  and  the  quality  of  the  interface  for  end  users,  but  also  the  extent  to  which  the  product  provides  flexibility  in  the 
development  of  on-line  learning  strategies.  It  may  be  that  while  such  tools  require  less  initial  investment  in  time  and  money,  the 
requirements  for  implementation,  maintenance,  and  their  inflexibility  with  regard  to  interface  and  the  types  of  learning  activities  they 
support  may  lead  to  less  effective  design,  and  consequently  less  successful  learning  than  customised  environments. 

This  paper  examines  the  use  of  a courseware  management  tool,  WebCT,  in  the  delivery  of  a single  undergraduate  unit  in 
Multimedia  Project  Management.  The  course  operated  using  WebCT  in  the  first  semester  of  the  academic  year,  and  was  then 
delivered  using  a customised  Web-based  environment  in  the  second  semester.  The  two  approaches  are  compared  in  terms  of: 

• ease  of  development  and  administration  by  lecturers; 

• ease  of  use  and  accessibility  for  the  students;  and 

• potential  for  supporting  instructional  strategies  appropriate  to  the  unit. 

A Taxonomy  of  Courseware  Management  Tools 

There  are  many  tools  available  which  have  been  designed  to  deliver  on-line  learning,  and  it  is  important  to  distinguish  between 
them.  While  some  products  are  designed  for  system  wide  implementation,  and  link  into  existing  enterprise  systems  such  as  student 
and  financial  databases,  many  are  less  complex  ‘shrinkwrapped’  products  which  attempt  to  offer  whole  solutions  to  education  and 
training  systems.  Typically,  these  maintain  student  records  in  proprietary  file  systems  and  are  designed  to  facilitate  courseware 
development  for  novice  developers  or  lecturers. 

Examples  of  courseware  management  systems  are  PlaNet  Manager  (Planet  Software,  1999),  Southrock  GEM  (Southrock 
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behavioural  objectives  as  well  the 


educational  philosophy  that  underpins  the  strategy. 

Traditional  approaches  to  computer  assisted  learning  have  generally  been  based  upon  object! vist  epistemology  and  behaviourist 
methods  (Reeves  & Reeves,  1997).  Contemporary  approaches,  however,  tend  to  focus  more  on  the  multitude  of  ways  in  which 
learning  can  be  mediated  through  cognition  and  social  interaction.  This  shift  in  educational  theory  has  been  in  tandem  with  the 
development  of  the  Internet  as  a learning  medium,  and  the  Internet  as  been  advocated  as  an  ideal  tool  for  social  constructivist 
teaming,  since  it  provides  the  networked  communication  necessary  for  the  collaboration  and  dynamic  content  delivery;  all  of  which 
were  impossible  in  fixed  media  such  as  CD-ROMs  (McMahon,  1997). 

Constructivist  approaches  emphasise  the  role  of  the  individual  in  generating  learning.  Rather  than  a process  of 
stimulus/response,  user’s  are  actively  engaged  in  making  meaning  through  cognitive  accommodation  and/or  assimilation  (Piaget, 
1977),  or  through  social  interaction  (Vygotsky,  1978).  Vygotsky  argued  that  learning  comes  about  through  social  negotiation  within  a 
cultural  context,  with  language  as  the  primary  enabling  tool.  This  social  constructivist  philosophy  has  been  expanded  on  recently, 
introducing  the  notion  of  cognitive  apprenticeship  (Brown,  Collins  & Duguid,  1989)  through  which  students  learn  in  a manner 
similar  to  traditional  apprenticeships.  The  students  access  expertise  through  mentors,  whose  role  is  to  facilitate  rather  than  teach,  and 
the  aim  of  learning  is  to  solve  realistic  and  practical  problems  in  an  authentic  setting.  Just  as  in  traditional  apprenticeships,  learners 
engage  in  activities  ‘on-the-job’  rather  than  through  the  teaching  of  abstract  concepts.  The  argument  is  that  students  are  better 
equipped  to  approach  non-familiar  problems  and  produce  solutions  that  are  appropriate  to  a given  culture. 

Social  Constructivism  formed  the  theoretical  basis  for  this  unitwith  the  ultimate  view  of  learners  as  developing  practitioners  in 
the  field  of  Interactive  Multimedia.  Strategies  used  in  the  project  management  course  included: 

• having  students  develop  a web  site  for  a client,  which  simulates  how  these  students  will  be  working  when  they  gain 
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employment; 

• having  students  form  project  teams  of  approximately  four  students  to  develop  a web  site  to  promote  collaboration; 

• promoting  asynchronous  discussion  and  collaboration  by  having  student  task  teams  (different  to  the  project  teams)  present  a 
weekly  topic  on  the  bulletin  boards; 

• promoting  synchronous  on-line  discussion  through  the  use  of  chat  facilities; 

• having  the  students  work  with  complex  problems  which  have  multiple  potential  solutions. 

WebCT  Development 

Transferring  this  course  to  an  on-line  environment  using  WebCT  required  careful  thought  and  planning.  Which  WebCT  icons 
would  be  useful  for  the  design  of  a home  page?  How  could  the  required  instructional  features  required  by  the  course  be  transferred  to 
WebCT?  In  order  to  answer  these  questions  it  was  necessary  to  firstly  learn  and  understand  how  to  use  all  the  functions  and  tools 
available  in  WebCT.  Much  reading,  advice  and  experimentation  with  WebCT  was  required  before  settling  on  a home  page  which 
appeared  to  satisfy  all  the  course  requirements.  It  became  obvious  that  there  were  many  ways  of  designing  the  home  page.  Using 
different  combinations  of  icons  in  WebCT  could  change  the  instructional  focus  of  the  course,  or  change  the  way  that  students 
accessed  the  content.  Also,  careful  consideration  of  interface  design  issues  was  needed,  as  too  many  or  inappropriately  designed  icons 
would  be  confusing  to  new  users  and  add  unnecessary  cognitive  load. 

After  much  consultation  with  other  WebCT  users  and  experimentation  with  WebCT  icons,  the  icons  shown  in  Figure  1 were 
used  on  the  WebCT  Home  Page.  Table  2 below  shows  the  order  of  importance  of  each  of  the  icons  for  the  instructional  strategy  used 
in  this  case. 


Table  2.  WebCT  Homepage  Icon  Ranking 


Rank 

WebCT  Icon  Used 

Description 

1 

Bulletin  Board 

Discussing  weekly  topics  and  any  general  questions 

2 

Password 

Changing  student  passwords 

3 

Homepage 

Each  student  presents  their  background  and  interests 

4 

Presentation 

Server  space  for  students  to  put  samples  of  work  for  comment  and  final  web 
sites 

4 

Technical  Support 

Help  for  students  in  down-loading  and  using  “Hotline”  to  transfer  files  into 
the  presentation  area 

4 

Past  Student  Projects 

Completed  web  sites  from  last  semesters  students 

5 

My  Progress 

Used  mainly  by  the  tutors  to  assess  the  amount  of  messages  posted  and  read 
by  each  student 

5 

Survey 

Survey  form  to  evaluate  the  course 

7 

PowerPoint  Presentations 

Lecture  slides  made  available 

7 

My  Record 

Shows  students  marks  for  different  assessment  points 

7 

Calendar 

A day  by  day  breakdown  of  student  activities  and  assessment  points 

10 

Chat 

Chat  tool 

10 

Whiteboard 

Interactive  whiteboard  tool 

* Ranking  Code:  1 = most  important,  10  ~ lease  important 


The  Bulletin  Boards  were  the  most  heavily  used  part  of  the  WebCT  environment.  As  shown  in  Figure  2,  a bulletin  board  was 
created  for  each  of  the  ten  weeks  that  content  was  delivered.  From  week  3 onwards,  a student  task  team  was  responsible  for  the 
delivery  of  this  content.  The  “Main”  bulletin  board,  which  had  over  one  third  of  the  total  messages  was  set  up  to  allow  students  to 
ask  questions  or  raise  issues  about  anything  other  than  the  weekly  discussion  topic. 

There  was  little  intervention  by  the  tutors  on  the  bulletin  boards,  except  to  assist  when  it  was  obvious  that  the  students  would 
not  resolve  an  issue  or  question.  Tutors  would  always  wait  a few  days  before  answering  a posted  question  in  order  to  give  other 
students  the  opportunity  of  answering,  and  hence  obtain  some  marks  for  bulletin  board  contributions  which  accounted  for  30%  of 
their  total  mark.  Students  were  encouraged  to  develop  discussion  threads,  which  were  relevant  to  a weekly  topic,  or  other  issues 
related  to  multimedia  project  management,  making  the  environment  truly  student  centred. 
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Figure  1.  WebCT  Home  Page 


The  five  elements  used  for  assessing  student’s  progress  in  the  unit  are  shown  in  table  3,  The  student’s  final  mark  was  a 
combination  of  project  team,  task  team,  individual,  client  and  peer  assessment  marks.  This  provided  a rich  form  of  assessment  for  the 
students,  but  was  found  to  be  time  consuming  by  all  the  tutors. 


Table  3.  Assessment  Methods 


Assessment  Method 

Project 

Team 

Task 

Team 

Individual 

Mark 

Client 

Mark 

Task 

Peer 

Mark 

Project 
Peer  Mark 

WebCT  - Quantitative  Measures  (Posted,  Read. 
Hits) 

✓ 

✓ 

WebCT  - Qualitative  Measures  Text  Analysis 

✓ 

✓ 

Web  Site  - Quality  of  final  product 

✓ 

Written  Assignment 

✓ 

✓ 

Reflective  Report 

V' 

Communication  and  Contribution 

✓ 

✓ 

WebCT  provided  comprehensive  quantitative  measures  for  recording  the  number  of  messages  “Posted”,  “Read”  and  “Hits” 
made  by  students.  This  gives  tutors  a quick  idea  of  how  students  are  using  the  WebCT  environment  ie  which  students  are  “lurking” 
and  not  posting  messages.  However,  while  it  can  provide  some  valuable  data  for  tutors  if  they  are  trying  to  assess  the  amount  of  effort 
individual  students  are  contributing  toward  the  unit,  it  does  not  give  a good  indication  of  the  ‘quality’  of  submissions.  In  order  to 
assess  this,  a content  analysis  model  based  on  Henri  (1992),  and  Gunawardena  et  al  (1997)  was  implemented.  While  this  was  an 
effective  approach,  which  supported  the  social  constructivist  philosophy  behind  the  unit,  it  was  cumbersome  to  implement,  and  was 
used  predominantly  because  bulletin  boards  are  the  primary  mode  of  collaboration  supported  by  WebCT. 


Figure  2:  WebCT  Bulletin  Board  Summary  of  Messages  Posted 


Customised  Web  Site  Development 

In  semester  2,  1999  the  same  project  management  course  presented  to  students  in  semester  1,  1999  using  WebCT,  was 
delivered  through  the  use  of  a customised  web  site.  The  web  site  fhttD://www>scam.cowan.edu.au:5425/units/imm3202/default.html) 
was  built  using  Macromedia  Dreamweaver  (Macromedia,  1998)  and  Filemaker  Pro  (Filemaker,  1999). 

As  with  constructing  the  WebCT  course  in  the  first  semester,  the  instructional  strategies  required  for  the  unit  were  carefully 
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considered  before  construction  on  the  site  started.  The  main  instructional  difference  that  occurred  in  the  second  semester  was 
changing  the  instructional  focus  from  the  bulletin  boards  in  the  first  semester  (as  with  WebCT)  to  a problem  solving  strategy. 


si  li^ 

.XX:  ■ ■■■  ■ ' ■ 

0 



Figure  3:  Customised  Web  Site 

As  shown  in  Figure  3,  the  site  is  relatively  simple  to  access  without  the  use  of  passwords.  There  was  no  need  to  set  up  student’s 
accounts  or  a database  to  enable  student  logins.  The  options  shown  were  specifically  chosen  to  meet  the  needs  of  the  syllabus  as  follows: 

• available  projects  option.  This  was  as  used  to  allow  students  to  choose  from  a list  of  available  project  topics  that  they  would  develop 
as  a team  during  the  semester.  All  the  topics  are  related  to  a project  management  concept,  such  as  interface  design,  legal  issues, 
analysis  etc.  When  students  selected  a project,  it  then  appears  in  the  right  hand  frame.  The  blue  underline  hyperlinks  to  a description 
of  the  project  and  the  circular  icon  hyperlinks  to  a student  presentation  area,  where  students  keep  putting  up  prototypes  for  peer  and 
tutor  evaluation; 

• Registration  option.  Was  used  to  collect  student  details  such  as  e-mail,  phone,  address,  skills,  previous  employment  and  other 
details.  Students  also  entered  a password,  which  then  enabled  them  to  go  back  into  the  database  and  change  their  details  if  necessary; 

• Message  Boards  option.  Allowed  the  use  of  bulletin  boards.  However,  in  this  semester  this  was  not  a high  priority  and  not  used  very 
heavily; 

• Post  a URL  option.  Allowed  students  to  share  useful  web  site  (and  also  attract  bonus  marks)  by  making  useful  URL’s  available  to 
other  students.  This  area  was  set  up  in  project  management  categories  so  that  when  URL’s  were  found  students  posted  them  to  the 
relevant  category,  along  with  a comment  about  the  site; 

• Software  option.  Presented  students  with  information  and  software  needed  to  transmit  files  from  their  desktop  computers  onto  the  file 
server  to  show  prototypes  and  in  order  to  receive  feedback; 

• Syllabus  Documents  option.  Allowed  students  to  down-load  syllabus  and  other  resources  needed  for  the  unit; 

• Previous  Projects  option.  Showed  past  student’s  project  work,  which  illustrated  the  standard  of  work  expected;  and 

• The  RON  SUB  Login  button.  This  gave  students  access  to  the  problem  submission  and  peer  assessment  environment,  which  was  a 
purpose  built  software  application.  It  provided  students  with  tools  to  submit  their  solution  and  then  rank  the  solutions  of  their  peers. 
Students  who  received  the  highest  number  of  votes  were  given  the  highest  scores. 

Discussion 

A comparison  of  the  WebCT  and  customised  web  site  environments  can  be  made  according  to  ease  of  development  and 
administration  by  lecturers,  ease  of  use  and  accessibility  for  the  students  and  potential  for  supporting  instructional  strategies  appropriate  to 
the  unit. 

Given  that  the  WebCT  site  was  not  a centralised  university  initiative,  without  direct  links  to  the  central  student  database  and  without 
central  technical  support,  it  was  a difficult  and  time  consuming  task  to  set  up  for  only  one  group  of  ninety  students.  Student  logins  and 
passwords  had  to  be  established  for  all  students  and  tutors,  along  with  other  initial  database  requirements.  Also,  any  WebCT  technical 
problems  were  generally  complex  in  nature  and  often  required  assistance  from  the  WebCT  on-line  help  facility,  which  sometimes  was  not 
fast  enough  to  solve  pressing  problems. 

In  comparison,  setting  up  the  customised  web  site  was  relatively  simple.  Even  without  any  technical  skills,  a site  such  as  shown  in 
Figure  3,  can  be  cheaply  and  quickly  created  by  a multimedia  developer  for  minimal  cost,  which  would  “pay  for  itself’,  as  it  would  not 
have  any  royalty  payments. 

With  the  customised  web  site  it  was  easier  to  add  customised  features  eg  the  inclusion  of  “RonSub”,  purpose  built  problem 
presentation  and  peer  assessment  tool,  was  easily  embedded  into  the  customised  web  page.  The  inclusion  of  previous  student  web  projects 
and  web  space  for  students  to  load  working  progress  of  their  prototypes  was  simpler  and  quicker  to  implement  in  the  customised  web  site 
environment.  To  present  these  within  the  WebCT  environment  requires  files  to  be  packaged  or  “zipped”  and  then  loaded  with  the  File 
Manager.  In  some  cases,  HTML  editing  is  required  to  make  the  web  pages  work  correctly. 

Training  students  to  use  the  WebCT  environment  required  one  week  of  the  course  getting  students  on-line  with  passwords,  and  also 
familiarising  them  with  WebCT  interfaces  for  Bulletin  Boards,  Presentation  areas.  File  Manager,  Personal  Home  Page  tools  etc.  The  tutors 
also  needed  training  to  become  familiar  with  the  WebCT  environment  and  interface.  Students  often  complained  about  difficulties  in 
logging  onto  the  WebCT  environment  (due  to  incorrect  passwords  or  technical  problems)  and  also  about  slow  response  times  and  screen 
refresh  rates.  This  has  not  been  the  case  with  the  customised  web  site  environment.  No  login  is  necessary  and  there  is  no  extra  cognitive 
load  in  learning  a new  interface  to  access  the  customised  web  site,  as  it  allows  the  flexibility  required  to  make  an  intuitive  interface. 
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Both  WebCT  and  the  customised  web  site  environment  supported  learning  strategies  needed  for  this  course,  based  on  social 
constructivist  learning  theory.  WebCT  offered  bulletin  boards,  chat,  whiteboard,  e-mail  (internal  to  WebCT),  public  home  pages, 
presentation  areas  and  on-line  note  taking  facilities.  The  customised  environment  used  a listserv,  e-mail  (university  accounts),  presentation 
areas  and  problem  posting  and  peer  assessment  area.  The  instructional  tools  used  in  the  customised  site  were  generally  easy  to  obtain, 
either  through  the  central  “Virtual  Campus”  (Edith  Cowan  University),  by  having  them  custom  built  (as  with  “RonSUB”)  or  by  accessing 
tools  from  different  suppliers  which  are  either  free,  or  very  cheap  to  implement  (see,  for  example,  the  freeware  ‘Hot  Potatoes’  suite  from 
Half-Baked  Software  [1999]). 

Conclusion 

The  choice  to  use  WebCT  or  any  other  courseware  management  tool  should  be  carefully  considered.  The  amount  of  initial  training 
needed  for  technicians,  lecturers,  tutors,  sessional  staff  members  and  students  is  significant.  Also,  there  are  considerable  overheads  in 
setting  up  logins  and  establishing  how  to  implement  the  tools  for  designing  a home  page. 

The  following  issues  should  be  carefully  considered  before  committing  to  a courseware  management  environment: 

• will  the  courseware  management  system  be  implemented  across  the  whole  university? 

• have  the  courseware  tools  been  carefully  considered  for  their  instructional  relevance? 

• how  easy  is  it  to  add  customised  features  if  needed? 

• has  a budget  been  allocated  for  training  lectures  and  students  in  how  to  access  and  use  the  new  environment? 

• has  a budget  been  allocated  for  central  technical  support  and  help  desk? 

• will  an  instructional  design  advisory  group  be  set  up  for  to  advise  lecturers? 

• do  all  the  levels  of  management  support  its  implementation? 

If  the  answer  to  some  of  the  above  is  “No”,  then  it  may  be  preferable  to  implement  a customised  solution  which  is  simpler,  cheaper 
and  more  efficient  on  server  resources.  Customised  solutions  would  still  require  lecturer  training,  technical  support  and  instructional 
support,  but  may  support  greater  flexibility  in  design  when  creating  an  environment  built  from  the  ‘ground  up’.  Also,  customised  websites 

support  finding  a solution  to  meet  pedagogical  needs  rather  than  manipulating  content  to  support  the  tools  available.  In  the  final  analysis, 

the  decision  must  be  driven  by  the  outcomes  required  from  the  course  that  is  being  developed  and  also  the  scale  of  the  site  being 
implemented. 
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Abstract.  Over  time,  strategies  have  evolved  from  learning  theory  that  aim  to 
ensure  learning  takes  place  within  a given  situation.  While  such  strategies  are 
well  understood  within  the  context  of  traditional  classroom  teaching,  and  to  a 
certain,  if  lesser,  extent  fixed  format  multimedia  (CD-ROM)  delivery  methods, 
there  has  been  little  study  beyond  individual  case  studies  as  to  which  strategies 
work  best  within  Web-based  learning  environments.  This  paper  seeks  to 
explore  the  concept  of  learning  on  the  Web  by  comparing  10  common 
strategies  across  the  three  media  of  face-to-face  classroom  environments. 

Multimedia  CD-ROM  software,  and  Web-based  learning  environments.  The 
strategies  vary  trom  didactic  modes  such  as  presentation  and  demonstration 
through  highly  interactive  student  centred  methods  such  as  problem  solving 
and  simulations,  to  collaboration  using  strategies  such  as  discussion.  It  is 
argued  that  the  Web  has  the  potential  to  deliver  learning  through  a range  of 
strategies,  but  that  the  constraints  and  opportunities  afforded  by  the 
environment  must  be  taken  into  consideration. 

Introduction  - The  Web  and  Learning 

Learning  is  a notoriously  difficult  concept  to  define.  It  can  be  described  as  a 
“persisting  change  in  human  performance  potential  ...  [brought]  about  as  a result  of 
the  learner’s  interaction  with  the  environment”  (Driscoll,  1994,  pp.  8-9),  but  in  reality, 
such  textbook  definitions  pose  more  questions  than  they  answer.  How,  for  example, 
do  learners  interact  with  the  environment?  What  types  of  experiences  enhance 
learning,  and  howl  As  will  be  seen,  these  questions  are  still  highly  contentious. 

Several  theories  have  been  proposed  to  explain  the  process  of  learning.  Some  view  it 
as  a process  of  enculturation  (Brown,  Collins  & Duguid,  1989),  a change  in  behaviour 
(Gagne  Briggs  & Wager,  1988),  or  the  acquisition  of  the  cognitive  skills  required  for 
“purposeful  remembering”  (Norman,  1982,  p.  361).  There  are  obviously  many 
distinctions  between  these  theories;  the  emphases  upon  external  variables,  social 
processes,  or  internal  mental  function  are  dependent  upon  the  theorists’  attitudes 
towards  learning  as  a science  as  well  as  deeply  ingrained  epistemological  and 
ontological  assumptions. 

Such  assumptions  inevitably  carry  through  to  the  final  learning  environment. 
Classroom  environments  have  formed  the  basis  of  formal  education  for  many  years, 
and  there  are  now  a number  of  well  understood  strategies  which  can  be  implemented 
to  enhance  learning  of  various  types.  In  the  area  of  Instructional  Technology,  too,  it 
could  be  argued  that  Interactive  Multimedia  (IMM)  in  CD-ROM  format  has  become  a 
mature  technology  for  delivering  educational  and  training  content.  With  this  maturity 
has  come  an  understanding  of  what  works  and  what  doesn’t.  IMM’s  potential  for 
“multi-sensory  delivery,  student  metacognitive  self  monitoring  assisted  by  instant 
feedback,  and  student  control  over  the  pacing  of  information  flow  and  the  pathways 
followed”  (Richardson,  1995)  has  marked  it  as  a medium  with  attributes  that  can  go 
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beyond  traditional  class  based  instruction.  It  has,  however,  also  lacked  the  social 
interaction  that  typifies  face  to  face  learning. 

In  some  respects,  the  World  Wide  Web  appears  to  bridge  these  two  approaches  to 
delivering  learning;  providing  a nexus  between  content-based  learning  through 
hyperlinked  Web  pages,  and  the  communication  afforded  by  real-time  chat  and 
asynchronous  e-mail.  Certainly,  it  seems  to  have  attracted  a good  deal  of  attention  as 
a learning  medium. 

This  is  particularly  true  of  the  tertiary  sector,  where  the  demand  for  flexible  modes  of 
study  and  the  need  to  reduce  costs  are  starting  to  erode  the  traditional  face  to  face 
learning  that  has  previously  characterised  it.  The  Web  is  promising  reduced  logistical 
overheads  since  it  is  significantly  less  expensive  to  produce  materials  electronically 
than  in  printed  form,  and  the  material  may  easily  be  kept  up  to  date  (Eklund,  Garrett, 
Ryan,  & Harvey,  1996).  While  it  has  been  argued  that  students  may  not  like  to  learn 
at  a distance  (Simonsen,  1995),  the  flexibility  afforded  through  distance  education  for 
the  student  and  the  savings  to  a university  in  contact  time  and  physical  infrastructure 
are  making  it  an  increasingly  attractive  option  for  both  providers  and  clients. 

It  can  not  be  said  yet,  however,  that  the  Web  in  its  current  form  is  a mature 
technology.  Certainly,  the  actual  use  of  the  Web  does  not  appear  to  live  up  to  its 
promise  as  yet.  A visit  to  the  World  Lecture  Hall  (University  of  Texas,  Austin,  1998) 
would  indicate  that  most  sites  are  based  around  the  use  of  the  Web  as  a repository  for 
information  rather  than  an  actual  learning  environment.  This  has  lead  to  the  criticism 
that  the  Web  is  simply  a “24  hour-a-day  glorified  whiteboard”  (Archee  & Duin, 
1995). 

Clearly,  the  strategies  common  to  classroom  environments  and  CD-ROM  based  IMM 
need  to  be  re-evaluated  in  terms  of  their  potential  for  the  Web.  This  paper  will 
examine  10  such  strategies.  Their  relative  potential  to  facilitate  learning  in  these  three 
environments  will  be  discussed,  and  recommendations  will  be  made  on  how  they  may 
be  best  implemented 

Learning  Strategies 

Such  examination  is  made  difficult  by  differing  notions  of  what  instructional  strategies  actually  are.  At 
a micro  level,  they  can  include  Cognitive  techniques  such  as  the  use  of  mnemonics,  elaboration,  and 
chunking  (West,  Farmer  & Wolff,  1991),  but  such  techniques  can  perhaps  better  be  defined  as  tactics 
since  they  deal  with  low  level  implementation  within  a broader  paradigm.  It  is  the  macro  level 
strategies,  such  as  the  use  of  tutorials,  drills  and  simulations  which  will  be  the  focus  of  this  analysis, 
since  it  is  these  broad  approaches  which  appear  to  categorise  most  types  of  learning  environments  and 
impact  most  heavily  on  the  nature  of  the  learning  environment. 

Newby,  Stepich,  Lehman  and  Russell’s  (1996)  ten  instructional  approaches  are  used  as  the  basis  for 
discussion,  and  are  outlined  below: 

• Presentation 

• Demonstration 

• Discussion 

• Co-operative  Learning 

• Discovery 

• Problem  Solving 

• Instructional  Games 
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• Simulation 

• Drill  and  Practice 

• Tutorial 

This  choice  is  necessarily  somewhat  arbitrary,  and  the  individual  approaches  are  not  mutually 
exclusive  (presentation,  for  example,  may  form  part  of  a tutorial).  However,  they  include  strategies  that 
are  often  used  in  many  settings.  Some  approaches,  such  as  tutorials,  drills  and  simulations,  are  common 
strategies  for  single  user  computer  assisted  learning  (Alessi  & Trollip,  1991),  while  techniques  such  as 
discussion  and  co-operative  learning  often  form  the  basis  of  classroom  experiences.  An  examination  of 
this  range  of  approaches  may  allow  for  a more  complete  discussion  of  the  potential  of  the  Web  for 
learning,  while  at  the  same  time  acknowledging  that  there  may  be  other  strategies  available  to  the 
instructional  designer. 

Each  strategy  will  be  discussed  in  terms  of  their  potential  of  each  approach  to  be  implemented  via  the 
Web  will  be  explored  with  the  aim  of  arriving  at  some  conclusions  about  how  best  the  Web  may  be 
used  to  foster  learning. 


Analysis  of  Strategies 


In  order  to  compare  strategies,  their  potential  for  learning  within  the  three  environments  of  the  Web, 
classroom,  and  stand-alone  Interactive  Multimedia  is  represented  as  a diagram  (figure  1).  For  the 
purposes  of  this  paper,  each  environment  is  treated  as  a ‘stand  alone’  approach  to  learning.  For 
example,  while  both  multimedia  and  the  Web  may  be  effective  tools  in  the  classroom,  the  focus  is  on 
the  features  of  each  environment  which  make  them  distinct  from  each  other,  rather  than  ways  in  which 
they  can  be  combined.  Each  strategy  may  pertain  to  any  of  the  three  orientations  in  each  diagram  to 
varying  degrees.  The  placement  of  the  strategies  within  the  diagrams  is  at  best  an  interpretation  based 
upon  existing  literature  and  is  open  to  discussion.  However,  they  serve  a useful  role  as  a heuristic  for 
exploring  the  relatedness  of  the  approaches  to  each  other  and  their  potential  implementations. 


Presentation 
Demonstration 
Discussion 
Co-operative 
Learning 
Discovery 
Problem  Solving 
Instructional  Games 
Simulation 
Drill  and  Practice 
Tutorial 


World  Wide  Web 


IMM  CD-ROM 


figure  1 . Analysis  of  Strategies  and  Learning  Environment 


Presentation 

Presentations  “relate,  dramatize,  or  otherwise  disseminate  information  to  learners”  (Newby,  Stepich, 
Lehman  & Russell,  1996,  p.  47)  and  are  a very  common  tool  in  many  learning  situations.  While  a 
limited  strategy,  they  are  an  efficient  means  of  delivering  information.  The  most  common  form  of 
presentation  in  higher  education  is  the  lecture,  which  is  usually  supplemented  by  a workshop  or 
tutorial.  In  traditional  computer  based  training,  too,  the  act  of  presenting  information  is  only  a 
component  of  the  learning  sequence  (Gagne,  Briggs,  & Wager,  1988).  The  cost  effectiveness  of 
presentations  (Newby,  Stepich,  Lehman  & Russell,  1996),  however,  means  that  they  will  continue  to 
be  integrated  into  traditional  instruction,  in  the  form  of  texts  and  lectures;  and  the  Web  is  a capable 
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medium  for  replicating  this  method. 

One  of  the  best  examples  of  this  is  the  latest  version  of  Microsoft  PowerPoint,  which  allows 
presentations  to  be  easily  converted  into  HTML.  Since  this  is  a common  lecture  presentation  tool,  it  has 
been  quickly  adopted  as  an  easy  approach  to  information  presentation.  The  danger,  of  course,  is  that 
the  Web  could  end  up  as  merely  an  information  resource,  rather  than  a genuine  learning  medium.  In 
this  sense  a presentation  strategy  may  even  be  viewed  as  ‘no  strategy’  beyond  the  convenience  of  using 
the  web  in  the  same  way  as  any  informational  material,  such  as  a text  book  or  encyclopedia. 

Demonstration 

Demonstrations  share  some  similarities  with  presentations  in  that  they  are  content  based  and 
transmissive  in  their  approach  to  learning.  However,  they  differ  from  presentations  in  that  the 
information  is  presented  in  a contextual  manner.  Students  “view  a real  or  life-like  example  of  the  skill 
or  procedure  to  be  learned”  (Newby,  Stepich,  Lehman  & Russell,  1996,  p.  48)  and  this  can  be  achieved 
both  on  the  computer  and  through  face-to-face  experiences. 

Demonstrations  are  common  practice  in  the  classroom;  science  experiments,  for  example,  can  be 
shown  to  students  saving  the  resources  in  time  and  money  that  would  be  expended  in  students 
completing  the  experiment  themselves.  Digital  video  can  replicate  this  experience  in  a multimedia 
environment.  The  high  level  of  realism,  and  the  ability  of  video  to  show  events  in  real  time  make  it  an 
ideal  medium  for  computer  based  demonstrations.  It  is  here,  however,  that  the  problems  of  the  Web 
become  manifest.  Media  intensive  environments  require  high  bandwidth  data  transmission,  which,  as 
has  been  shown,  is  still  an  unrealised  promise  of  on-line  learning.  Virginia  University's  award  winning 
Interactive  Frog  Dissection  (Kinzie,  1994)  exemplifies  this.  Videos  are  used  within  this  tutorial  to 
provide  safe  and  authentic  demonstrations  of  frog  dissections  for  use  in  high  schools,  but  despite  these 
ambitious  aims,  QuickTime  videos  prove  to  be  so  slow  to  download  that  any  sense  of  flow  is  lost. 

Discussion 

Discussion  “allows  students  to  actively  practice  problem  solving,  critical-thinking,  and  higher-level 
thinking  skills”  (Newby,  Stepich,  Lehman  & Russell,  1996,  p.  49)  thus  providing  access  to  deep 
learning,  making  use  of  prior  knowledge.  It  is  no  surprise  that  it  forms  a major  component  of  classroom 
learning,  however,  little  discussion  can  be  found  in  traditional  CD-ROM  based  multimedia,  which  is 
often  designed  as  a flexible  ‘stand-alone’  product.  The  global  networked  nature  of  the  Web  would 
appear  to  make  it  an  ideal  medium  for  this  strategy.  Traditional  Internet  communication  tools  such  as 
E-mail,  Newsgroups,  Internet  Relay  Chat,  and  MOOs  offer  both  the  rapid  synchronous  communication 
of  normal  speech  (in  the  form  of  text)  as  well  as  asynchronous  interaction  which  may  help  to  promote  a 
more  reflective  metacognitive  approach.  With  the  use  of  plug-ins  and  CGI  scripting,  such  facilities  are 
now  becoming  available  in  a more  cohesive  form  on  the  Web.  Examples  of  learning  through 
communication  can  be  seen  in  commercial  environments  such  as  TopClass  (WBT  Systems,  1997) 
which  provide  the  functionality  required  for  this  approach. 

Co-operative  Learning 

Group  work  forms  the  basis  of  many  classroom  activities,  and  is  a highly  valued  and  relevant  strategy 
since  it  could  be  argued  that  most  work  environments  involve  a high  level  of  collaboration  to  achieve  a 
common  goal  (Brown,  Collins  Duguid,  1989).  As  with  discussion  however,  co-operative  learning  is 
rare  in  multimedia  environments.  The  swing  away  from  networked  computing  in  the  1980s  and  the 
lack  of  any  practical  paradigm  for  intelligent  software  limited  students’  ability  to  interact  in  a co- 
operative way  with  each  other  or  the  machine. 

It  is  a trend,  however,  that  appears  to  be  reversing  with  the  advent  of  the  Web.  E-mail  is  a common 
collaborative  tool  among  academics  to  jointly  conduct  research  and  author  papers.  Such  an  approach  is 
now  trickling  down  to  high  schools  and  primary  schools.  This  author,  for  example,  was  recently 
approached  by  a high  school  teacher  in  the  United  States  of  America  who  wished  to  initiate 
collaborative  projects  between  his  students  and  Australian  students  of  geography.  Groupware,  a 
collaborative  tool  that  allows  users  to  share  work  files,  has  been  common  across  local  area  networks 
and  is  now  being  ably  adapted  to  the  Web  through  environments  such  as  Lotus  Corp’s  Notes,  which 
has  now  made  the  transition  from  local  area  network  to  the  Web  (Lotus  Corp,  1997). 


700 


Discovery  Learning 


The  learner  centredness  of  discovery  learning  and  emphasis  on  learning  by  induction  rather  than 
deduction  make  this  an  attractive  strategy  where  self  regulated  learning  is  the  goal.  A common 
argument  against  discovery  learning  though,  is  that  “for  the  majority  of  regular  school  subjects  and 
most  procedural  or  physical  skills,  a model  that  begins  with  the  presentation  of  information  [is]  more 
efficient  and  demonstrably  more  successful”  (Alessi  & Trollip,  1991).  Thus,  discovery  has  limited 
application  in  the  classroom,  though  many  make  use  of  it  in  a highly  structured  and  guided  way.  After 
all,  it  “is  legitimate  and  necessary  for  teaching  to  go  beyond  the  specific  experience,  to  abstract  the 
symbolic  representation  that  allows  the  learner  to  use  their  knowledge  in  an  unfamiliar  situation” 
(Laurrilard,  1993,  p.  19). 

Such  criticisms  can  also  be  applied  to  computer  based  discovery.  Rieber  (1993)  cites  Papert  (1980)  in 
arguing  that  “the  computer  offers  a powerful  medium  for  the  exploration  and  discovery  of  many  ideas, 
particularly  in  science  and  mathematics,  just  as  a young  child  might  explore  the  concepts  of  volume 
given  a sandbox,  and  mass  and  momentum  in  marbles”  (Rieber,  1993,  p.  5).  As  will  be  shown  later,  the 
multimedia  potential  of  computers  for  simulations  is  one  way  of  providing  such  an  experience, 
however  this  is  not  so  easily  achieved  through  the  Web.  In  one  sense,  it  may  be  argued  that  the  Web  is 
an  ideal  medium  for  discovery  learning.  The  self  directed  nature  of  web  ‘surfing’  may  undoubtedly 
lead  to  some  form  of  serendipity.  However,  a great  deal  of  frustration  is  associated  with  Web  use  when 
a student  has  not  developed  effective  searching  and  browsing  strategies  (Archee  & Duin,  1995)  and  it 
would  appear  that  care  is  required  when  implementing  such  a strategy. 

Problem  Solving 

Problem  solving  has  much  in  common  with  discovery,  but  is  necessarily  more  structured  in  that  a 
problem  must  be  solved.  Like  discovery  strategies,  problem  solving  usually  involves  the  generation  of 
learning  through  active  involvement,  in  this  case  through  a process  of  trial  and  error.  It  can  also  be  used 
as  a tool  for  the  practical  reinforcement  of  abstract  concepts. 

The  use  of  problems  as  reinforcement  is  common  to  many  classroom  situations;  in  a maths  class,  for 
example,  abstract  knowledge  of  calculus  can  be  applied  to  problems  involving  speed,  velocity,  and 
acceleration.  Situations  where  learning  is  actually  founded  in  authentic  problems  and  involves  trial  and 
error  are  less  common,  despite  the  potential  benefits  from  the  deeper  processing  required  within  this 
paradigm  (Reeves  & Reeves,  1997)  perhaps  as  much  because  of  time  and  curriculum  constraints  as 
pedagogical  philosophy. 

It  has  been  argued  that  one  role  of  the  computer  is  as  a cognitive  tool  that  can  be  used  as  a generic 
device  to  assist  the  development  of  problem  solving  skills.  Papert  (1993)  goes  so  far  as  to  describe 
them  as  tools  to  teach  one  to  think,  and  proposes  LOGO,  a simple  programming  language  as  an 
example  of  such  a tool.  It  is  an  environment  that  is  driven  by  the  problem  solving  process  rather  than 
any  inherent  content.  Such  highly  interactive  environments  are  yet  to  be  seen  on  the  Web,  but  there 
would  not  appear  to  be  any  great  hindrance  to  this  approach.  Indeed,  it  could  be  argued  that  the  trial 
and  error  use  of  the  Web  search  facilities  to  find  information  can  assist  students  in  developing  research 
skills. 

Instructional  Games 

Games  can  provide  an  engaging  scenario  for  learning,  containing  the  challenge,  curiosity,  control  and 
fantasy  required  for  a motivating  environment  (Malone,  1980) 

The  fact  that  games  seem  under-utilised  in  the  classroom  may  be  because  of  the  difficulty  of  designing 
games  which  intrinsically  support  learning  or  that  games  are  simply  perceived  as  incompatible  with 
learning,  particularly  as  students  become  older  (Rieber  1996).  It  is  on  the  computer  that  games  seem  to 
have  found  the  most  support,  and  in  fact  are  one  of  the  biggest  areas  of  software  development.  The 
relative  cost  and  time  associated  with  developing  games  for  specific  learning  goals  as  opposed  to 
traditional  computer  based  training,  however,  means  that  specifically  instructional  games  are  rare. 
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The  same  is  true  for  the  Web.  Rieber’s  Space  Shuttle  Commander  (1994),  an  Authorware  based 
microworld,  is  one  of  the  few  games  available  via  this  medium.  It  is  without  doubt  true  that 
instructional  gaming  via  the  Web  is  yet  to  become  a major  use  of  the  medium.  The  high  quality  of  the 
media  rich  games  available  on  CD-ROM  would  suggest  that  there  would  be  a lot  of  work  necessary 
before  students  would  accept  such  environments.  This  does  not  mean  that  it  is  an  impossible  task, 
however.  The  emergence  of  servers  which  provide  gaming  between  remote  players  of  Arcade  style 
games  such  as  DOOM  and  Quake  (Id  Software,  1997),  show  the  potential  of  hybrid  combinations 
where  the  major  media  elements  and  game  engine  resides  on  the  client  computer,  data  transfer  being 
restricted  to  player  to  player  interaction.  It  must  also  be  acknowledged  that  text  based  adventure  games 
were  once  popular  despite  their  lack  of  multimedia.  Whether  contemporary  players  would  tolerate  the 
paucity  of  stimulating  graphics  and  sound  at  the  expense  of  a rich  fantasy  scenario  is  debatable.  It  is 
clear,  however,  that  the  potential  for  the  Web  as  a gaming  medium  needs  consideration. 

Simulations 

Simulations  are  distinct  as  an  instructional  strategy  in  that  “the  student  learns  by  actually  performing 
the  activities  to  be  learned  in  a context  similar  to  the  real  world”  (Alessi  & Trollip,  1991,  p.  119)  The 
increasing  capacity  of  computers  to  model  aspects  of  the  real  world,  in  an  inexpensive,  safe  and  self 
contained  scenario  has  lead  to  an  increasing  use  of  this  strategy  within  I MM  learning  environments. 
Examples  such  as  Maxis’  Sim  City  2000  (1993)  show  how  generic  skills  can  be  constructed  in  a rich 
multimedia  environment. 

One  variation  of  simulations  is  the  micro  world.  “A  small,  but  complete,  subset  of  a specific 
environment  or  domain”  (Rieber,  1993,  p.  6),  Micro  worlds  have  the  potential  to  provide  a highly 
interactive  and  motivating  environment,  but  with  greater  teacher  control  of  contributing  variables. 

Small-scale  microworlds  are  now  possible  on  the  Web.  The  example  of  Space  Shuttle  Commander  is 
testament  to  this.  However,  as  the  requirements  for  authenticity  increase,  so  do  the  demands  placed 
upon  data  transfer  rates  as  a result  of  the  extra  media  and  coding  that  need  to  be  integrated.  Such 
requirements  place  a heavy  burden  upon  the  current  implementation  of  the  Web.  Until  such  issues  are 
resolved  it  is  unlikely  that  Simulations  will  be  able  to  replicate  the  kinds  of  experiences  generated 
through  media  rich  CD-ROMs. 

Drill  and  Practice 

With  drill  and  practice  “learners  are  led  through  a series  of  practice  exercises  designed  to  increase 
fluency  in  a new  skill  or  to  refresh  an  existing  one”  (Newby,  Stepich,  Lehman  & Russell,  1 996,  p.  56). 
Such  a strategy  can  form  an  important  component  of  learning  in  the  classroom  where  automaticity 
rather  than  reflection  is  required,  as  in  the  use  of  flash  cards  for  learning  multiplication  tables  or 
foreign  language  vocabulary.  While  computers  are  quite  capable  of  such  interactions,  there  does  appear 
to  be  a move  away  from  Drill  and  Practice  as  a strategy  in  IMM,  perhaps  because  of  its  foundings  in 
behaviourist  principles  and  the  fact  that  it  does  not  make  full  use  of  the  technology. 

While  computers  themselves  are  now  capable  of  far  more  than  simple  drill  and  practice  interactions  the 
same  can  not  be  said  of  the  Web  as  yet.  In  fact,  it  is  in  this  area  of  computer  interaction  that  the  Web 
appears  to  excel.  Alessi  and  Trollip  (1991)  define  the  anatomy  of  drill  and  practice  as  an  interaction 
involving  item  selection,  question/response,  judging  and  feedback,  and  such  elementary  processing  can 
easily  be  handled  by  Common  Gateway  Interface  scripting.  Even  at  a more  complex  level,  Authorware, 
which  is  designed  specifically  to  facilitate  such  interaction  can  be  integrated  into  the  Web  using 
Shockwave. 

Tutorials 

Tutorials  are  less  an  individual  strategy  as  an  instructional  sequence.  Specifically,  the  tutorial  “presents 
the  content,  poses  a question  or  problem,  requests  learner  response,  analyzes  the  response,  supplies 
appropriate  feedback,  and  provides  practice  until  the  learner  demonstrates  a predetermined  level  of 
competency”  (Newby,  Stepich,  Lehman  & Russell,  1996,  p.  57).  This  sequence  is  graphically 
represented  in  figure  2. 
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Fi|gure  2-  Outline  of  a Tutorial  (Alessi  & Trollip,  1991) 

Tutorials  are  a common  strategy  in  educational  multimedia,  and  there  is  no  reason  why  they  can  not 
use  the  medium  of  the  Web.  TTie  ease  of  creating  presentations,  where  large  media  files  are  not  an 
issue,  has  already  been  shown,  as  has  the  power  of  forms  for  question/answer  episodes.  While  there 
appear  to  be  few  good  examples  of  tutorials  on  the  Web,  most  being  characterised  by  excessive  use  of 
text  and  linear,  page  turning  interactions  (UT  Austin,  1997),  they  have  the  potential  to  provide  effective 
learning  where  objectives  are  clearly  defined. 

Recommendations 


Ostensibly,  there  appear  to  be  few  strategies  which  can  not  in  some  way  be  implemented  in  all  of  the 
environments  discussed.  In  categorising  particular  environments  as  supporting  certain  strategies  at  the 
expense  of  others,  due  care  is  necessary.  The  clustering  of  strategies  towards  the  centre  of  the  diagram 
depicting  learning  environments  would  suggest  that  many  approaches  work  equally  well  on  a variety  of 
platforms. 

Nevertheless,  some  conclusions  can  still  be  drawn  as  to  what  strategies  may  be  more  or  less  effective  in 
a Web  based  environment  than  in  a traditional  classroom  or  using  CD-ROM  based  Interactive 
Multimedia.  The  difficulties  of  the  Web  in  providing  a high  level  of  interactivity,  and  environments 
rich  in  multi-sensory  media,  certainly  limit  its  effectiveness  in  implementing  demonstration  and 
simulation  strategies.  Since  these  approaches  are  often  linked  with  Constructivist  tenets  of  active  and 
authentic  learning,  consideration  must  be  given  to  how  these  issues  can  be  addressed. 

In  highly  concrete  knowledge  domains  this  is  a very  real  problem.  A trade  such  as  Plumbing,  for 
example,  requires  skills  that  involve  enactment  on  a physical  environment.  The  difficulty  the  Web  has 
in  creating  such  authentic  simulations  appears  to  limit  its  capacity  to  accommodate  this.  Even  more  ill- 
structured  academic  areas  such  as  History  and  Philosophy  require  a level  of  context,  even  if  that 
context  is  the  academic  one  requiring  the  implementation  of  a culturally  specific  methodology  and  use 
of  language.  It  is  here,  perhaps,  where  the  Web  may  be  particularly  useful.  While  the  Web  does  not 
easily  support  a high  level  of  authenticity  in  terms  of  the  physical  environment  that  can  be  created,  it 
may  perhaps  be  used  to  create  an  authentic  culture  for  learning.  This  would  involve  communication 
between  practitioners  of  a knowledge  domain  within  a social  framework.  Thus  a high  level  of 
interaction  is  maintained  for  the  learner  through  the  sharing  of  ideas  within  an  appropriate  context 

Traditional  Internet  communication  tools  such  as  E-mail,  Newsgroups,  Internet  Relay  Chat,  and  MOOs 
offer  both  the  rapid  synchronous  communication  of  dialogue  in  the  form  of  text  as  well  as 
asynchronous  interaction  which  may  help  to  promote  a more  reflective  metacognitive  approach.  With 
the  use  of  Web  browser  plug-ins  and  server  software  such  as  Ichat(Ichat,  1998),  such  facilities  are  now 
becoming  available  in  a more  cohesive  form  on  the  Web. 

While  care  needs  to  be  taken  in  the  use  of  Web  based  communication  tools,  it  would  appear  that  they 
do  show  some  promise.  Wyld  and  Eklund  (1997),  for  example,  cite  Papert  (1994)  and  Betts  (1994)  in 
arguing  that  Web  communication  has  the  potential  to  support  “a  wide  range  of  learning  styles  as 
information  is  presented  in  a range  of  ways  that  is  flexible  to  students’  needs  and  abilities”  (p.  148). 
This  does  raise  questions  about  the  actual  nature  of  the  Web  as  a communication  medium.  How  does  it 
differ  from  the  traditional  forms  of  letter  writing  and  face  to  face  interaction?  These  questions  in  turn 
raise  issues  of  whether  students  need  to  be  taught  skills  in  communication  with  each  other  using 
Internet  technologies.  Even  if  this  is  so,  however,  the  goal  of  Web  communication  appears  to  be  one 
worth  pursuing.  Secondary  outcomes  alone,  such  as  the  potential  to  develop  “collaborative  global 
relationships  and  cultural  understandings  by  reducing  the  existence  of  stereotypes  and  prejudices  and 
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the  significance  of  location,  gender,  age  and  race”  (Caudell,  1994,  cited  in  Wyld  & Eklund,  1997,  p. 
148),  make  the  implementation  of  Web  based  communication  an  exciting  prospect. 
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Abstract:  This  paper  will  report  on  the  results  of  a DETYA*-funded  project  which 
investigated  the  extent  of  use  of  computer-facilitated  learning  (CFL)  materials  across  the 
higher  education  sector  in  Australia  in  order  to  determine  overall  trends.  In  addition,  the 
project  also  investigated  how  an  updated  national  inventory  of  such  materials  might  be 
developed  in  the  context  of  developing  a strategy  which  could  lead  to  a greater  adoption  of 
CFL  materials  in  Australian  higher  education.  * Department  of  Education,  Training  and 
Youth  Affairs  - a central  Australian  government  department 


Introduction  and  Scope  of  the  Study 

There  has  been  a great  deal  of  development  of  electronic  educational  resources  in  the  last  few  years,  fuelled  by  a 
climate  in  higher  education  which  demands  educating  more  students  with  less  resources.  This  has  occurred  together 
with  substantial  development  of  information  technology  (IT)  systems  and  infrastructure  in  all  Australian 
universities.  However,  the  evidence  is  that  there  is  little  dissemination  of  these  electronic  resources  and  practices. 
Greater  collaboration  and  sharing  of  resources  is  becoming  an  increasingly  urgent  issue.  There  are  several  existing 
databases  of  computer-facilitated  learning  (CFL)  materials  but  these  databases  do  not  appear  to  have  increased  the 
take-up  of  CFL  materials  and  strategies  a great  deal.  We  need  to  investigate  educational,  technological  and 
management  issues  in  designing  ways  in  which  more  use  can  be  obtained  from  the  valuable  resources  that  exist. 

The  term  computer-facilitated  learning  (CFL)  materials  is  used  to  describe  materials  which  use  information 
technology  in  some  way  to  facilitate  teaching  and  learning,  including:  educational  CD-ROMs,  online  course  content 
materials,  and  the  use  of  software  for  computer-mediated  communication  within  a course. 


Methods  Used 

The  study  used  a multi-method  approach,  employing  online  surveys  of  institutional  practice  (28  Australian 
universities  responded);  a literature  survey;  and  a case  study  of  five  universities  at  project,  faculty  and  institutional 
levels.  The  data  included  survey  results,  interview  transcripts,  focus  group  transcripts,  institutional  documentation 
and  short  descriptions  or  vignettes. 

The  study  explored  issues  which  facilitated  or  mitigated  against  teachers  being  able  to  work  in  an  environment 
which  facilitates  the  adoption  of  CFL,  in  terms  of: 

• appropriate  policies,  infrastructure  and  supports  within  the  institution; 

• access  to  information  about  CFL  resources;  and 

• being  able  to  work  collaboratively  both  within  and  across  institutions. 
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Existing  Information  about  CFL  Resources  at  Australian  Universities 

A substantial  amount  of  data  was  accumulated  from  a range  of  sources  about  the  resources  which  exist  to  support 
CFL  at  Australian  universities.  It  is  clear  that  many  universities  are  actively  engaged  in  producing  CFL  resources  to 
enhance  the  educational  offerings  they  have.  There  is  also  a clear  commitment  to  developing  appropriate 
infrastructure  to  support  the  use  of  technology.  The  diversity  in  universities'  structural  arrangements  made  it  difficult 
to  compare  data  across  institutions.  There  is  also  a variation  in  the  stage  of  adoption  of  new  IT  from  contexts  where 
there  are  a significant  number  of  early  adopters  and  the  use  of  technology  is  clearly  embedded,  to  contexts  where 
significant  use  of  technology  is  a newly  emerging  trend.  Precise  information  about  CFL  resources  may  not  have 
been  obtained  from  some  universities  because  the  survey  was  not  responded  to  by  the  person  with  most  accurate 
knowledge,  or  because  facilities  were  devolved  and  centralised  data  was  not  available.  The  rapid  change  in  the  area 
also  means  that  information  provided  to  the  investigators  may  be  soon  out  of  date.  Overall  the  information  about 
existing  CFL  resources  at  Australian  universities  is  patchy  and  incomplete.  In  particular,  there  is  limited  or  no 
information  about: 

• the  educational  design  of  the  CFL  resources  being  produced, 

• the  incentives  and  support  that  exist  for  individuals  to  produce  CFL  resources, 

• the  technical  design  and  access  specifications  for  using  these  CFL  resources, 

• the  experience  of  using  the  CFL  resources  in  actual  teaching  contexts, 

• evaluations  carried  out  to  determine  how  educationally  effective  these  resources  are  in  practice, 

• intellectual  property  and  copyright  issues  which  might  affect  the  use  by  others,  and 

• how  access  can  be  obtained  to  these  CFL  resources  from  either  colleagues  in  the  same  university  or  another 
institution. 


Major  Issues  Relating  to  the  Adoption  of  CFL  Resources  at  Australian  Universities 

The  major  issues  were  selected  from  the  case  studies  using  three  criteria: 

• frequency  of  being  mentioned, 

• intensity  of  expression  in  the  interview,  and 

• who  articulated  the  idea-senior  administration  and/or  teachers. 

Three  major  themes  emerged  with  a considerable  overlap  between  a number  of  the  issues  and  factors  involved  in 
each  theme. 

Policy 

The  policy  theme  looked  at  specific  institutional  policies,  such  as  equity  and  intellectual  property,  the  alignment  of 
policy  throughout  the  organisation,  the  direction  of  policy  change  (bottom-up  or  top-down)  and  a number  of 
strategic  processes  which  flowed  on  from  policies  such  as  grant  schemes.  There  was  a variation  in  the  direction  of 
policy  change  across  the  universities  studied.  The  universities  with  a traditional  emphasis  on  distance  education  had 
a strongly  top-down  policy  direction.  Two  other  universities  had  a predominantly  bottom-up  policy  direction,  and 
one  university  was  reported  by  case  participants  as  having  essentially  no  policy  with  respect  to  CFL.  In  the 
institutions  in  which  adoption  of  CFL  seemed  to  be  progressing  most  effectively,  there  was  a balance  between  top- 
down  and  bottom-up  policy  implementation.  This  reinforces  the  need  for  balance  between  policy,  culture  and 
support;  policy  on  its  own  will  not  succeed  unless  it  aligns  with  institutional  culture. 

Copyright  was  a particularly  complex  issue  which  impacted  in  different  ways  according  to  the  contextual  level 
(personal,  institutional  or  sectoral)  in  which  it  was  being  examined.  Some  individuals,  for  example,  believed 
university  copyright  policy  presented  a personal  barrier  to  the  sharing  and  development  of  CFL;  while  from  an 
institutional  perspective  there  was  concern  and  a growing  awareness  of  the  complexity  of  the  issues  which  limited 
organisational  capacity  to  maximise  specific  institutional  uses  of  CFL.  Finally,  across  the  higher  education  sector  in 
general  there  were  broad  ranging  legislative  issues,  with  respect  to  moral  rights,  licensing,  royalties  and  payments 
for  use  of  copyrighted  materials,  and  continuing  uncertainties  relating  to  technological  capability  and  usage. 
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Key  issues  relating  to  the  policy  theme: 

• Safety  net  policies  need  to  be  continued,  and  possibly  increased,  in  order  to  address  the  specific  needs  of 
the  minority  who  cannot  gain  access  to  computing  equipment  for  equity  reasons. 

• Funding  schemes  need  to  take  account  of  earlier  experiences  - mechanisms  are  needed  for  monitoring  the 
progress  of  initiatives  and  learning  from  failure, 

• Funding  schemes  need  to  explicitly  take  account  of  the  need  for  ongoing  maintenance  of  CFL 
developments. 

• Appropriate  criteria  for  teaching  performance  need  to  be  developed  in  order  to  provide  an  effective 
incentive  for  staff  to  adopt  CFL  materials  and  practices. 

• University  staff  need  access  to  a centralised  service  which  provides  practical  support  to  staff  on  intellectual 
property  and  licensing  issues. 

• There  needs  to  be  widespread  dissemination  of  the  legal  situation  with  respect  to  online  intellectual 
property. 

• If  a CFL  resource  is  made  available  to  the  academic  community,  mechanisms  are  needed  to  facilitate 
copyright  clearance  of  CFL  materials  so  that  due  recognition  is  given  to  the  originator. 

• Legislation  relating  to  the  use  of  online  resources  in  education,  the  ownership  of  copyright  on  materials 
used  in  offshore  courses,  and  moral  rights  is  in  train.  Appropriate  legislation  should  be  supported. 


Culture 

Culture  incorporated  factors  such  as  collaboration  within  institutions,  and  personal  motivation  of  staff  to  use  CFL,  as 
well  as  particular  aspects  of  funding,  staff  rewards  and  time,  leadership,  teaching  and  learning  models,  and  attitudes 
such  as  ‘not  invented  here’.  The  case  studies  showed  a great  difference  in  the  organisational  cultures  of  the 
universities  studied  and  of  the  infrastructures  set  up  to  nurture,  develop,  deliver  and  support  CFL.  The  climate  of  the 
organisation  is  made  up  factors  such  as: 

• strategic  vision  and  leadership, 

• attitudes  to  risk  taking  and  innovation  in  teaching  and  learning, 

• attitudes  to  adopting  CFL, 

• allocation  of  resources,  and 

• staff  recognition  and  rewards. 

Factors  that  motivated  staff  to  adopt  CFL  did  not  vary  greatly  over  the  universities  studied.  Institutions  with  large 
numbers  of  early  adopters  had  made  changes  to  their  cultures  and  policies,  and  had  well-supported  infrastructure. 
This  resulted  in  a high  level  of  acceptance  so  that  some  of  the  issues  were  no  longer  relevant  to  them.  Main  barriers 
to  take-up  were  seen  as: 

• lack  of  knowledge  about  CFL; 

• lack  of  academic  time  release; 

• pressure  to  keep  up  the  research  quantum; 

• non-recognition  of  teaching; 

• lack  of  funding  to  maintain  programs,  staff  and  technical  infrastructure;  and 

• lack  of  student  acceptance  of  the  new  approaches. 

On  the  other  hand,  factors  that  would  motivate  staff  to  use  CFL  included: 
a university  culture  which  supports  new  approaches, 

• good  leadership  from  academic  managers, 

• recognition  of  teaching  as  of  equal  value  as  research, 

• appropriate  support  infrastructures  for  staff  and  students, 

• workload  adjustments  to  develop  materials  and  become  computer  literate, 

• the  opportunity  to  think  about  learning  and  not  teaching, 

• evaluation  studies  showing  improvements  in  student  learning, 

• improving  learning  opportunities  and  outcomes  for  distance  students, 

• adding  value  to  existing  courses, 

• providing  a means  to  offer  courses  offshore, 

• the  chance  to  interact  more  with  students. 
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• positive  feedback  from  students, 

• solutions  to  problems  of  large  classes  and  funding  cuts, 

• support  and  sponsorship  from  mentors,  and 

• peer  pressure. 

Changing  educational  practices  and  styles  can  produce  many  negative  reactions  and  this  negativity  needs  to  be 
acknowledged  and  managed  effectively.  Change  should  be  introduced  and  implemented  within  a supportive 
environment.  The  culture  of  the  organisation  needs  to  be  able  to  embrace  change  while  offering  staff  opportunities 
to  develop  their  own  levels  of  comfort  with  the  change.  Within  the  university  environment,  leaders  need  to  develop 
vision  statements  that  are  clear  and  well  articulated  to  the  staff.  Appropriate  levels  of  infrastructure  and  support 
should  be  part  of  the  policy  formulation  to  match  the  vision. 

Key  issues  relating  to  the  culture  theme: 

• Universities  need  to  have  a clearly  articulated  vision  of  the  changes  to  teaching  and  learning  that 
technology  brings. 

• This  vision  should  have  ownership  and  commitment  from  all  levels  of  management. 

• The  Dean  or  Head  of  Department/  School  should  lead  and  support  moves  by  the  academic  unit  into 
increased  use  of  CFL  materials  and  strategies. 

• Policies  developed  from  the  vision  should  include  positive  values  and  well  funded  infrastructure  to  support 
staff  and  students. 

• Issues  of  staff  workloads  in  the  changeover  to  use  of  CFL  should  be  clarified. 

• The  tension  between  teaching  and  research  needs  to  be  resolved  so  that  staff  gain  appropriate  recognition 
for  the  development  and  use  of  CFL  materials. 


Support 

Support  incorporated  a whole  gamut  of  institutional  issues  including  IT,  library  and  administrative  infrastructure, 
professional  development  for  staff,  student  support,  educational  and  instructional  design  support  for  academic  staff, 
funding  and  grant  schemes,  and  IT  literacy.  There  was  a belief  that  real  efficiencies  could  be  possible  with 
supportive  infrastructure  and  good  planning.  Many  universities  are  now  focussing  on  an  overall  systems  approach 
across  the  university  so  that  infrastructure  planning  is  linked  to  the  development  of  professional  development  and 
support  services.  At  this  time,  however,  all  universities  are  in  transition  and  are  grappling  with  the  issues  involved  in 
designing  and  managing  new  ways  of  working. 

Key  issues  relating  to  the  support  theme: 

• Internal  structures  and  funding  for  CFL  development  units  create  tensions  between  central  administration 
and  the  faculties;  open  exploration  of  this  tension  is  needed. 

• Ongoing  budget  support  for  CFL  development  is  needed.  Funding  for  CFL  development  is  ephemeral  in 
many  universities. 

• Institutional  development  grants  should  focus  on  sharing  processes  and  experience,  as  well  as  looking  for 
product-oriented  outcomes. 

• The  costs  involved  in  hardware  and  facilities,  and  appropriate  technical  and  support  staff,  are  high. 
Mechanisms  are  needed  to  assist  universities  in  managing  these  costs. 

• The  diversity  of  roles  of  ITS  staff  are  high,  and  keeping  skilled  staff  is  difficult.  Schemes  to  recognise  and 
reward  technical  staff  are  needed  as  much  as  schemes  to  recognise  and  reward  academic  staff. 

• Professional  development  and  training  is  a complex  and  multi-faceted  area.  There  is  a need  for  high  quality 
staff  developers,  for  flexible  support  programs,  for  using  mentors,  and  allowing  adequate  time  for  staff  to 
engage  in  staff  development. 

• Student  support  services  cover  a wide  range  of  issues,  including  equity  and  access,  access  to  academic  help 
(this  has  implications  for  funding  staff  time),  access  to  technical  help,  and  access  to  computers.  Further 
attention  is  needed  to  all  these  issues. 


Adoption  and  Collaboration 
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Unless  the  climate  within  and  between  universities  supports  the  development  and  use  of  CFL  resources,  there  will 
not  be  an  increase  in  appropriate  uses  of  technology  in  teaching  and  learning.  Staff  will  use  technology  in  their 
teaching  when  culture,  policy  and  support  structures  are  congruent.  Some  particular  issues  that  were  highlighted 
were: 

• Time  and  scalability:  from  an  institutional  point  of  view,  the  current  situation  is  that  infrastructure  and  time 
allowance  is  possible  only  for  a relatively  small  number  of  staff  Mainstreaming  the  use  of  CFL  has 
enormous  resource  implications. 

• Good  evaluation  of  existing  CFL  programs  is  a crucial  aspect  of  adoption.  Late  adopters  want  evidence  that 
CFL  materials  and  strategies  can  enhance  student  learning 

• Adoption  depends  on  key  individuals  who  can  act  as  role  models  and  mentors  for  others.  Institutions  need 
to  recognise  and  support  these  individuals. 

The  tension  between  collaboration  and  competition  is  important.  We  argue  that  collaboration  can  assist  healthy 
competition  in  higher  education.  Using  resources  efficiently  in  collaborative  arrangements  can  allow  institutions  to 
develop  their  own  specialities  more  effectively.  During  the  next  few  years  each  university  will  need  to  find  its  own 
balance  point  between  external  collaborative  work  and  internal  work  aimed  at  developing  its  own  specialisations. 


Dissemination  and  Databases 

Dissemination  of  information  about  CFL  resources,  and  the  location  of  CFL  resources  via  databases  were 
fundamental  parts  of  the  study.  A framework  for  the  development  of  a national  inventory  system  is  proposed 
whereby  the  development  of  national  metadata  standards  will  enable  teachers  and  lecturers  to  search  across  a 
distributed  set  of  interest-group-based  databases. 

Dissemination  of  information  about  CFL  resources  through  marketing  programs  was  seen  to  be  problematic  by  case 
study  participants,  for  a number  of  reasons,  some  of  which  are: 

• Potential  purchasers/  users  had  varying  levels  of  IT  skills. 

• Everyone  is  keen  to  be  a seller  in  the  marketplace,  but  few  want  to  be  buyers. 

• Some  programs  proved  to  be  too  difficult  to  transfer  to  another  context-there  were  technical,  content  and 
remuneration  barriers. 

• Commercialisation  often  becomes  a dead  issue;  not  many  people  are  interested  in  pursuing  this. 

• Often  there  is  insufficient  kudos  involved  in  disseminating  information  about  CFL  developments. 

• Intellectual  property  is  an  issue,  in  that  some  people  might  not  want  their  material  to  be  used  by  others. 
There  are  concerns  about  compromising  the  competitive  advantage  of  universities. 

Nevertheless,  it  was  felt  that  databases  of  CFL  resources  could  be  useful  if  designed  well.  A number  of  features 
were  identified  which  could  characterise  a well-designed  teaching  and  learning  database.  These  are: 

• The  data  submission  and  retrieval  process  is  simplified. 

• It  has  a distributed  nature. 

• It  is  maintained  in  an  ongoing  sense. 

• It  is  owned  by  academics. 

• Resources  are  submitted  by  people  with  expertise  in  CFL  and  knowledge  of  the  discipline,  following  a 
scholarly  review  process. 

• Resource  submission  should  not  be  by  the  developer  of  the  resource. 

• It  should  contain  contextual  information  about  the  resource,  including  a full  description  of  the  product;  the 
rationale  behind  its  development;  its  unique  characteristics;  the  pedagogical  approach  used;  intellectual 
property  details,  and  how  it  might  be  obtained;  and  evaluation  data. 

• It  should  contain  a range  of  experiential  information  on  how  the  resource  was  used  in  a real-life  teaching 
context,  both  by  the  developer  and  others. 

• It  should  return  the  appropriate  level  of  information  to  the  queries  submitted  by  users. 

• Resources  successfully  submitted  to  the  database  should  attract  scholarly  recognition. 
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The  contextual  and  experiential  information  required  of  an  ideal  database  is  encapsulated  in  the  idea  of  metadata. 
There  is  a need  for  cooperation  amongst  database  owners  and  proponents  with  respect  to  metadata  standards.  The 
central  idea  of  the  framework  is  that  databases  can  be  created  which  fill  a particular  market  niche,  but  that  a global 
‘megasearch’  facility  enables  academics  searching  for  information  to  seamlessly  traverse  a range  of  databases.  The 
framework  we  are  proposing  for  the  development  of  a unified,  Australia- wide  collaborative  framework  for 
interoperable  online  databases  strongly  urges  that  international  metadata  standards  be  developed. 

A prototype  distributed  Links  database  has  been  developed  by  ASCILITE.  This  forms  a searchable  resource  that  is 
interoperable  with  other  databases  and  is  sustainable  and  credible.  It  allows  for  distributed  maintenance  of  hypertext 
links  within  the  system.  Loosely  based  upon  a Yahoo-style  system,  the  Links  database  allows  reviewers  to  moderate 
categories  in  which  they  have  expertise.  Any  person  can  submit  useful  links  to  a particular  category.  The  moderator 
provides  a third-party,  quality  assurance  role  that  adds  value  to  the  to  resources  maintained  within  the  database. 

The  recommendations  on  dissemination  and  databases  are  that  DETYA  support  and  funding  are  needed  for  the 
development  of  a high  level  technical  database  framework  which: 

• is  implementable; 

• encourages  open  discussion  with  the  various  teaching  and  learning  resource  database  owners  to  strive  to 
achieve  interoperability  between  their  sites; 

• develops  standards  to  ensure  inter-exchange  of  metadata  between  different  domain-specific  teaching  and 
learning  resource  databases,  by  working  with  the  various  domain  communities; 

• works  with  other  DETYA-funded  bodies  to  establish  standards  and  encourage  adoption  of  such  standards 
for  resolving  intellectual  property  issues,  with  the  aim  of  creating  of  a market  for  learning  artifacts;  and 

• ensures  that  all  standards  are  interoperable  with  emerging  international  standards. 

The  framework  we  are  proposing  for  the  development  of  a unified,  Australia-wide  collaborative  framework  involves 
interoperable  online  databases.  While  our  considerations  have  been  mainly  directed  towards  databases  of  CFL 
materials,  we  believe  that  this  framework  has  a wider  applicability.  To  be  successful,  the  development  of  the 
framework  needs  to  be  a national  initiative,  funded  appropriately,  and  coordinated  through  a suitable  coordinating 
body.  The  coordinating  body  will  need  to  develop  standards  for  metadata  by  leveraging  off  the  Dublin  Core  and 
IMS  standards,  and  working  closely  with  other  Australian  interest  groups,  such  as  the  Australian  Library  and 
Information  Association  (ALIA),  the  Committee  of  Australian  University  Librarians  (CAUL)  and  the  Distributed 
Systems  Technology  Centre  (DSTC). 

Professional  organisations,  such  as  ASCILITE,  can  also  participate  in  this  national  framework  of  collaboration  in 
areas  such  as: 

• educating  the  academic  community  about  the  need  to  provide  metadata, 

• providing  pedagogical  and  educational  design  advice  in  the  formulation  of  standards, 

• providing  a testbed  for  the  implementation  of  the  standards,  and 

• evaluating  the  effectiveness  of  the  standards  and  the  impact  of  the  standards  on  the  quality  of  education. 

The  size  of  the  Australian  economy  does  not  allow  us  to  re-invent  the  wheel  all  the  time.  The  Australian  education 
sector  needs  a collaborative  framework  to  capture  the  synergy  that  may  be  generated  by  the  aggregation  effect. 
Information  technology  changes  the  size  of  a market,  as  well  as  the  way  the  market  is  shared.  To  remain  competitive 
in  the  global  market,  Australian  higher  education  cannot  afford  to  wait  for  the  development  of  standards  which  will 
influence  how  education  will  be  delivered  and  how  effective  the  digital  education  system  will  be. 
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Abstract:  This  paper  presents  preliminary  findings  from  a two-semester  study  on  the  effects  of 
online  learning  on  student  perceptions  of  self-change  as  well  as  the  impact  of  online  learning 
environments  on  student  attitudes  concerning  their  learning  experiences.  There  have  been  several 
studies  that  examined  distance  learning  courses  versus  traditional  fonnats  and  found  no  significant 
differences  in  grade  point  averages  of  students  in  either  format  (The  Institute  for  Higher  Education 
Policy,  1999).  Overall  ratings  of  courses  by  students  who  complete  online  courses  are  equal  or 
higher  than  those  for  traditional  courses  (Hiltz,  1997).  There  have  been  few  studies,  however,  that 
examined  other  components  of  the  learning  process  such  as  changes  in  perceptions  and  attitudes. 


Teaching  and  learning  are  not  the  same  as  they  were  even  five  years  ago;  the  amount  and  nature  of  information 
continues  to  grow  far  too  rapidly  for  the  old  paradigms  and  systems  to  function  effectively  (Hannafin,  Hill,  & Land, 
1997).  The  World  Wide  Web  offers  a myriad  of  ways  to  deliver  instruction  and  provide  resources  and  support  for 
learning,  but  the  array  of  possibilities  is  confusing  for  both  teachers  and  students.  K-12  schools,  institutions  of 
higher  education,  and  corporations  are  all  feeling  the  pressure  to  provide  online  instruction.  In  many  instances  the 
change  from  a traditional  classroom  to  an  Internet-based  experience  has  been  instituted  with  little  or  no 
consideration  of  the  effect  on  student  learning. 

Concerns  about  the  impact  of  distance  learning  courses  on  educational  systems  have  been  addressed  in  much  of  the 
literature  to  date.  These  concerns  include  lack  of  technological  expertise  on  the  part  of  both  faculty  and  students, 
resistance  to  change  on  the  part  of  some  faculty,  the  cost  of  developing  and  implementing  programs,  hardware 
limitations,  poor  instructional  design,  and  learner  isolation  (Wegner,  Holloway,  & Garton,  1999;  Boettcher,  2000). 
While  they  represent  legitimate  areas  of  concern,  for  the  most  part,  these  problems  relate  to  training  and  technology 
issues  that  have  fairly  obvious  solutions.  It  is  less  obvious,  however,  what  the  effect  of  online  learning  has  on 
students'  perceptions  of  self-change  in  teaching  and  learning. 

This  paper  presents  preliminary  findings  from  a two-semester  study  on  the  effects  of  online  learning  on  student 
perceptions  of  self-change  as  well  as  the  impact  of  online  learning  environments  on  student  attitudes  concerning 
their  learning  experiences.  There  have  been  several  studies  that  examined  distance  learning  courses  versus 
traditional  formats  and  found  no  significant  differences  in  grade  point  averages  of  students  in  either  format  (The 
Institute  for  Higher  Education  Policy,  1999).  Overall  ratings  of  courses  by  students  who  complete  online  courses  are 
equal  or  higher  than  those  for  traditional  courses  (Hiltz,  1 997).  There  have  been  few  studies,  however,  that  examined 
other  components  of  the  learning  process  such  as  changes  in  perceptions  and  attitudes. 
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The  Problem 


As  faculty  members,  we  began  to  see  changes  in  the  students  who  finished  an  instructional  design  course  online.  At 
first,  our  feeling  was  that  the  changes  were  limited  to  one  particular  class,  but  each  semester  that  the  course  was 
taught,  students  began  to  exhibit  changes  in  their  perceptions  of  themselves  as  learners  and  users  of  technology. 
These  changes  were  evidenced  in  their  online  discussion  postings,  in  their  attitudes  and  interactions  towards  their 
peers,  and  in  the  collaborative  quality  of  their  assignments.  In  some  students,  the  change  was  subtle,  but  in  others,  a 
more  radical  transformation  took  place.  As  students  became  more  involved  in  the  discussion  and  using  the  material 
and  resources,  they  seemed  to  be  more  in  control  of  their  learning  and  more  involved  in  the  online  experience. 

This  course  is  offered  to  a wide  range  of  students,  not  only  students  in  the  instructional  technology  program  area,  but 
graduate  students  working  on  teacher  certification  as  well.  The  background  of  these  students  is  very  diverse  and  the 
common  thread  that  unites  the  content  of  the  course  is  the  instructional  design  process.  The  career  goals  of  these 
students  are  varied  as  well,  and  range  from  K-12  classroom  teachers  to  school  district  technology  specialists  to 
business  and  industry  trainers.  The  course  is  divided  into  three  modules  with  fifteen  assignments;  the  last  eight  are 
focused  on  creating  a final  project,  a process  that  guides  the  student  through  the  steps  of  a generic  instructional 
design  model.  Most  students  who  enroll  in  this  course  have  little  experience  with  designing  and  delivering 
instruction. 

As  several  semesters  went  by,  and  this  change  happened  with  regularity,  we  began  to  look  for  supporting 
information  that  would  help  clarify  what  was  happening.  We  began  discussing  the  change  with  the  students  after 
they  finished  the  course.  Several  of  them  made  comments  about  the  changes  that  occurred: 

This  course  has  been  time  consuming,  and  challenging.. .but  what  an  eye-opening  experience,  as  well. 

When  1 realize  what  we  are  capable  of  doing,  and  all  the  resources  that  are  available,  1 feel  overwhelmed. 

This  class  gave  me  the  incentive  to  find  some  of  those  resources,  find  better  ways  to  teach  content,  and  a 
more  positive  impression  of  tutorials.  (Student  One) 

1 think  the  class  has  been  beneficial.  I now  have  a clear  picture  of  what  an  instructional  designer  does,  and  1 
like  the  process.  The  support  that  we  have  received  in  this  class  is  phenomenal  compared  to  the  classes  that 
1 have  taken  in  the  past.  I never  had  a question  that  wasn't  answered,  and  1 have  never  felt  as  supported  by 
my  instructor  or  my  peers.  (Student  Two) 

This  change  is  exemplified  by  this  statement  from  an  online  learner  at  another  university: 

Within  a matter  of  days  of  logging  on  for  the  first  time,  1 began  to  see  the  worth  of  the  experience  that 
exceeded  earning  a grade  and  credit.  No  one  was  more  surprised  than  I was  by  the  way  in  which  the 
interactions  I became  involved  in  shaped  my  perception  of  technology  and  its  applications  in  my 
professional  life.  But  more  importantly,  those  online  interactions  exponentially  enhanced  my  own  learning. 

1 believe  my  experiences  were  transformational  because  the  online  conference  exemplified  the 
characteristics  of  a model  learning  environment.  (Allison,  1 998) 


Methodology 

This  study  investigated  the  perceptions  students  had  about  self-change  during  a distance  education  course  taught 
using  the  World  Wide  Web.  The  objective  of  the  research  was  to  investigate  several  questions  related  to  change 
issues  as  evidenced  through  pre-course  and  post-course  surveys. 

The  research  questions  were: 

1 . Do  students’  feelings  about  technology  change  during  an  online  course? 

2.  Do  students  feel  that  their  teaching  and  learning  abilities  and  skills  change  as  a result  of  taking  and  completing 
an  online  course? 
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This  research  study  focused  on  a master's  level  course  in  instructional  design.  Components  of  the  course  included: 

• A Classroom  of  course  information,  instructional  materials,  readings,  and  examples 

• A Library/Resource  Center  with  a glossary,  list  of  Web  resources  and  related  articles 

• A Communications  Center  with  Web-based  discussion  groups.  These  discussion  groups  included  three 
different  areas  of  interest  (general,  K-12,  adult  learning)  as  well  as  a technical  help  group  and  an  area  to 
share  homework.  There  was  also  an  area  designed  to  broadcast  course  news  and  updates  to  all  students. 


Participants 

Data  were  collected  from  two  different  groups  of  students  in  spring,  1999  (number=26)  and  summer,  1999 
(number=23).  The  49  students  involved  in  this  study  were  enrolled  in  two  sections  of  an  online  graduate  level  course 
called  "Instructional  Design."  Both  classes  had  the  same  instructor,  used  the  same  materials,  completed  the  same 
assignments,  and  used  the  same  learning  strategies.  Most  students  stated  that  they  chose  an  online  course  over  a 
traditional  lecture  course  because  their  schedule  did  not  allow  them  to  take  an  on-campus  section  (N=16).  Several 
students  enrolled  in  the  course  because  they  wanted  to  experience  an  online  environment.  There  were  2 students 
(4%)  in  the  18-23  age  category,  14  students  (29%)  in  the  24-29  group,  9 students  (19%)  in  the  30-35  group,  10 
students  (20%)  in  the  36-41  group,  9 students  (18%)  in  the  42-47  group,  and  5 students  (10%)  in  the  48-53  group. 
There  were  10  male  students  and  39  female  students.  There  were  12  full-time  students  and  37  part-time  students. 
This  demographic  data  is  representative  of  the  student  population  in  the  College  of  Education.  The  majority  of  the 
graduate  students  are  part-time  students,  are  employed  full-time,  and  are  working  toward  certification  or  a graduate 
degree. 


Data  Sources 

For  this  initial  study,  data  sources  included  a pre-course  survey  and  a post-course  survey.  Other  sources  that  will 
provide  additional  data  for  the  next,  more  in-depth  study  include  reflective  journals,  time  logs,  interviews  and 
analysis  of  course  communications  and  email. 

Quantitative  analysis  included  data  from  a pre-course  survey  that  gathered  information  about  students'  expectations 
of  the  course  as  well  as  perceptions  about  online  learning  and  level  of  technological  expertise.  A post-course  survey 
measured  student  satisfaction  and  reassessed  both  technological  expertise  and  time  spent  on  various  course-related 
activities  such  as  reading  the  Web-based  materials,  completing  homework  assignments,  and  participating  in  the 
online  discussions.  Both  surveys  were  anonymous,  but  they  were  coded  so  they  could  be  matched  for  pre-course  and 
post-course  analysis  techniques.  Students  responded  to  various  questions  by  checking  answers.  There  was  also  space 
for  additional  written  comments. 

It  should  be  noted  that  validity  issues  might  affect  self-reported  data  from  these  surveys.  Surveys  could  be  biased 
positively  or  negatively  by  how  a student  feels  about  the  instructor  or  the  course,  by  feelings  that  answers  may  affect 
the  grade  for  the  course,  or  by  other  influences  such  as  the  novelty  of  the  online  learning  environment.  In  order  to 
deal  with  these  issues,  students  completed  both  surveys  without  the  instructor  being  present,  and  surveys  were 
matched  using  information  that  gave  no  indication  of  the  student’s  identification.  Post-course  surveys  were 
completed  at  the  end  of  the  semester,  after  the  presentation  of  final  semester  projects. 

There  were  two  research  questions  that  guided  this  preliminary  study: 

Research  Question  1 : Do  students'  feelings  about  technology  change  during  an  online  course? 

Students  were  asked  to  rate  their  feeling  about  using  technology  on  a scale  from  1 : uncomfortable  using  it,  2:  know 
basics  of  how  to  use  it,  3:  relatively  skilled,  but  not  an  expert,  to  4:  expert  user. 

Research  Question  2:  Do  students  feel  that  their  teaching  and  learning  abilities  and  skills  change  as  a result  of  taking 
and  completing  an  online  course? 

Students  were  asked  to  rate  their  feeling  about  changes  that  occurred  in  their  teaching  and  learning  abilities  as  a 
result  of  the  course.  Students  used  the  following  likert  scale  to  indicate  their  perceptions:  1 : strongly  agree,  2:  agree. 
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3:  no  opinion,  4:  disagree,  5:  strongly  disagree.  Students  could  also  indicate  that  they  were  already  proficient  in  a 
particular  area.  These  data  were  not  used  in  the  statistical  interpretation,  but  are  reported  in  the  accompanying  table. 

Analysis 

Research  Question  1 : Do  students'  feelings  about  technology  change  during  an  online  course? 

Table  1:  Results  Obtained  from  the  Wilcoxin  Signed-Rank  Test. 


r=  2.594  p=.009 

N 

Mean 

Median 

SD 

Beginning  of  the  Course 

49 

2.77 

3.00 

0.65 

End  of  the  Course 

49 

2.98 

3.00 

0.58 

Table  2:  Frequencies  for  Feelings  about  Technology 


Beginning  of  the  Course 


End  of  the  Course 


1 : uncomfortable 
using  it 


2:  know  basics  of 
how  to  use  it 


2.5 

* reported  by 
students,  but  not 
given  as  a 
category 


3:  relatively  skilled, 
but  not  an  expert 


1 14 


29 


8 2 31 


4:  expert  user  N 


5 


8 


4 

9 

4 

9 


A Wilcoxin  Matched-Pairs  Signed-Rank  Test  was  used  to  determine  statistical  significance.  This  test  is  used  for 
correlated  samples  in  much  the  same  way  as  the  dependent  sample  test. 

As  shown  in  Table  1 above,  the  results  from  the  Wilcoxin  Signed  Rank  Order  Test  yielded  a T = 2.594  and  a 
statistically  significant  p value  of  .009 

The  students'  feelings  about  using  technology  after  completion  of  the  course  are  significantly  higher  than  their 
feelings  about  technology  before  taking  the  course.  In  light  of  these  findings,  the  following  implications  are 
suggested:  there  is  evidence  that  students'  feelings  about  using  technology  improve  while  enrolled  in  this  class. 
Whether  this  effect  is  related  to  the  phenomena  of  "novelty  effect"  (The  Institute  for  Higher  Education  Policy,  1999) 
is  not  yet  determined.  The  novelty  effect  is  a reactive  effect,  and  refers  to  increased  interest,  motivation,  or 
participation  on  the  part  of  students  because  they  are  doing  something  different,  not  better,  per  se. 

Several  factors  could  be  responsible  for  contributing  to  this  perception.  Students  are  required  to  use  a computer  to 
access  course  materials,  to  submit  assignments  and  to  communicate  with  their  peers  and  the  instructor.  As  their  use 
increases,  it  is  likely  that  students’  perceptions  of  their  abilities  and  level  of  comfort  also  increases. 

Research  Question  2:  Do  students  feel  that  their  teaching  and  learning  abilities  and  skills  change  as  a result  of  taking 
and  completing  an  online  course? 

Table  3:  Results  of  Survey  Questions  about  Teaching  and  Learning  Abilities 


This  course  has  helped  me: 

N 

Mean 

Median 

SD 

Strongly 

Agree 

Agree 

Disagree 

Already  Proliclenl 
or  No  Opinioiii 

become  a more  proficient  computer 
user 

36 

1.41 

1.00 

0.50 

21 

(58%) 

15  (42%) 

13 

become  a more  proficient  email  user 

25 

1.44 

1.00 

0.50 

14 

(56%) 

11 

(44%) 

24 

715 


become  a more  proficient  internet 
user 

34 

1.29 

1.00 

0.46 

24 

(71%) 

10 

(29%) 

15 

become  more  proficient  finding 
information  online 

38 

1.39 

1.00 

0.63 

25 

(66%) 

12 

(31%) 

1 

(3%) 

11 

become  a better  evaluator  of 
information 

47 

1.51 

2.00 

0.50 

23 

(49%) 

24 

51%) 

2 

become  a better  teacher  or  trainer 

46 

1.30 

1.00 

0.46 

32 

(70%) 

14 

(30%) 

3 

become  a better  writer 

38 

1.92 

2.00 

0.85 

11 

(29%) 

23 

(60%) 

4 

(11%) 

11 

communicate  my  ideas  more 
effectively 

41 

1.63 

2.00 

0.62 

17 

(42%) 

23 

(56%) 

1 

(2%) 

8 

collaborate  with  other  students  more 
effectively 

37 

1.59 

2.00 

0.49 

15 

(41%) 

22 

59%) 

12 

1 Data  in  the  category  “Already  Proficient  or  No  Opinion”  were  not  used  in  the  statistical  interpretation,  but  are 
reported. 

As  shown  in  Table  3 above,  students  overwhelmingly  felt  that  the  course  helped  them  become  more  proficient  and 
competent  in  a number  of  areas  of  teaching  and  learning.  It  is  not  yet  clear  whether  this  is  also  a reactive  effect  or  a 
real  one. 

Student  comments  support  the  change  in  perception,  but  investigating  this  effect  in  more  qualitative  ways  will 
provide  a more  complete  assessment  of  self-change. 

Online  learning  made  me  feel  that  with  a very  good  instructional  design,  people  can  learn  a lot  of  things. 

1 felt  more  of  a part  of  this  class  than  most. 

1 prefer  this  method  of  learning.  While  the  work  load  is  the  same  or  greater  than  regular  classes,  the  ’’self- 
paced"  nature  of  the  instruction  is  very  important  to  me. 

It  was  wonderful  to  learn  from  others  as  they  learn. 

Interesting  to  see  others’  knowledge  and  expertise.  The  variety  of  interests  in  the  group  was  amazing. 

Learned  more  about  the  other  students  and  homework  and  how  they  felt  about  different  topics. 

Other  data  collected  in  the  post-course  survey  indicated  that  students  tend  to  work  harder  in  this  online  course.  This 
may  be  caused,  in  part,  by  the  anytime,  anyplace  access  to  course  materials,  the  ability  of  all  students  to  read  others’ 
postings  and  homework,  and  the  motivation  to  keep  up  with  the  class  because  of  increased  interactions  and 
connections.  These  data  are  also  supported  by  research  in  other  asynchronous  learning  environments  (Hiltz,  1997). 

Table  4:  Enrollment  in  Another  Online  Course 


Would  You  Take  Another  Online  Course? 


Yes 

43 

(88%) 


No 

I 

(2%) 


Maybe 


5 

(10%) 


N 

49 


One  overall  measure  of  student  satisfaction  is  whether  or  not  students  would  take  another  online  course;  results  are 
shown  in  Table  4.  The  majority  of  students  (88%)  indicated  that  they  would  take  another  online  course;  10%  of  the 
students  indicated  that  they  might  take  another  online  course. 


er|c 
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Table  5:  Student  Perceptions  about  Learning 


Strongly  Agree  Agree  Disagree  N 

I felt  that  1 learned  as  much  in  this  online  33  14  1 

class  as  1 would  in  a face-to-face  classroom (69%) (29%) (2%) 

Table  5 shows  that  the  majority  of  students  (98%)  felt  that  they  learned  as  much  in  the  online  class  as  in  a face-to- 
face  class. 

Table  6:  Perceptions  about  Workload 


What  was  the  workload  for  this  online 
course  in  comparison  to  an  on-campus 
course  worth  the  same  number  of  units? 


Less  Work  About  the  Same  More  Work  N 


0 


9 

(19%) 


39 

(81%) 


48 


The  majority  of  students  (81%)  felt  that  the  workload  for  an  online  course  is  more  than  a face-to-face  class.  Only 
1 9%  felt  that  the  workload  was  the  same;  no  students  felt  that  the  workload  was  less. 


Next  Steps 

The  results  of  this  preliminary  study  support  the  hypothesis  that  students  feel  that  their  teaching  and  learning 
abilities  increased.  The  general  feelings  about  an  online  course  were  positive;  students  indicated  that  they  felt  they 
learned  as  much  as  a face-to-face  course  and  that  they  would  take  another  course  even  though  they  felt  that  the 
workload  was  higher  in  an  online  environment. 

Other  sources  that  will  provide  additional  data  for  the  next,  more  in-depth  studies  include  reflective  journals,  time 
logs,  interviews  and  analysis  of  course  communications  and  email.  The  role  of  online  learning  environments  as  a 
catalyst  for  change  should  be  studied  and  perceived  self-efficacy  of  the  students  should  be  measured  more 

accurately. 
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Abstract:  Electronically  generated  log-files  contain  a lot  of  pertinent  information,  but  the 
information  is  often  hard  to  access.  This  paper  describes  two  tools  that  have  been  developed  to 
assist  researchers  in  extracting  information  from  Team  Wave  Workplace  (TW)  generated  server 
log-files  and  repositories.  SLA  reads  in  one  log-file,  sorts  out  a researcher  specified  group  of 
persons,  and  then  uses  the  information  to  calculate  some  results  from  the  log.  TRA,  on  the  other 
hand,  is  a tool  to  help  researchers  in  classifying  the  use  of  various  tools  inside  TW,  organising 
their  classifications,  and  registering  how  the  use  of  the  tools  evolves  over  time.  Output  from  both 
tools  is  used  to  complement  various  interpretation  methods  being  used  to  identify  collaboration 
patterns  in  a collaborative  teleleaming  scenario. 


Introduction 

One  challenge  for  research  within  collaborative  teleleaming  is  what  and  how  to  collect  and  analyse  data  in  a 
distributed  environment.  Depending  on  the  research  question  and  study  design,  various  types  of  data  are  desired. 
Traditional  qualitative  methods  such  as  structured  and  unstructured  interviews,  participant  observations,  and  video 
recording  need  to  be  tailored  to  distributed  participants.  Once  the  obstacles  in  collecting  this  type  of  data  are 
overcome,  then  questions  about  how  to  interpret  and  analyse  the  data  persist.  On  the  other  hand,  the  collection  of 
electronically  logged  data  is  relatively  straightforward.  As  Garton  et  al.  (1999)  point  out,  the  restrictions  are  often 
“the  amount  of  server  storage  space  and  the  integrity  of  researchers  and  programmers  in  their  study  design  (p.  92)” 
and  electronic  data  gathering  “replaces  issues  of  accuracy  and  reliability  with  issues  of  data  management, 
interpretation,  and  privacy  (p.  92)”.  Generally,  the  amount  of  electronically  logged  data  can  easily  be 
overwhelming  and  again,  questions  of  interpretation  and  analysis  pose  real  problems. 

In  project  DoCTA  (http://www.ifi.uib.no/docta)  we  are  interested  in  identifying  collaboration  patterns  in 
various  collaborative  teleleaming  scenarios  (Wasson  & Morch,  1999;  Guribye  & Wasson,  1999).  A number  of 
evaluation  perspectives  are  taken  and  numerous  data  collection  methods  are  used  (for  a complete  report  see 
(Wasson,  Guribye  & Morch,  2000)).  The  VisArt  activity  scenario  took  place  during  February  and  March  1999  and 
provided  an  opportunity  for  the  DoCTA  project  to  design,  deploy  and  evaluate  an  authentic  collaborative 
teleleaming  activity.  The  VisArt  learning  activity  involved  students  taking  courses  at  three  Norwegian  educational 
institutions  (University  of  Bergen,  Nord-Trondelag  College,  Stord/Haugesund  College).  Teams  comprised  of  three 
students,  one  from  each  institute,  collaborated  to  design  a learning  activity  through  the  Internet.  There  were  no 
opportunities  for  the  teams  to  meet  face-to-face.  Collaborative  learning  tasks  and  the  technological  environment 
were  carefully  designed.  The  technological  environment  comprised  their  own  email  system,  a word  processor  of 
their  choice,  and  a groupware  system  TeamWave  Workplace  (TW).  One  week  of  training  in  both  using  the  TW 
tools  and  in  collaboration  proceeded  three  weeks  of  design  activity.  The  students  were  asked  to  design  a room  in 
TW  for  learning  about  some  subject  of  their  choice.  All  participants  had  the  option  to  sign  a consent  form  that 
enabled  us  to  electronically  collect  their  team  emails,  questionnaire  data,  and  to  use  the  electronically  logged  TW 
data  for  our  research. 

Evaluation  of  the  VisArt  scenario  (Wasson,  1999)  took  place  on  several  dimensions  and  the  results  are 
currently  being  written  up  as  Masters  dissertations.  The  evaluation  of  interest  in  this  paper  is  how  automatically 
generated  log-files  contribute  to  research  in  understanding  collaboration  patterns  (Meistad,  in  preparation).  Two 
tools  that  have  been  developed  to  assist  researchers  in  extracting  information  from  TW  generated  server  log- files 
and  repositories  are  described.  The  paper  begins  by  briefly  describing  TW  groupware  system.  Then  the  tools  that 
have  been  developed  are  presented.  The  paper  concludes  with  a brief  reflection  on  how  the  extracted  data  can  be 
used. 
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TeamWave  Workplace 


Team  Wave  Workplace  (TW)  is  based  on  the  metaphor  of  shared  networked  places.  Using  real  life  physical  team 
rooms  (Johansen,  1991)  that  provide  a permanent  shared  working  place  for  teams  as  inspiration,  the  notion  of 
virtual  team  rooms  has  been  adopted  in  TW.  These  virtual  team  rooms  provide  a permanent  shared  space  where 
teams  distributed  over  the  Internet  can  have  meetings,  store  documents,  share  URL  links  to  web  sites,  coordinate 
and  communicate  with  one  another  and  can  carry  out  collaborative  activities  such  as  brainstorming  or  participation 
in  a discussion  forum.  Each  team  can  build  a set  of  different  rooms  according  to  need.  For  example,  they  might 
create  a common  project  room  and  resource  room,  and  in  addition,  individual  rooms  for  each  team  member.  TW 
provides  1 9 tools  that  enable  teams  to  easily  plan,  set  up  a database,  design,  share  documents,  link  to  the  web,  chat 
with  one  another  etc.  For  a detailed  review  of  TW  and  for  examples  of  how  it  was  used  in  the  DoCTA  project,  see 
Wasson,  Guribye  & Morch  (2000).  One  feature  of  TW  is  that  it  comes  with  traditional  logging  mechanisms  such  as 
recording  that  a user  logs  in  and  out  of  the  server,  moves  between  rooms  etc.,  as  well  as  a non-traditional  feature,  a 
repository.  All  room  states  are  saved  in  the  repository,  thus  it  becomes  possible  to  recreate  earlier  states  of  any 
given  room  using  an  option  called  version  control.  Both  these  mechanisms  (the  server  log  and  version  control)  are 
used  in  conjunction  with  the  two  prototypes  tools  described  in  the  next  section. 

The  TeamWave  Workplace  Server  File.  All  interactions  with  the  TW  server  are  electronically  logged  in  a 
server  log- file  called  server.log.  The  file  is  automatically  generated  by  the  TW  server  on  its  startup,  and  is  then 
updated  as  the  various  users  interact  with  the  server.  Each  user  action  in  TW  is  recorded  in  the  file.  During  the 
Vis  Art  scenario  the  file  grew  from  0 KB  to  over  2.2  MB  and  has  38,207  lines.  The  following  are  extractions  from 
the  VisArt  server.log  file: 

Fri  Feb  19  15:25:08  1999  oyvind  entered  Overview-room 
Fri  Feb  19  15:25:26  1999  oyvind  left  Overview-room 
Fri  Feb  19  15:25:26  1999  oyvind  entered  Klasse-diagram 
Fri  Feb  19  15:32:25  1999  oyvind  left  Klasse-diagram 
Fri  Feb  19  15:32:25  1999  connection  closed  for  oyvind 

As  one  can  see,  each  line  starts  with  information  about  date  and  time.  Thereafter,  the  user  is  identified  and  the 
user’s  particular  action  is  recorded.  For  example,  the  second  entry  tells  that  on  Friday,  February  19^^  1999,  at 
15:25:26,  oyvind  left  the  Overview-room.  In  addition  to  logging  information  about  users  entering  and  leaving 
rooms  as  shown,  the  TW  server  logs  users  logging  in  and  out  of  TW,  information  about  the  server  starting  up,  the 
creation  of  rooms,  the  writing  of  rooms,  and  the  server  shutting  down.  Although  the  structure  of  the  server.log  file 
is  rather  simple,  the  large  number  of  users  involved  (32  students,  3 instructors  and  1 0 researchers)  created  such  a 
large  file  that  it  is  difficult  to  glean  information  from  the  file.  Therefore,  a software  tool  that  interprets  the  entries 
has  been  developed. 

The  TeamWaveWorkplace  Version  Control.  Every  room  created  in  TW  resides  on  the  TW  server,  and  is  stored 
in  a repository.  When  a TW  user  selects  the  menu  option  to  save  the  current  room,  or  leaves  a room  when  no  other 
person  is  visiting  the  room,  the  room  is  saved  to  the  repository.  Every  tool  created  in  each  room  is  also  stored  in  the 
repository.  As  a room  is  saved,  tools  that  have  been  changed,  or  added,  or  moved  around,  etc.  since  the  previously 
saved  version  of  the  room,  are  saved.  When  a user  enters  a room,  the  last  version  of  the  room  and  the  last  version 
of  the  tools  residing  in  the  room  are  automatically  retrieved  from  the  repository.  TW’s  version  control  enables 
earlier  versions  of  the  current  room,  or  earlier  versions  of  a specific  tool  to  be  retrieved  from  the  repository.  The 
average  number  of  versions  stored  in  the  repository  for  a single  room  at  the  end  of  VisArt  was  approximately  50. 
Depending  on  the  type  of  room,  however,  the  number  varied  a great  deal.  For  instance,  the  Classroom  existed  in  no 
less  than  498  versions.  The  version  control  option  enables  a researcher  to  recreate  the  state  of  various  rooms  and 
tools  during  one’s  analysis.  Thus,  long  after  the  completion  of  the  scenario,  one  is  still  able  to  follow  how  the 
scenario  evolved  for  a particular  team.  The  potentially  large  number  of  available  versions  of  one  room  or  tool,  does, 
however,  make  such  an  analysis  difficult.  Therefore  a software  tool,  TRA,  was  developed  to  help  researchers 
keeping  track  of  how  the  various  rooms  evolved  over  time. 
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Tools  for  Analysing  Log-files 

There  exists  a long  tradition  of  using  logged  data  in  usability  studies  focussed  on  identifying  events  from  such  as 
the  number  of  errors  occurring  or  when  a menu  is  selected  with  the  mouse,  and  graphically  visualising  the  results 
(e.g.,  Fjeld  et  al.  1998;  Guzdial  1993;  Okada  & Asahi  1999).  Another  interesting  approach  is  to  use  the  logged 
data  in  conjunction  with  video  analysis  to  effectively  retrieve  parts  of  a video  recording  (Badre  et  al.  1995). 
Common  in  these  types  of  studies  is  the  focus  on  the  interaction  between  a human  and  a local  computer.  Another 
somewhat  different  tradition  is  to  use  logged  data  to  measure  the  usage  of  various  Internet  resources  such  as  FTP- 
servers,  web-servers,  etc.  This  kind  of  information  is  likely  most  interesting  for  the  Internet  providers  supplying  the 
resources,  but  it  has  also  gained  attention  from  researchers  (e.g.,  Newhagen  & Rafaeli  1997;  Carton  et  al.  1999). 
The  focus  in  these  last  studies  is  on  humans  communicating  through  a local  computer  with  a remote  computer. 

The  tradition  of  using  logged  data  for  identifying  patterns  of  collaboration  between  two  or  more 
geographically  dispersed  persons  is  in  its  infancy.  Recently,  Nurmelea  et.  al.  (1999)  reported  on  a study  of  a CSCL 
environment  where  electronically  stored  files  (log-files  and  documents)  served  as  their  only  form  of  data.  They 
used  social  network  analysis  with  its  focus  on  uncovering  patterns  in  people’s  interactions,  together  with 
information  gleaned  from  log-files.  The  information  was  extracted  with  scripts  written  in  Perl.  CSCL  is  a field  of 
interest  for  researchers  with  backgrounds  as  diverse  as  computer  science  and  pedagogy.  Although  most  computer 
scientists  will  have  little  or  no  problem  writing  a script  in  Perl,  it  is  likely  that  a researcher  who  lacks  experience 
from  computer  science  related  fields  will  experience  problems.  To  address  this  issue,  we  explore  the  possibility  of 
creating  tools  that  use  logged  data  from  a distributed  CSCL  environment  to  make  the  data  more  easily  accessible  to 
researchers. 

To  support  the  evaluations  of  DoCTA  project  scenarios  where  TW  was  used,  it  was  necessary  to  develop  two 
prototypical  tools  for  the  analysis.  The  tools,  the  Server  Log  Analyser  (SLA),  and  the  TWWRoomAnalyzer  (TRA), 
were  both  developed  in  Java  1.1  using  the  Java  Foundation  Classes  1.1  as  graphical  user  interface  components.  The 
programs  differ  from  each  other  in  that  SLA  reads  in  one  log-file,  sorts  out  a researcher  specified  group  of  persons, 
and  then  uses  the  information  to  calculate  some  results  from  the  log.  TRA,  on  the  other  hand,  is  a tool  to  help 
researchers  in  their  classificiation  of  the  use  of  various  tools  inside  TW,  organising  their  classifications,  and 
registering  how  the  use  of  the  tools  evolves  over  time.  In  this  study  TRA  was  used  in  conjunction  with  the  TW 
version  control  and  the  output  from  SLA.  These  two  tool  s are  described  below. 

Server  Log  Analyser 

SLA  takes  as  its  starting  point  the  server.log  file.  The  VisArt  scenario  was  completed  by  1 1 teams  and  SLA  focuses 
on  identifying  the  times  at  which  a team  could  have  possibly  collaborated  synchronously  in  TW  (i.e.,  two  or  more 
of  the  team’s  members  must  have  been  logged  on  at  the  same  time). 

To  find  out  what  types  of  collaboration  were  possible  for  a given  team  at  a given  time,  SLA  reads  through  the 
server.log  using  a researcher- specified  names  of  the  team  members.  SLA  creates  a log  for  each  group  member,  as 
well  as  a group  log  consisting  of  every  entry  from  the  individual  logs,  appearing  in  the  same  order  as  found  in  the 
server.log.  The  resulting  logs  (4  in  this  case)  contain  redundant  information.  SLA  removes  unnecessary 
information  from  the  group  log,  and  adds  some  statistics  to  the  group  log.  This  log  is  used  by  SLA  to  generate  a 
graph  that  illustrates  the  times  at  which  the  various  group  members  were  logged  in  to  TW.  If  the  graph  shows  that 
two  or  more  group  members  were  logged  in  at  the  same  time,  they  had  the  possibility  to  collaborate  synchronously. 
Figure  1 shows  a graph  for  one  of  the  groups  from  VisArt.  The  left  column  of  the  graph  denotes  the  time  of  the 
day.  Each  day  starts  on  the  bottom  at  0,  and  goes  up  past  23  and  up  to  0 again.  The  bottom  axis  specifies  the  date 
that  is  displayed.  If  a date  is  missing  (e.  g.  Feb  26  and  Feb  27),  it  means  that  none  of  the  group  members  logged  in 
that  day. 

Interesting  information  may  be  found  looking  at  the  graph.  First  it  tells  us  that  from  February  25-March  24, 
there  were  only  three  times  when  it  was  possible  for  the  whole  group  to  collaborate  synchronously.  VisArt  started 
on  February  25  when  the  group  members  were  told  to  download  the  TW  client,  and  log  in  to  the  server.  They  were 
given  a few  days  to  carry  out  this  task.  March  1-7  was  a week  of  training  in  use  of  TW  and  collaboration,  and 
March  8-26  was  the  period  of  carrying  out  the  design  task.  By  examining  the  graphs,  it  becomes  apparent  that 
subject02  did  not  log  in  to  TW  before  the  fourth  day  of  the  training  period.  It  also  becomes  apparent  that  the 
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formerly  active  subjectOl,  did  not  log  in  once  between  March  4 and  March  15.  Consequently,  even  asynchronous 
collaboration  is  highly  unlikely  during  this  period.  To  explain  why  these  more  or  less  negative  patterns  occurred,  is 
ditficult  using  a simple  log-file  as  the  only  kind  of  data.  It  is  important,  however,  that  the  researcher  is  aware  that 
one  of  the  three  team  members  disappeared  for  10  days  in  the  middle  of  a 4 week  collaborative  learning  task.  This 
is  valuable  information  that  enables  the  situation  to  be  investigated  more  closely  using,  for  example,  interviews, 
and/or  questionnaires. 


Figure  1 Graph  of  Team  Activity  in  VisArt 


The  prototypical  version  of  SLA,  discussed  here,  is  completely  tied  to  the  server.log  of  TW,  limiting  its  usefulness 
to  studies  were  TW  have  been  used.  A new  version  is,  at  the  time  of  writing,  therefore  being  developed.  When 
ready,  the  new  version  will  support  the  addition  of  all  Java  classes  that  implement  an  interface  defined  in  SLA,  and 
which  enable  sorting  of  log-files  containing  the  type  of  information  found  in  TW’s  server.log.  Although  the  class 
that  sorts  the  TW  server.log  and  generates  SLA’s  output  (various  logs  and  graphs)  in  the  new  version  is  no  larger 
than  153  lines  of  code,  the  approach  is  not  without  problems.  It  require  someone  to  write  the  new  class  and  to 
make  the  class  available  to  those  interested.  A future  goal  will  therefore  be  to  attempt  to  implement  a feature  in 
which  users  of  SLA  can,  at  a very  high  level,  define  how  a log-file  is  organized. 

TWW  Room  Analyzer 

TRA  is  designed  to  be  used  in  an  analysis  of  collaborative  learning  environments  that  use  a room  metaphor  or 
another  analogous  structure  such  as  a web-page.  In  this  study,  TRA  helps  researchers  in  classifying  the  use  of 
various  tools  inside  TW,  organising  their  classifications,  and  registering  how  the  use  of  the  tools  evolves  over  time. 
TRA  requires  the  researcher  to  use  the  output  of  SLA  and  the  TW  version  control  to  specify  input  data  for  TRA. 
Ideally  it  should  be  possible  to  retrieve  most  of  the  information  needed  by  TRA  from  the  TW  repository,  but  that 
path  has  not  yet  been  investigated,  and  would  also  make  the  use  of  the  tool  less  general.  Even  then,  some  of  the 
information  needed  by  TRA,  such  as  a classification  of  the  uses  of  the  tools,  would  still  have  to  be  identified  by  a 
researcher.  Once  the  data  has  been  input  to  TRA,  the  researcher  can  use  TRA  to  track  changes  to  rooms  and  tools 
over  time. 

TRA,  shown  in  figure  2,  is  a dynamic  tool  that  is  used  to  track  how  a team’s  learning  environment  evolved  in 
the  course  of  the  scenario.  TRA  comprises  several  graphical  user  interface  components  that  present  information 
about  any  number  of  rooms.  For  each  room  there  may  be  any  number  of  room  versions,  with  each  room  version 
containing  any  number  of  tools,  and  any  number  of  tool  versions.  As  TRA  is  currently  being  used  during  analyses 
of  VisArt,  it  is  possible  new  features  may  be  required.  If  so,  then  TRA  will  be  revamped  and  the  user  interface  may 
change. 
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The  whiteboard  is  used  in  conjunction  with  the  postit  note  that  is  under  tool  3.  On  the  whiteboard  it  is  written  directy  postioned  over  tool  3;| 
Leggigjenbeskjedomatduharveerther.  husknavnogtidspunktj  | 

^ubjectX  leaves  a message  in  tool  3,  presumably  after  having  seen  the  texUjal  note  | 


Figure  2 One  of  many  possible  views  of  TRA 

In  order  to  use  TRA,  the  researcher  first  uses  SLA  to  obtain  information  about  the  times  when,  places  where,  and 
the  team  members  who  were  logged  in  (i.e.,  the  date,  time  and  room).  SLA  also  generates  information  about  the 
people  that  visited  a particular  room  during  the  scenario.  For  example  in  figure  2 the  rooms  MyTeam2  and 
Classroom  have  been  entered,  and  MyTeam2  room  is  active.  One  can  also  see  that  MyTeam2  was  originally 
created  by  Trond  Pedersen  on  January  23,  1999  at  13:59.  There  have  been  12  versions  of  MyTeam2  entered  into 
TRA.  In  figure  2,  version  10,  created  by  Rune  Baggetun,  is  active.  TW’s  version  control  is  then  used  to  retrieve  the 
active  version  of  the  room  and  the  researcher  can  see  which  tools  are  present  in  the  room.  This  information  is 
entered  into  TRA  along  with  the  number  of  versions  of  each  tool.  In  figure  2,  version  4 of  the  Whiteboard  tool, 
created  by  Helge  Underhaug  on  February  17,  1999  at  16:35  is  visible.  The  researcher  also  classifies  the  use  of  the 
particular  version  of  the  tool  indicating  information  such  as  if  the  current  use  has  changed  compared  to  previous 
uses,  if  the  tool  was  used  synchronously  or  asynchronously  (menu  selection),  the  role  of  the  actor  who  made  or 
changed  the  tool,  etc.  In  figure  2,  the  researcher  who  classified  the  use  of  the  Whiteboard  believes  that  the 
Norwegian  phrase  written  on  the  Whiteboard,  shown  in  figure  3,  meaning  “Leave  a note  that  you  have  been  here, 
remember  name  and  time!”  must  be  seen  in  conjunction  with  the  Postit  also  shown  in  figure  3 (the  Postit  is  refered 
to  as  tool  3 in  the  classification  of  the  use  of  the  Whiteboard  shown  in  figure  2). 


Legg  igjen  beslqed  om  at  du  har  vaert  her,  husk  navn  og  tidspunkt! 


Var  her  onsdag  12.15 
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Figure  3 Part  of  the  Whiteboard  and  a Postit  from  version  10  of  MyTeam2  room 
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Conclusions 


This  paper  has  described  two  tools  developed  for  the  ongoing  analysis  of  collaborative  teleleaming  scenarios  using 
TW.  The  tools  take  automatically  generated  data  as  their  input.  SLA  does  so  as  a fairly  automated  process,  whereas 
TRA  depends  on  a great  deal  of  interaction  with  the  researcher.  Information  produced  by  SLA  is  useful  for 
identifying  the  times  at  which  a team  had  the  potential  to  collaborate  asynchronously  or  synchronously.  To  say  that 
it  is  possible  to  collaborate  is,  however,  not  to  say  that  collaboration  occurred.  Thus,  SLA’s  output  can  be  used  to 
focus  other  methods  that  are  being  used  to  interpret  the  interaction  between  users  of  TW  through  its  various  tools. 

TRA  was  developed  as  aid  for  studying  the  learning  environment  of  a team  (e.g.,  the  rooms  involved,  the 
persons  influencing  the  environment,  the  available  tools,  etc.).  Incorporating  Tele-universite’s  tool  classification 
model  for  a virtual  campus  environment  (Paquette  et.  al.,  1995)  in  TRA,  enables  TRA  to  suggest  how  the 
particular  uses  of  the  tools  found  in  the  team’s  rooms  may  be  classified.  Using  these  tools  in  conjunction  with 
other  data  gathering  methods  such  as  interviews  or  video  observation,  the  researcher  will  be  able  to  develop  a 
richer  picture  of  collaboration  patterns. 
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Abstract:  This  paper  describes  the  results  of  a feature  analysis  aimed  at  evaluating  how 
adequate  hypermedia  models,  methodologies  and  systems  are  when  applied  to  authoring  for 
education. 

The  requirements  used  in  our  work  were  identified  by  educational  hypermedia  authors  and 
also  by  existing  literature  in  the  field  of  hypermedia  authoring.  Our  objective  was  to  assess 
whether  or  not  hypermedia  models,  methodologies  and  systems  offered  features 
corresponding  to  the  requirements  identified  by  educational  hypermedia  authors/existing 
literature. 

The  results  obtained  showed  that  none  of  the  hypermedia  models  and  methodologies  used  in 
our  evaluation  had  features  corresponding  to  the  essential  requirements  that  have  been 
identified.  Among  the  hypermedia  systems,  only  three  presented  features  which 
corresponded  to  requirements  considered  essential  by  hypermedia  authors. 


Introduction 

The  aim  of  a feature  analysis  (Kitchenham  and  Jones  1997a;  1997b;  1997c)  is  to  identify  which  of 
several  methods/systems/methodologies  best  fits  the  requirements  of  potential  users.  The  requirements  are 
translated  into  a list  of  desired  features,  which  later  on  will  be  compared  against  the  features  offered  by  the 
candidate  methods/systems/methodologies. 

The  feature  analysis  presented  in  this  paper  was  used  to  identify:  i)  which  models,  methodologies  and 
systems  would  fit  the  requirements  of  potential  users;  ii)  potential  areas  of  research  that  would  improve 
hypermedia  authoring  by  improving  either  the  models/methods,  methodologies  or  systems  selected  in  the 
evaluation. 

The  depth  of  investigation  for  the  candidate  hypermedia  models,  methodologies  and  systems  was 
based  on  published  literature.  Hypermedia  models  are,  in  general,  used  for  the  conceptual  design  of 
hypermedia  applications  whereas  hypermedia  methodologies  are  used  for  the  development  of  hypermedia 
applications.  The  models  and  methodologies  considered  in  our  work  are  presented  in  Table  1 : 


Hypermedia  Models 

Hypermedia  Methodologies 

HDM  (Garzotto  et  al.  1995) 

RMM  (Isakowitz  et  al.  1997) 

HDM2  (Garzotto  et  al.  1996b) 

PHD  (Schuler  and  Thiiring  1 994) 

OOHDM  (Schwabe  and  Rossi  1995) 

Thiiring’s  framework  (Thiiring  1994) 

EORM  (Lange  1996) 

Table  1:  Hypermedia  models  and  methodologies  used  in  the  feature  analysis  evaluation. 


Hypermedia  systems  are  tools  used  to  develop  hypermedia  applications.  The  hypermedia  systems  used 
in  the  feature  analysis  are  presented  in  table  2: 

Hypermedia  Systems 
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Conclusions 


This  paper  has  described  two  tools  developed  for  the  ongoing  analysis  of  collaborative  teleleaming  scenarios  using 
TW.  The  tools  take  automatically  generated  data  as  their  input.  SLA  does  so  as  a fairly  automated  process,  whereas 
TRA  depends  on  a great  deal  of  interaction  with  the  researcher.  Information  produced  by  SLA  is  useful  for 
identifying  the  times  at  which  a team  had  the  potential  to  collaborate  asynchronously  or  synchronously.  To  say  that 
it  is  possible  to  collaborate  is,  however,  not  to  say  that  collaboration  occurred.  Thus,  SLA’s  output  can  be  used  to 
focus  other  methods  that  are  being  used  to  interpret  the  interaction  between  users  of  TW  through  its  various  tools. 

TRA  was  developed  as  aid  for  studying  the  learning  environment  of  a team  (e.g.,  the  rooms  involved,  the 
persons  influencing  the  environment,  the  available  tools,  etc.).  Incorporating  Tele-universite’s  tool  classification 
model  for  a virtual  campus  environment  (Paquette  et.  al.,  1995)  in  TRA,  enables  TRA  to  suggest  how  the 
particular  uses  of  the  tools  found  in  the  team’s  rooms  may  be  classified.  Using  these  tools  in  conjunction  with 
other  data  gathering  methods  such  as  interviews  or  video  observation,  the  researcher  will  be  able  to  develop  a 
richer  picture  of  collaboration  patterns. 
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Abstract;  This  paper  describes  the  results  of  a feature  analysis  aimed  at  evaluating  how 
adequate  hypermedia  models,  methodologies  and  systems  are  when  applied  to  authoring  for 
education. 

The  requirements  used  in  our  work  were  identified  by  educational  h)q)ermedia  authors  and 
also  by  existing  literature  in  the  field  of  hypermedia  authoring.  Our  objective  was  to  assess 
whether  or  not  hypermedia  models,  methodologies  and  systems  offered  features 
corresponding  to  the  requirements  identified  by  educational  hypermedia  authors/existing 
literature. 

The  results  obtained  showed  that  none  of  the  hypermedia  models  and  methodologies  used  in 
our  evaluation  had  features  corresponding  to  the  essential  requirements  that  have  been 
identified.  Among  the  hypermedia  systems,  only  three  presented  features  which 
corresponded  to  requirements  considered  essential  by  hypermedia  authors. 


Introduction 

The  aim  of  a feature  analysis  (Kitchenham  and  Jones  1997a;  1997b;  1997c)  is  to  identify  which  of 
several  methods/systems/methodologies  best  fits  the  requirements  of  potential  users.  The  requirements  are 
translated  into  a list  of  desired  features,  which  later  on  will  be  compared  against  the  features  offered  by  the 
candidate  methods/systems/methodologies. 

The  feature  analysis  presented  in  this  paper  was  used  to  identify:  i)  which  models,  methodologies  and 
systems  would  fit  the  requirements  of  potential  users;  ii)  potential  areas  of  research  that  would  improve 
hypermedia  authoring  by  improving  either  the  models/methods,  methodologies  or  systems  selected  in  the 
evaluation. 

The  depth  of  investigation  for  the  candidate  hypermedia  models,  methodologies  and  systems  was 
based  on  published  literature.  Hypermedia  models  are,  in  general,  used  for  the  conceptual  design  of 
hypermedia  applications  whereas  hypermedia  methodologies  are  used  for  the  development  of  hypermedia 
applications.  The  models  and  methodologies  considered  in  our  work  are  presented  in  Table  1: 


Hypermedia  Models 

Hypermedia  Methodologies 

HDM  (Garzotto  et  al.  1995) 

RMM  (Isakowitz  et  al.  1997) 

HDM2  (Garzotto  et  al.  1996b) 

PHD  (Schuler  and  Thiiring  1 994) 

OOHDM  (Schwabe  and  Rossi  1 995) 

Thiiring’s  framework  (Thiiring  1994) 

EORM  (Lange  1996) 

Table  1:  Hypermedia  models  and  methodologies  used  in  the  feature  analysis  evaluation. 


Hypermedia  systems  are  tools  used  to  develop  hypermedia  applications.  The  hypermedia  systems  used 
in  the  feature  analysis  are  presented  in  table  2: 


Hypermedia  Systems 
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SEPIA  (Thiiring  et  al.,  1995) 

Intermedia  (Haan  et  al.  1992) 

MacWeb  (Nanard  and  Nanard  1995) 

StorySpace  (Bernstein  et  al.  1991b) 

IDE  (Jordan  etal.  1989) 

HyDesign  (Marmann  and  Schlageter  1992) 

the  Web  (Berners-Lee  et  al.  1994) 

Webcosm  (Hill  1998) 

Aquanet  (Marshall  et  al.  1991) 

HyperWave  (Maurer  1996) 

VIKI  (Marshall  and  Shipman  1997) 

ODMTool  (Lange  1996) 

RMCase  (Diaz  and  Isakowitz  1995) 

Microcosm  (Hall  et  al.  1996) 

Table  2:  Hypermedia  systems  used  in  the  feature  analysis  evaluation 


The  feature  analysis  evaluation  was  organised  using  a screening  mode  approach  (Kitchenham  and 
Jones  1997b)  and  was  performed  by  one  person.  The  screening  mode  is  the  fastest  and  least  costly  form  of 
feature  analysis  and  the  number  of  different  roles  is  minimised.  On  the  other  hand,  the  whole  evaluation  is 
based  on  the  subjective  assessment  of  one  person  or  team,  which  may  bias  the  results. 

The  requirements  used  in  the  feature  analysis  were  based  on  published  literature  in  the  fields  of 
hypermedia  authoring  for  education  and  experimentation  and  on  the  results  of  a qualitative  evaluation  that 
analysed  the  process  of  authoring  hypermedia  applications  for  education  (Mendes  and  Hall,  1999). 

The  remainder  of  this  paper  is  organised  as  follows:  Section  2.0  describes  the  feature  analysis 
evaluation  and  the  results  obtained.  Section  3.0  gives  our  conclusions  and  comments  on  future  work. 


The  Feature  Analysis  Evaluation 

The  Assessment  Scales  Used  for  Scoring  the  Features 

The  first  step  of  a feature  analysis  is  to  prepare  a feature  list,  where  each  feature  is  classified  into 
simple  (S)  or  compound  (CO).  Each  feature  also  has  a degree  of  importance  associated,  which  can  be: 
Mandatory  (M),  Highly  Desirable  (HD),  Desirable  (D)  and  Nice  to  have  (N).  Each  degree  of  importance  has 
also  a weighting  factor  associated,  which  will  be  used  by  compound  features.  The  corresponding  weighting 
factors  for  M,  HD,  D and  N are,  respectively,  10,  6,  3 and  1 (Kitchenham  and  Jones  1997b).  Simple  features 
are  scored  as  five  (for  Yes)  or  zero  (for  No). 

The  judgement  scales  that  we  have  used  to  assess  a method/methodology/system  support  for  a 
particular  feature  are  as  follows  (see  Tables  3 and  4): 


Generic  Scale 
Point 

Definition  Scale  Point 

Scale  Point 
Mapping 

Makes  things 
Worse 

Causes  confusion.  The  way  the  feature  is  implemented  makes  it 
difficult  to  use  and/or  encourages  its  incorrect  use 

-1 

No  support 

Fails  to  recognise  it.  The  feature  is  not  supported  nor  referred  to  in  the 
published  literature  available. 

0 

Little  support 

The  feature  is  supported  indirectly,  by  the  use  of  other  system  features 
in  non-standard  combinations. 

1 

Some  support 

The  feature  appears  explicitly  in  the  system  and  in  the  published 
literature.  However,  some  aspects  of  the  feature  use  are  not  catered  for. 

2 

Strong  support 

The  feature  appears  explicitly  in  the  system  and  in  the  published 
literature.  All  aspects  of  the  feature  are  covered,  but  its  use  depends  on 
the  expertise  of  the  user. 

3 

Very  strong 
support 

The  feature  appears  explicitly  in  the  system  and  in  the  published 
literature.  All  aspects  of  the  feature  are  covered,  and  the  system 
provides  tailored  dialogue  boxes  to  assist  the  user. 

4 

Full  support 

The  feature  appears  explicitly  in  the  system  and  in  the  published 
literature.  All  aspects  of  the  feature  are  covered,  and  the  system 
provides  user  scenarios  to  assist  the  user. 

5 
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Table  3:  Judgement  scale  to  assess  hypermedia  system  support  for  a feature 


Generic  Scale 
Point 

Definition  Scale  Point 

Scale  Point 
Mapping 

Makes  things 
Worse 

Causes  confusion.  The  way  the  feature  is  implemented  makes  it 
difficult  to  use  and/or  encourages  its  incorrect  use. 

-1 

No  support 

Fails  to  recognise  it.  The  feature  is  not  supported  nor  referred  to  in  the 
published  literature  available. 

0 

Little  support 

The  feature  is  supported  indirectly,  by  the  use  of  other 
model/method/methodology  features  in  non-standard  combinations. 

1 

Some  support 

The  feature  appears  explicitly  in  the  model/method/methodology  and 
in  the  published  literature.  However,  some  aspects  of  the  feature  use 
are  not  catered  for. 

2 

Strong  support 

The  feature  appears  explicitly  in  the  model/method/methodology  and 
in  the  published  literature.  All  aspects  of  the  feature  are  covered,  but 
its  use  depends  on  the  expertise  of  the  user. 

3 

Very  strong 
support 

The  feature  appears  explicitly  in  the  model/method/methodology  and 
in  the  published  literature.  All  aspects  of  the  feature  are  covered,  and 
the  model/method/methodology  provides  tailored  questions  and 
frequently  asked  questions  to  assist  the  user. 

4 

Full  support 

The  feature  appears  explicitly  in  the  model/method/methodology  and 
in  the  published  literature.  All  aspects  of  the  feature  are  covered,  and 
the  model/method/methodology  provides  user  scenarios  to  assist  the 
user. 

5 

Table  4:  Judgement  scale  to  assess  hypermedia  model/methodology  support  for  a feature 


The  List  of  Features 

The  Features  Obtained  from  the  Qualitative  Survey 

This  section  describes  a list  of  features,  which  was  based  on  the  results  of  a qualitative  survey 
evaluation  (Mendes  and  Hall  1999).  The  survey  aimed  at  analysing  the  processes  involved  in  the  authoring  of 
hypermedia  applications  for  education.  The  list  of  features  and  the  type  and  degree  of  importance  associated 
with  each  one  are  presented  below  (see  Table  5).  A more  detailed  explanation  of  each  feature  is  given  in 
Appendix  A: 


Feature  Description 

Type 

Importanc 

e 

M,  MD,  S 

N 

Supports  structural  knowledge  design 

Compound 

Mandatory 

All  three 

04 

Supports  structuring  different  domains 

Simple 

Mandatory 

All  three 

05 

Supports  a top-down  approach  to  structuring  the 
information 

Compound 

Mandatory 

All  three 

06 

Links  are  not  essential  in  order  to  structure  the  knowledge 
domain 

Simple 

Mandatory 

All  three 

07 

Supports  the  representation  of  the  author’s  cognitive  map 

Compound 

Mandatory 

Method, 

methodology 

08 

Supports  the  interface  design,  structural  knowledge  design 
and  the  authoring  of  the  nodes’  contents  separately 

Compound 

Mandatory 

All  three 

09 

Allows  readers  to  see  the  author’s  cognitive  map 

Compound 

Mandatory 

System 

10 

Supports  text 

Simple 

Mandatory 

System 

11 

Supports  images 

Simple 

Mandatory 

System 

12 

Supports  typed  links 

Compound 

Mandatory 

System 

13 
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Supports  contextual  links 

Compound 

Mandatory 

System 

14 

Supports  the  structuring  of  information  according  to  a 
problem-solving  approach 

Simple 

Mandatory 

System 

15 

Supports  different  levels  of  information  granularity 

Compound 

Mandatory 

System 

16 

Supports  collaboration  in  the  system 

Compound 

Highly 

desirable 

System 

17 

M - Method  feature  MD  - Methodology  feature  S - System  feature  N - # in  the  scoring  sheet 

Table  5:  Feature  list  obtained  from  the  qualitative  survey 


General  Features 

This  sub-section  presents  three  general  features,  taken  from  the  literature  in  hypermedia  authoring  for 
education  and  empirical  evaluation.  These  were  considered  relevant  for  the  feature  analysis  evaluation.  Each 
general  feature  is  described  below  and  a summary  is  given  in  the  end  of  this  sub-section  (see  Table  6). 

a)  Support  for  ill-structured  domains 

According  to  Rand  Spiro  et  al.  (1995)  all  domains  which  involve  the  application  of  knowledge  to 
unconstrained,  naturally  occurring  situations  (cases)  are  highly  ill-structured.  Even  those  knowledge  domains 
that  are  well-structured  also  have  aspects  of  ill-structuredness,  especially  at  more  advanced  levels  of  study,  e.g. 
mathematics. 

Ill-structuredness  is  not  a serious  problem  for  introductory  learning  as  learners  are  not  expected  to 
master  complexity  or  independently  transfer  their  acquired  knowledge  to  new  situations.  However,  when 
conceptual  mastery  and  flexible  knowledge  application  become  paramount,  the  complexity  and  across-case 
diversity  characteristic  of  ill-structured  domains  needs  to  be  addressed  (Carvalho  and  Dias  1997;  Jacobson  and 
Archodidou  1997;  Rana  and  Bieber  1997). 

b)  Has  been  validated  empirically  using  formal  experimentation 

In  a random  sample  of  all  the  papers  that  the  ACM  published  in  1993,  40  percent  of  them  contained 
claims  that  needed  empirical  support  but  had  none  at  all  (Tichy  1998).  In  software-related  journals,  this 
fraction  was  50  percent.  Tichy’s  data  shows  that  computer  scientists  validate  a smaller  percentage  of  their 
claims  than  other  scientists.  He  also  asserts  that  a balance  between  theory  and  engineering  with 
experimentation  would  have  the  following  benefits: 

• Experimentation  can  help  build  a reliable  base  of  knowledge  and  thus  reduce  uncertainty  about  which 
theories,  methods  and  tools  are  adequate. 

• Observation  and  experimentation  can  lead  to  new,  useful,  and  unexpected  insights  and  open  whole  new 
areas  of  investigation.  Experimentation  can  push  into  unknown  areas  where  engineering  progresses 
slowly,  if  at  all. 

• Experimentation  can  accelerate  progress  by  quickly  eliminating  fruitless  approaches,  erroneous 
assumptions,  and  fads.  It  also  helps  orient  engineering  and  theory  in  promising  directions. 

• Demonstrations  can  provide  proof  of  concepts  or  incentives  to  study  a question  further.  Too  often, 
however,  these  demonstrations  merely  illustrate  a potential.  To  obtain  evidence,  careful  analysis  involving 
experiments,  data,  and  replication  is  necessary. 

According  to  Fenton  et  al.  (1994)  much  of  what  is  believed  about  which  approaches  are  best  is  based  on 
anecdotes,  gut  feelings,  expert  opinions,  and  flawed  research,  rather  than  on  careful,  rigorous  software 
experimentation. 

c)  Was  proposed  using  a scientific  method  of  research 

Glass  (1994)  gives  a classification  of  possible  research  models: 

• The  scientific  method  - Observe  the  world,  propose  a model  or  theory  of  behaviour,  measure  and  analyse, 
validate  hypotheses  of  the  model  or  theory,  and  if  possible  repeat. 

• The  engineering  method  - Observe  existing  solutions,  propose  better  solutions,  build  or  develop,  measure 
and  analyse,  repeat  until  no  further  improvements  are  possible. 

• The  empirical  method  - Propose  a model,  develop  statistical  or  other  methods,  apply  to  case  studies. 
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measure  and  analyse,  validate  the  model,  repeat. 

• The  analytical  method  - Propose  a formal  theory  or  set  of  axioms,  develop  a theory,  derive  results,  and  if 
possible  compare  with  empirical  observations. 

Most  of  the  research  to  date  in  hypermedia  authoring  uses  an  analytical  advocacy  model  of  research 
without  further  empirical  justification,  transforming  the  process  of  authoring  into  art,  rather  than  into  science 
(Isakowitz  et  al.l995). 

As  engineering  involves  the  analysis  of  measurements  (Pfleeger  et  al.  1997),  hypermedia  engineering  will 
only  become  a true  engineering  discipline  when  a solid  foundation  of  measurement-based  theories  is  build. 

The  general  feature  list  and  the  corresponding  type  and  degree  of  importance  are  presented  below  (see 
Table  6): 


Feature  Description 

Type 

Importance 

M,  MD,  S 

N 

Supports  ill-structured  domains 

Simple 

Mandatory 

All  three 

1 

Has  been  validated  empirically  using  formal 
experimentation 

Simple 

Highly  desirable 

All  three 

2 

Was  proposed  according  to  a scientific  method  of  research 

Simple 

Highly  desirable 

All  three 

3 

M — Method  feature  MD  — Methodology  feature  S - System  feature  N — # in  the  scoring  sheet 

Table  6:  General  feature  list 


The  Evaluation  Profiles 

The  Evaluation  Profile  for  Hypermedia  Models,  Methods  and  Methodologies 

None  of  the  hypermedia  models,  methods  and  methodologies  evaluated  supports  the  structuring  of  ill- 
structured  domains,  which  is  one  of  the  mandatory  features.  Consequently,  they  were  all  eliminated  from  the 
list  of  candidates.  Isakowitz  and  Turing  (1994)  corroborate  this  result: 

“These  methodologies  are  apt  for  structured  domains  where  economies  of  scale  can  apply,  i.e,,  there  are 
many  objects  of  the  same  kind.  The  key  difference  between  the  approaches  is  that  Schwabe  and  Rossi  is 
object-oriented,  while  Balasubramanian,  Isakowitz  and  Stohr’s  is  based  on  an  entity-relationship 
approach.”. 

The  Evaluation  Profile  for  Hypermedia  Systems 

None  of  the  hypermedia  systems,  except  Microcosm,  Hy Design  and  Webcosm  support  contextual 
links.  Consequently,  apart  from  those  3,  all  the  others  were  eliminated  from  the  list  of  candidates.  The 
evaluation  profiles  for  the  three  hypermedia  systems  are  presented  below  (see  Table  7): 


Conformance  Score 

#Feature  Number 

Importance 

Microcosm 

HyDesign 

Webcosm 

01 

M 

5 

5 

5 

02 

HD 

0 

0 

0 

03 

HD 

0 

0 

0 

05 

M 

1 X 10 

2x10 

1 X 10 

06 

M 

5 

5 

5 

07 

M 

3x  10 

3x  10 

3x  10 

08 

M 

5 

5 

5 

10 

M 

2x10 

1 X 10 

2x  10 

11 

M 

3x  10 

3x  10 

3x  10 

12 

M 

5 

5 

5 

13 

M 

5 

5 

5 

14 

M 

2x10 

1 X 10 

1 X 10 

15 

M 

3x  10 

3x  10 

3x10 

16 

M 

5 

5 

5 

17 

M 

2x  10 

1 X 10 

3x  10 
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20 

HD 

0 

0 

2x6 

Total  score 

190 

170 

202 

M - Mandatory  HD  - Highly  desirable 

Table  7:  Evaluation  profile  for  the  hypermedia  systems 


Conclusions  and  Future  Work 

All  the  models,  methods  and  the  methodologies  and  nearly  all  the  hypermedia  systems  were 
eliminated.  The  two  main  requirements  that  have  contributed  to  these  results  were  ‘support  for  ill- 
structured  domains’  and  ‘support  for  contextual  links’.  The  scores  obtained  by  Microcosm  and  Webcosm  were 
quite  similar  and  higher  than  the  score  obtained  for  HyDesign, 

The  results  of  the  feature  analysis  evaluation  showed  that  in  practice  authors’  requirements  are  not  being 
mapped  by  state-of-the-art  methods  and  methodologies,  corroborating  Fenton’s  affirmation  that: 

“best  projects  do  not  necessarily  have  state-of-the-art  methodologies  or  extensive  automation  and 
tooling.  They  rely  on  basic  principles  such  as  strong  team  work,  project  communication,  and  project 
controls.  Good  organisation  and  management  appears  to  be  far  more  of  a critical  success  factor  than 
technology  or  methodology,”  (Fenton  1993), 

The  feature  analysis  evaluation  also  showed  that  none  of  the  hypermedia  system  candidates  has  scored 
points  for  the  general  features  ‘being  validated  empirically  using  a formal  experimentation’,  and  ‘being 
proposed  according  to  a scientific  method  of  research’. 

Our  next  step  is  to  apply  a scientific  approach  to  the  evaluation  and  proposal  of  several  metrics  that  will 
measure  the  authoring  quality  of  educational  hypermedia  applications  developed  using  either  Microcosm  or 
Webcosm.  By  applying  measurement  principles  we  hope  to  generate  quantitative  descriptions  of  hypermedia 
authoring  processes  and  applications  that  will  enable  us  to  control  and  improve  the  authoring  process  as  a 
whole. 
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Appendix  A 

Supports  structural  knowledge  design  - Does  the  hypermedia  system  allow  the  author  to  structure  the 
application  according  to  the  structural  knowledge  corresponding  to  a particular  domain?  The  structural 
knowledge  is  the  knowledge  of  how  concepts  within  a domain  are  interrelated. 

Supports  structuring  different  domains  - Does  the  hypermedia  system  allow  the  author  to  structure  the 
applications  in  different  ways,  corresponding  to  which  subjects  (domain)  is  being  taught? 
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Supports  a top-down  approach  to  structuring  the  information  - Does  the  hypermedia  system  allow  the  author 
to  structure  the  contents  using  a hierarchy? 

Links  are  not  essential  in  order  to  structure  the  knowledge  domain  - Does  the  author  need  to  define  the  links 
in  the  application  at  the  same  time  as  (s)he  structures  the  knowledge  domain? 

Supports  the  representation  of  the  author's  cognitive  map  - A cognitive  map  is  the  graphical  representation  of 
the  author’s  structural  knowledge.  Does  the  hypermedia  system  support  any  graphical  representation  of  the 
structural  knowledge? 

Supports  the  interface  design,  structural  knowledge  design  and  the  authoring  of  the  nodes*  contents  separately 
- Does  the  hypermedia  system  allow  authors  to  design  the  application’s  interface,  design  the  structural 
knowledge  and  author  the  contents  of  each  node  separately?  Or  does  everything  need  to  be  done  ”at  the  same 
time"? 

Allows  readers  to  see  the  author's  cognitive  map  - Does  the  hypermedia  system  allow  readers  to  see  the 
graphical  representation  of  the  structural  knowledge  that  represents  the  application’s  content? 

Supports  typed  links  - A typed  link  is  a link  that  has  a description  (label)  attached  to  it.  In  general,  types 
explain  what  kind  of  information  readers  will  find  when  they  follow  that  particular  link.  Examples  of  types  are 
‘example’,  ‘definition’,  ‘more  information’,  etc. 

Supports  contextual  links  - Context-dependent  links  are  links  that  are  recognised  by  the  hypermedia  system 
within  certain  boundaries  established  by  the  author.  A boundary  can  be  a document,  the  whole  hypermedia 
application,  an  anchor,  a set  of  documents  (nodes). 

Supports  different  levels  of  information  granularity  - Does  the  system  support  the  structuring  of  information 
using  different  levels  of  abstraction  (granularity)? 

Supports  collaboration  in  the  system  - Does  the  system  allow  authors  to  collaborate  in  the  authoring  process? 
Can  authors  develop  the  same  application  at  the  same  time? 
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Abstract:  While  it  may  be  quite  usual  for  proponents  of  some  cause  to  enthusiastically  portray 
their  case  in  the  best  possible  light,  when  it  comes  to  changes  in  education,  there  is  no  excuse 
for  not  considering  the  true  impact  on  the  final  target  of  those  changes,  the  students.  In  the  push 
for  electronic  delivery,  distance  education  students  have  been  found  to  carry  a significant 
number  of  costs  which  rarely  find  their  way  into  cost-benefit  analyses  carried  out  by  the 
institution  or  its  staff.  Wasted  time  and  effort,  decreased  learning  access  and  a variety  of  social 
costs  are  all  part  of  the  real  price  that  a distance  student  often  pays. 


The  Changing  Scene 

It  appears  inevitable  that  more  and  more  education  is  going  to  be  delivered  in  the  distance  mode  (National  Center 
for  Education  Statistics  1998)  and  that  an  active  debate  exists  in  the  move  to  electronic  delivery  of  education.  One 
only  has  to  look  at  any  of  the  reports  into  higher  education  such  as  that  of  the  Kellog  Commission  on  the  future  of 
state  and  land  grant  universities  (Stukel  1 998)  to  appreciate  how  determined  university  administrators  are  to  take 
advantage  of  the  use  of  Information  and  Communications  Technology  (ICT).  Given  all  the  rhetoric  and  posturing, 
one  could  be  forgiven  for  thinking  that  this  is  revolutionary  rather  than  evolutionary.  It  is  significant  to  note  that 
the  advent  of  another  communication  technology,  public  postage,  was  instrumental  in  the  establishment  of  early 
Distance  Education  (DE)  and  that  while  the  Internet  has  challenged  the  current  notions  of  what  DE  means  and 
how  it  is  to  be  delivered,  public  postage  and  DE  did  not  totally  destroy  the  existing  university  order  in  the  last  100 
years!  History  suggests  that  the  Internet  will  not  either.  ICT  provides  many  new  and  exciting  prospects  that  will 
complement  and  extend,  rather  than  totally  supplant,  existing  education. 

These  changes  to  education  make  it  imperative  that  the  impact  of  this  mode  and  medium  on  students  be  understood 
and  taken  into  account.  Literature  such  as  Twigg  (1999)  is  heavily  skewed  in  favour  of  the  institutional  effects  and 
the  impact  on  the  instructors.  An  analysis  of  the  costs  and  benefits  for  students  such  as  that  carried  out  for 
instructors  by  Collis,  Winnips  & Moonen  (1999)  is  not  easily  found.  Sibillin  et  al.  (1996  p205)  admit  that 
determinations  about  the  future  of  higher  education  are  "a  discourse  in  which  students  appear  all  too  irregularly.*' 
Hara  and  Kling  (1999)  while  broaching  the  topic  of  student  frustrations  with  using  web-based  education  conclude 
that  the  predominantly  positive  literature  emphasizes  potential  and  understates  the  problems  students  face.  Even 
more  difficult  to  find  is  reliable  data  about  the  impact  on  DE  students  who  are  not  at  all  like  full-time  students.  DE 
students  are  much  more  heavily  influenced  by  external  factors  associated  with  the  many  different  roles  that  they 
have  to  undertake.  This  alters  the  learning  process  for  each  in  a wide  variety  of  ways  resulting  in  a more 
andragogical  rather  than  pedagogical  way  of  learning.  (Roberts  1984,  Woodley  1987,  Garland  1993) 

Determining  how  electronic  forms  of  learning  are  to  feature  in  the  DE  environment  cannot  be  done  using  just 
pedagogical,  technological  or  economic  criteria.  These  are  necessary  and  may  ultimately  limit  the  scope  of  what  is 
feasible  at  any  time,  but  they  are  certainly  not  sufficient.  The  effects  that  such  strategies  have  on  the  end  user,  the 
learner,  are  equally  important.  Key  components  of  DE  are  learner  choice  over  what,  how,  where  and  when  to 
study.  Electronic  materials  have  the  potential  to  alter  these  choices.  Whether  this  changes  as  learners’  exposure  to 
the  use  of  ICT  increases,  must  also  be  considered.  The  philosophical  changes  to  educational  practice,  which  are 
moving  to  more  learner  participation  in  activities  and  the  social  impact  of  such  changes  are  additional  concerns. 
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Costs 


It  is  frequently  the  case  (cf  CAUT  1993;  Western  Governors  Association  1996;  Bates  1997,  Stukel  1998,  Sakamoto 
1999)  that  decisions  to  use  technology  to  deliver  education  make  mention  of  improved  access  to  higher  quality 
learning  materials  and  better  cost-effectiveness  of  the  process.  The  relationship  between  costs  and  quality  are  not 
simple,  nor  clear  (Bates  1995,  Moonen  1999),  Justifications  for  the  use  of  educational  technology  sometimes 
incorporate  assumptions  that  this  will  result  in  a better  learning  environment  for  the  student  and  suggestions  of 
possibly  greater  success  (Reushle  1 995).  Such  assumptions  are  not  easily  challenged  because  they  appear  plausible 
and  the  collection  of  reliable  data  is  difficult.  There  are  many  factors  involved  in  the  quality  of  the  education  that  a 
learner  receives  and  improvements  in  one  area  with  degradation  in  another  may  prove  inconclusive,  Russell’s 
(1999)  collection  of  reports  resulting  in  the  'No  significant  difference  phenomenon’  is  testament  to  that. 

Better  quality  learning  materials  which  place  greater  restrictions  on  the  'how,  when  and  where'  of  the  learning 
process  may  result  in  a decrease  in  satisfaction  or  performance.  The  interaction  of  many  other  variables  such  as 
the  effect  of  peer  support  and  face  to  face  context,  are  difficult  to  quantify  and  even  more  difficult  to  control.  Add 
in  new  variables  associated  with  life  long  learning  and  flexible  'education  on  demand'  and  the  ability  to  accurately 
predict  improvement  in  educational  quality,  regardless  of  the  technology,  becomes  exceedingly  difficult. 

Changing  the  focus  from  the  policies  of  the  institution  to  the  needs  of  the  students  is  exceptionally  difficult.  While 
the  technology  referred  to  may  have  altered  considerably,  the  words  of  Sewart  (1981  pi 72)  are  no  less  relevant  in 
the  current  circumstances. 

"...there  is  an  overwhelming  tendency  within  the  field  of  teaching  at  a distance  to  offer  systems  from  the 
standpoint  of  the  institution  teaching  at  a distance,  rather  than  from  the  standpoint  of  the  student  learning 
at  a distance.  The  response  to  the  individual  needs  of  the  student  has  often  become  lost  in  the  over-riding 
requirement  to  produce  a grandiose  package  of  materials. " 

It  is  impossible  to  imagine  that  the  many  projects  and  studies  reported  in  the  literature  are  all  genuine  responses  to 
student  needs.  An  examination  of  over  100  successful  applications  for  funding  under  the  National  Teaching 
Development  Projects  (CAUT  1993)  for  example,  revealed  that  even  though  there  were  expected  to  be  improved 
learning  outcomes  in  some,  all  but  three  were  teacher  centered.  The  predominant  focus  was  still  on  the  teaching  or 
institutional  standpoint. 

The  complete  costs  to  the  student  of  the  use  of  these  new  approaches  need  to  be  identified  and  quantified  as 
accurately  as  possible.  Not  all  of  these  may  be  expressed  in  financial  terms.  Additional  time  and  effort, 
inconvenience  and  the  effects  of  frustration  or  poor  system  performance  are  very  real  costs  for  which  it  is  not 
always  possible  to  attach  some  numerical  value  much  less  a dollar  equivalent.  Conversely,  as  Geasler  (1992) 
pointed  out,  the  failure  to  develop  and  use  more  effective  learning  environments  also  represents  a cost,  for  both 
students  and  academics,  in  lost  opportunities. 


Wasted  time 

Many  current  initiatives  involve  the  use  of  the  Internet  without  even  basic  consideration  of  the  cost  of  doing  so  for 
the  student.  Even  common  IT  practices  such  as  estimates  of  system  performance  are  being  ignored  in  the  rush  to  be 
seen  to  be  at  the  cutting  edge.  Receiving  a 100  kiIob>1;e  file  on  an  average  academic  network  may  not  incur  any 
substantial  delays  but  for  a DE  student,  the  whole  process  of  starting  the  computer,  making  the  connection  and 
then  reading  or  printing  the  file  after  transfer  may  add  5 or  10  minutes  to  the  process.  The  time,  effort  and 
financial  cost  incurred  must  be  considered  a burden  that  the  student  has  to  bear.  When  the  network  slows  down, 
the  cost  increases  substantially.  On  economic  grounds  alone,  this  is  not  a good  situation.  The  wasted  time  and  the 
uncertainty  factor  which  increases  anxiety,  makes  it  less  than  acceptable.  What  may  work  exceedingly  well  in 
campus  laboratories  supported  by  high  speed  optic  fibre  networks,  may  turn  out  to  be  a nightmare  for  students 
using  a modem  from  home  at  peak  times.  Some  stories  such  as  that  of  a class  required  to  download  a large  file, 
which  repeatedly  failed  to  download  in  the  connection  time  limit,  are  beginning  to  approach  legend  status.  The 
current  joke  of  WWW  standing  for  World  Wide  Wait  is  sadly  funny  because  of  its  proximity  to  the  truth. 
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Similar  difficulties  may  arise  in  the  use  of  electronic  learning  materials,  regardless  of  the  method  of  distribution. 
A book  has  obvious  advantages  in  that  it  requires  no  additional  hardware,  is  exceptionally  portable,  may  be  quickly 
accessed  and  may  be  used  in  a diverse  range  of  environments.  When  access  sessions  with  electronic  learning 
materials  are  relatively  brief,  the  time  taken  to  start  up  represents  a sizeable  proportion  of  the  time  involved.  On 
the  other  hand,  for  extended  sessions,  it  would  be  negligible.  This  either  changes  the  way  that  students  use  the 
materials  or  disadvantages  those  who  do  not  want  to,  or  are  not  able  to,  adjust  to  such  strategies  of  extended  use. 


Inconvenience 

Models  of  the  teaching-learning  process  such  as  that  proposed  by  Laurillard  (1993)  emphasize  the  discussion  and 
interaction  between  the  teacher  and  the  learner.  Such  models  may  be  appropriate  for  face  to  face  environments  but 
provide  a poorer  fit  for  most  DE  situations.  Attempts  to  synchronize  what  is  basically  an  asynchronous  paradigm 
cause  some  friction.  The  ultimate  challenge  of  future  education  is  to  completely  remove  synchronization  events. 
Such  events  include  fixed  enrolment  dates,  due  dates  for  assessment  items  and  even  expectations  by  the  teachers 
that  students  will  be  "up  to  topic  x"  by  any  particular  date.  This  is  especially  true  if  notions  of  life  long  learning 
and  flexibility  as  espoused  by  many  such  as  Cheng  (1999)  or  Jones  (1997,  1999)  are  also  taken  into  account. 
Education  using  ICT  also  has  a different  set  of  protocols  to  ’traditional’  education.  Many  students  have  yet  to 
master  these  and  the  effort  required  to  cope  may  be  asking  too  much  of  the  student. 

Experience  with  DE  students  (Roberts  1984,  McLachlan  1999)  has  shown  that  most  would  be  characterized  by 
Newbie  and  Clarke  (1986)  as  having  a ’strategic’  approach.  This  is  supported  by  studies  such  as  the  navigation 
behaviours  of  students  using  an  electronic  study  guide  (Messing  1997).  For  such  students,  electronic  learning 
materials  would  have  to  possess  identifiable  advantages  over  traditional  resources  or  they  would  be  rejected. 

What  is  often  overlooked  in  this  discussion  is  the  role  that  the  scale  of  the  operation  plays  in  encouraging  or 
discouraging  learner  engagement.  Van  der  Veen  et  al  (1999)  report  a situation  where  the  numbers  of  contributions 
by  students  actually  discouraged  deep  reading  of  the  contributions  by  fellow  students.  As  with  face  to  face  teaching, 
there  may  be  natural  limits  within  which,  strategies  involving  the  use  of  electronic  resources  may  be  effective  but 
outside  of  which,  they  have  little  chance  of  success.  The  old  cliche  of  "horses  for  courses"  does  not  appear  to  have 
been  recognised  by  many  of  the  grand  plans. 

Research  on  product  outcomes  such  as  higher  performance  has  been  extensive  but  far  from  conclusive  (Russell 
1999)  while  studies  on  the  costs  associated  with  ’learning  access’  are  scarce.  Martens  et  al.  (1997)  describe  learning 
access  as  the  extent  to  which  students  are  able  to  efficiently  achieve  their  learning  goals  by  means  of  the 
affordances  available  in  the  electronic  materials.  The  learning  access  needs  to  be  considered  at  the  level  of  general 
principles  as  well  as  how  these  are  applied  to  each  case  of  electronic  learning  materials. 

Perceived  problems  of  not  being  able  to  find  the  desired  information  or  achieve  desired  learning  outcomes  may  be 
the  product  of  students  not  fully  understanding  the  features  that  are  available  to  them  or  being  aware  of  the 
protocols  required  to  engage  in  the  materials.  Teachers  using  electronic  learning  materials  are  often  closely 
involved  in  their  design  thus  giving  them  a biased  perspective  which  sometimes  fails  to  appreciate  the  difficulties 
of  novices.  Evidence  in  studies  such  as  Messing  (1997,  1999)  illustrates  the  fact  that  students  are  not  always 
completely  conversant  with  the  features  at  their  disposal.  This  is  manifested  in  ’wandering’  behaviour  resulting  in 
wasteful  transitions  from  node  to  node  with  very  short  intervening  times,  often  too  short  for  any  purposeful 
learning.  Such  situations  can  easily  lead  to  frustration  (Kara  and  Kling  1999). 


The  Preference  for  Paper 

The  issue  of  printing  is  one  of  the  most  fundamental  in  the  discussion  of  the  viability  of  electronic  study  materials. 
It  is  common  to  find  (for  example.  Mason  1996;  Bates  1997)  that  the  reduction  of  printed  material  and  its 
associated  costs  is  given  as  one  of  the  principal  justifications  for  the  development  of  electronic  materials.  However, 
that  represents  only  the  institutional  perspective  and  may  simply  transfer  the  cost  of  printing  in  both  financial  and 
production  senses,  from  the  institution  to  the  student.  It  has  also  been  pointed  out  (Mason  1 996)  that  the  student 
printed  material  is  vastly  inferior  in  quality  and  presentation  to  the  standard  that  the  institution  would  normally 
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provide.  Such  reduction  in  standard  is  another  cost  that  the  student  must  bear.  In  circumstances  such  as  those 
described  by  Benson  and  Vincent  (1997)  where  there  was  very  little  functional  difference  between  the  electronic 
and  printed  versions  of  the  materials,  the  result  was  an  overwhelming  student  preference  for  the  printed  versions. 
The  'added  value'  that  electronic  materials  supposedly  provide  needs  to  be  objectively  evaluated  to  determine  if  it  is 
sufficient  compensation  for  the  inconvenience  of  being  electronic. 

Examining  only  the  functionality  of  electronic  materials  would  not  provide  a sufficiently  detailed  and  accurate 
evaluation.  As  well  as  understanding  the  nature  of  the  printing  that  takes  place,  it  is  important  to  understand  why 
it  is  taking  place.  For  DE  students  this  involves  issues  of  learning  style,  access,  portability  and  social  factors  which 
need  to  be  considered  as  part  of  the  evaluation. 

Mason  (1996)  in  a study  of  160  UK  Open  University  students  and  Messing  (1997)  in  a study  of  69  Australian  DE 
students  both  reported  approximately  half  of  the  students  printing  extensive  sections  of  the  electronic  learning 
materials.  Printing  large  numbers  of  pages  requires  the  expenditure  of  considerable  money,  time  and  effort  over  the 
course  of  the  semester.  Other  studies  (Martens  et  al.  1997,  Benson  & Vincent  1997)  reported  students  wanting  to 
have  both  print  and  electronic  versions  of  their  materials.  Given  organizations'  efficiencies  with  print  production, 
it  is  likely  that  they  would  be  able  to  provide  a printed  copy  for  all  the  students  at  a much  lower  unit  cost  than  any 
student's  personal  printing  efforts.  When  only  electronic  materials  are  provided,  transferring  printing 
responsibility  to  the  student  is  a cost  that  needs  to  be  acknowledged. 


Access 

The  question  of  access  to  electronic  learning  materials  often  arises  in  the  context  of  a policy  question  about 
requiring  students  to  own  a computer.  Bates  (1997  p94)  raises  the  equity  issue  that:  "...  a university  requiring  all 
students  to  have  access  to  the  Internet  can  at  an  administrator’s  stroke,  deny  access  to  all  students  who  cannot 
ajford  a computer,  who  are  not  skilled  or  confident  in  using  a computer,  or  who  cannot  get  Internet  access  where 
they  live. " The  issue  of  ownership  of  suitable  hardware  is  often  not  a real  consideration  as  many  students  often 
have  suitable  systems.  Organizations  such  as  the  University  Grants  Committee  (French  1999)  are  very  good  at 
building  infrastructure  and  redressing  inequity  when  it  comes  to  hardware.  Many  universities  have  similar 
programs  to  that  of  Acadia  (MacDougall,  Muldner  and  Tomek  1998)  in  which  they  provide  an  appropriate  level  of 
equipment  to  all  incoming  students.  However,  other  factors  which  could  also  be  termed  access  are  every  bit  as 
important  in  determining  how  any  electronic  learning  materials  are  to  be  used.  In  an  environment  of  electronic 
learning,  the  definition  of  access  will  need  to  be  extended  beyond  traditional  ownership  of  hardware.  Indeed,  for 
distance  education  students,  access  needs  to  consider  issues  of  where  and  when  students  study,  as  well  as  the 
physical  and  logistical  requirements  for  the  use  of  electronic  media. 

DE  students  study  in  quite  an  amazingly  diverse  range  of  circumstances.  Apart  from  the  obvious  environment  of  a 
set  time  in  a dedicated  area  of  the  home,  the  circumstances  in  which  students  in  both  Australia  and  Hong  Kong 
(Messing  1997)  carried  out  their  study  included: 

• at  work  (during  breaks,  before  and  after  work,  on  business  trips) 

• in  transit  (bus,  train,  plane,  car  - presumably  as  a passenger) 

• during  'dead'  times  (waiting  for  children,  airport  and  hotel  lobbies) 

• in  bed 

While  electronic  learning  materials  sometimes  confer  advantages  such  as  searchability,  editability  (particularly 
copy  and  paste  when  the  system  is  not  locked)  and  ease  of  duplication  they  also  severely  restrict  the  physical  access 
that  some  learners  require.  Apart  from  the  lack  of  portability,  limitations  which  have  been  reported  include: 

• not  being  able  to  spread  the  materials  out  on  the  desk 

• not  being  able  to  see  enough  information  on  the  screen  at  a time 

• not  having  multiple  'pages’  open  at  the  same  time  (without  overlap) 

• 'overhead'  time  incurred  in  starting  up 

• being  restricted  to  the  location  of  the  telephone  line  (ICT  applications) 

Screen  size  represents  a cost  in  the  form  of  an  inconvenience.  Not  only  are  large  screens  prohibitively  expensive 
but  their  bulk  would  further  restrict  the  physical  access  in  the  cramped  environments  that  many  students  work  in. 
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Some  such  as  Catenazzi  (1993),  design  systems  for  larger  screens  but  one  has  to  question  whether  such  designs  are 
applicable  to  any  but  the  most  well  resourced  campus,  let  alone  DE  situations. 

The  central  importance  of  the  telephone  line,  especially  in  materials  that  require  the  use  of  communications 
technology,  is  rarely  recognized.  The  many  proposals  for  delivering  electronic  learning  materials  over  the  Internet 
(Harasim  1995;  Western  Governors  Association  1996;  Daniel  1996;  Twigg  1999)  all  rely  on  access  to  a telephone. 
In  most  DE  situations,  the  single  telephone  line  into  the  home  both  determines  the  location  of  the  computer  (and 
hence  the  place  of  study)  as  well  as  the  times  when  it  may  be  used.  While  in  recent  times,  additional  telephone 
lines,  especially  in  the  form  of  mobile  phones,  have  become  more  frequent,  use  of  a telephone  as  part  of  the  study 
process  causes  the  student  to  incur  a cost  in  monetary,  time  and  inconvenience  terms. 


Social  costs 

Garland  (1993)  reported  that  the  student’s  learning  environment  was  a key  situational  variable  in  the  success  of  a 
DE  student.  The  learning  process  was  very  heavily  influenced  by  what  happened  all  around  the  student  as  well  as 
the  multiple  roles  that  the  learner  was  expected  to  fulfil,  Woodley  (1987),  in  examining  the  causes  of  adult  student 
drop-out,  identified  an  extensive  list  of  study  environment  factors  that  adversely  affect  the  learning  process.  They 
included: 

• Little  time  or  energy  given  to  other  domestic,  work  or  leisure  commitments 

• Lack  of  encouragement  by  spouse  or  employer 

• Personal  / domestic  changes 

• Loss  of  a quiet  place  to  study 

• Work  changes 

While  such  influences  are  often  studied,  they  are  seldom,  if  ever,  taken  into  consideration  in  decisions  regarding 
the  design  and  delivery  of  educational  materials.  Studying  in  the  distance  mode  affects  almost  everyone  that  forms 
part  of  the  student’s  social  network.  At  the  very  least  it  is  a disruption  to  the  established  order.  Imposing 
restrictions  by  tying  the  student  to  a computer  creates  added  pressures.  Decreasing  portability  reduces  the  student's 
opportunities  to  participate  in  social  activities  that  are  removed  from  the  computer.  Tensions  created  by  demands 
for  resources  such  as  the  telephone  or  the  only  desk  in  the  home  create  an  extra  level  of  anxiety  about  the  study 
process.  This  is  exacerbated  in  situations  where  such  demands  result  in  additional  sacrifices  by  other  members  of 
the  social  network.  Examples  include  the  relegation  of  children  to  do  home  work  at  the  kitchen  table  or  loss  of 
social  communication  ability  because  the  telephone  line  is  in  use  for  long  periods. 

Such  social  costs  are  ones  that  both  the  student  and  those  affected  must  bear.  The  consequences  of  electronic 
delivery  of  education  extend  beyond  the  learning  process  far  more  than  conventional  technologies.  Increasing 
these  social  costs  reduces  the  likelihood  of  electronic  delivery  being  a judged  a success  by  the  student. 


Conclusion 

Ellul  (1990  p39),  in  analyzing  the  ambivalence  of  technical  progress  in  a technological  society  set  out  four 
propositions  which  he  suggested  applied  in  varying  degrees  to  all  technical  developments.  They  were  that: 

"a//  technical  progress  has  its  price 

at  each  stage  it  raises  more  and  greater  problems  than  it  solves 
its  harmful  effects  are  inseparable  from  its  beneficial  effects 
it  has  a great  number  of  unforeseen  effects.  " 

These  form  an  important  backdrop  for  the  debate  about  the  future  of  electronic  delivery  of  education.  There  is  a 
significant  price  that  students  have  to  pay  for  such  technical  ’progress’.  The  questions  raised  do  not  have  simple 
solutions.  In  the  grand  scheme  of  things  however,  students  are  rarely  asked  if  they  are  prepared,  or  able,  to  pay  that 
price.  It  is  up  to  the  administrators  and  educators  to  consider  all  the  costs  in  a balanced  and  fair  way,  from  not  just 
their  personal  or  institutional  perspective,  but  also  that  of  their  students. 
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Abstract;  This  paper  reports  on  a study  of  student  use  of  various  online  learning  facilities 
using  comprehensive  log  data  that  was  collected  within  Charles  Sturt  University.  Analysis  of 
the  data  verified  that  many  students  were  silent  users  of  the  system  as  well  as  revealing 
some  surprising  results  regarding  features  that  were  not  used  at  all  and  preferred  patterns  of 
access.  These  results  have  implications  for  the  design  of  online  learning  facilities  for  other 
distance  education  ventures. 


Background 

Various  studies  such  as  OECD  (1994),  Jones  (1997,  1999)  predict  that  future  university  students  will  be  drawn 
from  wider  populations  with  more  varied  backgrounds  and  needs.  The  higher  education  system  in  many  parts 
of  the  world  is  changing.  Macknight  (1996)  attributes  most  of  these  changes  to  powerful  economic  forces;  a 
transition  from  the  current  teacher  centred  instructional  model  to  a learner  centred  one  and  the  global  nature  of 
digital  interactions  which  is  changing  the  scale  with  which  the  world  is  viewed. 

Technology  has  for  many  years  been  cited  (Goldsworthy  1980;  Godfrey  1996;  Bates  1997;  Sakamoto  1999)  as 
offering  the  promise  of  greater  learning  effectiveness,  more  learner  centred  approaches  and  a better  quality  of 
interaction.  However,  the  driving  force  for  change  does  not  come  from  some  technological  imperative  that 
many  such  as  Tiffin  (1996)  would  have  us  believe.  The  pressures  for  change  derive,  as  Luke  (1996)  or 
Margolis  (1998)  suggest,  from  political,  social  and  economic  roots.  Even  though  the  causes  are  debatable,  one 
thing  is  certain.  They  have  been  building  for  some  time  as  governments  cut  higher  education  budgets  while 
society  expects  progressively  more  and  more  of  its  school  leavers  to  aspire  to  post  secondary  education.  These 
pressures  are  complicated  by  falling  entry  standards  and  a perception  of  anachronistic  university  practices 
(Jones  1999). 

'Distance  Education'  (DE)  or  'Open  Learning'  as  it  is  sometimes  referred  to,  is  one  of  the  methods  by  which  the 
high  cost  of  face  to  face  teaching  may  be  reduced.  There  are  also  serious  doubts  about  the  intrinsic  value  of 
some  of  the  traditional  face  to  face  methods  such  as  the  lecture,  that  most  universities  continue  to  use  (Ramsden 
1992,  Laurillard  1993,  Ramsden,  Margetson,  Martin  and  Clarke  1995).  The  appropriate  use  of  technology  is  a 
significant  component  in  many  DE  ventures.  Computer  technology  has  been  used  in  education  for  many  years 
but  what  is  surprising,  is  the  relatively  superficial  effect  that  it  has  had.  For  more  than  a quarter  of  a century, 
computers  were  seen  as  an  important  tool  to  be  used  in  the  educational  process.  Society  was  promised  (Suppes 
1968)  the  ultimate  in  individualized  instruction,  an  infinitely  patient  tutor,  a system  that  would  revolutionize 
both  the  educational  processes  at  schools  and  in  the  home.  Bork  (1980),  in  attempting  to  predict  the  future  use 
of  technology  in  education  claimed  that  : 

" - the  computer  as  a learning  device  in  current  classes  is,  compared  with  all  other  learning  modes, 
almost  nonexistent  - the  pace  will  pick  up  rapidly  over  the  next  15  years.  By  the  year  2000  the  major 
way  of  learning  at  all  levels,  and  in  almost  all  subject  areas  will  be  through  the  interactive  use  of 
computers.  p53” 

While  his  predictions  were  clearly  well  short  of  the  mark,  the  widespread  adoption  in  recent  years,  of  the 
Internet  and  the  World  Wide  Web  (WWW)  by  academic  as  well  as  business  and  private  interests  around  the 
world  has  led  to  suggestions  (Daniel  1996)  that  this  is  the  means  by  which  cost  effective  delivery  of  quality 
education  may  be  achieved.  The  current  social  pressures  on  the  education  system  noted  by  Voss  (1995)  make  a 
good  case  for  the  rapid  development  of  educational  delivery  systems  which  are  more  flexible  in  time,  space  and 
the  type  of  technology  used.  The  real  effect  of  such  radical  changes  to  the  role  of  educational  institutions  will 
be  felt  more  by  the  consumers,  the  students,  than  the  educational  institutions  themselves.  Some  such  as  Feather 
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(1995),  fear  that  just  as  printing  divided  societies  on  the  basis  of  literacy,  this  will  lead  to  a further  subdivision 
based  on  wealth,  privilege  and  an  electronic  imperialism. 

The  nature  of  the  electronic  learning  materials  provided  by  educational  institutions  is  only  just  being  addressed. 
While  there  is  literature  available  on  many  aspects  of  Computer  Based  Learning  (CBL),  current  technology 
requires  a re-assessment  of  what  features  and  functions  such  learning  materials  might  contain. 

Charles  Sturt  University  (CSU)  is  one  of  the  major  providers  of  distance  education  in  Australia  with  over 
24,000  students,  three  quarters  of  whom,  study  in  the  DE  mode.  The  Technology  Strategy  Report  (Rebbechi 
and  Barnard  1994)  mapped  out  a comprehensive  set  of  alternatives  for  the  development  of  technology  used  in 
the  teaching  and  administration  of  CSU  students.  One  of  its  key  recommendations  involved  the  conversion  of 
as  many  as  1200  subjects  from  print  into  electronic  form  by  the  year  2000.  What  has  evolved  is  a system  which 
migrates  the  existing  print  based  version  of  the  teaching  materials  into  electronic  form  at  the  same  time  as 
adding  hyperlink  features  that  connect  different  parts  of  the  document  to  other  resources  that  CSU  has  placed 
online.  Fundamental  to  this  decision  to  move  to  electronic  format  for  teaching  materials  was  the  belief  that  in 
doing  so,  students  would  not  be  disadvantaged  and  indeed  the  provision  of  ’added  value’  features  such  as 
hyperlinks,  would  actually  make  this  method  of  delivery  more  attractive  to  students. 

CSU  has  for  various  compelling  reasons  such  as  being  a rural  university;  having  three  main  campuses  spread 
over  400  kilometers  as  well  as  a number  of  smaller  campuses  even  more  remote  from  the  main  ones;  teaching 
students  who  predominantly  study  in  the  DE  mode;  and  having  a significant  presence  in  many  Asian  countries, 
made  a concerted  effort  to  provide  effective  communications  technology.  The  problems  of  isolation  are,  in 
theory,  ameliorated  by  developing  information  technology  strategies  to  compensate. 


Online  support  at  CSU 

Distance  education  has  always  placed  more  of  an  emphasis  on  the  student  taking  charge  of  their  own  learning 
than  the  traditional  face  to  face  mode.  DE  students  are  given  the  necessary  materials  as  well  as  study  guides 
and  whatever  other  help  the  university  deems  appropriate  and  expected  to  take  it  from  there.  The  currently 
popular  term,  ’’Resource  Based  Learning”  is  an  expression  that  encompasses  the  concept  of  students  controlling 
their  learning.  Without  wanting  to  enter  into  the  debate  about  the  meaning  of  resource  based  learning,  it  is 
clear  that  there  are  two  different  types  of  resources  that  are  involved  in  the  process.  For  want  of  a better  set  of 
terminology,  these  are  referred  to  as  ’’static”  and  ’’dynamic”  resources.  Failure  to  differentiate  between  these 
types  and  the  role  that  they  play  is  an  all  too  common  cause  of  misunderstanding  of  the  way  that  electronic 
communications  technologies  might  assist  the  teaching  and  learning  process. 

The  ’’static”  resources  have  traditionally  been  provided  in  print  or  some  other  physical  format  (videotape, 
slides,  audio  cassettes  and  computer  disks).  Note  that  the  most  important  characteristic  of  these  resources  is 
not  the  medium  by  which  they  are  delivered  but  the  nature  of  the  material  itself  A lecture  can  be  static 
whether  it  is  delivered  en  masse  to  600  students,  comes  printed  in  a book  of  lecture  notes,  is  read  as  a word 
processing  file  on  the  student’s  computer  or  is  downloaded  off  the  network  as  an  HTML  document. 

’’Dynamic”  resources  involve  interactions  between  student  - teacher  or  student  - student.  The  main 
characteristic  of  these  is  that  they  are  ad  hoc  and  cannot  be  planned  for  in  advance.  They  include  styles  of ’real- 
time'  delivery  from  the  mass  lecture,  seminars,  tutorials  where  questioning  and  dialogue  are  encouraged  down 
to  one  to  one  conversations  in  person,  on  the  phone  or  via  e-mail.  What  conventional  DE  students  often  report 
is  a lack  of  dynamic  resources.  They  feel  isolated  and  that  the  concepts  of  a class,  fellow  students,  peer  support 
and  general  interaction  are  not  very  strong. 

Online  supported  subjects  at  CSU  consist  of  both  the  static  and  dynamic  resources.  While  there  are  numerous 
exceptions,  static  resources,  in  the  main,  are  material  converted  from  Open  Learning  Institute  (OLI) 
maintained  documents.  This  material  is  organized  as  a subject  site  on  a secure  information  server.  Dynamic 
resources  are  provided  via  e-mail,  list  servers  and  the  centrepiece  of  the  university’s  strategy,  the  subject  forum. 

Figure  1 shows  a typical  subject  page  in  a web  browser.  The  left-hand  frame  serves  as  a navigation  menu  for 
the  documents  that  appear  in  the  right  hand  frame.  On  entry,  the  initial  document  displayed  is  always  the 
overview.  A great  deal  of  redundancy,  seen  as  a positive  factor  in  human-computer  interface  design  guidelines. 
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is  built  into  the  design  so  that  students  may  access  components  from  a number  of  locations.  For  example, 
hyperlinks  to  the  subject  coordinator  may  be  found  in  the  overview,  communications  and  index  pages. 

Most  of  the  links  in  the  menu  are  self-explanatory.  While  some  such  as  ’library’  link  to  the  standard  CSU 
library  page,  others  such  as  communicate,  overview  etc.  are  customized  with  the  information  that  is  specific  to 
that  subject. 


Typical  online  supported  subject  page 
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Figure  1 


The  forum  is  a discussion  area  where  everyone  may  post  questions,  items  of  interest,  general  notes  or  responses 
to  previous  postings.  It  is  typical  of  this  type  of  communication  tool  but  has  the  added  feature  that  forums  are 
restricted  to  students  and  staff  assigned  to  that  subject.  The  postings  are  ’threaded’  and  give  the  user  a 
considerable  amount  of  control  in  managing  their  own  interface  features.  The  list  of  postings  and  any  special 
attributes  of  each  are  displayed.  Again,  a degree  of  redundancy  has  been  built  into  the  interface  to  make  it 
easier  to  use.  For  example,  a user  can  initiate  the  act  of  reading  a message  by  clicking  on  the  hypertext  link,  the 
window  icon  next  to  the  message  or  the  view  all  button.  The  forum  software  was  purpose  built  at  CSU  and  the 
current  version  is  a greatly  enhanced  form  of  what  was  used  prior  to  July  1999.  Even  so,  additions  are  still 
being  made  to  increase  the  functionality  even  further.  The  investment  in  developing  this  system  can  be  viewed 
as  a measure  of  conviction  that  this  was  an  important  dynamic  resource  in  the  delivery  of  the  education  of  the 
future. 


The  study 

Despite  all  the  consultation  that  went  into  the  design  and  the  effort  that  is  involved  in  producing  these  subject 
sites  and  subject  forums,  little  empirical  evidence  had  been  collected  to  date  which  analyzed  how  these  features 
were  being  used.  Answers  to  basic  questions  such  as: 

How  much  time  do  students  spend  using  these  resources? 

What  effect  docs  the  academic  involvement  have  in  the  usage  time  of  these  resources? 

What  is  the  relative  frequency  of  use  of  various  features  in  these  resources? 

What  improvements  may  be  made  to  the  design  of  the  resources? 
are  valuable  from  both  a pedagogical  as  well  as  a system  design  perspective. 

System  log  data  is  collected  for  security  purposes  on  all  actions  that  users  carry  out.  This  collection  of  data 
provided  a previously  untapped  source  of  information.  Based  on  a rough  activity  index  which  was  calculated  by 
dividing  the  number  of ’sessions’  by  the  number  of  students  enrolled  in  the  subject,  three  subjects  were  chosen: 


743 


BEST  COPY  AVAILABLE 


one  with  a high  activity  index  (Sub-A),  one  with  medium  (Sub-B)  and  one  at  the  lower  end  but  which  still 
indicated  a significant  level  of  activity  (Sub-C). 

The  records  from  the  log  files  between  June  30,  1999  and  September  7,  1999  were  extracted  from  the  database 
and  processed  by  custom  written  programs  that  discarded  superfluous  details  and  coded  certain  actions.  Since 
this  time  frame  represented  a significant  proportion  of  the  teaching  time  during  the  semester,  any  trends  in 
usage  of  the  resources  should  have  been  evident  by  that  point.  This  is  even  truer  when  one  considers  the  fact 
that  for  some  of  these  resources,  the  initial  use  might  be  the  only  use  and  one  would  expect  that  to  occur  very 
early  in  the  semester. 

The  important  components  of  the  records  from  the  log  files  include  the  user,  the  date,  time  to  the  nearest 
second  and  the  command  which  was  executed  by  the  system  as  a consequence  of  the  user's  action.  From  these 
records  it  was  possible  to  reconstruct  a particular  user's  interaction  with  the  online  system.  By  comparing  times 
of  successive  events  for  a particular  user  it  was  possible  to  calculate  time  intervals  between  events,  correct  to 
the  nearest  second. 

Inspection  of  the  usage  logs  and  initial  attempts  to  analyze  the  data  revealed  the  presence  of  large  time 
intervals  where  no  action  was  monitored  as  part  of  the  online  system.  Some  students  in  the  initial  tabulation  of 
data  had  average  use  times  that  were  extraordinarily  high,  but  showed  very  little  activity.  In  the  absence  of  any 
direct  observation  notes,  some  procedure  was  necessary  to  account  for  the  possibility  that  the  time  logged  at  the 
computer  was  not  actually  being  used  as  time  on  task.  This  phenomenon  had  been  observed  in  previous  studies 
(Messing  1997)  and  even  in  laboratory  studies  (Messing  1990)  in  which  there  was  a limited  time  for  the 
activity.  When  conducted  in  the  diverse  DE  environments  of  this  study,  it  was  likely  that  there  may  have  been 
tasks  unrelated  to  the  use  of  online  facilities,  which  affected  the  logged  event  time. 

In  order  to  reduce  the  impact  of  such  suspect  times,  all  entries  in  the  log  that  had  a time  difference  from  the 
previously  logged  event  of  15  minutes  or  more  were  assigned  a value  of  15  minutes.  This  was  in  line  with  a 
similar  technique  reported  in  the  literature  (Chen,  Homey  and  Anderson-lnman  1995).  All  documents  in 
either  the  online  subjects  page  or  forum  could  be  read  within  that  time  period,  most  of  them  several  times. 
Students  most  likely  read  what  they  had  to  and  then  performed  some  other  activity  such  as  sending  e-mail, 
opening  a word-processing  document  or  even  leaving  the  online  session  with  the  intention  of  resuming  at  a 
later  time  but  not  actually  quitting  the  system.  The  data  were  adjusted  so  that  these  large,  likely  to  be  inactive 
time  intervals,  were  replaced  as  indicated. 


Results 

Time  on  task 

It  is  quite  reasonable  to  assume  that  there  are  many  factors  which  affect  the  amount  time  that  students  spend  in 
online  related  learning  activities.  The  nature  of  the  subject,  the  level  of  integration  of  the  online  resources  with 
the  teaching  process,  the  teaching  style  used  by  the  academic  as  well  as  a host  of  personal  factors  such  as  the 
level  of  access  to  suitable  equipment,  confidence  in  using  the  technology  or  even  simply  the  time  available  for 
study  are  all  potential  variables  which  affect  usage.  The  influence  of  these  was  not  able  to  be  determined. 
However,  from  a quantitative  perspective,  one  important  factor,  the  impact  of  the  academics'  usage  was  able  to 
be  isolated. 


Usage  time 


Total  time  (hrs) 

Avg  time  (hrs) 

Avg  time 
minutes/week 

Sub-A-all 

247.63 

4.27 

26 

Sub-A-students  only 

239.71 

4.44 

27 

Sub“B-alI 

93.81 

1.71 

11 

Sub-B“Students  only 

90.37 

1.74 

12 

Sub-C-all 

38.04 

0.63 

4 

Sub-C-students  only 

36.53 

0.65 

4 

Table  1 


744 


Table  1 shows  both  the  total  time  on  task  as  defined  above,  as  well  as  the  average  time  on  task  for  each  of  the 
subjects  as  well  as  an  adjusted  value  for  the  number  of  weeks  since  first  usage  was  recorded.  An  additional  set 
of  values  was  determined  by  removing  data  that  was  able  to  be  attributed  to  non-student  entries.  This  included 
log  records  contributed  by  academics  and  other  CSU  personnel.  The  trend  across  the  three  subjects  reflected 
the  original  basis  for  their  selection  in  terms  of  activity  ratio.  Note  however,  that  in  all  three  subjects,  even  in 
Sub-C  which  has  quite  a low  average  usage,  when  the  non-student  use  is  removed,  the  average  increases 
slightly.  The  conclusion  to  be  drawn  from  this  is  that  academic  contributions  do  not  markedly  dominate  the 
overall  usage  in  terms  of  time  on  task.  Indeed,  the  suggestion  is  that  academics  access  these  resources  slightly 
less  than  the  average  student  does. 

Of  course,  the  nature  and  purpose  of  an  academic's  access  is  quite  different  from  that  of  a student  but  there  does 
not  appear  to  be  any  support  in  this  study  for  the  proposition  that  the  academics  dominate  the  usage.  One  might 
even  question  why  the  values  for  academic  usage  are  not  higher  than  they  are. 


Static  versus  Dynamic  Resources 

Reference  was  made  above  to  the  distinction  between  static  and  dynamic  resources.  Since  the  online  material 
represented  a static  resource  and  the  forum  a dynamic  one,  the  data  were  analyzed  on  the  basis  of  the  number 
of  'sessions'  of  each.  A session  being  defined  as  the  existence  of  an  initialization  event  for  either  the  online 
subject  pages  or  the  forum.  This  does  not  represent  'terminal  sessions'  i.e.  how  many  times  the  user  logged  on 
to  the  CSU  site  but  how  many  times  the  user  'switched  into'  the  online  material  or  forum.  The  results  are  shown 
in  Table  2. 


Numbers  of  sessions 


Sub-A 

Sub-B 

Sub;C 

Total  sessions 

710 

436 

330 

Online  only 

48 

16 

25 

Forum  only 

70 

158 

68 

Both 

592 

262 

237 

Table  2 


The  overwhelming  number  of  sessions  involved  access  to  first  the  online  subjects  page  and  then  shortly  after,  to 
the  forum.  This  is  not  surprising  since  the  online  subjects  page  acts  as  a 'gateway'  to  the  CSU  online  resources. 
The  relatively  high  proportion  of  direct  access  to  the  forum  in  the  case  of  Sub-B  is  explained  by  the  use  of 
bookmarks  to  avoid  going  through  the  gateway.  It  appears  that  most  non  Sub-B  students  were  either  unaware  of 
how  to  use  this  feature  or  chose  to  follow  the  normal  entry  method. 


Use  of  Online  Subject  Components 

Analysis  of  the  data  was  carried  out  to  determine  the  number  of  times  various  subsections  of  the  online  pages 
were  accessed  both  from  the  hypertext  menu  in  the  left  hand  frame  (see  Figure  1)  as  well  as  through  links 
embedded  in  the  various  documents.  The  data  were  reported  as  the  number  of  accesses  per  student  but  made  no 
adjustment  for  'transient'  actions  which  are  simply  navigation  actions. 

Access  to  the  overview  page,  both  as  an  introduction  or  a subsequent  selection,  accounted  for  83.5%  of  access. 
No  other  feature  selected  was  more  than  5%  of  all  actions.  Five  features.  Search,  CSU  Services,  Help,  Library 
and  the  Subjects  pages  were  not  accessed  even  once  in  the  total  of  8267  accesses  recorded.  While  this  does  not 
mean  that  students  did  not  use  these  features  from  other  parts  of  the  CSU  site,  it  does  indicate  that  students  did 
not  need  to  access  them  from  the  online  subjects  pages. 

What  is  more  surprising  though  is  the  fact  that  in  most  cases,  less  than  half  of  the  students  engaged  in  a 
deliberate  action  to  use  a particular  feature  even  once.  The  averages  were  calculated  on  the  number  of  actual 
users  rather  than  the  number  of  students  enrolled  in  the  class.  Using  the  number  of  students  in  the  class  would 


have  reduced  the  average  values  even  further.  The  low  average  usage  was  somewhat  surprising  since  one  would 
expect  that  even  from  the  perspective  of  curiosity,  each  student  would  enter  these  pages  at  least  once  to  see 
what  was  there. 

The  most  likely  explanation  lies  in  the  fact  that  a great  deal  of  what  exists  in  the  online  subject  site  is  a 
duplication  of  the  printed  subject  outline  that  students  also  receive.  Indeed,  this  printed  subject  outline  would 
probably  be  read  before  entering  the  online  subject  site.  If  this  were  the  case,  there  would  consequently  be  little 
need  for  students  to  access  many  of  the  sections.  While  verification  of  such  an  explanation  by  more  direct 
means  would  be  necessary  before  reaching  a reliable  conclusion,  it  does  raise  the  question  of  the  merit  of 
providing  the  same  static  material  in  two  media. 


Use  of  Forum  Components 

In  a similar  manner  to  that  of  the  online  subject  pages,  the  access  commands  related  to  the  use  of  forums  were 
tabulated.  The  ’posts’  that  is,  the  creation  of  a new  message  or  a reply  to  an  existing  message,  were  separated 
into  those  performed  by  students  and  those  by  the  subject  coordinator  while  all  other  data  is  combined.  One 
complication  was  that  the  forum  was  designed  with  a number  of  function  buttons  representing  tasks  such  as 
changing  the  way  that  the  forum  index  is  displayed,  setting  topics  to  be  ’watched’  by  sending  e-mail  notification 
of  new  postings  and  a number  of  other  tasks.  These  function  buttons  did  not  generate  a log  file  entiy  and 
consequently  could  not  be  included  in  the  analysis. 


More  than  80%  of  students  in  all  three  subjects,  used  the  forum  at  least  once.  The  most  significant  point  that 
these  data  highlight  is  the  fact  that  by  far  the  most  frequent  operation  is  the  reading  of  posted  messages.  This 
action  comprised  66.3%  of  all  the  actions  performed  and  when  compared  to  postings  (1.7%)  gives  a ratio  of 
almost  40  reads  to  every  post.  This  high  ratio  confirms  the  long  held  suspicion  that  there  are  many  ’lurkers’ 
who  make  use  of  the  forums  but  predominantly  as  a means  of  receiving  information  rather  than  contributing. 
This  suspicion  is  further  confirmed  by  calculating  the  ratio  of  the  number  of  students  who  were  'writers’  and 
’readers'.  These  ratios,  expressed  as  a percentage  for  each  of  the  subjects  in  the  study  were  calculated  as  77.1%, 
1 1.5%  and  16.0%  respectively. 

The  result  for  Sub-A  is  surprisingly  high  but  this  is  able  to  be  explained  by  the  nature  of  the  subject  and  the 
way  that  the  forum  was  integrated  into  the  learning  process.  Students  were  required  to  submit  contributions  for 
assessment  purposes.  The  data  also  reinforced  the  critical  role  that  the  subject  coordinator  has  in  making 
postings.  The  contributions  by  the  subject  coordinator  far  outweighed  that  of  any  other  student.  Even  in  Sub-A 
where  there  was  a very  high  level  of  posting  activity  that  was  student  initiated,  the  subject  coordinator  was 
responsible  for  more  postings  than  the  three  most  prolific  students  combined.  In  the  context  of  overall  low  staff 
usage,  one  can  only  conclude  that  staff,  unlike  students,  often  posted  responses  to  what  they  read. 


Conclusions 

The  three  subjects  chosen  were  not  a random  sample,  nor  were  they  representative  of  the  full  range  of  subjects 
offering  online  support.  If  anything,  they  were  chosen  to  highlight  more  extensive  use  of  the  online  support 
facilities  than  the  average  CSU  subject.  They  each  had  over  50  students  enrolled  in  the  subject  and  made  use  of 
the  online  facilities  in  different  ways  as  part  of  the  teaching/leaming  process.  Given  the  limitations  of  the  way 
that  the  sample  was  selected,  the  following  conclusions  are  suggested  by  the  data. 

* Students  did  make  significant  use  of  the  forums  as  one  avenue  of  receiving  ’dynamic’  resources. 

* The  high  ratio  of  reading  to  writing  in  the  forums  was  confirmation  of  the  belief  that  there  are  many 
more  students  who  benefit  from  the  postings  than  one  would  suspect.  Even  so,  there  were  still 
considerable  numbers  of  students  who  made  no  use  of  the  system  at  all. 

* The  important  role  of  the  subject  coordinator  in  contributing  to  the  material  in  the  forum  was 
verified,  even  in  forums  where  there  was  a very  high  level  of  student  activity. 

* The  access  to  ’static’  resources  which  were  largely  the  same  as  the  printed  subject  outline  material 
was  extremely  limited. 

* Little  or  no  use  was  made  from  within  the  online  subject  pages  of  links  to  standard  CSU  resources 
such  as  the  search  facility  or  access  to  the  library. 
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As  a consequence  of  these,  a number  of  recommendations  relating  to  the  design  of  the  online  subjects  pages 
and  a re-evaluation  of  their  usefulness  were  made.  The  evidence  from  this  study  suggested  that  the  strategy  of 
providing  static  resources  as  a set  of  related  HTML  documents  is  not  viable,  even  with  the  inclusion  of  a 
substantial  number  of  cross-linked  hypertext  links.  Other  avenues  will  need  to  be  explored  that  meet  the  policy 
as  well  as  pedagogical  requirements. 

With  respect  to  the  provision  of  dynamic  resources  as  evidenced  by  the  forums,  there  appears  to  be  qualified 
support  for  the  continuation  of  that  strategy.  However,  the  role  of  the  subject  coordinator  is  critical  if  the  forum 
is  to  be  used  effectively.  Designing  functional  systems  is  clearly  not  sufficient.  All  the  participants  need  to  be 
made  aware  of  the  strategies  available  in  working  more  effectively  with  such  tools. 


References 

Bates,  A.W.  (1997).  The  impact  of  technological  change  on  open  and  distance  learning.  In  Distance  education,  18(1),  93- 
109. 

Bork,  A.  (1980).  Interactive  learning.  In  R.P.  Taylor,  (ed),  The  Computer  in  the  School : Tool,  Tutor,  Tutee , Teacher's 
College  Press:  New  York. 

Chen,  D.,  Homey,  M.A.  and  Anderson-Inman,  L.  (1995).  The  standardized  states  technique:  A cluster  analysis  in  analyzing 
students'  hypertext  processing  patterns.  In  Journal  of  educational  multimedia  and  hypermedia,  4(1 ),  53-73. 

Daniel,  J.  (1996),  New  kids  on  the  box:  Distance  education  enters  its  third  generation.  In  C.  McBeath  and  R.  Atkinson 

(eds).  Proceedings  of  the  3rd  International  Interactive  Multimedia  Symposium,  PROMACO  Conventions:  Perth,  1- 
9. 

Feather,  J.  (1995).  The  information  society:  A study  of  continuity  and  change.  Library  Association  Publishing,  London, 

Goldsworthy,  A.W.  (1980),  Technological  change-  The  impact  of  information  technology,  Australian  Government 
Publishing  Service,  Canberra 

Jones,  G.R.  (1997),  Cyberschools:  An  educational  renaissance,  Jones  Digital  Century  Inc.,  Colorado 

Jones,  G.R.  (1999),  The  human  face  of  electronic  education,  in  F.  Castro,  R.  Lai,  and  M.  Wong,  e-Education:  Challenges 
and  opportunities.  Social  Science  Research  Centre,  The  University  of  Hong  Kong,  83-104. 

Laurillard,  D.M.(I993).  Rethinking  university  teaching,  Routledge:  London 

Luke,  T.  (1996).  The  politics  of  cyberschooling  at  the  virtual  university.  In  G.  Hart  and  J.  Mason,  Symposium  proceedings 
& case  studies  of  the  virtual  university  symposium,  Melbourne  University,  23-47. 

Margolis,  M.  (1998),  Brave  new  universities  in  First  Monday,  3(5), 

(http:www.firstmonday.dk/issues/issye3_5/margolis/index.html) 

Messing,  J.  (1990).  The  use  of  content  and  teaching  strategy  control  features  in  computer  assisted  learning  courseware, 
MEd  thesis,  Charles  Sturt  University. 

Messing,  J.  (1997).  Learning  computing  concepts  from  electronic  books,  PhD  thesis,  Charles  Sturt  University. 

OECD  (1994).  Background  report:  The  international  conference  on  learning  beyond  schooling:  New  forms  of  supply  and 
new  demands.  Centre  for  Educational  Research  and  Innovation,  Organisation  for  Economic  Cooperation  and 
Development:  Paris. 

Ramsden,  P.,  Margetson,  D.,  Martin,  E.  and  Clarke,  S.  (1995).  Recognising  and  rewarding  good  teaching  in  Australian 
higher  education,  Australian  Government  Publishing  Service:  Canberra. 

Rebbechi,  M.  and  Barnard,  I.  (1994),  Technology  Strategy  Report,  Internal  report,  Charles  Sturt  University:  Wagga  Wagga. 

Sakamoto,  T.  (1999),  Educational  reform  by  information  and  communications  technology:  ICT  strategies  for  educational 
improvement-  A Japanese  perspective,  in  In  Proceedings  of  ED-MEDIA-99  World  Conference  on  Educational 
Multimedia,  Hypermedia  amd  Telecommunications,  Association  for  the  Advancement  of  Computing  in  Education, 

2. 

Suppes,  P.  (1968).  Computer  technology  and  the  future  of  education.  In  Phi  Delta  Kappan,  49,  420-423. 

Tiffin,  J.  (1996).  In  search  of  the  virtual  class.  In  G.  Hart  and  J.  Mason,  Symposium  proceedings  & case  studies  of  the 
virtual  university  symposium,  Melbourne  University,  49-54. 

Voss,  L.  (1995).  Technology  supported  learning:  Education  on  the  edge  of  transition.  Keynote  address,  ED-MEDIA-95 

World  Conference  on  Educational  Multimedia  and  Hypermedia,  Association  for  the  Advancement  of  Computing  in 
Education. 


747 


One  teacher,  one  text,  four  lessons:  a case  study  of  collaboration  and 
innovation  in  a multilingual  New  Zealand  classroom. 


Duncan  Meyer 
Long  Bay  College 
Ashley  Ave,  Torbay 
Auckland,  New  Zealand 
duncan@chalksoft.com 

Glenda  Jacobs 
UNITEC 
P Bag  92025 
Auckland 

gjacobs@unitec.ac.nz 


Abstract:  New  Zealand  is  a multi-lingual  society,  and  this  is  reflected  in  classrooms  across  the 
country.  Since  most  instruction  (with  the  exception  of  some  Maori-language  instruction)  is  in 
English,  the  large  Chinese,  Japanese,  and  Korean  (CJK)  student  population  is  placed  at  a distinct 
disadvantage.  Many  of  these  students  arrive  in  the  country  with  the  most  rudimentary  English 
skills,  and  it  is  not  uncommon  for  most  CJK  students  to  fare  poorly  in  English  medium  classes. 

This  is  a case  study  of  collaboration  between  a K-12  school,  a local  university,  professional 
translators,  and  ethnic  community  groups  to  address  the  challenges  faced  by  CJK  students  in  an 
almost  exclusively  English  medium  classroom. 


Some  background 

In  1998,  the  entire  year  12  Computer  Studies  course  as  taught  at  Long  Bay  College  was  rewritten  for  web-based 
delivery.  Students  logged  in  to  the  school  Intranet,  ran  their  Browser,  and  accessed  all  course  materials  online 
(Meyer  & Jacobs,  1998).  Until  this  time,  course  materials  consisted  of  various  text  books,  journal  articles  and  tutor- 
written  notes.  Since  effective  web-based  materials  differ  substantially  from  their  paper-based  counterparts,  the 
materials  could  not  be  adapted  without  being  stilted.  The  opportunity  was  taken  to  rewrite  the  course  completely. 

While  web-based  delivery  proved  extremely  effective  in: 

• accommodating  the  widely  divergent  computer  skills  of  the  120  students  in  the  study 

• encouraging  self-paced  study 

• providing  valuable  insights  into  which  technologies  were  optimal  in  the  K-12  online  context 

the  problems  facing  CJK  (Chinese,  Japanese  & Korean)  students  immersed  into  an  English-medium  class  (which 
required  good  English-reading  skills)  remained  unaddressed,  since  although  the  course  was  extremely  practical  by 
nature,  the  students  were  now  expected  to  read  their  lessons  rather  than  listen  to  them.  This  placed  a heavy  burden 
on  these  students. 

This  problem  is  partially  dealt  with  by  (Faulkner  & Chen,  1999)  in  a study  trialling  dual-medium  course  content. 
While  a step  in  the  right  direction,  their  reproducing  the  entire  course  content  in  an  Asian  language  was  (we  felt) 

1 . too  labour  intensive 

2.  did  not  lend  itself  to  regular  updates 

3.  and  ignored  the  students’  own  expressed  desire  to  become  more  familiar  with  English  as  their  teaching 
medium. 


Our  approach  - a four-tier  system: 

We  opted  instead  for  a four-tier  system,  as  follows: 

1 , Glossary  Overlays 

2,  Tapping  into  the  local  Asian  community  for  online  “mentors”  for  each  student, 

3,  Eliciting  the  aid  of  a local  university  to  host  our  course  on  their  “Web  Shell”  server. 

4,  Using  the  “Web  Discussion”  feature  embedded  in  Office  2000  server  extensions. 


1.  Glossary  Overlays: 

Since  the  original  web-based  course  materials  were  constructed  using  NetObjects  Fusion,  it  was  a simple  matter  to 
identify  key  words  and  concepts  which  appeared  in  the  English  text,  and  to  provide  (on  every  page)  a Chinese, 
Japanese  and  Korean  “button”  which  when  clicked  popped  up  a new  window  which  displayed  explanatory  notes  in 
the  appropriate  language. 

There  is  an  in-built  problem  with  using  this  technique:  if  one  opens  a “child”  window,  and  then  moves  to  another 
page  on  the  website,  that  child  window  simply  becomes  minimized  - it  doesn’t  go  away.  After  a very  short  time, 
one  can  have  a profusion  of  very  confusing  minimized  windows  crowding  up  the  task  bar. 


Plug-ins: 

NetObjects  Fusion  has  the  ability  to  install  “plug-in”  components,  to  extend  the  functionality  of  the  original 
application.  A Fusion  component  (also  available  for  Macromedia's  Dreamweaver)  was  sourced  which  enabled  us  to 
create  Java-based  popup  windows  in  which  contained  a glossary  of  the  terms  used  in  the  main  page.  These  glossary 
windows  are  activated  by  clicking  on  a button  which  appears  on  the  main  page.  Unlike  most  popup  browser 
windows,  however,  they  automatically  close  themselves  when  either  another  language  is  selected  or  the  user  moves 
to  another  web  page.  Without  this  automatic  closing  of  “child”  windows,  this  system  would  soon  have  sunk  under 
its  own  weight,  since  neither  Netscape  nor  Internet  Explorer  do  this  by  default  - leaving  in  their  navigation  trail  an 
ever-growing  number  of  orphaned  popup  help  windows. 


2.  Community  Involvement: 

In  order  to  establish  the  viability  of  the  concept,  use  was  made  of  senior  Asian  students  within  the  school  to  translate 
the  glossaries  for  a single  module.  This  module  was  published  without  comment  to  the  school’s  intranet,  and 
student  reactions  to  it  observed.  Encouraged  by  the  ready  acceptance  of  the  new  feature,  we  approached  members 
of  the  Asian  communities  who  were  eager  to  become  involved  in  the  project. 


Pilot  Testing  and  adjustment: 

To  test  the  effectiveness  and  suitability  of  the  glossaries  (as  well  as  the  user  interface),  we  had  two  Chinese,  two 
Japanese  and  two  Korean  students  work  through  the  course  in  pairs.  The  students  were  asked  to  vocalise  their 
thoughts  as  they  progressed  through  the  material,  so  that  their  comments  could  be  taped,  and  subsequently  played 
back  to  a Language  professional  for  translation.  Armed  with  these  student  comments,  final  editing  was  done,  and 
the  site  published. 

Online  Mentors 

Computer-literate  “mentors”  were  sought  in  the  local  Asian  communities  who  agreed  to  “adopt”  one  or  more 
students,  and  to  follow  their  progress  through  the  online  lessons.  Since  the  course  materials  were  completely  online, 
their  task  was  (1)  to  become  familiar  with  these  materials,  and  (2)  to  be  available  both  to  answer  emails  from  their 
student(s),  and  to  show  regular  interest  in  their  progress. 
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Asian  (CJK)  characters: 


It  was  imperative  that  the  CJK  students  be  able  to  communicate  in  their  own  language  with  their  mentors.  To  this 
end,  all  student  computers,  as  well  as  those  of  the  mentors,  were  configured  to  run  Global  I ME,  allowing  them  to 
type  Asian  characters  directly  into  email  messages.  Most  of  the  CJK  students  were  already  familiar  with  this 
method  of  entering  Asian  characters,  and  this  proved  not  to  be  a problem  at  all. 


Orientation: 

We  arranged  an  initial  orientation  meeting  at  which  the  mentors,  parents,  homestay  parents,  and  members  of  the 
wider  Asian  communities  could  meet  both  the  students  and  the  staff  involved  in  the  project.  This  personal  encounter 
helped  put  a face  to  the  email  communication  which  was  to  be  fundamental  to  the  student/mentor  relationship.  This 
was  followed  up  with  regular  reviews  to  renew  the  personal  contacts.  In  some  cases,  the  mentors  were  known  to  the 
students  already;  however,  these  meetings  “formalized’*  their  new  relationship. 


3.  Using  a “Web  Shell” 

Secondly,  a local  university  agreed  to  host  the  course  on  one  of  their  servers  running  “Courseinfo”,  so  that  all 
administration  could  be  handled  online.  Students  were  thus  able  to  continue  their  lessons  from  home  (if  they  had 
Internet  access). 

Web  shells  offer  many  features  not  available  on  a simple  website  - like  student  drop-boxes  (enabling  students  to 
submit  assignments),  secure  and  private  pages  displaying  each  student’s  marks,  personal  student  web-pages,  online 
chat,  daily  notices,  a course  calendar  and  so  on.  While  these  features  certainly  could  be  written  into  any  website,  the 
time  involved  in  doing  so  would  be  prohibitive.  With  a web  shell,  they  are  almost  “incidental”,  significantly 
enhanced  the  effectiveness  of  the  online  course  delivery. 

One  feature  not  well  implemented  in  “Courseinfo”  was  online  testing.  After  a few  attempts  at  getting  a class  to  do 
tests  online  within  Courseinfo,  we  elected  to  use  computer-based  testing  software  developed  by  ourselves  (Jacobs  & 
Meyer,  1998)  instead.  This  software  is  now  mature,  and  able  to  handle  local  as  well  as  distance  testing  in  a secure, 
robust  and  feature-rich  manner. 

Fortunately,  Courseinfo  allows  a teacher  to  manually  enter  marks  in  the  online  gradebook,  so  that  even  though  tests 
were  conducted  using  our  own  testing  engine,  results  could  still  be  entered  into  the  Courseinfo  system,  where 
students  could  access  them  securely. 


4.  Web  Discussions 

Office  2000  comes  equipped  with  a very  useful  feature  called  “Web  Discussions”.  Once  these  extensions  are 
installed  on  an  NT  Server,  any  IE5  browser  with  access  to  this  Server  is  able  to  add  comments  to  any  web  page 
anywhere  on  the  Internet.  These  comments  appear  to  be  part  of  the  target  web  page,  but  in  reality  exist  as  files  on 
the  NT  Server,  and  are  dynamically  added  to  the  target  web  page  when  viewed  by  IE5. 

Using  this  feature,  our  staff  were  able  to  add  extra  comments  to  our  already  published  web-based  course  materials  as 
the  need  arose  - customizing  these  materials  to  the  needs  of  their  particular  students.  Both  our  students  and  their 
mentors  were  able  to  see  these  comments,  and  to  add  comments  of  their  own  at  any  stage  - creating  instant  context- 
sensitive  discussions  - visible  only  to  those  involved  in  the  project. 

A valuable  feature  of  these  web  discussions  is  that  any  addition  (ie  follow-on  comment)  to  a comment  already  on  the 
web  page  would  trigger  an  email  notification  to  the  author  of  the  comment,  alerting  him/her  to  the  fact  that  someone 
had  responded  to  their  posting. 
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As  an  additional  bonus,  these  impromptu  discussions  became  rich  sources  of  ideas  to  be  incorporated  in  the  next 
version  of  the  published  course  materials. 


Challenges  faced  in  implementing  the  project. 

Workload 

1 . The  year  12  Computer  Studies  course  consists  of  twelve  separate  modules.  This  represented  a considerable 
number  of  individual  web  pages  which  needed  to  be  enhanced  with  three  language-specific  glossaries. 
Each  page  had  to  be  supported  by  three  independent  language  popup  screens.  For  some  of  the  larger 
courses,  this  ran  into  hundreds  of  extra  pages  which  needed  to  be  translated,  and  published. 

2.  In  addition,  to  ensure  that  the  popup  glossaries  displayed  the  Asian  fonts  consistently  across  all  browsers 
and  on  all  computers,  we  elected  to  represent  the  CJK  text  as  a graphic.  This  required  each  CJK  glossary  to 
be  created  in  a word  processor,  and  copy/pasted  into  a graphics  package  before  it  could  be  embedded  into 
the  glossary  pages. 


Language  expertise 

Initially,  we  prepared  a single  course  (Intro  to  Win9x)  using  three  senior  Asian  students  at  the  school.  However, 
since  staff  had  little  way  of  gauging  the  accuracy  of  the  translations,  and  little  or  no  additional  funding  to  enlist  the 
skills  of  independent  experts,  we  resorted  to  a “round-robin”  approach.  We  presented  the  CJK  texts  to  other  CJK 
native  speakers,  and  asked  them  to  translate  them  back  into  English.  We  used  any  marked  discrepancies  identified 
in  this  manner  to  more  accurately  translate  the  original  texts. 

With  this  course  as  a “working  model”,  we  were  able  to  approach  local  interested  parties  for  funding  to  complete  the 
translation  using  qualified  translators. 


Results/Findings 

We  would  love  to  report  that  this  experiment  was  an  unqualified  success.  However,  since  we  were  only  able  to 
translate  three  of  the  twelve  modules  in  the  course,  our  results  are  less  than  conclusive. 


Glossary  Overlays: 

Even  though  only  three  modules  sported  the  CJK  popup  windows,  students  found  this  feature  helpful.  Students 
were  interviewed,  and  asked  if  they  would  like  future  courses  to  continue  to  offer  this  feature.  Of  the  20  students 
interviewed  4 thought  that  the  translations  were  too  abbreviated,  and  should  be  expanded  into  full  translations  of  the 
English  text;  14  considered  the  help  “just  righf’,  and  2 thought  that  although  the  translations  were  a good  idea 
generally,  they  did  not  personally  benefit  from  them,  since  their  English  was  already  of  an  adequate  standard. 


Community  Involvement 

Although  the  communities  met  the  idea  of  mentoring  with  great  enthusiasm  and  interest,  and  although  offers  to 
translate  the  help  pages  were  many,  it  soon  became  apparent  that  relying  on  the  goodwill  of  busy  people  was  not  a 
workable  solution  for  a project  which  was  bound  by  the  inexorable  march  of  the  school  calendar.  Deadlines  came 
and  went,  and  students  progressed  to  modules  for  which  there  was  no  CJK  popup  help  available.  Since  the 
mentoring  program  relied  heavily  on  mentors  being  familiar  with  the  online  course  content,  when  this  familiarity 
was  not  apparent,  the  helpfulness  of  their  replies  was  (as  some  students  confessed)  sometimes  of  little  value.  While 
the  students  were  unequivocal  as  the  value  of  being  able  to  email  a native  speaker  of  their  language  with  problems 


encountered  in  the  course,  one  of  the  singular  challenges  facing  us  next  year,  is  to  identify  outside  mentors  with  the 
time  and  expertise  to  play  this  role  more  effectively. 


Web  Shell 

As  far  as  staff  were  concerned,  the  Web  Shell  was  a solid  success.  It  streamlined  the  administration  of  the  course, 
allowed  them  to  post  regular  notices,  publish  grades  on  secure  student-only  pages,  facilitate  the  submission  of 
student  work,  and  to  return  the  graded  assignments  both  from  home  and  at  school.  Some  of  the  features  provided  by 
Courseinfo  proved  to  be  less  than  useful  in  a high  school  context,  and  were  soon  deactivated.  One  such  feature  was 
the  online  chat  - or  virtual  classroom  feature.  Teachers  rapidly  came  to  the  conclusion  that  16  or  17  year  old 
students  were  simply  too  tempted  by  the  technology,  and  were  too  easily  lured  off-task  into  endless  personal 
messaging. 


Web  Discussions 

Again,  the  temptation  of  students  to  write  fatuous  comments  rather  than  to  either  ask  questions  related  to  the  text,  or 
to  offer  suggestions  germane  to  the  task  at  hand,  became  immediately  apparent.  So  much  so,  in  fact,  that  staff  were 
forced  to  limit  student  participation  in  the  web  discussions  to  read-only.  Several  attempts  were  made  to  re-open  the 
discussions,  but  with  the  same  results.  Even  though  students  knew  that  eveiy  posting  carried  their  login  name,  they 
persisted,  claiming  that  they  never  wrote  the  message,  but  that  someone  must  have  discovered  their  password. 

However,  even  as  a tool  used  by  staff  only,  the  web  discussions  proved  invaluable,  in  that  it  facilitated  the  ongoing 
comment  by  staff  on  the  body  of  the  text  - sometimes  in  amplification,  sometimes  in  alternative  points  of  view. 
These  comments  provided  rich  and  useful  material  for  incorporating  into  future  versions  of  the  course. 


Funding 

One  of  the  rather  unexpected  outcomes  of  our  search  for  sponsors,  was  the  discoveiy  that  schools  enjoy  a low 
profile  in  the  academic  community,  and  as  a result  can  call  on  few,  if  any,  research  grants.  Local  private  enterprise 
saw  little  to  be  gained  by  allocating  funds  for  this  academic  research,  and  the  school  Board  of  Trustees,  while  they 
thought  the  idea  veiy  interesting,  the  fact  remained  that  there  was  little  enough  money  available  for  other,  more 
substantial,  projects.  In  short,  the  project  had  to  be  funded  on  a shoestring,  with  almost  all  of  the  work  being  done 
by  teachers  in  their  free  time.  This  was  a major  reason  for  not  being  able  to  translate  the  entire  course  into  CJK. 


Bibliography: 

Jacobs,  G.A.  (1997)  Test  Authoring  for  Techno  Cynics,  Proceedings , Ed-Media97,  Calgary,  Canada 

Meyer  D.  (1997)  QuizMaster:  A Software  Solution  to  a National  Problem.  Proceedings  , Ed-Media97,  Calgaiy, 
Canada 

Meyer  D.G.  & Jacobs  G.  A.  (1998)  At  Amis  Length:  Distance  Delivery  within  the  Classroom,  Proceedings, 
ICCE98,  Beijing,  China 

Faulkner  E.G.  & Chen  Y.  (1999)  Towards  a Time-Efficient  Methodology  for  Asynchmonous  Mixed- Language 
Presentations,  IEEE  Learning  Technology  Taskforce,  Vol  1 Issue  2,  http://lttfieee.org/leam_tech/  , Oct  '99 

Montgomerie  C et  al  (1999)  Scaling  Web-Based  Instruction,  Proceedings,  Ed-Media99  Seattle  Washington,  USA 

Sakamoto  T (1999)  Educational  Reform  by  Information  and  Communications  Technology:  ICT  Strategies  for 
Educational  Improvement  - A Japanese  Perspective,  Proceedings,  Ed-Media99  Seattle  Washington,  USA 


Australian  National  Training  Authority  (ANTA)  (Feb  1998)  Teaching  and  Learning  Styles  that  Facilitate  Online 
Learning,  (WWW  document)  http://www.tafe.sa.edu.au/isrsc/one/natproj/tal 

Leubke  S.M.  et  al  (1999)  Annotating  the  World-Wide  Web,  ?roce^\ngs,  Ed-Media99  Seattle  Washington,  USA 


753 


Description  of  a web-based  program  suitable  for  easy  adaptation  for  on-line 
delivery  of  many  educational  resources. 


Geoffrey  T.  Meyer 

Department  of  Anatomy  and  Human  Biology 
The  University  of  Western  Australia 
Nedlands,  6907,  Perth,  Western  Australia 
gmeyer@anhb.uwa.edu.au 


Abstract:  This  paper  reports  the  creation  of  a web-based  computer  program  now  used  successfully 
to  teach  "histology"  in  our  Medical  School.  Unlike  any  other  similar  histology  computer  program 
now  available,  the  "template"  of  this  program  can  be  made  available  to  individual  course 
coordinators  in  their  own  institutions  to  create  their  own  "on-line  histology  course"  that  reflects  the 
specific  and  unique  learning  objectives  of  an  educator’s  individual  histology  course.  A further 
objective  of  this  paper  is  to  emphasis  that  the  "program  template"  is  easily  adapted  for  the  "flexible, 
on-line  delivery"  of  many  other  subject  areas  in  High  School  and  University  Curricula. 


The  ’’Histology  Practical  Assistant” 

The  presentation  of  this  paper  will  be  simultaneously  accompanied  by  an  "on-line"  demonstration  of  this 
teaching  resource  but  a website  demonstrating  the  features  of  this  resource  is  also  available  (see  below). 

The  program  is  called  the  Histology  Practical  Assistant"  ("HPA")  and  was  created  initially  to  save  money! 
Teaching  any  histology  course  has  always  been  an  expensive  exercise  due  to: 

(a)  large  numbers  of  staff  required  to  assist  students  in  the  practical  sessions. 

(b)  the  need  for  expensive  teaching  resources  such  as  high  quality  microscopes  and  class  slide  collections  for 
each  student. 

(c)  the  constant  maintenance  of  these  resources  by  staff,  or  the  replacement  of  these  resources  due  to  "wear 
and  tear". 

Increases  in  student  numbers  in  our  histology  courses  over  the  past  few  years  has  meant  that  we  do  not 
have  sufficient  numbers  of  quality  microscopes  or  slide  collections  for  each  student.  We  have  reluctantly  shared 
these  items  amongst  the  students,  but  sharing  is  not  an  ideal  teaching  and  learning  exercise.  So  this  dilemma  was 
another  reason  to  develop  the  "HPA". 

The  ”HPA**  was  also  designed  for  flexible  delivery” 

This  program  was  also  created  to  enhance  the  effectiveness  of  teaching  and  learning.  Many  students  of 
histology  have  difficulty  visualising  a three-dimensional  structure  from  the  study  of  a two-dimensional  tissue 
section.  Images  in  a computer  program  can  be  labeled  to  show  important  histological  features.  Diagrams  can  be 
added  so  students  can  see  the  plane  of  section  of  the  image  through  the  organ,  and  the  location  of  the  image  in 
relation  to  the  whole  organ. 

The  ”HPA”  was  also  designed  for  ” flexible  delivery” 

The  "HPA"  was  also  designed  to  allow  students  to  examine  histological  sections  of  tissues  and  organs 
outside  of  normal  teaching  hours  and  at  their  own  pace  in  a non-threatening  environment.  This  has  reduced  the  need 
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for  large  numbers  of  microscopes  and  slide  collections  (and  reduced  maintenance)  to  be  available  at  any  one  time 
during  the  academic  teaching  week.  It  also  replaces  the  need  for  academic  staffing  in  the  practical  classes  by 
incorporating  into  the  program  the  "tutor"  i.e.  a "computerised  microscope"  which  details  how  to  examine  each 
histological  section  to  identify  the  necessary  histological  features.  This  has  led  to  substantial  savings  in  teaching 
costs.  Another  feature  of  the  "HP A"  is  that  it  provides  images  of  a variety  of  demonstration  material,  limited 
preparations  etc.  that  are  not  in  sufficient  numbers  to  provide  in  slide  collections.  This  no  longer  requires  staff  time 
and  room  space  to  "set  up"  each  week  and  this  material  is  stored  within  the  computer  and  hence  readily  available 
throughout  the  course  for  revision  purposes. 

The  "HPA"  provides  a comprehensive  self-assessment  module.  This  enables  students  to  test  their  skill  in 
identifying  histological  structures,  or  their  knowledge  of  the  theory  of  histology,  by  answering  an  extensive  quiz 
following  each  topic  to  enhance  and  not  compromise  effectiveness  in  teaching  and  learning. 

Why  is  the  ”HPA”  so  unique? 

(a)  It  is  web-based  so  easy  to  maintain. 

(b)  It  is  interactive  in  that  if  students  cannot  identify  features  then  they  can  link  to  extensive  use  of 
pointers/labels  etc.  to  indicate  relevant  histological  components  to  them. 

(c)  It  is  written  in  a language  that  reflects  the  personality  /sense  of  humour  of  the  lecturer  and  so  personalises 
the  interface  between  the  computer  program  and  the  student. 

(d)  It  is  a versatile  program  and  so  can  be  immediately  adapted  to  any  changes  in  teaching  emphasis. 
Additional  topics,  images,  quiz  question  etc.  can  be  inserted  (or  deleted)  very  easily.  This  is  particularly 
useful  so  teaching  Departments  from  other  Universities  can  substitute  their  images  - particularly  in  the 
"tutor"  component  of  this  program. 

Because  of  (d)  above,  anatomical,  plant,  insect  specimens,  etc.  can  replace  the  histological  content  of  this 
program  so  it  can  be  utilised  within  any  University  by  a group  of  educators  from  many  different  disciplines.  It  is 
designed  to  be  "user  friendly".  Students  can  navigate  their  way  through  the  program  very  easily.  An  independent 
survey  has  indicated  that  when  students  have  used  this  program  it  has  made  a very  positive  impact  on  student 
learning. 

The  major  features  of  the  "HPA"  will  be  demonstrated  during  the  presentation  of  this  paper. 

There  is  an  On-line  demonstration  website! 

Most  features  can  be  seen  by  visiting  the  following  website: 
http://www.anhb.uwa.edu.au/hpa/hpademo.html 

The  "Histology  Practical  Assistant"  is  written  in  javascript  and  word  documents.  On  opening  the  program 
a list  including  each  major  topic  on  the  histology  curriculum  appears  within  a narrow  frame  on  the  left  hand  side  of 
an  800X600-pixel  screen  frame  - this  list  of  topics  is  called  the  "labmap".  A welcoming  "splash"  page  also  appears 
on  the  remainder  of  the  screen. 

A "Noticeboard"  is  available  as  a link  to  convey  messages  to  students  about  any  aspect  of  the  study  program. 

The  U*  level  of  the  "labmap""  lists  major  topics  of  a typical  histology  curriculum  in  a medical,  dental,  or  a 
biological  science  course.  The  " "labmap""  begins  with  the  basic  tissues  leading  down  to  the  major  organ  systems. 
Any  topic  can  easily  be  deleted  if  it  doesn’t  suit  the  objectives  (or  time  restraints)  of  a particular  histology  course. 
Conversely,  it  is  easy  to  add  further  topics  to  this  " "labmap"". 

By  clicking  on  the  + icon,  each  topic  on  the  level  of  the  "labmap"  expands  to  include  the  many 
tissues/organ  components  of  that  topic.  Many  components  on  this  2"^^  level,  can  be  further  expanded  to  a 3"*  level  by 
once  again  clicking  on  the  + icon.  Clicking  the  icon  again  can  contract  topics. 


Each  topic  generally  depicts  images  at  the  gross  anatomy  level;  then  low  to  high  magnification,  light 
microscope  images,  and  finally  electronmicrographs  for  relevant  ultrastructural  details.  Thus,  the  computer  is 
imitating  student  use  of  their  microscope  to  examine  the  particular  tissue  or  organ.  This  feature  alone  may  be 
sufficient  for  histology  courses  where  the  constant  use  of  the  microscope,  class  slide  collections,  and  associated 
costs  may  not  be  of  major  emphasis  in  the  course  objectives. 

The  ”HPA”  is  interactive 

Students  need  to  interact  with  the  program!  By  clicking  on  a topic  at  any  level,  an  image  appears  in  the 
middle  frame.  At  the  same  time,  concise  text  (the  ’’guide")  appears  in  narrower  right  hand  frame,  with  all  the  major 
histological  features  hypertexted.  Students  can  work  individually,  or  in  small  groups,  to  identify  each  feature  and 
are  only  tempted  to  click  on  the  hypertexted  feature  if  they  have  doubts  or  wish  to  test  themselves.  Once  the 
student  clicks  on  any  one  hypertexted  feature  it  is  immediately  labeled  and  indicated  by  an  arrow  or  circle,  etc.  The 
concise  text  is  written  in  a language  that  reflects  the  personality/sense  of  humour  of  the  lecturer  and  so  personalises 
the  interface  between  the  computer-aided  teaching  program  and  the  student.  Like  all  components  of  this  program 
the  text  can  be  easily  amended.  At  the  base  of  the  text  for  each  feature  are  two  links;-the  "more  info"  link  and  the 
"tutor"  link. 

The  ’’more  info”  link 

This  link  can  have  a variety  of  uses.  For  instance,  it  can  contain  diagrams,  information  and  questions  for 
the  student  to  research  in  their  recommended  texts,  lecture  notes,  etc.  There  is  no  limit  to  the  length  of  the  material, 
or  method  of  presentation  of  the  material,  inserted  into  this  link.  The  "labmap"  on  the  left-hand  side  remains  visible 
at  all  times  which  enables  the  user  to  open  any  topic  in  any  order.  This  is  particularly  useful  when  students  wish  to 
review  a topic  before  investigating  another  topic. 

The  "tutor”  link 

The  other  link  is  the  "tutor".  The  innovation  of  the  "tutor"  link  has  two  unique  features: 

(1)  The  microscope  objective  lenses  on  the  computer  screen  link  images  (taken  at  different  magnifications)  of 
the  student’s  class  tissue  section  and  so  simulates  the  student's  use  of  a typical  class  microscope.  This 
unique  feature  has  eliminated  use  of  the  microscope  in  most  of  our  practical  classes  except  to  reinforce  the 
skill  of  microscopy  in  the  histological  study  of  body  tissues  and  organs. 

(2)  Accompanying  each  image  is  text  to  direct  the  student  to  locate  (on  high-resolution  images  of  their  class 
slide)  the  specific  histological  feature/s  opened  on  the  "labmap".  This  feature  promotes  an  independent, 
self-paced  learning  opportunity  for  the  student  and  eliminates  the  need  for  teaching  staff  to  be  present  in 
practical  sessions. 

Self-assessment  is  available  on-line 

After  studying  each  topic  a quiz  is  available.  Questions  are  MCQ,  "roll-down"  questions,  identification  of 
structures  on  labeled  diagrams,  or  histological  images  (often  requiring  a written  answer),  or  requests  for  functional 
descriptions  of  tissues/organs.  At  the  completion  of  the  quiz  a student  submits  his/her  answers  and  receives 
immediate  feedback  of  his/her  success.  It  is  easy  to  utilise  a link  that  transfers  the  performances  of  eacli  student  to  a 
secure  result  sheet  where  it  can  be  stored  and  count  towards  the  final  grade  for  each  student. 

A brief  section  on  clinical  considerations  and  research  is  included  following  at  the  end  of  each  topic  of  the 
curriculum.  The  main  purpose  of  these  two  features  is  to  impress  upon  students: 

(1)  the  close  relationship  of  histology  to  pathology/disease  processes. 

(2)  the  variety  of  current  research  investigations  at  the  histological  level  of  many  tissues. 

(3)  the  use  of  histological  techniques  as  research  tools. 


The  program  ’’template”  is  versatile 

In  the  presentation  a demonstration  will  be  performed  explaining  how  versatile  the  "program  template"  is  to 
access  via  any  major  HTML  editor  and  make  amendments  to  the  components  of  the  study  subject.  This  will  include 
a demonstration  of  how  easy  other  subject  disciplines  eg.  botany,  geography,  many  biological  science  subjects 
including  anatomy,  anthropology,  genetics;  geology  and  many  High  School  subjects  can  be  included  in  this 
"Flexible  Delivery"  format  by  a teacher  who  need  not  be  computer  literate! 
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Abstract:  Learning  protocols  are  computational  descriptions  of  collaborative  learning  policies 
(strategies).  They  can  help  geographically  distributed  people  to  overcome  the  difficulty  of 
coordination  by  guiding  and  controlling  social  interactions  in  virtual  learning  environments.  To 
perform  a certain  collaborative  learning  activity,  a number  of  strategies  can  be  used  to  achieve  the 
same  goal.  However,  no  one  strategy  can  be  effective  in  all  situations,  because  learning  groups  are 
different  in  group  size  and  group  stmcture,  and  group  members  vary  in  knowledge,  skills,  interests, 
learning  style,  and  so  on.  Furthermore,  in  learning  processes  the  situation  changes  over  time. 
Learning  groups  may  want  to  shift  from  the  currently  used  strategy  to  another  strategy  to  fit  some 
changes.  In  this  paper,  a mle-based  method  to  switch  learning  protocols  is  proposed.  This  method  is 
implemented  and  demonstrated  in  our  virtual  learning  environment  CROCODILE. 


Introduction 

In  education  a revolution  is  taking  place,  in  which  we  see  a shift  from  the  teacher-centered  approach  to  the  learner- 
centered  approach.  The  underlying  philosophy  is  that  people  learn  best  when  they  actively  engage  in  learning  to 
acquire  knowledge  and  skills,  which  they  need  to  solve  the  problem  at  hand  (Norman96).  The  advance  of 
technologies  accelerates  this  change  and  enables  many  ideas  of  the  philosophy  of  constructivism  to  be  carried  out.  In 
the  past  decade,  many  systems  have  been  developed  to  support  learner-centered  learning  in  virtual  learning 
environments,  such  as  CSILE  (Scardamalia94),  CNB  (Edelson95),  Web-SMILE  (Guzdial97),  and  Belvedere 
(Suthers97). 

In  the  same  line  we  developed  a virtual  learning  environment  (VLE)  based  on  principles  from  constructivism  and 
situation  theory.  We  suggest  that  losing  physical  contact  in  a VLE  makes  it  difficult  for  learners  to  understand  and 
interact  with  learning  environment,  to  construct  shard  knowledge,  and  to  coordinate  their  contribution.  In  order  to 
solve  these  problems,  we  adopted  a development  approach  that  can  be  characterized  as  1)  using  a virtual  institute 
metaphor  to  structure  shared  workspace  (Miao99),  2)  using  learning  nets  to  facilitate  collaborative  construction  of 
shared  knowledge  (MiaoOOa),  3)  using  learning  protocols  to  mediate  social  interaction  (Wessner99,  MiaoOOb),  4) 
using  learning  plan  to  coordinate  tasks  performed  by  different  people. 

In  this  paper,  we  focus  on  discussing  an  open  issue  of  learning  protocols,  A learning  protocol  is  a scripted 
collaborative  learning  strategy  in  computational  form  that  helps  people  to  coordinate  a collaborative  learning 
activity  in  a VLE.  In  different  situations,  learning  groups  may  adopt  different  strategies  to  perform  learning 
activities.  For  example,  to  perform  problem-based  learning  different  process  models  are  proposed  in  the  literature 
(Stepien93).  Therefore,  a set  of  pre-defined  learning  protocols  should  be  available  for  learning  groups,  so  that  they 
can  choose  one  of  them  to  guide  and  control  their  collaborative  learning  activity  in  a VLE  when  needed.  However, 
to  shift  from  the  learning  protocol  currently  used  at  run-time  to  another  learning  protocol  is  an  open  problem.  This 
paper  addresses  this  problem.  First  of  all,  we  introduce  learning  protocols  in  detail.  The  main  part  of  the  paper 
discusses  the  problem  of  shifting  between  learning  protocols  and  presents  a rule-based  method  to  solve  this  problem. 
Then  we  briefly  discussing  the  implementation  of  a protocol  control  tool.  Finally,  we  summarize  our  work  and 
discuss  related  research. 


Learning  Protocols  and  Protocol  Instances 

In  this  section,  we  briefly  present  the  concept  of  learning  protocols,  how  a learning  protocol  is  defined  and  executed. 
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Learning  Protocols 

Learning  protocols  are  theoretically  based  on  psychological  script  theory  (Wessner99).  According  to  the  script 
theory,  generalized  knowledge  about  a sequential  list  of  the  characteristic  events  involved  in  a common  routine  is 
called  a script  (Schank77).  Scripts  can  be  used  to  organize  knowledge,  to  assist  recall,  to  guide  behavior,  to  predict 
likely  happenings  and  to  help  us  to  make  sense  of  our  current  experiences.  Learning  protocols  are  computational 
scripts  to  guide  and  control  social  interaction  in  collaborative  learning  processes  in  a VLE. 

A learning  protocol  is  described  as  an  extended  state-transition-diagram.  A node  in  the  protocol  diagram  represents  a 
state  of  collaboration  within  the  learning  protocol.  In  problem  based  learning  (PBL),  for  instance,  the  possible  states 
are  “identifying  problem”,  “identifying  learning  issue”,  “applying  knowledge”,  etc.  Each  node  is  labeled  by  a unique 
name  within  a learning  protocol.  Each  state  of  collaboration  is  bound  to  a set  of  behavior  rules.  A behavior  rule 
specifies  which  protocol-role  is  permitted  to  perform  which  operation  on  which  type  of  object.  A protocol-role  is 
protocol  specific  (e.g.,  tutor,  learner,  and  expert  in  PBL  protocols).  Protocol-roles  are  assigned  to  participants  of  the 
collaborative  learning  activity.  There  are  two  types  of  objects  in  a learning  protocol.  The  first  type  of  object  are 
artifacts  that  can  be  manipulated  collaboratively  in  learning  processes.  Examples  of  types  of  artifacts  in  PBL  are 
“problem”,  “learning  issue”,  “hypothesis”,  “solution”,  “hint”.  Operations  on  artifacts  such  as  “learning  issue” 
objects,  are  “create”,  “delete”,  “modify”,  “refer__to”,  “I_know”,  etc.  Such  artifact  objects  carry  the  knowledge 
constructed  and  exchanged  in  learning  processes.  The  second  type  of  objects  are  so  called  coordination  objects, 
which  are  used  to  coordinate  group  interaction  (e.g.,  floor  token,  state  of  collaboration,  etc).  Operations  on  such 
objects  include,  for  example,  “request”,  “drop”,  “start”,  “finish”,  and  so  on.  A link  in  a protocol  diagram  represents 
an  event.  The  occurrence  of  an  event  causes  a transition  between  the  departure  state  and  the  arrival  state  connected 
by  the  link.  A link  is  also  bound  to  a behavior  rule  in  which  the  object  is  a coordination  object.  An  event  occurs 
when  an  allowed  operation  is  performed.  Now,  we  formally  describe  a learning  protocol. 

First  of  all,  we  define  the  behavior  rule.  A behavior  rule  can  be  defined  as  a tuple  element  of  B = (R  x Op  x O), 
where  R is  a finite  set  of  protocol-roles;  O is  a finite  set  of  objects;  and  Op  is  a finite  set  of  operations.  Hence,  a set 
of  behavior  rules  can  be  defined  to  specify  all  possible  behaviors  to  support  a certain  collaborative  learning  activity. 
We  use  B*  to  denote  the  set  of  all  subsets  of  set  B.  Then,  we  formally  define  a learning  protocol.  A learning  protocol 
is  defined  as  a tuple  P = < S,  T,  lab,  bind,  L,  B*  > where: 

• S is  a finite  set  of  states  of  collaboration.  Within  S,  there  is  a start  state  denoted  as  that  is  the  access  point  of 
a learning  protocol. 

• TcSxSisa  subset  of  relation  on  S that  is  called  as  transitions. 

• L is  a finite  set  of  names  that  are  represented  as  strings. 

• lab:(SuT)^Lisa  labeling  function.  Notice  that  the  label  of  each  state  of  collaboration  is  unique,  while  a 
label  can  be  carried  by  more  than  one  transition.  A label  of  a state  5 e S is  represented  as  lab(5)  and  a label  of  a 
transition  / g T is  represented  as  lab(0- 

• bind:  (SuT)^B*  is  a binding  function.  The  set  of  behavior  rules  bound  to  a state  5 g S is  represented  as 
bind(5). 

Protocol  Instances 

When  group  members  perform  a collaborative  learning  activity,  they  can  use  a pre-defined  learning  protocol  to 
guide  and  control  the  group  interaction.  A learning  protocol  can  be  executed  as  a protocol  instance.  One  learning 
protocol  can  have  more  than  one  protocol  instance  at  the  same  time.  Each  protocol  instance  is  executed 
independently  following  the  definition  of  the  chosen  learning  protocol. 

Before  or  during  execution  of  a protocol  instance,  the  initiator  of  the  protocol  instance  needs  to  assign  protocol-roles 
to  the  potential  participants  of  the  collaborative  learning  activity.  When  the  execution  of  a protocol  instance  is 
initiated,  it  will  begin  execution  at  the  start  state  of  the  learning  protocol.  That  is,  the  current  state  of  the  instance  is 
the  start  state.  When  participants  with  appropriate  protocol-roles  perform  an  operation  that  is  bound  to  a transition, 
the  corresponding  event  occurs,  which  triggers  a state  transition  according  to  the  definition  of  learning  protocol.  The 
destination  state  of  the  transition  then  becomes  the  current  state  of  the  protocol  instance.  In  the  current  state, 
participants  are  guided  and  controlled  to  perform  operations  on  the  shared  artifact  according  to  the  behavior  rules 


bound  to  this  state.  In  this  way,  step  by  step,  all  participants  carry  out  their  collaborative  learning  activities 
coordinated  by  following  the  learning  protocol. 


Shifting  between  Learning  Protocols 

As  discussed  above,  group  interaction  is  supported  at  run  time  by  the  execution  of  a protocol  instance  of  a pre- 
defined learning  protocol.  However,  to  support  a certain  collaborative  learning  activity  (e.g.,  problem  based 
learning),  a family  of  learning  protocols  can  be  defined.  As  we  known,  learning  groups  are  different  in  group  size 
and  group  structure,  and  group  members  vaiy  in  knowledge,  skills,  interests,  learning  style,  and  so  on.  Different 
learning  groups  may  exploit  different  learning  protocols  to  carry  out  their  collaborative  learning  activity,  because  of 
the  characteristics  of  the  groups  or  their  customs.  There  is  no  single  learning  protocol  that  is  suitable  to  eveiy  type  of 
group  and  to  eveiy  problem.  Furthermore,  as  groups  conduct  a collaborative  learning  activity,  some  factors  may 
change  over  time  such  as  group  size,  structure  of  group,  the  understanding  and  expertise  of  learners,  etc.  Therefore, 
groups  may  want  to  shift  from  the  currently  used  learning  protocol  to  another  learning  protocol  to  fit  the  changes. 

A simple  approach  to  shift  between  protocols  is  to  terminate  the  currently  executed  protocol  instance  first  and  then 
to  initiate  a new  protocol  instance  based  on  the  target  learning  protocol.  However,  in  such  a way,  some  data 
recorded  in  the  current  protocol  instance  will  be  lost  upon  terminating  the  current  protocol  instance.  The  participants 
then  have  to  repeat  efforts  to  reach  the  equivalent  state  in  the  new  protocol  instance.  For  example,  when  a group 
uses  the  “first-request-first-take”  floor  token  control  protocol,  the  learners’  requests  for  the  floor  are  recorded  in  the 
request  queue  of  the  current  protocol  instance.  If  the  group  wants  to  change  the  floor  token  control  protocol  by 
stopping  the  current  protocol  instance  and  initiating  a new  protocol  instance  of  another  floor  token  control  protocol 
(e.g.,  the  “teacher- assigning”  floor  token  control  protocol),  the  request  queue  of  the  current  protocol  instance  will  be 
lost.  Learners  need  to  request  the  new  floor  token  in  the  new  protocol  instance  again. 

An  alternative  approach  is  to  enable  end-users  to  modify  the  definition  of  the  learning  protocol  in  use  at  run  time. 
However,  our  experiences  reveal  that  it  is  a difficult  and  time-consume  task  for  end-users  to  change  learning 
protocol  dynamically. 

Therefore,  we  adopted  an  approach  where  end-users  are  enabled  to  shift  between  learning  protocols  by  simply 
choosing  another  protocol  from  a family  of  pre-defined  learning  protocols  for  a certain  learning  activity.  Because  all 
learning  protocols  in  a family  are  carefully  defined  to  support  the  same  collaborative  learning  activity,  they  use 
identical  sets  of  objects,  such  as  the  types  of  objects  and  their  associated  operations,  and  identical  protocol-roles. 
Learning  protocols  within  the  same  family  vaiy  in  the  definition  of  the  state  of  collaboration  and  transitions,  and  in 
the  binding  relation.  An  analysis  of  learning  protocols  in  the  same  family  showed  that  some  states  of  collaboration 
defined  in  different  learning  protocols  are  bound  to  the  same  set  of  behavior  rules,  and  that  some  states  of 
collaboration  defined  in  different  learning  protocols  are  refined  to  different  degrees.  Manually,  for  any  state  of  any 
learning  protocol  from  which  to  shift  we  can  find  an  appropriate  state  in  any  target  learning  protocol  from  where  to 
continue  execution  after  shifting.  Therefore,  a family  of  learning  protocols  can  be  defined  by  a protocol  diagram  in 
which  all  learning  protocols  are  contained  as  sub-diagrams.  A new  shift  relation  is  defined  by  edges  that  connect 
each  state  in  any  sub-diagram  to  a state  in  any  other  sub-diagram.  This  diagram  becomes  veiy  complex  when  the 
number  of  learning  protocols  and  the  numbers  of  states  in  these  protocols  are  large.  It  is  also  difficult  to  maintain  the 
diagram  when  adding  or  removing  or  modifying  learning  protocols. 


A Rule-based  Method  to  Shift  between  Learning  Protocols 

In  order  to  shift  between  learning  protocols  in  a systematic  manner,  we  defined  a rule-based  method. 

Readers  should  remember  that  a state  within  a learning  protocol  is  distinguished  from  other  states,  because  a certain 
sub  set  of  behavior  rules  is  bound  to  this  state.  Within  a learning  protocol,  two  different  states  are  bound  to  two 
different  sub  sets  of  behavior  rules.  Given  there  are  two  states  s j and  $2  (s^  ^^2)ina  learning  protocol  P,  then 

(bind(.yj ) = bind(^2  ))  equivalent  to  true. 
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Different  learning  protocols  are  defined  to  support  the  same  collaborative  learning  activity.  That  is,  the  set  of 
protocol -roles,  objects  and  their  associated  operations  are  the  same.  Learning  protocols  are  different  from  each  other 
because  the  states  in  different  protocols  are  defined  in  different  way.  That  is,  in  different  learning  protocols,  states 
are  refined  or  specialized  in  different  degree  by  binding  to  different  sub  sets  of  behavior  rules.  According  to  the  sub 
set  of  behavior  rules  bound  to  states,  the  relation  between  two  states  within  two  different  learning  protocols  can  be 
fall  into  one  of  four  categories.  Given  5 is  a state  of  a learning  protocol  P and  5'  is  a state  of  a learning  protocol  P’  (P 
^ P’),  the  relation  between  bind(5)  and  bind(^ ')  leads  to  one  of  the  situations: 
equal:  bind(^)  = bind(^ '); 

contained:  bind(^)  c:  bind(5')  v bind(5)  3 bind(5'); 

intersected:  bind(5)  bind(^ ')  a bind(5')  (X  bind(^)  a (bind(5)  n bind(5')  0); 
separated:  bind(^)  n bind(^ ')  = 0. 

According  to  these  four  situations  we  can  establish  a shift  relation  between  labels  of  states  instead  of  establishing  a 
shift  relation  between  states  of  two  learning  protocols.  For  all  learning  protocols  in  the  same  family,  we  define  a 
unified  set  of  labels  and  a unique  labeling  function  that  is  denoted  as  Lab.  In  the  first  case,  we  label  two  states  by  the 
same  name.  In  the  rest  of  three  cases,  we  have  to  use  different  names  for  these  two  states. 

Generally  speaking,  given  there  are  m learning  protocols  in  a family  Pj,P2,..P^  , while  Sj,  S2,  ...  S^, 

denote  the  sets  of  state  of  collaboration  of  the  respective  learning  protocols.  s*  e Sj  bind(5)  = 

bind(^ ')  iff  Lab(^)  = Lab(^ '),  that  is,  these  two  states  will  be  labeled  by  the  same  name.  Otherwise,  if  bind(5)  ^ 
bind(^ '),  then  Lab(^)  ^ Lab(^  'j.  Now,  we  define  a label  relation  graph  as  a directed  graph  G = < L,  U,  B > where: 

L is  a finite  set  of  nodes  that  represent  the  unified  set  of  labels  of  states  for  all  protocols  in  the  family.  U c ( L x L ) 
is  a set  of  uni-directional  edges  U ={(/,/')  | / = Lab(5)  a / ' = Lab(5')  a bind(5)  3 bind(5 ')  }.  If  (/,  /')  ^ U,  then  / is 
called  a father  of/',  while  /'  is  called  a child  of  /.  B c ( L x L ) is  a set  of  bi-directional  edges  B c {(/,/')  | / = 
Lab(^)  A /'  = Lab(^')  a bind(^)  (X  bind(^')  a bind(5')  <x  bind(^)  a (bind(5)  n bind(5')  ^ 0)}.  It  is  important  to  notice 
that  not  all  intersected  states  can  lead  to  drawing  such  bi-directional  edges  between  their  labels.  In  some  cases,  the 
behavior  rules  bound  to  two  states  are  almost  the  same  but  they  have  subtle  difference  because  they  are  defined  to 
emphasize  or  ignore  intentionally  some  details  more  or  less.  They  can  be  regarded  as  mutually  shiftable  states  and  a 
bi-directional  edge  can  be  defined  between  the  labels.  In  other  cases,  no  shift  should  be  possible  between  two  states 
although  the  behavior  rules  bound  to  the  two  states  are  associated  to  more  or  less  common  behavior  rules.  Thus,  the 
bi-directional  edges  need  to  be  designed  carefully  by  protocol  designers. 

After  defining  the  label  relation  graph,  we  can  shift  between  protocols  by  using  the  algorithm  described  below.  Let  P 
be  the  currently  used  protocol  and  s be  the  current  state  {s  e S)  of  a protocol  instance  of  P.  If  it  is  desired  to  shift 
from  the  current  learning  protocol  to  another  learning  protocol  P’  (P  ^ P’)  we  need  to  find  a state  in  the  new 
protocol.  This  state  .y'  g S’  can  replace  the  current  state  in  the  new  protocol  while  maintaining  the  current  protocol 
instance  by  using  the  following  algorithm: 

1 . Try  to  find  a state  in  S’ : If  3 5 ' g S’  | Ldb{s)  = Lab(^ '),  then  return  s ' 

2.  Try  to  find  a state  in  S’:  If  3 5'  g S’  j (Lab(5),  Lab(^'))  g B,  then  return 

3.  Try  to  find  a state  in  the  set  of  children  of  Lab(^):  C = {c  | c g La  ((Lab(5),  c)  g U} 

• lf3!cG  CI^'g  S’ac  = Lab(5 '),  then  return  s ' 

® I f there  are  k labels  c,.  (i  = 1, 2, k)  ((c.  =Lab(^,-  ')a.s,-  'g  S’)  (i  = 1,  2,  ...,  k)),  search  a path  in 

',  ^2  --.s 'are points. 

• If  the  path  is  not  a loop  and  s ' is  the  start-point  of  the  path,  then  return  s '; 

• If  the  path  is  a loop  and  .y ' is  the  nearest  point  to  the  start  state  of  the  protocol  P’,  then  return  s ' 

• If  there  is  no  such  c e C|^'g  S’ac  = Lab(5'),  then  recursively  search  children  of  elements  of 
repeat  step  3. 

4.  Try  to  find  a state  in  the  set  of  fathers  of  Lab(^):  F = {f\f^  L a(/,  (Lab(^))  g U} 

• If3!/GF|5'GS’A/=  Lab(^ '),  then  return  s ' 


which 


C and 
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• If  there  are  k labels /.  (i  = 1, 2,  k)  ((/'.  = Lab(^  . ) a ^ ' e S’)  (i  = 1,  2,  k)),  search  a path  in  which 

.y  j ^2  k ' points. 

• If  the  path  is  not  a loop  and  s ' is  the  start-point  of  the  path,  then  return  s 

• If  the  path  is  a loop  and  ^ ’ is  the  nearest  point  to  the  start  state  of  the  protocol  P’,  then  return  s ’ 

• If  there  is  no  such  f e F | e S’  a /=  Lab(^  % then  recursively  search  fathers  of  elements  of  F and  repeat 
step  4. 

5.  Return  the  start  state  of  learning  protocol  P’  (i.e.  no  better  matching  state  in  the  target  protocol  could  be  found). 


Implementation 

The  rule-based  method  to  shift  between  learning  protocols  has  been  implemented  in  a protocol  control  tool  in  our 
prototype  VLE  system  called  CROCODILE  (an  abbreviation  for  CReative  Open  Cooperative  Distributed  Learning 
Environment).  It  is  implemented  in  Visual  Works  Smalltalk  and  available  on  Window’ 95,  ’98,  NT  and  Solaris. 


In  Fig.  1,  we  illustrate  the  three  windows  that  users  see.  The  lower  window  is  the  hypermedia  document  editor  in 
which  learners  can  build  their  hyperdocument.  The  upper  left  window  shows  a virtual  room  in  which  learners  are 
located.  The  upper  right  hand  window  shows  the  protocol  control  panel. 

A learner  can  initiate  a learning  protocol  instance  by  clicking  on  the  “protocol”  icon  (see  Fig.  1)  in  the  editor 
window.  A dialog  window  will  pop  up  and  the  protocol  initiator  can  select  a learning  protocol  from  a list  of  pre- 
defined learning  protocols.  In  the  current  prototype  we  prepared  three  learning  protocols  for  problem  based  learning 
(PBL-protocols).  After  choosing  a PBL-protocol,  users  currently  working  in  the  editor  will  see  that  the  protocol 
control  panel  pops  up.  Information  about  the  initiated  protocol  instance  is  presented  in  the  panel,  such  as  the  name  of 
selected  learning  protocol,  current  state  of  the  protocol  instance,  and  assigned  role  membership  (which  is  assigned 
by  using  this  panel).  Learning  groups  can  construct  shared  knowledge  in  the  editor  window  by  following  the 
selected  PBL-protocol.  A detailed  description  of  how  a learning  group  conducts  problem-based  learning  by  using 
our  system  can  be  found  in  (MiaoOOb).  It  is  important  to  notice  that  learning  group  can  shift  between  PBL-protocols 
by  clicking  on  the  “shift”  button  in  the  protocol  control  panel  and  choosing  another  PBL-protocol. 
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Summary 


In  this  paper,  we  discussed  the  concept  of  learning  protocols,  their  definition,  and  execution.  Then,  we  identified  the 
problem  of  shifting  between  learning  protocols.  After  analyzing  and  exploring  possible  approaches  to  solve  this 
problem,  we  presented  a rule-based  method  to  shift  between  learning  protocols.  Finally,  we  briefly  described  our 
virtual  learning  environment  CROCODILE,  in  which  the  method  has  been  implemented  in  a protocol  control  tool. 

In  terms  of  related  work.  Belvedere  and  Web-SMILE  provide  students  with  guidance  for  the  execution  of  learning 
processes.  The  guidance  is  implemented  as  something  like  a protocol  description.  However,  in  both  of  these  systems 
this  protocol  description  can  not  be  instantiated  and  shifted.  In  the  Workflow  Management  Systems  area,  process 
models  are  different  and  cannot  easily  be  used  for  our  purpose  of  controlling  behavior  in  collaborative  learning 
processes.  Also,  in  WfMS  usually  the  definition  of  a process  is  changed  instead  of  replacing  the  process  definition 
of  a running  process.  Compared  to  finite  state  machines  our  process  model  differs  with  respect  to  the  use  of  behavior 
rules  in  the  process  definition.  Exactly  these  behavior  rules  form  the  basis  for  our  algorithmic  computation  of 
transitions  between  protocols. 

Our  plans  for  future  work  include  extensive  evaluation  of  our  approach.  Furthermore,  we  will  use  this  method  to 
develop  other  families  of  learning  protocols  to  support  other  types  of  collaborative  learning  activities. 
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Abstract:  Based  on  the  findings  of  this  study,  it  can  be  claimed  that  the  process  of  acceptance  of 
computer  technology  is  related  to  gender  where  the  women  are  far  more  cautious  about  exhibiting  a 
positive  attitude  than  men.  However,  when  the  functionality  of  a technology  has  been  demonstrated  within 
the  context  of  a specific  process,  in  this  case  the  'learning  process,'  women  tend  to  embrace  the  technology 
and  are  often  more  positively  pre-disposed.  Furthermore,  there  are  no  specific  categories  of  use  where  this 
process  appears  to  become  more  significant,  and  the  differences  between  men  and  women  with  regards  to 
computer  use  categories  is  unevenly  and  somewhat  randomly  distributed  suggesting  again  that  it  is  the 
practice  of  technology  that  is  more  critical  than  the  specific  technology  itself 


Introduction 

Early  studies  reveal  that  males  are  less  anxious,  more  confident,  and  like  using  computers  more  than 
females.  Despite  these  findings  researchers  also  conclude  the  opposite;  gender  does  not  directly  impact  a student’s 
attitude  toward  computers  (Rosen,  Sears,  & Weil,  1985).  These  findings  are  a result  of  taking  into  account  the 
variable  ‘computer  experience.’  Chen  (1985)  concludes  that  overall  males  show  more  positive  attitudes  of  interest 
and  confidence  with  computers  than  do  females.  However,  when  the  amount  of  computer  experience  is  controlled, 
males  and  females  respond  with  similar  levels  of  interest.  Hunt  & Bohlin  (1993)  reported  on  a research  study 
designed  to  measure  the  entry  attitudes  of  students  enrolling  in  educational  computing  classes  and  found  that 
previous  experiences  were  the  best  predictors  of  student  attitudes.  Males  indicated  having  more  experience  with 
programming  and  recreational  uses  of  computers  but  gender  did  not  significantly  correlate  with  any  of  the  attitude 
variables.  Levin  and  Gordon  (1989)  confirm  the  impact  of  computer  experience  on  attitudes  and  add  that  having  an 
actual  computer  in  the  home  has  a greater  affect  on  attitude  toward  computers  than  does  gender. 

Only  when  the  trend  that  males  and  females  exhibit  similar  attitudes  toward  computers  was  recognized  that 
researchers  began  asking  if  these  results  were  the  same  for  all  types  of  computer  use.  C.R.  Scott  & Rockwell  (1997) 
explored  the  role  of  computer  anxiety,  communication  apprehension,  and  writing  apprehension  in  predicting  future 
use  of  computers  as  communication  tools.  They  found  that  past  computer  use  appears  to  be  a strong  predictor  of 
future  use;  however,  gender  differences  are  rare:  male  respondents  reported  a greater  likelihood  of  joining  online 
services  and  playing  video  games.  These  two  instances  were  exceptions;  there  were  no  other  differences  between 
the  genders  regarding  likelihood  to  use  new  (or  more  traditional)  technologies.  Rockwell’s  research  points  to  a 
fundamental  distinction  not  made  in  previous  research  — the  difference  in  types  of  computer  use  by  gender.  More 
importantly,  this  information  can  be  used  to  determine  if  there  is  a difference  in  attitude  toward  computers  because 
of  specific  use  patterns  between  genders. 

As  this  brief  review  suggests  there  is  some  confusion  about  the  relationship  between  gender  and 
contemporary  attitudes  and  use  of  computers.  In  the  educational  setting,  this  confusion  leads  to  doubts  about  the 
way  in  which  differences  in  learning  in  a computer-enriched  environment  could  be  related  to  gender.  We  argue  that 
there  are  two  areas  that  need  further  exploration:  1)  there  is  an  alteration  in  the  relation  between  gender  and  different 
kinds  of  the  computer  use  as  computer  enriched  environments  are  emerging,  and  2)  the  emergent  environments  and 
the  cultures  that  these  environments  spawn  can  lead  to  new  areas  of  differences  between  genders  when  computers 
are  considered  as  the  key  technology  of  the  new  academic  environment.  These  considerations  drive  us  to  advance 
one  broad  research  question: 

What  are  the  relationships  between  gender,  categories  of  computer  use,  and  attitudes  towards  computers  in  the 
emergent  computer  enriched  environments  of  academic  institutions? 


Methodology  and  Data 


Given  the  breadth  of  the  question,  we  have  utilized  the  data  from  an  ongoing  longitudinal  assessment  of  the 
impact  of  computerization  at  a small  liberal  arts  institution  that  initiated  a process  of  computer  enrichment  by 
entering  into  collaboration  with  a major  computer  industry  leader.  The  study  used  a survey  design  to  collect  self- 
response data  from  a sample  of  students  and  faculty  at  the  University,  The  surveys  continue  on  an  annual  basis  and 
use  questionnaires  that  were  developed  after  extensive  focus  group  interviews  with  students  and  faculties. 

Results 

The  results  of  the  analysis  point  towards  several  inter-related  findings.  The  results  are  presented  here  in  a 
thematic  order  to  capture  the  various  relationships  between  gender,  use  and  attitudes  towards  computers. 

First,  the  results  indicate  that  before  the  computer  enrichment  was  initiated  on  campus,  and  before  the 
arrival  of  the  laptop  students  on  campus,  the  men  on  campus  had  a significantly  more  positive  attitude  towards  the 
use  of  computers  and  were  also  more  frequent  users  of  computers.  As  illustrated  in  Table  1,  men  reported  that  they 
felt  that  computers  can  make  the  learning  process  easier,  preferred  to  be  in  classes  where  they  could  use  computers, 
felt  more  comfortable  with  computers  than  women,  were  less  apprehensive  about  computers  than  women  and  were 
less  concerned  that  computers  in  teaching  would  make  the  teaching  process  too  impersonal.  Finally,  men  reported 
that  computers  enabled  them  to  interact  more  with  teachers  using  computers. 


Table  1:  Year  1 comparison 


Attitude  Item 

All 

Women 

All  Men 

Increased  use  of  technology  makes  learning  easier 

3.39 

3,56 

* 

1 prefer  classes  in  which  1 get  to  use  computers 

2.72 

3,05 

* 

I feel  comfortable  using  the  computer 

3.38 

3.78 

* 

I have  a certain  apprehension  about  computer  use 

3.03 

2,55 

* 

Computers  in  teaching  makes  the  learning  process  too  impersonal 

3.25 

3,07 

* 

Computer  use  increases  the  college  work  load 

3.08 

3.06 

Computers  are  effective  for  communicating  with  faculty 

3.33 

3.31 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3,38 

3.32 

Computers  enable  me  to  interact  more  with  professors 

2.94 

3.07 

* 

In  a similar  way,  men  on  campus  also  reported  a more  frequent  use  of  computers  than  women.  Computer 
use  was  reported  as  higher  by  men  in  all  categories  of  computer  use,  particularly  in  the  task  categories  of  use 
ranging  from  specialized  use  to  the  more  routine  use  of  the  technology.  As  illustrated  in  Table  2,  this  difference 
disappeared  only  in  the  case  of  the  use  of  electronic  mail  with  teachers. 


Table  2:  Year  1 comparison 


Use 

All  W 

AllM 

Spreadsheet 

1,50 

1,71 

* 

Database 

1,45 

1,69 

* 

Statistical 

1,44 

1,65 

* 

Mathematical 

1.56 

1,78 

* 

Internet  for  Info 

2,71 

2,97 

* 

E-mail  with  teachers 

2,11 

2,20 

This  trend,  continues  in  the  first  year  of  computer  enrichment.  At  this  point,  a quarter  of  the  students  in  the 
University  had  access  to  personal  laptop  computers,  the  network  system,  albeit  somewhat  unreliable,  was  in  place 
and  the  computer  labs  were  being  phased  out.  The  data  suggests  that  both  the  legacy  and  the  laptop-program 
women  became  less  positive  towards  computers  and  their  use  was  less  than  the  men  (Table  3 and  4)  were. 


However,  at  the  same  time  the  overall  attitude  towards  computers  among  both  men  and  women  saw  an  increase  in 
the  positive  direction  (Tables  5 and  6),  Furthermore,  there  were  more  significant  differences  in  attitude  between  the 
legacy  women  and  the  legacy  men  than  the  laptop  women  and  the  laptop  men  (Tables  7 and  8), 


Table  3:  Year  2 comparison  (aggregate) 


Men 

Women 

Increased  use  of  technology  makes  learning  easier 

3,48 

3.25 

* 

I prefer  classes  in  which  1 get  to  use  computers 

3,13 

2.75 

* 

I feel  comfortable  using  the  computer 

4.04 

3.66 

I have  a certain  apprehension  about  computer  use 

2.16 

2.56 

Computers  in  teaching  makes  the  learning  process  too  impersonal 

2.78 

3 

Af 

Computer  use  increases  the  college  work  load 

2.92 

3,01 

Computers  are  effective  for  communicating  with  faculty 

3,57 

3,68 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3.57 

3,74 

Computers  enable  me  to  interact  more  with  professors 

3.41 

3,35 

Table  4:  Year  2 comparison  (aggregate) 


AllM 

All  W 

Spreadsheet 

1,95 

1,74 

Af 

Database 

1,74 

1,52 

Af 

Statistical 

1,81 

1.76 

Mathematical 

1.9 

1.73 

Af 

Internet  for  Info 

3.41 

3,13 

Af 

E-mail  with  teachers 

2.6 

2.61 

Table  5:  Changes  in  women  over  time 


Item 

Year  1 W 

Year  2 W 

Year  3 W 

Increased  use  of  technology  makes  learning  easier 

3,39 

3.48 

3,63 

1=2,2= 

3,1*3 

1 prefer  classes  in  which  I get  to  use  computers 

2,72 

3.13 

3,27 

1*2=3 

1 feel  comfortable  using  the  computer 

3,38 

4.04 

4,13 

1*2=3 

I have  a certain  apprehension  about  computer  use 

3,03 

2,16 

2,13 

1*2=3 

Computers  in  teaching  makes  the  learning  process  too  impersonal 

3.25 

2.78 

2,67 

1*2=3 

Computer  use  increases  the  college  work  load 

3,08 

2.92 

2,86 

1=2=3 

Computers  are  effective  for  communicating  with  faculty 

3.33 

3,57 

3,68 

1*2=3 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3.38 

3,57 

3,65 

1=2,2= 

3.1*3 

Computers  enable  me  to  interact  more  with  professors 

2.94 

3.41 

4.19 

1*2*3 

* stands  for  significant  difference  between  means 


Table  6:  Changes  in  men  over  time 


Year  1 M 

Year  2 M 

Year  3 M 

Increased  use  of  technology  makes  learning  easier 

3.56 

3.25 

3.38 

1=3,2= 

3,1*2 

I prefer  classes  in  which  I get  to  use  computers 

3.05 

2.75 

2.77 

1*2=3 

I feel  comfortable  using  the  computer 

3.78 

3.66 

3.9 

1=3,2= 

3,1*2 

I have  a certain  apprehension  about  computer  use 

2.55 

2.56 

2.32 

1=2=3 

Computers  in  teaching  makes  the  learning  process  too 
impersonal 

3.07 

3 

2/77 

1=2*3 
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Computer  use  increases  the  college  work  load 

3.06 

3.01 

3.04 

1=2=3 

Computers  are  effective  for  communicating  with  faculty 

3.31 

3.68 

3.86 

1*2=3 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3.32 

3.74 

4.05 

1*2*3 

Computers  enable  me  to  interact  more  with  professors 

3.07 

3.35 

4.32 

1*2*3 

*stands  for  significant  difference  between  means 


Table  7:  First  year  of  computer  enrichment,  comparison  of  legacy  and  laptop  students 


LM 

LW 

TPM 

TP  W 

Increased  use  of  technology  makes  learning  easier 

3.36 

3.1 

* 

3.72 

3.49 

I prefer  classes  in  which  1 get  to  use  computers 

2.98 

2.68 

* 

3.42 

2.87 

* 

I feel  comfortable  using  the  computer 

3.97 

3.6 

* 

4.17 

3.76 

* 

I have  a certain  apprehension  about  computer  use 

2.23 

2.57 

* 

2.02 

2.54 

* 

Computers  in  teaching  makes  the  learning  process  too  impersonal 

2.94 

3.17 

2.49 

2.72 

Computer  use  increases  the  college  work  load 

2.92 

3.1 

2.89 

2.86 

Computers  are  effective  for  communicating  with  faculty 

3.48 

3.56 

3.75 

3.86 

Communicating  with  professors  by  e-mail  is  generally  gratifying 

3.43 

3.62 

3.84 

3.92 

Computers  enable  me  to  interact  more  with  professors 

3.25 

3.17 

3.73 

3.62 

*stands  for  significant  difference  between  means 


Table  8:  First  year  enrichment  comparison 


LM 

LW 

TPM 

TP  W 

Spreadsheet 

2.02 

1.75 

♦ 

1.78 

1.7 

Database 

1.66 

1.5 

1.89 

1.55 

♦ 

Statistical 

1.85 

1.81 

1.68 

1.69 

Mathematical 

1.8 

1.68 

2.11 

1.82 

♦ 

Internet  for  Info 

3.35 

3.02 

♦ 

3.52 

3.31 

♦ 

E-mail  with  teachers 

2.5 

2.52 

2.77 

2.76 

* stands  for  significant  difference  between  means 


The  same  trend  continued  in  the  second  year  of  the  computer  enrichment  with  the  single  difference  being 
that  women  agreed  significantly  more  to  the  notion  that  communicating  with  professors  by  electronic  mail  was 
generally  more  gratifying.  In  all  other  respects,  the  women  held  a more  negative  attitude  towards  computers  than 
men  or  held  attitudes  similar  to  the  men.  This  tendency  continued  with  respect  of  use  of  computers  as  well. 

The  data  suggests  that  the  attitude  towards  computers  of  women  among  the  laptop  students  was  somewhat 
less  positive  than  the  men  in  the  same  group  for  both  the  first  and  second  years  of  computer  enrichment.  Finally,  the 
data  suggests  that  the  general  attitude  towards  computer  has  become  increasingly  positive  as  the  computer 
enrichment  progresses  and  use  of  computers  has  increased  as  well.  This  finding  is  a throwback  to  the  earlier  days  of 
computing  when  research  had  indicated  that  men  were  more  positively  disposed  towards  computers  than  women. 

Discussion 

First,  it  is  possible  to  claim  that,  like  members  of  many  other  institutions,  the  University  students  had 
displayed  little  differences  in  attitudes  towards  computers  and  use  of  computers  prior  to  the  introduction  of  a 
complex  and  elaborate  technological  enrichment  program.  The  few  significant  differences  based  on  gender  tended 
to  suggest  that  the  men  were  more  positively  predisposed  towards  computers  and  tended  to  use  computers  more  than 
women.  This  finding  is  consistent  with  previous  research  on  the  relationship  between  gender  and  computer 
attitudes.  This  finding  also  has  to  be  placed  in  the  context  of  a liberal  arts  institution  where  the  thrust  of  the 
education  was  not  on  skill -development  and  professional  or  technical  training  but  on  disciplines  that  traditionally 
were  not  highly  computer-use  intensive.  It  can  be  argued  that  in  such  environment  men  and  women  do  not  come  in 
with  specific  expectations  and  predisposition  about  computer  use.  In  such  environments,  the  computer  remains  a 
“tool”  that  can  be  used  to  advance  the  learning  goals  which  would  not  necessarily  be  learning  how  to  use  a computer 
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or  how  to  program  a computer,  but  more  on  how  the  computer  can  make  a pedagogic  and  learning  goal  more 
efficiently  achieved.  We  would  thus  make  the  argument  that  when  the  computer  was  not  the  center  of  attention  and 
publicity  for  the  University,  the  actual  difference  in  computer  use  level  and  attitudes  towards  computers  were 
minimal,  even  if  statistically  significant. 

The  second  major  finding  follows  from  the  earlier  one  in  logical  progression.  Once  the  environment  was 
altered  and  the  overall  “ethos”  of  the  University  shifted  towards  a technological  orientation,  albeit,  in  enriching  the 
campus  only  and  not  in  revamping  the  curriculum  to  make  it  a “technical”  institution,  the  expectations  about  the 
education  that  could  be  obtained  from  the  University  could  have  shifted.  Given,  the  way  the  University  altered  its 
image  in  the  academic  marketplace,  there  was  a specific  emphasis  placed  on  the  computerization  process  as  the 
University  geared  up  for  the  new  Millenium.  Thus,  the  plan  called,  “Plan  for  Year  2000,”  was  publicized  as  one  that 
would  technically  enrich  the  University.  Indeed,  many  other  significant  aspects  of  the  plan,  such  as  the  reduction  of 
class  size,  and  the  addition  of  new  faculty  sometimes  were  made  less  important  given  the  emphasis  that  was  placed 
on  the  technological  aspect  of  the  plan.  This,  as  demonstrated  in  the  content  analysis,  resulted  in  well-documented 
displeasure  among  the  legacy  students,  and  that  displeasure  could  have  resulted  in  the  extension  of  some  of  the  more 
traditional  relationships  between  gender  and  attitudes  towards  computers.  Thus  the  women  being  less  favorable 
towards  computers  could  be  based  on  the  fact  that  the  new  environment  that  was  perceived  to  be  emerging  did  not 
appeal  to  the  women  and  the  blame  was  placed  on  technological  enrichment  and  women  became  more  critical  of  the 
process  than  men. 

In  association  with  this  change,  there  could  have  been  a third  shift  in  terms  of  the  expectations  about  the 
institution.  The  laptop  students  also  displayed  differences  between  genders  with  men  being  more  positive  towards 
computers.  This  phenomenon  could  be  related  to  two  different  processes  at  work  in  the  University.  On  one  hand, 
the  incoming  students  now  had  a somewhat  different  expectation  of  the  environment  that  they  would  encounter  at 
the  University.  While,  there  was  no  evidence,  particularly  in  focus  groups,  that  there  was  any  school-choices  based 
on  the  laptop  program  the  students  did  enter  the  environment  expecting  that  computers  would  play  a significant  role 
in  their  learning  process.  This  expectation,  which  was  different  from  the  traditional  perceptions  of  liberal  arts 
institutions  could  in  itself  have  led  to  the  fact  that  women  came  in  greater  trepidation  than  men.  Such  a conclusion 
is  somewhat  congruent  with  earlier  studies  about  gender  and  computing  where  there  was  some  evidence  to  suggest 
that  men  are  somewhat  more  positively  predisposed  to  computing  than  men.  On  the  other  hand,  this  positive 
predisposition  was  also  related  to  the  newness  of  computers,  and  as  other  studies  have  suggested,  as  the  newness 
disappears  and  the  computer  becomes  more  ubiquitous  the  gender  difference  becomes  insignificant  or  switches  to  a 
more  positive  valence  for  women.  In  other  words,  as  the  computer  becomes  an  expected  and  standard  part  of  the 
process  of  learning,  women  are  likely  to  embrace  it  more  readily.  In  many  ways,  the  computer-enrichment 
phenomenon  could  be  duplicating,  in  a microcosm,  the  phenomenon  that  used  to  be  observed  about  the  relation 
between  gender  and  computer  technology  in  the  past  when  computers  were  being  introduced  into  the  practice  of  our 
everyday  life.  If  that  is  the  case,  it  can  be  hypothesized  that  the  differences  observed  in  the  first  two  years  of  the 
computer  enrichment  will  begin  to  disappear  with  time. 

In  conclusion  it  can  be  claimed  that  the  process  of  acceptance  of  computer  technology  is  related  to  gender 
where  the  women  are  far  more  cautious  about  exhibiting  a positive  attitude  than  men.  However,  when  the 
functionality  of  a technology  has  been  demonstrated  within  the  context  of  a specific  process,  in  this  case  the 
'learning  process,'  women  tend  to  embrace  the  technology  and  are  often  more  positively  pre-disposed.  Furthermore, 
there  are  no  specific  categories  of  use  where  this  process  appears  to  become  more  significant,  and  the  differences 
between  men  and  women  with  regards  to  computer  use  categories  is  unevenly  and  somewhat  randomly  distributed 
suggesting  again  that  it  is  the  practice  of  technology  that  is  more  critical  than  the  specific  technology  itself 

The  prescriptive  aspect  of  these  findings  is  that  technology  enrichment  needs  to  be  accompanied  with  a 
clear  description  of  the  way  in  which  the  technology  would  be  introduced  in  the  primary  practice  of  an  institution. 
In  the  case  of  an  academic  institution  this  would  suggest  that  there  needs  to  be  a clear  rationale  explicitly  stated 
about  the  way  in  which  ubiquitous  computing  would  be  utilized  in  the  process  of  teaching.  Such  explicit  definition 
of  the  technology  practice  and  its  functionality  would  generate  a more  positive  affect  within  women,  who  otherwise 
would  be  far  more  likely  to  consider  the  computer-enrichment  with  a greater  degree  of  criticism.  Such  criticism  can 
then  translate  to  a negative  affect  eventually  creating  undesirable  gaps  between  genders.  On  the  other  hand,  if  the 
expectations  and  functionality  of  the  technology-enrichment  is  made  explicit  at  the  onset,  or  when  in  time  they 
become  explicit,  it  can  be  anticipated  that  the  differences  between  genders  could  either  disappear  or  the  women 
could  exhibit  a more  positive  pre-disposition  than  men.  All  of  this  suggests,  that  planners  of  technology-enrichment 
need  to  be  sensitive  of  the  gender  of  the  users  and  consequently  make  adopt  specific  planning  strategies  to  ensure 
that  unwanted  gender-based  rifts  do  not  result  from  the  process  of  technology  enrichment. 
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Abstract:  A growing  number  of  researchers  have  come  to  the  conclusion  that  we  have  moved 
from  an  earlier  paradigm  (computer-based  education,  CBE)  to  a new  paradigm  which  will  be 
called  network-based  learning  (NBL)  in  this  article.  So  far,  however,  too  little  attention  has 
been  paid  to  the  methodological  aspects  needed  for  creating  NBL  environments.  This  article 
aims  to  analyze  these  developments  by  linking  them  closely  to  new  emerging  electronic 
learning  environments  and  to  methodology.  As  a concrete  example  we  will  analyze  the  use  of 
one  integrated  distributed  learning  environment  (IDLE)  tool  called  "Future  Learning  Environ- 
ments" (FLE).  Our  focus  is  on  academically  oriented  university  students  in  a teacher  education 
program  at  the  University  of  Helsinki,  Finland. 


Learning  Environment  as  Dialog 

Network-based  learning  relates  directly  to  the  concept  of  learning  environments.  There  are  multiple  views  of  a 
learning  environment,  each  eliciting  in  our  minds  different  images  about  teaching,  learning  and  studying.  Mono- 
nen-Aaltonen  (1998)  has  noted  that  these  are  often  divided  into  three  categories:  learning  environments  as 
(i)  ecosystems,  (ii)  places,  and  as  (iii)  space.  As  a novel  way  of  seeing  a learning  environment,  Telia  & Mono- 
nen- Aaltonen  (1998)  have  suggested  that  a learning  environment  should  be  defined  as  dialog  in  its  Bakhtinian 
and  Vygotskian  sense.  In  the  framework  of  dialogism  (Telia  & Mononen-Aaltonen,  1998)  and  the  cultural  de- 
mocracy approach  (Appelbaum  & Enomoto,  1995),  this  article  argues  for  dialogic  learning  environments  and 
dialogic  pedagogy.  The  research  focus  is  based  on  the  concepts  of  modem  media  education  and  social  re- 
constructionism. 

Our  initial  arguments  are  that 

1 . Dialog  as  a learning  environment  is  a social  environment,  in  which  knowledge  is  co-constructed  and  appro- 
priated. 

2.  A network-based  learning  environment  is  a computer-mediated  and  technology-enhanced  dialog. 

3.  A prerequisite  (or  a co-requisite)  for  empowering  NBL  environments  is  the  fundamental  flexibility  of  any  so- 
cial environment.  An  IDLE  should  be  able  to  enable  social  interaction  between  the  different  communicators. 

4.  When  planning  and  designing  NBL  environments,  one  guiding  principle  should  include  the  idea  of  providing 
the  users  with  structures  that  they  can  tailor  to  best  accommodate  their  own  learning  needs. 

5.  Combining  NBL  environments  with  undominated  dialog  contributes  to  constructing  a model  of  social  inter- 
action within  an  unoppressed  and  equal  study  community. 

In  order  to  implement  these  principles,  we  will  have  to  enlarge  the  contemporary  discussion  usually  focused  on 
learning  into  a broader  concept  of  the  teaching-studying-leaming  paradigm  (Uljens,  1997)  in  which  theoretical 
knowledge  plays  an  important  role. 
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The  Teaching-Studying-Learning  Paradigm 


In  new  electronic  environments,  the  teacher  must  have  a strong  and  theoretically  well  justified  methodological 
view  of  her  own.  It  can  be  grounded  on  different  theories  and  conceptions  of  learning  and  knowledge,  such  as 
constructivism.  In  the  teaching-study ing-leaming  paradigm,  we  point  out  the  importance  of  teaching,  studying 
and  the  emergence  of  a study  community,  as  demonstrated  later  in  this  article. 

Admittedly,  the  teacher  will  change  from  a ’sage  on  the  stage’  to  a ’guide  on  the  side’.  In  an  NBL  envi- 
ronment, and  in  our  conception  of  a dialog,  this  is  not  quite  enough:  the  real  question  is  the  dual  stance  (Willis 
1995,  14-15)  of  the  learner  and  the  teacher,  in  which  the  teacher  is  still  on  the  center  stage  as  an  actor  and  as  a 
moderator  of  all  activities  but  at  the  same  time  (s)he  will  be  on  the  side,  observing  the  teaching-learning  process 
with  an  attentive  eye,  reviewing  the  whole  situation.  The  teacher,  then,  is  both  an  actor  and  a critic.  And  so  is 
the  student:  playing  his  or  her  part  but  also  analyzing  his  or  her  own  studying  process  at  the  metacognitive  level. 
The  teacher  can  easily  contribute  to  this  process  by  giving  cognitive  support,  such  as  scaffolding.  Telia  (1999, 
213)  has  argued  that  media  educators  will  need  a ’’media  educational”  eye  in  the  spirit  of  Bourdieu’s  reflexive 
sociology  (Bourdieu  & Wacquant,  1992,  248).  The  question,  then,  is  not  only  of  reciprocity  and  addressivity 
between  the  different  actors  but  also  of  the  changes  of  the  changing  roles.  One  way  to  express  this  change  is  to 
describe  the  teacher  as  the  student’s  cognitive  coach  and  as  a motivating  and  emotional  counselor.  These  will 
also  help  her  to  act  as  an  interpreter  of  the  student’s  relations  with  the  world.  This  kind  of  ’eye’  is  needed  when 
we  think  of  a dialog  as  the  metaphor  of  an  NBL  environment,  which  contextualizes  the  process  of  individual 
empowerment  and  raises  the  awareness  of  individual  actors.  This  is  exactly  our  link  to  social  reconstructionism 
and  to  the  cultural  democracy  approach. 

The  studying  component  consists  of  tools  and  the  context,  and  the  context  helps  the  emergence  of  a 
study  community,  in  which,  again,  different  tools,  like  IDLEs,  will  be  used. 


Tools  and  the  Context 

In  the  emergence  of  a study  community,  the  dual  stance  of  teachers  and  students  will  facilitate  the  adoption  of  a 
dialog  as  a learning  environment.  In  this  community,  certain  tools,  media  and  technologies  will  be  needed  to 
support  dialog.  At  the  same  time,  this  community  will  create  an  educational  context  in  which  these  tools  will  be 
used. 

Tools 

Tools  can  be  divided  into  technological  and  intellectual  ones.  While  it  is  necessary  to  have  a fair  command  of 
’technological’  tools,  it  is  at  least  equally  important  for  everyone  to  cope  with  intellectual  tools.  These  include 
methods  of  creating,  organizing,  and  using  information  and  knowledge.  Lin  et  al.  (1995)  note  the  ability  to  criti- 
cal and  sustained  thinking  and  to  reason  about  important  content  as  well  as  the  ability  and  motivation  to  life- 
long learning  and  autonomous  study.  Rieber’s  findings  (1994;  cited  in  Lin  et  al.  1995,  59)  show  that  learners’ 
personal  discoveries,  possibility  to  explore,  feeling  of  ownership,  and  construction  of  knowledge  can  help  opti- 
mize not  only  intrinsic  motivation  but  also  learning  proper.  In  this  sense,  intellectual  tools,  technological  tools, 
and  learning  are  deeply  intertwined  and  in  close  interrelationship  with  one  another. 

Context 

We  contend  that  in  addition  to  the  tools  component,  the  TSL  paradigm  is  likely  to  lead  to  a new  kind  of  thinking 
of  the  learning  context,  which  could  best  be  depicted  as  an  ’’ideal  public  space”  in  the  spirit  of  Habermas  (1974). 
This  space  is  important,  as  it  is  the  meeting  point  of  all  learners  at  the  very  heart  of  the  study  community.  At  its 
best,  this  thinking  leads  to  contextualism,  which,  according  to  Pettigrew  (1985;  also  Poikela  1999,  38),  is  one  of 
the  ’’world  hypotheses”,  whose  mission  it  is  to  create  order  to  the  chaos.  History,  time  and  change  are  the  basic 
principles  of  contextualism  (Poikela  1999,  38).  Therefore,  the  metaphor  of  contextualism  is  a historical  event, 
with  a special  view  to  the  discussion  of  the  independence  of  time,  place  and  location,  which  is  often  regarded  as 
an  intrinsic  feature  of  an  IDLE. 

Technology  can  be  regarded  as  context,  representing  and  simulating  meaningful  real-world  problems, 
situations,  beliefs,  perspectives,  arguments,  and  stories  of  others.  Technology  as  context  also  supports  discourse 
among  knowledge-building  communities  of  learners  (Jonassen  1995,  62)  and  could  therefore  be  seen  as  suppor- 
tive of  our  argument  of  a dialog  being  the  learning  environment. 


IDLES  (Integrated  Distributed  Learning  Environments) 


Above,  the  dual  stance  of  the  teachers  and  students  was  derived  from  the  TSL  paradigm.  At  the  same  time,  new 
emerging  study  communities  will  greatly  benefit  from  technological  and  intellectual  tools,  as  well  as  from  see- 
ing the  learning  environments  as  a new  context.  In  our  thinking,  IDLEs  (Integrated  Distributed  Learning  Envi- 
ronments; also  called  groupware  tools)  are  ideal  to  be  used  in  the  framework  of  contextualism.  IDLEs  represent 
two  mainstream  cultural  theories  (see  Creswell,  1998,  86-87):  first,  they  represent  ideational  theories,  suggest- 
ing that  change  is  a result  of  mental  activities  and  ideas  enabled  by  language.  Second,  they  represent  materialis- 
tic theories,  holding  that  material  conditions,  like  resources,  money,  and  modes  of  production,  are  prime  mov- 
ers. In  IDLEs,  we  believe,  these  two  theories,  merge  beneficially.  Their  use  call  for  mental  ideas  related  to 
teaching,  studying  and  learning,  but  they  also  represent  a context  or  a forum  of  action,  a ’’center  stage”,  on 
which  different  actors  interact.  What  counts  from  our  point  of  view  is  that  IDLEs  can  be  especially  designed  for 
communal  modes  of  inquiry  and  studying,  as  in  our  research  project.  But  once  on  the  ’’center  stage”,  one  needs 
some  principles  for  ’’acting"  properly.  We  will  discuss  these  in  the  following. 


The  Four  Basic  Elements  of  the  Dialogic  Learning  Environment,  in  relation  to  IDLEs 

In  our  thinking,  the  four  basic  elements  of  a learning  environment  are  the  study  community,  the  zone  of  proxi- 
mal development,  the  We  relationship  and  the  dialogic  field  (Figure  1). 


Study  Community 

Jiialogic  FJeld.  ;;  , i ,i  a , ^ 

' I : ,1:1.  ireRelationshiR 

Zone  of  Proximal  Development 

Figure  1.  The  basic  elements  of  a learning  environment  (Mononen-Aaltonen  1998,  195; 
http://www.helsinki.fi/~tella/montreal4field.htmn. 

In  Buber’s  philosophy,  the  concept  of  We  is  overshadowed  by  the  dominating  doctrine  of  I-Thou,  and  as  a con- 
sequence, the  We  has  failed  to  attract  sufficient  attention.  The  Buberian  We  is  animated  in  the  shared  speech 
and  finds  its  beginning  when  one  addresses  another  on  the  I-Thou  basis.  The  Buberian  We  suggests  a setting 
for  dialog  to  realize  itself 

It  is  through  a dialogic  field  that  learners  discover  their  ways  of  thinking,  their  truths,  their  belief  sys- 
tems, their  fields  of  vision,  which  of  them  they  share  with  others  and  which  are  unique  to  them. 

For  Vygotsky,  the  most  essential  feature  of  teaching  and  education  is  that  it  creates  the  zone  of  proxi- 
mal development  (ZPD).  The  dynamic  movement  within  the  ZPD  is  a prerequisite  for  growth  and  knowledge 
construction  that  moves  the  learner  from  one  stage  of  development  to  another.  This  is  done  with  the  help  from 
outside  one’s  mind,  with  the  help  of  another  voice,  another  mind.  Through  technological  tools,  especially  when 
combined  with  intellectual  tools  such  as  cognitive  support,  coaching,  modeling,  and  scaffolding,  we  can  help 
elevate  learners  through  their  zones  of  proximal  development. 

The  study  community  is  interpreted  in  line  with  Bruner’s  ideas  of  the  classroom  being  a subcommunity 
that  specializes  in  learning  among  its  members-a  subcommunity  of  mutual  learning.  Mutuality  is  tangible  in  the 
sense  that  we  can  feel,  notice  and  see  it  as  well  as  identify  with  it. 


The  Research  Project 

The  Media  Education  Center  of  the  University  of  Helsinki  has  recently  launched  a research  project  on  dialog, 
communication  and  technology  in  teaching  and  learning  at  the  university  level.  The  project  is  carried  out  in  a 
foreign  language  methodology  research  seminar,  where  the  traditional  face-to-face  mode  of  studying  and  learn- 
ing is  complemented  with  working  in  an  NBL  environment  by  using  an  IDLE  tool  called  Future  Learning  Envi- 
ronments (FLE),  designed  for  communal  modes  of  inquiry  and  studying.  The  study  project  is  part  of  the  FLE 
Project,  launched  in  1 998  and  co-ordinated  by  the  Medialab  of  the  Helsinki  University  of  Art  and  Design.  The 
teacher  in  charge  of  the  research  seminar  has  been  Maija  Mononen-Aaltonen. 

Our  aim  is  two-fold:  (i)  to  create  a learning  environment  that  would  empower  the  students  to  partici- 
pate in  the  discourse  of  educational  sciences  and  to  conduct  research  on  foreign  language  methodology,  and 
(ii)  to  support  each  student’s  research  project  as  a whole  within  a dialogic  frame  of  reference.  We  aim  at  pro- 
moting sharing  and  discussing  ideas  at  all  stages  of  the  scientific  inquiry  process.  The  research  seminar  has  been 
chosen  for  two  reasons:  First,  the  students  come  from  other  university  departments  in  order  to  do  their  teacher 
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education  module  (one  academic  year)  at  the  Department  of  Teacher  Education.  They  start  our  research  seminar 
only  three  weeks  after  they  have  first  entered  the  world  of  educational  sciences,  which  is  to  them  a totally  new 
academic  culture  with  its  own  discourse  and  traditions,  concepts  and  research  problems.  Second,  the  program 
includes  two  two-month  school  placements.  The  students  are  expected  to  continue  their  research  papers  also  in 
schools,  and  complete  them  by  the  end  of  the  academic  year. 

As  stated  above,  our  study  is  based  on  the  notion  of  a learning  environment  as  dialog  in  the  Bakhtinian 
and  Vygotskian  sense.  The  seminar  is  conducted  in  such  a way  that  dialog  is  likely  to  become  the  way  to  be  a 
seminar  member:  the  way  to  teach,  to  study  and  to  learn  the  culture  and  the  language  of  the  academic  commu- 
nity of  education,  and  to  participate  in  its  scientific  discourse.  The  study  aims  at  developing  general  pedagogic 
principles  and  teaching  practices  to  support  dialog  in  an  academic  learning  community. 

We  will  now  briefly  analyze  how  the  four  components  of  the  dialogic  learning  environment  (Figure  1) 
can  be  worked  on  as  elements  of  an  academic  study  community. 

For  instance,  when  creating  the  dialogic  field,  the  participants  need  to  be  engaged  in  the  dialog  from 
the  very  first  research  idea.  Then  thinking  becomes  a text  and  the  students  as  well  as  the  teacher  has  a 360°  field 
of  vision.  This  might  lead  to  a temporary  chaos,  which  is  shown  in  the  unstructured  and  non-linear  nature  of  the 
research  process. 

One  more  step  is  to  activate  the  ZPD  by  making  the  research  process  visible  and  understandable  to  the 
learners.  More  data  will  be  needed,  browsed  and  analyzed  during  the  on-going  process  than  in  traditional  semi- 
nars. Continuous  feedback  is  needed.  The  target  at  this  stage  is  telos,  the  cognizance  of  common  and  shared 
educational  aims  to  be  achieved  through  the  research  project. 

At  the  same  time,  an  interface  to  the  We  relationship  needs  to  be  created.  It  means,  among  other  things, 
that  the  openness  of  the  relationship  becomes  an  opportunity  and  a risk.  In  this  process,  the  teacher's  contribu- 
tion is  crucial.  However,  the  teacher  needs  to  remember  that  participating  in  a dialog  is  voluntary,  too  much 
control  or  pushing  will  destroy  the  budding  dialogic  atmosphere. 


Research  Findings 

One  of  the  foci  of  our  research  project  has  been  an  authentic  use  of  information  and  communication  technolo- 
gies in  the  real  world  and  their  potential  in  research-focused  teaching  in  teacher  education.  The  data  gathered 
during  the  academic  year  1998-99  are  being  analyzed  at  the  moment.  Therefore,  these  findings  are  preliminary 
but  indicative. 

The  members  of  the  seminar  are  interpreted  to  form  a study  community.  This  study  community  is  seen 
as  a ’common  obligation’  to  everybody  in  the  Latin  sense  of  the  word:  cum  (with,  together)  and  munis  (duty, 
obligation),  thus,  a common  obligation,  shared  by  everybody  but  not  imposed  on  anybody.  In  this  way,  the  stu- 
dents’ minds  will  be  geared  towards  research  and  thus  the  process  is  more  likely  to  result  in  a ’minds-on’  ap- 
proach, instead  of  an  earlier  more  pragmatic  hands-on  approach.  The  work  is  therefore  moving  from  brawn  to 
brain,  with  a distinctly  conceptual  character.  The  many  comments  by  the  students  indicate  that  we  have  suc- 
ceeded in  this. 

The  support  given  by  an  NBL  environment  to  the  emergence  of  a study  community  and  the  We  rela- 
tionship is  indirect  and  mediated.  In  research-focused  teaching,  NBL  appears  to  be  very  important  as  the  creator 
and  maintainer  of  the  zone  of  proximal  development  and  the  dialogic  field. 

We  believe  that  it  is  through  dialog  with  others  that  learners  discover  their  ways  of  thinking,  their 
truths,  their  belief  systems,  their  visions,  all  of  which  they  can  then  share  but  which  will  still  be  unique  to  them. 
Their  ways  of  thinking,  being,  believing  and  studying  open  up  as  a dialogic  field.  As  Bakhtin  put  it:  ’’What  we 
see  can  never  be  what  you  see,  if  only  because  I can  see  what  is  behind  your  head”. 

Our  students  participate  in  a real  research  project  with  the  aid  of  an  authentic  IDLE.  From  the  very  be- 
ginning, they  are  aware  of  belonging  to  a large  research  project.  They  understand  that  they  are  not  only  writing 
an  ’academic  piece  of  paper’  but  are  fully  committed  to  developing  a new  area  of  methodology  and  the  tools  to 
be  used  in  it. 

The  approach  we  promote  also  includes  a few  hazards  and  threats.  The  FLE  interface  consists  of  each 
participant’s  desk,  open  to  everybody  else  in  the  same  study  group.  Everybody’s  documents  and  other  materials 
are  openly  accessible  by  everybody  else.  Even  access  to  each  other’s  messages  is  open.  In  this  sense,  the  FLE 
functions  as  a technological  and  intellectual  interface  to  the  We  relationship,  but,  in  our  case,  again  in  an  indi- 
rect way:  the  students  knew  each  other  and  the  teacher  before  entering  the  seminar,  as  they  had  participated  in 
her  other  courses  before  this  seminar.  The  openness  of  the  FLE  to  all  the  participants  of  the  seminar  with  open 
desks  and  files  is  a new  opportunity  for  collaboration  and  sharing  ideas  and  thoughts.  But  at  the  same  time  the 
students  may  find  it  awkward  to  go  to  their  co-researchers’  desks  and  open  their  files.  Clearly,  more  emphasis 
should  be  laid  on  the  personality  characteristics  of  learners  who  find  it  valuable  to  use  these  emerging  technolo- 
gies, such  as  the  IDLEs.  We  simply  refer  to  Dede’s  argument  (1995,  47)  that  there  might  exist  a whole  new  di- 
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mension  of  learning  styles  orthogonal  to  the  visual/auditory/kinesthetic/symbolic  categories  now  underlying 
pedagogical  approaches  to  individualization. 

One  of  our  conclusions  is  that  the  metaphor  of  a desk  is  not  appropriate;  rather,  we  should  speak  of  a 
public  space,  like  a virtual  library,  in  which  everybody  can  browse  through  the  documents.  If  the  concept  of 
virtuality  represents  cultures  based  on  the  logical  rather  than  the  physical  (as  argued  by  Agres,  Edberg  & Igbaria 
1998,  72),  it  seems  that  ours  is  one  of  the  cases  in  which  bringing  reality  into  the  virtual  world  might  encounter 
some  unexpected  barriers  at  the  conceptual  level  of  the  learners. 

Lin  et  al.  (1995,  54;  as  reported  in  Telia  1997)  argue  that  social  considerations  belong  to  the  second 
wave  of  the  cognitive  revolution,  focusing  attention  on  the  social  contexts  of  learning  that  have  pervasive  cog- 
nitive and  motivational  effects,  while  the  first  wave  mainly  dealt  with  individual  thinkers  and  learners,  de- 
emphasizing affect,  context,  culture,  and  history.  In  this  light,  the  use  of  a groupware  tool  such  as  FLE  is  most 
likely  to  amplify  the  relations  between  the  teacher  and  the  students,  and  between  the  students  themselves,  and 
finally  helping  construct  a study  community,  in  which  the  second  wave  cognition  and  motivation  prove  indis- 
pensable. We  would  argue  that  a traditional  research  seminar  is  much  more  likely  to  point  out  the  first  wave 
emphases,  i.e.,  individual  work  done  without  any  real  co-construction  of  knowledge. 

The  teacher’s  role  seems  to  prove  most  crucial  in  creating  a dialogic  learning  environment  in  the  con- 
text of  FLE.  It  is  the  teacher  who  either  creates  the  We  relationship  or  destroys  the  opportunity  to  establish  any 
relationships  on  a dialogic  basis.  The  teacher’s  role  becomes  even  more  important  in  creating  and  maintaining 
successful  mediated  learning  experiences.  Importantly,  an  IDLE  may  open  up  the  chaotic  and  non-linear  char- 
acter of  almost  any  research  process,  but  at  the  same  time  it  helps  to  give  order  to  this  chaos.  In  the  final  analy- 
sis, then,  the  mediated  learning  experience  consists  of  the  individual  empowerment  properly  contextualized 
within  the  virtual  working  concept  of  a study  community,  and  inducive  to  fostering  a better  comprehension  of 
individual  participation  in  a mutually  shared  research  project. 
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Abstract  Educational  multimedia  systems  have  a potentially  important  role  to  play  in  the  education 
of  students  with  autism.  This  potential  has  thus  however,  been  largely  unexplored.  This  paper 
therefore  makes  a case  for  the  use  of  multimedia  systems  for  such  students,  and  reviews  the  current 
state  of  the  field.  It  will  be  argued  that  a fruitful  way  of  furthering  research  and  development  in  this 
area  is  to  develop  systems  that  directly  address  autism-specific  impairments.  Proposals  for 
multimedia  systems  for  each  of  these  areas  will  be  put  forward,  and  our  current  development  work 
outlined. 


Introduction  - A case  for  Multimedia  Systems  for  Students  with  Autism 

A commonly  held  view  of  the  nature  of  autism  is  that  it  involves  a “triad  of  impairments”  (Wing  1996).  There  is 
a social  impairment:  the  person  with  autism  finds  it  hard  to  relate  to  other  people.  Secondly,  there  is  a 
communication  impairment:  the  person  with  autism  finds  it  hard  to  understand  and  use  verbal  and  non-verbal 
communication.  Finally,  there  is  a tendency  to  rigidity  and  inflexibility  in  thinking,  language  and  behaviour. 
Much  current  thinking  is  that  this  triad  is  underpinned  by  a “theory  of  mind  deficit”  - people  with  autism  may 
have  a difficulty  in  understanding  mental  states  and  in  ascribing  them  to  themselves  or  to  others. 

Why  might  it  be  thought  that  multimedia  systems  could  be  beneficial  in  the  context  of  the  education  of  students 
with  autism?  First,  it  has  been  argued  that  many  people  with  autism  have  a natural  affinity  with  computers 
(Murray  1997)  and  that  pupils  with  autism  tend  to  enjoy  using  computers  (Swettenham  1992).  Indeed,  Jordan 
(1991)  has  suggested  of  children  with  autism  that  “..  it  is  often  valuable  to  view  their  thinking  and  learning  as 
analogous  to  computers  - you  only  get  out  what  you  put  in”.  Concerning  multimedia  in  particular,  it  is  well 
known  that  children  with  autism  are  attracted  to  visual  images  (Virart  1997)  and  since  it  is  thought  that 
individuals  with  autism  may  not  have  3D  vision  (Grandin  1995)  then  the  world  of  the  flat  screen  may  be  more 
acceptable  and  less  frightening  than  the  real  world.  Multimedia  systems  may  be  beneficial  in  that  they  can 
eliminate  the  need  for  reading  in  order  to  engage  in  computer  aided  learning  (CAL)  activities  (lacono  and 
Miller  1989)  and  Murray  (1997)  suggests  that  through  computer  use  the  student  with  autism  may  in  fact  become 
motivated  to  read. 

A further  advantage  is  that  the  computer  presents  a relatively  controlled  environment  and  can  eliminate  many 
distractions  of  the  normal  classroom  (Green  1993);  attempting  to  reduce  potentially  disturbing  stimuli  is  in  line 
with  the  “SPELL”  approach  to  teaching  (Siddles  et  al  1997).  A further  benefit  concerns  the  individualised,  one 
to  one  tutoring  that  may  be  made  possible  by  interactive  multimedia  systems.  Such  a teaching  mode  is  often 
seen  as  important  for  students  with  autism.  Peeters  (1995),  for  example,  talks  of  the  “unbelievable  luxury  of 
individual  attention”  and  Higgins  and  Boone  (1996)  see  individualisation  as  “key  to  appropriate  instruction”  for 
students  with  autism. 


Existing  Multimedia  Systems  for  Students  with  Autism 

There  appears,  then,  great  potential  for  beneficial  multimedia  systems  for  students  with  autism.  And  there  is  a 
growing  body  of  such  applications.  Tjus  et  al  (1998)  use  multimedia  for  teaching  literacy  to  pupils  with  autism, 
allowing  the  student  to  construct  a sentence  in  text  and  receive  “immediate  multi-channel  feedback”.  Anthony 
(1992)  discusses  a non-oral  approach  for  teaching  communication  skills  to  severely  handicapped  “autistic-like” 
students.  The  approach  involves  “picture  communication”,  using  dedicated  communication  devices  that  can  be 
programmed  to  reflect  the  daily  activities  and  choices  in  the  classroom.  Anthony  claims  that  the  results  of  the 
approach  include  reductions  in  “non-social  behaviours”  and  increased  verbal  interactions. 
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In  a related  approach,  Arora  and  Goodenough*Trepagnier  (1989)  adopt  a game  show  format  in  their  visual 
communication  systems.  Another  system  adopting  a games-based  approach  is  “Danny’s  Rooms”.  This  is  a 
“recreational  multimedia  application  for  the  autistic  population”  (Eddon  1992)  with  seven  games,  such  as  a 
photograph  game  invoking  spoken  messages  from  on-screen  photographs  of  the  family  and  the  rooms  game  in 
which  the  user  is  required  to  navigate  a series  of  increasingly  complex  rooms.  Another  room-based  application 
is  “AVATAR”  (Enyon  1997),  in  which  the  user  activates  objects  in  the  room  which  respond  with  an  appropriate 
sound  effect,  with  the  word  in  writing  and  in  Makaton  and  with  the  word  spoken. 

More  generally,  Higgins  and  Boone  (1996)  have  provided  a set  of  software  design  guidelines  for  use  with  CAL 
systems  for  people  with  autism,  and  a group,  ASILeSP  (Autism  Specific  Interactive  Learning  Software 
Packages)  has  been  set  up  to  look  specifically  at  “effective  ways  to  use  technology  to  empower  and  to  support 
individuals  with  autism”. 

Despite  this  potential,  however,  the  field  remains  relatively  unexplored.  Higgins  and  Boone  see  educators  as  in 
a position  only  to  '’'"begin  to  explore  the  possibilities  of  the  new  technologies  for  students  with  autism”  (Higgins 
and  Boone  1996,  emphasis  added),  and  the  British  National  Autistic  Society  (NAS)  points  to  “....  a general  lack 
of  computer  programs  which  have  been  written  specifically  for  children  with  autism”  (NAS  1 996).  Similarly, 
Eddon  (1992)  points  to  a “void  in  recreational  computer  applications  for  the  autistic  population”  and  Enyon 
(1997)  claims  a “growing  need  voiced  by  parents  and  professionals  for  specially  developed  computer 
programs”. 


A Proposal  for  Research  and  Development 

Given  that  there  is  a compelling  case  for  educational  multimedia  systems  for  students  with  autism  and  that  the 
field  is  thus  far  largely  un-researched,  an  important  issue  concerns  delineating  areas  that  might  usefully  be 
addressed  by  such  systems.  The  priority  should  be,  we  argue,  to  develop  interactive  systems  specifically  aimed 
at  the  core  autistic  deficiencies.  We  argue  for  this  in  detail  elsewhere  (Moore  et  al  2000)  but  essentially  our 
argument  is  that  such  a prioritisation  is  in  line  with  current  pedagogic  theoiy  relating  to  autism.  For  example, 
Jordan  argues:  “of  course,  it  will  be  a teaching  priority  to  enable  the  pupil  to  improve  interpersonal  and 
communication  skills”  (Jordan  1992a). 

The  importance  of  targeting  autism-specific  issues  is  given  greater  urgency  by  the  fact  that  the  limited  provision 
of  existing  software  designed  for  users  with  autism  tends  not  to  directly  address  the  autistic  condition  as  such. 
Eddon  (1992),  for  example,  points  out  that  many  applications  relate  to  basic  skills  such  as  knowledge  of  the 
alphabet,  and  that  these  are  less  appealing  to  older  children  who  might  be  slighted  by  their  childish  nature.  Even 
Eddon’s  own  Danny’s  rooms  itself,  though,  seems  not  to  be  directly  targeted  at  the  autistic  condition  per  se,  in 
that  navigating  rooms  or  invoking  sounds  from  farm  animals  is  intuitively  unlikely  to  help  autism  specific 
concerns  such  as  a lack  of  social  skills.  More  generally,  Trehin  (1996)  argues  that  the  attempt  to  act  on  “one  or 
several  specific  deficiencies  encountered  in  autism”  is  “only  in  its  infancy”.  This  is  not  of  course  to  deny  the 
worth  of  the  work  outlined  earlier,  far  from  it,  but  merely  to  argue  for  an  urgent  need  for  programs  targeted 
directly  at  the  difficulties  characterising  the  autistic  condition. 

In  the  remainder  of  the  paper,  therefore,  we  will  consider  means  by  which  these  difficulties  might  be  addressed 
by  multimedia  systems. 


Social  Skills  Education 

Direct  teaching  of  social  skills  to  students  with  autism  is  widely  advocated  (e.g.  Jordan  1991).  Multimedia 
systems  with  varying  levels  of  interactivity  can  be  developed  to  assist  in  this  endeavour.  The  lowest  level  of 
interactivity  would  involve  simply  observing  social  events.  Even  this  may  be  educationally  useful.  Jordan 
(1992b)  suggests  that  pupils  with  autism  should  be  able  to  observe  how  non-autistic  children  play  with  each 
other.  Jordan  points  out  that  the  “pupils’  observations  need  to  be  directed  to  salient  points”,  and  this  can  be 
aided  by  multimedia  systems  in  that  they  can  be  paused,  speeded  up  and  repeated  endlessly,  and  graphics  and 
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animations  can  be  overlaid  to  highlight  the  crucial  points.  Further,  the  multimedia  facilities  provide  a readily  re- 
usable resource,  which  may  be  important  in  specialist  settings  where  opportunities  to  observe  play  may  be 
sparse  (Jordan  1992b).  This  facility  can  be  further  enhanced  by  creating  examples  of  “good”  and  “bad”  social 
interactions,  asking  the  user  to  pick  out  sub-optimal  interactions,  and  allowing  the  user  to  invoke  displays  of  the 
motives  and  thoughts  of  the  various  participants,  in  the  manner  of  “bubble  dialogues”  (O’Neill  and  McMahon 
1991). 

A higher  degree  of  interactivity  can  be  created  in  simulated  role  play  activities.  Role-play  (which  may  include 
the  use  of  video)  is  used  to  teach  social  skills  to  adults  and  children  with  autism  (e.g.  Yates  1993),  and 
simulated  versions  can  be  built.  Interactive  video  disks  for  management  training  have  been  available  for  some 
time.  Social  scenarios  are  introduced  and  a number  of  alternatives  offered  to  the  user;  the  user’s  choice  invokes 
further  video  and  further  decision  points,  so  that  the  user  is  branched  through  at  least  some  of  the  on-line 
material  in  line  with  their  answers.  Similar  work  has  been  carried  out  in  connection  with  more  general  social 
skills  (lacono  and  Miller  1989). 

Much  of  our  current  development  work  is  centred  around  building  simulations  of  a similar  kind,  involving 
contexts  likely  to  be  found  particularly  problematic  by  people  with  autism,  such  as  the  school  lunch  hour 
(Cumine  et  al  1998),  As  an  example,  one  of  our  prototypes  presents  the  student  with  a video  clip  depicting  a 
dining  room  queue  and  asks  where  the  queue  should  be  joined.  Different  clips  of  video  are  then  shown, 
consequent  upon  the  selection  made,  before  the  system  moves  onto  other  aspects  of  dining  room  protocol  such 
as  choosing  an  appropriate  place  to  sit.  Evaluation  of  interactive  systems  for  students  with  autism  is,  we  argue, 
particularly  problematic  (Moore  et  al  2000),  but  preliminary  formative  evaluation  of  the  system  has  proved  very 
encouraging 

To  some  extent  such  work  involves  creating  on-line  versions  of  existing  practice,  and  it  might  be  argued  that 
such  an  approach  may  diminish  the  impact  of  the  technology  and  that  it  would  be  better  to  go  beyond  what  can 
be  done  in  conventional  methods  (cf  Green  1990).  However,  the  computer-based  simulations  do  offer  the 
advantages  of  interactivity,  of  being  repeatable  and  of  offering  “safe”  experimentation  with  ideas.  Further,  they 
are  readily  adaptable  in  that,  for  example,  text  can  be  turned  on  and  off  depending  on  the  perceived  needs  of  the 
user. 


Communication  Skills  Education 

Here  we  follow  lacono  and  Miller’s  (1989)  suggestion  that  individuals  with  poor  communication  skills  “may 
stand  to  gain  most  from  computer  applications  specifically  designed  to  teach  or  enhance  communication  skills”. 

At  the  level  of  the  individual  word,  talking  word  processors  may  be  useful  (NCIP  1995).  There  has  been 
considerable  work  in  automating  symbol  systems  for  speech  and  hearing-impaired  people  (Sale  and  Carey 
1995,  Anthony  1992)  and  systems  such  as  Makaton  are  frequently  used  by  people  with  autism  (e.g.  Rutherford 
1996).  At  the  semantic  and  syntactic  levels,  drill  and  practice  may  be  valuable,  e.g.  lacono  and  Miller  (1989) 
suggest  using  drill  and  practice  for  initial  teaching  of,  for  example,  verbs,  following  which  “the  verbs  can  be 
utilised  in  computer  games,  exploratory  play  and  simulations  of  fantasy  then  real  life”. 

At  the  pragmatic  level,  lacono  and  Miller  (1989)  suggest  computer  or  video  simulations  as  a way  of 
encouraging  communication.  Concerning  non-verbal  communication,  Yates  (1993)  advocates  video  for  the 
teaching  of  body  language,  and  video  tapes  of  differing  facial  expressions  are  already  used,  as  are  graphically 
based  face  games  (Jordan  1992b,  Yates  1993).  Tantam  (1993),  however,  suggests  that  “systematic  training  of 
emotional  expressiveness  has  never  been  carried  out  in  people  with  autism”  and  interactive  systems  may  have 
an  important  role  here. 

Our  current  prototypes  in  this  area  seek  to  address  non-verbal  communication  by  requiring  users  to  identify 
facial  expressions  and  body  language.  Users  are  presented  with  examples,  via  a mixture  of  line  drawings,  on- 
line still  photographs  and  videos,  and  asked  to  identify  the  likely  messages  being  given.  They  are  then  offered 
explanations  and  further  examples,  dependent  upon  their  progress  to  date.  Preliminary  formative  evaluation 
results  are  again  very  encouraging.  A range  of  similar  scenarios  can  be  created,  such  that  the  series  of 
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interactions  a user  might  engage  in  becomes  of  a drill  and  practice  nature.  Drill  and  practice  may  be  unlikely  to 
generate  a deep  understanding  of  the  relevant  concepts,  but  may  at  least  enable  the  associating  of,  for  example, 
expressions  with  emotions,  which  may  be  very  helpful:  “[people  with  autism]  will  not  ...  directly  perceive 
someone’s  joy  or  despair,  although  they  can  come  to  ‘work  out’  how  certain  facial  expressions  and  behaviours 
are  associated  with  certain  given  labels  such  as  ‘sad’  or  ‘happy’”  (Powell  and  Jordan  1997). 


Rigidity  of  Thought 

This  impairment  can  generate  a number  of  learning  difficulties.  Many  of  these  may  be  amenable  to  assistance 
from  multimedia  systems.  The  problem  of  lack  of  imagination  (Green  1990)  and  lack  of  creativity  and 
spontaneity  (Jordan  1993)  may  be  helped  by  simulation,  by  the  interactivity  and  animation  of  multimedia  as 
opposed  to  paper  based  material  and  by  being  able  to  create  one’s  own  multimedia.  One  approach  to  the 
difficulty  of  restricted  play  is  to  encourage  role-play  (Jordan  1993),  and  multimedia  simulations  may  help  here. 
“Difficulties  in  sequencing  a prediction”  (Simmonds  1993)  can  be  helped  by  multimedia  simulations,  for 
example  the  sequences  in  a plant’s  life  cycle  may  be  understood  more  readily  if  speeded  up  in  a simulation.  The 
adverse  affect  of  rigidity  of  thought  on  problem  solving  (Simmonds  1993)  can  be  addressed  by  the  sort  of 
problem  driven  interaction  to  which  CAL  is  well  suited. 

The  autistic  tendency  to  “tunnel  vision”  (Jordan  1993)  and  difficulties  with  enquiry  (Simmonds  1993)  may  be 
helped  by  hypermedia  systems.  A student  with  autism  may  have  a major  interest  in  a narrow  topic  area,  such  as 
deltic  trains  (Attwood  1998).  A hypermedia  page  about  deltic  trains  could  have  links  from  the  initial  page  to 
(say)  the  workings  of  a diesel  engine,  and  from  there  into  topics  from  general  physics.  The  student  may 
therefore  be  encouraged  to  take  a wider  perspective. 

A further  problem  caused  by  rigidity  of  thought  is  “difficulties  in  accepting  the  essentially  open-minded  attitude 
that  is  a feature  of  scientific  enquiry”  (Jordan  1992b).  Clearly  it  is  essential,  with  students  with  autism  and 
indeed  all  students,  to  seek  to  produce  individuals  who  can  think  for  themselves.  Computer  programs  have  been 
developed  to  help  teach  critical  thinking  skills  (Jordan  and  Powell  1990)  and  our  current  research  in  multimedia- 
based  argumentation  systems  (Moore  2000)  may  have  a role  to  play  here. 


Multimedia  to  Address  the  “Other  Minds  Deficiency” 

It  has  been  postulated  that  people  with  autism  have  a cognitive  deficit  in  the  development  of  a “theory  of  mind” 
(Swettenham  1996).  People  with  autism,  it  is  held,  do  not  understand  mental  states  and  cannot  ascribe  them 
either  to  themselves  or  to  others.  They  do  not  understand  what  people  are  thinking  or  feeling  or  even  that  people 
are  thinking  and  feeling  (Jordan  1993). 

Again,  multimedia  is  a potentially  powerful  teaching  medium.  Simulations  involving  predicting  people’s 
emotions  in  and  reactions  to  different  situations  can  be  developed.  People  with  autism  may  for  example  hit 
other  people  without  realising  it  hurts,  may  stare  at  people  without  realising  it  may  make  them  uncomfortable 
(Carlton  1993),  and  such  scenarios  can  readily  be  modelled  in  interactive  multimedia.  One  of  our  current 
prototypes,  for  example,  plays  a number  of  videos  of  youngsters  interacting  in  different  ways,  e.g.  stealing 
sweets  from  each  other  or  helping  each  other  out.  It  then  asks  the  user  to  suggest  how  the  participants  in  the 
video  might  be  feeling;  this  is  a preliminary  attempt  at  using  multimedia  to  address  the  concern  of  a possible 
lack  of  empathy.  Tantam  (1993)  argues  that  people  with  autism,  “like  people  who  are  not  autistic,  modify  their 
behaviour  according  to  its  predicted  result”  and  the  use  of  such  simulations  may  help  increase  the  accuracy  of 
the  predictions. 

A different  but  related  approach,  is  to  use  “bubble  dialogues”  (O'Neill  and  McMahon  1991).  The  essential  idea 
here  is  that  dialogue  participants  are  provided  with  an  on-screen  character,  in  strip  cartoon  format,  together  with  a 
"say  bubble"  and  "thought  bubble"  for  their  character.  Participants  participate  in  dialogue  by  typing  their 
contributions  into  the  say  bubble,  and  their  private  reflections  into  the  thought  bubble.  In  all  this  the  aim  is  to 
encourage  ascription  to  others  of  thoughts,  emotions,  motives  and  intentional ity,  in  line  with  claims  that,  for 
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example,  pupils  with  autism  need  to  be  “specifically  taught  that  others  have  points  of  view”  (Jordan  1991).  The 
ability  of  students  with  autism  to  understand  thought  bubbles  is  controversial,  but  Parsons  and  Mitchell  (1999)  argue 
that  “at  least  some  children  with  autism  are  capable  of  understanding  thought  bubbles  as  a representational  device”. 
Further,  there  is  some  evidence  of  the  successful  use  of  bubble  dialogues  with  children  with  autism  (McMahon 
1999,  Rajendran  and  Mitchell  2000). 


Summary  and  Further  Work 

This  paper  has  given  a brief  overview  of  our  on-going  research  and  development  work  concerning  multimedia 
for  students  with  autism.  Many  important  research  issues  are  raised  and  are  being  investigated  (cf  Moore  et  al 
2000).  Chief  among  these  are  how  best,  if  at  all,  to  utilise  the  apparent  attraction  of  computer  games,  ways  of 
catering  for  student  differences  and  deciding  a suitable  evaluative  regime,  A particular  concern,  perhaps,  is  the 
danger  than  computer-based  work  may  itself  become  a dominant  interest  of  the  user  with  autism  and  thus  fail  to 
ease,  or  perhaps  even  exacerbate,  the  autistic  condition.  The  answer  to  this,  we  argue,  lies  in  the  successful 
integration  of  computer-based  work  with  other  aspects  of  educational  provision.  How  best  to  manage  this 
integration  is  an  important  research  question. 

Much  remains  to  be  done,  therefore,  but  the  potential  benefits  are  great.  Murray  (1997)  argues,  concerning  the 
education  of  students  with  autism,  that  “the  auto  didactic  possibilities  are  endless”.  The  hope  is  that  the  research 
discussed  in  this  paper  will  contribute  in  some  measure  to  the  achievement  of  these  possibilities. 
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Abstract:  The  goal  of  this  study  is  to  test  the  hypothesis  that  animated  pedagogical  agents  can 
promote  constmctivist  learning  in  a discovery-based  learning  environment.  We  do  this  by  first, 
comparing  the  learning  outcomes  of  students  who  learn  in  the  context  of  social  interaction  with  a 
computer-mediated  character  with  the  learning  outcomes  of  students  who  learn  in  a computer  text 
context.  Second,  we  focus  on  the  relative  contributions  of  pedagogical  agents'  image,  voice,  and 
language  style  in  promoting  deeper  learning. 


The  Case  for  Software  Pedagogical  Agents 

Discovery-based  learning  environments  are  intended  to  facilitate  constructivist  learning  through  creative  problem- 
solving experiences.  However,  the  characteristics  of  discovery-based  environments  which  offer  the  greatest 
potential  for  fostering  learning  also  pose  the  greatest  challenge:  the  complexity  of  the  learning  experience,  based  on 
the  learner’s  freedom  to  explore  and  design  artifacts  in  microworlds,  constantly  threatens  to  overwhelm  learners.  To 
combat  this  complexity,  discovery-based  learning  environments  should  provide  scaffolding  in  the  form  of  highly 
contextualized  problem-solving  advice  that  is  customized  to  each  learner.  Perhaps  the  most  intriguing  vehicle  for 
providing  such  dynamically  individualized  scaffolding  is  the  emerging  technology  of  animated  pedagogical  agents— 
lifelike  on-screen  characters  who  provide  contextualized  advice  and  feedback  throughout  a learning  episode  (Lester 
& Stone  1997,  Lester  et  al.  1997). 


A Life-Like  Pedagogical  Agent  Example 

The  discovery-based  learning  environment  used  in  this  study  is  a microworld  called  ”Design-A-Plant”, 
developed  by  the  Multimedia  Laboratory  at  the  Department  of  Computer  Science  of  North  Carolina  State  University 
(Lester  & Stone  1997,  Lester  et  al.  1997).  In  this  program,  the  student  travels  to  an  alien  planet  that  has  certain 
environmental  conditions  (e.g.,  low  rainfall,  light  sunlight)  and  must  design  a plant  that  would  flourish  there  (e.g., 
including  designing  the  characteristics  of  the  leaves,  stem,  and  roots).  It  includes  an  animated  pedagogic  agent 
called  Herman— a lovable  alien  with  appealing  facial  expressions,  human-like  movements,  and  an  amusing  voice, 
who  offers  individualized  advice  concerning  the  relation  between  plant  features  and  environmental  features, 
encouragement  when  students  encounter  difficulties,  and  feedback  on  the  choices  that  students  make  in  the  process 
of  designing  plants.  Herman's  actions  are  dynamically  selected  and  assembled  by  a behavior  sequencing  engine  that 
guides  the  presentation  of  problem-solving  advice  to  learners,  as  described  more  fully  in  Lester  et  al.  (1997).  The 
program  starts  with  Herman  introducing  the  student  to  the  first  set  of  environmental  conditions.  Then,  he  asks  the 
student  to  choose  the  appropriate  root  from  the  library  of  roots'  names  and  graphics  shown  on  the  computer  screen. 
After  the  student  had  chosen  a root,  Herman  gives  two  explanations:  first,  a narrated  explanation,  and  second  a 
QuickTime  movie  that  contains  a similar  explanation  for  the  correct  root.  The  same  procedure  applies  to  the  stem 


and  leaves,  with  Herman  first  asking  the  student  to  make  a choice,  and  giving  the  student  feedback  afterwards. 
Once  the  student  was  given  the  last  explanation  on  the  leaves  for  each  environment,  he  is  taken  to  the  next 
environment.  The  same  procedure  follows  for  the  rest  of  the  environments. 


Issue  1:  Agent  versus  Text-Based  Environments 

Does  learning  with  an  animated  pedagogical  agent  in  a discovery  environment  promote  constructivist 
learning?  In  order  to  help  answer  this  question,  we  conducted  a preliminary  study  where  the  learning  outcomes  of 
students  who  learned  about  environmental  science  in  the  Design-A-Plant  microworld  (personal  agent  or  PA  group) 
was  compared  with  the  learning  outcomes  of  students  who  received  the  identical  verbal  and  visual  materials  in  a 
computer-based  text  environment  (no  personal  agent  or  no-PA  group). 

Method  and  Results 

The  participants  were  44  college  students  from  the  Psychology  Subject  Pool  at  the  University  of  California, 
Santa  Barbara.  Twenty-four  participants  served  in  the  no-PA  group  and  twenty  participants  served  in  the  PA  group. 
The  computerized  materials  consisted  of  two  multimedia  computer  programs  on  how  to  design  a plant.  The  PA 
version  was  the  original  multimedia  program  called  "Design-a-Plant",  where  students  see  a library  of  plant  parts, 
pick  the  plant  part  that  they  consider  appropriate  for  the  respective  environment,  and  receive  spoken  feedback  in  a 
conversational  style  from  the  agent  (Lester  & Stone  1997,  Lester  et  al.  1997).  The  no-PA  version  was  presented  with 
the  same  library  plant  parts  and  explanations  than  the  PA  version,  but  Herman's  image  was  deleted.  In  addition, 
students  in  the  no-PA  version  were  not  allowed  to  design  the  plant  before  reading  the  verbal  explanations,  but  rather 
received  the  explanation  directly  in  a monologue  style,  similar  to  when  science  material  is  read  from  a text  book. 

The  participants  were  randomly  assigned  to  a treatment  group  and  seated  at  an  individual  cubicle  in  front 
of  a computer.  First,  students  were  given  a questionnaire,  which  solicited  participant’s  demographic  information. 
Second,  the  respective  version  of  the  multimedia  program  was  presented.  After  the  respective  program  was  over, 
participants  were  tested  on  three  important  measures  of  learning:  retention— in  which  we  assessed  memory  for  the 
basic  factual  information  that  was  presented;  problem-solving  transfer-in  which  we  asked  students  to  solve  new 
problems  based  on  the  principles  learned  in  the  respective  program;  and  motivation  and  interest— in  which  we  asked 
students  to  rate  their  level  of  motivation,  interest,  understanding,  and  the  perceived  difficulty  and  friendliness  of  the 
lesson.  We  determined  whether  the  groups  differed  on  measures  of  retention,  transfer,  and  self-ratings. 

Do  students  who  learned  interactively  with  a pedagogical  agent  show  deeper  understanding  from  a multimedia 
science  lesson  than  students  who  learned  in  a conventional  environment? 

Students  in  the  PA  and  no-PA  groups  received  identical  factual  information  consisting  of  the  same  verbal  and  visual 
materials.  As  a retention  test,  we  simply  asked  students  to  name  as  many  types  of  roots,  stems,  and  leaves  as  they 
could-highly  relevant  information  that  was  prominently  displayed  in  both  instructional  programs.  The  mean 
number  of  correctly  recalled  items  by  the  PA  group  was  not  significantly  different  than  the  mean  number  of 
correctly  recalled  items  by  the  no-PA  group.  The  results  suggest  that  when  retention  of  factual  information  is  the 
goal  of  instruction,  then  environments,  which  allow  for  interacting  with  an  animated  pedagogical  agent  are  not 
warranted.  Although  these  results  demonstrate  that  both  groups  learned  the  basic  factual  information,  the  major 
prediction  in  this  study  is  that  students  in  the  PA  group  will  learn  more  deeply  than  no-PA  students.  As  predicted, 
students  in  the  PA  group  produced  significantly  more  correct  solutions  on  transfer  problems  than  students  in  the  no- 
PA  group  (p  < .005). 

The  foregoing  results  are  consistent  with  the  idea  that  students  who  learn  with  animated  pedagogic  agents 
work  harder  to  make  sense  of  the  material  than  do  students  who  learn  in  a more  conventional  text-based 
environment.  Another  way  to  test  this  idea  is  to  assess  students'  motivation  to  continue  learning  and  their  interest  in 
the  material,  so  we  predicted  that  the  PA  group  would  produce  a higher  mean  program  rating  than  the  no-PA  group. 
As  predicted,  the  PA  group  rated  their  motivation  to  continue  learning  and  their  interest  in  the  material  significantly 
higher  than  the  no-PA  group  (p  < .01). 


Conclusion 


This  preliminary  study  contributes  to  multimedia  learning  theory  (Mayer  1997)  by  comparing  the  cognitive 
consequences  of  students'  experience  with  or  without  the  aid  of  a pedagogical  agent.  Students  in  both  groups 
received  identical  verbal  and  visual  material  presented  in  the  context  of  social  interaction  with  a computer-mediated 
character  (PA  group)  or  in  an  non-personalized  text  context  (no-PA  group).  These  findings  give  preliminary 
evidence  in  favor  of  using  pedagogical  agents  as  software  mentors,  and  demonstrate  a personal-agent  ejfect  in 
multimedia  learning  environments:  Students  are  more  motivated,  interested,  and  achieve  better  transfer  when  the 
lesson  is  imparted  by  a pedagogic  agent  rather  than  by  on-screen  non-personalized  text. 

How  do  agents  affect  student  learning?  Reeves  and  Nass  (1996)  have  provided  convincing  evidence  that 
students  view  their  contact  with  computer-based  characters  as  social  interactions.  Congruent  with  this  approach, 
students’  learning  with  the  pedagogical  agent  could  have  been  promoted  by  at  least  three  social-cues:  the 

pedagogical  agents’  image,  voice,  and  personalized  language  style.  Therefore,  we  investigated  first,  the  role  of 
pedagogical  agents’  auditory  and  visual  presence  in  a discovery-based  computer  lesson.  For  this  purpose,  we  varied 
whether  the  agent’s  words  were  presented  as  speech  or  on-screen  text  and  whether  or  not  the  agent’s  image  appeared 
on  the  screen,  both  with  an  animated  fictional  agent  (Experiment  1),  and  a video  of  a human  face  (Experiment  2). 
Second,  we  investigated  the  role  of  the  agents'  language  style  in  the  computer  lesson.  For  this  purpose  we  varied 
whether  or  not  the  agent's  words  were  presented  in  conversational  style  (i.  e.  as  dialogue  or  monologue)  both  using 
speech  (Experiment  3),  and  on-screen  text  (Experiment  4). 


Issue  2:  The  Role  of  Life-Like  Pedagogical  Agents’  Visual  and  Auditory  Presence 

We  tested  the  predictions  arising  from  two  opposing  hypothesis:  the  social-cue  hypothesis,  and  the 
cognitive  load  hypothesis.  According  to  the  former,  students  communicate  better,  become  more  interested,  and 
therefore  learn  better  and  rate  more  favorably  computer  lessons  that  include  social  cues—such  as  facial  expressions 
or  human  voices,  than  those  which  do  not  include  social  cues  (Dewey  1913,  Rutter  1984).  Three  predictions  can  be 
made  for  students  who  learn  a multimedia  lesson  with  the  image  of  an  agent.  First,  they  will  be  more  likely  to 
remember  the  materials  of  the  lesson  than  students  who  are  not  provided  with  the  visual  presence  of  the  agent;  this 
we  call  an  image  effect  on  retention.  Second,  they  will  be  more  likely  to  use  what  they  have  learned  to  solve 
problems  than  students  who  are  not  provided  with  the  visual  presence  of  the  agent;  this  we  call  an  image  effect  on 
transfer.  Third,  they  will  rate  the  lesson  more  favorably  as  expressed  in  the  program  ratings  than  students  who  are 
not  provided  with  the  visual  presence  of  the  agent;  this  we  call  an  image  effect  on  program  rating.  In  contrast, 
according  to  a cognitive  load  of  learning  (Chandler  & Sweller  1991,  Moreno  & Mayer  in  press,  Sweller  1989),  the 
inclusion  of  irrelevant  adjuncts  in  a science  lesson  will  divert  the  limited  cognitive  resources  available  for  the 
processing  of  the  relevant  materials  of  the  lesson.  As  a result,  learning  and  problem  solving  will  be  impaired. 

An  additional  goal  of  the  present  study  was  to  investigate  the  role  of  pedagogical  agents’  auditory  presence 
in  a constructivist  multimedia  lesson.  We  investigated  if  prior  findings  on  modality  effects  in  Ieaming--where 
students  learn  better  from  visual  and  verbal  presentations  when  the  verbal  information  is  presented  as  speech  rather 
than  as  on-screen  text,  extend  to  discovery-based  multimedia  environments  (Mayer  & Moreno  1998,  Moreno  & Mayer 
1999,  Mousavi,  Low,  8c  Sweller  1995).  Congruent  with  past  research  on  modality  effects  in  multimedia  learning,  two 
predictions  can  be  made.  First,  students  who  learn  with  the  voice  of  an  agent  will  be  more  likely  to  remember  the 
materials  of  the  lesson  than  students  who  leam  the  same  verbal  materials  as  on-screen  text;  this  we  call  a modality 
effect  on  retention.  Second,  students  who  learn  with  the  voice  of  an  agent  will  be  more  likely  to  use  what  they  have 
learned  to  solve  problems  than  students  who  learn  the  same  verbal  materials  as  on-screen  text;  this  we  call  a 
modality  effect  on  transfer.  An  additional  prediction  was  made  based  on  the  social-cue  hypothesis  (Dewey  1913). 
Students  who  leam  with  the  voice  of  an  agent  will  be  more  motivated  and  will  rate  the  lesson  more  favorably  than 
students  who  leam  the  same  verbal  materials  as  on-screen  text;  this  we  call  a modality  effect  on  program  rating. 

Method  and  Results 

In  the  first  study,  17  students  learned  by  interacting  with  the  image  of  a life-like  fictional  agent  who  spoke 
to  them  (Group  IN),  16  students  learned  by  interacting  without  the  image  of  an  agent  who  spoke  to  them  (Group  - 
IN);  15  students  learned  by  interacting  with  the  image  of  a life-like  fictional  agent  who  gave  explanations  as  on- 
screen text  (Group  IT);  and  16  students  learned  by  interacting  without  the  image  of  an  agent  who  gave  explanations 
as  on-screen  text  (Group  -IT).  The  second  study  was  identical  but  the  video  and  voice  of  a human  agent  replaced 
the  fictional  agent.  Nineteen  students  participated  in  the  IN  group,  20  students  participated  in  the  -IN  group,  19 


students  participated  in  the  IT  group,  and  21  students  participated  in  the  -IT  group.  The  procedure  was  identical  to 
that  of  the  preliminary  study. 

Do  students  who  are  presented  with  the  image  of  a pedagogical  agent  show  deeper  understanding  from  a 
multimedia  science  lesson  than  students  who  are  not  presented  with  the  image  ? 

The  findings  from  Experiments  1 and  2 did  not  provide  evidence  in  favor  or  against  presenting  students  with  the 
image  of  a pedagogical  agent,  failing  to  confirm  what  we  have  called  an  image  effect  in  program  ratings,  recall,  and 
transfer:  Students  who  are  presented  with  the  image  of  an  agent  do  not  rate  the  lesson  more  favorable,  recall  more, 
or  are  better  able  to  use  what  they  have  learned  to  solve  problems  than  students  who  are  not  presented  with  the 
visual  presence  of  the  agent. 

Do  students  who  communicate  with  a pedagogical  agent  via  speech  show  deeper  understanding  from  a multimedia 
science  lesson  than  students  who  communicate  with  a pedagogical  agent  via  on-screen  text? 

The  findings  from  Experiments  1 and  2 gave  evidence  in  favor  of  students'  communicating  with  a pedagogical  agent 
by  means  of  speech  by  demonstrating  what  we  have  called  a modality  effect  in  program  ratings,  recall,  and  transfer: 
Students  who  learn  with  the  voice  of  an  agent  rate  the  lesson  more  favorably,  recall  more,  and  are  better  able  to  use 
what  they  have  learned  to  solve  problems  than  students  who  learn  the  same  verbal  materials  as  on-screen  text.  In 
both  experiments,  the  mean  program  ratings  for  the  narration  groups  was  significantly  higher  than  the  mean  program 
ratings  for  the  text  groups  (p  < 0.05  for  both  experiments);  the  narration  groups  recalled  significantly  more  than  the 
text  groups  (p  < .005  and  p < 0.005  for  Experiments  1 and  2,  respectively);  and  the  narration  groups  gave 
significantly  more  correct  answers  in  the  transfer  tests  than  the  text  groups  (p  < .0005  and  p = .0001  for  Experiments 
1 and  2,  respectively).  These  results  extend  the  modality  effect  in  learning  from  visual  and  verbal  materials  to 
discovery-based  multimedia  environments  and  confirm  the  social-cue  prediction  according  to  which  students  are 
more  motivated  and  interested  in  programs,  which  communicate  via  speech. 


Issue  3:  The  Role  of  Life-Like  Pedagogical  Agents*  Language  Style 

Regarding  the  language  style  used  by  the  pedagogical  agent,  we  tested  the  opposing  predictions  from  the 
transmission  hypothesis  and  the  conversational  hypothesis.  According  to  the  former,  human  communication 
involves  encoding  an  idea  into  a signal  by  a sender,  transmitting  the  signal  to  the  receiver,  and  decoding  the  signal 
by  the  receiver  (Reddy  1979).  Consequently,  as  the  content  material  is  identical  when  the  language  style  is  changed 
from  a dialogue  style  to  a monologue  style,  it  should  not  affect  students'  learning.  In  contrast,  the  conversational 
hypothesis  views  communication  as  inherently  cooperative  and  claims  that  a uni-directional  model  of 
communication  such  as  the  one  supported  by  the  transmission  hypothesis,  fails  to  capture  the  social  and  individual 
processes  involved  in  knowledge  construction  through  conversations  (Brennan  1990)..  This  hypothesis  predicts  that 
students  will  learn  deeper  from  instructional  conversations  rather  than  from  non-personal ized  monologue-style 
communications. 

Method  and  Results 

In  the  third  study,  18  students  learned  with  a personalized  conversation  spoken  by  the  human  agent  used  in 
Experiments  1 and  2,  (Group  P)  and  21  students  learned  with  a non-personalized  monologue  spoken  by  the  same 
human  agent  (Group  NP).  For  the  fourth  study,  21  students  learned  with  a personalized  conversation  displayed  as 
text  (Group  P)  and  21  students  learned  with  a non-personalized  monologue  displayed  as  text  (Group  NP).  In  both 
studies  the  image  of  the  agent  was  not  presented.  The  procedure  was  identical  to  that  of  the  prior  studies. 

Do  students  who  communicate  with  a pedagogical  agent  via  a personalized  dialogue  show  deeper  understanding 
from  a multimedia  science  lesson  than  students  who  communicate  with  a pedagogical  agent  via  an  non-personalized 
monologue? 

The  findings  from  Experiments  3 and  4 gave  evidence  in  favor  of  students'  communicating  with  a pedagogical  agent 
by  means  of  a personalized  conversation  by  demonstrating  what  w have  called  a dialogue  effect  in  recall  and 
transfer:  Students  who  learn  by  communicating  with  a pedagogic  agent  via  a personalized  dialogue  recall  more  and 


are  better  able  to  use  what  they  have  learned  to  solve  problems  than  students  who  communicate  via  a non- 
personalized  monologue.  The  mean  number  of  ideas  recalled  for  dialogue  groups  was  significantly  larger  than  for 
monologue  groups  (p  < .005  and  p < .05  for  Experiments  3 and  4,  respectively)  and  the  mean  number  of  correct 
answers  in  the  transfer  test  was  significantly  larger  for  the  dialogue  groups  than  for  the  monologue  groups  (p  < 
.0001  for  both  experiments).  In  addition,  Experiment  4 demonstrated  a dialogue  effect  for  program  ratings. 
Students  who  learned  by  reading  dialogue-style  text  rated  more  favorably  the  lesson  than  students  who  learned  by 
reading  monologue-style  text  (p  = 0.05). 


General  Discussion 

The  reported  results  have  important  theoretical  and  practical  implications. 

First,  the  modality  effect  found  in  Experiments  1 and  2 might  be  explained  as  a combination  of  the 
superiority  in  recall  when  words  are  processed  auditorily  rather  than  visually  (Penney  1989),  the  relatively  effortless 
maintenance  of  the  auditory  input  in  comparison  to  the  visual  input  provided  by  text  (Anderson  & Craik  1974),  and 
the  extra  emotional  cues,  which  voices  carry  and  text  lacks.  According  to  communications  research,  voices  are 
powerful  indicators  of  social  presence  and  its  incorporation  in  the  interaction  might  promote  richer  processing  by  the 
incorporation  of  the  additional  attitudes  and  beliefs  that  are  attached  to  the  agent  (Reeves  & Nass  1996).  These 
findings  extend  pioneering  demonstrations  of  modality  effects  in  learning  from  visual  and  verbal  materials  (Mayer  & 
Moreno  1998,  Moreno  & Mayer  1999,  Mousavi,  Low,  & Sweller  1995)  in  three  ways:  (a)  by  examining  a modality 
effect  in  an  interactive  constructivist  computer-based  multimedia  environment  rather  than  a paper-based  or  non- 
interactive computer  environment,  (b)  by  employing  multiple  dependent  measures  including  students'  rating  of  the 
learning  materials,  and  (c)  by  using  fictional  and  non-fictional  pedagogic  agents  to  deliver  the  verbal  materials  of  the 
lesson.  The  finding  that  students  rated  the  computer  program  more  favorably  when  it  communicates  via  speech 
rather  than  via  text  is  congruent  with  the  social-cue  hypothesis  according  to  which  the  agents'  voice  may  help 
students  feel  an  emotional  connection  with  the  agent  therefore  promoting  interest  in  the  learning  task  which  in  turn 
fosters  constructivist  learning . 

Second,  the  failure  to  find  an  image  effect  in  Experiments  1 and  2 does  not  support  the  social  cue 
hypothesis  or  the  cognitive  load  hypothesis.  Several  interpretations  for  this  finding  can  be  offered.  First,  the  lack  of 
image  effects  might  have  relied  on  the  strength  of  the  auditory  cues  given  by  the  voice  of  the  agents  which  in  both 
applications  was  extremely  expressive  and  clear.  When  a voice  carries  these  qualities,  it  is  less  likely  that  facial 
expressions  or  lip  movements  will  provide  extra  information,  or  help  disambiguate  the  message  in  the  lesson.  An 
alternative  explanation  may  reside  on  the  specific  content  of  the  computer  lesson:  The  scientific  nature  of  the  lesson 
used  in  the  reported  studies  is  not  essentially  emotional  and  facial  expressions  may  not  offer  any  informational 
advantage. 

Cognitive  load  predicted  that  the  introduction  of  the  agent's  image  in  the  computer  lesson  would  be 
detrimental  to  students'  learning.  This  prediction  was  not  confirmed.  However,  in  our  studies,  the  agent's  image 
was  never  presented  simultaneously  with  other  visual  materials  of  the  lesson.  According  to  cognitive  load  theory,  a 
detrimental  effect  in  learning  occurs  in  the  cases  that  students  need  to  split  their  attention  between  mutually  referring 
materials.  Therefore,  it  is  more  likely  that  if  the  multimedia  lesson  contained  animations  of  the  agent  presented 
simultaneously  with  graphics  or  text,  his  visual  presence  would  have  been  detrimental  rather  than  neutral  to  learning 
(Mayer  & Moreno  1998,  Moreno  & Mayer  1999,  Sweller  1989). 

The  dialogue  effects  on  retention  and  transfer  found  in  Experiments  3 and  4 support  the  conversational 
hypothesis.  We  attribute  these  effects  to  a combination  of  less  cognitive  effort  and  more  active  processing.  First,  it 
might  be  argued  that  students  who  listen  to  non-personal ized  explanations  need  to  work  harder  than  students  who 
listen  to  personalized  explanations  because  they  need  to  map  the  agent's  generalized  explanations  to  their  specific 
choices  in  the  computer  lesson.  Second,  conversations  might  act  as  attention  enhancers  and  produce  more  active 
processing  of  the  materials  (Reeves  & Nass  1996)  by  creating  a state  of  vigilance  in  which  the  participant  might  be 
asked  to  intervene  in  the  dialogue  at  any  time. 

It  is  tempting  to  conclude  that  the  personal  agent  effect  found  in  our  preliminary  study  resided  in  the 
effectiveness  of  teaching  via  instructional  conversations  (Tharp  & Gallimore  1991).  However,  our  focus  in  the  present 
study  was  solely  on  the  role  that  the  agent's  social  cues  may  have  in  a multimedia  constructivist  environment. 
Although  speech  and  language  style  proved  to  help  students  understanding,  other  factors,  such  as  students' 
interaction  with  the  computer  program,  might  play  an  important  role  in  learning  as  well.  More  research  is  needed  to 
determine  the  contribution  of  alternative  interactive  designs  in  promoting  deep  learning  from  constructivist 
environments. 


The  most  direct  practical  implication  of  our  studies  is  that  in  a constructivist  science  lesson  where  students 
learn  with  the  help  of  a pedagogical  agent,  technological  advances  in  education  should  focus  on  extending  the 
capabilities  of  speech  recognition  and  natural  language  systems  to  facilitate  instructional  conversations.  Moreover, 
the  reported  findings  suggest  that  an  agent  interface  metaphor  might  already  be  present  once  students  interact  with  a 
computer,  and  trying  to  make  it  more  visually  apparent,  may  not  necessarily  lead  to  better  results. 
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Abstract:  How  can  we  help  students  understand  scientific  systems?  One  promising  approach 
involves  multimedia  presentations  of  explanations  in  visual  and  verbal  formats,  such  as  presenting 
a computer-generated  animation  synchronized  with  narration  or  on-screen  text.  In  this  paper,  we 
present  a cognitive  theory  of  multimedia  learning  from  which  principles  of  instructional  design  are 
derived  and  tested.  The  split-attention  principle  states  that  learning  is  impaired  when  the 
presentation  requires  the  learner  to  mentally  integrate  disparate  sources  of  information  before  the 
instructional  material  can  be  rendered  intelligible.  The  spatial  contiguity  principle  states  that  it  is 
better  to  present  words  and  graphics  integrated  rather  than  physically  separated.  The  temporal 
contiguity  principle  states  that  students  learn  better  when  corresponding  words  and  pictures  are 
presented  simultaneously  rather  than  successively.  The  modality  principle  states  that  it  is  better  to 
present  words  auditorily  as  narration  than  visually  as  on-screen  text  The  coherence  principle  states 
that  adding  sufficient  amounts  of  entertaining  but  irrelevant  words  or  sounds  to  a multimedia 
instructional  message  is  detrimental  to  student  learning. 


Designing  a Multimedia  Explanation 

Humans  can  integrate  information  from  different  sensory  modalities  into  one  meaningful  experience— such 
as  when  they  associate  the  sound  of  thunder  to  the  visual  image  of  lightning  in  the  sky.  They  can  also  integrate 
information  from  different  modes  (verbal  or  non-verbal)  into  a mental  model— such  as  when  they  watch  lightning  in 
the  sky  and  listen  to  an  explanation  of  the  event.  Therefore,  when  the  process  of  lightning  formation  is  to  be  taught 
with  a multimedia  explanation,  the  instructional  designer  is  faced  with  the  need  to  choose  between  several 
combinations  of  modes  and  modalities  to  promote  meaningful  learning  (Mayer  & Moreno,  1998).  Should  the 
explanation  be  given  auditorily  in  the  form  of  speech,  visually  in  the  form  of  text,  or  both?  Would  entertaining 
adjuncts  in  the  form  of  words,  environmental  sounds,  or  music  help  students'  learning?  Should  the  visual  and 
auditory  materials  be  presented  simultaneously  or  sequentially?  To  help  answer  these  questions,  we  conducted  a 
series  of  studies  where  the  learning  outcomes  of  students  who  viewed  alternative  presentation  formats  was 
compared.  In  particular,  we  concentrated  on  three  learning  measures:  retention  of  the  steps  in  the  cause-and-effect 
chain  (retention  test),  matching  of  the  pictures  and  names  of  parts  (matching  test),  and  using  what  they  have  learned 
to  solve  problems  (transfer  test).  By  beginning  with  a cognitive  theory  of  how  learners  process  multimedia 
information,  we  have  been  able  to  conduct  focused  research,  which  has  yielded  some  preliminary  principles  of 
instructional  design  for  multimedia  explanations. 


A Cognitive  Theory  of  Multimedia  Learning 


Our  cognitive  theory  of  multimedia  learning  draws  on  dual  coding  theory,  cognitive  load  theory,  and 
constructivist  learning  theory.  It  is  based  on  the  following  assumptions:  (a)  working  memory  includes  an  auditory 
working  memory  and  a visual  working  memory,  analogous  to  the  phonological  loop  and  visuo-spatial  sketch  pad, 
respectively,  in  Baddeley’s  (1986)  theory  of  working  memory;  each  working  memory  store  has  a limited 
capacity,  consistent  with  Sweller’s  (1988,  1994;  Chandler  & Sweller,  1992)  cognitive  load  theory;  (c)  meaningful 
learning  occurs  when  a learner  retains  relevant  information  in  each  store,  organizes  the  information  in  each  store 
into  a coherent  representation,  and  makes  connections  between  corresponding  representations  in  each  store, 
analogous  to  the  cognitive  processes  of  selecting,  organizing,  and  integrating  in  Mayer's  generative  theoiy  of 
multimedia  learning  (Mayer,  1997);  and  (d)  connections  can  be  made  only  if  corresponding  pictorial  and  verbal 
information  is  in  working  memory  at  the  same  time,  corresponding  to  referential  connections  in  Paivio’s  (1986) 
dual-coding  theory. 


The  Split-Attention  Principle 

How  should  verbal  information  be  presented  to  students  to  enhance  learning  from  animations:  auditorily  as 
speech  or  visually  as  on-screen  text?  In  order  to  answer  this  question,  Mayer  and  Moreno  (1998)  asked  students  to 
view  an  animation  depicting  the  process  of  lightning  either  along  with  concurrent  narration  (Group  AN)  or  along 
with  concurrent  on-screen  text  (Group  AT). 

According  to  the  cognitive  theory  of  multimedia  learning,  in  the  AN  treatment,  students  represent  the 
animation  in  visual  working  memoiy  and  represent  the  corresponding  narration  in  auditoiy  working  memory. 
Because  they  can  hold  corresponding  pictorial  and  verbal  representations  in  working  memoiy  at  the  same  time, 
students  in  group  AN  are  better  able  to  build  referential  connections  between  them. 

In  the  AT  treatment,  students  try  to  represent  both  the  animation  and  the  on-screen  text  in  visual  working 
memory.  Although  some  of  the  visually-represented  text  eventually  may  be  translated  into  an  acoustic  modality  for 
auditory  working  memory,  visual  working  memory  is  likely  to  become  overloaded.  If  students  pay  full  attention  to 
on-line  text  they  may  miss  some  of  the  crucial  images  in  the  animation,  but  if  they  pay  full  attention  to  the  animation 
they  may  miss  some  of  the  on-line  text.  Because  they  may  not  be  able  to  hold  corresponding  pictorial  and  verbal 
representations  in  working  memory  at  the  same  time,  students  in  group  AT  are  less  able  to  build  connections 
between  these  representations.  Therefore,  our  theory  predicts  that  students  in  group  AT  perform  more  poorly  than 
students  in  group  AN  on  the  retention,  matching,  and  transfer  tests. 

Method  and  Results 

The  participants  were  78  college  students  who  lacked  knowledge  of  meteorology.  They  first  viewed  the 
animation  with  either  concurrent  narration  in  a male  voice  describing  8 major  steps  in  lightning  formation  (Group 
AN)  or  concurrent  on-screen  text  involving  the  same  words  and  presentation  timing  (Group  AT).  Then,  all  students 
took  the  retention,  transfer  and  matching  tests. 

Group  AN  recalled  significantly  (p<  .001)  more,  correctly  matched  significantly  (p  < .01)  more  elements 
on  diagrams,  and  generated  significantly  (p<  .001)  more  correct  solutions  than  Group  AT.  These  results  are 
consistent  with  the  predictions  of  the  cognitive  theory  of  multimedia  learning  and  allow  us  to  infer  the  first 
instructional  design  principle,  called  the  split-attention  principle  by  the  cognitive  load  theory  (Chandler  & Sweller, 
1992;  Mousavi,  Low,  & Sweller,  1995). 

Split-Attention  Principle 

Students  learn  better  when  the  instructional  material  does  not  require  them  to  split  their  attention  between  multiple 
sources  of  mutually  referring  information. 

The  Modality  Principle 

Why  do  students  learn  better  when  verbal  information  is  presented  auditorily  as  speech  rather  than  visually 
as  on-screen  text?  Mayer  and  Moreno’s  (1998)  study  showed  that  students  who  learn  with  concurrent  narration  and 
animations  outperform  those  who  learn  with  concurrent  on-screen  text  and  animations.  However,  concurrent 


multimedia  presentations,  force  the  text  groups  to  hold  material  from  one  source  of  information  (verbal  or  non- 
verbal) in  working  memory  before  attending  to  the  other  source.  Therefore,  the  narration  group  might  have  had  the 
advantage  of  being  able  to  attend  to  both  sources  simultaneously,  and  the  superior  performance  might  disappear  by 
using  sequential  multimedia  presentations,  where  verbal  and  non-verbal  materials  are  presented  one  after  the  other. 
The  next  logical  step  was  to  test  if  the  advantage  of  narration  over  on-screen  text  resides  in  a modality  principle 
(Moreno  & Mayer,  1999,  Experiment  2).  If  this  is  the  case,  then  the  advantage  for  auditory-visual  presentations 
should  not  disappear  when  they  are  made  sequential. 

Method  and  Results 

The  participants  were  1 37  college  students  who  lacked  knowledge  of  meteorology.  They  first  viewed  the 
animation  in  one  of  the  following  six  conditions,  one  group  of  students  viewed  concurrently  on-screen  text  while 
viewing  the  animation  (TT),  a second  group  of  students  listened  concurrently  to  a narration  while  viewing  the 
animation  (NN),  a third  group  of  students  listened  to  a narration  preceding  the  corresponding  portion  of  the 
animation  (NA),  a fourth  group  listened  to  the  narration  following  the  animation  (AN),  a fifth  group  read  the  on- 
screen text  preceding  the  animation  (TA),  and  the  sixth  group  read  the  on-screen  text  following  the  animation  (AT). 
After  viewing  the  animation,  students  took  the  retention,  transfer,  and  matching  tests. 

The  text  groups  (TT,  AT,  and  TA)  scored  significantly  lower  than  the  narration  groups  (NN,  AN,  and  NA) 
in  verbal  recall  (p  < .001),  problem  solving  transfer  (p  < .001),  and  matching  (p  < .005).  These  results  reflect  a 
modality  effect.  Within  each  modality  group,  only  the  simultaneous  and  sequential  text  showed  a significant 
difference  in  their  performance  for  matching  tests  (p  < .05),  with  the  sequential  text  groups  outperforming  the 
simultaneous  text  group.  This  finding  might  be  interpreted  as  another  example  of  the  split-attention  effect,  where 
presenting  two  competing  visual  materials  simultaneously  has  negative  effects  on  the  matching  of  verbal  and  visual 
materials  in  a multimedia  lesson.  The  results  from  this  study  are  consistent  with  prior  studies  on  text  and  diagrams 
(Mousavi,  Low,  & Sweller,  1995),  and  allow  us  to  infer  a second  instructional  design  principle-the  Modality 
Principle. 

Modality  Principle 

Students  learn  better  when  the  verbal  information  is  presented  auditorily  as  speech  rather  than  visually  as  on-screen 
text  both  for  concurrent  and  sequential  presentations. 


The  Spatial  Contiguity  Principle 

How  does  the  physical  integration  of  on-screen  text  and  visual  materials  affect  students’  learning?  Mayer 
and  Moreno’s  study  on  split-attention  (1998)  showed  that  students  who  learn  with  concurrent  narration  and 
animations  outperform  those  who  learn  with  concurrent  on-screen  text  and  animations.  One  interpretation  for  this 
result  is  that  students  might  be  missing  part  of  the  visual  information  while  they  are  reading  the  on-screen  text  (or 
vice  versa).  In  a review  of  ten  studies  concerning  whether  multimedia  instruction  is  effective,  Mayer  (1997) 
concluded  that  there  was  a consistent  evidence  for  what  was  called  a spatial-contiguity  effect.  Students  generated  a 
median  of  over  50%  more  creative  solutions  to  transfer  problems  when  verbal  and  visual  explanations  were 
integrated  than  when  they  were  separated  (Mayer,  1997).  In  order  to  extend  these  studies  to  multimedia  learning 
with  animations,  we  conducted  a study  where  the  physical  proximity  of  the  on-screen  text  and  the  animation  was 
manipulated  (Moreno  & Mayer,  1999,  Experiment  1).  One  group  of  students  had  on-screen  text  that  was  integrated 
or  physically  close  to  the  animation  (IT  group)  while  a second  group  of  students  had  on-screen  text  that  was 
separated  or  physically  far  from  the  animation  (ST  group).  A third  group  of  students  saw  a presentation  with 
concurrent  animation  and  narration  (N  group).  This  study  allows  to  interpret  performance  differences  between  the 
text  groups  (IT  and  ST)  in  terms  of  spatial-contiguity  and  performance  difference  between  the  narration  (N)  and  text 
groups  (IT  and  ST)  in  terms  of  modality. 

Method  and  Results 

The  participants  were  132  college  students  who  lacked  knowledge  of  meteorology.  They  first  viewed  the 
animation  in  one  of  the  following  three  conditions:  N,  ST  or  IT.  One  group  of  students  listened  to  concurrent 
narration  describing  each  of  the  major  events  in  words  spoken  at  a slow  rate  by  a male  voice  (Group  N)  and  the 
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other  two  groups  of  students  learned  by  reading  concurrent  on-screen  text  using  the  same  words  and  timing  as  the 
narration  used  in  the  N group.  For  the  separated  text  version  (Group  ST),  the  text  was  physically  far  from  the 
animation  and  for  the  integrated  text  version  (Group  IT),  the  text  was  physically  close  to  the  relevant  part  of  the 
animation.  After  viewing  the  animation,  all  students  took  the  retention,  transfer  and  matching  tests. 

The  N group  scored  significantly  higher  than  the  IT  and  ST  groups  in  the  verbal  recall,  and  transfer  tests, 
with  the  IT  group  scoring  significantly  higher  than  the  ST  group  (p  < .001).  These  results  are  evidence  for  both,  a 
modality  and  a spatial-contiguity  effect  in  recall  and  transfer.  For  matching  tests,  students  in  group  N group  scored 
significantly  higher  than  the  IT  and  ST,  which  did  not  differ  from  each  other  (p  < .05).  These  results  are  evidence 
for  a modality  effect  in  matching  but  not  for  a spatial-contiguity  effect.  One  possible  explanation  for  not  finding  a 
significant  difference  in  the  matching  scores  of  the  IT  and  ST  groups  is  a ceiling  effect  in  which  all  groups  did  very 
well  on  this  particular  measure.  Overall,  it  is  possible  to  infer  a third  instructional  design  principle  that  extends  prior 
findings  on  text  and  diagrams  to  multimedia  learning  with  animations-the  Spatial  Contiguity  Principle. 

Spatial  Contiguity  Principle 

Students  learn  better  when  on-screen  text  and  visual  materials  are  physically  integrated  rather  than  separated. 


The  Temporal  Contiguity  Principle 

How  synchronized  in  time  do  the  verbal  and  visual  materials  have  to  be  for  students  to  learn  better  from 
animations?  Congruent  with  a dual-processing  model  of  working  memory,  constructivist  learning  is  fostered  when 
the  learner  is  able  to  hold  a visual  representation  in  visual  working  memory  and  a corresponding  verbal 
representation  in  verbal  working  memory  at  the  same  time.  The  model  implicates  working  memory  load  as  a major 
impediment  to  learning.  Although  all  learners  may  receive  identical  animation  and  narration  in  a multimedia 
environment,  the  amount  of  information  that  they  are  forced  hold  in  working  memory  at  one  time  might  affect 
performance.  For  example,  presenting  the  whole  animation  preceded  or  followed  by  the  whole  narration 
successively  (which  we  will  call  large  bites),  can  overload  working  memory  such  that  it  is  not  possible  to  hold  all  of 
the  narrative  in  working  memory  until  the  animation  is  presented  (or  vice  versa).  However,  if  we  present  one  chunk 
of  animation— depicting  only  a short  sequence-preceded  or  followed  by  one  corresponding  chunk  of  narration  and 
so  on  (which  we  will  call  small  bites),  and  if  the  size  of  the  chunk  does  not  exceed  working  memory  capacity,  then 
the  learner  should  be  able  to  make  connections  between  the  corresponding  words  and  pictures— in  the  same  way  as 
when  animation  and  narration  are  presented  concurrently. 

Our  prior  study  (Moreno  & Mayer,  1 999)  failed  to  find  performance  differences  between  the  simultaneous 
and  sequential  narration  groups.  Similar  results  were  obtained  for  learning  with  geometry  examples  by 
simultaneous  and  sequential  explanations  (Mousavi,  Low  & Sweller,  1995).  In  both  cases,  the  successive 
presentations  of  the  animation  and  narration  were  small  bites,  only  a line  or  two  at  a time,  and  thus  unlikely  to 
overload  working  memory.  However,  in  another  set  of  studies  (Mayer  & Anderson,  1991,  1992),  where  large  bites 
of  animation  and  narrations  were  presented  successively,  students  in  the  simultaneous  conditions  outperformed 
students  who  learned  with  sequential  presentations.  In  order  to  reconcile  the  above  findings,  we  varied  the  size  of 
the  units  that  were  presented  sequentially  to  the  learners  (Mayer,  Moreno,  Boire  & Vagge,  1999).  One  group  of 
students  saw  a presentation  with  concurrent  animation  and  narration  (concurrent  group),  a second  group  of  students 
either  viewed  the  whole  animation  followed  or  preceded  by  the  whole  explanatory  narration  (large  bites  group)  and 
a third  group  of  students  either  viewed  small  chunks  of  animation  followed  or  preceded  by  small  chunks  of 
explanatory  narration  (small  bites  group). 

Method  and  Results 

The  participants  were  60  college  students  who  lacked  knowledge  of  meteorology.  They  first  viewed  the 
animation  in  one  of  the  following  conditions:  concurrent  narration  and  animation,  large  bites  of  narration  and 
animation  or  small  bites  of  narration  and  animation.  As  in  our  prior  studies,  after  viewing  the  animation,  all  students 
took  the  retention,  transfer  and  matching  tests. 

The  large  bites  group  scored  significantly  lower  than  the  concurrent  and  small  bites  groups  in  the  verbal 
recall,  matching  and  transfer  tests,  (p  = .002,  p < .0001,  and  p < .0001,  respectively).  The  concurrent  and  small  bites 
groups  did  not  differ  from  each  other.  These  results  allow  all  prior  studies  in  temporal  contiguity  to  be  reconciled  if 
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the  length  of  the  cycles  in  the  sequential  presentations  are  taken  into  account.  Therefore,  a fourth  instructional 
design  principle  for  multimedia  learning  with  animations  can  be  inferred. 

Temporal  Contiguity  Principle 

Students  learn  better  when  verbal  and  visual  materials  are  temporally  synchronized  rather  than  separated  in  time. 


The  Coherence  Principle 

Would  entertaining  adjuncts  in  the  form  of  sounds,  or  music  help  students’  learning  from  a multimedia 
explanation?  According  to  the  cognitive  theory  of  multimedia  learning,  learners  process  multimedia  messages  in 
their  visual  and  auditory  channel s--both  of  which  are  limited  in  capacity.  In  the  case  of  a narrated  animation,  the 
animation  is  processed  in  the  visual  channel  and  the  narration  is  processed  in  the  auditory  channel.  When  additional 
auditory  information  is  presented,  it  competes  with  the  narration  for  limited  processing  capacity  in  the  auditory 
channel.  When  processing  capacity  is  used  to  process  the  music  and  sounds,  there  is  less  capacity  available  for 
paying  attention  to  the  narration,  organizing  it  into  a coherent  cause -and-effect  chain,  and  linking  it  with  the 
incoming  visual  information.  Based  on  this  theory,  we  can  predict  that  adding  interesting  music  and  sounds  to  a 
multimedia  presentation  will  result  in  poorer  performance  on  tests  of  retention,  matching,  and  transfer.  In  short,  the 
cognitive  theory  of  multimedia  learning  predicts  a coherence  effect  in  which  adding  interesting  material— in  the  form 
of  music  and  sounds— hurts  student  learning.  In  order  to  test  this  prediction,  Moreno  and  Mayer  (in  press)  asked 
students  to  view  an  animation  depicting  the  process  of  lightning  either  with  concurrent  narration  (Group  N),  with 
concurrent  narration  and  environmental  sounds  (Group  NE),  with  concurrent  narration  and  music  (Group  NM),  or 
with  concurrent  narration,  environmental  sounds  and  music  (Group  NEM). 

Method  and  Results 

The  participants  were  75  college  students  who  lacked  knowledge  of  meteorology.  They  first  viewed  the 
animation  in  one  of  the  four  conditions:  narration,  narration  and  sounds,  narration  and  music,  or  narration,  sounds 
and  music.  After  viewing  the  animation,  all  students  took  the  retention,  transfer  and  matching  tests. 

Students  scored  significantly  lower  in  both  retention  and  transfer  tests  when  music  had  been  presented  than 
when  no  music  had  been  presented  (p  < .0001);  but  there  was  no  significant  difference  between  students  who 
received  environmental  sounds  and  those  who  didn’t.  There  was  a significant  interaction  between  music  and  sounds 
(p  < .05),  in  which  the  combination  of  music  and  environmental  sounds  (Group  NEM)  was  particularly  detrimental 
to  retention  and  transfer  performance.  Supplemental  tests  indicated  that  students  in  Group  NEM  had  retention  and 
transfer  scores  significantly  lower  than  each  of  the  other  groups  and  that  Group  NEM  had  retention  and  transfer 
scores  significantly  lower  than  Groups  N and  NE,  which  did  not  differ  from  each  other.  The  mean  matching  scores 
for  the  four  groups  were  no  significantly  different,  with  no  main  effect  for  music,  sounds,  and  no  significant 
interaction  between  music  and  environmental  sounds. 

This  pattern  obtained  for  the  retention  and  transfer  tests  therefore  supports  the  hypothesis  derived  from  the 
cognitive  model  of  multimedia  learning.  Adding  extraneous  auditory  material— in  the  form  of  music— tended  to  hurt 
students'  understanding  of  the  lightning  process.  Adding  relevant  and  coordinated  auditory  material— in  the  form  of 
environmental  sounds  did  not  hurt  students'  understanding  of  the  lightning  process.  Together  with  the  retention 
results,  these  findings  suggest  that  auditory  overload  can  be  created  by  adding  auditory  material  that  does  not 
contribute  to  making  the  lesson  intelligible.  The  results  of  this  auditory  overload  are  that  fewer  of  the  relevant 
words  and  sounds  may  enter  the  learner’s  cognitive  system  and  fewer  cognitive  resources  can  be  allocated  to 
building  connections  among  words,  images,  and  sounds 

Therefore,  A fifth  instructional  design  principle  for  multimedia  learning  with  animations  can  be  inferred. 

Coherence  Principle 

Students  learn  better  when  extraneous  material  is  excluded  rather  than  included  in  multimedia  explanations. 
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Conclusion 


Multimedia  explanations  allow  students  to  work  easily  with  verbal  and  non-verbal  representations  of 
complex  systems.  The  present  review  demonstrates  that  presenting  a verbal  explanation  of  how  a system  works 
with  an  animation  does  not  insure  that  students  will  understand  the  explanation  unless  research-based  principles  are 
applied  to  the  design. 

The  present  studies  have  important  theoretical  implications.  According  to  a generative  theory  of 
multimedia  learning  (Mayer,  1997),  active  learning  occurs  when  a learner  engages  three  cognitive  processes— 
selecting  relevant  words  for  verbal  processing  and  selecting  relevant  images  for  visual  processing,  organizing  words 
into  a coherent  verbal  model  and  organizing  images  into  a coherent  visual  model,  integrating  corresponding 
components  of  the  verbal  and  visual  models.  To  foster  the  process  of  selecting,  multimedia  presentations  should  not 
contain  too  much  extraneous  information  in  the  form  of  words  or  sounds,  and  should  not  present  . To  foster  the 
process  of  organizing,  multimedia  presentations  should  represent  the  steps  in  order  and  with  clear  signals  for  both 
the  verbal  and  visual  information.  To  foster  the  process  of  integrating,  multimedia  presentations  should  present 
words  and  pictures  concurrently  using  modalities  that  effectively  use  available  visual  and  auditory  working  memory 
resources.  The  major  advance  in  our  research  program  is  to  identify  techniques  for  presentation  of  verbal  and  visual 
information  that  minimizes  working  memory  load  and  promotes  meaningful  learning. 
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Abstract:  We  present  a description  of  a fiilly  distributed  system  that  can  be  used  to 
distribute  data  among  any  number  of  authenticated  users.  The  name  server  is  used  only  for 
authentication;  otherwise  the  load  is  evenly  distributed.  The  system  has  been  used  to  create  a 
distributed  repository  of  programming  examples. 


0.  Introduction 

With  the  increased  availability  of  the  Internet,  personal  computers  and  fast  networks,  there  are  many  research 
efforts  directed  at  various  network  applications,  and  in  general,  distributed  software.  Another  reason  for  the 
interest  in  this  research  is  the  growing  number  of  universities,  which  have  built  fully  computerized  campuses. 
For  example,  Acadia  University,  see  (MacDougall  et  al.  1998),  has  been  involved  in  building  such  a campus;  it 
provides  fast  network  connections  for  all  students  living  on  the  campus  as  well  as  electronic  classrooms  (a 
setting  in  which  every  student  and  teacher  have  access  to  a personal  computer,  and  all  the  computers  are 
networked  allowing  for  various  types  of  computer-supported  interaction,  see  (Shneiderman  et  al  1995)).  For 
another  example  of  a computerized  campus,  see  (Holmes  1996).  In  this  paper  we  tackle  the  issue  of 
distributing  data  among  a number  of  networked  computers. 

We  consider  both,  connected  and  partially  connected  computers  (the  latter  are  mobile  computers  that  can 
connect  to  various  networks).  This  type  of  arrangement  is  typical  for  computerized  campuses,  but  also  applies 
to  mobile  users  who  travel  with  their  laptops.  There  are  many  applications  for  which  one  would  wish  to 
distribute  information,  but  here  we  concentrate  on  a specific  case  to  support  an  example-driven  learning 
process;  namely  a distributed  repository  of  programming  examples,  for  instance  examples  of  programs  in  C or 
Java.  Examples  are  useful  if  they  can  be  easily  browsed  and  searched,  and  various  users  can  share  them.  When 
learning  programming,  examples  are  particularly  useful  because  in  this  case  one  always  learns  from  examples 
of  small  programs. 

This  paper  is  organized  as  follows.  In  the  first  section,  we  provide  basic  concepts  of  distributed  information 
systems  and  discuss  various  models.  The  second  section  presents  our  system,  the  Distributed  Repository  of 
User-Classified  documents,  DRUCO;  and  the  final  section  describes  a case  study,  a repository  of 
programming  examples  in  C. 

1.  Distributed  Information  Systems 

A distributed  information  system  can  be  defined  as  a system  where  documents  containing  some  information 
are  distributed  across  multiple  machines  connected  by  a network.  Therefore,  data  (or,  documents)  are 
accessible  as  a shared  resource,  see  (Booth  1981).  These  systems  are  useful  because  the  collective  storage  of 
multiple  computers  provides  a more  powerful  system.  Additionally,  with  the  duplication  of  resources,  the 
failure  of  one  component  does  not  necessarily  imply  losing  the  entire  set  of  data.  Thus,  distributed  systems 
provide  parallelism  and  fault  tolerance,  making  them  potentially  much  more  powerful  than  their  individual 
components,  see  (Mullender  1989).  In  the  context  of  computerized  campuses  and  a wide  accessibility  of  the 
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Internet,  a distributed  information  system  has  an  additional  important  task  to  fulfill;  namely  to  support 
information  sharing.  Below,  we  review  various  types  of  information  sharing  systems. 

The  most  basic  thing  we  want  to  be  able  to  do  is  to  exchange  and  share  information.  From  now  on,  we  will 
assume  that  the  information  is  stored  in  a document,  which  serves  as  a persistent  medium  for  this  information 
(and  therefore,  we  will  use  the  term  ’’document”  interchangeably  with  the  term  ’’information").  A document  is 
not  necessarily  the  same  thing  as  a file;  it  could  be  a file,  a number  of  files  logically  grouped  together,  or  an 
entry  in  a database.  (Note  that  this  definition  of  a distributed  system  is  more  general  than  that  of  a distributed 
file  system,  such  as  Coda,  see  (Satyanarayanan  1990)).  There  are  three  basic  considerations  when  it  comes  to 
sharing  documents  using  an  information  system:  operations  on  the  system,  security  of  the  system,  and  the 
interface  to  the  system.  For  a system  to  be  useful,  it  has  to  provide  various  operations  such  as  search,  browse, 
compare,  export/import  from  and  to  the  file  system,  and  others.  Some  systems  support  only  homogenous 
information,  that  is  force  all  the  users  to  use  the  same  type  of  documents.  Other  systems  may  support  more 
heterogeneous  documents  as  long  as  they  conform  to  a certain  standard.  The  system  should  be  secure  by 
allowing  each  user  to  specify  for  every  document,  who  are  the  users  for  whom  this  document  is  accessible  (for 
example,  to  view,  copy,  traverse,  etc.).  This  functionality  can  be  provided  by  associating  with  each  document 
the  list  of  users  and  their  passwords;  then  the  user  has  to  be  authenticated  before  they  can  obtain  any 
information.  As  an  alternative,  digital  signatures  can  be  used,  see  (Greenleaf  1997). 

The  interface  to  the  system  determines  one  of  two  possible  roles  the  user  has  to  play  to  obtain  documents  that 
belong  to  other  users.  These  roles  roughly  correspond  to  pulling  information  and  pushing  information.  If  a 
system  supports  only  pulling  information,  then  the  user  has  to  play  an  active  role  in  obtaining  required 
information.  A notification  system  can  be  added  to  notify  a user  for  example  when  new  information  becomes 
available  (addition  notification)  or  is  changed  (change  notification).  An  information  push  system  tries  to  push 
required  information  to  interested  users.  These  systems  may  be  profile-based,  or  context-  and  situation -based; 
see  (Van  de  Velde  at  al.  1997).  An  alternative  would  be  to  consider  active,  autonomous  providers,  such  as 
software  agents,  using  planning  to  search  for  the  required  information;  see  (Huhns  1998).  A system  of 
intelligent  agents,  such  as  Cooperative  Information  System,  CIS,  see  (Papazoglou  1992)  can  actively  reason 
about  the  information  it  contains  and  search  for  these  pieces  of  information  by  connecting  to  other  systems. 
Now,  we  elaborate  on  the  push  and  pull  models.  In  order  to  describe  the  task  of  exchanging  and  sharing 
information  in  a pull  model,  we  consider  two  kinds  of  applications,  here  called  respectively  providers  and 
fetchers.  A provider  application  gives  access  to  available  information  for  authorized  users;  a fetcher 
application  is  designed  to  fetch  or  browse  information  from  one  or  more  providers.  Typically,  a provider  is 
implemented  as  a server,  and  a fetcher  as  a client;  for  more  information  about  clients  and  servers,  see 
(Mullender  1989).  A single  application  may  be  both  a provider  (server)  and  a fetcher  (client)  at  the  same  time. 
For  an  application  A to  communicate  with  another  application  B,  A has  to  be  able  to  locate  B,  using  some  kind 
of  a naming  system.  A standard  convention  used  by  the  Internet,  is  to  use  Universal  Resource  Locators,  URLs. 
When  A provides  B’s  URL,  the  Domain  Name  Server,  DNS  finds  B’s  IP  address,  and  now  A can  use  this 
address  to  communicate  with  B.  Unfortunately,  the  above  technique  does  not  work  if  B is  currently  off-line,  or 
B uses  an  Internet  service  Provider,  ISP  which  provides  dynamic  IP  addresses.  Therefore,  often  we  assume  that 
there  exists  additional  name  server  residing  at  a "well-known”  static  IP.  When  the  application  B goes  on-line, 
it  connects  to  the  name  server,  which  at  that  time  retrieves  its  current  IP  and  associates  it  with  the 
application’s  name.  When  the  application  A wants  to  connect  to  the  application  B,  it  does  it  through  the  name 
server,  which  guarantees  that  B is  on-line  and  its  current  IP  address  is  known.  (Additionally,  the  name  server 
can  be  used  to  store  user  names  and  passwords,  and  to  authenticate  users.) 

The  best  known  example  of  the  above  model  is  that  of  a Web  server  (provider)  and  a Web  browser  (fetcher). 
The  user  of  a browser  pulls  information  provided  by  the  Web  server.  Recently,  Internet  Explorer  introduced 
channels,  which  are  lists  of  web  sites  that  are  regularly  (with  the  required  frequency)  pushed  onto  the  users 
desktop.  However,  this  is  not  a real  push  model;  instead  it  is  a "scheduled"  pull.  As  mentioned  above,  the  push 
model  is  best  implemented  using  mobile  agents  (for  the  description  of  mobile  agents,  see  (Wong  1999)).  The 
user  interested  in  an  operation,  such  as  search,  or  compare  may  create  an  agent  to  move  to  various  sites  and 
perform  this  operation  on  her  or  his  behalf  (the  agent  may  perform  some  action  on  that  site,  if  allowed,  or  it 
may  collect  the  information  and  carry  it  to  its  home  site).  While  it  is  easy  to  see  how  to  specify  the  required 
information,  it  is  more  difficult  to  decide  where  the  agent  should  go  to  perform  the  required  task.  Here,  there 
are  several  possible  solutions.  First,  a solution  similar  to  one  described  above,  i.e.  using  a central  naming 
server  can  be  used.  The  server  will  know  of  all  kinds  of  information  and  the  corresponding  sites.  Of  course. 
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instead  of  using  a central  server,  a distributed  naming  service  such  as  JNDI  can  be  used.  The  problem  is  that 
all  sites  associated  with  a single  type  of  information  will  have  to  be  homogenous;  for  example  the  information 
about  technical  reports  will  have  be  provided  in  a consistent  manner.  It  would  be  best,  if  there  is  a generally 
accepted  standard  for  specifying  data,  and  for  this  reason  below  we  briefly  describe  XML. 

An  extensible  Marking  Language,  XML,  see  (David  1999),  is  a meta-language  that  allows  the  user  to  create  a 
specialized  language  to  define  a structure  of  a document.  XML  has  been  used  in  a variety  of  applications,  for 
example  to  define  Internet  Explorer’s  channels,  see  (Microsoft).  The  main  strength  of  XML  is  that  it  provides 
a domain  specific  standard,  and  XML  applications  support  operations  such  as  search,  compare  and  merge.  For 
each  XML-generated  language  there  is  a Document  Type  Definition,  DTD,  which  is  basically  a grammar  for 
this  language.  Therefore,  an  XML  document  can  be  verified  against  its  DTD.  In  addition,  an  XML  document 
can  be  easily  converted  to  a different  XML  format,  if  the  conversion  is  defined  by  another  DTD.  XML  is 
accompanied  by  an  extensible  Style  Language,  XSL  and  an  extensible  Link  Language,  XLL.  XSL  provides 
formatting  instructions  to  display  an  XML  document,  while  XLL  supports  links  that  generalize  HTML  links, 
to  rectify  the  problem  of  maintenance  of  links,  and  to  allow  to  create  links  for  read-only  media.  Therefore, 
these  products;  XML,  XSL  and  XLL  provide  three  independent  ways  to  collectively  define  the  document’s 
structure,  format  and  links.  In  the  absence  of  XML,  one  may  want  to  use  an  information  system,  in  which  the 
user  explicitly  provides  her  or  his  own  structure  of  the  document.  In  the  following  section  we  describe  a 
specific  example  of  a general  distributed  information  system,  based  on  user-defined  classifications.  This 
system  is  called  Distributed  Repository  of  User-Classified  documents,  DRUCO. 


2.  Distributed  Repository  of  User-Classified  Documents 

Here  we  consider  a system  of  documents,  in  which  every  user  can  define  her  or  his  classifications.  Each 
document  may  be  stored  or  classified  within  one  or  more  classifications,  and  classifications  may  be  nested. 
Therefore,  in  a classical  file  system,  classifications  resemble  folders,  or  directories,  and  documents  resemble 
files.  The  entire  system  can  be  seen  as  a tree;  leaves,  however,  may  have  more  than  one  father  (classification). 
The  user  will  be  able  to  make  a part  of  its  system  available  to  other  users;  and  pull  (download)  a classification 
from  another  user  (this  classification  corresponds  to  the  node  of  the  tree;  and  the  entire  subtree  rooted  at  this 
node  may  be  pulled).  Permissions  may  be  set  so  that  only  selected  users  will  have  access  to  some  documents. 
In  what  follows,  we  will  refer  to  this  specific  information  system  as  a distributed  repository  of  user-defined 
classifications,  or  briefly  as  a repository. 

DRUCO  is  a system  designed  to  satisfy  the  above  requirements.  In  this  system,  a single  unit  that  resembles  a 
file  represents  each  document.  DRUCO  documents  are  stored  separately  from  the  file  system,  but  they  can  be 
easily  imported  to,  and  exported  from  that  system.  Now,  we  briefly  describe  the  design  and  functionality  of 
DRUCO.  It  is  a program  that  can  operate  both,  in  off-line  mode  or  in  on-line  mode,  and  in  the  latter  mode  as  a 
server,  or  as  a client.  All  the  operations  that  can  be  performed  in  off-line  mode  can  also  be  performed  in  on- 
line mode.  Off-line  operations  facilitate  organizing  a repository,  that  is  creating  new  classifications,  modifying 
and  deleting  existing  classifications,  importing  and  exporting,  and  browsing  and  viewing  documents;  see  (Fig. 
1).  In  (Fig.  1)  the  leftmost  pane  shows  the  current  state  of  the  repository.  Folder  icons  indicate  whether  or  not 
they  are  currently  folded,  or  expanded.  The  middle  pane  shows  the  current  state  of  the  DownloadSpace;  the 
outermost  folder  (Shiv)  is  a placeholder  for  all  the  documents  that  have  been  pulled.  The  rightmost  pane  can 
be  used  to  show  the  contents  of  the  selected  file.  The  user  can  use  menus/right  button/drag  and  drop.  Also,  in 
off-line  mode,  the  user  may  specify  "permissions",  that  is  give  or  revoke  the  right  to  access  documents.  Here,  a 
classification  is  the  smallest  unit;  that  is  the  user  can  grant  permission  to  one  or  more  users  for  a specific 
classification;  then,  these  users  can  view  and  possibly  pull  all  documents  in  this  classification.  However, 
whether  or  not  they  will  be  able  to  view  the  contents  of  any  nested  classifications,  depends  on  their 
permissions  for  these  classifications.  (Below,  we  describe  how  the  user  can  get  the  list  of  all  registered  users.) 
To  manage  permissions,  the  user  selects  the  SecurityManagement  pane,  see  (Fig.  2).  The  UserList  pane  shows 
the  list  of  registered  users  (see  (Fig.  2);  the  user  can  select  folders  and  documents  in  the  leftmost  pane,  and  use 
buttons  to  assign  and  revoke  permissions. 


Figure  1 : DRUCO  in  off-line  mode). 
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Figure  2:  DRUCO  in  off-line  mode 
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To  facilitate  the  location  of  other  users,  especially  in  the  presence  of  users  who  use  dynamic  IP  addresses,  a 
central  name  server  has  been  incorporated  into  DRUCO.  An  administrator  of  this  name  service  manages  users, 
that  is  registers  these  users,  assigns  passwords,  etc.  The  name  server  is  running  on  a computer  with  a "well- 
known"  static  IP  address,  which  is  provided  in  the  configuration  file  read  when  the  user’s  application  starts. 
The  interface  to  the  name  server  is  shown  in  (Fig.  3). 


Figure  3:  DRUCO’s  name  server 

The  administrator  manages  the  name  server;  that  is  adds  and  removes  users,  and  sets  their  passwords.  On-line 
operations  can  be  divided  into  two  types;  interactions  with  the  name  server  and  interactions  with  other  users. 
In  order  to  interact  with  the  name  server,  the  user  has  to  login  to  the  server,  by  providing  the  name  and 
password.  Upon  the  successful  login,  the  name  server  retrieves  the  current  IP  of  this  user,  and  saves  this 
information;  therefore  DRUCO  supports  users  who  have  dynamic  IP  addresses,  for  example  those  using  ISPs. 
The  user  may  perform  the  following  operations  that  interact  with  the  name  server:  change  the  password, 
retrieve  the  list  of  all  registered  users,  and  retrieve  the  list  of  all  active  users  (i.e.  users  who  are  currently  on- 
line). The  list  of  active  users  is  used  to  start  an  interaction  with  another  user;  to  connect  to  another  user  it  is 
enough  to  select  this  user  from  the  active  user  list.  Note  that  this  connection  doesn’t  require  authentication, 
because  both  users  must  be  currently  connected  to  the  name  server,  and  therefore  they  have  already  been 
authenticated  by  that  server.  Also,  the  name  server  provides  the  current,  dynamic  IP  address  of  all  users.  The 
list  of  all  currently  active  users  can  be  used  to  select  one  user  and  then  connect  to  them.  User  A,  upon 
connecting  to  another  user  B,  may  browse  these  classifications  for  which  have  been  permitted  by  the  user  B, 
and  she  or  he  can  also  download  some  of  these  classifications. 

DRUCO  has  been  implemented  in  Java  1.2,  see  (SUN  1999a),  using  RMI,  see  (SUN  1999b). 


3.  Case  Study:  Distributed  Repository  of  Programming  Examples 

Example-based  learning,  see  (Neal  1 989)  promotes  the  idea  of  using  numerous  examples  to  help  to  understand 
various  concepts,  and  to  move  these  concepts  from  short-term  memory  to  long  term  memory.  Examples  are 


useful  if  they  can  be  easily  browsed  and  searched,  and  they  can  be  shared  by  various  users.  When  learning 
programming,  examples  are  particularly  useful  because  in  this  case  one  always  learns  from  examples  of  small 
programs.  Consider  as  an  example  a specific  class  for  teaching  programming  in  the  C programming  language, 
taking  place  in  an  electronic  classroom.  Before  the  beginning  of  the  class,  the  instructor  (provider)  makes 
available  examples  of  programs  in  C to  all  students  in  the  classroom.  (As  an  alternative,  the  instructor  may 
divide  students  into  groups  by  giving  each  group  a different  example.)  Now,  the  students  can  pull  these 
examples  to  their  computers.  They  can  view  these  examples,  export  them  to  their  favorite  compiler,  modify 
them  by  creating  new  versions,  and  making  these  versions  available  to  all  students,  or  specific  students.  The 
entire  process  can  result  in  collaborative  development  of  a useful  repository  that  can  be  used  not  only  for 
learning,  but  also  for  real  every-day  programming. 

DRUCO  is  an  ideal  candidate  to  implement  a shared  repository  of  examples  with  a hierarchical  structure.  It 
has  been  used  to  implement  DRUCOC  - a repository  of  examples  of  C. 
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Abstract:  Hypermedia  has  been  identified  as  a promising  medium  for  education. 
However,  it  is  beset  with  problems  concerning  the  complexity  of  navigation  and 
meeting  the  individual  needs  of  the  user.  This  paper  is  concerned  with  a domain- 
independent  metric  that  can  be  used  to  extract  information  about  a user  of  an 
educational  hypermedia  system  with  a view  to  tailoring  the  hypermedia  to  meet  the 
user’s  individual  needs.  The  metric  attempts  to  discover  a user’s  information  seeking 
goal  when  they  are  using  a hypermedia  system  based  upon  the  patterns  they  make  as 
they  navigate  the  hypermedia.  Discussed  within  are  the  types  of  patterns  that  a 
hypermedia  navigator  may  use  and  how  they  may  be  used  to  infer  the  user’s 
information  seeking  goal.  Experiments,  used  to  demonstrate  the  validity  of  this 
approach  are  described  and  discussed. 


Introduction 

Leeds  Metropolitan  University  has  been  investigation  the  possibility  of  producing  a tutoring  system 
that  offers  aid  to  the  student  whilst  allowing  reuse  over  a number  of  domains.  The  use  of  a generic 
system  reduces  the  amount  of  information  that  can  be  collected  about  the  student  for  the  purpose  of 
providing  aid.  In  an  attempt  to  solve  this  problem  a prototype  hypermedia  tutoring  system,  called 
“Hypemet”,  has  been  developed  to  explore  issues  relating  to  extracting  information  about  the  student  in 
an  information  sparse  environment.  A potential  candidate  for  a metric  in  a generic  hypermedia 
environment  has  is  browsing  pattern  information.  The  remainder  of  this  paper  is  concerned  with  this 
metric  and  describes  some  experiments  that  were  used  to  support  the  contention  that  browsing  patterns 
are  of  potential  use  in  this  area.  Hypemet  collect  other  metrics,  these  are  discussed  elsewhere  (Mullier 
etal  1998,  1999,  Mullier  1999). 

Browsing  Patterns  and  Browsing  Strategies 

Hypermedia  browsing  strategies,  as  outlined  by  Canter  et  al  (1985)  and  McAleese  (1993),  and 
discussed  further  in  Mullier  (1999),  are  characteristic  methods  for  moving  through  a network  of 
hypermedia  nodes.  Canter  et  al  and  McAleese  are  referring  to  browsing  strategies  since  they  are 
referring  to  a user  undertaking  a particular  information  seeking  strategy  in  order  to  fulfil  some 
information  seeking  goal.  For  example,  a user  searching  for  a specific  piece  of  information  may  induce 
a “seeking”  browsing  strategy.  Therefore  the  user  may  “quickly  skip  from  one  node  to  the  next,  seldom 
returning  to  a previously  visited  node,  seldom  spending  more  than  a few  seconds  at  each  node  and 
ignoring  the  associative  trails  that  lead  off  in  varying  directions”  (Canter  et  al  (1985)’s  description  of 
scanning).  It  is  the  movements  that  the  user  makes  as  they  move  through  the  hypermedia  that  are 
termed  a “browsing  pattern”,  which  can  be  thought  of  as  a physical  manifestation  of  a browsing 
strategy.  It  has  been  argued  previously  by  Canter  et  al  (1985)  and  Mullier  et  al  (1999)  that  browsing 
patterns  offer  the  potential  to  extract  information  about  the  user’s  information  seeking  goal.  Research  at 
Leeds  Metropolitan  University  has  produced  a neural  network  based  system  that  is  capable  of 
recognising  browsing  patterns  and  ascribing  meaning  to  them,  a description  of  this  system  can  bee 
found  in  Mullier  et  al  (1998,  1999)  and  a thorough  discussion  in  Mullier  (1999).  The  remainder  of  this 
paper  is  concerned  with  the  potential  uses  of  browsing  patterns  and  some  experimental  evidence  to 
support  the  contention  that  they  are  of  use. 
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Canter  et  al  (1985)  and  McAleese  (1993)  have  formally  identified  five  browsing  strategies. 

1)  Scanning  - covering  a large  area  without  depth. 

2)  Browsing  - following  a path  until  a goal  is  achieved. 

3)  Searching  - striving  to  find  an  explicit  goal. 

4)  Exploring  - finding  out  the  extent  of  the  information  given. 

5)  Wandering  - purposeless  and  unstructured  “globe  trotting”. 


Uses  for  Browsing  Patterns 

Canter  et  al  (1985)  suggest  that  browsing  patterns  represent  some  “important  psychological 
information  about  the  user”.  Since  the  browsing  pattern  is  a result  of  the  user’s  information-seeking 
goal  it  may  be  possible  to  identify  this  information-seeking  goal  from  the  browsing  pattern.  The 
information-seeking  goal  may  then  be  used  to  tailor  the  hypermedia  system  to  aid  the  user.  For 
example,  current  web  agents  use  keyword  matching  to  aid  a user  of  the  World  Wide  Web,  by  offering 
pages  with  similar  content  (keywords)  (Armstrong  et  al  1985).  Further,  offering  web  pages  of  similar 
content  is  only  useful  for  certain  information-seeking  strategies,  namely  those  that  involve  the  location 
of  specific  information.  However,  as  Cartledge  and  Pitkow  (1995)  point  out,  keyword  matching  can  not 
take  into  account  the  possible  dynamic  behaviour  of  the  browser,  in  terms  of  the  potential  for  the 
browser  to  change  their  goals  and  strategy  at  any  instant.  Similarly,  Armstrong  et  al  (1995)  note  that 
their  browsing  support  system  ”Web  Watcher"  is  restricted  to  being  an  adviser  only  because  it  "might 
not  perfectly  understand  the  user's  information  seeking  goal".  Information-seeking  strategies,  such  as 
general  browsing,  are  not  aided,  and  may  be  hindered,  by  keyword  matching.  The  identification  of 
browsing  patterns  may  therefore  provide  additional  information  for  a web  agent  (or  hypermedia  agent) 
so  that  keyword  matching  can  be  enabled  or  disabled,  as  the  situation  requires. 

The  Meaning  of  Browsing  Patterns 

Once  a particular  browsing  pattern  has  been  identified  it  is  necessary  to  link  the  pattern  to  information 
that  can  be  used  to  directly  aid  the  student.  It  is  envisaged  that  identification  of  the  browsing  pattern 
could  be  employed  to  tailor  the  presentation  of  information  to  a user  in  such  a way  so  as  to  model  their 
interests  more  closely.  For  example,  if  the  student  is  generally  interested  in  the  domain  then  it  may  be 
useful  to  present  them  with  a selection  of  different  topics  from  the  domain,  whereas  it  would  be  better 
to  offer  more  detailed  information  if  they  are  looking  for  a specific  item  of  information.  The  Hypemet 
system  developed  at  Leeds  Metropolitan  University  to  explore  issues  relating  to  educational 
hypermedia  (Mullier  et  al  1998,  Mullier  1999)  contains  a sub-system  (the  Browsing  Pattern 
Recogniser)  which  is  able  to  identify  browsing  patterns.  Browsing  patterns  are  then  mapped  onto  a 
student  ability  level  (obtained  by  Hypemet  is  response  to  a student’s  interactions  with  tutorials)  and  the 
task  the  browsing  pattern  is  employed  for  (as  provided  by  the  Hypemet),  by  a second  sub-system 
termed  the  Fuzzy  Rule  Base  (FRB).  It  is  an  advantage  that  Hypemet  is  a tutoring  system,  since  this 
enables  a dialogue  with  the  student,  the  results  of  this  dialogue  can  then  be  used  to  ascribe  meaning  to  a 
browsing  pattern.  For  example,  if  a novice  student  uses  a particular  browsing  pattern  to  solve  a 
particular  problem  (and  the  type  of  student  and  type  of  task  are  known)  then  it  may  be  that  this  type  of 
browsing  pattern  is  often  used  in  such  a way.  Obviously  it  is  not  desirable  to  make  such  a distinction 
based  upon  one  example,  therefore  the  Hypemet  system  has  been  designed  to  map  general  trends  from 
a large  number  of  examples.  Hypemet  achieves  this  by  using  a combination  of  fuzzy  logic  and  neural 
networks,  this  is  discussed  further  in  Mullier  et  al  (1998  and  1999)  and  is  discussed  in  detail  in  Mullier 
(1999). 

Recognising  Browsing  Patterns 

Canter  et  al  (1985)  suggest  that  browsing  patterns  are  formed  from  lower  level  browsing  stmctures 
(constmcts);  termed  “spikes,  “rings”,  “paths”  and  “loops”.  These  constmcts  connect  together  to  form 
higher  level  browsing  patterns  (searching,  scanning  etc.).  These  browsing  patterns  may  then  form  part 
of  a user’s  overall  browsing  strategy.  This  presents  a problem  for  automatic  recognition,  since  it  is  not 
readily  apparent  where  one  construct  ends  and  another  constmct  begins.  It  could  also  be  argued  that 
constmcts  merge  into  each  other.  For  example,  a large  loop  can  also  be  thought  of  as  a path,  until  the 
point  where  it  crosses  a previously  visited  node.  Further,  the  diagrammatic  representations  of  browsing 
patterns  are  not  accurate  representations  of  the  actual  constmcts  and  browsing  patterns.  This  is  because 
the  diagrams  are  drawn  in  two-dimensional  space,  whereas  the  constmcts  themselves  do  not  exist  in 
such  a space.  For  example,  the  ring  diagram  shows  a trail  arcing  over  upon  itself  until  it  crosses  a 
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previously  crossed  node.  In  reality,  it  is  only  because  one  node  has  been  visited  twice  that  this  construct 
becomes  a loop.  Although  Canter  et  al  (1985)  do  not  note  this  deficiency  in  representation  explicitly 
they  do  provide  a mechanism  for  overcoming  the  problem.  They  do  not  attempt  to  identify  constructs 
individually,  instead  they  describes  a user’s  browsing  pattern  as  being  comprised  of  several  indices, 
namely  the  path,  spike,  ring  and  loop  indices,  identified  on  figure  1 as  Path  PQ,  Ring  RQ,  Loop  LQ  and 
Spike  SQ.  However,  they  admit  that  their  indices  present  "calculation  problems"  since  indices  can 
overlap  and  become  fuzzy  in  nature  and  there  is  therefore  a requirement  for  "more  complex  and  subtle 
algorithms".  It  can  be  seen  then  that  the  low-level  browsing  constructs  merge  into  one  another  to  form 
the  complex  browsing  patterns.  The  task  then,  is  to  identify  these  constructs  and  relate  them  to  the 
overall  browsing  pattern.  The  browsing  pattern  may  then  be  used  to  infer  information  about  the 
student’s  overall  browsing  strategy. 

Automatic  recognition  of  browsing  patterns,  however,  presents  several  problems  and  their 
interpretation  several  more.  Firstly,  recognising  the  high  level  patterns  is  made  complex  because  a user 
may  adopt  more  than  one  pattern  for  a particular  session  and  may  appear  to  be  utilising  different 
patterns  depending  upon  how  many  movements  are  examined  (Bates  1989).  For  example,  a student 
may  be  involved  in  an  overall  “searching”  strategy  (and  thus  invoke  a searching  pattern),  i.e.  they  are 
looking  for  a specific  piece  of  information.  However,  they  may  become  interested  in  a piece  of 
information  that  they  have  discovered  whilst  engaged  upon  their  “searching”.  The  student  may  then 
make  a short  deviation  from  their  current  “searching”  strategy  and  "browse"  around  the  area  that  has 
caught  their  interest,  before  returning  to  their  original  strategy.  How  then  is  this  to  be  interpreted?  Are 
there  three  distinct  strategies?  Are  there  two?  Or  is  this  to  be  interpreted  as  one  complex  strategy?  If  a 
symbolic  artificial  intelligence  (SAI)  approach  to  pattern  recognition  were  to  be  employed  then  the 
answer  to  such  questions  must  be  decided  upon  before  a student  uses  the  system.  For  the  SAI  designer 
must  decide  beforehand  the  construction  of  every  browsing  pattern  (that  is  invoked  by  a particular 
strategy)  that  the  system  must  recognise  and  encode  appropriate  production  rules. 

FATKENESS  - A route  through  the  nodes  that  does  not  visit  any  node  twice  LOOPINESS  - A ringwHch  contains  no 


RINGirrESS  - A routs  that  rotumt  to  Uts  start  node  (can  bs  nested) 


SFIKENESS  • A route  uiUt  a return  journey 


O 


retracing  the  original  p ath 


Figure  1 

A further  problem  is  that  a student  may  use  a particular  strategy  at  one  instant  and  then  swap  to  another 
in  response  to  some  stimulus  from  the  hypermedia  domain  itself  The  rate  of  sampling,  i.e.  how  many 
movements  the  user  makes  before  the  system  determines  the  browsing  pattern,  can  therefore  result  in 
the  system  providing  differing  results.  For  example,  a user  may  be  involved  in  an  overall  browsing 
strategy  that  is  built  up  of  several  smaller  browsing  strategies,  in  that  the  student’s  overall  information 
seeking  goal  may  be  served  by  the  employment  of  several  distinct  strategies.  It  is  difficult  to  say 
whether  the  student  has  engaged  one  overall  browsing  strategy  or  several  strategies.  It  is  therefore 
difficult  to  determine  exactly  what  the  student  is  doing  at  one  particular  instant  in  time.  The  result  of 
these  problems  is  that  different  results  are  obtained  from  a number  of  recognition  methods. 

Low-level  Constructs  to  Browsing  Pattern 

In  order  to  combat  the  above  recognition  problems  it  is  necessary  to  identify  high-level  browsing 
patterns  since  the  low-level  constructs  do  not  provide  enough  information  about  the  student,  since  they 
are  only  formed  in  a relatively  short  time  period  and  encompass  a small  amount  of  the  domain  (in 
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comparison  to  an  overall  browsing  pattern  formed  by  the  student  to  solve  a particular  information 
seeking  problem).  This  is  possible  in  Hypemet  because  it  can  identity  the  start  of  a browsing  pattern. 
The  start  point  can  be  defined  as  the  point  at  which  time  the  student  leaves  a tutorial  node,  i.e.  they 
have  been  given  a task  to  complete  by  the  tutoring  system.  They  then  engage  upon  a browsing  strategy, 
which  induces  a browsing  pattern,  in  order  to  complete  the  requirements  of  the  tutorial  node.  Once  they 
have  satisfied  the  requirements,  the  browsing  pattern  has  concluded.  Once  the  high-level  browsing 
pattern  has  been  identified,  in  terms  of  its  start  and  end  points,  then  the  system  may  process  it.  It  is 
stressed  that  Hypemet  is  a research  system  and  is  used  here  to  extract  these  browsing  patterns  (and 
record  them)  so  that  they  can  be  used  afterwards  in  other  systems  (such  as  the  WWW).  It  is  therefore 
not  necessary  to  always  identify  the  start  and  end  points  of  a browsing  pattern,  once  Hypemet  has 
recorded  a representative  sample  the  start  and  end  points  are  patterns  themselves  within  the  sample. 
This  situation  is  analogous  to  recognising  the  handwriting  of  an  unknown  script,  in  that  enough 
examples  should  make  the  script  clear.  This  is  similar  to  the  Case-Based  Reasoning  approach  used  by 
Micarelli  and  Sciarrone  (1998)  to  identify  content-based  browsing  patterns,  in  that  recognition  of  the 
current  case  is  based  upon  similar  cases  encountered  previously.  A possible  criticism  of  this  approach 
is  that  browsing  patterns  recorded  as  part  of  Hypemet  may  be  specific  to  Hypemet  and  may  not  exists 
in  other  systems.  As  McAleese  (1993)  points  out  certain  browsing  strategies  are  facilitated  by  certain 
interface  styles  (graphical  browsers  and  text-based  browsers),  it  therefore  follows  that  the  interface 
style  (in  this  case  Hypemet’ s interface)  has  a direct  influence  upon  the  browsing  pattern  produced  by 
the  student’s  browsing  strategy.  However,  it  is  likely  to  be  the  case  that  there  is  a core  of  patterns  that 
are  common  to  all  styles.  The  recognition  system  may  therefore  record  general  information  as  part  of 
Hypemet  and  may  then  adapt  to  individual  browsing  systems  that  it  may  subsequently  be  employed 
within. 

In  order  to  demonstrate  that  relationships  exist  between  the  browsing  patterns  made  by  similar  types  of 
user,  in  terms  of  interests  (information  seeking  goals)  and  abilities,  several  experiments  were 
undertaken.  This  information  is  to  be  collected  by  Hypemet  as  it  is  used  over  a prolonged  period. 
However,  this  is  a long-term  goal  of  the  current  research  project,  in  the  meantime  a smaller  scale  study 
was  performed.  This  study  is  described  below. 

Browsing  Experiments 

The  purpose  of  the  browsing  experiments  is  to  establish  whether  different  types  of  students  browse  the 
hypermedia  in  different  ways  and  whether  similar  abilities  of  students  show  similarities  in  their 
browsing  behaviour.  The  experiments  have  provided  some  interesting  information  that  warrants  further 
study. 

The  initial  results  demonstrate  that,  for  the  population  of  subjects  tested,  there  are  distinct  differences 
between  expert  browsers  and  novice  browsers.  Further,  results  have  demonstrated  that  there  are  two 
distinct  types  of  novice  students.  A novice  student  may  show  reticence  to  move  away  from  the  initial 
node  and  therefore  does  not  explore  the  domain.  By  contrast  another  type  of  novice  student  recklessly 
moves  away  from  the  initial  node  and  become  quickly  lost. 

Method 

A simplified  astronomy  domain  was  dispatched  to  volunteers  contacted  via  Internet  news  groups.  The 
domain  has  a few  tens  of  nodes  representing  information  to  varying  depths.  High-level  nodes  represent 
such  items  as  planets  and  satellites,  detailed  nodes  represent  such  items  as  weather  systems,  plate 
tectonics  and  probe  missions.  Volunteers  were  required  to  grade  themselves  into  one  of  three  grades  of 
expertise  with  the  solar  systems  domain:  1)  Expert,  2)  Intermediate,  3)  Novice.  Volunteers  who 
described  themselves  as  experts  were  obtained  from  astronomy  related  news  groups  on  the  Internet. 
Novice  were  generally  the  children  of  the  expert  volunteers.  Intermediate  users  were  people  who  has  a 
passing  knowledge  of  Astronomy  but  had  not  studies  it  at  University. 

The  prototype  system,  supplied  to  the  volunteers,  recorded  all  browsing  information  and  stored  it  on 
the  volunteer's  hard  drive.  The  volunteer  could  then  email  the  browsing  data  back  for  examination. 
Information  recorded  by  the  system  was  a real  number  value  representing  whether  a node  had  been 
previously  visited  or  not.  Every  time  a student  moves  from  one  node  to  another  the  current  node  has  its 
'‘visited  value”  set  to  one,  whilst  all  other  nodes  that  have  been  previously  visited  have  their  visited 
values  reduced  by  a small  amount.  The  actual  amount  is  a function  of  the  number  of  total  nodes  in  the 


domain.  This  method  allows  for  the  representation  of  all  of  Canter  et  al’s  (1985)  browsing  patterns  and 
provides  some  measure  of  temporal  information. 

Volunteers  were  given  two  tasks  to  accomplish,  the  first  gave  them  a certain  piece  of  information  to 
find,  the  level  of  difficulty  being  dependent  upon  their  self-perceived  ability  level.  The  second  task  was 
to  browse  the  hypermedia  for  as  long  as  they  liked.  Participants  then  emailed  the  results  back  at  their 
leisure. 

Results 

The  results  described  below  are  from  a population  of  1 1 volunteers.  These  were  separated  into  three 
categories:  6 novices,  2 intermediate  and  3 expert. 

Results  for  expert  level  browsers  demonstrated  that  they  tended  to  use  rings  and  loops  with  a high 
degree  of  pathiness.  This  is  indicative  of  them  being  sure  of  the  layout  of  the  domain  (since  it  is 
organised  in  a structured  manner)  and  they  are  only  crossing  previously  visited  nodes  as  a means  of 
getting  to  their  destination,  which  could  be  determined  because  they  only  spent  a second  or  so  at  each 
node  on  their  return  journey.  The  free  browsing  exercise  produced  more  spikes,  although  these  tended 
to  be  more  distinct  than  for  the  novice  levels  of  browser,  in  that  they  were  likely  to  be  returning  along 
the  same  path  to  get  to  their  next  point  of  interest. 

Novice  browsers  were  more  complex  than  expert  browsers.  They  tended  to  be  more  chaotic  in  their 
behaviour,  in  that  it  was  more  difficult  to  discern  regular  patterns,  although  several  distinct  patterns  did 
emerge.  Novice  browsers  tended  to  use  spikes  much  more  than  expert  browsers.  These  spikes  also 
tended  to  be  short  and  often  centred  on  a small  cluster  of  nodes.  This  is  indicative  of  the  browser  being 
reluctant  to  leave  an  area  with  which  they  are  familiar.  Novice  browsers  tended  not  to  use  loops  and 
rings,  suggesting  that  it  is  important  to  have  an  understanding  of  the  structure  of  the  domain  before  the 
browser  can  return  to  a previously  visited  node  via  a different  path.  It  is  also  important  to  note  that 
returning  to  a previously  visited  node  is  easily  accomplished  by  selecting  the  "back"  button  (as  in  most 
hypermedia  systems).  Tliis  will  automatically  produce  a spike.  Hence  novice  students  could  return  to  a 
known  point  by  the  easier  option  of  selecting  the  back  button  (and  hence  producing  a spike),  whereas 
intermediate  and  expert  users  were  more  likely  to  navigate  themselves,  perhaps  via  a shorter  route 
which  was  already  known  to  them. 

Novice  browsers  tended  to  fall  into  two  categories,  the  first  category  are  reluctant  to  leave  the  area  with 
which  they  are  familiar.  The  second  category  tended  to  be  more  adventurous  and  move  away  quickly 
from  the  area.  It  is  not  a simple  matter  to  determine  which  strategy  is  generally  the  more  productive. 
The  reticent  browsers  were  obviously  intent  upon  the  domain  and  did  not  want  to  lose  themselves,  thus 
demonstrating  their  interest.  It  is  possible  that  the  more  adventurous  browser  was  just  aimlessly 
wandering  (a  strategy  identified  as  being  undesirable  by  Canter  et  al  (1995)).  It  was  not  possible  to 
determine  this  with  the  free  browsing  task,  since  if  the  browser  is  not  tested  then  it  is  impossible  to 
determine  whether  they  are  involved  in  their  own  novel  browsing  strategy.  The  given  task  did  however, 
provide  some  information  with  which  to  make  a determination  on  a particular  browsing  pattern,  in  that 
the  student’s  success  with  the  task  is  an  indication  of  the  usefulness  of  their  strategy.  Results 
demonstrated  that  some  browsers  were  indeed  wandering,  in  that  they  were  unable  to  fulfil  the  task  or 
took  much  longer  to  accomplish  it  than  other  browsers  of  the  same  level.  Obtaining  this  browsing 
pattern  information  whilst  the  student  was  using  the  system  may  have  enabled  the  presentation  of 
material  to  aid  the  student.  However,  some  low-level  students  that  appeared  to  be  adventurous,  did 
accomplish  the  task  and  often  in  times  much  quicker  than  reticent  browsers.  There  were  not  enough 
examples  of  each  type  of  novice  browser  to  make  a further  determination  about  how  these  types  of 
pattern  differ  and  may  be  independently  recognised.  However,  it  is  likely  that  there  are  differences  that 
could  be  determined,  given  a larger  set  of  example  data.  This  is  a major  reason  why  a mechanism  has 
been  developed  as  part  of  the  current  research  project  to  automatically  obtain  this  information. 

In  summary  then,  novice  browsers  tend  to  use  spikes,  probably  because  they  are  unsure  of  the  structure 
of  the  domain  and  need  to  use  navigational  cues.  More  expert  users  tend  to  use  rings  and  loops, 
probably  because  they  are  more  certain  of  the  structure  of  the  domain  and  know  which  node  to  return 
to  in  order  to  further  their  travels.  Therefore  there  may  be  a strong  correlation  between  spikes  and  more 
novice  students  and  rings  and  more  advanced  users.  Larger  path  values  indicate  the  more  adventurous 
types  of  user.  However,  even  this  small-scale  study  demonstrated  that  all  levels  of  user  did  exhibit  all 
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of  the  browsing  constructs  to  some  degree,  it  is  therefore  not  just  a matter  of  relating  a particular 
construct  to  a particular  ability  level. 

Discussion 

The  experiments  were  limited  in  that  they  employed  a restricted  domain  of  only  a few  tens  of  nodes 
and  only  a few  people  were  involved.  However,  the  results  have  demonstrated  some  interesting 
relationships  between  individual  browsing  constructs  and  certain  ability  levels  of  student.  This,  along 
with  research  conducted  by  Canter  et  al  (1985)  and  Micarelli  and  Sciarrone  (1998),  indicates  that 
different  types  of  student  may  use  different  browsing  patterns  and  goes  some  way  therefore,  towards 
justifying  the  use  of  browsing  patterns  as  a method  for  extracting  information  about  the  users  of 
information  resources. 

An  important  outcome  of  this  study  was  the  identification  of  two  types  of  novice  student,  namely 
adventurous  and  reticent  browsers,  with  the  possibility  of  the  adventurous  browsers  being  wanderers  or 
’’advanced  novices”.  It  should  be  stressed  however,  that  this  was  a small  study  and  this  result  may  be 
unique  to  the  study.  It  may  also  be  that  larger  studies  reveal  more  distinctions  between  people  of  the 
same  level 

The  experiments  give  some  evidence  that  Canter  et  al’s  (1985)  low-level  constructs  are  utilised  to 
different  degrees  by  different  types  of  student.  Expert  students  tended  to  use  loops  and  rings  with  a 
high  degree  of  pathiness  and  not  to  use  spikes  as  much  as  more  novice  students.  However,  it  must  be 
remembered  that  the  domain  in  use  was  highly  structured.  It  may  be  that  different  patterns  emerge  for 
different  types  of  domain. 

Conclusion 

This  paper  has  discussed  the  patterns  made  by  users  of  hypermedia  systems  and  means  of  identifying 
and  utilising  them  in  order  to  determine  information  about  the  student,  this  information  may  then  be 
used  to  aid  them  use  the  hypermedia.  The  browsing  experiments  suggest  that  different  types  of  student 
do  use  low-level  browsing  constructs  to  different  degrees  and  that  a method  for  recognising  these 
patterns  is  a potentially  interesting  and  useful  method  for  extracting  some  information  about  students 
and  users  of  information  structures.  Further  research  is  currently  investigation  a method  for  collecting 
browsing  data  over  the  World  Wide  Web,  which  potentially  offers  the  large  amount  of  browsing  data 
required  to  make  some  determination  as  to  what  each  pattern  is  used  for. 
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Abstract:  The  computer  program  PROSA  is  an  instructional  environment  for  learning  legal 
case  solving.  PROSA  stands  for  PROblem  Situations  in  Administrative  law.  PROSA  enables 
the  student  to  engage  in  legal  case  solving.  PROSA  supports,  guides  and  evaluates  the 
student’s  activities  and  outcomes. 


The  Computer  Program  PROSA 

The  computer  program  PROSA  is  an  instructional  environment  for  learning  legal  case  solving.  The  instructional 
environment  consists  of  two  horizontal  layers  and  three  vertical  columns  (see  Fig.  1). 

In  the  upper  layer  the  stimulus  materials  to  elicit  the  required  performance  are  presented  specified  to  type  of 
material  in  the  different  columns.  In  the  lower  layer  the  learner  guidance  and  response  are  presented  specified  to 
type  of  support.  In  PROSA  the  student  is  required  to  engage  in  legal  case  solving.  Legal  case  solving  is  the 
construction  of  a legal  solution  for  a specific  legal  case.  In  the  construction  process  the  legal  rules  are  the  legal 
problem  solving  tools.  To  construct  a legal  solution  for  a legal  case  the  student  has  to  select  legal  rules.  A 
distinction  is  made  between  legal  case  solving  activities,  the  process,  and  the  legal  solution  constructed,  the 
product.  Both  process  and  product  are  assessed  in  PROSA.  PROSA  offers  two  types  of  support:  elaborations 
and  response.  An  elaboration  presents  prerequisite  knowledge  and  examples.  The  student  may  request  an 
elaboration  at  any  moment  in  the  legal  solution  construction  process.  A response  is  a reaction  of  PROSA  to  an 
activity  of  the  student  or  to  an  outcome  produced  by  the  student.  A legal  case  solving  session  in  PROSA  is 
described  in  more  detail  to  see  how  the  instructional  environment  makes  a student  perform  in  legal  case  solving. 


A Legal  Case  Solving  Session  in  PROSA 

The  student  has  to  carry  out  a series  of  activities  to  solve  a specific  legal  case.  The  student  has  to  use  the  output 
of  the  activities  to  construct  the  legal  solution.  There  is  one  series  of  activities  that  is  classified  as  the  most 
recommended  route.  However,  there  is  some  variation  possible  in  the  sequence  of  activities.  These  routes  are 
classified  as  possible  routes.  Variations  of  activities  that  are  not  desired  are  not  possible  in  PROSA.  The 
description  of  a session  in  PROSA  is  based  on  the  most  recommended  route. 


Constructing  a Legal  Solution 

In  PROSA  legal  case  solving  starts  when  the  student  has  selected  a legal  case.  The  student  first  has  to  select  a 
legal  case  from  the  set  of  available  legal  cases  using  the  menu  button  legal  case.  The  legal  cases  in  PROSA  are 
arranged  to  topic.  Within  each  topic  the  legal  cases  are  arranged  to  level  of  difficulty.  The  selected  legal  case  is 
presented  in  the  upper  layer  in  the  legal  case  part  (e.g.  the  Alexander  Boer  case  topic:  interested  party;  difficulty 
level:  easy).  At  the  same  time  the  question  that  belongs  to  the  case  is  presented  in  the  upper  layer  in  the 
construct  legal  solution  part  (e.g.  Is  Alexander  Boer  an  interested  party  according  to  the  General  Administrative 
Law  Act?  see  Fig.  1). 


The  Process  Activities  Select  and  Match 
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The  student  now  has  to  construct  the  legal  solution  for  the  legal  case  presented.  This  construction  process 
involves  using  the  buttons  process  and  product  in  the  construct  legal  solution  part  in  the  upper  layer.  The 
student  first  has  to  select  the  menu  button  process.  The  process  button  shows  the  two  basic  legal  case  solving 
activities:  select  and  match.  The  first  activity  to  carry  out  is  select.  Being  presented  with  a legal  case  either  a 
legal  rule  or  a fact  from  the  legal  case  has  to  be  selected.  The  option  select  causes  a change  in  the  construct 
legal  solution  screen.  A distinction  is  made  between  select  legal  rule  and  select  fact.  There  also  appears  a 
specific  part  in  the  construct  legal  solution  screen  that  is  titled  legal  solution.  This  is  were  the  student  has  to  put 
the  outcomes  of  the  various  activities,  and  so  construct  the  legal  solution  (see  Fig.  1). 


Select  a Legal  Rule 

The  student  now  has  to  select  a legal  rule  by  choosing  the  legal  rules  button  in  the  legal  rules  part  in  the  upper 
layer.  This  button  shows  three  different  source  categories  of  legal  rules:  statutes,  other  regulations  and 
precedents.  Within  the  statute  option  a further  classification  of  statutes  is  made  based  on  the  area  of  law  the 
statutes  belong  to.  The  student  selects  the  option  statutes  from  the  legal  rules  button  and  then  selects  the  statute 
that  is  applicable  given  the  specific  legal  case  and  question  to  be  answered  (e.g.  the  General  Administrative 
Law  Act).  The  selected  statute  is  presented  in  the  legal  rules  part  of  the  screen.  Then  the  student  has  to  select  an 
applicable  article  (e.g.  Interested  party  means  the  person  whose  interest  is  directly  affected  by  an  order. 
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This  article  has  to  be  dragged  to  the  construct  legal  solution  part  of  the  screen,  in  the  specific  sub  part  select 
legal  rules.  The  student  can  then  bring  this  article  to  the  legal  solution  (see  Fig.  1). 


Match  Article  Component  and  Fact 

An  article  is  still  too  abstract  to  be  matched  to  a specific  case  fact.  Therefor  the  student  has  to  decompose  the 
article  in  its  article  components  (e.g.  the  person;  an  order;  interest  is  directly  affected).  The  selection  of  an 
article  component  is  followed  by  the  selection  of  a fact  from  the  legal  case  that  can  be  related  to  the  article 
component  (e.g.  Alexander  Boer).  Using  the  match  option  in  the  process  menu  relates  the  article  component  to 
the  selected  fact  (e.g.  the  person  = Alexander  Boer).  This  relation  is  then  automatically  placed  in  the  legal 
solution. 


Answer  the  Question 

The  select  activity  has  to  be  repeated  until  there  are  no  statutes,  articles,  article  components  and  facts  left.  The 
match  activity  has  to  be  repeated  until  there  are  no  more  article  components  or  facts.  At  that  stage  the  student 
has  to  formulate  the  final  answer  to  the  question.  She  has  to  select  the  menu  button  product  and  choose  the 
option  answer  (e.g.  A.  Boer  is  not  an  interested  party  in  the  meaning  of  the  GALA)  (see  Fig.  2).  The  product 
button  available  in  the  construct  legal  solution  part  in  the  upper  layer  of  the  screen  offers  a set  of  option.  These 
are:  bring  to  solution,  cut,  up,  down,  large  screen  and  answer.  The  option  large  screen  is  offered  to  enlarge  the 
legal  solution  part  to  get  a better  overview  of  the  legal  solution  constructed  so  far. 
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Figure  2:  formulate  answer 


Elaborations  and  Response 

In  the  lower  layer  the  support  is  presented.  There  are  two  types  of  support:  elaborations  and  response.  The 
student  may  request  support  at  any  moment  in  the  legal  solution  construction  process.  An  example  of  an 
elaboration  available  in  the  legal  case  part  of  the  lower  layer  in  the  form  of  a list  of  domain  concepts  is 
presented  in  Fig.  3.  When  selecting  a particular  domain  concept,  for  instance  interested  party,  the  background 
knowledge  about  the  concept  and  a reference  to  statutes  and  articles  within  statutes  are  presented. 

The  student  can  use  the  assess  button  in  the  construct  legal  solution  part  in  the  upper  layer  to  ask  for  an 
assessment  of  her  activities  (the  process)  and  her  legal  solution  (the  product).  The  process  is  assessed  according 
to  the  route  followed.  The  product  is  assessed  on  completeness,  correct  sequence  and  correct  answer.  The 
assessment  results,  specified  to  process  and  product,  are  presented  in  the  construct  legal  solution  part  in  the 
lower  layer. 
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Figure  3:  presentation  of  an  elaboration 


Implementation  of  the  computer  program  PROSA 

The  program  Authorware  is  used  for  implementing  the  specification  of  the  design  of  the  instructional 
environment.  Authorware  is  an  authoring  environment  for  creating  and  publishing  interactive  information  and 
can  be  used  for  the  construction  of  interactive  learning  and  training  applications.  Authorware  has  many 
evaluation  functions  that  make  it  possible  to  handle  all  kinds  of  input.  The  program  Authorware  has  been 
chosen  for  the  realization  of  PROSA  on  the  basis  of  these  specific  Authorware  cispects:  interactivity  and 
evaluation  of  input.  As  Authorware  is  an  icon-based  authoring  tool,  a program  is  made  by  assembling  icons  on  a 
flow  line.  Different  types  of  icons  contain  different  types  of  objects  like  text,  graphics  or  a set  of  instructions 
and  herewith  the  content  of  a program.  The  way  in  which  these  icons  are  arranged  on  the  flow  line  forms  the 
architecture.  In  Fig.  4 the  top  level  of  the  architecture  of  PROSA  is  shown.  The  icons  on  the  main  flow  line  are 
visible  at  this  level. 


best  copy  available 


SIO 


Figure  4:  the  top  level  of  the  PROSA  architecture 


Initialize 

When  PROSA  is  run,  Authorware  executes  the  icons  from  top  to  bottom  along  the  flow  line.  The  first  icon  that 
is  executed  is  the  map  icon  ‘initialize’.  This  map  icon  contains  a number  of  display  icons  that  contain  the  first 
screen,  the  so-called  startup  screen.  Furthermore  the  variables  used  in  PROSA  are  defined  and  initialized  in  a 
calculation  icon.  These  variables  are  used  to  keep  track  of  the  students’  actions  and  to  store  and  use  general 
PROSA  information  like  available  items  for  the  menu  buttons.  And  finally  the  ‘initialize’  icon  contains  display 
icons  which  contain  the  standard  PROSA  screen  with  the  two  layers  and  three  parts  (see  Fig.  1).  This  ‘initialize’ 
icon  is  executed  only  once  per  session. 


Perpetual  interactions 

The  second  icon  ‘perpetual  interactions’  (see  Fig.  4)  contains  two  kinds  of  so  called  perpetual  interactions.  An 
interaction  is  an  interaction  icon  with  different  types  of  response  type  symbols  attached  to  it.  These  response 
type  symbols  tell  the  interaction  icon  whether  to  display  a button,  a menu,  a text-field  or  some  other  element. 
An  interaction  monitors  the  actions  of  the  student  and  sends  that  information  to  the  response  type  symbols 
attached  to  it.  If  Authorware  encounters  a perpetual  interaction  it  activates  the  interaction  and  immediately 
continues  down  the  flow  line.  This  is  used  in  PROSA  because  the  student  is  given  personal  control  in  learning 
to  solve  legal  cases.  The  first  perpetual  interaction  displays  the  menu  buttons  available  for  the  six  different  parts 
and  defines  the  reaction  of  PROSA  when  the  student  uses  the  buttons.  The  student  throughout  the  whole  session 
of  solving  a legal  case  can  use  these  buttons.  The  second  perpetual  interaction  defines  the  responses  of  PROSA 
to  the  students  activities  regarding  the  construction  of  a legal  solution  in  the  sub  parts  select  rule,  select  fact  and 
legal  solution  part  in  the  construct  legal  solution  part  of  the  screen.  An  example  of  such  a student  activity  is 
dragging  an  article  to  the  ‘select  legal  rule’  sub  part  of  the  construct  legal  solution  part. 


Main  loop 

After  the  perpetual  interactions  the  main  loop  of  the  program  occurs,  the  so  called  decision  icon  ‘legal  case’ 
with  map  icons  for  every  case  topic  attached  to  it.  Within  each  case  topic  map  a similar  decision  icon  for  the 
different  levels  of  difficulty  is  used.  When  Authorware  encounters  a decision  icon  it  branches  to  a path 
according  to  certain  criteria.  In  PROSA  these  criteria  are  the  choices  the  student  makes.  The  student  uses  the 
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menu  button  legal  case  and  the  choices  she  makes  are  stored  in  two  variables  caseTopic  and  caseDifFiculty level. 
On  the  basis  of  these  variables  Authorware  first  branches  to  the  map  icon  of  the  chosen  case  topic  and  then  to 
the  map  icon  of  the  chosen  difficulty  level.  A difficulty  level  map  icon  contains  (1)  display  icons  which  contain 
the  legal  case  text  and  the  accessory  question  (2)  a calculation  icon  in  which  the  correct  legal  solution  is  stored 
in  a variable  (3)  an  interaction  icon  to  monitor  the  students  activities  specific  to  the  chosen  case.  In  the 
architecture  a specific  legal  case  and  the  accessory  question  are  considered  to  be  the  basic  element,  because  it  is 
the  current  problem  to  be  solved  by  the  student  and  in  this  way  PROSA  is  able  to  give  case-  and  student  specific 
feedback.  Also  many  student  characteristics  can  be  recorded  per  case,  like,  for  instance,  the  sequence  of  the 
students  activities  in  solving  the  legal  case,  the  legal  solution  the  student  constructs  and  the  cases  the  student 
selects.  These  student  characteristics  are  an  example  of  characteristics  that  are  recorded  and  maintained  during 
all  sessions  of  the  student  working  with  PROSA.  In  this  way  a student  history  is  built  to  be  able  to  adapt  to  the 
individual  students  activities  and  to  evaluate  the  individual  student. 


Subroutines 

The  last  icon  at  the  top  level  flow  line  is  the  map  icon  ‘subroutines’  and  Authorware  never  automatically 
encounters  it.  This  icon  contains  a number  of  subroutines  implemented  as  map  icons  attached  to  framework 
icons.  These  subroutines  appear  only  once  in  PROSA,  but  are  called  many  times  by  various  parts  of  the 
program.  An  example  of  a subroutine  is  adding  a student  activity  to  the  list  that  is  used  to  keep  track  of  the 
series  of  activities  the  student  carries  out  to  construct  a legal  case  solution.  Because  of  the  way  the  main  loop  in 
PROSA  is  structured,  new  legal  cases  of  the  existing  topics  and  difficulty  levels  can  be  added  easily. 
Furthermore,  legal  cases  of  new  topics  and  difficulty  levels  can  be  added.  For  each  new  case  topic  and  difficulty 
level  a new  map  icon  containing  the  case  specific  display,  calculation  and  interaction  icons  has  to  be  added. 
The  same  structure  can  also  easily  be  used  for  a different  domain  if  the  problems  to  be  solved  can  be  divided  in 
a hierarchy  of  topics  and  difficulty  levels. 
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Abstract:  In  this  study,  we  identify  the  characteristics  of  project-based  learning  activities  in  an 
online  environment.  Based  on  the  data  collected  from  17  students  in  a Web-based  graduate 
telecommunications  course,  we  found  that  the  project-based  activities  were  authentic  and  useful  in 
terms  of  gaining  a variety  of  skills  like  group  collaboration  and  communication,  facilitation  of 
online  activities,  and  intercultural  communication.  We  describe  the  strategies  used  by  the  instructor 
and  by  the  student  facilitators  and  participants  in  project-based  learning  activities  in  this  online 
course.  We  also  stress  the  importance  of  these  kinds  of  characteristics  and  strategies  of  project- 
based  learning  activities  in  terms  of  course  design  and  implementation. 


Introduction 

The  surge  in  Web-based  and  Web-supported  courses  has  prompted  course  designers  and  instructors  to 
identify  appropriate  ways  to  adapt  traditional  courses  to  a Web  environment  (Khan,  1997).  An  instructor’s  typical 
responsibility  is  to  design,  develop,  and  teach  courses  and  to  conduct  research.  However,  in  online  learning 
environments,  the  focus  tends  to  change:  the  instructor’s  role  shifts  from  that  of  the  sole  dispenser  of  information  to 
that  of  facilitator  and  resource  for  learning  activities  (Gunawardena,  1992).  The  online  environment  is  constructivist 
in  nature,  providing  an  arena  for  collaborative  learning  and  learner-centered  activities.  Active  construction  of 
meaning  (Vygotsky,  1978)  about  experiences  should  take  place  through  "experiential  exercises  followed  by 
interpersonal  interaction  in  small  groups,  and  with  facilitators  to  guide  the  group  towards  useful  conclusions" 
(Romiszowski,  1997,  p.  33).  Project-based  learning  involves  students  working  collaboratively  to  solve  a problem 
that  results  in  a product.  Project-based  learning  on  the  Web  presents  special  challenges  and  opportunities  that  groups 
in  conventional  settings  may  not  encounter. 

This  case  study  describes  a Web-based  course  in  instructional  telecommunications  at  Texas  A&M 
University.  The  following  questions  guided  the  inquiry  into  project-based  learning  on  the  Web: 

1 . What  are  the  characteristics  of  project-based  learning  activities  that  produce  the  desired  learning  outcomes  in  a 
Web-based  course? 

2.  What  are  the  strategies  that  the  instructor,  student  facilitators,  and  student  participants  use  to  prepare  and 
conduct  project-based  learning  activities  in  a Web-based  course? 

Theoretical  Framework 

Two  theoretical  assumptions  frame  this  study  about  Web-based  courses:  learner-centered  instruction  can 
foster  positive  learning  outcomes,  and  project-based  learning  can  encourage  learner-centered  instruction. 


Learner-centered  instruction.  Researchers  have  described  a role  change  among  distance  education  faculty  from 
the  instructor  as  a content  expert  to  a facilitator  of  learning  and  an  accompanying  shift  from  teacher-centered  to 
learner-centered  instruction  (Gunawardena,  1992;  Harasim,  Hiltz,  Teles,  & Turoff,  1995).  Providing  learner- 
centered  instruction  requires  the  instructor  to  understand  the  distance  learner.  In  the  introduction  to  her  edited  book 
about  distance  learners  in  higher  education,  Gibson  (1998)  challenges  faculty  to  “do  more  than  provide  access  to 
information.  We  need  to  truly  understand  that  learner  and  design  learning  environments  that  facilitate  learning, 
environments  that  enhance  access  to  and  success  in  higher  education’*  (p.  viii). 

In  constructivist  learning  environments,  the  instructor  provides  the  overall  structure  and  the  parameters  for 
the  course.  In  computer  conferencing  environments,  ’’course  design  becomes  more  important,  and  preparation  entails 
the  structuring  of  conferences  and  topics,  and  the  design  of  activities  and  small  group  work”  (Romiszowski  & 
Mason,  1996,  p.  447).  Students  in  learner-centered  settings  construct  their  own  learning  within  the  framework  of  the 
course  in  their  attempts  to  make  sense  of  their  experiences. 

Project-based  learning.  To  meet  learners’  needs  and  allow  them  to  profit  directly  from  their  learning  experience, 
instructors  need  to  provide  authentic  activities.  Such  ’’coherent,  meaningful,  and  purposeful  activities"  (Brown, 
Collins,  & Duguid,  1989)  are  similar  to  those  that  Romiszowski  (1997)  recommends  for  developing  learners’  critical 
thinking  skills:  small-group  discussions,  simulation  games,  project-based  work,  and  collaborative  problem-solving 
activities. 

One  of  these  activities,  project-based  learning,  is  described  as  learners  working  collaboratively  over  an 
extended  period  of  time  to  solve  an  authentic  and  challenging  problem  that  results  in  an  end  product  (Fung,  1996; 
Moursund,  1998).  Project-based  activities  have  explicit  educational  goals  and  are  typically  learner-centered,  rooted 
in  constructivism,  and  facilitated  by  a teacher  who  acts  as  a facilitator.  The  ’’rich  environments  for  active  learning" 
in  which  project-based  learning  is  likely  to  occur  engage  learners  ”in  a continuous  collaborative  process  of  building 
and  reshaping  understanding”  (Dunlap  & Grabinger,  1996,  p.  67).  Fung  (1996)  lists  the  following  advantages  of 
project-based  learning — the  encouragement  of  student  initiative,  self-directedness,  inventiveness,  and  independence. 

The  literature  has  identified  ideal  characteristics  and  responsibilities  of  learners  and  instructors  involved  in 
project-based  learning.  For  example,  a study  of  British  Open  University  students  participating  in  project-based 
learning  in  a telecommunications  environment  found  that  the  students  were  required  to  have  heightened  levels  of 
self-confidence,  motivation,  and  the  ability  to  organize  their  own  work  plans  (Fung,  1996). 

The  Project 

"Applications  of  Telecommunications  in  Education”  is  a Web-based  graduate  course  that  focuses  on 
instructional  applications  of  telecommunications;  the  analysis  of  characteristics  of  varied  systems,  both  dedicated 
and  public  networks;  and  the  design  of  appropriate  instructional  strategies  and  methods  using  those  systems.  The 
course  is  designed  to  help  learners  apply  the  latest  developments  in  telecommunications  technologies  to  specific 
education  or  training  contexts. 

The  instructor  designed  the  Fall  1999  course  for  students  to  engage  in  constructivist  learning  through  a 
combination  of  independent  and  project-based  learning  activities.  The  course  materials  were  available  on  the  Web, 
and  the  students  used  FirstClass^^  computer  conferencing  software  to  communicate  with  their  classmates  and  the 
instructor  and  to  do  all  of  their  course  work.  The  students  sent  private  messages  through  the  FirstClass  closed  email 
system  and  public  messages  to  the  conferences,  and  they  used  the  chat  function  to  hold  scheduled  and  impromptu 
real-time  text-based  discussions  with  their  project  groups.  FirstClass  also  allowed  them  to  attach  files  and  work 
together  on  collaborative  documents. 

Students  had  three  independent  activities  in  this  course.  The  major  independent  activity  was  a "report  for 
action”  addressing  an  educational  or  training  problem  that  might  be  solved  through  the  application  of 
telecommunications.  Students  were  also  required  to  keep  a telecommunications  journal  for  their  reflections  on 
learning  and  teaching  in  an  online  environment.  The  students’  classmates  and  the  instructor  could  read  these 
"public”  journals.  Finally,  the  students  completed  pre-existing  evaluation  forms  for  group  presentations,  Web  sites, 
formative  evaluations,  and  pre-  and  post-course  surveys. 

Two  project-based  learning  activities  that  took  place  in  small  groups  — generally  three  to  four  students  per 
group — were  facilitating  the  unit  activities  and  developing  a Web  site.  The  five  two-week  units  were  based  on  the 
assigned  readings  and  included  discussions,  role-play,  case-study  analysis,  Web  site  evaluations,  and  panel 
discussions.  The  students  were  told  in  advance  which  type  of  activity  would  be  used  for  each  unit  theme.  This 
project  was  designed  to  assist  students  with  their  individual  action  reports,  to  guide  them  through  the  content  of  the 
course,  and  to  foster  skills  in  communicating  effectively  with  co-facilitators  in  planning,  implementing,  and 
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synthesizing  the  unit.  The  other  project-based  learning  activity  was  creating  and  publishing  a Web  site  about  an 
issue  critical  to  telecommunications  in  education.  The  students  selected  their  topics  from  a list  of  options  including 
equity  issues,  digital  libraries,  futuristic  issues,  literacy,  and  copyright  issues  and  were  assigned  to  those  groups  on 
the  basis  of  their  common  interests.  The  five  critical  issues  Web  sites  were  designed  to  be  used  by  future  classes  as 
well  as  by  the  general  public. 

Methods 

This  study  is  an  investigation  of  two  project-based  learning  activities  in  a graduate  level  Web-based  course. 
We  asked  about—  (a)  the  characteristics  of  project-based  learning  activities  that  produce  the  desired  learning 
outcomes,  and  (a)  the  participants'  strategies  for  preparing  and  conducting  the  project-based  learning  activities. 

Participants.  The  participants  were  the  17  students  enrolled  in  the  Web-based  course  in  Fall  semester  1999. 
Thirteen  of  the  students  were  females  and  only  four  were  males.  Ten  were  master's  students  primarily  in  the 
educational  technology  program,  while  the  seven  doctoral  students  represented  three  departments  in  the  College  of 
Education,  and  one  was  from  Computer  Science.  The  class  consisted  of  an  unusually  high  percentage  of 
international  students:  nine  of  the  students  were  from  the  United  States,  while  the  other  eight  were  international 
students  who  spoke  English  as  a second  language  (ESL). 

No  students  had  taken  a Web-based  course  before  enrolling  for  this  course,  though  two  of  the  students  were 
taking  a second  Web-based  course  simultaneously  from  the  same  instructor.  The  students'  previous  experience  with 
taking  coursework  at  a distance  was  limited  to  two-way  interactive  videoconference  in  combination  with  computer 
conferencing.  Their  telecommunications  skills  were  varied;  one  student  used  email  only  rarely  prior  to  the  beginning 
of  the  course;  most  were  accustomed  to  using  computers  and  telecommunications  in  their  studies;  and  a few  had 
jobs  that  were  based  on  telecommunications. 

Data  sources.  The  data  sources  included:  (a)  students'  online  communication  through  FirstClass  threaded  computer 
conferencing  messages,  collaborative  documents,  and  real-time  chats;  and  (b)  students'  evaluations  and  assessments, 
and  interviews  with  the  students  via  FirstClass. 

Evaluations  and  assessments  were  used  prior  to  beginning  the  course,  during  the  semester,  and  at  the  end  of 
semester  for  varied  reasons.  The  pre-course  surveys  provided  participants'  background  information  and  their 
perceived  readiness  for  an  online  class.  During  the  semester,  students  assessed  their  own  and  their  group  members' 
contributions  to  the  two  group  projects  by  completing  and  sending  private  group  evaluation  forms  directly  to  the 
instructor.  Formative  evaluations  took  place  as  a single  collaborative  document,  to  which  the  students  contributed 
anonymously.  Beginning  in  the  fifth  week  of  the  semester,  the  instructor  asked  the  students  to  "assess  how  things  are 
going"  by  contributing  to  the  formative  evaluation.  During  the  last  two  weeks  of  the  semester,  the  instructor  asked 
the  students  to  contribute  to  another  collaborative  document  "Final  Thoughts"  to  get  feedback  on  the  various  aspects 
of  the  course,  including  the  books,  assignments,  and  the  use  of  FirstClass  for  communication  and  the  Web  for 
obtaining  course  information.  Students  also  completed  an  online  course  evaluation  administered  by  the  university. 

The  graduate  assistant  used  FirstClass  to  conduct  the  interviews.  Interviews  were  open-ended  and  included 
questions  about  students'  experiences  as  facilitators  and  participants  in  project  groups.  The  questions  aimed  to 
uncover  perceptions  about  the  different  types  of  activities. 

Data  Collection  and  Analysis.  The  investigation  relied  on  case  study  methods,  which  are  characterized  by  rich 
description  of  processes  (Merriam,  1998).  The  case  study  describes  the  characteristics  of  problem-based  learning 
activities  and  the  strategies  used  to  prepare  and  conduct  these  activities  in  a Web-based  course.  We  conducted  the 
electronic  interviews  and  downloaded  all  of  the  electronic  evaluations  and  assessments  from  FirstClass  conferences 
and  from  the  Web,  copying  them  into  individual  files  for  each  data  source.  We  conducted  content  analyses  of  the 
data  sources  (Krippcndorff,  1980).  This  process  consisted  of  coding  the  data  by  categorizing  them  according  to  the 
two  research  questions,  using  QSR  NUD*IST  (Non-numerical  Unstructured  Data*Indexing  Searching  and 
Theorizing)  qualitative  data  analysis  software.  For  each  major  theme  regarding  the  research  questions,  we  created 
“nodes”  or  categories;  and  for  each  sub-theme  we  created  “child  nodes”  or  sub-categories.  We  generated  four  nodes 
(i.e.,  Website  Projects,  Group  Activities,  Facilitator  Strategies,  and  Participant  Strategies)  and  40  child  nodes  (e.g.. 
Preparation  in  Advance  and  Periodic  Contact  for  the  node  Facilitator  Strategies). 

We  reported  on  the  data  using  the  participants'  verbatim  accounts.  We  established  the  following  method  to 
identify  messages  that  are  quoted  verbatim  in  the  text:  the  data  source  and  the  author  of  the  message  (if  known).  The 
data  sources  are:  I=Interview,  F=Final  Thoughts,  and  S=End  of  Course  Surveys.  Each  student  was  assigned  a 


number.  Message  identification  is  enclosed  in  brackets  after  the  verbatim  messages  in  the  text.  For  example,  a 
message  that  Student  1 1 wrote  in  the  Interview  would  be  identified  with  this  description:  [I-l  1]. 

Results 

The  results  of  the  two  research  questions  are  addressed  in  this  section.  These  questions  are  concerned  with 
the  characteristics  of  project-based  learning  activities  that  produce  the  desired  learning  outcomes,  and  the 
participants’  strategies  for  doing  project-based  learning  activities. 

1.  What  are  the  characteristics  of  project-based  learning  activities  that  produce  the  desired  learning 
outcomes  in  a Web-based  course? 

The  intended  learning  outcomes  for  project-based  learning  activities  were  based  on  the  learning  objectives 
specified  at  the  beginning  of  the  course.  The  project-based  learning  objectives  were-- 

• In  a small  group,  create  and  publish  a well-designed  research-based  Web  site  that  focuses  on  a critical  issue 
facing  telecommunications. 

• Demonstrate  proficiency  in  participating  in  and  facilitating  online  activities  and  discussions. 

We  identified  four  characteristics  of  project-based  learning  activities  that  produce  the  desired  learning 
outcomes  in  a Web-based  course:  (a)  authentic  activities;  (b)  collaborative  work  and  communication  via 
telecommunications;  (c)  opportunities  for  knowledge  enhancement  and  skill  building;  and  (d),  intercultural 
communication  skills. 

First,  the  learning  experiences  were  designed  to  be  authentic,  i.e.,  the  students  were  to  engage  in  actual 
real-life  situations.  The  students  created  real  Web  sites,  which  were  research-based,  published  on  the  Web,  and  used 
by  others  to  obtain  needed  information  about  critical  issues  in  telecommunications.  Most  students  responded  that  the 
Web  site  activity  was  fairly  authentic,  although  many  students  ’’would  have  liked  one  less  group  activity  and  more 
time  to  improve  on  those  completed”  (F-anonJ.  The  students  perceived  the  unit  group  activities  to  be  somewhat 
more  authentic,  as  they  planned,  implemented,  and  synthesized  the  activities  related  to  the  assigned  readings.  One 
student  shared  that,  to  him,  authentic  ’’means  purpose  was  generated  by  the  learner  and  acted  upon  ...  in  terms  of 
learner  arranged  tasks,  i believe  it  was  authentic”  [1-12].  However,  he  continued  to  explain  that  ”to  be  truly 
authentic,  the  students  would  have  had  to  recognized  the  problem"  [M2],  possibly  conveying  that  the  students  didn’t 
select  their  own  topics  and  instead  were  presented  with  a list  from  which  to  choose.  Two  students  wrote  in  the  final 
course  survey  that  the  course  was  strengthened  by  projects  that  were  "meaningful”  to  them  [F-anon]. 

Second,  the  project-based  learning  activities  provided  opportunities  for  the  students  to  work  as  a team, 
which  is  an  activity  encountered  frequently  in  real-life.  By  participating  fully  in  these  activities,  the  students  learned 
how  to  work  collaboratively  in  a group  and  communicate  via  telecommunications.  An  ESL  student  explained  how 
the  group  projects  helped  her:  "By  working  with  my  partners,  I improved  my  language  skills,  ability  of  organizing 
my  thinking,  and  rate  of  interactions  with  others,  also  technological  skills  for  handling  online  communications"  [I- 
17].  To  another  ESL  student,  the  projects  were  "a  means  of  learning  how  cooperate  and  exchange  ideas,  knowledge, 
materials"  [1-8].  The  same  ESL  student  continued  to  explain  how  the  Web  site  project  provided  a good  opportunity 
for  a "teamwork  learning  environment": 

1)  It  has  several  different  aspects  such  as  site  development,  presentation  to  content  selection;  2) 

Many  times  group  members  need  to  decide  things  for  its  group  and  a compromising  decision 
process  happens;  3)  Each  group  member  has  strengths  and  weaknesses.  When  members  working 
together  they  can  learn  to  improve  weakness  and  creating  new  ideas  with  strengths.  [1-8] 

Third,  the  activities  fostered  the  development  and  enhancement  of  a variety  of  skills.  Some  students 
referred  to  their  learning  gains  in  terms  of  objectives,  while  others  identified  a variety  of  skills  they  gained  as  an 
outcome  of  these  activities.  They  referred  to  having  built  content  knowledge,  even  becoming  an  expert,  as  one 
student  expressed,  “at  the  end,  I became  an  expert  in  that  issue  to  some  extent”  [1-8].  They  also  referred  to  the  dual 
benefit  of  applying  new  content  knowledge  to  Web  site  creation.  One  student  claimed  that  "the  project  was  a good 
one  because  not  only  was  I learning  about  the  topic,  . . . but  I was  also  becoming  better  at  making  web  pages"  [1-5]. 
Similarly,  "It  not  only  help  me  understand  the  topic  itself,  I also  learn  a lot  about  how  to  design  a web  site"  [1-4]. 

Fourth,  the  activities  provided  for  the  development  of  intercultural  communication  skills  by  working  in  a 
multicultural  environment.  The  American  students  developed  the  skills  as  “a  conscious  effort  to  keep  from 
regressing  into  Texas  slang  and  colloquialism”  [1-12].  Another  American  student  reflected,  “I  believe  that  in  the 
planning  and  implementation  of  the  activities,  groups  had  to  be  sensitive  to  the  differences  among  cultures”  [1-5]. 
On  the  other  hand,  the  ESL  students  encountered  difficulties  in  online  communications,  particularly  in  real-time 
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chats,  when  they  "cannot  express  what  [they]  really  meant  and  understand  what  other  meant”  [1-5].  Still,  they 
reported  several  benefits  of  group  projects.  For  example,  an  ESL  student  acknowledged,  "When  we  had  a chat  to 
discuss  our  project,  my  language  skill  prevented  me  from  exactly  expressing  my  thinking.  However,  the  members 
in  my  group  encourage  me  a lot  and  helped  me  when  I encountered  some  difficulties"  [1-17].  This  effort  to  pay 
attention  was  described  by  an  American  student  who  said,  "I  did  realize  that  my  informal  writing  tends  to  include 
many  terms  people  from  other  countries  might  not  understand  - and  caught  myself  many  times  backspacing  and 
pondering  a more  understood  phrase!"  [1-4]. 

2.  What  are  the  strategies  that  the  instructor,  student  facilitators,  and  student  participants  use  to  prepare 
and  conduct  project-based  learning  activities  in  a Web-based  course? 

The  instructor,  the  students  as  facilitators,  and  the  students  as  participants  used  a variety  of  strategies  in 
preparing  and  conducting  project-based  learning  activities  in  the  Web-based  course. 

Before  the  course  began,  the  instructor,  who  acted  as  a facilitator  of  learning  rather  than  as  a content  expert, 
developed  the  course  with  project-based  learning  activities  in  mind.  Preparation  for  project-based  learning  involved 
structuring  the  conferences  and  topics,  and  designing  the  activities  and  small  group  work.  The  instructor  arranged 
for  the  students  to  develop  group  learning  contracts  to  govern  their  group  behavior.  During  the  semester,  the 
instructor  had  real-time  chats  with  each  group  to  help  plan  the  unit  activities.  The  instructor  also  monitored  the 
progress  of  groups,  gave  feedback  to  the  groups,  and  contacted  specific  individuals  and  groups  as  needed. 

The  co-facilitators  scheduled  and  posted  periodic  live  chats  before,  during,  and  after  their  two-week  unit 
activities.  The  facilitators  saved  and  posted  a total  of  17  live  chats  for  the  5 group  activities  during  the  semester.  One 
chat  strategy  was  “to  wait  for  other  group  members  who  didn’t  type  as  fast  as  I do,  or  who  needed  more  time  to 
formulate  a response”  [I-l].  One  facilitator  gave  value  to  the  group  learning  contracts  written  at  the  beginning  of  the 
semester,  explaining  that  she  needed  “to  finish  all  the  required  reading  in  time  and  not  break  the  contract  in  the 
group  work"  [1-14].  The  facilitators  posted  drafts  of  communications,  instruction,  and  syntheses;  and  they  edited 
each  other’s  work.  They  used  collaborative  documents  to  develop  their  plans  and  create  outlines  and  documents, 
often  taking  care  to  be  “very  detailed  in  the  writing  of  the  procedures  for  the  activity  ...  I was  forced  to  look  at  every 
detail  of  the  process  . . . from  the  perspective  of  the  participants"  [1-5].  The  same  facilitator  explained  her  strategy: 

1 think  as  the  prepare  of  the  information,  it’s  best  come  at  the  information  asking  "What  is  that  I 
would  hope  to  get  out  of  this  web  site?"  "How  will  I provide  the  content  for  the  audience?”  and 
"What  can  I do  to  make  it  more  real  to  the  audience?"  "Are  there  other  technologies  that  can  be 
used?"  "Are  there  other  sites  I can  take  them  to?"  [1-5]. 

The  co-facilitators  monitored  their  activities  by  replying,  weaving  responses,  and  sending  private  email  to  individual 
participants  to  encourage  them  to  participate  by  “more  than  just  post[ing]  a response"  [1-5].  They  even  covered  for 
each  other  when  co-facilitators  had  a conflict  or  problem  during  the  activity. 

The  students  as  group  project  participants  deciphered  the  instruction  that  the  facilitators  developed  and 
posted,  and  they  logged  in  frequently  to  maintain  their  participant  role.  One  participant  explained,  “I  tried  to  check 
email  frequently  and  respond  immediately,  whether  I knew  the  answer  to  the  question  being  asked  or  not  - ’I’ll  get 
back  to  you’  is  better  than  not  hearing  from  someone  at  all”  [I-l].  They  also  assisted  other  participants  by 
contributing  their  ideas  and  editing  their  work,  and  they  conducted  peer  reviews  of  the  other  groups’  work.  In  the 
group  projects,  they  recognized  the  importance  of  giving  "enough  time  to  other  participants  for  conducting  learning" 
[1-2].  They  needed  this  time  for  preparing  to  participate  in  the  group  activities,  as  one  student  described: 

As  a participant,  I have  to  read  and  organize  the  thought  in  the  books,  then  answer  the  questions 
and  observe  the  discussion  among  all  participants  and  facilitators.  The  best  stance  of  a participant 
is  always  be  ready  to  answer  and  give  feedback,  so  it  is  natural  that  I have  to  finish  all  the  reading 
for  joining  the  discussion  [1-15]. 

Educational  Signiflcance 

Answers  to  the  research  questions  provide  insight  for  Web-based  course  designers  and  instructors  as  well 
as  students  in  Web-based  courses.  Knowing  important  characteristics  of  project-based  learning  activities  will  assist 
those  who  plan  to  incorporate  such  activities  into  the  design  of  a Web-based  course.  The  strategies  that  the 
instructor,  student  facilitators,  and  student  participants  used  to  prepare  and  conduct  project-based  learning  activities 
are  transferable  to  other  types  of  activities. 

The  findings  of  this  study  support  the  developing  body  of  literature  on  designing  for  online  environments. 
Researchers  posit  that  there  are  six  design  considerations  for  computer  conferencing,  two  that  involve  administrative 
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design  considerations-grading  system  and  grouping— and  four  instructional  design  considerations-collaboration, 
relevance,  learner  control,  and  technological  preparation  (Cifuentes,  Murphy,  Segur,  & Kodali,  1997).  Based  on  the 
data  collected  from  the  students  in  a Web-based  graduate  telecommunications  course,  we  found  that  the  project- 
based  activities  were  authentic,  or  relevant-useful  in  terms  of  gaining  a variety  of  skills  like  group  collaboration  and 
communication,  facilitation  of  online  activities,  and  intercultural  communication.  The  students  in  this  study 
recognized  the  value  of  trying  to  understand  each  other  as  they  worked  toward  common  goals.  For  example,  one 
ESL  student  concluded,  ”I  also  listened  to  the  others  and  saw  the  difference  and  similarities  among  us.  Sometimes  I 
needed  to  comprise  to  others  and  sometimes  I needed  to  stick  to  my  opinions.  Whatever  I did  was  to  show  I am  a 
responsible  and  collaborative  person  to  work  with  others  for  the  goals  of  successfully  finishing  the  project”  [1-15]. 
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Abstract:  For  most  students  inquiry-based  learning  environments  such  as  problem-based 
learning  (PBL)  comprise  a major  shift  away  from  approaches  to  learning  with  which  they 
are  most  comfortable  and  familiar.  We  have  been  observing  PBL  students  in  situ  with 
particular  focus  on  the  cognitive  processes  with  which  they  are  having  greatest  difficulty, 
the  processes  that  students  need  help  with,  and  specifically  when  in  the  PBL  process  they 
require  this  help.  The  outcomes  of  this  empirically-based  understanding  of  the  cognitive 
processes  with  which  students  have  difficulties,  is  assisting  us  with  the  development  of 
cognitive  support  tools  for  incorporation  into  the  inquiry-based  learning  environments. 


Research  Context  and  Background 

Problem-Based  Learning  (PBL)  which  represents  a form  of  inquiry-based  learning  is  being  widely  adopted  as  a 
powerful  teaching  and  learning  strategy  at  all  levels  of  education.  Situated  in  a constructivist  paradigm,  PBL  is  a 
learner-centered  pedagogical  strategy  where  the  students  themselves  assume  major  responsibility  for  their  learning. 
Generally  PBL  is  divided  into  several  phases  spread  over  periods  of  small  group  work  and  independent  study 
(Barrows  & Tamblyn,  1980;  Schmidt,  1993).  The  PBL  process  begins  with  an  encounter  of  a problem  situation 
through  the  use  of  a case  study  or  incident,  which  the  learners  analyze.  During  this  phase  learners  generate 
hypotheses  about  the  occurrence  of  the  problem  in  the  case  or  incident.  Based  on  this  exercise  learners  identify  what 
they  already  know  about  the  problem  and  determine  what  they  need  to  know,  and  develop  plans  to  research  the 
information.  Students  then  embark  on  a period  of  self-directed  learning  where  they  research  or  investigate  issues  that 
emerged  in  the  first  phase.  During  the  next  phase,  the  findings  of  individual  inquiry  are  shared  with  the  group,  which 
results  in  a re-evaluation  of  the  problem  and  perhaps  revision  of  their  first  perceptions  of  the  problem.  Through  this 
iterative  process,  students  develop  the  required  content  knowledge  and  problem-solving  skills.  Central  to  the  PBL 
process  are  the  following  cognitive  processes:  (1)  activation  of  prior  knowledge,  (2)  elaboration  of  content,  (3) 
restructuring  of  semantic  networks/schemata,  (4)  development  of  an  intellectual  scaffold. 


Activation  of  Prior  Knowledge 

The  first  step  in  PBL  involves  the  presentation  of  a problem,  which  the  learner  relates  to  his/her  prior  knowledge. 
Schmidt  (1993)  suggests  that  the  extent  of  prior  knowledge  about  a subject  is  one  of  the  major  determinants  of  the 
“nature  and  amount  of  new  information  that  can  be  processed’*  (p.  424).  Ausubel  (1968)  has  suggested  that  the 
essential  factor  in  the  acquisition  of  knowledge  is  that  content  must  be  linked  to  what  is  previously  known.  The 
exchange  of  ideas  between  participants  of  the  PBL  session,  under  the  guidance  of  the  PBL  tutor,  assists  in  activating 
or  mobilizing  prior  knowledge  of  students.  This  has  the  effect  of  creating  ‘learner  readiness’  for  subsequent  stages 
of  learning  by  asking  students  to  generate  hypotheses  for  the  problem. 


Elaboration 


This  comprises  exploration  of  alternative  interpretations  of  the  problem  and  an  understanding  of  relationships 
among  concepts.  The  storage  and  retrieval  of  information  is  said  to  be  enhanced  when  elaboration  of  the  material 
takes  place  (Schmidt,  1993^  This  elaboration  of  the  content  by  the  student  may  take  the  form  of  listening  to  other 
students'  hypotheses  and  determining  the  merit  and  worth  of  these  different  hypotheses.  The  cognition  involved  in 
evaluating  these  different  perspectives  begins  the  process  of  elaboration.  Use  of  other  learning  resources  such  as 
journal  articles,  books,  multimedia  teaching  modules  and  relevant  websites  should  also  help  elaborate  the  student’s 
understanding  of  the  problem. 


Restructuring 

Depending  upon  the  existing  knowledge  of  the  student,  a process  of  accretion,  tuning  or  restructuring  occurs  to 
actively  change  existing  schemata.  Gagne  (1986)  defines  schemata  as  “a  set  of  interconnected  propositions  centering 
around  a general  concept,  and  linked  peripherally  with  other  concepts”  (p.  8).  Schemata  are  not  static  but  continually 
evolving  in  content  and  structure.  When  new  learning  occurs,  new  schemata  develop  or  old  schemata  undergo 
structural  changes.  When  more  information  is  incorporated  into  an  existing  data  structure,  accretion  occurs.  Tuning 
refers  to  the  adjustment  of  existing  schemata.  The  continual  tuning  or  modification  in  categories  of  interpretation 
occurs  to  bring  the  categories  more  in  congruence  with  the  functional  demands  placed  on  them  (Gordon  & Rennie, 
1987).  Restructuring  is  an  important  process  for  changing  existing  schemata  or  developing  new  schemata  to 
interpret  new  information  (Keppell,  Elliott  & Harris,  1998). 

Group  discussion  is  an  asset  to  this  process.  Brown  and  Palincsar  (1989)  assert  that  change  is  more  likely  when 
individuals  are  required  to  explain,  elaborate  or  defend  their  positions  to  others  as  well  as  to  themselves,  which  gives 
rise  to  cognitive  conflict  in  the  individual.  Cognitive  conflict  arises  when  the  learner  is  exposed  to  disagreement 
between  existing  knowledge  and  new  anomalous  information.  Through  observation  and  explanation,  learners  can 
elaborate  and  integrate  knowledge  in  new  ways.  Chin  and  Brewer  (1993)  attribute  conceptual  change  to  the 
following:  status  of  the  anomalous  data  in  the  perception  of  the  students;  characteristics  of  prior  knowledge; 
learner’s  perception  of  the  credibility  and  validity  of  the  new  information;  and  processing  strategies.  Problem-based 
learning  attempts  to  create  and  induce  cognitive  conflict  within  learners  with  the  aim  of  bringing  about  conceptual 
change.  This  conflict  may  emanate  from  a disagreement  or  mismatch  between  existing  knowledge  of  learners  and 
the  knowledge  that  is  generated  from  the  problem  situation.  In  fact  this  is  an  axiom  of  PBL.  Typically,  alternative 
explanations  are  not  provided  by  tutors  but  are  constructed  in  small  group  discussions,  based  on  a combination  and 
evaluation  of  the  knowledge  of  all  group  members  and  verified  later  on,  during  self  study. 


Scaffolding 

In  the  process  of  PBL  the  student  will  develop  a framework  for  use  with  subsequent  clinical  problems.  Ausubel 
(1960)  referred  to  this  framework  as  an  “intellectual  scaffold”.  Intellectual  scaffolding  is  an  infrastructure  of 
information  to  which  new  material  can  be  anchored  (Ausubel,  1 960).  “By  beginning  a task  embedded  in  a familiar 
activity,  it  demonstrates  to  students  the  legitimacy  of  their  implicit  knowledge  and  its  availability  as  scaffolding  in 
apparently  unfamiliar  tasks”  (Brown,  Collins  and  Duguid,  1989,  p.  38). 


Challenges  and  Constraints 

While  PBL  has  substantial  advantages  for  teaching  and  learning  in  practice-based  professions  (Barrows  & Tamblyn, 
1980;  Barrows,  1994;  Koschmann,  Kelson,  Feltovich,  & Barrows,  1996),  for  most  educators  PBL  comprises  a 
significant  shift  away  from  approaches  to  teaching  with  which  they  are  most  comfortable  and  familiar.  For  most 
students,  at  least  in  the  beginning  stages,  PBL  is  somewhat  disorienting.  So  for  all  its  promise,  problem  based 
learning  poses  a number  of  challenges  for  both  teachers  and  students  alike.  A major  barrier  to  the  success  of  PBL  is 
students’  initial  discomfort  with  the  increased  degree  of  freedom  they  experience  in  a PBL  environment.  Students 
have  generally  been  enculturated  into  a model  of  schooling  in  which  their  role  is  more  or  less  limited  to 
comprehension  and  synthesis  of  instructor-specified  information,  based  on  instructor-formulated  learning  objectives, 
and  participation  in  instructor-led  learning  activities.  Many  students  often  lack  the  skills  even  to  know  where  to 
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begin  when  confronted  with  ill-defined,  real-world  problems,  let  alone  manage  the  complex  cognitive  processes  that 
are  part  of  PBL. 


Aims 

The  aims  of  this  research  program  have  been  to  observe  PBL  students  in  situ, 

• The  focus  of  this  observation  has  been  the  above-mentioned  cognitive  processes  associated  with  PBL:  (1) 
activation  of  prior  knowledge,  (2)  elaboration  of  content,  (3)  restructuring  of  semantic  networks/schemata,  (4) 
development  of  an  intellectual  scaffold. 

• Our  goal  has  been  to  pinpoint  the  cognitive  processes  and/or  components  of  these  processes  with  which  students 
have  been  having  difficulties. 

• We  expect  to  be  able  to  identify  the  cognitive  processes  that  students  need  help  with,  as  well  as  specifically 
when  in  the  PBL  process  they  require  this  help. 


Signiflcance  of  this  Research 

The  use  of  PBL  as  an  instructional  method  reflects  the  rise  of  teaching  and  learning  programs,  which  encourage 
students  to  take  responsibility  for  their  own  learning.  Such  programs  commonly  present  students  with  an  ill- 
structured  subject  area  and  require  them  to  conduct  their  own  inquiry  which  involves  formulating  hypotheses, 
developing  questions,  gathering  and  interpreting  data,  and  communicating  their  findings  to  peers  and  tutors  (Schank, 
1997;  Glasersfeld,  1987;  Brown,  Collins,  & Duguid,  1989;  Linn,  Songer,  & Eylon,  1996).  Our  work  seeks  to 
understand  the  challenges  and  constraints  of  these  processes  so  that  as  educators,  we  are  better  prepared  to  help 
students  achieve  the  desired  learning  outcomes.  Results  from  this  study  should  highlight  the  phases  of  PBL  and  the 
cognitive  processes  with  which  students  are  having  real  difficulty.  These  results  are  being  used  to  develop  software- 
based  support  tools  to  help  students  with  their  learning.  By  focussing  on  cognitive  processes  with  which  students  are 
experiencing  difficulties,  such  tools  have  the  advantage  of  being  tailored  to  the  specific  learning  needs  of  students. 
The  immediate  benefits  of  this  work  accrue  to  students  and  educators  associated  with  the  course  principally  being 
investigated.  However,  we  believe  that  our  work  has  significance  for  any  course  which  supports  student-centered 
learning  in  inquiry-based  educational  environments  either  here  at  the  University  of  Melbourne  or  elsewhere. 


Software-Based  Support  Tools 

While  a considerable  amount  of  work  has  gone  on  in  supporting  student  learning  with  various  types  of  cognitive 
tools  and  strategies  (see  for  example,  Jonassen,  Peck,  & Wilson,  1999;  Kommers,  Jonassen,  & Mayes,  1992;  Pea, 
1985;  Salomon,  Perkins,  & Globerson,  1991;  Naidu  & Bernard,  1992;  and  Bernard  & Naidu,  1992),  there  has  been 
little  work  carried  out  on  the  area  of ''software-based  cognitive  support  tools  or  strategies".  Existing  software-based 
cognitive  support  tools  for  ill-structured  learning  environments  is  believed  to  impose  an  inflexible  structure  on 
students’  learning  (see  Edelson,  & O’Neill,  1994;  Scardamalia,  & Bereiter,  1991).  These  support  tools  can  help 
students  organize  arguments  for  presentation  after  they  have  reached  a critical  summary  point  in  their  inquiry,  but 
they  are  less  useful  in  guiding  students  in  cognitive  processes  associated  with  PBL.  However,  before  we  can  begin 
building  any  such  software-based  support  tool  to  provide  students  with  the  kind  of  cognitive  support  they  need  for 
learning  in  ill-structured  learning  contexts,  we  need  to  know  exactly  what  cognitive  processes  students  have 
problems  with,  and  what  is  the  exact  nature  of  these  problems.  These  are  the  preliminary  questions,  which  this 
research  program  has  been  seeking  to  answer. 


Expected  Outcomes 

The  expected  outcomes  of  this  research  program  are  as  follows: 

• An  empirically  based  and  reliable  understanding  of  the  cognitive  processes  that  students  have  difficulties  with, 
while  engaged  in  the  PBL  process. 

• To  be  able  to  identify  the  specific  stages  in  the  PBL  process,  where  students  are  having  these  difficulties. 


• Assist  students  and  educators  identify  areas/phases  where  learners  require  assistance  and  areas/phases  where 
courses  require  refinement. 

• Empirical  data  from  this  investigation  provides  the  foundation  for  the  design  and  development  of  suitable 
software-based  cognitive  support  tools.  These  tools  will  be  incorporated  into  the  PBL  curriculum,  and  their 
effectiveness  will  be  evaluated  in  terms  of  their  use  as  well  as  benefits  to  the  learning  and  teaching  process. 


Research  Plan,  Methods,  and  Techniques 

Research  plan  and  methods 

This  research  utilizes  a "case  study"  approach.  The  classes  that  we  observe  comprise  the  case  studies.  Yin  (1994) 
defines  the  case  study  as  an  empirical  inquiry  that  investigates  a contemporary  phenomenon  within  its  real-life 
context.  Case  studies  are  able  to  explain  the  causal  links  in  real-life  situations  that  are  too  complex  for  the  survey  or 
experiment. 


Study  sample  and  procedure 

The  study  sample  is  a group  of  second  year  Agricultural  Science  students.  An  instructional  package  using 
multimedia  materials  designed  for  Intranet/Intemet  delivery  is  under  development  for  application  in  this  subject 
area.  The  students  will  be  the  first  trial  group  for  the  multimedia  materials.  Data  from  the  trial  group  will  be  used  to 
formatively  evaluate  the  multimedia  package,  in  preparation  for  its  implementation  with  students  on  several 
campuses.  The  study  will  focus  on  the  cognitive  processes  the  students  engage  in  while  learning  within  a PBL 
framework.  The  multimedia  package  introduces  the  students  to  a range  of  real  scenarios,  presented  using 
photographs  and  interviews  with  farmers.  Students  need  to  interpret  the  farmer's  conception  of  the  problem,  and 
develop  a solution  using  the  range  of  resources  built  into  the  program.  These  resources  include  interactive  maps, 
databases,  maps,  spreadsheets,  and  a pasture  growth  simulator  developed  during  a research  project,  and  interactive 
tutorials  on  key  concepts.  Students  select  whatever  they  feel  they  need  to  solve  the  problem  from  this  range  of 
resources. 

In  this  trial,  students  will  work  individually  to  cover  the  conceptual  base,  and  come  together  in  small  groups  to  reach 
their  resolution  of  the  problems.  As  the  students  will  be  all  together  for  this  formative  trial,  group  work  will  be  face 
to  face.  The  multimedia  materials,  however,  are  structured  to  allow  computer  conferencing  for  problem  solving. 
This  will  enable  students  to  work  collaboratively  with  students  on  other  campuses  or  external  students  to  work  in 
collaborative  groups. 


Development  of  data  gathering  instruments 

Data  gathering  in  case  study  research  typically  uses  multiple  sources  of  data.  For  this  study  direct  observation, 
questionnaires  and  a focus  group  will  all  be  oriented  to  the  identification  of  difficulties  that  the  students  encounter, 
that  may  hinder  their  problem  solving  processes.  Any  technical  problems  will  be  identified  by  observation  and, 
hopefully,  rectified  at  the  time.  The  questionnaire  and  focus  group  will  address  issues  relating  to  the  learning 
method  and  the  resources.  Tlie  questionnaire  will  be  oriented  towards  providing  some  quantifiable  data  that  can  be 
used  to  identify  proportions  of  students  who  encounter  specific  difficulties  or,  conversely,  feel  that  the  range  of 
aspects  of  the  materials  work  well.  The  focus  group  will  be  oriented  towards  more  subjective  issues  such  as  how  the 
group  functioned  in  the  problem  solving  process,  and  how  realistic  the  experience  of  working  this  way  is  perceived 
to  be. 


Data  gathering  and  analysis 

Data  from  direct  observation  will  be  analyzed  to  identity  any  major  impediments  to  problem  solving  that  the 
students  appear  to  be  encountering.  This  data  tends  to  reveal  significant  issues  such  as  technological  problems  or 
major  misconceptions  caused  by  inadequacies  in  the  project  materials  or  in  the  students'  perception  of  the  nature  of 
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the  task.  An  example  of  this  is  that  the  students  must  feel  that  the  topic  is  important  to  them,  rather  than  that  they 
are  actually  testing  software.  If  there  are  problems  of  this  nature  then  they  need  immediate  rectification.  If  the 
situation  cannot  be  properly  rectified  however,  then  direct  observation  data  provides  a context  for  consideration  in 
the  analysis  of  other  data.  Questionnaire  data  will  be  analyzed  to  reveal  trends  or  tendencies  that  indicate  the 
effectiveness  of  different  aspects  of  the  program  in  relation  to  stages  in  learning,  such  as  activation  of  prior 
knowledge,  elaboration,  and  restructuring.  The  way  the  students  felt  about  the  PBL  process,  and  how  they  perceived 
the  effectiveness  of  the  overall  learning  task  will  be  extrapolated  from  the  focus  group  data.  Following  this,  all  three 
sources  will  be  considered  together  to  enable  an  interpretation  of  major  themes  and  issues,  and  how  these  have  either 
helped  or  hindered  effective  learning. 


Results 

This  is  an  exploratory  study  that  is  in  progress.  Results  from  the  trial  will  reveal  a number  of  aspects  relating  to 
PBL  as  a method,  and  the  use  of  instructional  technology  for  delivery.  As  PBL  is  essentially  a way  of  structuring  a 
learning  process,  data  relating  to  how  students  experience  the  process,  and  its  effectiveness  in  promoting  effective 
cognitive  processes  is  important.  The  approach  to  analysis  of  the  case  study  is  designed  to  reveal  this.  It  is 
important  that  the  critical  processes  of  PBL  are  identified,  and  more  important  that  the  processes  in  PBL  with  which 
students  have  difficulty  with  are  identified.  Too  many  inquiry-based  technology-enhanced  learning  environments 
place  students  in  information  dense  learning  situations  without  appropriate  cognitive  supports  or  scaffolding.  Our 
research  agenda  is  to  develop  a suite  of  supports  that  students  can  use  in  such  learning  environments.  We  argue  that 
the  development  of  these  sorts  of  supports  and  scaffolding  must  be  based  on  empirical  data,  and  that  is  what  this 
research  program  is  all  about. 
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Abstract:  This  paper  shows  that  ability  grouping  in  the  middle  of  a semester  is  effective  for 
teaching  computer  literacy  to  freshmen  who  have  a variety  of  skill  levels.  Three  teachers  including 
the  authors  have  been  teaching  a computer  literacy  course  to  150  students  for  about  five  years.  In 
1998  and  1999,  we  reorganized  three  50-student  classes  based  on  an  achievement  exam  in  the 
middle  of  the  semester.  After  the  reorganization,  we  adjusted  lectures  according  to  the  students' 
ability.  Our  statistical  analysis  shows  that  the  exam  scores  were  improved  and  the  students'  self- 
assessment  scores  of  their  own  skills  were  upgraded  in  all  classes.  The  effect  was  significant  in  the 
slow-leamers  class  compared  to  the  other  two  classes. 


Introduction 

In  Japan,  computer  literacy  education  in  colleges  and  universities  is  facing  a problem  of  diversity  of  students* 
computer  skill  levels.  The  reason  for  this  is  twofold.  One  is  that  the  computer  literacy  education  is  incorporated  in 
very  small  number  of  high  schools.  The  other  is  that  as  computer  price  is  falling  down  the  number  of  students  is 
increasing  who  learn  how  to  use  computers  by  themselves  at  home.  Besides  the  skill  levels  at  the  beginning  of  the 
class,  there  are  differences  between  students  how  fast  they  get  accustomed  to  computer  operations  in  class.  These 
were  the  problems  to  overcome  for  us  to  help  students  learn  this  subject  in  good  quality. 

Since  1994,  the  computer  literary  education  has  been  a required  subject  for  all  freshmen  of  Osaka  University. 
Three  teachers  including  the  authors  have  been  teaching  a computer  literacy  course  to  over  150  freshmen  majoring 
in  Human  Sciences  for  five  years.  The  course  consists  of  15  sessions,  and  each  session  lasts  90  minutes,  which 
includes  demonstrations  by  a teacher  and  hands-on  activities  by  students.  Usually  an  assignment  is  also  given  which 
takes  an  hour  or  two  to  complete. 

At  the  beginning  of  the  course  in  1995  and  1996,  we  organized  three  parallel  50-student  classes  according  to 
the  students*  computer  experience  using  a questionnaire  survey.  Other  background  variables  (age,  gender,  future 
academic  field)  were  not  considered  for  this  class  organization.  The  organization  method  aimed  at  ability  grouping, 
that  would  provide  non-experienced  students  with  a slower  work  pace  and  allow  high-achieving  students  to  be 
sufficiently  challenged  by  more  demanding  lessons.  In  a few  months,  we  realized  that  our  approach  was  somewhat 
idealistic  and  did  not  work  well.  Nevertheless,  six  months  after  the  semester  had  ended,  we  conducted  a follow  up 
study  to  see  the  students'  attitudes  toward  computer  use,  and  compared  the  log-on  time  and  the  number  of  mails  sent 
when  they  were  freshmen  to  those  when  they  were  sophomores.  This  result  showed  that  the  sophomores  who  took 
this  course  in  their  first  year  used  computers  more  actively  than  when  they  were  freshmen.  There  was,  however, 
weak  positive  correlation  between  computer  experience  and  log-on  time  or  number  of  mails.  This  is  because,  we 
think,  students*  skills  were  not  objectively  grasped  by  the  questionnaire  and  because  experienced  students  are  not 
necessarily  diligent. 

In  1997,  we  introduced  a written  examination  in  the  middle  of  the  semester  in  order  to  reorganize  classes 
based  on  the  exam  score.  This  trial  showed  us  that  this  class  organization  was  not  so  effective  because  students* 
skills  were  not  objectively  grasped  by  questionnaire  and  written  examination. 

In  1998,  we  introduced  hands-on  work  into  the  exam  in  the  middle  of  spring  semester,  such  as  word  document 
processing,  looking  into  online  dictionaries,  and  finding  web  sites  using  a web  browser.  Then  we  reorganized  the 
three  classes  based  on  the  exam  score.  At  the  end  of  the  spring  semester,  we  again  gave  them  an  exam  that  would 
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measure  how  they  made  progress.  This  ability  grouping  strategy  brought  us  success.  In  1999,  we  refined  this 
method  and  compared  the  result  of  two  years  trial. 

In  this  paper,  we  begin  with  related  work  and  a brief  explanation  of  our  computer  literacy  course,  followed  by 
the  methodology  of  class  reorganization.  Then,  we  give  the  statistical  data  analysis  to  show  that  both  exam  score 
and  self-assessment  score  of  low-ability  class  were  significantly  improved. 

Related  work 

Although  there  has  been  much  previous  research  on  ability  grouping,  tracking,  and  class  organization,  they  are 
mainly  for  K-12  school  education,  e.g.  (Wiggins,  1993,  Mills,  1998).  Furthermore,  as  far  as  the  authors  know,  there 
has  not  been  any  research  on  class  reorganization  in  the  middle  of  a semester  for  computer  literacy  education.  The 
authors  have  tried  class  reorganization  for  years  and  presented  its  result  in  (Nakanishi  & Harada,  1999).  This  paper 
includes  the  result  in  1 999  and  completes  our  method. 

Computer  literacy  education 

Since  the  1970s,  the  definition  of  computer  literacy  has  evolved,  and  many  researchers  have  discussed  the 
courseware  and  teaching  methodology  of  the  computer  literacy  course,  e.g.  (Higdon,  1 995),  (Keizer,  1 997),  (Mills, 
1 998).  The  authors  believe  that  to  learn  details  of  word  processing  and  spread  sheet  application  is  not  important  but 
to  grasp  the  concept  and  principal  facilities  of  those  applications  is  a key  for  students  to  become  literate. 
Furthermore,  the  attitude  to  learn  by  himself  or  herself  is  also  a key. 

In  the  syllabus  of  the  computer  literary  course  in  Osaka  University,  topics  consist  of  two  categories, 
'Requisites'  and  'Options'. 

Requisite  category  consists  of  the  following  items. 

- Login/logout,  password,  touch  typing 

- File  system,  file  operation,  floppy  disks 

- Word  processing,  kana-kanji  conversion  (for  Japanese  characters) 

- Concept  of  the  Internet,  network  etiquette 

- Electronic  mail,  web 

- Draw  and/or  paint  software 

Options  are 

- Net  news 

- LaTeX  with  graphics 

- Spreadsheet 

- Writing  HTML 

- Mathematica 

-SAS 

- Script  languages  (perl,  awk,  shell,  etc.) 

- Computer  ethics,  social  problems 

- Computability. 

Since  one  course  session  lasts  90  minutes  and  is  held  once  a week,  all  topics  of  requisite  category  which 
authors  think  are  the  minimum  competencies,  are  covered  in  about  seven  weeks.  For  the  rest  of  one  semester, 
teachers  select  some  of  the  items  from  optional  category' depending  on  the  students’  majors.  For  example,  LaTeX 
and  Mathematica  are  chosen  for  students  majoring  in  Physics  and  Mathematics,  while  spreadsheets  and  SAS  for 
Economics. 

Class  reorganization  Method 

Reorganizing  by  a written  exam 

In  April  1997,  we  started  our  three  parallel  classes  with  a questionnaire  on  computer  experience.  The  class 
organization  was  based  not  on  their  computer  experience,  but  on  the  enrollment  order.  The  questionnaire  showed 
60%  of  the  students  had  experience  of  word  document  processing  using  computers  or  word  processors. 

After  we  completed  topics  in  requisite  category,  we  conducted  a written  examination  to  reorganize  classes 
based  on  students'  competencies.  The  exam  requested  students  to  write  computer  operations,  e.g.,  how  to  change 
fonts,  how  to  cite  the  body  of  the  mail  to  make  a reply,  why  the  login  password  is  important,  etc.  A total  of  150 


#1  Change  this  plain  text  file  to  rich  text  format,  then  change  the  font  of  the  marked  words  to  Ryumin 
1 2point,  and  place  it  in  the  center. 

#2  Explain  how  to  use  Japanese  Kana-Kanji  translation  and  how  to  register  your  name  in  your  own 
dictionary. 

#3  Look  up  three  words  (omitted  here)  in  the  following  dictionaries:  English-Japanese,  Japanese- 
English,  and  Webster. 

#4  Draw  a tiny  picture  and  insert  it  into  this  page. 

#5  Find  URLs  of  the  National  Diet  Library  and  two  online  book  stores. 

#6  Look  into  “/SharedLibrary/Literature/”  and  describe  whafs  in  it. 

#7  What  should  be  considered  when  deciding  your  password  string? 

#8  Find  Akutagawa's  books  using  Online  Public  Access  Catalog  (OPAC)  of  the  University  Library. 

#9  What  should  you  do  if  you  find  a nice  picture  of  Madonna  with  a notice  saying  “copy  free”  and 
you  would  like  to  have  its  copy  on  your  web  page? 

#10  Describe  how  to  limit  the  line  length  to  fewer  than  65  characters  in  the  body  of  an  e-mail. 

■■  

Table  1:  Mid-term  exam  sample  in  1998. 

- Touch  typing 

- Mouse  operation  (click,  double  click,  drag) 

- Window  operation  (resize,  move,  iconify,  hide) 

- Japanese  Kana-Kanji  translation 

- File/Folder  operation  (move,  rename,  create) 

- Word  document  processing  (fonts,  centering,  insert  graphics) 

- Mail  (MIME,  signature,  save  to  file,  reply,  delete,  re-file) 

- Web  (search  engine,  book-mark) 

- Use  of  online  dictionary  (Japanese,  English) 

- Canceling  your  printer  job 

^ : ;■ - 

Table  2:  A questionnaire  survey  for  self-assessment 
students  in  three  classes  were  reorganized  according  to  the  exam  score.  In  all  reorganized  classes,  spreadsheets  and 
HTML  were  selected  from  optional  category.  LaTeX  was  also  selected  in  the  two  high-scored  classes,  while  in  the 
low-scored  class  word  document  processing  was  selected  instead. 

At  the  end  of  the  semester,  we  again  gave  the  students  a written  examination  and  a questionnaire,  which  asked 
whether  the  students  were  satisfied.  The  scores  of  the  two  written  exams  showed  no  remarkable  difference  between 
classes;  however,  most  of  the  low-ability  class  students  were  satisfied  with  the  class  reorganization.  There  remained 
a problem  that  the  student  competencies  were  not  so  exactly  evaluated  by  our  written  examination. 

Reorganizing  by  a hands-on  performance  exam 

In  1998  and  1999,  we  conducted  a mid-term  exam  which  demands  hands-on  work  in  an  hour,  in  order  to 
reorganize  classes  based  on  students'  competencies  (Table.  1). 

We  included  two  questions,  which  had  not  been  taught  in  detail  in  all  classes,  to  see  whether  students  could 
solve  it  using  the  online  manual  (question  #3,  #8). 

We  marked  the  examination  papers  out  of  100  (each  question  of  10),  and  used  the  scores  io  reorganize  classes. 
Because  of  the  limit  of  PCs  available  in  computer  labs,  each  class  could  not  exceed  60  students.  So,  53  students  who 
got  more  than  67  points  were  grouped  to  A-class.  The  threshold  between  B-class  (50  students)  and  C-class  (50 
students)  was  52  points.  The  average  total  score  in  1999  was  60.3  (5=15.7). 

The  average  scores  of  the  new  classes  A,  B,  and  C were  77  (7.4),  60  (3.9),  42  (7.5)  points,  respectively.  The 
average  score  for  A-class  students  was  more  than  5.5  for  all  ten  questions,  whereas  the  students  who  were  grouped 
into  C-class  got  only  2.2  and  2.3  for  question  #5  (web  search)  and  #6  (file  search),  respectively,  which  were 
significantly  lower  than  the  other  two  classes.  This  tendency  was  very  similar  to  the  score  of  1 998. 
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Self-assessment  by  students 

After  the  mid-term  exam,  a questionnaire  survey  paper  was  distributed  to  the  students.  The  survey  consisted 
of  1 0 questions,  which  would  measure  students'  mastery  of  competencies  taught  by  that  time  (Table.2). 

The  answers  to  these  questions  were  collected  on  a 5-point  scale.  The  scores  of  the  collected  questionnaire 
showed  a positive  correlation  between  the  exam  and  the  students'  self-assessments.  That  is,  A-class  students  felt 
good  at  the  following  competencies:  inserting  graphics  into  a document,  operation  of  files/folders,  and  searching 
web  site,  whereas  C-class  students  answered  that  they  had  not  mastered  those  topics.  As  for  searching  files  from 
hierarchical  file  structure,  C-class  students  perceived  that  they  were  good  at  it;  however,  its  exam  score  is  poor. 
This  indicates  that  C-class  students  did  not  understand  the  file  structure  itself.  Touch-typing  and  Email  competencies 
were  not  statistically  significant  between  three  classes. 


What  and  how  should  we  teach  after  the  class  reorganization 

The  result  of  the  exam  and  a questionnaire  showed  us  what  we  should  teach  in  the  new  classes  after  the  class 
reorganization.  We  provided  C-class  students  with  a slower  work  pace  and  revisited  those  competencies  which  they 
were  not  good  at.  For  A-class  students,  self-teaching  of  LaTeX  was  given  as  an  assignment  during  summer  vacation. 

In  order  to  allow  them  to  have  an  active  participation  in  the  shaping  and  augmenting  of  their  learning,  we  also 
took  an  independent  study  approach  for  teaching  spreadsheets  in  all  classes.  That  is,  after  a brief  explanation  given 
by  a teacher,  students  follow  tutorials  on  the  teacher's  web  pages.  Since  graduate  teaching  assistants  (two  assistants 
per  classroom)  were  also  available  to  assist  the  students,  we  asked  them  to  give  students  only  hints  not  to  give 
detailed  step-by-step  operations. 


Statistical  evaluations  of  exam  and  questionnaire 


At  the  end  of  the  course,  we  gave  a final  examination  that  included  three  questions  similar  to  the  mid-term 


exam  and  required  hands-on 
work  of  about  an  hour.  After  the 
exam,  self-assessment 

questionnaire  survey  was 
conducted.  Here  we  begin  with 
the  analysis  of  the  questionnaire. 

Analysis  of  exams 

The  total  score  and  the 
scores  of  three  questions  (use  of 
online  dictionaries,  OPAC 
search,  and  limiting  line  length 
of  Mail  body),  which  are  similar 
in  two  exams,  were  considered 
in  a two-way  ANOVA.  As  a 
result,  these  four  items  showed 
an  "interaction"  between  the 
two  exams  and  classes  belonged 
to  (p  > 0.05). 

Figure  1 shows  a 
comparison  of  the  average 
scores  of  two  examinations. 
Note  that  Figure  1(a)  shows  the 
"ceiling  effect",  that  is,  the 
scores  in  A-class  are  distributed 
downward  after  the  class 
reorganization. 
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Figure  1:  Comparison  of  the  average  scores  oftwo  examinations 
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Figure  2:  Self-assessment  scores  in  1998  and  1999 


As  for  spreadsheets,  two  sessions  (which  means  180  minutes)  were  assigned  for  C-class  and  one  session  was 
assigned  for  A,  B classes  using  self-learning  web  based  text.  But  the  score  was  is  8 points  in  A-class  and  5 points  in 
C-class.  The  score  of  writing  HTML  also  supports  this  tendency.  From  this  fact,  it  is  proved  that  A-class  students 
had  higher  ability  in  computer  operations  than  other  two  class  students. 


Analysis  of  self-assessment  questionnaire 

As  for  the  27  items  that  were  included  both  mid-term  and  term  end  questionnaires,  the  average  term  end  exam 
scores  were  higher  than  mid-term  scores  in  all  classes  in  1998  and  in  1999.  In  Figure  2,  the  x-axis  shows  the  mid- 
term score,  the  y-axis  shows  term  end  score,  and  each  scatter  point  in  the  graph  represents  the  score  of  one  item. 
Two  years  analysis  shows  same  tendency. 

We  took  the  exam  timing  (mid-term  or  term  end)  and  classes  the  students  belonged  to  as  the  factors  of  the 
two-way  analysis  of  variance  (ANOVA).  The  following  four  items  of  the  questionnaire  shows  that  there  is  an 
"interaction"  between  exam  timing  and  classes  {p  > 0.05). 

- Creating  a new  folder  (directory) 

- Editing  a word  document 

- Browsing  web  pages 

- Searching  web  site 

Figure  3 compares  the  average  scores  of  these  four  items  obtained  through  the  questionnaire  survey.  From  this 
figure,  we  conclude  that  C-class  students  perceived  they  were  as  self-confident  in  these  items  as  the  students  in  the 
other  two  classes. 

Though  the  exam  scores  are  not  significantly  correlated  with  students*  self-assessments,  the  class 
reorganization  was  effective  for  slow  learners  class  students. 
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Figure  3:  Comparison  of  the  average  scores  of  these  four  items  obtained  through  the  questionnaire  survey. 


Conclusions 

This  paper  described  the  efficiency  of  ability  grouping  strategy.  Class  reorganization  in  the  middle  of  a term 
has  enhanced  learning  outcomes  in  computer  literacy  classes.  It  is  also  good  tool  for  teachers  to  know  the  way  to 
lead  students  to  higher  skill  level. 

We  are  planning  to  refine  the  examinations  and  questionnaire  items  so  that  we  can  more  accurately  measure 
the  students'  competencies  and  introduce  RUBICS  for  assessment.  In  2000,  we  will  incorporate  group  work, 
discussion  and  presentation  for  high  ability  students  after  the  class  reorganization. 
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Abstract:  A course  delivery  model  is  a framework  promoting  the  scalable  application  of  effective 
pedagogies  and  educational  technologies.  This  paper  describes  how  course  delivery  models  were 
designed  and  implemented  for  the  inaugural  first  year  program  at  the  Technical  University  of 
British  Columbia  (TechBC).  In  Fall  1999,  ninety  first  year  university  students  took  6 courses 
delivered  through  a central  Course  Management  System  (CMS)  providing  online  conferencing, 
announcement  posting,  online  quizzes,  support  for  team-based  learning,  and  course  content  in  the 
form  of  web  pages,  streaming  video,  and  animation.  The  courses,  representing  5 distinct  delivery 
models,  differed  substantially  in  their  application  of  these  features  aaid  in  the  use  of  face-to-face 
class  time.  A comparative  analysis  of  questionnaire  and  web  log  data  focusing  on  learner 
engagement  and  achievement  with  the  delivery  models  will  be  presented  at  ED-MEDI A 2000. 


In  a widely  publicized  comparison  of  online  and  face-to-face  teaching,  Schutte  (1997)  attributed  the  higher 
achievement  of  the  online  group  to  “higher  perceived  peer  contact  and  time  spent  on  class  work”.  Schutte’s 
conclusion  is  compelling  because,  to  no  one’s  surprise,  time  spent  learning  has  long  been  confirmed  (e.g.,  Carroll, 
1963)  as  perhaps  the  most  powerful  determinant  of  learning  achievement  and,  more  interestingly,  decades  of 
research  on  cooperative  learning  (Slavin,  1 995)  have  established  structured,  team-based  learning  as  among  the  most 
effective  educational  methods. 

In  addition,  research  on  metacognition  and  self-regulated  learning  (Butler  & Winne,  1 995)  is  revealing  the  critical 
role  of  these  aspects  of  engagement  in  learner  achievement.  Thus,  for  the  purposes  of  this  paper,  learner  engagement 
is  regarded  as  a combination  of  three  inter-related  factors:  Time  spent  working  on,  metacognitive  processing  of,  and 
social  interaction  in  relation  to,  learning  resources  and  tasks. 

This  paper  describes  the  design  and  implementation  of  5 course  delivery  models  constructed  from  different 
combinations  of  educational  technologies  and  teaching  methods.  In  so  doing  it  presents  the  groundwork  for  a study, 
to  be  presented  at  ED-MEDIA  2000,  of  how  learner  engagement  varies  across  delivery  models. 

Institutional  Goals  for  Educational  Technology  and  Learning 

The  Technical  University  of  British  Columbia,  Canada’s  newest  public  university,  welcomed  its  initial  cohort  of 
first-year  students  in  August,  1999  with  a curricular  emphasis  on  Information  Technology,  Management  & 
Technology,  and  Interactive  Arts.  The  development  of  a strategy  for  the  use  of  innovative  pedagogy  and  educational 
technology  began  before,  and  extended  throughout,  the  planning  and  development  of  the  first  year  curriculum. 

In  the  early  institutional  planning  for  educational  technology  and  learning,  four  standards  emerged  with  regard  to  the 
first  year  program: 

• Most  courses  would  emphasize  collaborative  learning  with  classes  organized  into  teams. 

• Averaged  across  the  program,  about  half  of  learning  contact  time  would  occur  online. 

• Most  courses  would  integrate  online  and  face-to-face  learning. 

• All  courses  would  include  a substantial  web-based  component. 

It  was  apparent  that  achieving  these  standards  for  the  first  set  of  courses  and  the  small  initial  cohort  of  students 
would  be  easier  than  maintaining  them  as  student  enrollment  and  course  offerings  grew  rapidly.  To  enable 
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scalability  it  was  decided  to  design  a relatively  small  set  of  delivery  models  consisting  of  appropriate  combinations 
of  learning  methods  and  technologies.  Each  course  is  assigned  to  a delivery  model  so  that  technical  and  pedagogical 
support  and  planning  can  be  organized  around  the  delivery  models  rather  than  the  unique  requirements  of  each 
course.  Having  each  of  the  delivery  models  represented  in  the  initial  set  of  course  offerings  also  ensured  that  a 
variety  of  approaches  would  be  explored  and  compared  at  the  outset. 


The  TechBC  Course  Management  System 


Crucial  to  the  successful  implementation  of  course  delivery  models  is  the  support  for  learner  engagement  provided 
by  the  course  management  system  (CMS).  After  reviewing  several  commercially  available  systems  it  was  decided 
to  build  a web-based  CMS  with  core  functionality  written  in  XML/RPC,  Dynamic  HTML,  and  JavaScript. 
Specialized  functions  were  provided  by  commercial  products.  For  example,  we  used  SoftArc’s  FirstClass  for  online 
conferencing  and  QuestionMark’s  Perception  for  online  tests  and  quizzes. 
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Figure  1.  The  iSpace  page  provides  a calendar,  announcements,  and  links  to  courses. 


To  promote  learner  engagement,  features  of  the  TechBC  CMS  were  designed  to  provide  convenient  access  to  course 

content  and  interaction  with  instructors  and  other  learners: 

• An  entry  page,  was  designed  to  support  community  building  by  providing  general  announcements,  Java-based 
text  chat  serving  as  a virtual  cafe  or  hallway,  voting  on  community  issues,  and  links  to  online  conferences 
dealing  with  a variety  of  student  concerns. 

• A personal  page  (Figure  1)  provides  links  to  courses,  email,  a calendar,  announcements,  and  personal  file  space 
on  a university  server. 

• Each  course  has  a syllabus  page  (Figure  2)  providing  links  to  learning  objectives,  presentations,  online 
conferences,  and  descriptions  of  assignments  and  exams. 

• Most  courses  have  a teams  page  listing  all  class  members  by  team.  The  teams  page  also  provides  access  to  team 
conferences. 

• A navigation  bar  shown  at  the  bottom  of  the  Figure  1 is  available  throughout  the  CMS. 


The  course  developers,  faculty  and  teaching  staff,  were  provided  with  preformatted  web  page  templates  specific  to 
the  delivery  model  for  which  they  were  working.  Most  developers  were  subject  matter  experts  with  little  or  no 
experience  developing  for  the  web.  They  entered  text  and  graphic  content  into  server  files  using  Dreamweaver  and 
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received  support  from  educational  technology  staff  for  the  creation  and  licensing  of  interactive  and  multimedia 
elements. 
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Figure  2,  Example  of  a syllabus  page  in  the  TechBC  course  management  system. 

Many  of  the  web  page  templates  included  interactive  components.  For  example,  a template  in  one  delivery  model 
included  links  to  pre- study  and  post-study  quizzes.  A template  in  another  model  included  a voting  poll  in  which  each 
student  had  one  anonymous  vote. 

Some  design  goals  intended  to  promote  learner  engagement  were  postponed  to  later  versions  of  the  CMS: 

• Team  management  tools  enabling  instructors  to  easily  assign  learners  to  teams  and  evaluate  team  projects. 

• Team  participation  tools  assisting  team  members  in  collectively  managing  project  work. 

• Interactive  features  of  the  CMS  interface  further  supporting  self-monitoring  and  self-regulation  of  individual 
learning,  such  as  meters  or  check  lists  showing  progress  through  the  objectives  of  the  course. 

Network  Access  Requirements  and  the  Expansion  of  Broadband  Service 

Learners  accessed  the  course  management  system  from  on-campus  labs  and  from  home.  TechBC  required  that  they 
have  access  to  the  internet  through  3"*  party  ISPs  at  a minimum  56  kbps.  In  fact,  learners’  home  subscriptions  and 
modem  speeds  ranged  from  28.8kbps  dial-up  to  cable  modem  and  ADSL. 

TechBC  is  located  in  the  lower  mainland  of  British  Columbia,  a region  in  which  both  ADSL  and  cable  subscriptions 
are  available  for  under  $30  USD  per  month.  Throughout  the  Fall  of  1999  local  cable  companies  were  aggressively 
expanding  internet  subscriptions  at  a rate  of  about  300%  per  year. 

As  shown  in  Figure  3,  There  was  a substantial  increase  in  learners’  subscriptions  to  broadband  internet  services 
(ADSL  and  cable)  in  the  early  months  of  the  term.  By  the  3'^'*  week  of  classes  in  September,  46%  of  ISP  accounts 
accessing  the  CMS  were  through  broadband  services.  This  figure  had  risen  to  66%  by  mid-November. 
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Figure  3.  Percent  of  learners’  ADSL,  Cable,  and  other  ISP  accounts  accessing  the  course  management  system 
(CMS)  in  September  and  November,  1999 


Despite  developers  attempts  to  optimize  streaming  video  for  56  kbps,  anecdotal  reports  from  TechBC  learners 
indicated  that  broadband  access  may  be  necessary  to  enable  effective  home  use  of  streaming  video  presentations 
provided  through  the  CMS.  Some  learners  with  dial-in  connections  report  having  to  come  to  on -campus  labs  to 
effectively  use  streaming  video. 

When  broadband  access  can  be  mandated  for  all  learners,  likely  within  a year  or  two,  new  deliveiy  models  using 
real-time,  one-way  video  presentations  with  interactive  components  will  become  increasingly  feasible. 


The  Course  Delivery  Models 

A course  delivery  model  is  a combination  of  logistical,  technological,  and  pedagogical  features.  Each  model  has 
implications  for  the  scheduling  of  classrooms  and  labs,  division  of  roles  among  teaching  staff,  course  development 
time,  demographic  profile  of  learners,  learners’  hardware  and  network  requirements,  fit  to  curricula,  and  so  on.  Our 
design  of  logistical  and  technological  components  of  the  delivery  models  was  influenced  by,  among  others.  Bates 
(1995)  who  discussed  access,  cost,  teaching  and  learning,  interactivity  and  user-friendliness,  and  organizational 
issues  as  key  criteria  for  making  decisions  about  educational  technology  and  media. 

The  pedagogical  components  of  the  delivery  models  were  influenced  by  theory,  research  and  practice  in 
asynchronous  learning  networks  (e.g.,  Harasim,  1995),  cooperative  learning  (e.g.,  Millis  & Cotell,  1998)  and 
instructional  design  (e.g..  Diamond,  1998).  The  models  were  designed  to  shift  one-way  delivery  of  information  out 
of  lectures  toward  print  or  web  media,  or  better  yet,  to  replace  one-way  delivery  with  interactive  approaches. 
Among  the  methods  adopted  were  the  use  of  explicit  learning  objectives,  team-based  learning  in  both  face-to-face 
(F2F)  and  online  settings,  moderated  discussions,  primary  trait  analysis  for  assessment,  and  interactive  simulations. 

Curricular  needs  also  influenced  the  design  of  the  models.  For  example,  a program  focus  on  Interactive  Arts  led  to 
the  development  of  a Studio  Lab  model  building  on  art  school  traditions.  In  fact,  each  of  the  models  can  be  seen  as  a 
re-working  of  one  or  more  traditional  teaching  models. 

The  following  sections  summarize  the  five  delivery  models  used  by  TechBC ’s  first  year  courses. 

Presentational  Cooperative  (PC)  Model 


Section  size: 

F2F  cooperative  learning: 


48  learners 
1 .5  hours  per  week 


Study  of  web  presentation: 


~ 1.5  hours  per  week 


Derived  from  the  lecture-seminar  method,  this  model  replaces  the  lecture  with  a web  presentation  containing 
multimedia  and  interactive  elements.  During  weekly  face-to-face  sessions  students  work  in  teams  of  4 to  6 on 
structured  cooperative  learning  tasks.  An  asynchronous  online  conference  is  provided  for  students  to  ask  questions. 
Some  courses  may  add  open  lab  sessions  where  students  come  to  receive  help  and  work  on  assignments. 

Studio  Lab  (SL)  Model 

Section  size:  24  learners 

F2F  studio  session:  3 hours  per  week 

Study  of  web  presentation:  ~ 1.5  hours  per  week 

This  model  bears  some  similarity  to  the  PC  model,  but  differs  in  the  duration  of  required  on-campus  sessions,  and  in 
the  learning  activities  carried  out  in  those  sessions.  Studio  sessions  are  held  in  a computer  lab,  and  the  learning 
activities  tend  to  be  project  based.  Although  learners  are  pre-assigned  to  teams  of  6,  they  are  just  as  likely  to  be 
working  individually  as  working  within  their  teams.  This  model  provides  a weekly  web  presentation  containing 
multimedia  and  interactive  elements,  and  an  asynchronous  online  conference  for  students  to  ask  questions.  Some 
courses  may  add  open  lab  sessions  where  students  can  come  to  receive  help  and  work  on  projects. 

Computer-Mediated  Classroom  (CMC)  Model 

Section  size:  24  learners 

On-campus  activities:  infrequent  if  any 

Online  conferencing:  ~ 3.0  hours  per  week 

The  CMC  model  is  built  on  the  tradition  of  asynchronous  learning  networks  (Harasim  et  al.,  1995)  and  makes  heavy 
use  of  online  conferencing.  Every  week,  a section  of  24  learners  takes  up  a new  topic  in  an  online  conference.  Some 
web-based  interactive  and  multimedia  resources  may  be  provided,  but  these  are  not  expected  to  be  extensive.  The 
CMC  model  imposes  a relatively  low  development  cost,  but  the  need  to  moderate  and  manage  conferences  results  in 
high  delivery  costs.  Students  are  pre-assigned  to  teams  of  six,  with  each  team  carrying  out  some  of  its  work  within 
its  own  conference. 

Flexible  Study  (FS)  Model 

Section  size:  unlimited 

On-campus  activities:  optional  open  labs 

Study  of  web  presentation:  ~ 3.0  hours  per  week 

Derived  from  the  distance  learning  tradition,  this  is  a self-paced  model  with  no  requirement  to  attend  face-to-face 
sessions.  The  flexible  study  model  demands  the  production  or  acquisition  of  high  quality  web-based  and  multimedia 
resources.  Compared  to  other  models  described  here,  it  imposes  the  greatest  development  costs  but  the  least  delivery 
costs.  There  are  frequent  self-diagnostic  quizzes  and  less  frequent  formal  examinations.  An  asynchronous  online 
conference  is  provided  for  students  to  ask  questions.  To  compensate  for  the  absence  of  scheduled  on-campus 
sessions,  web-based  conferences  and  contact  support  are  provided  to  support  the  formation  of  informal  study  groups. 
Some  courses  may  add  open  lab  sessions  where  students  visit  campus  to  receive  help  and  work  on  assignments. 

Threaded  Process  (TP)  Model 

Section  size:  unlimited 

F2F  integrated  activity:  1 .5  hours  per  week 

Study  of  web  presentation:  ~ 1.5  hours  per  week 

This  model  arose  from  the  curricular  need  to  integrate  the  learning  of  communication,  writing,  team  process, 
research  and  learning  skills  with  other  courses  in  the  first  year  program.  Thus  the  learning  activities  in  this  model 
tend  to  piggyback  on  the  learning  activities  of  other  courses.  For  example,  for  a unit  on  concept  mapping  learners 
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studied  a web  presentation  on  concept  maps.  Then  later,  in  a PC  class  covering  operating  systems  in  a different 
course,  they  were  asked  to  create  a concept  map  to  illustrate  structures  in  a operating  system.  The  concept  map  they 
created  could  be  used  in  a portfolio  to  assess  their  learning  of  concept  mapping. 

Implementation  of  Delivery  Models 

Table  1 shows  how  courses  were  assigned  to  delivery  models.  Online  components  of  the  first  term  courses  included 
a total  of  about  5000  web  pages  and  90  interactive  or  multimedia  objects. 

A few  outcomes  were  apparent  even  during  the  early  weeks  of  the  term.  First,  delivery  models  placing  more 
emphasis  on  online  learning  did  not  necessarily  provoke  greater  activity  in  the  CMS.  Table  1 shows  the  mean 
number  of  hits  to  the  course  syllabus  pages  per  learner  during  a fairly  typical  week  in  early  December.  In  that  week 
the  delivery  models  with  the  greatest  emphasis  on  f2f  contact  actually  garnered  the  most  syllabus  hits.  Second,  the 
demanding  schedule  imposed  by  the  6 first-year  courses  inclined  learners  to  assign  low  priority  to  the  self-paced 
flexible  study  course.  The  model  was  subsequently  adapted  by  introducing  assignments  and  exams  on  a fixed 
schedule.  Third,  pressure  to  converge  the  delivery  models  emerged  as  instructors  and  learners  requested  features 
from  other  models  and  declined  use  of  some  features  specific  to  the  model  assigned  to  their  course. 


Table  1.  Courses  and  delivery  models  used  in  first  term  courses. 


Delivery  Model 

Course 

Mean  syllabus  hits  per 
learner  in  one  week 

Threaded  Process  (TP) 

Tech  100  Process  Elements 

3.2 

Computer-Mediated  Classroom  (CMC) 

Techl20  Business  in  a Global  Economy  I 

3.1 

Studio  Lab  (SL) 

Tech  130  Systems  of  Visual  Representation 

5.7 

Presentational  Cooperative  (PC) 

Tech  140  Linear  Systems 

5.5 

Tech  150  Introduction  to  Computer  Systems 

7.8 

Flexible  Study  (FS) 

Tech  142  Probability  and  Statistics 

3.4 
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Abstract:  Southern  Utah  University  has  over  five  years  of  successful  programming  on  the  state’s 
closed  circuit  two-way  video  educational  network.  We  have  recently  capitalized  on  this 
experience  and  expertise  to  create  a unique  web-based  humanities  course  supported  by  video-on- 
demand  lectures  using  RealNetworks®  media  streaming  technology  and  SMIL  (Synchronized 
Multimedia  Integration  Language). 


Introduction 

Streaming  video  technology  is  transforming  multimedia  content  on  the  Internet  and  adding  powerful  new 
possibilities  to  on-line  education.  More  powerful  compression  algorithms  and  broadening  bandwidth  capabilities 
make  video  delivery  over  the  web  more  and  more  feasible  for  serious  distance  educators  who  want  to  maximize 
content  delivery  options. 

Just  as  video-on-demand  programs  using  VHS  tapes  added  the  power  of  video  and  moving  images  to 
“traditional”  distance  education  programs  now  video  streaming  technologies  make  it  possible  to  bring  the  power  of 
video  to  web  based  courses  as  well.  Mention  of  video  as  a possible  ingredient  in  new  on-line  educational  offerings 
began  appearing  in  the  literature  as  early  as  July  1999  (Thornton,  1999).  Though  relatively  rare  there  are  a few 
institutions  using  streaming  video  to  support  web  based  instruction.  The  majority  of  these  programs  are  using  in 
house  products  developed  by  their  institution  to  create  Internet  courses  that  include  video  and  “electronic 
presentations”  (Brusilovsky,  2000).  At  Southern  Utah  University  (SUU)  we  have  created  a general  education 
humanities  course  for  delivery  over  the  Internet  that  includes  video-on-demand  capability  providing  students  with 
over  150  mini  lectures  in  support  of  the  content  material.  In  seeking  to  create  a new,  exciting  Internet  offering  we 
chose  a general  education  humanities  course  as  an  ideal  vehicle  for  testing  the  efficacy  of  multimedia  streaming 
technology  in  Internet  based  university  instruction.  In  short,  we  wanted  to  make  our  Humanities  1010  Internet 
course  as  human  as  possible. 


The  Course 

The  Humanities  course  selected  is  part  of  SUU’s  growing  distance  education  program  and  is  taught  yearly 
over  live  interactive  TV  as  part  of  our  concurrent  enrollment  program.  Students  across  the  state  can  take  the  course 
live  via  Utah’s  closed  circuit  educational  network,  EDNET.  The  course  was  ideal  for  this  project  in  that  it  is  taught 
in  a studio  setting  where  a high  quality  recordLng  of  the  lecture  was  possible.  In  addition,  the  instructor  has  designed 
the  course  to  be  delivered  over  TV  so  many  of  the  instructional  aids  and  projects  have  been  optimized  to  the 
camera/TV  environment  adding  quality  to  the  video  recording  as  well. 

Lectures  for  the  entire  course  were  recorded  digitally  during  Fall  1998  in  preparation  for  developing  the 
Internet  course.  Editing  the  recorded  lectures  into  5-10  minute  mini  lectures  and  securing  still  images  to  illustrate 
them  all  was  a very  time  consuming  process.  The  faculty  instructor  worked  with  a team  of  three  student  assistants  to 
create  the  illustrated  mini  lectures.  Development  of  the  Internet  course  was  funded  in  part  with  grants  from  the  Utah 
Education  Network  (UEN)  and  the  Higher  Education  Technology  Initiative  (HETI). 


S38 


The  Technology 


The  innovative  structure  and  delivery  of  the  course  is  made  possible  by  a powerful  WWW  multimedia  tool, 
SMIL,  (Synchronized  Multimedia  Integration  Language)  pronounced,  ’’smile.”  Recommended  by  the  World  Wide 
Web  Consortium  (W3C)  it  allows  for  the  creation  of  time  based  multimedia  delivery  over  the  web  (World  Wide 
Web  Consortium,  1 998).  Based  on  XML,  it  allows  developers  to  mix  many  types  of  media  together  including,  text, 
video,  graphics,  audio  and  vector  based  animation,  and  to  synchronize  them  to  a timeline.  Specifically,  SMIL 
provides  Web  users  with: 

• Easily  defined  basic  timing  relationships 

• Fine-tuned  synchronization 

• Spatial  layout 

• Direct  inclusion  of  non-text  and  non-image  media 

• Hyperlink  support  for  time-based  media 

• Adaptability  to  varying  user  and  system  characteristics. 


Our  Humanities  course  for  the  Internet  uses  these  capabilities  to  deliver  video  and  still  images  to  create  an  illustrated 
video  lecture  format  unique  to  on-line  instruction. 

The  simultaneous  streaming  of  a variety  of  media  is  the  key  feature  of  SMIL  that  we  use  to  create  the 
“illustrated  lecture”  format  of  the  class.  We  use  RealNetworks®  technology  for  our  video  encoding,  streaming  and 
player  system.  When  we  first  envisioned  video-on-demand  support  for  a course  delivered  over  the  WWW 
RealNetworks®  was  the  industry  standard  for  streaming  media  on  the  Internet.  Since  beginning  our  program  Apple, 
Microsoft  and  others  have  introduced  streaming  systems  as  well  but  we  are  pleased  with  our  choice  to  use 
RealNetworks®  and  with  the  improvements  they  continue  to  make  in  their  product  and  support.  In  addition, 
RealNetworks®  was  the  first  of  the  major  streaming  technologies  to  make  their  player  SMIL  compatible. 

SMIL  is  a programming  language  similar  in  structure  to  html.  Layout  tags  allow  the  player  window  to  be 
customized,  including  the  creation  of  multiple  player  regions,  each  capable  of  displaying  a different  media  stream. 
Our  mini  lectures  use  a simple  rectangular  window  divided  into  three  separate  regions:  a video  region,  a menu 
region  and  a slide  region  (Fig.  1). 


Figure  1:  General  layout  of  player  window 

The  video  and  menu  regions  take  up  the  left  side  of  the  window  with  the  video  region  immediately  above  the  menu 
region.  The  video  region  displays  the  instructor  in  a tight  shot,  as  well  as  any  visuals  he  used  during  the  TV 
broadcast  of  the  class.  These  visual  supplements  include  such  things  as  computer  images,  overheads  and  use  of  the 
white  board.  The  menu  region  lies  just  below  the  video  window  and  is  a simple  point  and  click  menu  for  navigating 
through  the  current  lesson. 
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The  slide  region  is  the  largest  window  and  lies  to  the  right  of  the  other  two.  The  slide  region  displays  still 
images  that  illustrate  what  the  instructor  is  discussing.  Standard  video  controls  (play,  stop,  pause,  fast  forward, 
rewind)  above  the  video  window  allow  students  to  control  the  flow  of  the  presentation  (Fig.  2). 


Rivotal  Point  of  History 
Solon 
Buddah 
Zoroaster 
Isaiah/Jeremiah 
Confucius 


Figure  2:  Screen  capture  of  player  window  in  operation 

Another  important  SMIL  feature  beyond  its  layout  capability  is  SMIL’s  ability  to  create  simultaneous 
streams  of  various  media.  This  is  the  heart  of  the  “narrated  slide  show”  format  of  our  class.  SMIL  allows  the 
designer  to  synchronize  various  media  streams  to  produce  a multimedia  product.  Our  design  uses  SMIL  to  “tie”  a 
series  of  still  images  to  the  time  line  of  the  video  so  that  they  stream  simultaneously  with  each  graphic  appearing  in 
the  image  window  “on  cue”  as  the  lecture  progresses. 

We  initially  demonstrated  the  video-on-demand  courseware  to  a small  number  of  students  and  were 
encouraged  by  their  response.  They  found  the  video  to  be  engaging  and  personal  and  they  react  very  positively  to 
the  instructor’s  excitement  and  enthusiasm  for  the  subject  matter.  In  fact,  students  tended  to  rely  too  much  on  the 
video  which  is  meant  to  supplement  and  interact  with  the  course  reading  assignments,  not  replace  them. 

One  draw  back  of  Internet-based  multimedia  is  the  fairly  high  bandwidth  required  for  quality  streaming. 
Our  suggested  minimum  requirement  for  online  students  is  a pentium  class  ( or  comparable  Macintosh)  computer, 
with  64K  RAM,  300MhZ  processor  and  a true  56K  Internet  connection  (or  faster.)  The  high  end  technology  does 
limit  access  to  the  course  but  we  are  confident  that  streaming  video  will  be  a major  element  in  the  future  of  web- 
based  instruction  and  we  feel  it  is  necessary  to  begin  working  in  this  arena  now.  The  necessary  bandwidth  and 
processor  speeds  for  streaming  video  are  already  in  place  at  most  colleges  and  universities  and  at  many  public 
schools  as  well.  With  cable  modems  and  other  emerging  technologies  increasing  the  receiving  capabilities  of  home 
users  our  early  foray  into  this  field  will  prepare  us  to  better  serve  the  needs  of  students  in  the  future  . 


A Beginning 

The  course  was  officially  opened  winter  2000  with  ten  students.  We  are  using  this  first  course  as  a test  of 
the  program  and  pedagogy  we  have  employed  in  creating  it.  All  students  will  be  expected  to  fill  out  a survey  as  part 
of  the  exit  requirements  for  receiving  credit  for  the  course.  The  survey  instrument  will  be  distributed  from  a 
continuing  education  web  site,  unrelated  to  the  course  site,  in  hopes  to  encourage  students  to  be  frank  and  open  in 
their  response.  The  survey  will  focus  on: 

General 

• Student  demographics 

• Satisfaction  with  course 

• Satisfaction  with  the  technology 

• Satisfaction  with  student  support 
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Specific 

• Satisfaction  and  perceived  value  of  the  video  lectures 

• Satisfaction  and  perceived  value  of  the  streaming  technology 

• Satisfaction  and  perceived  value  of  the  on-line  assignments  and  quizzes 

• Satisfaction  and  perceived  value  of  the  student-teacher  interaction 

Conclusion 

Overall,  we  are  excited  about  what  streaming  video  can  bring  to  Internet  instruction  and  look  forward  to 
implementing  our  first  course  using  this  new  technology.  We  hope  to  have  enrolled  sufficient  numbers  by  the  fall 
of  2000  to  be  able  to  publish  significant  findings  from  our  research  into  the  operation,  teaching  effectiveness  and 
student  satisfaction  related  to  streaming  video-on-demand  in  support  of  Internet  based  college  courses. 
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Abstract:  CoCoA  (Communicative  Correction  Assistant  System)  was  developed  for  supporting  students 
and  teachers  to  exchange  online  marked-up  documents  via  email.  Its  environment  is  very  similar  to  a real 
one  in  which  people  use  paper  and  pen.  In  order  to  effectively  reuse  marked-up  digital  documents  created 
with  CoCoA,  this  paper  proposes  a computer  supported  proofreading  exercise  environment  called  CoCoA2. 
CoCoA2  can  reduce  the  overload  of  the  teacher,  and  it  can  make  composition  learning  more  effective. 
CoCoA2  has  been  developed  and  used  it  experimentally. 


Introduction 

Recently,  researchers  in  educational  systems  attempt  to  provide  technological  support  for  cooperative  and 
collaborative  learning  advocated  by  educational  theories  (O’Malley,  1994).  The  explosive  growth  of 
networking,  in  particular,  raises  the  possibility  of  widespread  collaborative  and  open-ended  learning  activities 
(Harasim,  et  al.,  1995).  We  have  investigated  technological  support  for  open-ended  and  collaborative  learning 
activities  (Ogata,  et  al.,  1996;  1997).  In  composition  studies,  coupled  with  the  shift  in  writing  pedagogy  toward 
an  open-ended  and  learner-centered  approach,  the  theoretical  view  of  learning  has  created  a surge  of  interest  in 
collaborative  learning  in  such  way  as  to  prize  conversation  and  interaction  among  students  and  among  students 
and  teachers. 

In  computer-based  classrooms,  writing  teachers  and  students  use  various  software  tools  based  on  some  theories 
of  writing.  Writing  compositions  includes  various  sub-processes  such  as  planning,  transcribing,  and  revising, 
which  do  not  need  to  occur  in  any  fixed  order  (Scardamalia  & Bereiter,  1986).  Hence,  it  is  very  important  for 
students  to  receive  the  instruction  through  the  review.  There  are  some  asynchronous  editing  systems  that 
support  teachers  to  review  and  correct  the  students’  drafts  with  online  mark-up.  These  systems  adapt  the 
traditional  mark-up  model  to  the  computer  screen.  The  symbols  are  both  familiar  and  intuitive  for  editors  and 
authors;  for  example,  deletion,  insertion,  and  move  (Farkas  & Poltrock,1995).  For  instance,  MATE  (Hardock, 
Kurtecbach,  & Buxton,  1993)  allows  the  editors  to  use  both  online  markup  and  comments.  In  this  model, 
authors  and  editors  can  readily  interpret  the  editor’s  markings.  Although  there  are  many  systems  that  employ 
markup  that  allows  multiple  users  to  correct  an  electronic  document  as  if  they  were  marking  up  a printed  copy 
of  the  document,  the  systems  do  not  provide  a generalized  format  for  exchanging  the  documents  using  Internet. 

CoCoA  (Communicative  Correction  Assistant  system)  has  been  developed  for  supporting  foreigners  and 
teachers  to  exchange  marked-up  documents  by  e-mail  (Ogata  et  al.,  1997).  Its  environment  is  very  similar  to  a 
real  one  in  which  people  use  paper  and  pen.  CoCoA  allows  teachers  not  only  to  correct  the  compositions  sent 
from  foreigners  by  E-mail,  but  also  foreigners  to  see  where  and  why  the  teacher  had  corrected  them.  CoCoA 
improves  the  opportunities  that  foreigners  have  for  writing  Japanese  compositions  and  for  receiving 
instructions  from  teachers.  CCML  (Communicative  Correction  Mark-up  Language)  (Ogata  et  al.,  1998)  has 
also  been  proposed  for  the  representation  of  marked-up  documents,  which  is  based  on  SGML  (Standard 
Generalized  Mark-up  Language)  (Henvijnen,  1990).  With  CCML,  teachers  and  students  can  exchange 
marked-up  documents  via  e-mail  (Ogata  et  al.,  1999).  In  the  experimental  use  of  CoCoA,  most  of  users 
commented  that  CoCoA  was  easy  for  them  to  understand  the  mistakes  in  documents  because  of  the  use  of 
marks,  and  that  the  optional  view  of  the  original,  marked  or  revised  text  was  very  useful. 

This  paper  proposes  an  online  markup  based  collaborative  learning  environment  called  CoCoA2,  for  Japanese 
language  composition  studies.  In  order  to  effectively  re-use  marked-up  digital  documents  created  with  CoCoA, 
CoCoA2  reuses  the  documents  in  proofreading  exercise  for  other  learners.  In  CoCoA2,  a learner  sends  a 
teacher  an  original  text.  The  teacher  then  corrects  the  text  with  CoCoA2,  and  returns  the  learner  the  marked- 
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up  text.  The  marked-up  texts  of  all  the  learners  are  stored  into  a repository  of  the  teacher.  From  the  students  in 
a networked  writing  classroom,  CoCoA2  chooses  an  appropriate  learner  for  proofreading  exercise  by  analyzing 
writing  errors  in  the  CCML  formatted  documents  repository.  CoCoA2  presents  the  selected  learner  with  an 
original  text  in  which  there  are  a number  of  errors  that  the  learner  has  to  find  and  correct.  This  exercise 
elevates  error-correcting  skills  and  self-monitoring  skills  in  writing  a composition  (Coniam,  1997).  After  the 
learner  finishes  proofreading,  CoCoA2  evaluates  the  result  through  comparing  the  correction  of  the  learner 
with  that  of  the  teacher.  CoCoA2  creates  the  opportunity  of  collaborative  learning  by  connecting  the 
proofreading  learner  and  the  learner  who  wrote  the  original  text.  The  document  can  be  improved  through  the 
collaboration.  Because  of  the  reuse  of  online  marked-up  digital  documents,  CoCoA2  can  reduce  the  teacher’s 
load,  and  it  can  make  composition  learning  more  effective. 


Overview  of  CoCoA  and  CCML 

CoCoA-Editor  & CoCoA-Viewer 


CoCoA  system  consists  of  CoCoA-Editor  for  a teacher  and  CoCoA-Viewer  for  a student.  Figure  1 shows  the 
interface  of  CoCoA.  By  selecting  a mark  from  the  mark  palette  shown  in  the  window  (A),  a teacher  can  correct 
learners’  documents.  Moreover,  the  teacher  can  annotate  in  the  document  using  the  annotation  palette,  and 
he/she  classifies  the  marks  using  a window  which  looks  like  (B).  The  “undo”  button  erases  corrections.  In 
figure  1,  the  teacher  changes  the  words  which  means  “attend”,  and  determines  the  importance  level  of  the 
correction  is  three.  A learner  obtains  a marked  text  in  a window  (C).  In  this  case,  CoCoA-Viewer  provides 
only  the  marks  over  the  level  two  of  importance.  “?”  denotes  a question  and  shows  a explanation  from  the 
teacher.  The  system  displays  the  contents  of  the  annotation  after  clicking  the  mark.  As  shown  in  the  message 
window,  the  learner  can  reply  to  the  message  and  send  it  after  revising  the  document. 
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Question:  What  do  you  want  to  write? 

Rqjly:  1 wanted  to  write  the  word  which  means 
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Figure  1:  Screen  snapshots  of  CoCoA-Viewer  and  CoCoA-Editor. 


CCML 

Japanese  Industrial  Standard  (JIS)  defines  20  marks  and  18  sub-marks  as  the  marks  for  proofreading 
(Shimano,  1986).  A mark  shows  the  spot  of  correction,  e.g.,  text  insertion.  All  the  marks  are  not  required  in 
our  situation  because  documents  are  inputted  with  a computer.  Based  on  this  standardization,  seven  marks 
were  selected  through  our  experiments.  The  seven  marks  for  correction  were  selected;  insert,  replace,  delete, 
separate,  join,  move  and  annotation.  Based  on  SGML,  CCML  for  exchanging  marked-up  documents  has  been 
proposed.  SGML  is  an  ISO  standard  (ISO  8879:1986)  which  supplies  a formal  notation  for  the  definition  of 
generalized  mark-up  languages.  SGML  is  device-independent  methods  of  representing  texts  in  electronic  form. 
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CCML  document  consists  of  three  parts:  header,  body  and  close.  A header  part  has  an  editor  name,  an  author 
name,  and  the  revision  time.  A close  part  shows  the  editor’s  comments.  A body  part  consists  of  the  six  marks 
and  the  annotations.  Every  tag  has  an  attribute  “level”  that  a teacher  gives  a number  from  one  to  three;  1 :weak 
correction,  2:normal  correction,  and  3:strong  correction.  It  is  very  important  for  a teacher  to  annotate  the 
marked  text  for  instruction  in  composition.  For  example,  PREP  Editor  (Neuwirth  et  al,  1996)  is  a word 
processor  that  allows  writers  and  reviewers  to  create  electronic  margins,  or  columns,  in  which  they  can  write 
and  communicate  through  their  annotations.  In  CCML,  annotations  are  classified  into  three  categories: 
explanation,  question,  and  comment.  The  main  characteristics  of  CCML  are: 

(1)  Based  on  the  experiment,  CCML  presents  six  marks  and  annotation  CCML  tags. 

(2)  The  marks  have  three  degrees  of  importance  levels  against  respective  corrections. 

(3)  Removing  all  the  CCML  tags  generates  the  original  text. 

(4)  The  revised  text  is  derived  from  the  CCML  document. 

(5)  Because  CCML  documents  are  text-formatted,  it  is  easy  to  send  them  by  e-mail. 

Needless  to  say,  CCML  inherits  its  features  from  SGML. 


CoCoA2:  On-line  Markup  Based  Collaborative  Learning  Environment 


Learning  process 

Correcting  exercise  can  help  learners  understand  reasons  for  writing  errors.  Exercise  materials  can  also 
transform  the  learning  experience  and  enhance  learner  motivation.  In  making  full  use  of  every  corrected 
document  and  writing  errors  data  processed,  in  CoCoA,  the  correcting  exercise  method  is  suggested  to  assist 
learners  for  enhancing  their  writing  ability.  Figure  2 shows  the  method  of  providing  learner  with  some  similar 
documents  for  learner’s  correcting  exercises. 

(1)  Learner  A writes  an  original  text  and  sends  it  to  a teacher  by  e-mail. 

(2)  The  teacher  corrects  the  text  with  CoCoA-Editor. 

(3)  The  teacher  sends  learner  A the  marked-up  text  by  e-mail. 

(4)  The  teacher  stores  the  marked-up  text  of  learner  A into  the  CCML  document  database. 

(5)  CoCoA2  selects  a suitable  document  for  proofreading  exercises  of  learner  B using  the  CCML  document 
database.  Analyzing  the  document  database,  the  system  recommends  a candidate  document  containing 
similar  errors  to  learner  B’s  errors.  In  figure  2,  the  system  selects  a learner  A’s  document  for  the  exercise. 

(6)  Teacher  sends  learner  B the  CCML  document  of  learner  A. 

(7)  CoCoA  generated  the  original  document  from  the  CCML  text,  and  provides  learner  B the  uncorrected 
document  that  includes  similar  writing  errors  of  learner  B.  Then,  learner  B revises  the  text  with  CoCoA2. 

(8)  After  learner  B finishes  correction,  CoCoA2  evaluates  it  through  the  comparison  between  the  correction  of 
learner  B and  that  of  the  teacher.  Moreover,  learner  B can  be  helped  to  re-correct  the  document  with  some 
advice  of  learner  A who  received  the  corrected  text  by  the  teacher  before. 


In  CoCoA2,  it  is  very  important  how  the  system  can  support  process  5 and  8.  These  designs  are  described  in 
the  following  section. 
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(3)  Teacher  sends  leaner  A 
the  corrected  text  with  marks. 
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with  learner  A in  correcting  exercise. 
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Figure  2:  Learning  process  of  collaborative  learning  in  CoCoA2. 
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Selecting  a target  document  for  proofreading  exercise 

In  order  to  select  an  appropriate  document  for  proofreading  exercise,  CoCoA2  analyzes  the  CCML  document 
using  natural  language  processing  technologies  (Feng,  Ogata  & Yano,  1998).  Error  words  are  identified  by  a 
morpheme  analyzer  called  Chasen  (Matsumoto,  1997).  Statistical  analysis  of  these  errors  can  help  the  system 
to  find  a suitable  document  that  includes  similar  types  of  errors  to  a target  learner  for  the  exercise.  Using  a TF- 
IDF  vector  space  model  (Salton  et  al.,  1994),  writing  errors  are  classified.  TF-IDF  is  one  of  the  most  successful 
and  well-tested  techniques  in  information  retrieval.  A document  is  represented  as  a vector  of  weighted  terms. 
The  computation  of  the  weights  reflects  empirical  observations  regarding  text.  Terms  that  appear  frequently  in 
one  document  (TF  = term  frequency),  but  rarely  on  the  outside  (IDF  = inverse  document  frequency),  are  more 
likely  to  be  relevant  to  the  topic  of  the  document.  Therefore,  the  TF-IDF  weight  of  a term  in  one  document  is 
the  product  of  its  term-frequency  (TF)  and  the  inverse  of  its  document  frequency  (IDF).  In  addition,  to  prevent 
longer  documents  from  having  a better  chance  of  retrieval,  the  weighted  term  vectors  are  normalized  to  unit 
length.  With  this  method,  CoCoA2  compares  the  number  of  each  morpheme  of  the  learner's  errors  with  that  of 
other  learners’  errors,  and  CoCoA2  finds  similar  documents  including  the  same  errors  as  the  target  learner’s 
errors. 

Evaluation  of  learner’s  correction 

The  system  evaluates  learner's  correction  as  follows: 

1 . Standardizing  the  corrected  documents; 

2.  Comparing  correction  of  the  respective  document; 

3.  Scoring  a point  on  the  learner’s  correction. 

4.  Advising  the  learner  about  the  different  correction. 

The  standardization  is  needed  to  make  it  easy  to  compare  the  two  corrections,  because  the  learner  may  correct 
the  text  with  the  different  marks  from  the  teacher.  The  standardization  is  based  on  the  following  two  rules: 

(1)  Replacement  rule:  If  there  is  an  insert  mark  after  delete  mark,  or  if  there  is  an  delete  mark  after  insert 
mark,  then  the  two  makes  change  one  replacement  mark. 

(2)  Union  rule:  If  the  two  same  marks  adjoin,  they  are  united.  For  instance,  the  delete  mark  and  the 
neighboring  delete  mark  are  combined  into  one  delete  mark. 

After  the  standardization,  the  system  compares  the  correction  of  the  learner  with  the  model  correction  of  the 
teacher.  Correction  marks  of  two  standardized  documents  are  classified  into  the  following  states: 

1 . Same  correction:  Learner  completely  corrected  the  same  words/sentences  as  the  teacher. 

2.  Different  correction:  The  correction  of  learner  is  different  from  that  of  the  teacher.  For  instance,  corrected 
words  are  different,  or  learner  was  not  aware  of  writing  errors. 

In  order  to  make  it  easy  for  learner  to  understand  the  evaluation  result,  the  system  gives  a score  on  the 
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proofreading  exercise.  The  point  of  same  answer  (A^  is  proposed  as  follows: 

n is  the  total  number  of  teacher's  correction.  Mi  shows  the  correspondent  number  of  correction  between  the 
learner  and  the  teacher.  Ci  shows  the  number  of  the  teachers  correction.  The  more  similar  to  the  teacher  the 
correction  of  the  learner  is,  the  higher  X 


Implementation  of  CoCoA2 

CoCoA2  consists  of  a server  for  a teacher  and  a client  for  a learner.  With  CoCoA-Server,  teacher  stores  all  of 
the  learners’  documents  that  the  teacher  has  corrected  with  CoCoA-Editor  and  the  teacher  selects  the 
document  for  proofreading  exercise.  CoCoA2-Client  shows  the  proofreading  learner  the  uncorrected  document 
that  is  generated  from  the  CCML  document.  After  finishing  correction,  CoCoA2-Client  evaluates  it  and 
connects  the  learner  and  the  original  writer. 
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CoCoA2-Server 


Figure  3 shows  an  interface  of  the  document  selection  function  of  CoCoA2-Server,  Window  A lists  all  of  the 
learners’  names.  Teacher  selects  a learner  for  proofreading  exercise  from  this  list,  e.g.,  Wang.  Then,  the 
system  shows  some  files  for  proofreading  in  window  B,  comparing  errors  in  all  of  Wang’s  documents  and  that 
of  the  other  documents.  Moreover  a list  of  degree  of  similarity  appears  in  window  C.  The  degree  shows  the 
similarity  between  errors  of  Wang’s  documents  and  that  of  another  documents.  With  this  window,  teacher  can 
easily  select  a suitable  document  for  the  learner’s  proofreading  exercise  by  referring  the  degree. 
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Figure  3:  Interface  for  the  document  selection  for  proofreading  exercise. 


CoCoA2-Client 


CoCoA2-Client  is  extended  from  CoCoA-Editor.  Whenever  a learner  wants  to  study  about  compositions,  s/he 
can  start  a proofreading  exercise  with  CoCoA2  and  correct  a text  with  (A).  The  text  is  generated  from  the 
CCML  document  that  the  teacher  sent  to  the  learner.  This  system  begins  to  evaluate  the  results  of  the  learner’s 
correction  by  pushing  "evaluate”  button.  The  results  are  shown  in  (B).  The  same  answer  rate  is  53%  in  this 
case,  comparing  the  learner's  correction  with  the  teacher's  correction.  "Show  answer"  button  displays  the 
teacher's  correction  as  an  answer.  "Same  answer"  button  shows  only  the  same  corrections  of  learner  as 
teacher's  correction.  On  the  other  hand,  "different  answer"  button  displays  the  different  corrections  of  learner 
from  teacher's  corrections.  "Advise"  button  shows  helpful  hints  in  (C),  e.g.  indicating  the  places  to  correct. 
"Collaboration"  button  creates  a communication  channel  between  the  learner  and  the  original  writer  of  the 
document.  The  learner  reviews  and  corrects  in  collaboration  with  the  original  writer.  They  can  communicate 
with  an  online  chat  tool  or  email.  If  the  original  learner  agrees  with  the  learner's  correction,  the  exercise  will 
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Figure  4:  Interface  of  CoCoA2-Client. 


Conclusions 

This  paper  proposed  a computer  supported  proofreading  exercise  environment  called  CoCoA2  where  a teacher 
and  learners  can  exchange  electronic  marked-up  documents.  Although  we  tried  to  use  CoCoA2  in  Japanese 
language  learning,  CoCoA2  is  independent  of  a language  domain.  Therefore,  CoCoA2  will  be  able  to  be  used 
for  learning  any  language.  In  the  future  work,  we  will  evaluate  CoCoA2  in  detail  during  long  term,  and  we 
will  also  investigate  how  to  support  multi-lingual  language  learning  in  a cyber  space.  CoCoA2  have  been 
implemented  in  Tcl/Tk  on  both  Microsoft  Windows  95/98/NT  and  Sun  workstations.  A Japanese  version  of 
this  software  is  distributed  to  other  researchers  and  developers  for  demonstration.  For  further  information,  our 
home  page  (http://www-yano.is.tokushima-u.ac.jp/research/cocoa/cocoa.html)  is  available.  A commercial 
product  of  CoCoA  that  is  called  E-Correct  is  also  download-able  from  the  following 
URL:http://www3.apex.co.jp/'^apex/  corret/english/ 
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Abstract:  During  a qualitative  case  study  in  which  middle  school  learners  used  computer  tools  to 
scaffold  scientific  problem  solving,  three  research  questions  were  addressed:  how  do  students  use 
technological  tools  to  find,  frame,  and  resolve  open-ended  problems;  what  is  the  nature  of  science 
learning  in  problem-based,  Internet  environments;  and  how  do  attitudes  and  environmental 
variables  influence  students’  problem  solving?  While  the  students  understood  how  to  use  most 
tools  procedurally  (i.e.,  collecting  basic  information),  they  lacked  strategic  understanding  for  why 
tool  use  was  necessary  (i.e.,  organizing,  evaluating,  justifying  ideas).  Students  scored  average  to 
high  on  assessments  of  general  content  understanding,  but  developed  artifacts  suggesting  their 
understanding  of  specific  micro  problems  was  naive  and  rife  with  misconceptions.  Process 
understanding  was  also  inconsistent,  with  some  students  describing  basic  problem  solving 
processes,  but  most  students  unable  to  describe  how  tools  could  support  open-ended  inquiry. 
Personal  barriers  to  effective  problem  solving  included  naive  epistemologies,  while  environmental 
barriers  included  a lack  of  communication  activities  and  training.  In  future  studies,  tools  may 
support  higher-order  thinking  by  addressing  study  recommendations:  training  students 

procedurally  and  strategically,  perhaps  longitudinally;  using  students’  personal  problem  models  as 
an  explicit  part  of  instructional  strategies;  balancing  students’  epistemological  comprehension  of 
problem  complexity  with  self-directed,  self-management  tactics;  and  employing  the  teacher  as 
dialectic  facilitator. 


Introduction 

National  science  standards  call  for  increasing  student  exposure  to  inquiry  and  real-world  problem  solving  (Hurd, 
1997;  National  Academy  of  Sciences,  1995).  Students  can  benefit  from  environments  that  stress  the  engagement  of 
real  problems  and  the  development  of  thinking  skills  over  the  memorization  of  discrete  facts.  The  Internet  is  an 
ideal  vehicle  for  contextualizing  problems  with  its  seemingly  endless  supply  of  resources.  Problems  arise,  however, 
since  young  students  are  developmentally  ill-prepared  to  handle  open-ended  learning  and  have  cognitive  difficulties 
processing  hypermedia.  Computer  tools  were  used  to  determine  how  they  might  support  the  process  of  solving  open- 
ended  problems.  A preliminary  hypothesis  suggested  students  would  solve  open-ended  problems  more  appropriately 
if  they  used  tools  in  a manner  consistent  with  higher-order  critical  and  creative  thinking. 

Methods 

Design  and  Participants 

An  embedded  case  study  design  was  employed  with  an  eighth  grade  science  class.  Multiple  embedded  units  were 
analyzed:  12  individual  participants  who  were  grouped  into  6 dyads  that  together  represented  the  case  or  whole 
classroom. 

Materials:  Tools 

The  Knowledge  Integration  Environment  (KIE)  computer  tools  were  chosen  to  scaffold  student  inquiry  (Linn  & 
Slotta,  in  press).  The  tools  were  designed  to  support  students  in  developing  advanced  scientific  understanding  by 
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integrating  and  reconciling  scientific  evidence  on  the  Internet  with  individual  scientific  models.  The  KIE  tools  were 
of  three  primary  types:  processing,  scaffolding,  and  communicating.  The  technical  infrastructure  of  the  research 
setting  disallowed  the  use  of  communication  tools  for  this  research. 

Processing  tools  seek  to  activate  and  model  cognitive  processes  while  extending  the  capacity  of  human  memory 
(Kozma,  1987;  1992).  KIE  information  processing  tools  included  collecting  tools  to  save  web  evidence  into  a 
conceptual  framework,  organizing  tools  to  categorize  evidence,  and  integrating  tools  to  take  notes  and  reflect  on  the 
data.  Students  in  this  study  also  used  non-KIE  information  generating  tools  to  develop  sketches  and  web  pages 
defending  their  ideas. 

Researchers  suggest  scaffolding  tools  in  particular  are  needed  to  assist  learners  who  seek  to  process  hypermedia 
effectively  (Kinzie  & Berdel,  1990;  Lee  & Lehman,  1993).  Scaffolding  tools  are  "engineered  to  assist  learners  in 
performing  tasks  for  which  they  would  otherwise  be  unprepared"  (Laffey,  Tupper,  Musser,  & Wedman,  1998,  p. 
75).  They  fall  into  three  classifications:  procedural,  conceptual,  and  metacognitive  (Hannafin,  Land,  & Oliver,  in 
press;  liyoshi  & Hannafin,  1996).  KIE  tools  allow  for  the  embedding  of  procedural  instructions  suggesting  how  to 
go  about  tasks,  conceptual  prompts  suggesting  what  topics  were  important  to  consider,  and  metacognitive  advice 
suggesting  strategies  for  resolving  problems  posed. 

Materials:  Curriculum  Units 

A theory  comparison  unit  was  employed  in  a pilot  study,  asking  students  to  examine  then  choose  and  defend  either 
the  uniform  or  catastrophic  theory  of  earth  formation.  A design  unit  was  employed  for  the  research  study,  asking 
students  to  find  and  frame  various  earthquake  engineering  problems,  then  resolve  them  with  an  original,  evidence- 
based  solution. 

Data  Sources 

Data  sources  included  artifacts,  interviews,  field  notes,  and  tests.  Eight  artifacts  collected  included  such  items  as 
student  research  questions,  notes,  argument  graphs,  and  web  pages.  Field  notes  were  taken  daily  during  the  four- 
week  study.  Each  of  six  dyads  was  interviewed  three  separate  times  as  the  study  progressed  about  their  use  of  tools, 
their  scientific  understanding,  and  their  attitudes  toward  inquiry-based  learning.  The  teacher  participant  was 
interviewed  twice  regarding  the  same  student  outcomes.  Individual  students  were  interviewed  after  the  study  to 
clarify  their  understanding  of  the  open-ended  problem  solving  process.  A content  test  and  ordered  tree  instrument 
(Naveh-Benjamin,  McKeachie,  Lin,  & Tucker,  1986)  were  used  to  determine  conceptual  understanding. 

Procedures 

Students  were  trained  on  tool  use  by  means  of  a 2-day  mini-project  and  a 3-week  pilot  study,  during  which  they 
were  explicitly  instructed  how  and  when  to  use  specific  tools.  In  the  research  study,  however,  students  were  given 
the  choice  of  using  or  not  using  the  tools  to  accurately  depict  "intentional"  tool  use.  They  were  not  told  which  tools 
to  use  in  response  to  their  tasks,  but  were  provided  with  procedural  scaffolding  describing  how  to  use  tools  if  they 
forgot. 

Data  Analysis 

As  recommended  by  Yin  (1994),  data  analysis  sought  evidence  confirming  or  disproving  a theoretical  proposition: 
young  students  who  use  computer  tools  to  manipulate  hypermedia  resources  in  higher-order  creative  and  critical 
ways  (e.g.,  organize,  annotate,  represent)  will  effectively  solve  open-ended  problems.  Hypothesis  refinement 
situated  this  research  within  the  analytic  induction  tradition-  beginning  with  theory  in  mind  and  building  to  more 
focused  theory  (Merriam,  1 998).  Data  analysis  consisted  of  coding  and  reduction,  then  display  and  pattern  seeking 
(Miles  & Huberman,  1994).  The  data  were  reduced  by  coding  against  typological  frameworks  of  critical  and 
creative  thinking  to  address  tool  use.  Further  codes  were  developed  openly  by  seeking  instances  of  learning, 
attitudes,  and  environmental  factors  in  the  data  that  would  directly  inform  the  theory-built  research  questions.  A 
lengthy  aggregated  text  and  partially-ordered  checklist  matrices  were  developed  from  codes  to  seek  categorical 
evidence  of  students  thinking  with  tools,  learning  science,  or  displaying  attitudes  and  behaviors  (Creswell,  1998; 
Miles  & Huberman,  1994;  Stake,  1995)  (see  Figure  1).  Matrices  illustrated  the  thinking  skills  exhibited  by  each 
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dyad  and  the  tools  associated  with  that  thinking.  Asterisks  indicated  a tool  was  used,  but  ideal  or  expected  thinking 
did  not  occur  (e.g.,  students  created  a web  page  to  describe  their  solution,  but  did  not  justify  their  idea  with  web 
evidence).  Categories  were  generated  by  studying  the  matrices  for  commonalities  in  thinking  (e.g.,  '‘the  first  dyad 
rarely  used  metacognitive  scaffolding,  did  the  second,  did  the  third,  ...  ?”).  The  process  was  consistent  with 
“...discriminant  sampling”  suggested  by  Lincoln  & Cuba  (1985).  Categories  were  then  related  by  studying  their 
"semantic”  relationships  (e  g.,  X is  a kind  of  Y,  X is  a cause  of  why)  (Spradley,  1979).  Shared  relationships 
suggested  themes  that  were  developed  into  an  "analytic  text"  of  findings  (Miles  & Huberman,  1994).  For  example, 
one  tool  use  theme  noted  was  students  organize  information,  but  rarely  use  their  frameworks  as  the  basis  for 
decision  making  or  integration  of  further  evidence. 
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Figure  1,  Checklist  matrix  partially  ordered  across  three  study  dimensions. 


Findings 

On  Tool  Use 

Students  did  not  use  tools  strategically,  as  evidenced  by  three  underlying  themes.  First,  students  used  processing 
tools  more  appropriately  when  scaffolded  and  when  provided  with  limited  resource  sets.  During  the  initial  problem 
finding  stage,  students  were  provided  with  13  web  pages  and  conceptual  scaffolding  in  the  form  of  question  prompts 
in  their  note-taking  tool.  Students  answered  prompts  explicitly,  identifying  underlying  premises,  conclusions,  and 
restrictions,  but  neglected  to  describe  patterns  across  and  personal  reactions  about  the  information.  During  a problem 
framing  stage,  students  were  provided  with  65  web  pages  and  no  conceptual  scaffolding.  Students  identitied  less 
information  overall,  and  skimmed  resources  by  searching  for  keywords  related  to  their  problems. 

Second,  students  did  not  attribute  strategic  values  to  the  tools,  using  most  infrequently  and  often  inappropriately.  For 
instance,  some  students  did  not  effectively  use  their  organization  tool  to  seek  patterns  in  the  web  evidence  by 
generating  problem-specific  categories.  Only  1 of  12  participants  made  use  of  a system  function  to  integrate 
additional  web  information  as  "evidence"  for  a specific  category  or  problem.  Further,  few  students  used  print-based 
documents  to  evaluate  resources  or  their  own  solution  ideas  with  criteria,  and  most  did  not  use  information 
generation  tools  to  argue  for  and  justify  their  ideas  (e.g.,  making  hyperlinks  on  web  pages  to  supporting  evidence). 
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Finally,  students  relied  heavily  upon  explicit  procedural  scaffolding,  but  neglected  to  use  metacognitive  guidance 
suggesting  appropriate  inquiry  strategies. 

On  Learning 

Three  types  of  learning  were  evident  in  the  study:  problem,  content,  and  process.  Students  identified  basic  problem 
information  and  used  that  to  generate  some  original  solutions.  Students  based  these  ideas  on  limited  evidence, 
however,  so  many  of  the  original  ideas  were  flawed  in  design.  For  example,  one  dyad  suggested  an  hourglass- 
shaped structure  with  external  bracing.  External  bracing  was  a suggested  technique  found  in  the  web  evidence,  but 
the  odd-shaped  hourglass  design  contradicted  further  evidence  showing  the  structural  failure  of  asymmetrical 
structures.  Another  dyad  created  an  alternative  design  for  building  columns,  employing  layers  of  discs  that  could 
slide  or  pop-out  during  an  earthquake  instead  of  a single  column  that  was  shown  to  buckle  and  collapse.  The 
solution  was  both  original  and  relevant,  but  would  not  likely  support  an  overhead  load.  The  ability  of  students  to  test 
and  revise  their  ideas  within  a lab  or  computerized  simulation  might  have  helped  them  move  beyond  partial 
understanding  and  misconception. 

Beyond  student  understanding  of  specific  self-selected  earthquake  engineering  problems,  most  students  acquired 
basic  content  understanding  about  general  earthquake  engineering  constructs.  Only  one-half  of  the  participants, 
however,  demonstrated  process  understanding  for  open-ended  problem  solving  following  the  study.  Most  were  able 
to  list  steps  in  the  process  during  post-study  interviews,  but  few  could  describe  the  strategic  value  of  tools  to  support 
that  process. 

On  Student  and  Environmental  Factors 

Student  and  environmental  themes  suggested  some  possible  causes  for  inadequate  tool  use.  First,  students  exhibited 
characteristics  of  a naive  epistemology:  evaluating  ideas  summatively  rather  than  formatively,  infrequently  revising 
and  updating  ideas  on  the  basis  of  evidence,  perceiving  open-ended  problem  solving  as  non-complex,  and  seeking 
help  from  others  rather  than  the  system  in  a self-directed  manner.  Further,  they  managed  resources  and  time 
improperly:  skimming  evidence,  neglecting  to  use  non-electronic  resources,  and  rushing  through  activities.  In  terms 
of  environmental  factors,  some  students  needed  additional  procedural  training  on  tool  use.  Almost  all  students 
needed  explicit  training  on  strategic  tool  use,  or  at  least  more  direction  from  the  facilitators  regarding  which  tools  to 
use  for  strategic  purposes.  Finally,  when  student  activities  involved  communicating  with  peers  and  facilitators, 
students’  thinking  reached  high  levels  never  managed  by  tool  use  alone  (e.g.,  evaluation  and  justification  of  ideas). 
More  communication  activity  during  the  study  would  have  supported  better  inquiry  and  thinking. 

Interpretations  and  Recommendations 

Support  Tool  Use  through  Extended  Training  and  Improved  Interfaces 

Students  did  not  understand  the  value  of  their  tools  in  this  study.  Myers  (1993,  p.  75)  found  students  using 
hypermedia  to  solve  open-ended  problems  needed  time  to  learn  to  process  that  information  in  higher-order  ways, 
"...perhaps  a semester  or  school  year."  Such  implicit,  longitudinal  training  will  eventually  lead  students  to  more 
appropriate  tool  use,  or  a more  proactive  technique  would  involve  teachers  explicitly  training  students  on  the 
strategic  value  of  specific  tools. 

Students  will  tend  to  use  scaffolding  tools  when  they  are  conveniently  located.  Tool  designers  should  embed 
scaffolding,  particularly  metacognitive,  within  information  processing  tools.  These  tools  should  not  be  far  removed 
from  the  processes  they  seek  to  support.  Further,  tool-based  scaffolding  by  itself  is  insufficient  to  support  open- 
ended  problem  solving.  Myers  (1993)  found  teachers  are  also  required  to  prompt  students  into  appropriate  action. 

Support  Learning  with  More  Facilitation  and  Feedback  Mechanisms 

Students  lacked  endogenous  models  of  earthquake  engineering  from  which  to  interpret  the  evidence  presented 
(Bruning,  Schraw,  & Ronning,  1995).  In  such  situations,  dialectical  scaffolding  is  called  for  whereby  peers  and 
teachers  with  prerequisite  facts  and  skills  help  students  to  reach  proper  levels  of  task  engagement  with  selective 
help.  Other  techniques  include  the  provision  of  manipulation  tools  which  can  promote  personal  theory  development 
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(Land,  1995),  or  the  provision  of  ongoing  feedback  from  the  teacher  regarding  student  progress  and  performance. 
Finally,  students  should  be  required  to  state  a prediction  or  hypothesis  early  in  their  inquiries,  constantly  reflecting 
on  their  stated  assumptions  as  they  encounter  new  evidence  and  revising  their  understanding  when  appropriate. 

Encourage  Student  Intentions  Strategically 

Some  students  in  this  study  exhibited  characteristics  of  a naive  epistemology  which  may  simply  be  age-related  or  a 
function  of  the  traditional  school  environment.  While  the  problems  provided  to  students  in  this  study  were  open- 
ended,  the  KIE  activity  structure  itself  was  rather  explicit,  structuring  a series  of  activities  for  students.  Students  did 
not  learn  to  apply  processes,  but  gathered  and  learned  basic  information  as  one  might  expect  from  traditional 
learning  environments.  It  seems  likely  that  open-ended  environments  need  to  support  students  more  implicitly  in 
learning  to  handle  the  complexities  of  ill-defined  problems.  Laffey  et  al.  (1998)  designed  tools  that  allow  students  to 
set  their  own  goals  and  objectives,  plan  responsibilities,  and  allocate  work  to  specific  periods  of  time.  To  help 
students  overcome  their  naive  epistemologies,  tool  sets  should  strive  to  support  processes  implicitly  rather  than 
activities  explicitly. 

Implications  and  Conclusion 

This  research  has  implications  for  tool  and  instructional  designers  who  can  learn  to  design  better  open-ended 
systems  and  activities.  Further  implications  apply  to  administrators  and  science  teachers  who  seek  to  appropriately 
implement  such  curriculum  reform.  Authentic  scientific  inquiry  is  a model  of  learning  that  is  far  from  prevalent  in 
today’s  classroom.  It  stresses  the  development  of  lifelong  metacognitive  and  strategic  skills  (Duffield  & Grabinger, 
1997;  O’Reilly,  1991).  Realistically,  open-ended  activities  take  more  time  to  develop  and  implement,  also  limiting 
the  number  of  topics  that  can  be  taught  (O’Reilly,  1991).  However,  educators  will  better  prepare  students  for  the 
scientific  community  if  they  cover  fewer  topics  in  more  depth  and  stress  the  development  of  strategic  thinking 
processes  (Linn,  1987;  Norris,  1988,  p.  219;  O’Reilly,  1991). 

The  study  hypothesis  was  neither  supported  or  refuted.  Students  did  not  exhibit  higher-order  thinking,  nor  did  they 
use  tools  effectively.  If  students  had  used  tools  with  strategic  intent,  it  is  still  possible  they  would  have  thought 
critically  and  creatively  about  their  resources  to  generate  elaborate,  original  solutions.  Future  research  can  build 
upon  the  study  recommendations  to  improve  tool  use  and  develop  better  process  understanding.  Longitudinal  studies 
of  students  learning  to  strategically  use  tools  over  time  are  highly  recommended. 
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Abstract:  This  paper  reports  on  a study  undertaken  in  Australia  where  the  practice  and 
development  of  generic  skills  was  studied  in  the  context  of  a Web-based  learning 
environment.  The  results  showed  that  students  perceived  the  learning  environment  provided 
many  opportunities  for  them  to  practice  the  generic  skills  being  explored.  Furthermore  the 
results  showed  high  levels  of  student  perception  of  development  of  these  generic  skills.  We 
conclude  that  Web-based  learning  environments  offer  many  opportunities  for  teachers 
wishing  to  develop  students’  generic  skills  and  this  is  yet  another  advantage  of  using  this 
technology  for  program  delivery. 


Introduction 

Education  is  often  viewed  by  teachers  and  students  as  collections  of  courses  and  units,  with  units  being 
comprised  of  learning  objectives  linked  to  domain  specific  outcomes.  Increasingly  we  are  becoming  aware 
that  effective  university  teaching  and  learning  extends  far  beyond  the  development  of  skills  and  knowledge 
in  specific  subject  domains  (eg.  Bearing  Report,  1 997).  An  holistic  view  of  education  suggest  other  forms 
of  skills  and  knowledge  that  are  important  outcomes  of  university  education.  These  holistic  skills  are  often 
common  to  all  courses  and  units  irrespective  of  their  subject  domain  and  are  often  referred  to  as  generic  or 
key  skills.  The  generic  skills  are  those  that  students  need,  to  become  successful  learners  and  successful 
practitioners  in  their  fields  of  study  and  work  and  in  other  aspects  of  their  life.  In  a society  where  change  is 
rapid  and  where  knowledge  and  information  are  now  marketable  commodities,  this  extended  set  of 
outcomes  in  curriculum  and  teaching  are  gaining  prominence  in  all  sectors  of  education. 

One  of  the  main  factors  causing  the  emergence  of  generic  skills  as  a required  outcome  of  formal  education 
is  the  growing  use  of  technology  in  all  facets  of  life.  Many  of  the  generic  skills  that  are  defined  and 
described  relate  to  students’  abilities  to  make  meaningful  use  of  technology.  The  influence  of  technology 
as  the  subject  of  generic  skills  coupled  with  emerging  opportunities  for  students  learning  with  technology 
appears  to  create  a powerful  synergy  for  generic  skills  development.  The  use  of  technology  in  teaching  and 
learning  already  has  been  shown  to  provide  many  opportunities  to  teachers  and  learners.  These 
opportunities  include  among  other  things,  increased  access  to  learning,  increased  flexibility  for  learners  and 
enhanced  learning  outcomes  in  domain  specific  knowledge  outcomes.  If  it  can  be  shown  that  particular 
forms  of  technology  use  can  also  provide  opportunities  for  learners’  generic  skills  development,  teachers 
and  institutions  can  be  guided  by  these  findings  into  modifications  and  changes  to  their  technology-based 
learning  environments  to  provide  even  greater  returns  on  their  investments  and  efforts. 

Defining  Key  Skills  and  Competencies 

Defining  the  full  range  of  generic  and  transferable  skills  that  are  useful  (or  essential)  for  university  students 
is  an  exhaustive  process.  It  is  as  almost  as  exhaustive  as  finding  agreement  in  the  terms  which  might  best 
be  used  to  describe  the  set.  In  the  context  of  this  paper  we  use  the  term  generic  skills  to  describe  the  full 
range  of  domain  independent  skills  that  are  considered  to  be  essential  life  skills  for  people  both  in  and  out 
of  the  workforce.  There  has  been  interest  in  key  skills  as  outcomes  of  education  for  many  years  now.  In 
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different  countries,  different  sets  of  skills  are  listed,  all  with  similarity  and  consistency  to  each  other.  The 
1993  New  Zealand  Curriculum  Framework,  for  example,  proposed  8 essential  skills  as  important  outcomes 
of  New  Zealand  Schooling.  These  were:  communication  skills;  information  skills;  self-  management  and 
competitive  skills;  physical  skills,  numeracy  skills,  problem  solving  skills  and  co-operative  skills,  work  and 
study  skills.  In  the  United  Kingdom,  The  Qualifications  and  Curriculum  Authority  have  developed  a set  of 
key  skills  that  the  government  and  much  of  industry  consider  as  essential  for  successful  lifelong  learning 
and  a flexible  workforce.  The  QCA  key  skills  comprised  6 areas;  communication;  information  technology; 
application  of  number;  working  with  others;  improving  learning  and  performance  and  problem  solving. 

The  conceptualisation  of  generic  skills  is  problematic  in  many  ways.  Bennett,  Dunne  & Carre  (1999) 
describe  a number  of  synonyms  including  personal,  transferable,  generic,  common,  work  and  employment 
related  skills.  Are  they  skills  or  are  they  competencies,  capabilities  or  learning  outcomes?  The  skills  that 
are  reported  as  important  outcomes  of  schooling  tend  to  be  broad  and  extensive.  In  university  teaching,  the 
skills  set  is  often  narrowed  to  focus  on  those  diat  are  not,  or  cannot,  be  taught  as  discrete  components  of 
coursework.  At  the  same  time,  the  generic  skills  sought  by  university  education  assume  learners  are 
numerate  and  literate  as  a consequence  of  the  requirements  of  university  entrance.  The  key  skills  included 
in  the  mission  statements  of  most  universities  tend  to  include  higher  level  aims  relating  to  critical  thinking, 
inquiry  and  a capacity  for  lifelong  learning.  Typical  generic  skills  described  for  graduates  include: 

• skills  that  students  need  to  develop  to  becoming  successful  and  self-sufficient  learners.  For  example, 
information  literacy,  metacognitive  skills  (eg.  Candy,  1994); 

• intellectual  and  imaginative  powers,  understanding  and  judgement,  problem  solving  and  critical 
thinking  skills  and  an  ability  to  see  relationships  (eg.  Squires,  1990;  Ramsden,  1992); 

• personal  and  interpersonal  skills  needed  for  communication,  cooperative  and  collaborative  teamwork, 
and  leadership  (eg.  Ashcroft  & Foreman-Peck,  1994;  Gibbs  et  al.  1994). 

• skills  required  for  successful  work  practices  including  time  management,  task  management  leadership 
and  self  evaluation  (eg.  Gibbs  et  al.  1992;  Blumhof  et  al.  1996). 


Management  of  Self 

• Manage  time  effectively 

• Set  objectives,  priorities  and  standards 

• Take  responsibility  for  own  learning 

• Listen  actively  with  purpose 

• Use  a range  of  academic  skills 

• Develop  and  adapt  learning  strategies 

• Show  intellectual  flexibility 

• Use  learning  in  new  or  different  situations 

• Plan/work  towards  long-term  goals 

• Purposefully  reflect  on  own  learning 

• Clarify  with  criticism  constructively 

• Cope  with  stress 

Management  of  Others 

• Carry  out  agreed  tasks 

• Respect  the  views  and  values  of  others 

• Work  productively  In  a cooperative  context 

• Adapt  to  the  needs  of  the  group 

• Defend/justify  views  and  actions 

• Take  initiative  and  lead  others 

• Delegate  and  stand  back 

• Negotiate 

• Offer  constructive  criticism 

• Take  the  role  of  chairperson 

• Learn  in  a collaborative  context 

• Assist/support  others  In  learning 

Management  of  Information 

Management  of  Task 

• Use  appropriate  sources  of  information 

• Identify  key  features 

• Use  appropriate  technologies 

• Conceptualise  ideas 

• Use  appropriate  media 

• Set  and  maintain  priorities 

• Handle  large  amounts  of  Information 

• Identify  strategic  options 

• Use  appropriate  language  and  form 

• Plan/implement  a course  of  action 

• Interpret  a variety  of  information  forms 

• Organise  sub-tasks 

• Present  information  competently 

• Use  and  develop  appropriate  strategies 

• Respond  to  different  purposes/contexts  and 

• Assess  outcomes 

audiences 

• Use  information  critically 

• Use  information  in  innovative  and  creative  ways 

Table  1:  A framework  for  the  development  of  key  skills  (Bennett,  Dunne  & Carre,  1999) 

Bennett,  Dunne  & Carre  (1999)  offer  an  elegant  model  (Table  1. 

) to  conceptualise  key  skills  in  the  higher  education  sector  by  suggesting  a framework  comprising  4 broad 
managerial  skills.  These  authors  argue  that  the  important  key  skills  are  fundamentally  those  associated 
with  being  able  to  manage  self,  others,  information  & task.  They  contend  that  the  bulk  of  all  generic  skills 
can  be  embraced  within  the  four  categories  of  this  framework  and  they  propose  that  this  model  can  be 
applied  “to  any  discipline,  to  any  course  and  to  the  workplace  and  indeed  to  any  other  context  “ (p.  77). 

Developing  Keys  Skills 

The  provision  of  generic  skills  and  competencies  can  be  undertaken  through  a variety  of  forms. 
Traditionally  it  has  been  attempted  through  3 main  types  of  learning  activity:  integrated  approaches,  stand 
alone  approaches  or  approaches  where  key  skills  are  developed  in  parallel  with  the  conventional 
curriculum,  (eg  Drummond  et  al.  1997  cited  in  Bennett,  Dunne  & Carre,  1999).  Many  writers  question 
whether  it  is  really  possible  for  the  learning  taking  place  in  university  settings  to  be  transferable  to 
vocations  and  the  work  place.  One  school  of  thought  suggests  that  through  situating  learning  in  meaningful 
contexts,  this  transfer  can  be  facilitated  (eg.  Brown,  Collins  & Duguid,  1989).  Contemporary  thinking  is 
that  university  learning  can  be  significantly  strengthened  through  workplace-based  practica  and 
applications  (eg.  Seagraves,  Kemp  & Osborne,  1996). 

On-line  problem-based  learning 

We  have  been  using  the  WWW  in  our  university  teaching  for  several  years  now  and  recognise  that  one  of 
the  major  benefits  of  the  use  of  this  technology  is  the  facility  it  provides  to  incorporate  learning  activities 
which  help  to  develop  students’  generic  and  transferable  skills  (eg.  (eg.  Herrington  & Oliver,  1997;  Oliver, 
Omari  & Stoney,  1999).  In  the  introductory  multimedia  course  in  our  undergraduate  program,  we  have 
embraced  a Web-supported  problem-based  learning  environment  with  which  the  students  engage  and 
interact  with  the  course  content.  The  system  involves  learners  working  in  collaborative  groups  to  explore 


356 


the  solutions  to  open  ended  and  ill-defined  problems.  From  these  activities  we  have  sensed  learners 
developing  a raft  of  generic  skills  through  their  interactions  and  activities  in  the  Web-based  course. 

The  on-line  system  we  have  been  using  is  a database  driven  Web-based  learning  system  designed  to 
support  a form  of  problem-based  learning.  The  database  elements  enable  the  system  to  record,  manage  and 
support  the  interactions  of  a large  number  of  students  and  a large  number  of  tutors.  The  system  supports 
problem-based  learning  by  providing  a means  for  students  to  collaborate  on  set  problems,  to  share 
resources,  to  post  solutions  and  to  compare  and  review  answers  from  other  groups.  The  system  was 
designed  with  a degree  of  flexibility  in  mind  to  enable  it  to  be  used  in  a variety  of  ways  in  a variety  of  units 
and  courses.  Typical  use  of  the  system  in  a course  of  study  is  expected  to  revolve  around  the  following 
activities  which  were  characteristic  of  our  initial  implementations  and  trials.  The  system  is  Web-based  and 
all  the  following  activities  are  undertaken  using  on-line  technologies: 

• Each  week  a problem  is  presented  to  students,  the  purpose  of  which  is  to  contextualise  and 
authenticate  the  weekly  content  of  the  course. 

• Students  are  required  to  work  within  groups  of  3 or  4 to  explore  the  topic,  locate  relevant  information 
and  resources,  consider  the  various  options  and  outcomes  and  to  create  a response  which  is  informed 
and  well  argued. 

• The  group  post  this  solution  to  the  bulletin  board,  an  action  which  then  reveals  to  them  the  solutions  of 
the  other  groups  in  their  cohort . Each  group  is  asked  to  review  the  solutions  of  the  others  and  through 
their  feedback,  the  solutions  are  given  a peer-assessed  grade.  Each  tutor  also  reads  the  solutions  and 
gives  a mark  which  is  added  to  the  peer-assessed  grade  to  give  an  overall  mark  for  the  solution. 

• Students  are  able  to  view  the  marks  achieved  each  week  in  a number  of  ways  and  this  mark 
accumulates  throughout  the  semester  as  each  new  problem  is  solved. 

The  nature  of  the  learning  involves  high  degrees  of  self-regulation  on  the  part  of  the  students.  There  is  very 
little  support  or  help  provided  and  students  have  to  judge  when  and  where  their  research  will  stop  and  when 
and  where  their  summarising  and  reviewing  will  stop.  At  first  glance,  this  learning  environment  appears  to 
provide  many  opportunities  for  students  to  practice  and  develop  their  generic  skills.  It  was  our  impression 
from  observing  the  students’  learning  activities  and  levels  of  engagement  in  the  problem  solving  tasks  that 
students  had  many  opportunities  to  practice  a large  number  of  the  generic  skills  shown  in  Table  1 . What 
was  not  clear  though  was  the  extent  to  which  these  skills  were  being  practised  and  the  influence  of  this 
practice  on  actually  developing  and  extending  students’  existing  levels  of  these  skills.  The  aim  of  this 
study  became  to  explore  the  generic  skills  which  the  problem-based  learning  environment  caused  learners 
to  practise  and  the  extent  to  which  this  developed,  or  had  the  potential  to  develop,  the  particular  skills. 

Method 

The  learning  environment  was  implemented  in  a first  year  university  course  with  an  enrolment  of  90 
students.  The  theoretical  component  of  the  course  was  well  suited  to  problem-based  learning  and  the 
system  was  used  to  support  students’  problem  solving  activities  through  weekly  problems  for  1 0 weeks  of 
the  course.  A questionnaire  was  administered  to  students  in  the  8^  week  of  the  course  after  they  had 
completed  7 problem  solving  activities.  The  questionnaire  provided  students  with  a list  of  the  generic  skills 
described  by  Bennett,  Dunne  & Carre  (1999)  and  students  were  asked  to  reflect  on: 

1 . the  level  of  practice  that  the  problem-based  learning  environment  encouraged;  and 

2.  the  extent  to  which  this  practice  contributed  to  their  development  of  the  discrete  skills. 

The  instrument  used  a Likert  Scale  to  enable  learners  to  show  their  responses.  The  questionnaire  took  15 
minutes  to  administer  and  was  completed  by  the  students  in  their  class  settings.  After  the  questionnaire  had 
been  given,  a small  number  of  students  were  interviewed  to  gather  some  first  had  data  on  the  strategies 
employed  in  the  problem  solving  processes.  It  was  clear  that  students  were  using  a range  of  different 
approaches  in  creating  their  weekly  problem  solutions  and  the  different  strategies  were  likely  to  influence 
their  practice  and  development  of  the  generic  skills. 

Results 

a.  Practising  Generic  Skills 

To  explore  the  level  of  practice  encouraged  by  the  problem-based  learning  environment,  the  questionnaire 
showed  each  of  the  generic  skills  listed  in  Table  1 and  asked  students:  How  often  did  you  practise  the 
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following  during  the  problem  solving  activities?  Responses  were  made  using  a Likert  scale  with  the 
options:  never,  a little  and  regularly.  Table  2 below  shows  the  students  responses  as  a percentage  of  the 
options  chosen.  A weighted  index  was  calculated  to  determine  the  relative  amount  of  practice  each  skill 
received  on  a scale  from  0 to  2,  with  0 representing  no  practice  and  2 representing  regular  practice. 


Generic  Skill 

never  % 
0 

a little  % 
1 

regularly  % 
2 

Weighted 
Index  (0-2) 

Management  of  Self 

3 

57 

40 

1.37 

Management  of  Other 

15 

47 

38 

1.31 

Management  of  Task 

4 

52 

44 

1.39 

Management  of  Information 

1 

44 

55 

1.55 

Average 

6% 

50% 

44% 

Table  2:  Students’  Perceptions  of  the  Relative  Levels  of  Practice  of  Generic  Skills 

The  striking  outcome  from  this  first  dataset  was  the  extent  to  which  students  perceived  the  learning 
environment  encouraged  them  to  practise  the  generic  skills.  There  is  insufficient  space  to  show  the  discrete 
scores  achieved  for  the  40  individual  skills  listed  in  the  table  but  the  summary  data  conveys  the  overall 
information  quite  clearly.  For  example,  in  terms  of  practising  generic  skills  associated  with  self- 
management,  the  sample  of  students  perceived  that  they  failed  to  practise  any  the  listed  generic  skills  only 
3%  of  the  time.  In  the  sample  of  32  students,  there  were  only  10  instances  when  students  felt  that  their 
learning  activities  did  not  practise  one  of  the  12  generic  skills  listed  for  self-management. 

The  summary  data  shows  that  students  perceived  that  overall  their  learning  activities  involved  a little 
practice  of  the  generic  skills  50%  of  the  time  and  regular  practice  of  the  generic  skills  44%  of  the  time. 
Among  the  4 categories,  the  one  category  that  seemed  to  be  least  practised  was  that  of  management  of 
others.  When  we  say  least  practised,  there  were  more  instances  in  this  category  than  the  others  where 
students  perceived  they  were  practising  these  skills  least  but  the  overall  practice  level  still  remained  quite 
high.  The  skills  in  this  category  related  to  working  with  others  and  practising  the  skills  associated  with 
delegation,  negotiation,  collaboration  etc.  It  was  clear  that  the  way  we  had  structured  the  approach  limited 
students’  opportunities  to  continually  practise  some  of  these  skills.  The  students  remained  in  the  same 
group  throughout  the  semester  and  from  one  week  to  the  next,  some  members  assumed  roles  within  the 
group  and  this  led  to  a lessening  of  their  practise  of  such  tasks  as  delegation,  chairmanship,  etc.  The 
category  that  students  appeared  to  practise  the  most  was  that  of  information  management.  The  use  of  the 
Internet  and  WWW  as  a resource  base  was  a fundamental  component  of  the  Web-based  activity  and  each  of 
the  problems  involved  many  skills  associated  with  access,  reading,  reflecting  and  using  information.  It  is 
difficult  to  imagine  another  learning  environment  where  more  practice  was  possible. 

b.  Development  of  Generic  Skills 

The  study  also  sought  to  establish  students’  perceptions  on  how  much  skills  development  was  associated 
with  the  practice  they  were  doing.  Although  we  often  associate  practise  with  skills  development,  it  is 
useful  to  explore  whether  this  is  the  students’  perception  and  to  explore  whether  or  not  students  feel  that 
their  skills  are  being  developed  through  their  learning  activities.  Table  3 shows  the  results  of  students’ 
responses  to  the  question.*  How  much  did  the  problem  solving  tasks  help  you  to  develop  these  skills?  against 
each  of  the  4 generic  skills  categories.  To  explore  this  area  in  more  detail,  students  were  given  a 5 point 
Likert  scale  on  which  to  record  their  results,  the  scale  ranging  from  not  at  all,  through  to  very  much. 

Once  again,  the  results  were  very  positive  and  showed  that  the  students  perceived  that  the  learning 
environment  helped  them  in  more  than  minor  ways  to  develop  the  various  generic  skills.  There  were  only  a 
small  number  of  instances  where  students  felt  that  the  learning  environment  did  not  contribute  to  the 
development  of  particular  generic  skills.  The  value  of  6%,  as  an  average  of  those  perceiving  no  learning  at 
all,  in  Table  3 is  indicative  of  a very  small  number  of  students  and  an  even  smaller  number  of  discrete 
skills.  The  perceived  levels  of  skills  development  correlate  highly  with  students  perceived  levels  of 
practice  and  if  we  assume  that  students  can  tell  when  they  are  developing  skills,  then  the  amount  of  practice 
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is  a metric  which  can  be  used  to  reflect  skills  development.  Interviews  with  the  students  tended  to  support 
this  assumption.  In  interviews,  the  students  gave  examples  of  their  increased  abilities  to  seek  and  use 
information,  to  work  with  others  and  to  manage  open-ended  tasks,  all  examples  of  the  skills  in  Table  1 . 


Generic  Skill 

not  at  all 
0 

1 

2 

3 

very 
much  4 

weighted 
index  (0-4) 

Management  of  Self 

3 

21 

37 

31 

8 

2.12 

Management  of  Other 

12 

20 

31 

26 

11 

2.00 

Management  of  Task 

5 

20 

33 

31 

11 

2.23 

Management  of  Information 

4 

9 

31 

38 

18 

2.50 

Average 

6% 

18% 

33% 

31% 

12% 

Table  3:  Students’  Perceptions  of  the  Relative  Levels  of  Generic  Skills  Development 

Table  3 provides  an  indication  of  the  relative  levels  of  skills  development  between  the  4 categories  and 
once  again,  the  weighted  index  provides  a measure  by  which  this  can  be  compared.  Once  again  students 
perceived  the  management  of  others  to  be  the  skills  which  developed  to  the  least  extent  and  the 
management  of  information  to  the  skill  which  they  developed  the  most.  In  all  cases  however,  both  the 
results  and  the  student  interviews  suggested  that  the  students  felt  that  had  developed  skills  across  all  4 
categories.  The  weighted  indices  were  all  equal  or  greater  than  2.00,  the  middle  value  of  the  scale.  Once 
again,  the  discrete  data  for  each  of  the  40  skills  provides  some  interesting  material  for  discussion  and 
highlights  particular  skills  which  a small  minority  of  students  felt  were  less  developed  than  others. 

Summary  and  Conclusions 

The  results  from  this  study  have  provided  strong  confirmation  of  our  expectations  that  the  Web-based 
learning  environment  helps  students  to  practise  and  develop  their  generic  skills.  The  study  shows  that  the 
students’  perceptions  are  that  there  were  few  generic  skills  in  the  framework  described  by  Bennett,  Dunne 
& Carre  (1999)  that  were  not  practised  in  the  process  of  learning  through  the  Web-based  learning 
environment  which  we  had  developed.  Likewise,  the  results  showed  that  the  students  found  few  instances 
where  their  learning  activities  did  not  also  contribute  to  the  development  of  the  generic  skills  listed.  While 
there  were  some  small  differences  observed  in  the  perceived  levels  of  practice  and  development  across  the 
4 categories,  the  results  were  very  positive  for  them  all. 

It  is  our  intention  now  to  create  our  own  set  of  generic  skills  based  on  the  needs  and  demands  of  our  course 
and  programs  and  to  use  the  data  from  this  study  to  see  which  skills  might  need  to  be  intentionally 
considered  in  modifications  we  will  make  to  the  learning  environment.  We  sense  that  it  will  be  possible  to 
modify  the  learning  environment  in  a number  of  ways  to  address  those  skills  which  we  are  particularly 
interested  in  developing  and  to  reduce  the  levels  of  practice  of  students  in  skills  which  are  of  lesser 
importance.  One  limitation  with  this  paper  has  been  its  reliance  on  student  perceptions  as  its  measure  of 
the  amount  of  practice  and  level  of  skills  development.  For  the  purpose  of  this  exploration,  we  feel  that  our 
results  still  have  validity  and  we  base  this  feeling  on  substantiating  evidence  from  student  interviews  and 
the  consistency  of  the  data  that  was  gathered.  We  suspect  that  in  our  future  work  we  will  need  to  explore 
ways  in  which  we  might  find  more  empirical  measures  on  which  to  base  assumptions  about  generic  skills 
development  and  plan  to  make  this  an  integral  part  of  our  future  work. 

Note:  An  earlier  version  of  this  paper  was  presented  at  ASCILITE’99  held  in  December,  1999  in  Brisbane, 
Australia. 


References 

Ashcroft,  K.,  & Foreman-Peck,  L.  (1994).  Managing  Teaching  and  Learning  in  Further  and  Higher 
Education.  London,  UK:  The  Falmer  Press. 

Bennett,  N.,  Dunne,  E.  & Carre,  C.  (1999).  Patterns  of  core  and  generic  skill  provision  in  higher  education. 
Higher  Education,  5 7(  1 ),  7 1 -93 . 


859 


Biggs,  J.  (1987).  Student  Approaches  to  Learning  and  Studying.  Melbourne:  Australian  Council  for 
Educational  Research. 

Blumhof,  J.,  Honeybone,  A.,  Pearlman,  D.,  & Finn,  K.  (1996).  Tackling  the  problem  of  skills  development 
in  a modular  degree  programme:  the  Skillswise  Project.  In  G.  Gibbs  (Ed.),  Improving  Student  Learning: 
Using  Research  to  Improve  Student  Learning  (pp.  328-339).  Oxford,  UK:  Oxford  Centre  for  Staff 
Development. 

Brown,  J.,  Collins,  A.  & Duguid,  P.  (1989).  Situated  cognition  and  the  culture  of  learning.  Educational 
Researcher,  18,  32-42. 

Bearing  Report  (1997).  Higher  Education  in  the  Learning  Society.  London:  HMSO. 

Gibbs,  G.,  Rust,  C.,  Jenkins,  A.,  & Jaques,  D.  (1994).  Developing  Students*  Transferable  Skills.  Oxford: 
The  Oxford  Centre  for  Staff  Development. 

Herrington,  J.  & Oliver,  R.  (1997).  Multimedia,  magic  and  the  way  students  respond  to  a situated  learning 
environment  Australian  Journal  of  Educational  Technology,  /3(2),  127-143. 

Havard,  M.,  Hughes,  M.  & Clarke,  J.  (1998).  The  introduction  and  evaluation  of  key  skills  in 
undergraduate  courses.  Journal  of  Further  and  Higher  Education,  22(1),  61-688. 

Laurillard,  D.  (1993).  Rethinking  University  teaching:  A Framework  for  the  Effective  Use  of  Educational 
Technology.  London:  Routledge. 

Oliver,  R.  Omari,  A.  & Stoney,  C.  (1999).  Collaborative  learning  on  the  World  Wide  Web  using  a 
problem-based  learning  approach  In  (J.  Chambers  Ed.)  Selected  Papers  from  the  Tenth  International 
Conference  on  College  Teaching  and  Learning^.  Jacksonville:  FCCJ. 

Ramsden,  P.  (1992).  Learning  and  Teaching  in  Higher  Education.  London:  Routledge. 

Seagraves.  L.,  Kemp,  I.  & Osborne,  M.  (1996).  Are  academic  outcomes  of  higher  education  provision 
relevant  to  and  deliverable  in  the  workplace  setting?  Higher  Education,  32(2),  157-176. 

Squires,  G.  1990.  First  Degree:  the  Undergraduate  Curriculum.  Open  University  Press,  Buckingham. 

The  New  Zealand  Curriculum  Framework  1993.  Wellington:  Ministry  of  Education. 


360 


The  ANTA  Flexible  Toolbox  Project:  Developing  Sustainable  and  Scalable 
On-Line  Learning  Materials  for  Vocational  Education  and  Training 


Ron  Oliver, 

School  of  Communications  and  Multimedia, 
Edith  Cowan  University,  Western  Australia. 

r. oliver@cowan.edu.au 

Stephen  Towers, 

Queensland  Open  Learning  Network 
Brisbane,  Australia. 

s. towers@qoln.net.au 

Nic  Pearl 

Australian  National  Training  Authority, 
Brisbane,  Australia, 
pearln@anta.gov.au 


Abstract:  This  paper  describes  a project  undertaken  in  Australia  where  national  flexible  learning 
materials  were  developed.  The  paper  describes  the  rationale  and  background  of  the  project  and  the 
processes  by  which  a set  of  Toolboxes  were  developed  for  widespread  use  in  the  Australian  context. 
The  paper  discusses  issues  that  arose  out  of  the  project  and  provides  information  and  ideas  for 
others  who  are  looking  at  ways  to  create  scalable  and  sustainable  technology-based  resources  for 
education. 


Introduction 

On-line  technologies  such  as  the  Internet  and  World  Wide  Web  hold  considerable  promise  for  improving 
accessibility  and  quality  of  education  and  training  programs.  In  the  context  of  the  vocational,  education  and  training 
(VET)  sector,  these  technologies  have  particular  relevance.  In  the  Australian  setting,  many  authorities  within  the 
VET  sector  are  using  and  promoting  flexible  learning  as  a preferred  delivery  form  for  their  courses  and  programs 
(eg.  ANTA,  1996).  Within  the  VET  sector,  flexible  delivery  can  support  a wide  array  of  learning  settings  including 
open  learning  and  work-based  learning.  On-line  learning  materials  are  seen  to  be  fundamental  for  increasing 
flexibility.  However,  the  move  to  on-line  learning  among  these  groups,  while  desirable  from  many  perspectives,  has 
been  slow  and  fragmented. 

There  has  been  considerable  interest  paid  from  all  sectors  in  recent  years  to  strategies  that  can  support  and 
encourage  efficiencies  and  economies  in  both  the  development  and  delivery  of  on-line  teaching  and  learning.  Much 
of  the  early  development  of  materials  has  been  conducted  by  individuals  or  small  teams  for  local  and  restricted  use 
(Collis  & Oliver,  1999).  Whereas  with  CD-ROMs,  people  often  planned  large  projects  with  big  budgets  and  gained 
returns  on  their  investment  through  widespread  use  of  the  product,  the  same  cannot  be  said  for  the  majority  of 
developments  of  on-line  learning  materials.  National  and  state  authorities  responsible  for  VET  within  Australia  have 
recognised  that  developing  online  materials  is  expensive  and  best  achieved  through  a collaborative  approach  (EdNa 
VET  Advisory  Group  1999).  It  was  this  challenge  that  prompted  the  Australian  National  Training  Authority 
(ANTA)  to  consider  strategies  that  could  motivate  and  support  registered  training  organisations  (RTOs)  to  embrace 
modes  of  flexible  delivery  for  their  students.  A resource  strategy  was  formulated  by  which  sets  of  generic  and 
customisable  materials  could  be  developed  and  applied  widely  throughout  the  training  sector.  The  intention  was  to 
create  efficiency  and  economies  of  scale  though  large  scale  production  of  learning  materials  capable  of  wide 
acceptance  and  use.  This  plan  became  the  ANTA  Flexible  Toolbox  Project  and  is  further  described  in  this  paper. 

Vocational  Education  and  Training  in  Australia 

In  Australia,  the  mission  of  ANTA  is  to  “ensure  that  the  skills  of  the  Australian  labour  force  are  sufficient  to  support 
internationally  competitive  commerce  and  industry  and  to  provide  individuals  with  opportunity  to  maximise  their 
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potential”  (ANTA,  1998a).  ANTA’s  agenda  has  been  “to  deliver  training  more  responsively  and  efficiently  and  to  a 
wider  catchment  area”  (ANTA,  1998).  The  Authority  recognises  that  technological  advances  have  been  the  catalyst 
for  new  forms  of  educational  and  training  communication  systems  and  that  the  demand  for,  and  supply  of  vocational 
education  and  training  is  globalising.  Consequently  it  has  enacted  and  sponsored  strategies  to  encourage  the  VET 
providers  to  provide  more  flexible,  technological-based,  delivery  for  domestic  and  international  markets.  ANTA 
recognised  that  implications  of  technology  advances  and  globalisation  required  “new  ways  of  creating  and 
customising  training  material”  (Eccles,  1998).  The  Toolbox  initiative  was  one  of  ANTA’s  strategies  to  encourage 
development  and  delivery  of  more  flexible  learning  materials  for  the  training  market  - particularly  for  online 
learning.  In  1998-1999  ANTA  invested  S3. 8 million  in  “toolboxes”  with  the  aim  of  producing  a “smorgasbord  of 
multimedia  resources  from  which  providers  can  pick  and  choose  in  designing  online  training  programs”  (Eccles, 
1998). 

The  setting  of  a national  framework  with  many  discretionary  elements  provided  a perfect  opportunity  for  the 
Australian  National  Training  Authority  to  explore  the  concept  of  Flexible  Toolboxes  to  support  learning.  In  the  first 
instance,  any  products  developed  in  this  setting  would  have  widespread  application  providing  significant  economies 
of  scale.  Secondly  the  use  of  on-line  technologies  appeared  to  provide  and  promote  the  many  forms  of  flexibility 
associated  with  the  delivery  of  the  Training  Packages  in  the  national  setting. 


Flexible  Toolboxes 

The  concept  of  a Toolbox  as  a building  block  for  flexible  delivery  and  open  learning  is  relatively  common  in 
education.  The  concept  imbues  the  notion  of  a set  of  tools  that  can  be  implemented  by  teachers  and  trainers  to 
create  and  implement  on-line  and  computer-based  learning  environments.  The  concept  is  a powerful  strategy  as  a 
means  to  create  sustainable  and  scalable  materials  for  technology-based  teaching  and  learning  (eg.  Hanley, 
Schneebeck  & Zweier,  1998).  Where  a Toolbox  creates  unique  and  innovative  opportunities  is  through  the  provision 
of  alternative  options  to  the  creative  teacher.  Furthermore  a Toolbox  is  intended  to  facilitate  its  use  through  its 
ability  to  stand-alone  and  to  be  employed  through  customisation  and  adaptation.  Given  these  inherent 
characteristics.  Toolboxes  are  often  chosen  as  productive  solutions  to  the  development  and  delivery  of  flexible  and 
on-line  courses  (eg.  Oliver,  Omari  & Herrington,  1998). 

Toolboxes  come  in  varying  shapes  and  forms  and  the  term  itself  is  quite  flexible  in  the  way  in  which  it  is  applied. 
The  single  most  important  characteristic  of  the  Toolbox  concept  is  its  flexibility  of  use  and,  in  the  various 
interpretations  that  have  been  made  of  the  concept,  this  characteristic  is  extremely  variable.  Figure  1 shows  a 
continuum  describing  flexibility  of  use  as  it  can  be  applied  to  toolboxes  used  for  on-line  learning.  Across  the 
continuum,  it  is  possible  to  identify  discrete  forms  of  toolbox  applications.  Consideration  of  the  discrete  forms 
shows  that  there  are  opportunities  across  each  for  open  and  flexible  materials  development.  At  the  same  time,  each 
of  the  forms  carries  limitations  and  pitfalls  that  can  act  to  impede  their  application  and  use. 


Because  of  the  impetus  to  create  resources  for  Training  Packages,  ANTA  deliberately  sought  to  develop  Toolboxes 
that  were  towards  the  “restricted”  end  of  the  continuum.  It  was  recognised  that  it  would  be  an  advantage  if  the 
materials  that  were  developed  could  be  implemented  in  their  immediate  form.  While  a number  of  providers  of  VET 
courses  have  substantial  information  technology  resources,  many  are  not  in  a position  to  make  major  changes  to  on- 
line materials.  Providing  online  materials  which  demanded  high  levels  of  input  was  not  seen  as  desirable.  At  the 
same  time,  it  was  considered  important  for  the  materials  to  have  the  capacity  to  support  user  customisation,  and  thus 
an  important  component  of  the  initial  design  brief  was  to  require  that  any  materials  developed  for  the  project  had  a 
capacity  for  customisation  by  the  teachers  and  training  providers. 


restricted  flexibility  of  use  total 


-► 


• curriculum  packages 

• on-line  courses 

r complete  programs 

• course  exemplars 


• course  modules 

• Java  Applets 

• course  resources 

• curriculum  materials 


• generic  tools 

• content-free  applications 

• construction  templates 

• construction  shells 


J 


Figure  1:  A flexibility  continuum  describing  Toolboxes  for  on-line  learning 
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Therefore,  in  the  context  of  this  project,  a Toolbox  was  defined  as  “a  set  of  multimedia  resources  that  provides  a 
framework  for  the  development  of  training  programs  for  online  delivery  to  meet  the  requirements  of  Training 
Packages.  As  discussed,  a key  feature  of  the  Toolbox  was  that  providers  could  customise  the  contents  to  meet  the 
needs  of  learners  and  enterprises.  The  toolbox  was  required  to  constitute  two  essential  components  (1)  a completed 
multimedia  program  designed  to  satisfy  specific  training  competencies  and  (2)  a range  of  tools  and  functions  which 
provide  access  to  the  program’s  components  and  resources.  The  building  blocks  that  could  come  together  to  form 
the  multimedia  package  were  conceived  to  be  available  as  separate  resources.  Each  toolbox  was  expected  to  utilise  a 
wide  range  of  the  building  blocks.  For  example: 

• Packages  of  generic  resources  in  a variety  of  media; 

• Developer’s  notes,  guidelines,  supporting  documentation  and  technical  guidance  to  facilitate  online  provision; 

• Learning  materials  ‘translated’  from  the  Training  Package  and  combined  with  authoring  software  to  facilitate 
the  development  of  further,  customised  materials; 

• Assessment  instruments  which  meet  the  assessment  guidelines  contained  in  the  Training  Package  and  combined 
with  authoring  software  to  facilitate  the  development  of  further,  customised  materials; 

• Interactive  tutorial  packages  on  researching  through  the  internet,  including  directory  services,  and  WWW  with 
simulated  search  examples,  search  tasks; 

• Generic  case  studies,  and  interactive  simulation  games, 

• Guidelines  for  communication  functions  (e.g.  email,  web  forums  and  chat)  and  student  management  systems; 

• Tips,  hints,  ideas  for  student  friendly,  student  ‘grabbing’  instructional  design; 

• Web  sites  with  email  for  comments,  support  and  feedback  on  material  developed  as  part  of  the  Toolbox  and 

• Listings  of  publicly  accessible  Web  sites  relevant  to  the  content  of  the  targeted  Training  Package. 

The  Development  Project 

A detailed  specification  was  developed  and  applications  were  called  nationally  for  parties  interested  in  developing 
Flexible  Toolboxes.  Over  100  expressions  of  interest  were  received  and  after  an  exhaustive  selection  process, 
twelve  were  finally  selected  for  funding.  In  all  instances  the  Toolboxes  represented  cohesive  sets  of  course  materials 
for  various  qualifications  comprising  around  12  areas  of  study,  each  with  many  defined  competencies.  The  various 
Toolboxes  covered  a cross-section  of  training  areas  and  qualification  levels.  Their  proposed  construction  included  a 
range  of  authoring  tools  and  included  CD-ROM,  LAN  and  Web  as  intended  delivery  media.  Portability,  the 
capability  of  the  software  to  run  on  a range  of  hardware  systems,  differed  across  the  Toolboxes.  Some  Toolboxes 
were  designed  to  be  platform  independent  whereas  others  required  the  purchase  or  access  to  proprietary  delivery 
systems,  eg  WebCT.  Similarly  the  ease  of  customisation  (sequencing  learning  events,  adding/deleting/modifying 
resources  and  content,  changing  text/graphics)  varied  considerably  across  a range  of  dimensions  (user  skill  levels, 
pre-requisite  software  and  hardware).  An  additional  project  called  the  ‘Toolbox  Central’  was  established  to  provide 
an  on-line  professional  development  and  quality  systems  to  support  the  implementation  and  application  of  the 
Toolboxes  once  they  had  been  developed.  The  Toolboxes  covered  a variety  of  training  areas  including  Hospitality 
and  Tourism,  Metal  and  Engineering,  Community  Services,  Agriculture,  Information  Technology,  Workplace 
Training  and  Assessment,  Printing  and  Graphic  Arts,  Rural  Business  Management  and  Agriculture. 

The  Evaluation  Team 

The  Toolboxes  were  developed  over  an  eight-month  period  and  were  completed  by  the  middle  of  1999.  ANTA 
recognised  that  this  was  rather  a bold  and  innovative  project  and  contracted  an  external  team  to  evaluate  and  monitor 
the  development  of  the  individual  Toolboxes  and  to  provide  a mentoring  facility  for  the  developers  to  assist  with  the 
timely  completion  of  a quality'  product.  The  Evaluation  Team  established  close  contact  with  all  the  developers  and 
created  channels  of  communication  between  them  to  create  a sense  of  community  and  a sharing  of  knowledge. 
Communication  and  contact  between  the  various  teams  was  facilitated  by  a Web-based  Communication  Hub  that 
was  built  for  the  project  with  a range  of  communication  facilities  (Figure  2).  The  Evaluation  Team  monitored  all 
stages  of  the  development  process  and  provided  constructive  feedback  in  such  areas  as  project  management 
strategies,  instructional  design  and  system  architecture.  It  was  the  intention  of  ANTA  to  use  this  project  as  a 
learning  experience  to  inform  and  guide  subsequent  projects  of  this  nature. 

It  was  interesting  but  not  surprising  to  find  that  the  various  development  teams,  while  anxiously  pursuing  best 
practice  in  all  aspects  of  the  design  and  development,  were  generally  reticent  and  reluctant  to  work  in  the 
collaborative  ways  which  the  evaluation  team  was  seeking.  The  teams  tended  to  have  tight  and  fixed  timeframes 
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with  firm  development  ideas  and  tended  to  know  what  they  wanted  to  produce  and  the  best  approaches  to  be  used. 
For  these  sorts  of  reasons,  the  planned  high  levels  of  communication  and  interaction  planned  for  the  teams  were  not 
achieved  nor  was  collaboration  and  cooperative  sharing.  The  Communication  Hub  did  however  provide  a vehicle 
for  enabling  public  scrutiny  of  the  project  and  its  various  components  and  it  served  well  as  a distribution  centre  for 
project  information  and  document  delivery. 


Figure  2:  The  Components  of  the  Communication  Hub 


Project  Outcomes 

On  completion  of  the  project,  the  development  teams  had  produced  multimedia  resources  for  nine  Training 
Packages.  Each  development  team  created  resources  for  ten  to  fifteen  units  of  competency,  each  representing 
approximately  40  hours  of  independent  study.  The  set  of  Toolboxes  represented  an  enormous  collection  of  new  and 
exciting  technology-based  resources  for  the  Australian  VET  sector.  An  on-line  distribution  system  was  established 
to  enable  institutions  and  training  providers  to  view  selections  of  the  available  resources  and  to  order  them  for  their 
own  use.  All  of  the  Toolboxes  were  made  available  to  any  Registered  Training  Organisation  for  the  cost  of  the 
media  and  distribution.  In  practice,  this  meant  people  could  buy  $A3.5  million  dollars  worth  of  teaching  and 
learning  resources  for  about  $A2500.  In  most  instances  though,  RTOs  would  buy  only  those  Toolboxes  appropriate 
to  their  own  training  areas. 

Despite  the  broad  range  of  subject  areas  involved  and  the  range  of  industries  represented,  there  were  many  patterns 
and  consistencies  identified  by  the  evaluation  team  between  the  learning  materials.  Some  comparisons  and 
descriptions  of  the  various  toolboxes  in  terms  of  teaching  and  learning  processes  functionality,  flexibility  and  utility 
are  given  below.  There  were  many  indicators  by  which  the  success  of  this  project  could  be  measured.  Some  of  the 
more  important  indicators  of  success  included: 

• The  timely  completion  of  the  Toolboxes; 

• Extent  of  completion  to  the  agreed  technical  and  functional  specifications; 

• The  quality  of  the  learning  materials  within  such  functional  criteria  as:  scope,  flexibility,  and  portability;  and 

• The  quality  of  the  learning  materials  from  an  instructional  perspective. 

Despite  the  array  of  hurdles  and  obstacles  that  faced  the  teams  embarking  on  new  projects  and  working  in  areas  and 
ways  which  were  distinctly  different  from  previous  experience,  timely  completion  was  met  by  most  and  completion 
to  the  technical  and  functional  specifications  was  generally  achieved.  The  evaluation  team  observed  an  array  of 
issues  associated  with  large  scale  project  development  and  many  recommendations  and  new  strategies  were  fed  back 
to  the  organising  panel  as  guides  for  future  projects.  Perhaps  the  most  pressing  item  that  emerged  from  the 
evaluation  was  the  need  for  detailed  specifications  to  guide  all  aspects  of  the  design  and  the  development  processes. 
While  developers  were  given  strong  indicators  at  the  commencement  of  the  project  as  to  the  intended  audience  and 
the  need  for  flexibility  in  their  development  approaches,  the  resulting  products  differed  broadly.  The  differences 
occurred  across  a number  of  areas  and  it  became  evident  that  the  varying  interpretations  placed  by  developers  on  the 
stated  requirements  and  product  utility  had  the  capacity  to  limit  aspects  of  the  projects’  intentions. 
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Technical  Requirements  The  requirements  for  utility  as  defined  in  the  project  brief  included  such  technical 
requirements  as  multiple  platform  capability,  modularity  in  structure,  and  consistent  file  organisation  and  directory 
structures.  The  large  variations  in  approaches  that  were  ultimately  used  created  a number  of  challenges  for 
determining  how  the  products  would  be  stored  and  distributed.  Across  the  12  Toolboxes,  there  ended  up  being 
many  Gigabytes  of  information  to  store  and  distribute.  While  the  various  Toolboxes  each  represented  independent 
entities,  they  still  needed  forms  of  consistency  to  ensure  maximum  scope  for  use  among  the  various  users  and 
students. 

Performance  The  specifications  that  the  developers  followed  indicated  that  the  Toolboxes  needed  to  provide 
minimum  levels  of  functionality  in  terms  of  progress  tracking,  privacy  safeguards  and  ability  to  be  customised  to 
meet  the  needs  of  the  users.  Once  again,  providing  descriptions  of  the  functional  requirements  alone  resulted  in  a 
range  of  different  approaches  and  the  extent  of  the  differences  proved  to  be  a factor  that  had  the  prospect  to  limit 
application. 

Flexibility  Developers  were  required  to  provide  flexibility  in  the  form  of  an  easily  customisable  set  of  resources 
with  a range  of  customisable  features.  The  end-products  tended  to  meet  the  specified  requirements  but  developers 
had  chosen  different  approaches  and  solutions.  The  majority  of  the  customisable  elements  and  strategies  appeared 
at  the  screen  level  and  involved  a high  overhead  time  to  implement.  Once  again,  the  extent  of  the  variations  gave 
rise  to  the  feeling  among  the  evaluators  that  these  specifications  needed  expansion  for  the  best  outcomes  to  be 
achieved. 

The  second  area  of  inquiry  and  guidance  for  the  evaluation  team  related  to  the  quality  of  the  products  from  a 
teaching  and  learning  perspective  and  as  with  the  functional  elements,  large  differences  emerged  in  this  area  as  the 
following  sections  demonstrate. 

Content  Developing  multimedia  learning  materials  is  different  in  many  respects  to  developing  conventional 
learning  materials.  One  major  difference  exists  in  terms  of  the  content  that  can  be  used.  The  various  Toolboxes 
differed  considerably  in  the  scope  and  extent  of  the  media  elements  used.  It  became  clear  that  some  developers  were 
using  design  and  development  processes  more  suited  to  development  of  print-based  materials  and  were  quite  limited 
in  their  thinking  and  resourcing.  On  the  other  hand,  the  best  examples  from  the  project  provide  evidence  of 
outstanding  uses  of  the  media  and  content  to  support  learning.  Developing  guidelines  which  create  more 
consistency  in  terms  of  the  media  applications  among  products  was  seen  as  an  important  future  activity. 

Learning  Strategies  As  with  the  content,  many  differences  were  observed  in  the  forms  of  learning  strategies 
employed  by  the  instructional  designers  across  the  various  projects.  As  might  be  predicted  when  conventional 
design  processes  are  being  employed,  many  of  the  learning  settings  appeared  as  electronic  forms  of  paper-based 
materials.  Across  the  various  Toolboxes  there  were  many  strong  examples  of  effective  design  for  electronic  and 
interactive  learning  and  many  examples  of  learning  strategies  designed  to  take  advantage  of  the  best  features  of 
technology-based  learning.  For  example,  use  of  video  and  sound  files  in  case-study  settings,  interactive  modules  to 
engage  the  learners  and  activities  using  the  Web  to  facilitate  cooperation  and  communication.  Some  development 
teams  were  constrained  by  the  content  provided  and  the  view  of  their  content  experts  and  industry  expectations  in 
terms  of  how  a learning  package  should  appear. 

Assessment  The  challenge  for  developers  of  technology  and  Web-leaming  environments  is  to  creatively  use  the 
facilities  to  create  assessment  strategies  and  activities  that  enhance  the  learning  experience  as  well  as  measuring 
achievement.  All  of  the  Toolboxes  were  required  to  provide  activities  for  learners  to  self-assess  their  com.petency. 
While  there  were  some  sound  examples  among  the  various  Toolboxes  that  could  guide  future  developments, 
assessment  tasks  tended  to  measure  learning  outcomes  through  conventional  processes  such  as  objective  tasks  and 
multiple  choice  quizzes. 

Media  Usage  One  of  the  major  criticisms  of  education  and  schooling  in  the  past  is  that  it  is  abstract  and  removed 
from  reality.  Multimedia  and  on-line  learning  can  reduce  the  abstractness  of  education  by  providing  learners  with 
access  to  real  life  settings,  realistic  environments  and  authentic  information  and  cases.  Several  Toolbox 
development  teams  used  the  opportunity  provided  by  this  project  to  create  information  and  content  that  reflected  this 
real  world  use.  They  used  examples  drawn  from  real  life  and  presented  cases  rather  than  abstracted  descriptions. 
These  types  of  Toolboxes  contrasted  sharply  with  those  that  tended  to  present  text  descriptions  instead  of  videos  or 
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pictures  to  portray  real  life  settings;  and  subject  matter  experts  talking  about  processes  in  place  of  videos  showing 
the  process  studio  re-enactments  in  place  of  real  life  footage.  It  was  clear  to  the  Evaluation  Team  that  there  were 
many  opportunities  where  Toolbox  projects  could  have  made  better  use  of  the  technologies  from  a teaching  and 
learning  perspective.  However,  it  should  be  noted  that  there  were  many  constraints  on  the  designers  in  terms  of 
expectations  from  teachers  and  industry  about  the  forms  they  perceived  the  learning  materials  should  take.  It  was 
clear  that  a process  of  education  was  needed  to  move  thinking  along  the  path  to  more  student-centred  and  active 
learning  settings.  The  first  set  of  Toolboxes  contained  many  fine  examples  for  people  to  follow  and  these  were  seen 
as  a good  way  to  start  and  support  this  learning  process. 

Summary  and  Conclusions 

This  was  a very  large  multimedia  development  project  by  any  standards.  The  budget  for  the  project  exceeded  $A3.5 
million  dollars  and  the  completed  Toolboxes  contained  on-line  and  technology-based  learning  resources  for  over 
150  competencies,  each  a discrete,  self-contained  online  learning  program.  ANTA  purposely  used  the  1998  project 
as  a pilot  and  as  a learning  exercise  for  future  policy  and  resource  emphasis.  It  was  an  intentional  strategy  to  provide 
flexibility  to  the  developers  to  encourage  and  stimulate  creativity  and  innovation.  The  Evaluation  Team  was 
established  as  a means  to  gather  data  and  to  provide  feedback  which  could  inform  future  practice.  The  main 
findings  of  the  Evaluation  Team  related  to  the  forms  of  specification  and  prescription  needed  to  be  given  to 
developers  to  provide  consistency  between  the  completed  products  in  relation  to  their  technical  and  instructional 
characteristics.  In  essence  these  specifications  could  provide  consistent  forms  of  quality  between  the  Toolboxes  in 
all  the  important  areas. 

As  this  paper  was  being  written,  the  second  stage  of  the  project  was  in  its  early  stages  with  an  increased  budget  and 
the  benefit  of  hindsight  influencing  the  procedures  and  policies  being  used.  An  evaluation  is  planned  that  includes 
investigating  the  use  and  application  of  the  first  set  of  Toolboxes.  At  the  same  time,  the  second  project  has 
concentrated  on  developing  effective  environments  for  on-line  learning  and  exploring  ways  to  encourage  developers 
and  designers  to  use  the  technologies  to  promote  students’  knowledge  and  understanding.  Feedback  from  the  first 
project  has  been  used  extensively  to  guide  and  inform  the  second  development  round. 

There  were  many  things  learned  through  the  development  and  evaluation  processes  associated  with  the  ANTA 
Toolbox  project.  The  process  was  designed  not  only  to  produce  learning  materials  but  also  to  develop  sustainable 
strategies  and  processes  for  these  endeavours.  As  this  project  drew  to  an  end  and  a follow-on  project  commenced,  it 
was  heartening  to  see  the  many  lessons  that  had  been  learned  by  the  stakeholders  in  this  initial  project  being 
reflected  on  and  used  in  meaningful  ways  in  the  follow-up  project. 
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Abstract:  The  growing  use  of  information  and  communications  technologies  in  commerce  and 
industry  is  once  again  encouraging  debate  and  questioning  of  the  development  of  students’  skills 
and  knowledge  in  this  domain.  Whereas  in  the  past,  the  debate  has  centred  on  school  education, 
questions  about  ICT  literacy  are  now  being  raised  in  the  context  of  post-compulsory  education. 
Post-compulsory  education  sits  ahead  of  employment  and  ICT  literacy  in  the  population  and 
workforce  is  seen  as  an  important  component  of  a nation’s  competitiveness  and  advancement. 
This  paper  explores  the  concept  ICT  literacy  in  the  light  of  new  technologies  and  suggests  factors 
that  are  currently  seen  to  limit  and  impede  its  attainment. 


Introduction 

In  the  early  days  of  computers  in  schools,  a common  rationale  for  the  investment  in  hardware  and  software  was 
the  notion  of  computer  awareness.  This  concept  was  premised  on  the  view  that  schools  had  a responsibility  to 
introduce  students  to  this  emerging  technology  and  to  prepare  them  for  the  fact  that  one  day  computers  might  have 
a role  to  play  in  their  daily  living.  As  time  went  on,  and  the  impact  of  computers  in  society  grew,  the  notion  of 
computer  awareness  evolved  to  computer  literacy,  not  only  knowing  about  computers  but  being  able  to  use  them. 
Throughout  the  1980s  the  relevant  literature  abounds  with  descriptions  and  comment  on  both  the  rationale  for 
such  programs  and  evaluations  of  their  success  (eg.  McCormick,  1992). 

In  the  past  ten  years,  computer  technologies  have  diffused  into  many  aspects  of  everyday  life  and  in  recent  years 
this  diffusion  has  been  accelerated  and  exaggerated  by  the  convergence  of  computer  and  communication 
technologies.  The  place  of  computer  technologies  in  schools  has  evolved  from  objects  of  study  or  specialist 
applications  to  tools  for  teaching,  learning  and  administration.  Computer  literacy  disappeared  from  main  gaze  and 
focus  of  attention  as  the  stand  alone  computer  lost  its  novelty.  But  today,  the  concept  is  regaining  attention  as  the 
new  technologies  once  again  gain  novelty  and  public  attention  through  the  next  phase  of  their  development.  This 
renewed  interest  appears  to  be  fuelled  by  Internet  and  communication  technologies,  which  are  causing  yet  another 
revolution  their  use  and  growing  influence  of  life  and  society. 

The  renewed  interest  in  computer  literacy  appears  to  be  driven  not  only  by  those  in  education  and  industry  but 
also  by  governments  looking  to  maintain  or  to  upgrade  their  world  competitiveness  through  leadership  in  an 
activity  set  to  dominate  the  trade  and  corporate  world  in  the  new  millennium.  Whether  it  is  the  place  of  schools  to 
expose  students  to  computers  and  information  technology  (IT)  for  vocational  purposes,  or  simply  for  personal 
needs,  remains  a much  argued  point.  It  is  clear  that  many  in  government  perceive  the  development  of  IT  skills  in 
schools  as  an  economic  and  vocational  imperative.  This  view  is  not  shared  by  all  and  we  are  frequently  reminded 
that  the  role  of  education  is: 
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“Not  so  much  to  closely  align  to  the  ephemeral  demands  of  industry  but  to  equip  students  with  the  more 
fundamental,  expansive  skills  of  being  able  to  critique  and  reflect  on  the  changes  taking  place  in  their 
society”  (Watkins,  1986,  p.  85) 

The  concept  of  computer  literacy  appears  now  to  have  moved  beyond  schools  to  the  post-compulsory  education 
institutions  that  prepare  graduates  for  employment.  The  purpose  of  this  paper  is  to  explore  this  renewed  interest  in 
computer  literacy  and  to  establish  the  growing  implications  for  the  post-compulsory  education  sector. 

Defining  Computer  Literacy 

There  have  been  many  studies  over  the  past  two  decades  into  both  the  concept  and  attainment  of  computer 
literacy.  These  have  focussed  on  many  different  aspects  of  the  perception  of  computer  literacy.  Much  effort  has 
been  put  into  defining  set  of  skills,  which  adequately  describe  a computer  literate  person.  Other  efforts  have 
focussed  on  devising  assessment  tools  to  measure  levels  of  these  skills,  a difficult  task  in  such  a multi-faceted  and 
complex  area.  Further  studies  have  compared  computer  literacy  levels  and  attitudes  towards  computers  with 
different  groups  of  people,  based  on  gender,  ethnicity  or  socio-economic  factors. 

Through  the  1980s,  computer  literacy  tended  to  be  considered  a grab-bag  of  different  skills  and  attributes.  A 
widely  accepted  definition  of  computer  literacy  is  that  of  Simonson,  Maurer,  Montag-Torardi  & Whitaker  (1987) 
who  define  computer  literacy  as: 

“An  understanding  of  computer  characteristics,  capabilities  and  applications,  as  well  as  an  ability  to 
implement  this  knowledge  in  the  skilful  and  productive  use  of  computer  applications  suitable  to  the 
individual  roles  in  society,  (p.  232)” 

In  this  definition,  the  knowledge  and  skills  of  a computer  literate  person  were  divided  into  four  components 
comprising  computer  attitudes,  computer  applications,  computer  systems  and  computer  programming.  While  this 
is  now  a dated  definition  and  classification,  it  has  been  used  and  modified  in  subsequent  studies  (e.g.  Smith  & 
Necessary  , 1996)  and  the  concepts  broadened  to  be  independent  of  a given  time  frame.  The  essential  elements  of 
this  form  of  definition  still  hold  currency  today. 

The  Evolution  to  ICT  Literacy 

The  rapid  developments  in  communications  technologies  which  have  occurred  over  the  past  ten  years  has  seen  a 
broadening  in  the  range  of  skills  that  are  considered  to  be  imperative  to  the  constantly  evolving  notion  of 
computer  literacy.  We  now  see  a spectrum  of  differing  expressions  and  terms  which  relate  to  computer  literacy. 
For  example,  information  technology  literacy^  Net  literacy,  digital  literacy  or  on-line  literacy  (eg.  Gilster,  1 997). 
Overwhelmingly  however  more  recent  references  in  the  literature  still  often  use  computer  literacy  as  an 
appropriate  keyword  for  reference  listings.  Definitions  of  computer  literacy  have  varied  in  their  breadth  over  time 
and  with  the  rapid  changes  in  technology  are  evolving  accordingly. 

Where  once  “basic”  computer  literacy  was  narrowly  defined  and  could  be  used  to  easily  differentiate  students  that 
were  deficient.  Such  students  could  be  directed  to  specific  “context  free”  courses  to  build  their  computer  literacy. 
Over  the  past  decade,  the  notion  of  “basic”  is  too  obscure  when  considering  the  myriad  of  potential  contextually 
driven  applications  of  computers  in  contemporary  society.  Increasingly  there  appear  to  be  diverse  cohorts  of 
students,  each  with  different  bundles  of  computer  skills,  knowledge  and  attitudes.  Students  could  be  considered 
literate  in  one  setting  but  illiterate  in  another.  For  example,  many  school  leavers  have  the  ability  to  confidently 
browse  and  use  the  WWW  but  may  have  little  or  no  file  management  or  word  processing  skills.  Similarly,  there 
are  a large  number  of  mature  aged  students  who  have  had  no  prior  computer  experience. 

The  move  toward  the  broader  term  of  information  and  communication  technology  (ICT)  literacy  has  at  its  roots 
the  need  to  involve  the  acquiring  and  advantageous  use  of  information  through  the  use  of  technology.  Use  of  a 
range  of  communication  tools  such  as  the  Internet,  e-mail  and  the  World  Wide  Web  (WWW)  for  the  location  of 
information  and  dissemination  are  now  considered  to  be  components  of  ICT  literacy  and  yet  not  necessarily  that 
of  computer  literacy.  In  addition,  many  of  the  skills,  which  had  previously  been  associated  with  those  that  an 
individual  would  need  to  have  acquired  in  order  to  be  considered  computer  literate,  are  now  commonly  seen  to  be 
components  of  the  more  encompassing  term  of  ICT  literacy.  Many  authors  describe  the  place  of  computer  literacy 
as  a component  of  a more  encompassing  list  of  ICT  skills  (eg.  Eisenburg  & Johnson,  1 996;  Shapiro  & Hughes, 
1996;  Bruce;  1998). 
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Generic  Skills 

The  notion  of  computer  literacy  is  tending  to  be  considered  a component  of  a larger  set  of  key  skills  that  have 
been  identified  as  important  outcomes  of  schooling  outside  the  domain  of  traditional  curriculum  areas.  These 
generic  skills  include  such  elements  as  reading  and  writing  literacy,  communication  skills,  numeracy,  critical 
thinking  and  people  skills  (eg.  Gibbs  et  al.,  1994;  Harvard,  Hughes  & Clark,  1998).  An  emerging  trend  is  the 
perception  that  ICT  literacy  forms  part  of  a generic  skill  set,  with  implications  for  teaching  and  learning  in  the 
schools,  as  well  as  for  life-long  learning  and  just-in-time  learning.  The  American  Library  Association,  for 
example,  recognizes  the  need  for  ICT  skills  in  the  form  of  information  literacy  skills  as  necessary  for  America’s 
continuing  success  in  the  information  age.  The  ALA  (1999)  discusses  the  future  role  of  the  teacher  moving  from 
textbook  lecturer,  to  coach  and  that  an  effective  information  literacy  (IL)  curriculum  moves  away  from  learning 
traditional  library  location  skills  taught  in  isolation  to  one  of  learning  IL  skills  which  have  been  embedded  in  the 
curriculum.  The  desired  outcomes  are  for  people  who  are  prepared  for  lifelong  learning,  because  they  can  always 
find  the  information  needed  for  any  task  or  decision  at  hand. 

The  role  of  ICT  skills  as  being  ftindamental  to  general  work  place  literacy  is  reiterated  by  Mikulecky  and  Kirkley 
(1998)  and  Tomei  (1999).  The  implications  of  this  for  post-secondary  education  are  that  there  appears  to  be  a 
significant  shift  to  providing  a range  of  generic  skills  that  allow  the  learner  to  prepare  for  a life  where  learning  is 
an  ongoing  process.  The  need  for  appropriate  and  sustainable  ICT  skills  is  implicit  in  these  types  of 
recommendations. 

Operationalising  ICT  Literacy 

The  net  result  of  the  incorporation  of  ICT  literacy  into  the  realm  of  generic  skills  has  seen  a move  away  from  its 
definition  as  a discrete  series  of  a skills  and  attitude  and  knowledge  components  to  a more  competency-based 
description.  ICT  literacy  is  increasingly  being  seen  as  a capacity  for  purposeful  and  effective  use  of  ICT 
technologies  in  one’s  own  setting  and  this  creates  different  needs  for  different  people.  In  the  context  of  schooling 
and  education,  it  is  now  recognised  that  ICT  literacy  describes  a range  of  personal  competencies  that  are  in  many 
cases  distinct  from  ICT  components  specific  to  the  needs  of  individual  subject  disciplines  and  domains.  It  is 
interesting  to  note  that  there  is  now  a perception  of  ICT  literacy  specific  to  being  a successful  student  and  across 
all  sectors  of  education,  discrete  and  appropriate  ICT  skills  are  being  suggested. 

For  example,  Collis  (1999)  argues  that  post-compulsory  students  today  need,  at  a minimum,  skills  and 
competency  in  the  use  of  appropriate  presentation  systems  such  as  word  processors,  electronic  display  systems. 
Students  need  to  be  able  to  use  e-mail  to  send  and  receive  communications  and  need  to  be  able  to  use  Web 
browsers  to  access  and  retrieve  information  from  this  source.  When  one  deconstructs  exactly  what  levels  of 
computer  skills  and  use  underpin  these  forms  of  competency,  this  level  of  specification  necessarily  includes  an 
array  of  associated  skills  including  competence,  confidence  and  efficiency  in  the  use  of  hardware,  software  and 
operating  systems  associated  with  personal  computers,  networked  computers  and  the  Internet. 

Yet  another  component  of  the  ICT  literacy  issue  which  is  gaining  prominence  among  teachers  and  educators  is 
the  impact  of  these  technologies  on  the  basic  forms  of  reading  and  writing  literacy.  There  are  a number  of  writers 
who  have  demonstrated  that  computers  and  the  Internet  are  changing  the  ways  in  which  many  people 
communicate  through  the  written  word  (eg.  Warschauer,  1999;  Orlikowski  & Yates,  1994).  The  Internet  in 
particular  is  changing  the  way  many  people  write  and  the  screen-based  technologies  are  impacting  on  the  way 
many  people  read.  Researchers  are  arguing  that  given  the  rapid  uptake  of  ICT  technologies,  these  changes  will 
soon  need  to  be  recognised  in  the  way  literacy  is  taught  and  developed  in  schools  and  universities  (eg.  Bruce  and 
Candy,  1994). 

The  use  of  Internet  and  communication  technologies  in  the  delivery  of  both  information  and  instruction  for 
learners  has  seen  in  recent  years  a significant  lifting  of  the  perceived  baseline  level  of  ICT  literacy  deemed 
necessary  for  post-compulsory  students.  Trends  suggest  that  this  list  of  competencies  will  likely  grow  in  the  near 
future  as  ICT  technologies  continue  to  develop  and  have  application  to  teaching  and  learning.  The  increasing  use 
of  computers  in  all  aspects  of  life  seems  set  to  bring  about  even  more  changes  to  our  perceptions  of  what  actually 
constitutes  ICT  literacy.  The  research  suggests  that  different  disciplines  within  education  are  now  recognising  the 
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impact  of  ICT  on  their  core  business  and  are  creating  strategies  to  deal  with  it,  for  example,  linguists  and  language 
teachers  and 

teachers  of  writing  and  reading  literacy,  information  skills,  and  teachers  of  training(eg.  Victorian  University  of 
Technology,  1998). 

With  growing  use  of  ICT  in  the  workplace,  demands  for  literacy  have  now  heightened  with  growing  emphasis  on 
reading,  comprehension  and  information  literacy,  and  on  the  ability  to  gather  information  from  multiple  sources 
and  to  use  the  information  meaningfully  and  critically.  This  is  leading  (in  many  low  and  middle-level  jobs),  to  a 
more  complex  understanding  of  literacy  that  includes  making  critical  judgements  about  the  accuracy,  current 
relevance,  and  unexpressed  messages  implicit  in  information  (eg.  Mikulecky  & Kirkley,  1998). 

The  concept  of  ‘just  in  time  learning’  is  diminishing  in  many  instances  the  need  for  the  general  resources 
employees  must  bring  to  an  occupation.  Technology  can  now  manage  many  of  the  operations  in  a job  by  sending 
exactly  the  right  information  to  exactly  the  right  terminal  at  the  right  time  (eg.,  Talbot,  1999;  Flew,  1998;  Tomei, 
1999).  The  scenario  appears  to  exist  for  future  workers  to  go  to  a workplace  without  the  specific  skills  to  operate 
in  that  particular  environment  but  with  the  technology  skills  and  abilities  to  rapidly  adapt  to  the  potentially 
changing  requirements  of  the  job.  The  rapidly  changing  nature  of  the  workplace  can  mean  that  given  the 
appropriate  information  the  worker  has  the  skills  needed  to  seek  out  information,  as  it  required. 

Achieving  Computer  Literacy 

There  are  formal  programs  now  in  many  countries  to  encourage  and  support  the  development  of  IT  skills  in 
schools.  The  results  of  these  programs  and  activities  ensure  that  many  students  entering  post-compulsory 
education  have  some  level  of  ICT  literacy  although  all  too  often,  students  demonstrate  largely  varying  levels  of 
skills  and  knowledge.  There  is  evidence  that  many  post-compulsory  institutions  are  moving  positively  to  address 
the  need  for  ICT  literacy  among  students.  Some  require  prospective  students  to  undertake  ICT  literacy  tests  and 
to  take  bridging  courses  prior  to  commencing  their  studies.  A survey  by  the  American  Association  of  State 
Colleges  and  Universities  in  1995  reported  that  22%  of  American  State  Colleges  and  Universities  require 
computer  literacy  of  their  students  (Davis,  1999).  Many  of  these  have  initiated  computer  survey  courses  to  redress 
the  lack  which  was  reported  at  this  time.  In  addition  a number  of  universities  worldwide  require  that  students 
purchase  their  own  computers  to  use  in  their  learning  programs  (eg.  Davis,  1 999). 

Increasingly  students  at  all  levels  of  education  are  gaining  ICT  skills  at  home  as  home  ownership  of  PCs  increases 
to  almost  saturation  levels.  In  Australia,  for  example  a recent  survey  of  student  Internet  users  in  school,  vocational 
education  and  universities,  revealed  that  for  96%  of  them,  the  primary  point  of  access  for  students  was  through  a 
home  computer.  Sixty  seven  percent  of  those  surveyed  indicated  that  the  educational  institution  was  their 
secondary  point  of  access  (White,  1999).  In  Australia  in  1999,  96%  of  the  educational  institutions  had  access  to 
on-line  services.  Recent  studies  among  school  students  have  clearly  indicated  the  strong  influence  of  home  use 
on  skills  development.  The  students  with  the  highest  levels  of  ICT  literacy  are  frequently  those  with  home  access 
to  computers  and  Internet  technologies. 

Achieving  ICT  skills  going  forward  into  the  next  century  is  becoming  increasingly  complicated  as  education 
authorities  move  toward  embedding  a whole  raft  of  ICT  skills  into  the  general  curriculum.  Models  of  different 
approaches  are: 

• to  either  include  intensive  courses  in  ICT  at  the  commencement  of  enrolment  or  at  periods  throughout  an 
individuals  life 

• to  present  individual  modules  which  occur  at  the  precise  time  when  the  skills  and  tools  are  required  by  the 
learner  to  satisfy  other  learning  objectives. 

Tomei  (1999)  describes  these  models  with  respect  to  adult  learners  who  were  reported  to  prefer  the  second  model. 
This  ‘infusion’  approach  differs  from  the  more  traditional  approach  of  delivering  a single  intensive  course  at  a less 
appropriate  but  probably  at  more  convenient  times,  the  ‘concentration’  approach. 

Whilst  both  approaches  have  benefits  and  disadvantages  educational  institutions  at  all  levels  may  find  that  a 
hybrid  of  these  two  models  may  be  desirable.  Certainly  there  are  often  subject  specific  components  of  ICT  which 
have  to  be  dealt  with  in  an  ‘infusion  context  as  they  are  required  whereas  more  generic  ICT  skills  outcomes  can  be 
achieved  by  the  ‘concentration’  approach.  But  in  the  long  run,  most  appropriate  forms  of  skills  development  are 
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those  where  the  skills  are  practised  and  developed  from  relevant  experience.  Karsten  & Roth  (1998)  report 
results  from  their  study  that  suggest  that  it  is  the  relevance  rather  than  the  quantity  of  computer  experience  that 
students  bring  to  class  that  is  the  most  predictive  of  performance.  The  results  also  suggested  that  although  a wide 
variety  of  computer  experiences  enhance  student  perceptions  of  their  computer  competencies,  only  those 
experiences  that  develop  or  enhance  the  specific  computer  skills  defined  to  comprise  computer  literacy  in  a 
particular  context  are  likely  to  have  an  impact  on  computer-dependent  course  performance. 

There  is  a growing  awareness  in  all  circles  of  education  of  the  need  for  context  and  relevance  in  the  teaching  and 
learning  processes  especially  those  associated  with  ICT  literacy.  For  both  adolescents  and  adults,  the  most  likely 
path  to  success  lies  in  integrating  the  new  technologies  into  the  activities  of  everyday  learning.  Computer 
simulations,  tutorial  programs,  the  Internet,  and  information-processing  and  electronic  communication  tools  can 
be  used  to  address  these  learning  goals.  These  goals  are  (a)  learning  to  think  critically  while  solving  problems,  (b) 
becoming  familiar  with  the  use  of  technology  tools  for  processing  information,  (c)  expanding  the  breadth  of 
materials  encountered  by  learners,  and  (d)  developing  habits  of  mind  suitable  for  lifelong  learning  (Tomei,  1999). 

ICT  In  Post-Compulsory  Education 

For  the  majority  of  students  post-compulsory  education  sits  between  schools  and  workplace.  It  is  in  this  setting 
that  many  believe  ICT  literacy  is  most  importantly  dealt  with.  Studies  have  revealed  differences  in  ICT  literacy 
between  minority  groups  and  discipline  areas  (eg.  Smith  & Necessary,  1996).  Institutions  with  active  social 
Justice  views  and  policies  are  now  stipulating  benchmark  levels  of  generic  graduate  skills  for  the  workplace 
among  which  ICT  literacy  in  its  various  forms  can  be  found.  In  most  studies  where  ICT  literacy  has  been 
explored,  an  important  indicator  of  students’  skills  development  has  always  been  the  extent  to  which  students  have 
been  exposed  to  the  new  technologies  in  their  courses  and  programs.  And  research  in  this  field  has  always 
demonstrated  that  this  is  heavily  influenced  by  the  ICT  literacy  of  the  teaching  staff  (eg.  Carr,  1998).  Students 
who  learn  in  technology-enriched  learning  environments  develop  practical  experience  in  the  use  of  ICT 
technologies  as  a matter  of  course  (eg.  Davis,  1999).  This  practice  becomes  problematic  when  it  is  common, 
students  across  education  sectors  to  frequently  demonstrate  higher  levels  of  ICT  literacy  than  their  teachers. 
Consequently,  many  writers  now  express  the  view  that  an  efficient  way  to  improve  the  ICT  literacy  levels  of 
students  and  school  leavers  is  to  firstly  address  the  ICT  literacy  levels  of  the  teaching  staff. 

It  is  clear  that  the  ICT  literacy  levels  of  students  is  very  much  influenced  by  the  level  of  ICT  use  in  their 
educational  programs  and  this  in  turn  is  influenced  by  the  degree  of  confidence  and  competence  of  their  teachers 
to  appropriately  integrate  ICT  applications  within  learning  experiences.  Across  all  sectors  large  differences  exist 
in  the  use  of  IT  in  educational  programs.  The  use  of  ICT  technologies  has  long  been  seen  (and  proven)  as  a way 
to  create  enhanced  learning  outcomes  in  subject  domains  (Davis,  1999).  Its  capacity  to  provide  important  and 
useful  generic  skills  for  students’  working  and  private  lives  is  yet  another  reason  supporting  ICT  technology  use 
as  a teaching  and  learning  tool.  The  use  of  the  ICT  technologies  in  teaching  provides  a means  to  create 
meaningful  learning  experiences  which  can  develop  a range  of  generic  and  transferable  skills,  among  which  are 
those  associated  with  ICT  literacy  (eg.  Oliver  & McLoughlin,  1 999). 

Summary  And  Conclusions 

A reading  of  the  extant  literature  in  relation  to  computer  and  ICT  literacy  reveals  that  this  topic  is  still  an 
important  issue  for  many  stakeholders  in  the  education  process  but  that  its  scope  is  often  underestimated  by  policy 
makers  and  education  and  training  providers.  As  a consequence  of  the  burgeoning  use  of  the  Internet  and  Web, 
ICT  technologies  are  becoming  integral  parts  of  all  facets  of  life,  from  education  to  work.  We  are  seeing 
governments  and  industry  pushing  for  graduates  to  demonstrate  appropriate  skills  and  knowledge  in  this  area  to 
enable  them  to  retain  and  push  their  competitive  edge  in  commerce  and  trade.  And  we  are  seeing  learning 
enhanced  through  applications  of  ICT  technologies  as  information  and  learning  tools.  We  are  seeing  new  ways  of 
learning  and  new  ways  of  communicating  through  these  technologies.  Among  all  this,  however,  there  are 
examples  of  vastly  differing  levels  of  ICT  literacy  among  students  and  graduates. 

It  is  clear  that  ICT  literacy  is  most  effectively  developed  through  relevant  and  contextual  use  of  ICT  technologies. 
Students  with  home  computers  regularly  show  higher  levels  of  ICT  literacies  than  their  counterparts  whose  only 
access  and  use  is  through  school  and  education.  The  key  to  improving  ICT  literacy  levels  would  appear  to  be 
through  an  increased  and  targeted  use  of  technologies  in  teaching  and  learning  programs  appropriately  modelled 
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by  teachers.  Impediments  to  this  are  the  levels  of  available  infrastructure  and  the  capacity  of  teachers  to  make 
meaningful  use  of  the  technologies  in  their  programs.  There  is  a growing  body  of  evidence  which  suggests  that 
the  rate  determining  step  in  moving  along  the  path  of  ICT  literacy  is  the  general  low  levels  of  ICT  usage  in 
educational  courses  brought  about  by  low  levels  of  ICT  literacy  among  teachers  in  general.  The  way  ahead  in  all 
of  this  is  clearly  through  the  promotion  of  ICT  literacy  among  teaching  staff  and  by  immersing  students  in 
technology  rich  and  stimulating  learning  environments  where  information  and  computer  literacy  is  both  a learning 
enabler  and  a learning  outcome. 
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Abstract:  This  paper  considers  new  ^proaches  to  assessment  in  the  online  environment.  Increased 
use  of  computer-mediated  communications  for  teaching  and  learning  is  affecting  assessment 
strategies  across  disciplines.  Initial  investigations  are  equivocal  and  point  to  the  need  for  more 
questions  regarding  whether  assessment  of  online  interaction  in  a variety  of  formats  is  helping  or 
hindering  the  goals  of  educators  and  learners.  Data  was  collected  in  1998  via  interview  and  email 
questionnaire  to  discover  the  goals  of  educators  in  their  approaches  to  assessing  online  interaction. 
Responses  were  received  from  nine  educators  across  four  continents  - Australia,  South  Africa, 
Canada  and  the  United  Kingdom.  Initial  results  demonstrate  that  the  conditions  for  effective  use  of 
assessable  online  interaction  rely  upon  matching  educators*  goals  with  their  ^proaches  to 
assessment.  Questions  around  student  responses  to  assessable  online  interaction  were  explored 
through  the  literature,  yet  further  research  is  needed  for  a fuller  picture. 


Background 

Over  the  past  decade,  discussions  have  proliferated  on  the  educational  use  and  effectiveness  of  computer-based 
technologies  such  as  the  World  Wide  Web  and  email.  Most  questions  have  focused  on  the  nature  and  purpose  of 
online  communications;  possibilities  of  this  medium  for  education  in  general  and  learning  outcomes  in  particular; 
opportunities  for  development  of  new  teaching  and  learning  strategies;  and  implications  for  changing  roles  of 
educators,  learners,  experts  and  novices. 

Broad-ranging  as  these  questions  are,  we  can  take  some  heart  from  recent  reports  on  the  effectiveness  of  online 
strategies  for  teaching  and  learning  and  the  concomitant  staff  development  responses  (e.g.  McNaught,  1995;  Allen 
and  Otto,  1996;  Ellis,  O’Reilly  and  Debreceny,  1998;  Sherry,  1998).  In  regards  to  assessment,  the  online 
environment  has  promised  to  bring  greater  support  to  learners  and  educators  alike  through  immediacy  of  feedback 
and  availability  of  a collaborative  learning  community  (Jonassen,  1995;  Parry  and  Dunn,  1999;  O’Reilly  and 
Morgan,  1999). 

When  used  effectively,  online  interaction  has  the  potential  to  build  communities  of  learners,  foster  the  development 
of  meaningful  collaboration  and  support  deeper  levels  of  learning  (Mason,  1990;  Harasim,  1992;  Yeoman,  1996; 
Holt  el  al,  1998).  Coupled  with  assessment,  online  interaction  is  now  being  shown  to  be  affecting  individual  student 
approaches  to  learning,  students’  motivation  to  collaborate  and  provide  mutual  support  to  peers,  as  well  as 
influencing  both  group  and  individual  learning  outcomes  (Rimmington,  1998;  Thorpe,  1998).  For  example.  Parry 
and  Dunn  (1999)  found  that: 

...while  collaboration  can  foster  “deep”  or  “meaning”  approaches  to  learning,  it  is  important  that 
meaning  approaches  to  learning  tasks  are  valued  in  the  assessment  regime;  students  are  unlikely 
to  invest  much  time  and  effort  in  any  activity  that  is  not  rewarded  by  an  assessment  grade,  and 
this  applies  equally  to  learning  ‘how  to  learn’  strategies  [in  the  online  environment]. 


Methodology 

Interviews  were  conducted  in  early  1998  with  educational  designers  across  seven  Universities  in  Australia  to 
determine  the  location  and  activities  of  educators  who  were  experimenting  with  dialogue  and  assessment  in  the 
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online  environment.  Similar  questions  were  posted  to  four  international  email  discussion  lists  seeking  introductions 
to  educators  identified  as  innovative  in  their  adoption  of  the  online  environment  for  student  assessment. 
Approximately  30  contacts  were  made,  though  several  of  these  proved  to  be  ‘blind  alleys’,  where  the  online 
interaction  of  learners  was  not  ultimately  assessed.  Through  this  ‘snowballing’  method  of  contacts  (Roberts  and 
Taylor,  1 998),  a small  sample  of  educators  reported  to  be  using  online  interaction  as  a component  of  assessment  was 
identified  and  surveyed  by  email  questionnaire. 

Nine  educators  from  a total  of  four  continents  provided  responses  - six  from  Australia,  one  each  from  South  Africa, 
Canada  and  the  United  Kingdom.  The  results  of  these  surveys  were  processed  according  to  a mixture  of  qualitative 
and  quantitative  approaches  to  the  data.  Responses  to  survey  questions  were  viewed  by  frequency  and  specific 
comments  were  collated  with  the  help  of  computer  software  to  assist  in  the  sorting  processes.  Making  sense  of  the 
responses  on  email  questionnaire  — the  verbal  categorical  data  — was  the  crux  of  the  task,  since  each  of  the  responses 
were  in  respect  to  different  discipline  areas  and  reflected  somewhat  different  objectives  and  strategies. 

The  principle  question  for  investigation  concerned  the  ways  in  which  technologies  helped  or  hindered  the  meeting  of 
educators’  goals.  Further  investigations  are  needed  to  complete  the  picture  from  students’  perspectives  as  to  whether 
assessment  of  online  interaction  is  seen  to  help  or  hinder  their  own  goals  and  learning  outcomes. 


Features  of  online  interaction  and  assessment 

Since  commencement  of  this  project,  adoption  of  online  technologies  in  teaching  and  learning  has  grown 
exponentially.  Although  the  most  common  uses  of  online  technologies  are  for  communication  and  assignment 
submission  via  email  or  web-pages,  there  is  an  increasing  adoption  of  structured  online  assessment  processes.  These 
include  group  discussion  to  support  individual  assignment  preparation,  collaboration  on  a common  project  or 
presentation,  peer-review  and  self-review,  and  problem-based  approaches  to  cases,  scenarios  and  consultancies,  to 
name  just  a few. 

This  paper  considers  only  those  features  exemplified  by  respondents  to  the  questionnaire  and  the  resultant  roles 
emerging  for  both  educators  and  learners.  Before  moving  onto  the  formal  results  section,  let’s  take  a brief  look  at  the 
individual  cases  reported  and  their  unique  features. 

Teaching  and  Learning  Strategies  for  Interaction 

(a)  Group  support  for  individual  product  - both  formal  and  informal  structures  can  provide  learners  with 
opportunities  for  discussion  without  full  dependence  on  each  other  for  a grade.  For  example,  listserv  discussions  can 
provide  collegial  practice  in  developing  professional  discourse  in  a supported  environment  such  as  in 
“Discrimination  and  the  Law”  (Macquarie  University,  Australia).  In  this  case  students  are  required  to  (1)  submit 
weekly  critiques  of  a specified  article  to  the  group,  drawing  out  the  legal  argument  (2)  participate  in  informal 
discussion  with  peers  concerning  these  critiques,  the  evidence  of  legal  argument  or  assumptions,  and  (3)  continue  to 
actively  participate  in  online  interaction. 

(b)  Collaboration  for  common  product  - students  are  asked  to  work  together  in  teams  to  coordinate  development  of 
a conference  and  website  (“Science  and  Cornmunication”  University  of  Melbourne,  Australia)  or  a professional  co- 
authored paper  for  publication  (“Environmental  Psychology”,  University  of  Pretoria,  South  Africa;  “Nursing 
Honours”,  Curtin  University  of  Technology,  Australia).  Development  of  skills  includes  teamwork,  collaboration, 
negotiation,  presentation,  a shared  sense  of  achievement  and  academic  standing  in  the  field. 

The  products  in  these  examples  each  demonstrate  elements  of  professional  standards.  Specifically,  “Science  and 
Communication”  students  experience  the  organisation  and  participation  in  a conference  of  professional  standing 
where  discipline  based  experts  participate  as  keynote  speakers,  and  these  first  year  students  present  their  own  work 
to  a mixed  audience  of  peers,  staff,  experts  in  the  field  and  family  supporters.  “Environmental  Psychology”  students 
publish  a co-authored  paper  of  suitable  quality  to  submit  for  refereeing  by  professional  journals.  “Nursing  Honours” 
students  publish  a refereed  online  nursing  journal  for  access  by  peers  and  professionals  alike. 
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(c)  Peer  review  plus  self-review  - having  worked  together  towards  a common  goal,  students  may  then  be  asked  to 
assess  their  own  progress  and  to  evaluate  the  contribution  of  their  peers.  This  may  be  an  assessment  of  group 
processes  or  the  final  product,  but  must  be  according  to  given  or  previously-negotiated  criteria. 

The  projects  for  “Science  and  Communication”  University  of  Melbourne;  “Environmental  Psychology”,  University 
of  Pretoria;  and  “Nursing  Honours”,  Curtin  University  of  Technology  have  all  been  briefly  described  above. 
Students  in  these  subjects  follow  up  their  project  work  with  specific  activities  which  lead  to  self  and  peer 
assessment  In  another  example  from  Curtin  University  of  Technology,  “Foundations  and  Issues  in  Science, 
Mathematics  and  Technology  Education”,  students  are  acknowledged  as  co-professionals.  As  qualified  teachers 
themselves,  students  have  a rich  resource  in  their  own  daily  classroom  experiences  from  which  to  draw  upon  in 
reflective  exercises.  The  choice  of  a social  constructivist  approach  to  learning  in  this  subject  therefore  readily  lends 
itself  to  critical  reflection,  and  assessment  includes  the  journal-based  input  of  students  regarding  their  own 
leaming-in-action  as  well  as  that  of  their  peers. 

(d)  Debate  - students  may  form  teams  and  through  negotiation  select  a topic  and  their  position  on  the  debating  team. 
Following  a given  preparation  period  where  work  may  be  collegial,  a debate  can  take  place  according  to  established 
protocols.  A very  sophisticated  example  of  online  debate  is  the  international  collaboration  called  “Communication 
and  the  Media”  conducted  between  University  of  South  Australia;  Governors  State  University,  Chicago;  and 
University  of  Technology,  Sydney. 

In  this  instance,  the  debate  is  a particularly  authentic  genre  of  expression  in  such  a subject.  For  students  to  formally 
research  and  develop  a more-or-less  provocative  position,  and  then  to  challenge  the  arguments  of  members  on 
opposite  sides,  is  highly  applicable  in  the  vocational  contexts  of  a range  of  media.  Debating  topics  concerned  with 
globalisation  and  the  media,  and  to  further  be  engaging  in  debate  across  cultural  differences  significantly  enhanced 
students’  learning,  sometimes  in  the  most  poignant  experiential  ways. 

New  Roles 

New  roles  for  educators  which  can  be  gleaned  from  these  examples  include  facilitation,  moderation  and  the 
orchestration  of  a distributed  community  of  learners.  We  have  already  seen  how  the  educators  involved  with 
“Communication  and  Media”,  UniSA/UTS/GSU;  “Science  and  Communication”  University  of  Melbourne;  and 
“Discrimination  and  the  Law”,  Macquarie  University,  had  the  work  of  coordinating  and  facilitating  students’ 
activities  in  novel,  applied  and  appropriate  ways. 

Another  example  is  the  subject  “Designing  Training  for  the  Internet”  from  Open  University,  UK.  Here  students  do 
just  that,  they  are  required  to  develop  training  materials  in  the  web  environment  for  their  own  organisation’s 
purposes,  using  templates  and  asynchronous  text  based  support  from  teaching  staff  The  resulting  outcomes  from 
this  subject  not  only  provide  students  with  useful  resources  which  they  may  implement  in  their  workplace,  but  also 
provide  the  educators  with  new  resources  and  a range  of  authentic  approaches  to  build  upon  in  future  iterations  of 
delivery  and  for  future  cohorts  of  students. 

Additional  online  assessment  methods  illustrated  in  this  study  include  the  use  of  automated  quizzes  and  exams 
(“Business  Finance”,  University  of  Technology,  Sydney;  and  “Microeconomics”,  University  of  New  Brunswick, 
Canada).  In  these  cases,  the  interaction  demanded  of  students  while  online  is  their  response  to  assessable  questions 
presented  within  formative  and  summative  activities.  The  new  role  of  the  instructor  in  these  instances  includes 
development  of  assessment  tasks  which  support  knowledge  testing  and  reduce  the  likelihood  of,  or  desire  for 
cheating. 

Learners  now  also  find  themselves  developing  a range  of  new  skills  simultaneously  with  their  development  of 
subject  specific  knowledge.  These  skills  include  - research  and  critical  review  of  resources  found  online;  experience 
in  the  formation  of  virtual  groups,  where  dependence  on  non-verbal  cues  is  unreliable  or  even  impossible; 
publishing  web  pages;  collaboration  in  either  synchronous  or  asynchronous  modes  which  take  place  across  times 
zones  and  cultural  contexts;  and  the  review  of  peers  which  though  anonymous  must  be  ethical  and  constructive. 
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Results 


Results  show  effective  use  of  assessable  online  interaction  where  such  interaction  is  an  inherent  objective  and  thus 
supports  the  educators’  goals.  Examples  include  areas  which  value  development  of  communication  skills, 
collaborative  activities  such  as  problem-solving  and  peer  review.  Figure  1 summarises  (a)  the  technologies  used  in 
the  nine  case  studies  to  facilitate  interaction  and  (b)  the  educators’  reflections  on  the  impact  of  these  technologies 
upon  their  teaching  strategies  and  their  goals  as  educators.  It  can  be  seen  from  this  summary  that  in  many  cases 
educators  are  using  both  the  capacity  for  dialogue  and  speed  of  the  technology  as  well  as  the  capability  of  students 
themselves  to  engage  and  support  group  and  individual  student  learning. 

Questions  around  student  responses  to  assessable  online  interaction  were  explored  through  the  literature,  yet  further 
research  is  needed  relating  students’  perspectives  of  assessable  online  interaction  to  their  own  learning  goals.  A 
small  glimpse  of  the  student  perspective  is  revealed  in  Figure  1 through  feedback  quoted  by  educators. 

Online  assessment  approaches  were  identified  as  hindrances  where  the  technology  failed  to  be  transparent  and 
presented  obstacles  to  learning.  In  some  cases,  for  example  the  international  debate  between  USA  and  Australia, 
breakdowns  in  technology  were  treated  as  inevitable  and  students  were  not  penalised  nor  ultimately  prevented  from 
carrying  out  their  assessment  activities.  Rather,  students’  motivation  to  collaborate  and  provide  mutual  support  in 
the  face  of  technical  problems  revealed  approaches  to  learning  which  move  away  from  an  entirely  competitive 
culture. 

Hindrances  were  identified  in  the  form  of  escalating  workloads  arising  from  the  very  appealing  opportunities  for 
dialogue.  These  hindrances  were  not  however  a fundamental  drawback  of  the  educators’  goals  but  point  to  the  need 
for  careful  design  of  teamwork,  discussion  and  support  for  collaboration.  Of  note  is  the  problem  of  keeping  up  with 
the  changing  environment  of  educational  technology,  identified  when  the  study  was  conducted  in  1998  and  which 
continues  today.  Students’  objections  to  mandatory  dialogue  and  dependence  upon  fellow  learners  for  assessable 
activities  further  challenges  notions  of  flexibility  and  independence  boasted  by  traditional  print-based  distance 
education. 


Conclusions 

Research  such  as  this,  which  principally  considers  educators’  goals,  must  further  consider  learners’  goals. 
Hindrances  identified  in  this  study  suggest  appropriate  educational  design  for  teaching  with  reliable,  robust 
technologies  and  through  purposeful  dialogue  will  ensure  rich  and  rewarding  teaching  and  learning  experiences. 

The  study  reinforces  the  work  of  John  Biggs  (1996)  in  identifying  the  need  for  a constructive  alignment  between 
objectives,  teaching  and  learning  strategies,  and  assessment  as  being  fundamental  to  successful  online  teaching  and 
assessment.  Such  a matching  of  goals  with  assessment  is  not  only  critical  from  the  educator’s  perspective  but  is  also 
essential  to  an  effective  learning  experience  by  reflecting  the  process  of  learning  in  the  strategies  for  assessing  the 
outcomes  of  learning. 

Used  with  care  and  consideration,  it  seems  that  assessment  of  online  interaction  can  enhance  group  and  individual 
learning  outcomes  and  precipitate  motivations. 


“Interactive”  Technologies 
Used 

Helping  or  Hindering  Educators^  Goals 

1 . • Computer  conference  with 
tutors  and  trainees 

• Email 

• Javascript  self-tests 

Technologies  defined  the  educators’  goals. 

2.  • Listsever 

It  appears  as  though  goals  were  achieved,  students  benefited 

• Personal  email  contact 

• Individual  feedback  on 
assignments 

significantly  from  this  course,  switching  from  being  taught  to 
becoming  independent  learners. 

3.  • Notice  board  discussion 

• Small  group  discussion 

• Formal  debate 

Technology  helped  a great  deal  in  actual i sing  much  of  the  content 
e.g,  cultural  diversity  (Australian  students  interacted  with  US 
students);  ‘new  technologies’  (subject  considers  impact  of  new 
technologies  on  the  nature  of  communication). 

Only  hindered  by  limits  of  the  technology  [organisational  policy] 
e.g,  no  provision  made  by  IT  dept,  for  real  time  chat. 

4.  • Asynchronous  conference 

• Journal  for  submissions 

• Email 

WWW  facilitated  learning  about  information  literacy  in  addition  to 
enhancing  learning  in  regards  to  objectives  of  the  subject. 

5.  • MCQ  self-assessment 

• Timed  online  exams 

• Email  and  group  conference 

Students  appreciate  one-to-one  responses  to  email  questions. 
On-screen  graphics  help  illustrate  key  concepts  and  elucidate  verbal 
arguments.  Auto  grading  for  tests  and  exams  ensure  immediate 
feedback,  fundamental  to  successful  learning. 

6,  • One-to-one  and  group  email 

• Collaborative  web 
publishing 

• Database  searches  on 
CDROM 

We  are  moving  quickly  each  year  and  the  learning  curve  for 
revisions  is  always  a problem.  For  example,  electronic  conferencing 
[systems]  were  at  our  disposal  but  simply  no  time  to  learn  to  use 
[them]  effectively.  Battle  to  find  productivity  tools  e.g.  drag-and 
drop  web  authoring  packages  that  are  affordable,  always  a 
dilemma.  Extra  instruction  is  needed  to  overcome  the  deficiencies 
of  what  we  can  afford 

7 • Email  (tutor-student, 

student-student) 

• Discussion  Room  or  Notice 
Board 

• Course  Materials  on  WWW 

Our  goals  are  about  social  constructivism  and  connected  learning, 
so  the  older  forms  of  distance  education  were  very  frustrating  for 
us.  It  is  so  much  more  powerful  for  our  (practicing  teacher)  students 
to  be  able  to  share  ideas  and  experiences,  tell  ‘war  stories’  from 
their  school  context  and  know  one  another  (and  us)  as  struggling 
co-professionals. 

8.  • Online  Q&A  discussion  with 
24  hour  turnaround  from  staff 

• Self-assessment  quizzes, 
auto  graded 

• Package  of  CBL  scenarios 

[Online  technologies]  help  to: 

- deliver  opportunities  for  conversation 

arrange  relevant  learning,  practical  and  non-cheatable 
assessment  tasks 

- reduce  time  spent  answering  repetitive  questions  and  providing 
feedback 

- communicate  in  a caring  and  consistent  manner  to  a very  large 
and  diverse  student  population  (~  1100  students) 

9.  • Assessable  critique  of  article 
by  each  student 

• Assessable  informal 
discussion  among  peers 

• Constant  discussion  with 
tutors  on  the  listserv 

The  combination  of  printed  study  materials  and  listserv  discussions 
enabled  the  students  to  learn  with  each  other  and  from  each  other. 
Some  students  found  the  weekly  contributions  a bit  much  but  it  also 
meant  that  they  studied  regularly  rather  than  in  bursts. 

Figure  1:  Extracts  from  questionnaire  demonstrating  technologies  used 
and  their  impact  upon  educator’s  goals 


References 

Allen,  BS  and  Otto,  RG  1996  ‘Media  as  lived  environments:  The  ecological  psychology  of  educational  technology’. 
In  DH  Jonassen  (edj  Handbook  of  research  for  educational  communications  and  technology  New  York:  Simon 
and  Schuster  Macmillan,  pp  199-225 


878 


Biggs,  J 1996,  Assessing  Learning  Quality:  reconciling  institutional,  staff  and  educational  demands’  Assessment  and 
Evaluation  in  Higher  Education  Vol  21,  No  1,  pp5~15. 

Ellis,  A,  O'Reilly,  M & Debreceny,  R 1998  ‘Staff  Development  Responses  to  the  Demand  for  Online  Teaching  and 
Learning’.  In  ASCILITE98  Woolongong:  Woolongong  University  Press,  pp  191-201 

Harisim,  L (1992)  Foreword,  in  Paulsen,  M F From  Bulletin  Boards  to  Electronic  Universities:  Distance  Education, 
Computer-Mediated  Communication  and  Online  Education  Pennsylvania:  American  Centre  for  the  Study  of 
Distance  Education 

Holt,  ME,  Rees,  F,  Swenson,  JD,  Klieber,  PB  1998  ‘Evolution  of  Evaluations  for  Critical,  reflective  and 
Deliberative  Discourse:  National  Issues  Forum  On-Line’ 

http://www.coe.uga.edu/adulted/faculty/mholt/paperl.html 

Jonassen,  DH  1995  ‘Supporting  communities  of  learners  with  technology:  A vision  for  integrating  technology  with 
learning  in  schools’  Educational  Technology  (July-August)  pp  60-63 

Mason,  R 1990  ‘Computer  Conferencing  in  Distance  Education’.  In  Bates,  A Media  and  Technology  in  European 
Distance  Education  Milton  Keynes:  European  Association  of  Distance  Teaching  Universities,  pp.  221-226 

McNaught,  C 1995  ‘Overcoming  Fear  of  the  Unknown!  Staff  Development  Issues  for  Promoting  the  Use  of 
Computers  in  Learning  in  Tertiary  Education’.  In  JM  Pearce  and  A Ellis  (eds)  ASCI  LITE  95  Proceedings  of  the 
1 2 th  Annual  Conference  of  the  Australian  Society  for  Computer  in  Learning  in  Tertiary  Education,  University  of 
Melbourne:  Park vi lie,  pp4 10-4 16 

O’Reilly,  M and  Morgan,  C 1999  ‘Online  Assessment:  Creating  Communities  and  Opportunities’.  In  S,  Brown,  J, 
Bull  and  P,  Race  (1999)  Computer-Assisted  Assessment  London:  SEDA  pp  149-161 

Parry,  S and  Dunn,  L (in  press)  ‘Benchmarking  as  a Meaning  Approach  to  Learning  in  Online  Settings’  paper 
presented  at  4^*^  Northumbria  Conference  on  Assessment,  UK,  September  1 999 

Rimmington,  G 1998  ‘Some  Useful  Student  Feedback  on  the  Online  Learning  Experience’.  In  ONLINE-ED  20 
March 

http://www.edfac.unimelb.edu.au/online-ed/mailouts/1998/march20.html 

Roberts,  K and  Taylor,  B 1998  Nursing  Research  Processes:  an  Australian  perspective  Melbourne:  Nelson  ITP 

Sherry,  L 1998  Nature  and  Purpose  of  Online  Discourse:  A brief  synthesis  of  current  research  as  related  to  The 
WEB  project  http ://www.cudenver.edu/^lsherry /pubs/di alogue.htm 

Thorpe,  M 1998  ‘Assessment  and  ‘third  generation’  distance  education’  Distance  Education  Vol.  19,  No.  2,  pp265- 
286 

Yeoman,  E 1996  cited  in  Haughey,  M and  Anderson,  T 1998  Networked  Learning:  The  Pedagogy  of  the  Internet 

Montreal 


879 


Commercialization  of  Higher  Education 
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This  panel  will  discuss  the  role  of  Internet  advertising  in  higher  education.  Advertising  on  college  and 
university  campuses  has  long  existed  through  sporting  events,  soft  drinks,  magazines,  newspapers  and 
bookstores  and  more.  Yet  with  the  introduction  of  the  Internet  in  recent  years,  the  impact  and  prevalence 
of  ads  on  college  and  university  campuses  has  changed.  Computer  savvy  Generation  Y is  demanding 
schools  provide  costly  Internet  tools  and  resources  and  some  universities  are  struggling  to  find  ways  to 
offer  these  tools  to  stay  competitive.  Collegiate  Web  sites  offering  these  tools  and  resources  are  extending 
themselves  to  Universities  free  of  charge  with  the  condition  that  the  schools  allow  the  sites  to  present  ads. 
Universities  are  now  faced  with  a tough  decision.  Do  they  accept  the  free  resources  Internet  companies  are 
offering  with  advertising  or  do  they  try  to  create  a comparable  Internet  offering  or  their  own? 
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Abstract.  Teleleaming  research  conducted  in  our  research  centre  in  the  last  five  years  has  resulted 
in  a Virtual  Learning  Centre  (VLC)  model  and  an  implementation  for  Web-based  training  called 
Explora.  This  system  is  now  in  operation  at  Tele-universite,  in  three  professional  corporations  and 
in  a company.  Our  VLC  model  focuses  on  the  interaction  spaces  between  five  theoretical  actors: 
the  learner,  the  informer  (content  expert),  the  trainer,  the  manager  and  the  designer.  The  Explora 
implementation  supports  learners  and  the  other  actors  in  their  interactions.  Each  actor  has  a specific 
environment  enabling  him/her  to  manage  his/her  activities,  consult  and  produce  information, 
engage  in  collaboration  and  receive  assistance  from  different  sources.  We  will  finally  present  a 
reengineering  process  that  has  been  applied  to  transform  a traditional  distance  education  course  into 
a web-based  course  within  the  Explora  environment. 


1.  An  Open  Virtual  Campus  Model 

In  our  societies,  individuals  and  organisations  are  coping  with  an  exponential  growth  of  information  and  the 
knowledge  management  challenge.  The  rapidly  evolving  availability  of  multimedia  telecommunication  is  an  answer 
to  this  challenge  but  it  has  to  be  mastered,  to  mature,  not  mainly  technologically  but  in  the  way  people  use  it,  in  the 
way  it  is  integrated  to  practice,  conceptions  and  attitudes. 

Behind  terms  like  “distance  education’*,  “on-line  learning”,  “teleleaming”  and  “multimedia  training”,  is  a multi- 
facetted  reality  from  which  we  can  identify  many  paradigms.  High-tech  classrooms,  virtual  classrooms  (Hiltz, 
1990),  multimedia  Web-based  training,  on-line  learning  communities  (Harasim,  1990  and  Ricciardi-Rigault  et 
Henri,  1994)  and  electronic  performance  support  systems  (EPSS)  (see  Gery,  1997)  are  all  modalities  that  can  be 
integrated  in  a tmly  open  virtual  campus.  Such  a virtual  campus  will  enable  individuals  and  organisations  to  manage 
transitions  from  the  predominant  classroom  presentation  to  frilly  interactive  teleleaming. 

We  see  a teleleaming  system  as  a society  of  agents,  to  use  Marvin  Minski’s  term,  some  of  them  providing 
information  and  explanations,  others  constmcting  new  information,  still  others  fostering  collaboration  between 
agents  or  providing  assistance  to  the  other  agents  on  content,  pedagogical  process  or  organisation  of  activities. 

A Virtual  Learning  Centre  (VLC)  is  the  central  part  of  a Virtual  Campus  where  instmctional  resources,  learning 
events  as  well  as  technological  and  organisational  infrastructure  are  created  and  assembled  to  forward  the  delivery 
of  teleleaming  systems.  Our  VLC  model  (Paquette,  1997)  emphasises  the  concept  of  a process-based  learning 
scenario  coupled  with  assistance  resources.  Basically,  the  learner  proceeds  to  a scenario,  a network  of  learning 
activities,  using  different  kinds  of  information  resources  to  help  her  achieve  the  tasks  and  produce  some  outcome:  a 
problem  solution  or  new  information  that  can  be  used  in  other  activities.  The  assistance  resources  for  each  task  are 
also  defined  during  design  process.  The  assistance  can  be  distributed  among  many  agents;  trainers  interacting 
through  e-mail  or  teleconferencing,  other  learners,  contextual  help  or  intelligent  advisors. 
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2.  The  Explora  Model:  Actors,  Roles  and  Agents 


We  have  described  elsewhere  (Paquette  et  al,  1993  and  Paquette  et  al,  1997)  how  we  have  built  an  object-oriented 
model  of  a Virtual  Campus.  In  our  Virtual  Learning  Centre  architecture,  we  identify  five  actors,  each  personified  by 
different  persons  or  media  agents. 

The  Learner  transforms  information  into  personal  knowledge.  “ Knowledge  ” means  the  information  that  has  been 
integrated  by  a cognitive  entity  into  its  own  cognitive  system,  in  a situated  context  and  use.  The  learner  achieves 
knowledge  acquisition  and  construction  by  managing  a learning  environment  planned  by  another  actor,  the  designer, 
through  collaboration  with  other  learner  agents,  consulting  and  producing  information,  receiving  assistance  from 
other  actors. 

The  Informer  (the  content  expert)  makes  the  information  available  to  the  learner.  It  may  be  a person  or  a group  of 
persons  presenting  information,  but  also  a book,  a video,  a software  or  any  other  material  or  media.  Conversely,  the 
learner  will  also  produce  information  that  can  be  made  available  to  others  through  an  informer  agent,  as  a result  of 
his/her  production  activities. 

The  Designer  is  the  actor  planning,  adapting  and  sustaining  a teleleaming  system  (TLS)  that  integrates  information 
sources  (human  informers  or  learning  materials),  and  also  self-management,  assistance  and  collaboration  tools  for 
the  other  actors. 

The  Trainer  provides  pedagogical  assistance  by  giving  advice  to  the  learner  about  his  individual  process  and  the 
interactions  that  may  be  useful  to  him  based  on  the  learning  scenarios  defined  by  the  designer. 

Finally,  the  Manager  provides  organisational  assistance  to  the  learner  (and  other  actors)  by  managing  actors  and 
events,  for  example  creating  groups  or  making  tele-services  available  in  order  to  insure  the  success  of  the  learning 
process,  based  on  the  scenarios  defined  by  the  designer. 


Figure  1 - Actors  and  interaction  spaces 
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Figure  1 shows  the  five  theoretical  actors  and  their  interactions  with  the  learner.  In  our  VLC  model,  there  are  five 
interaction  spaces:  management,  consultation,  production,  collaboration  and  assistance,  each  giving  access  to  a 
number  of  resources. 


3.  The  Delivery  System:  the  Hyperguide  and  the  Explora  Environments 

At  delivery  time,  the  learner  and  the  other  actors  interact  within  a computer-based  learning  environment  called  the 
Virtual  Learning  Centre.  The  Explora  implementation  of  such  a system  is  a web-based  server  that  helps  designers 
build  a learning  environment  for  each  actor,  adapted  to  their  role  in  a certain  course  or  teleleaming  event.  Then  the 
other  actors  can  use  their  own  Explora  environment  to  intervene  in  relation  to  the  course  web  site. 

Each  course  can  be  any  web  site  we  called  the  Hyperguide.  This  web  site  describes  the  course  structure,  down  to 
learning  scenarios  and  learning  activities  distributed  into  modules  or  learning  units.  Each  scenario  gives  access  to 
specific  resources  to  be  consulted  or  productions  to  make. 

Figure  2 shows  such  a HyperGuide  for  an  introductory  course  in  artificial  intelligence.  The  main  window  here  is  one 
of  the  Hyperguide  pages  showing  the  scenario  for  module  3.  Each  oval  shape  represents  a learning  activity.  A click 
on  it  brings  the  description  of  the  learning  activity:  goal  and  type  of  the  activity,  expected  duration,  resources  to 
consult,  productions  to  make,  proposed  assignment  steps. 
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Figure  2 . A host  HyperGuide  within  an  Explora  environment 


The  rectangular  shapes  give  access  to  the  input  resources:  a text  to  read,  a video  to  watch,  a CBT  or  a multimedia 
simulation  to  interact  with.  They  also  give  access  to  output  resources  storing  the  learner’s  production  for  future  use 
in  other  activities,  exchange  with  other  learners  or  submission  to  a trainer  for  evaluation. 
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Different  actors  need  different  points  of  view  on  the  host  system  in  each  interaction  space.  For  example,  in  the 
information  space,  a learner  will  need  different  input  resources  such  as  a list  of  related  web  sites,  videos  from  a 
server  or  didactic  software  on  AI  content.  In  the  same  information  space,  a trainer  needs  other  information 
resources:  traces  of  the  learners  activities  for  diagnosis,  information  on  the  group  of  learners  and  on  learner 
productions  and  annotation  tools  to  identify  and  organise  information  for  assistance. 

These  resources  are  made  available  through  an  external  palette  as  shown  in  the  floating  window  of  figure  2.  This  is 
what  we  call  an  Explora  environment.  It  is  an  actor’s  set  of  resources  for  a course  or  program  supported  by  the 
Virtual  learning  centre.  It  groups  resources  into  five  interaction  spaces  (self-management,  information,  production, 
collaboration  and  assistance)  according  to  an  actor’s  role  and  course  specifics.  In  the  example  here,  if  we  open  any 
one  of  the  five  menus  in  the  Explora  window,  we  can  gain  access  to  the  following  resources. 


INTERACTION 

SPACE 

RESOURCE 

TYPE 

DESCRIPTION 

Self-management 

resources 

Personal  Profile 

Java 

Personal  information  accessible  to  all 

Progress  status 

Java 

Bar  graph  displaying  progress  levels 

Calendar  of  events 

Java 

Dates  where  activities  were  looked  upon 

Course  schedule 

HTML 

Gantt  distribution  of  activities 

Evaluation 

HTML 

Suggestions  and  course  evaluation  by  learners 

Information 

resources 

Texts 

HTML 

Access  to  texts  to  be  consulted  or  produced 

Videos 

HTML 

Access  to  video  streaming 

Webography 

HTML 

Access  to  interesting  web  sites 

Search  engines 

HTML 

To  search  for  other  web  sites 

Production 

resources 

CBTs 

EXE 

Triggers  seven  CBTs  illustrating  lA  concepts 

Text  editor 

EXE 

Link  to  a recommended  text  editor 

Knowledge  editor 

EXE 

Link  to  LICEF’S  MOT  knowledge  editor 

Productions  made 

HTML 

Simple  file  transfer  to  trainer  for  production  evaluation 

Collaboration 

resources 

Group  profile 

Java 

Display  of  other  learner’s  progress  and  chat 

E-mail 

EXE 

Link  to  recommended  email  software 

Forums 

HTML 

Asynchronous  teleconferencing  system 

Showcase 

HTML 

Simple  upload/download  to  a server  to  facilitate  the 
exchange  of  productions 

Assistance 

resources 

Explora  guide 

HTML 

Information  on  use  of  the  environment 

Study  guide 

PDF 

Access  to  a PDF  description  of  the  course 

Technical  help 

HTML 

A frequently  asked  questions  (FAQ)  facility 

Resource  persons 

HTML 

An  e-mail  list  of  persons:  professor,  tutor,  manager, 
technician,  etc. 

Table  1.  Example  of  the  distribution  of  resources  in  an  Explora  Environment 


These  resources  are  all  external  to  the  web  course.  There  are  three  types  of  resources: 

• Java  applets  that  we  have  developed  mainly  to  exploit  the  learner’s  trace  in  the  web  course,  to  build  individual 
progress  reports,  group  profiles,  advice  generation  or  support  to  peer  collaboration  tools; 

• Executables  downloaded  or  resident  on  the  user’s  workstation,  that  trigger  CBTs,  simulations,  or  generic 
software  shared  by  the  group  so  that  file  exchange  is  facilitated;  this  is  particularly  needed  for  some 
communication  tools  wherein  the  use  of  different  software  will  simply  not  work. 

• HTML  generic  services  that  sometimes  have  to  be  sometime  adapted  to  the  course  content;  examples  of  these 
are  course  evaluation  questionnaires  through  which  learners  and  trainers  will  give  feedback  to  designers  to 
improve  the  course,  and  also  the  webography  which  is  a structured  list  of  commented  web  sites. 
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Reengineering  a course  for  a Virtual  Learning  Centre 


We  will  now  focus  on  four  major  steps  in  a process  through  which  a distance  learning  course  can  be  re-engineered 
into  an  HyperGuide  within  Explora  environments.  In  the  example  above,  we  started  from  a distance  course  at  Tele- 
universite  based  on  a package  containing  printed  documents,  a 500  page  book,  a study  guide  describing  the  course 
and  its  learning  activities  and  a technical  guide  for  the  software  material).  The  package  also  contained  8 small 
software  on  3 diskettes  to  illustrate  lA  concepts  and  8 videos  on  VHS  cassettes  for  a total  of  4 hours’  viewing  time. 

1-  Engineering  the  teleleaming  system.  We  have  re-designed  this  course  with  MIS  A (Paquette  et  al  1997).  The 
MISA  method  presents  the  ID  processes  and  tasks  according  to  an  engineering  perspective  analogous  to 
software  engineering.  This  method  innovates  by  using  cognitive  modelling  techniques  to  represent  knowledge, 
as  well  as  pedagogical  scenarios,  learning  materials  specs  and  delivery  plans.  In  this  process,  we  have  made  all 
the  fundamental  decisions,  separating  the  components  specific  to  the  AI  content  that  must  be  integrated  in  the 
HyperGuide  web  site  and  the  generic  resources  (documents,  tools,  services)  that  will  be  useful  for  this  course  in 
an  Explora  environment. 

2-  Building  the  HyperGuide  web  site.  This  second  step  was  rather  straightforward,  based  on  the  MISA  models  that 
provided  precise  orientations  to  the  development  team.  They  could  then  focus  on  the  ergonomics  of  the  web  site 
and  the  presentations  of  the  pages  with  very  little  interaction  needed  from  the  designers.  Three  other  courses 
have  been  rapidly  constructed  afterward  on  the  same  model. 

3-  Selecting  actors  and  resources.  This  task  was  achieved  in  parallel  to  the  previous  one,  based  on  the  MISA 
models.  At  this  point,  we  have  decided  to  have  only  two  actors  at  delivery  time;  learners  and  trainers  (called 
tutors  here).  An  informer  actor  was  not  necessary,  since  all  the  content  had  been  mediated.  A manager  actor 
may  be  added  in  the  future.  The  learner’s  and  the  trainer’s  environments  are  quite  similar  but  specific  tools  for 
the  trainer  have  been  designed  to  give  them  access  to  a more  synthetic  group  progress  profile,  to  maintain  a 
FAQ,  to  animate  teleconferences  and  to  integrate  their  evaluation  into  the  learner’s  progress  results. 

Linking  a course  HyperGuide  and  the  Explora  environments.  This  last  step  has  been  presented  recently  with  more 
details  in  (Girard  et  al,  1999).  Essentially,  the  designer  is  here  using  some  of  the  Explora  designer’s  tools  to  define 
progress  levels  for  each  learning  activity  or  knowledge  unit.  Then  these  progress  levels  are  updated  according  to  the 
learner’s  actions  in  the  HyperGuide  web  site  by  the  system,  according  to  conditions  defined  by  the  designer,  or  by 
the  learner’s  direct  action  in  a viewing  interface.  These  progress  levels  form  a user  model  employed  by  the  system  to 
give  feedback  and  support  to  the  learner.  Support  is  given  using  help  messages,  avatars  demonstrations,  but  also 
adapting  the  interface  with  graphical  cues  to  guide  the  learner  to  aspects  of  his  task  or  to  resources  that  may  be 
useful  to  him.  This  aspect  is  being  developed  in  a new  graphic  adaptive  interface  described  in  (Dufresne  et  al,  1999). 


Conclusion 

There  are  many  advantages  to  the  open  architecture  that  has  been  presented  here.  The  Virtual  Learning  Centre  is  at 
the  learning  organisation’s  level,  thus  avoiding  duplication  and  facilitating  evolution  and  reuse  of  resources  from 
one  course  to  another.  It  also  speeds  up  the  design  process  because  each  individual  web-course  (the  Hyperguide)  is 
freed  from  all  the  generic  resources  and  the  information  management  load  between  different  actors.  Furthermore, 
each  course  can  have  a very  different  content,  structure  and  presentation.  In  fact,  it  can  be  any  web  site,  regardless  of 
the  software  used  to  produce  it.  On  the  other  hand,  all  of  the  Explora  environments  are  similarly  structured,  thus 
facilitating  their  use  in  different  courses  by  learners,  trainers  or  managers.  From  an  organisation’s  viewpoint,  this 
architecture  facilitates  management  through  standardisation  of  resources,  without  constraining  the  design  teams  in  a 
particular  authoring  software. 
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Abstract:  This  paper  discusses  two  systems  that  exploit  multiple  representations  to  facilitate 
learning.  The  Programming  Tutor  facilitates  the  acquisition  of  fundamental  programming  concepts 
through  interaction  with  textual  descriptions  and  flowchart  simulations  of  program  behavior.  The 
Electricity  Laws  Tutor  supports  understanding  of  some  of  the  basic  laws  of  electricity  through  the 
use  of  co-operating  representations:  text,  formulas,  simulated  circuits,  and  animated  metaphors. 
Dimensions  of  multiple  representations  are  discussed,  as  are  their  potential  advantages  and 
disadvantages.  The  paper  discusses  the  evaluation  of  the  two  systems,  and  concludes  with  a brief 
discussion  of  the  problems  arising  in  attempting  to  evaluate  multiple  representations  for  learning. 


Introduction 

Most  forms  of  learning  involve  information  that  is  encoded  into  different  representations.  We  can  have 
“paraphrases”  of  information  expressed  in  the  same  medium  (e.g.,  a textual  explanation  of  a corresponding 
mathematical  formula,  for  example).  Alternatively,  we  may  find  representations  of  some  aspects  of  given 
information  expressed  in  different  media  (a  flowchart  and  a fragment  of  programming  code  representing  the  same 
algorithm,  for  example).  Certain  representations  may  be  chosen  in  preference  to  others  because  they  make  otherwise 
abstract  or  hidden  features  and  processes  concrete  and  visible.  Concepts  that  are  usually  formally  expressed  and  can 
be  difficult  to  conceptualize  may  benefit  from  being  expressed  in  an  alternative  form.  An  obvious  application  of  this 
is  the  drawing  of  metaphorical  representations,  such  as  rope  and  pulley  diagrams,  to  capture  salient  details  of  an 
abstract  notion  such  as  force  (Larkin  & Simon  1987). 

It  is  often  the  case  that  different  representations  exploit  and  express  different  properties  of  the  represented 
thing.  The  diagrams  used  to  express  geometry  problems  are  excellent  for  expressing  the  relationship  between  lines. 
There  are  conventions  for  indicating  that  two  lines  are  parallel,  for  example.  However,  reasoning  must  also  be 
applied  to  solve  problems,  observational  evidence  alone  is  insufficient.  The  incorporation  of  multiple  representations 
into  interactive  multimedia  learning  environments  presents  exciting  opportunities  to  support  and  enrich  the  learning 
experience,  in  three  main  ways.  Firstly,  representations  can  be  animated.  Secondly,  the  representation  can  be 
manipulable  so  the  learner  can  control  and  explore  the  effects  of  changes  to  the  representations.  Thirdly, 
representations  can  co-operate,  in  the  sense  that  the  effects  of  changes  in  one  representation  can  be  seen  in  terms  of 
corresponding  changes  in  the  other  representations. 

This  paper  describes  the  evaluation  of  two  interactive  learning  environments.  Both  embody  one  or  more  of 
the  above  three  dimensions  of  animation,  manipul ability,  and  co-operation.  The  domains  featured  are  in  one  case, 
the  learning  of  basic  laws  of  electricity,  and  the  other,  the  acquisition  of  basic  programming  concepts. 

The  structure  of  the  paper  is  as  follows.  The  next  section  outlines  certain  dimensions  of  multiple 
representations,  and  the  advantages  and  disadvantages  of  multiple  representations  for  learning.  Section  3 briefly 
describes  the  two  systems.  Section  4 discusses  their  evaluation  and  the  results  obtained.  The  paper  concludes  with  a 
brief  general  discussion  of  the  evaluation  of  co-operating  multimedia  representations. 

Multiple  Representations  in  Learning  Environments 

This  section  considers  some  useful  properties  that  can  be  use  to  analyze  and  categorize  multiple  representations,  and 
then  discusses  the  potential  advantages  and  disadvantages  of  multiple  representations. 
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Dimensions  of  Multiple  Representations 

The  key  dimensions  of  multiple  representations  have  been  identified  as  perspective,  precision,  modality, 
specificity,  and  complexity  (van  Someren  et  al.  1998). 

The  perspective  taken  by  a representation  may  be  functional,  behavioral  or  physical,  or  some  combination 
of  these.  A functional  perspective  of  a program  may  describe  its  major  components  and  the  interface  between  them. 
A flowchart  represents  one  possible  behavioral  perspective  on  the  program.  The  code  of  the  program  itself 
represents,  to  some  extent,  a physical  perspective. 

The  precision  of  a representation  is  the  level  of  accuracy  with  which  it  represents  the  target  entity.  A 
representation  may  be  precise  in  some  ways  and  not  in  others.  Consider  a diagram  of  an  electrical  circuit  showing 
the  components  (battery,  lamp,  etc.).  The  diagram  precisely  defines  the  relative  placing  of  the  components  in  the 
circuit,  but  the  depictions  may  bear  little  or  no  resemblance  to  the  components  themselves. 

The  modality  of  a representation  refers  to  its  predominant  form  (text,  graphics,  video,  and  so  on). 

The  specificity  of  a representation  refers  to  its  informational  economy.  Representations  have  computational 
properties  (Larkin  & Simon  1987).  The  computational  properties  of  a representation  refer  to  its  support  for  tasks 
such  as  inference  and  search.  For  example,  it  is  much  easier  to  locate  a component  of  a software  system  in  a 
structure  diagram  than  by  scanning  the  code. 

The  final  dimension  of  multiple  representations,  according  to  van  Someren  et  al,  is  complexity.  Consider  a 
flowchart  for  a program  and  the  corresponding  program  code.  Many  of  the  same  features  are  present  in  both 
representations  (e.g.,  a given  Boolean  test),  albeit  in  different  forms  (a  loop,  visually  represented  in  the  flowchart, 
may  appear  in  the  program  as  a a “WHILE”  loop).  In  this  sense,  there  is  some  redundancy  in  the  representational 
scheme,  which  enables  each  representation  to  offer  a unique  viewpoint  on  the  salient  details. 

In  multiple  representation  schemes,  the  different  representations  are  usually  conceptually  linked.  However, 
they  could  also  be  physically  linked,  in  the  sense  that  operations  in  any  of  the  representations  result  in  the 
corresponding  operation  being  carried  out  automatically,  where  appropriate,  in  the  other  representations.  Such  “co- 
operative” multiple  representations  have  been  considered  in  (Cox  & Bma  1996),  (Parkes  1991),  (Parkes  1992), 
(Parkes  1 994a),  and  (Parkes  1 994b). 


Advantages  and  Disadvantages  of  Multiple  Representations 

As  indicated  by  the  preceding  discussion,  to  represent  all  of  the  salient  details  of  the  topic,  several 
complementary  representations  may  be  necessary.  Text  may  provide  a detailed  description,  but  an  accompanying 
diagram  may  be  indispensable  in  orientating  the  learner  to  the  key  elements  to  which  the  text  refers.  Moreover,  for  a 
complex  domain,  it  is  possible  to  separate  key  details  out  into  different  representations,  which  avoids  overloading 
the  learner  with  a single  all-encompassing  and  highly  complex  representation. 

Expertise  in  a given  domain  is  often,  at  least  in  part,  related  to  the  ability  to  use  several  different 
representations  as  and  when  needed,  and  to  be  able  to  translate  concepts  into,  and  between,  such  representations  as 
necessary.  Co-operating  multiple  representations  can  support  the  learner  in  the  task  of  integrating  the  different 
representations  into  a conceptual  whole,  as  learning  progresses.  Such  schemes  can  also  help  the  learner  to  pace  the 
learning  process.  In  the  initial  stages,  the  learner  focuses  on  one  or  more  of  the  simpler  representations  (e.g.,  a 
circuit  diagram).  As  knowledge  and  confidence  increases,  the  learner  can  devote  more  attention  to  the  more  complex 
or  abstract  representations  (the  formula  for  Ohm’s  law,  for  example). 

The  redundancy  present  in  multiple  representation  schemes  reinforces  the  encoding  of  information  in  long 
term  memory,  by  providing  the  opportunity  for  rehearsal  of  that  information  in  different  forms.  Moreover,  the  fact 
that  the  redundant  information  can  be  acquired  through  several  forms  may  provide  several  “memory  traces”  for  that 
information,  thus  ensuring  that  the  information  is  more  likely  to  be  retrievable  from  long  term  memory  in  the  future. 
Multiple  representations  can  provide  a learner  with  access  to  a preferred  representation.  Learners  can  choose  which 
representation  to  attend  to.  In  a single  representation  system,  a learner  who  encounters  problems  in  understanding 
the  sole  representation  must  seek  alternative  representations  outside  of  the  learning  environment  in  question. 

A potential  problem  with  the  use  of  multiple  representations  is  that  each  representation  may  be,  in  some 
senses,  a statement  in  a different  “language”,  and  a learner  may  need  to  acquire  its  syntax  and  semantics,  which  may 
obstruct  the  learning  of  the  target  domain  itself  Co-operative  representations  may  help  here:  the  user  can  observe 
the  corresponding  outcomes  in  several  representations.  The  user  will  be  able  to  learn  about  the  unfamiliar 
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representation  by  observing  the  translation  of  changes  between  an  understood  representation  and  an  unfamiliar  one. 
However,  while  such  positive  transfer  of  knowledge  can  take  place,  there  is  always  the  possibility  that 
misconceptions  about  one  representation  may  propagate  misunderstandings  throughout  all  of  the  representations. 

A final  potential  disadvantage  of  multiple  representations  concerns  development  cost.  Each  representation 
is  essentially  an  interface  in  itself,  and  must  be  designed,  implemented  and  evaluated  as  such.  Moreover,  if  the 
representations  are  co-operative,  each  representation  must  have  an  associated  set  of  controls  to  manipulate  it.  As 
designers,  we  intend  each  representation  to  be  effective  and  necessary.  The  provision  of  a representation  that 
benefits  no  learner  is  a pointless  exercise.  This  point  is  discussed  Either  in  the  concluding  section  of  this  paper. 

The  preceding  discussion  suggests  that  the  potential  advantages  of  the  use  of  multiple  representations  for 
learning  outweigh  the  disadvantages.  We  now  describe  case  studies  involving  two  interactive  learning  environments, 
both  of  which  feature  co-operating  multiple  representations. 


Two  Interactive  Learning  Environments 

The  systems  described  here  were  developed  by  the  second  and  third  authors  as  students  on  Lancaster’s 
MSc  in  Distributed  Interactive  Systems  (Drakakis  1999),  (Marselos  1999).  The  target  domains  of  the  systems  were, 
in  one  case,  fundamental  programming  concepts  and,  in  the  other,  basic  laws  and  formulas  of  electrical  circuits. 

We  now  briefly  describe  the  two  systems,  before  considering  their  evaluation. 


The  Programming  Concept  Tutor 

This  system  (Fig.  1)  facilitates  the  acquisition  by  complete  novices  of  basic  programming  concepts,  such  as 
input,  output,  assignment,  simple  arithmetic  expressions,  conditional  expressions,  and  loops.  The  tutor  uses 
animated  flowcharts  to  express  the  programming  concepts. 

The  learner  can  proceed  through  the  lessons  in  a suggested  order,  but  also  has  direct  access  to  any  lesson  at 
any  time.  Interactive  exercises  are  provided  for  each  topic,  as  described  later. 

Each  lesson  occupies  one  complete  screen.  At  the  leE  of  the  screen  is  a textual  description  of  the  concept. 
In  the  center  of  the  screen  is  a picture  of  the  flowchart  construct  that  represents  the  lesson  topic.  On  the  right  of  the 
screen  is  a flowchart  representing  a complete  program  featuring  the  lesson  concept.  This  flowchart  can  be  animated, 
so  the  user  can  watch  the  execution  of  the  flowchart  (continuously  or  via  single  stepping).  Along  the  bottom  of  the 
lesson  screen  are  other  representations,  such  as  a textual  description  of  the  current  statement,  and  a small  “display 
screen”  showing  output  from  the  animated  “program”  as  it  executes.  Clicking  on  any  flowchart  symbol  anywhere  in 
the  system  results  in  information  about  that  symbol  being  presented  to  the  learner. 

The  exercises  involve  the  learner  in  identifying  the  errors  in  example  flowcharts.  They  are  answered  by 
clicking  on  appropriate  flowchart  segments,  which  are  then  placed  into  the  appropriate  part  of  the  flowchart, 
replacing  the  erroneous  part.  Explanations  are  provided  for  both  correct  and  incorrect  answers. 


The  Electricity  Laws  Tutor 

The  aim  of  this  system  (Fig.  2)  was  to  facilitate  an  understanding  of  the  basic  concepts  of  electricity,  and 
formulas  for  Ohm 's  Law.  Power  (Wattage),  Serial  and  Parallel  Resistors,  etc. 

A main  screen  provides  direct  access  to  each  lesson,  and  a suggested,  but  not  enforced,  order  is  given. 
Interactive  exercises  are  provided  for  each  topic,  as  described  later. 

Each  lesson  occupies  one  complete  screen.  A screen  features  a textual  explanation  of  the  law  or  concept, 
the  actual  formula  of  interest,  an  animated  circuit  diagram  and  a metaphorical  animation  exemplifying  the  topic 
concept.  There  are  a number  of  (+  and  -)  buttons  that  are  used  to  increase  or  decrease  the  values  of  one  of  the 
relevant  properties  (e.g.,  voltage).  Typical  examples  of  the  metaphors  used  are  water  pipes  with  changing  diameter 
to  represent  resistance,  and  a lorry  carrying  a varying  size  of  load  to  represent  electrical  load. 

When  the  buttons  are  used  to  change  the  values,  the  circuit  diagrams,  any  dependent  button  values 
(according  to  the  calculation  of  the  formula)  and  the  metaphorical  animation,  undergo  corresponding  changes. 

All  exercises  involve  the  application  of  the  lesson  formula  when  one  value  is  missing.  The  given  values  are 
randomly  generated.  Each  exercise  screen  consists  of  a circuit  diagram  showing  the  problem,  with  the  missing 
property  indicated.  The  answer  is  chosen  from  a list  of  “radio  button”  options. 
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Figure  1:  Part  of  a screen  from  the  Programming  Tutor,  paused  while  a flowchart  simulation  is  in  progress. 
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Figure  2:  Part  of  a screen  from  the  Electricity  Tutor,  showing  the  various  co-operating  representations  involved. 


Evaluation  of  the  Two  Systems 

This  section  provides  an  overview  of  the  evaluation  trials  conducted  for  the  Programming  Concepts  Tutor 
and  the  Electricity  Lav^s  Tutor  described  in  the  preceding  section.  First,  the  evaluation  methodologies  are  outlined, 
following  which  the  outcomes  of  the  evaluation  trials  are  discussed. 
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The  Evaluation  Methodologies 

Each  system  was  empirically  evaluated  with  a small  number  of  adult  users  who  were  deliberately  chosen 
for  their  lack  of  knowledge  of  the  target  domains.  Eight  subjects  took  part  in  the  evaluation  of  the  programming 
concept  tutor.  The  electricity  tutor  was  evaluated  in  trials  using  six  subjects.  We  acknowledge  the  small  scale  of 
these  trials.  At  this  stage,  our  aim  is  qualitative  analysis  only. 

In  each  case,  subjects  were  presented  with  a pre-session  questionnaire  to  establish  their  backgrounds  and 
experience.  Following  their  session  with  the  system,  subjects  completed  a post-session  questionnaire.  The  post- 
session questionnaire  featured  questions  on  all  aspects  of  the  interaction  with  the  system,  such  as  usability, 
navigability,  understandability,  interactivity,  enjoyment  of  use,  and  so  on. 

Each  user’s  session  with  the  respective  system  was  video-recorded.  The  experimenter  also  observed  the 
sessions  and  took  notes.  Subjects  were  encouraged  to  think  aloud  as  they  interacted  with  the  system.  Considerable 
effort  was  expended  to  ensure  that  subjects  were  relaxed  and  not  intimidated  by  the  evaluation  process. 

The  rationale  for  the  combined  use  of  three  evaluation  methods  was  to  attempt  to  exploit  the  effectiveness 
of  each  while  overcoming  its  potential  drawbacks.  Direct  observation  is  a simple  and  cost  effective  method,  but 
notes  taken  by  an  observer  may  be  incomplete,  while  video  recordings  are  a permanent  record  that  can  be  studied 
many  times,  and  thus  aspects  missed  by  direct  observation  may  be  subsequently  revealed.  On  the  other  hand  video 
recording  (unless  done  using  many  cameras  from  different  angles)  does  not  capture  all  aspects  of  the  interaction  (it 
may  record  the  subject’s  body  language,  at  the  expense  of  what  is  actually  happening  on  screen  at  the  time 
aloud  protocols”  can  be  useful  in  indicating  what  the  subject  thinks  they  are  trying  to  do.  However,  attempting  to 
verbalize  a complex  task  while  actually  trying  to  learn  how  to  carry  it  out  can  actually  interfere  with  the  learning  of 
that  task.  Similarly,  interventions  by  the  experimenter  to  ensure  continued  verbalization  by  the  subject  when  a 
challenging  task  is  being  attempted  would  also  impede  effective  learning.  That  a session  is  recorded  in  several  forms 
makes  it  unnecessary  for  an  unwilling  subject  to  constantly  verbalize.  A more  thorough  discussion  of  evaluation 
issues  can  be  found  in  (Preece  et  al.  1994). 

With  each  system,  a typical  end-user  session  took  around  1 hour  15  minutes. 

For  the  programming  concept  tutor,  subjects  were  also  given  a post-session  test.  This  is  discussed  later. 


Outcomes  of  the  Evaluation 

For  both  systems,  evaluation  results  were  very  encouraging.  Subjects  performed  very  well  in  the  in-system 
tests.  For  the  electricity  tutor,  more  than  90%  of  the  first  answers  given  were  correct,  while  for  the  programming 
concept  tutor,  the  corresponding  figure  was  greater  than  95%. 

For  the  electricity  tutor,  the  animated  graphical  circuit  was  the  favored  representation  for  all  subjects.  It 
should  nevertheless  be  observed  that  learning  the  corresponding  formula  could  only  take  place  if  the  subject  used 
that  representation  in  conjunction  with  the  formula  description.  The  circuit  diagram  provides  an  advance  organizer 
that  facilitates  the  learning  of  the  formula.  The  extent  to  which  the  metaphorical  animation  was  effective  is  not  so 
readily  determined.  Given  that  this  representation  has  the  highest  cost  in  terms  of  design  and  development  time  and 
effort,  the  question  of  its  effectiveness  is  highly  significant.  However,  the  users  did  state  that  the  metaphor  was 
useful  in  encouraging  comprehension  of  the  more  abstract  material. 

As  stated  above,  a specific  post-test  was  undertaken  by  the  subjects  of  the  programming  concept  tutor.  In 
this  test,  the  subjects  were  first  asked  to  define  each  of  the  programming  concepts  in  their  own  words.  Then  the 
subjects  were  asked  to  solve  four  programming  exercises,  expressing  their  solution  as  a flowchart.  The  subjects  did 
not  have  access  to  the  training  materials  while  taking  the  test.  The  four  exercises  were  of  increasing  difficulty,  the 
final  exercise  requiring  a nested  loop  construct.  Though  only  one  subject  answered  all  four  exercises  correctly,  the 
errors  made  by  the  others  were  mostly  of  an  extremely  trivial  nature  (e.g.  correctly  represented  logic,  but  expressed 
using  the  wrong  flowchart  symbol).  This  is  impressive,  given  that  no  subject  had  previous  programming  or 
flowcharting  experience.  Moreover,  the  fact  that  they  were  able  to  define  the  programming  concepts  in  their  own 
words  indicates  that  more  than  superficial  acquisition  of  the  concepts  was  fostered  by  exposure  to  the  system.  These 
results  relate  to  Parkes’  assertion  (Parkes  1992)  that  an  effective  learning  environment  must  do  more  than  facilitate 
the  acquisition  of  skills  and  techniques  that  can  only  be  applied  in  the  environment  in  which  they  are  acquired. 
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Conclusions 


The  case  studies  described  above  lend  some  support  to  the  hypothesis  that  co-operative  multiple 
representations  can  be  an  effective  facilitator  of  learning.  In  creating  learning  environments  of  the  type  described 
above,  we  would  like  to  be  confident  that  each  representation  is  playing  its  part  in  providing  enjoyable  and  effective 
learning.  However,  evaluating  any  of  the  representations  in  isolation  may  yield  misleading  results.  The  system 
exploits  the  interaction  between  the  representations,  and  therefore  results  applying  to  a representation  considered 
individually  may  differ  from  those  pertaining  to  that  representation  when  it  is  located  in  the  multi-representational 
scheme.  On  the  other  hand,  a representation  within  the  scheme  may  not  be  realizing  its  potential  if  it  is  poorly 
designed,  but  the  roles  played  by  the  other  representations  in  the  scheme  may  obscure  the  effects  of  this.  Truly 
effective  methods  for  evaluating  the  effectiveness  of  co-operative  multiple  representations  for  learning  have  yet  to 
be  devised.  This  is  an  important  aim.  Learning  environments  need  to  be  multi-representational,  because  knowledge, 
skills  and  expertise  are  multi -representational.  Co-operative  multiple  representation  schemes  provide  the  potential  to 
reify  the  computational  processes  involved  in  expertise,  so  that  learners  can  explore  and  subsequently  acquire  these 
processes  as  part  of  an  engaging  and  effective  learning  experience. 
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Abstract:  This  paper  presents  findings  from  a research  project  that  examined  students' 
behaviors  during  mass  lectures  that  utilized  interactive  multimedia.  Students'  engagement 
behaviors  and  disengagement  behaviors  during  the  computer-assisted  mass  lectures  were 
influenced  by  three  types  of  lecture  elements:  interactive  multimedia  (IMM),  the  lecturer, 
and  student-idiosyncratic  factors.  The  significance  of  these  lecture  elements  and  factors  that 
triggered  the  student's  behaviors  are  categorized  and  discussed.  Being  able  to  understand 
what  influences  students'  behaviors  and  attention  in  computer-assisted  mass  lectures  can 
provide  guidelines  for  promoting  effective  learning  in  computer-assisted  mass  lectures. 


Introduction 

There  is  increasing  use  of  interactive  multimedia  (IMM)  in  university  lectures  for  teaching  and 
learning.  Yet  IMM  does  not  guarantee  better  content  learning  than  do  traditional  instruction  methods.  Studies 
report  that  IMM  is  most  effective  when  learners  work  with  IMM  individually  or,  better  still,  in  pairs  or  small 
groups  (Park  and  Hannafin,  1993;  Simpson,  1994).  However,  in  a mass  lecture,  the  IMM  is  controlled  by  the 
lecturer.  Learners  play  a passive-receiver  role.  As  they  do  not  have  direct  access  to  control  the  IMM,  what  do 
they  do  during  lectures?  What  triggers  their  attention?  What  causes  their  behaviors?  Are  they  focused  on  the 
content? 

There  is  still  too  little  research  concerning  how  students  engage  with  the  new  computer  mediated 
technologies,  particularly  the  increasingly  popular  use  of  PowerPoint  and  IMM  in  mass  lectures.  Most  research 
on  the  increasingly  popular  use  of  IMM  in  lectures  has  focused  on  "how-to-tips"  and  critiques  (for  example, 
Broughton,  1997;  Paldy,  1997;  Seaman,  1998).  There  were  few  qualitative  or  empirical  research  studies.  One 
by  Nowaczyk,  Santos,  and  Patton  (1998)  examined  student  perceptions  of  various  characteristics  of  multiple 
media  such  as  color  transparencies,  video,  and  IMM  in  the  undergraduate  classroom,  mainly  in  terms  of 
gaining  and  holding  attention.  Others  (Jackson,  1997;  Van-Meter,  1994)  sought  student  perceptions  of  the 
effectiveness  of  PowerPoint  presentations  rather  than  IMM  presentations  to  support  note  taking,  interest, 
attention,  and  clarity  of  content  organization.  Faraday  and  Sutcliffe's  (1996)  empirical  study  tracked  eye 
movement  and  ascertained  memorization  of  content  during  a short  medical  IMM  presentation.  Research  by 
Putt,  Henderson,  Patching  (1996)  and  Henderson,  Putt,  Ainge,  and  Coombs  (1997)  examined  teachers’ 
thinking  processes  about  the  academic  content  of  IMM  and  WWW  courseware,  respectively.  However,  these 
did  not  focus  on  a mass  lecture  context.  None  investigated  students'  behaviors  triggered  by  a combination  of 
the  features  of  IMM  and  the  lecturer  in  mass  lectures. 

Heeding  current  literature  in  the  field,  the  study  does  not  aim  to  ascertain  whether  learning  with  the 
new  technologies  produces  better  learning  or  test  outcomes  than  traditional  lectures.  Rather,  it  utilizes 
qualitative  methodologies  to  ascertain  the  students’  behaviors  as  they  learned  in  the  authentic  context  of  a 
lecture  theatre.  Thus  the  study  sought  to  identify,  categorize,  and  analyze  the  engagement  and  disengagement 
behaviors  that  were  reported  by  the  students. 


Methodology 

The  research  context  and  methodology  capitalize  on  authenticity.  The  students’  behaviors  were 
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obtained  in  realistic,  ecologically-valid  situations  as  the  data  were  collected  from  students  working  in  their 
regular  mass  lecture  environment,  a medical  school  setting.  The  research  was  conducted  with  a physiology 
class  in  a mass  lecture  theatre,  Faculty  of  Medicine  Siriraj  hospital,  Mahidol  University,  Bangkok,  Thailand. 
The  lecturer  used  A.D.A.M.  (A.D.A.M.  Software,  Inc.,  1995)  in  two  one  hour  lectures.  The  lecture  sessions 
were  video  recorded.  Six  students  volunteered  to  participate  in  stimulated-recall  interviews.  There  were  four 
males  and  two  females  with  ages  ranging  from  17  to  19  years.  They  were  in  the  third  semester  of  a six-year 
medical  degree.  The  six  participants  self  selected  into  2 male  pairs  and  1 female  pair.  The  first  male  pair  was 
interviewed  after  the  first  lecture  session;  the  others  after  the  second  lecture  session.  The  content  of  both 
lectures  was  the  same  topic  and  used  the  same  IMM  application.  All  six  participants  attended  both  lectures. 


Data  Collection 

The  participants  were  interviewed  using  the  stimulated  recall  technique  that  follows  strict  guidelines 
(Putt,  Henderson,  & Patching,  1996).  The  interviews  were  conducted  with  one  pair  at  a time.  The  interviewer 
and  each  pair  of  participants  together  viewed  a videotape  of  the  lecture  and  a synchronized  computer  screen 
showing  the  IMM  that  was  used  in  the  lecture.  The  video  picture  included  the  lecturer’s  verbal  and  nonverbal 
behaviors  and  the  content  of  the  computer-assisted  lecture.  The  computer  screen  showed  the  IMM  (A.D.A.M.) 
content  which  appeared  on  the  videotape.  Both  videotape  and  computer  screen  facilitated  the  participant's 
recall  and  verbalization  of  their  thinking  and  behaviors  during  the  lecture.  The  three  stimulated  recall 
interviews,  one  hour  duration  for  each  interview,  were  conducted  immediately  after  the  lecture  sessions  and 
were  audio  taped  for  later  transcription  and  analysis. 


Results 


From  the  interview  transcripts  of  the  students’  stimulated  recall  interviews,  their  behaviors  were 
jointly  identified  and  categorized  on  the  first  transcript  by  the  researchers.  The  remaining  transcripts  were 
analyzed  individually  and  the  data  from  each  researcher  were  compared  and  discussed.  Consensus  was 
achieved  when  disagreement  occurred.  The  behaviors  reported  by  the  participants  were  classified  as  those  that 
involved  their  engagement  with  the  content  and  those  that  indicated  disengagement  with  the  content.  These 
two  major  categories  were  then  subdivided  according  to  what  influenced  their  engagement  and  disengagement: 
the  "IMM"  product  (A.D.A.M),  "The  Lecturer",  and  "Student-Idiosyncratic  Factors"  that  were  not  specifically 
triggered  by  either  A.D.A.M.  or  the  lecturer  (see  Table  1). 

The  frequencies  for  each  of  these  types  of  behaviors  were  tallied.  A total  of  203  reported  behaviors 
were  identified  from  the  transcripts.  The  engagement  behaviors  were  reported  158  times  (78%)  while  the 
disengagement  behaviors  were  reported  45  times  (22%).  For  the  factors  of  engagement,  students’  behaviors 
influenced  by  the  IMM,  the  lecturer,  and  student-idiosyncratic  factors  were  68%,  23%,  and  9%  of  the  total 
reported  behaviors,  respectively.  For  the  factors  of  disengagement,  students'  behaviors  influenced  by  the  IMM, 
the  lecturer,  and  student-idiosyncratic  factors  were  47%,  15%,  and  38%  of  the  total  reported  behaviors, 
respectively.  Each  of  the  mass  lecture  elements,  IMM,  the  lecturer,  and  student- idiosyncratic  factors,  that 
triggered  each  of  the  behaviors  are  tabulated  in  Table  1.  Features  of  the  IMM  influenced  the  highest  number  of 
engagement  behaviors.  However,  it  influenced  the  highest  number  of  disengagement  behaviors  as  well. 


Discussion 

Engagement 

Interactive  multimedia  (IMM) 

Because  IMM  played  a major  role  in  the  computer-assisted  mass  lectures,  it  is  not  surprising  that 
IMM  is  the  factor  that  most  influenced  students’  engagement  and  disengagement  behaviors.  From  students’ 
reports,  IMM  influenced  68%  of  their  engagement  behaviors  and  47%  of  their  disengagement  behaviors. 

(a)  Focusing 
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Focusing  on  the  IMM  presentation  was  the  highest  reported  engagement  behavior  during  the 
computer-assisted  mass  lecture.  These  behaviors  were  influenced  by  animations,  pictures,  text,  sound  effects, 
voice-overs,  graphs,  and  color  changes.  The  findings  show  animation  was  the  feature  to  which  students  paid 
most  attention.  "Animation  has  a strong  effect  on  attention"  (Faraday  & Sutcliffe,  1996,  p.270)  because  it 
simultaneously  contains  pictures,  sound,  and  movement  concerning  the  concept  being  explained  (Hillstrom  & 
Yantis,  1994).  Because  the  animations  were  replayed  immediately,  the  students  paid  more  attention  to  the 
detail  during  the  second  or  third  time  it  was  shown.  The  benefit  is  that  all  students  received  the  correct 
information.  They  might  have  had  different  understandings  if  they  had  had  to  imagine  the  animations  by 
themselves  as  they  would  have  had  to  do,  if  the  graph  or  picture  was  on  an  OHT  or  if  the  lecturers’  verbal 
explanations  had  been  the  medium  of  delivery. 


Influenced 

by 

Behaviors 

Fre- 

quenc 

y 

Triggering  Factors 

IMM 

• focusing  (looking. 

39 

• pictures,  animations,  text,  sound  effects,  voice 

reading,  listening) 

over,  graphs,  zoom,  color  changes 

• note  taking 

19 

• text  on  screen,  graph 

• anticipating/waiting 

19 

• pictures,  animations,  text,  graphs,  color,  sound. 

zoom,  other  topics  shown  on  screen 

• linking 

10 

• pictures,  graphs 

• understanding 

6 

• pictures,  animations,  color  changes 

• affective 

4 

• animations,  computer  problems 

• generating  (ideas) 

3 

• pictures,  IMM  usage,  IMM  laboratory 

• using  textbook 

3 

• graph,  data 

• not  sleeping 

3 

• exciting,  fun  and  interesting  medium 

• recall 

1 

• picture 

e 

total 

107 

E 

Lecturer 

• focusing  (listening. 

17 

• explanations 

&£ 

following) 

A 

&£ 

• replying 

9 

• explanations,  questions 

M 

N 

• note  taking 

3 

• explanations 

• imaging 

3 

• explanations 

• anticipating 

2 

• lecturer-computer  interaction 

• understanding 

1 

• explanations 

• not  sleeping 

1 

• voice 

total 

36 

Student- 

• generating 

4 

• content 

Idiosyncratic 

• anticipating 

3 

• content 

Factors 

• reviewing 

3 

• content 

• focusing 

3 

• of  own  accord 

• affective 

1 

• correct  answer 

• understanding 

1 

• content 

total 

15 

IMM 

• unfocused  (not  looking. 

16 

• small  text,  listening,  focused  on  other  element 

missing  or  not  listening) 

• graph,  sound,  pictures 

• linking 

4 

• picture 

• sleepy 

1 

total 

21 

Lecturer 

• unfocused 

3 

• lecturer's  IMM  usage  (e.g.,  kept  clicking). 

lecturer's  speech 

e 

• sleeping 

2 

• explanations 

E 

• affective 

1 

• lecturer's  speech 

&£ 

a 

• perplexing 

1 

• lecturer's  IMM  usage  (passed  many  screens) 

QMj 

e 

V) 

total 

7 

ERIC 


8S5 


Student- 

• unfocused 

9 

• absent-mind,  thinking  about  something  else 

Idiosyncratic 

• not  understanding 

2 

• content 

Factors 

• bored 

2 

• same  content 

• perplexing 

2 

• new  topic,  answer 

• lost 

1 

• question-answer 

• sleepy 

1 

total 

17 

total 

203 

Table  1:  Students'  engagement  and  disengagement  behaviors  influenced  by  IMM,  the  lecturer,  and  other 


factors  and  the  factors  that  triggered  their  behaviors. 

Text  was  the  second  highest  ranked  IMM  feature  that  influenced  students'  attention.  The  reason  for 
this  finding  is  text  is  readable  and  notable  data,  that  is,  it  lends  itself  to  note  taking.  Graphs  were  the  third 
highest  ranked  IMM  feature  that  influenced  students'  engagement  behavior.  The  graphs  contained  statistical 
data  that  helped  explain  the  content.  However,  the  findings  show  graphs  were  reported  less  than  text.  This 
could  be  that  graphs  were  presented  on  screen  less  than  text.  It  is  interesting  that  pictures  were  the  fourth 
highest  ranked  IMM  feature  that  influenced  students’  attention  even  though  they  had  been  used  many  times 
during  the  lecture.  According  to  Levie  (1987),  a possible  explanation  is  that  students  have  to  spend  more  time 
decoding  information  from  animations  and  text  compared  with  pictures.  Therefore,  they  paid  more  attention  to 
those  features  than  pictures. 

(b)  Note  taking 

Note  taking  is  the  second  highest  ranked  (17.7%)  engagement  behavior  influenced  by  IMM.  Note 
taking  is  one  thing  that  student  would  do  to  record  information  for  review,  especially  for  the  exams.  However, 
note-taking  behaviors  were  reported  less  than  focusing  on  the  IMM  presentation.  A possible  explanation  is  that 
students  may  learn  by  remembering  the  combined  features  of  IMM  such  as  animation,  pictures,  and  sound 
without  need  for  note  taking.  In  other  words,  the  animations  made  the  information  more  memorable  and 
understandable  (Nowaczyk,  Santos,  & Patton,  1998)  and,  therefore,  students  may  not  need  written  "evidence" 
(notes)  that  help  them  recall  the  content.  Also  it  is  not  easy  to  take  notes  of  animations  and  pictures.  Students 
reported  they  took  notes  from  computer-assisted  mass  lectures  less  than  traditional  lectures  and  lectures  that 
used  OHTs  and  PowerPoint.  Less  note  taking  could  help  students  focus  more  on  the  content  and  not  worry 
about  writing.  They  may  engage  with  the  content  more  by  learning  with  computer-assisted  mass  lectures. 

(c)  Anticipating 

Anticipating  includes  predicting,  looking  forward  to,  and  speculating  about,  the  likelihood  of 
encountering  problems,  types  of  content,  and  features  of  the  medium.  Anticipation  is  also  the  second  highest 
ranked  engagement  behavior.  A possible  explanation  for  this  finding  is  that  the  lecturer  was  the  only  person 
who  controlled  the  IMM,  thus  the  students  anticipated  and  predicted  what  the  lecturer  would  present. 
Students'  anticipations  concerned  animations,  graphs,  pictures,  text,  color,  and  sound  in  descending  order  of 
reported  occurrences.  Predicting  what  would  happen  next  in  the  animation  influenced  47%  of  the  reported 
anticipating  behaviors.  A probable  reason  for  the  high  percentage  for  animations  is  because  they  knew  (after 
the  first  showing)  that  the  animations  provided  information  that  showed,  for  example,  how  the  muscles 
contracted  and  expanded  and  how  this  affected  the  rest  of  the  body,  and  they  therefore  attempted  to  second- 
guess  the  consequences  of  subsequent  animations. 

(d)  Linking 

"Linking”  is  a mental  activity  in  which  students  associate  or  bring  together  two  or  more  ideas,  topics, 
contexts,  personal  experiences,  etc.  It  is  the  third  highest  ranked  engagement  behavior.  Students  reported 
linking  from  pictures  and  graphs  to  their  past  experiences  with  the  content  of  the  lecture,  such  as  experiments 
in  the  laboratory.  However,  linking  behaviors  were  reported  as  disengagement  behaviors  as  well.  Students, 
sometimes,  linked  pictures  and  graphs  to  objects  and  images  that  were  outside  the  content,  for  example,  the 
movie,  “Titanic". 

(e)  Understanding 

"Understanding”  is  the  fourth  ranked  engagement  behavior.  Students  may  have  understood  more  than 
the  six  reported  comments  suggest;  however,  given  the  number  of  reported  engagement-with-the-content 
behaviors,  it  seems  that  students  distinguished  between  understanding  (6  reports)  and  focussing  (39  reports). 
Research  in  this  area  could  be  worthwhile.  Students  reported  that  some  of  their  understanding  was  influenced 
by  pictures,  animations,  and  color  changes.  The  possible  explanation  for  this  finding  is  that  pictures, 
animations,  and  color  changing  indicated  the  points  that  the  lecturer  or  the  IMM  voice-over  was  discussing 
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and  thus  highlighted  aspects  of  the  concept,  for  example,  by  changing  the  color  of  the  text  or  graph. 

(f)  Generating,  using  textbook,  and  not  sleeping 

“Generating”,  “using  the  textbook”,  and  “not  sleeping”  have  the  same  frequency.  There  are  several 
reports  of  using  the  textbook  during  the  lectures.  The  students  reported  that  they  wanted  to  follow  the  lecturer's 
explanations  in  the  textbook.  This  was  also  a reason  given  for  not  taking  notes  of  what  the  lecturer  was  saying. 
The  A.D.A.M.  IMM  software  that  was  used  in  the  lecture  had  a beneficial  effect  in  terms  of  maintaining 
students’  engagement.  Students  reported  not  sleeping  during  the  lectures.  Their  reason  was  that  the  IMM  was 
an  exciting,  fun,  and  interesting  medium.  The  males  also  reported  that  they  did  sleep  more  during  lectures  that 
used  OHTs  or  traditional  lectures.  Their  reasons  point  to  the  advantage  of  using  IMM  in  mass  lectures. 

The  Lecturer 

The  lecturer  influenced  engagement  behaviors  as  well.  From  the  students*  reports,  the  lecturer 
influenced  their  listening,  replying  to  whole  class  questions,  following  his  explanations,  note  taking,  imaging, 
anticipating,  focusing,  understanding,  and  ’’not  sleeping"  behaviors.  Most  of  these  behaviors  were  triggering 
by  the  lecturer's  explanations.  Listening  was  reported  3 times  as  being  influenced  by  IMM  but  was  reported  10 
times  as  being  influenced  by  the  lecturer.  The  reason  for  this  finding  is  that  the  lecturer  provided  verbal 
information  in  Thai  whilst  A.D.A.M.  had  English  voice  over  explanations.  Therefore,  the  students  chose  to 
listen  to  the  lecturer  rather  than  IMM  voice  over.  Generally,  engaging  with  the  lecturer’s  explanations  was  also 
because  he  explained  and  clarified  the  content  being  shown  in  A.D.A.M.  Because  the  students  reported  that 
they  might  not  be  able  to  comprehend  the  information  from  IMM,  partly  because  they  had  no  control  over 
pacing  or  reviewing,  they  therefore  focussed  on  the  lecturer's  explanations  in  order  to  understand  the  content. 
Their  "not  sleeping"  behavior  was  triggering  by  the  lecturer's  voice  as  well. 

Student-idiosyncratic  factors 

Beside  IMM  features  and  the  lecturer,  there  were  other  factors  that  influenced  engagement  behaviors. 
The  most  frequent  factor  that  influenced  engagement  behaviors  was  content.  Students  reported  that  some 
generating,  waiting,  reviewing,  focusing,  and  understanding  behaviors  were  influenced  by  the  content  itself 
(rather  than  the  IMM  or  the  lecturer).  Engagement  with  the  content  naturally  depends  on  the  individual. 
Students  who  perform  well  would  engage  with  the  content  without  the  IMM.  However,  engagement  with  the 
content  that  was  not  triggered  by  IMM  or  the  lecturer's  keying  devices  (for  example,  "note  this"  statements) 
represented  just  9%  of  the  students'  idiosyncratic  behaviors.  Compared  with  the  percentage  of  engagement 
behaviors  influenced  by  IMM  (68%),  IMM  was  a significant  factor  helping  students  engage  with  the  content. 

Disengagement 
Interactive  multimedia  (IMM) 

IMM  also  triggered  student  disengagement  behaviors  that  were  reported  21  times.  This  number  is 
higher  than  disengagement  behaviors  that  were  influenced  by  the  lecturer  (7  times)  and  student-idiosyncratic 
factors  (17  times).  Not  looking  at  the  IMM  software  projected  on  the  screen  was  the  highest  ranked 
disengagement  behavior.  Small  text  was  the  main  factor  that  made  students  decide  not  to  look  at  information 
on  the  screen.  This  finding  shows  the  significance  of  presenting  readable  text  data  on  the  screen  in  mass 
lecture  situations.  Moreover,  readability  problems  could  affect  note-taking  behaviors  as  well. 

Lecturer 


The  findings  show  that  students  did  not  focus  while  the  lecturer  was  searching  for  the  information 
and  paid  less  attention  when  the  lecturer  stopped  interacting  with  the  computer  and  adopted  the  traditional 
lecture  style  of  talking  to  the  students.  This  also  caused  students’  sleeping  behaviors;  this  was  confirmed  from 
the  authors*  observations  of  the  students  during  the  mass  lectures.  Therefore,  the  lecturer's  explanations  were 
the  triggering  factor  that  most  influenced  listening  behaviors  yet  it  also  caused  disengagement  behaviors  as 
well,  particularly  when  the  students  had  become  used  to  the  combination  of  IMM  and  lecturer  explanation. 

Student-Idiosyncratic  factors 
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The  "student-idiosyncratic  factors"  influenced  disengagement  behaviors  more  than  the  lecturer  did. 
Student-idiosyncratic  factors  that  triggered  disengagement  behaviors  were  varied.  Not  focusing  on  the  content, 
in  terms  of  being  absent-minded  and  thinking  about  something  else,  was  the  first  ranked  disengagement 
behavior.  This  relies  on  individual  differences  that  could  be  dependent  on  attitudes  toward  content  and 
motivation  to  learn  (Reeves,  1 998). 


Conclusion 

According  to  the  findings,  IMM  features  triggered  students'  engagement  with  the  content  more  than 
the  lecturer  and  other  features  did.  However,  most  engagement  behaviors  influenced  by  IMM  were  triggered  by 
visual  effects.  The  first  ranked  IMM  feature  that  triggered  engagement  behaviors,  animation,  has  a strong 
effect  on  engagement  (Hillstrom  & Yantis,  1994).  Students'  attention  will  be  locked  when  processing  the 
animations  (Faraday  & Sutcliffe,  1996).  The  replay  function  of  the  animation  feature  gains  more  students' 
attention.  Color  changes  indicate  the  important  points.  It  can  shift  students'  attention  to  other  elements  on  the 
screen  that  the  lecturer  wanted  to  emphasize.  Changing  screens  also  create  movement  on  the  screen,  thereby 
influencing  students  to  refocus  on  the  content.  Leaving  the  information  on  one  screen  for  a long  time  may 
cause  disengagement  behavior  such  as  day  dreaming.  Sound  effects  may  focus  students'  attention  but  they 
might  not  be  able  to  provide  better  understanding  (Bradey,  1997).  Students  paid  most  of  their  audio  attention 
to  the  lecturer's  explanations.  At  this  point,  this  research  shows  that,  in  a computer-assisted  mass  lecture 
environment,  the  students'  engagement  could  be  triggered  by  IMM  visual  effects  and  the  lecturer's  instruction 
(Paivio,  1990).  The  "collaboration"  between  IMM  and  the  lecturer  in  computer-assisted  mass  lectures  should 
encourage  engagement  behaviors.  However,  inappropriate  visual  information  may  trigger  disengagement 
behaviors.  Small  text  is  an  example  in  this  study.  This  may  lead  to  difficulties  in  comprehension  (Faraday  & 
Sutcliffe,  1996). 

Even  though  this  study  cannot  be  generalized,  the  results  should  help  lecturers  and  IMM  designers 
understand  some  of  the  factors  that  influence  engagement  and  disengagement  behaviors  during  learning  with 
computer-assisted  mass  lectures. 


References 

Bradey,  S.  (1997).  The  impact  of  digital  audio  on  students'  perceptions  of  learning  through  interactive  multimedia 
courseware.  B.Ed  (Hons)  thesis.  Townsville:  James  Cook  University. 

Broughton,  P.  (1997).  Designing  dynamic  PowerPoint  presentations.  Paper  presented  in  Partnerships  for  workforce 
development  in  business  and  marketing  education.  Annual  Atlantic  Coast  Business  and  Marketing  Education  Conference 
Proceedings,  February  21-22,  Greenville,  North  Carolina. 

Faraday,  P.,  Sutcliffe,  A.  {\996)  An  Empirical  study  of  Attending  and  Comprehending  Multimedia  Presentations.  City 
University,  London,  pp  265-275. 

Henderson,  L.,  Putt,  I.,  Ainge,  D.,  Coombs,  G.  (1997).  Comparison  of  student  thinking  process  when  studying  with 
WWW,  IMM,  and  Text-based  material.  Paper  submitted  to  IFIP  3.3  and  3.6  Joint  Working  Conference,  Madrid. 

Hillstrom,  A.  & Yantis,  S.  (1994).  Visual  motion  and  attentional  capture.  Perception  & Psychology,  55(4),  399-41 1. 

Jackson,  S.  (1997).  Case  study:  The  use  of  PowerPoint  in  teaching  comparative  politics/  [online] 
http://horizon.unc.edu/TS/featured/1997-05a.asp  Accessed  27  July  1999. 

Levie,  W.  (1 987).  Research  on  Picture:  A guide  to  the  literature.  In  D.  Willows  & H.  Houghton  (Eds./  The  psychology  of 
illustration.  Vol.  I (pp.  1-49).  New  York:  Springer- Verlag. 

Nowaczyk,  R.,  Santos,  L.,  & Patton,  C.  (1998).  Student  Perception  of  Multimedia  in  the  Undergrad  Classroom. 
International  Journal  of  Instructional  Media,  25(4),  367-382 


Paldy,  L.  (1997)  Graphic  in  the  lecture  hall.  Journal  of  College  Science  Teaching,  27(1),  4-5. 

Paivio,  A.  (1990).  Mental  representations:  A dual  coding  approach.  New  York:  Oxford  University  Press. 

Park,  1.,  Hannafin,  M.  (1993).  Empirically-based  guidelines  for  the  design  of  interactive  multimedia.  Educational 
Technology  Research  & Development,  4 IQ),  63-85. 

Putt,  1.,  Henderson,  L.,  & Patching,  W.  (1996).  Teachers’  Thinking  Elicited  from  Interactive  Multimedia  Professional 
Development  Courseware.  Educational  Technology  Research  & Development,  44{4),  7-22. 

Reeves,  T.  (1998).  User  characteristics,  [online]  http://mimel.marc.gatech.edu/MM  Tools/analvsis.html.  Accessed  6 April 
1999. 

Seaman,  M.  (1998).  Developing  visual  displays  for  lecture-based  courses.  Journal  of  Psychology.  25(2),  141-45. 

Simpson,  M.  (1994).  Neurophysiological  consideration  related  to  interactive  multimedia.  Educational  Technology 
Research  & Development,  42(1),  75-80. 

Van-Meter,  D.  (1994)  Computer-aided  lecturing.  Journal  of  Natural  Resources  and  Life  sciences  Education.  25(1),  62-64 


ERIC 


899 


The  Essen  Learning  Model  - a Multi-Level  Development  Model 


Jan  M.  Pawlowski 

Information  Systems  for  Production  and  Operations  Management 
University  of  Essen,  Germany 
jan.pawlowski@wi-infuni-essen.de 

Abstract;  The  Essen  Learning  Model  is  a generic  development  model  supporting  development 
processes  on  three  levels:  development  of  curricula,  learning  sequences,  and  learning  units.  We 
focus  on  finding  an  adequate  combination  of  didactical  methods  and  educational  technologies. 

Secondly,  we  focus  on  interoperability,  reusability,  and  ease  of  use,  using  standards  on  the 
modeling  level  (ARIS),  architecture  level  (LTSA,  IMS),  and  implementation  level  (XML).  The 
approach  will  enable  educators,  project  managers,  and  authors  to  efficiently  develop  and 
implement  Computer  Supported  Learning  Environments. 

Introduction 

The  Essen  Learning  Model  is  a development  model  to  ensure  the  overall  quality  of  the  development  process  of 
learning  environments  on  different  levels.  New  technologies  in  the  educational  sector  require  a well  planned  and 
controlled  development  process.  Our  experience  has  shown  weaknesses  of  current  development  models  for  the 
utilization  of  new  technologies  in  the  educational  sector. 

The  use  of  new  technologies  is  often  seen  as  a universal  solution  to  all  problems  and  challenges  in  the 
educational  sector.  Educational  concepts  like  Virtual  University,  Multimedia,  or  Edutainment  have  gained  an 
increased  attention.  Despite  of  extensive  research  efforts,  many  immature  and  unsuitable  products  and  concepts 
can  be  found.  Quality  management  is  not  yet  a natural  part  of  educational  projects. 

The  use  of  development  models,  originating  from  the  field  of  software  engineering,  is  a first  step  towards  quality 
management  and  assurance.  Nevertheless,  several  restrictions  remain:  software  development  models  do  not 
include  didactical  concepts  adequately.  Specific  models  for  the  development  of  hypermedia  systems  or  CBT- 
systems  require  the  selection  of  didactical  methods  before  the  start  of  the  project.  To  overcome  those  weaknesses 
of  most  development  models,  a multilevel  development  model  was  designed,  supporting  educational  processes 
on  three  levels:  the  design  and  implementation  of  curricula,  learning  sequences,  and  learning  units. 

I present  basic  concepts  of  learning  environments  and  the  Learning  Technology  Systems  Architecture  which  is 
the  base  for  the  development  of  learning  environments.  By  following  this  standard  architecture,  systems 
implemented  using  the  Essen  Learning  Model  can  easily  be  integrated  and  coupled  with  other  systems,  reducing 
the  development  effort  significantly.  Thereafter,  I describe  design  principles  and  the  levels  of  the  Essen  Learning 
Model  together  with  a sample  implementation  for  the  development  of  a learning  sequence  on  “Simulation”. 

Computer  Supported  Learning  Environments 

Our  development  model  is  based  on  the  Learning  Technology  Systems  Architecture.  I will  briefly  describe  this 
standard  and  our  implementation. 

Classification  of  CSLE 

According  to  (Euler,  1992),  Computer  Supported  Learning  Environments  might  be  classified  into  five  classes: 
Drill  & Practice  Systems,  Intelligent  Tutoring  Systems,  Hypermedia  Systems,  Cognitive  tools,  and 
Simulations/Micro  Worlds.  Those  criteria  only  describe  trends  of  developments.  They  cannot  be  used  for 
evaluating  or  selecting  a certain  learning  environment.  In  contrast,  the  Essen  Learning  Model  uses  a detailed 
multi-dimensional  classification  including  attributes  like  learning  objectives,  strategies,  learning  contents, 
temporal  aspects,  location,  interaction,  and  users. 

Architectures  of  CSLE 

Concepts  like  the  Virtual  University  or  distance  learning  are  based  on  the  exchange  and  combination  of  courses 
and  learning  contents  distributed  world-wide.  Unfortunately,  the  majority  of  learning  environments  and 
authoring  systems  do  not  use  standardized  methods  in  order  to  support  exchange  of  materials  or  courses.  This 
leads  to  an  enormous  number  of  isolated  systems,  not  utilizing  and  being  utilized  by  other  systems.  To  overcome 
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those  weaknesses,  several  standardization  projects  have  been  started.  The  Essen  Learning  Model  uses  an 
implementation  of  the  proposed  standard  architecture  by  the  Learning  Technology  Standards  Committee  of 
IEEE. 

The  Learning  Technology  Systems  Architecture  (LTSA)  is  an  abstract  model  describing  the  architecture  of 
learning  environments  (Fig.  1).  It  contains  the  following  components:  Processes  describe  the  main  subsystems  of 
learning  processes:  Learner,  Delivery,  Evaluation,  System  Coach.  Flows  describe  in-  and  outputs  of  the 
processes:  Behavior,  Assessment,  Performance,  Query  Index,  Content  Index,  Locator  Index,  Learning  Content, 
Multimedia,  Learning  Style.  Finally  Stores  store  relevant  information:  Records  Database,  Knowledge  Library. 


Fig.  1:  Learning  Technology  Systems  Architecture 


Furthermore,  the  LTSA  defines  formats  for  the  communication  and  interaction  of  systems.  The  implementation 
of  ELM  (Fig.  2)  uses  the  standards  of  the  LTSA  specification  in  order  to  combine  and  integrate  different  learning 
environments. 

The  main  element  of  a CSLE  is  a Learning  Engine  for  the  steering,  control,  and  co-ordination  of  the 
components.  The  Knowledge  Base  contains  learning  objectives  and  contents  used  by  the  CSLE.  The  Methods 
Base  of  a Learning  Environment  contains  different  didactical  methods  and  concepts,  supporting  authors  and 
educators.  A User  Model  is  designed  in  the  development  phase.  This  model  contains  attributes,  characteristics, 
and  knowledge  of  a user.  In  collaborative  learning  environments,  additional  information  about  the  group  and  the 
group  process  is  modeled  (Kinshuk,  Patel,  1 997).  The  Presentation  Component  allows  the  generation  and 
presentation  of  learning  contents  in  different  ways.  The  Communication  Component  determines  the  level  of 
interactivity  of  a Learning  Environment.  It  contains  methods  for  communication,  co-operation,  and  its  technical 
realization.  Communication  between  learner,  teacher,  and  system  is  to  be  co-ordinated  in  different  forms 
(Heeren,  1996).  The  Evaluation  Component  determines  the  performance  of  the  learner,  providing  tests  or  ratings. 


Development  Models  in  Software  Engineering 

The  main  function  of  a software  engineering  model  is  to  establish  the  phases  in  which  major  tasks  are  performed 
within  a project,  and  to  establish  the  transition  criteria  for  proceeding  from  one  task  to  the  next.  A software 
development  process  model  supports  the  implementation  of  software  projects  together  with  the  planing, 
scheduling,  and  controlling  of  time  and  budget  constraints.  According  to  (Balzert,  1998),  a development  model 
specifies  the  following  issues:  Processes  (scope,  activities,  work  breakdown  structure,  schedule,  prioritization), 
Organization  (competencies,  responsibilities,  roles,  budget),  General  Issues  (standards,  guidelines,  methods, 
techniques,  tools,  reporting).  There  are  various  software  development  models,  following  different  strategies. 

For  many  years  the  Waterfall  model  (Royce,  1 970)  was  the  most  popular  process  model.  It  is  a highly  structured 
approach,  breaking  down  the  development  process  into  multiple  phases.  Each  phase  creates  a result  which 
triggers  the  next  phase  after  approval.  If  the  result  is  rejected,  the  performed  phase  is  repeated.  Backtracking  is 
limited  to  successive  stages  due  to  complexity  reasons  and  to  minimize  expensive  re-work.  In  general,  the  model 
is  transparent  and  easy  to  understand,  however,  there  are  essential  disadvantages.  Requirement  changes  during 
the  development  are  hard  to  handle  because  of  the  limited  backtracking  mechanism.  Involvement  of  the  users  is 
only  provided  during  the  requirement  specification  in  a very  early  phase.  Especially  for  learning  environments, 
the  lack  of  motivational  factors  leads  to  acceptance  problems.  Due  to  the  sequential  approach,  concurrent 
activities  are  not  supported,  hence  extending  the  duration  of  a project. 

Prototyping  (Balzert,  1998)  is  a popular  method  to  provide  „user  friendly^  software.  It  is  based  on  the 
development  of  executable  models  and  patterns,  defining  software  characteristics  and  requirements  successively 
during  the  development  process.  These  basic  steps  are  repeated  until  all  requirements  are  well  defined.  The 
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prototype  itself  is  discarded  and  the  real  system  is  build,  based  on  the  prototype.  In  comparison  to  the  Waterfall 
model,  the  development  effort  is  higher  due  to  the  additional  development  of  prototypes.  The  main  advantage  of 
Prototyping  is  early  user  involvement.  Because  of  this  important  advantage,  the  Essen  Learning  Model 
incorporates  prototypes  into  the  development  process. 

The  Spiral  model  (Boehm,  1986)  combines  positive  aspects  of  the  Waterfall  model  and  the  Prototyping 
approach.  It  can  be  used  for  rapid  development  of  incremental  versions.  The  definition  of  the  software  product 
and  software  process  is  elaborated  in  a series  of  repetitive  steps.  These  steps  are  represented  as  a spiral 
progression  through  a project.  Each  cycle  of  the  spiral  elaborates  product  and  process  objectives  and  constraints, 
alternative  product  and  process  solutions,  identifying  and  resolving  major  risks.  The  disadvantages  are  concerns 
about  the  high  level  of  control  and  negotiation  difficulties  caused  by  the  amount  of  control.  High  expertise  in  risk 
management  is  required.  Advantages  are  the  match  of  changing  requirements  in  the  whole  development  process, 
and  the  consideration  of  risks  at  all  stages  of  the  development.  Especially  for  learning  environments  these 
characteristics  are  significant:  therefore  this  approach  was  chosen  for  the  Essen  Learning  Model. 

Development  Models  for  CSLE 

There  are  various  models  specifically  designed  for  the  development  of  CSLE.  Major  approaches  are: 
models  based  on  Didactical  Frameworks,  e.g.,  on  the  „Berliner  ModeH“  (Heimann,  Schulz,  Otto,  1965).  They  do 
not  consider  to  use  technical  innovations.  Therefore,  they  can  be  used  for  didactical  planing  but  not  be  applied  to 
new  methods  and  technologies.  Specific  CSLE  models  for  certain  instances,  e.g.,  CBT  development  (Steppi, 
1997)  or  Hypermedia  Development  Model  (Garzotto,  Paolini,  Schwabe,  1993).  The  weaknesses  of  these  models 
are  obvious  because  didactical  decisions  must  be  made  before  the  selection  of  the  CSLE.  Instructional  Design 
models  (Issing,  1 997)  describe  the  structured  development  process  of  the  educational  systems,  courses,  and 
teaching  products.  They  fulfill  the  requirements  much  better  than  the  above  described  models,  however,  they 
support  authors  and  teachers  only  insufficiently  because  the  conceptual  phase  lacks  flexibility. 

As  a conclusion,  all  the  above  mentioned  models  do  not  consider  technological  and  didactical  aspects 
sufficiently.  Either  they  lack  flexibility,  or  developers  and  users  are  highly  restricted  in  their  creativity.  The 
Essen  Learning  Model  is  an  approach  which  integrates  and  improves  positive  aspects  of  different  development 
models. 


The  Essen  Learning  Model 

The  Essen  Learning  Model  was  developed  at  the  University  of  Essen.  It  is  currently  being  evaluated  to  be  used 
in  several  departments  of  our  university.  I will  describe  the  design  principles  and  the  general  flow  of  the 
approach.  
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Fig.  3:  The  Essen  Learning  Model 


The  Essen  Learning  Model  (Fig.  3)  is  a multi-level  development  model.  The  complex  development  process  is 
divided  into  modules  supporting  both,  the  development  process  and  the  use  of  the  system  on  different  levels.  The 
support  of  curriculum  development  and  design  (C-level),  planning  and  implementation  of  learning  sequences  (D- 
level),  and  the  design  and  implementation  of  learning  units  (E-level). 
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We  distinguish  three  abstraction  levels:  The  development  model  provides  generic  knowledge  for  a variety  of 
contexts.  This  generic  model  is  customized  depending  on  the  user’s  needs  and  preferences,  and  transformed  into 
a specific  process  model  for  each  project.  The  process  model  provides  a framework  for  educational  technology 
projects,  specifying  the  timeline  of  the  project,  involved  participants,  and  information  technologies.  It  supports 
conceptual  and  implementation  decisions.  Furthermore,  it  can  be  used  to  integrate  learning  processes  and 
business  processes  to  close  the  traditional  gap  between  education  and  every-day-work  (Adelsberger,  Komer, 
Pawlowski,  1998).  The  third  level  is  the  result  of  the  development  process,  resulting  in  certain  implementations 
on  each  level.  The  result  of  ELM-C  is  a detailed  network  of  learning  objectives  and  goals.  The  structure  and 
relations  of  learning  sequences  (courses)  are  determined.  Based  on  these  results,  learning  sequences  are 
developed  in  ELM-D.  The  focus  of  this  phase  is  to  find  an  adequate  didactical  method  including  the  right 
technology  depending  on  learning  objectives  and  the  user  group.  Single  learning  units  are  designed  and 
implemented  in  ELM-E. 

ELM  follows  the  approach  of  the  spiral  model.  By  using  this  approach,  changes  concerning  the  needs, 
requirements,  and  wishes  of  the  project  participants  can  be  integrated  dynamically.  Furthermore  the  user  is 
involved  in  the  development  process,  evaluating  prototypes  and  contributing  actively  to  the  development.  By 
using  this  approach  we  ensure  the  quality  of  the  product  from  the  beginning  to  the  final  implementation. 

One  of  the  most  important  goals  is  the  interoperability  with  other  systems.  The  architecture  of  the  learning 
environment  is  consistent  to  the  Learning  Technology  Systems  Architecture  of  the  Learning  Technology 
Standards  Committee  of  IEEE  and  to  the  specifications  of  the  Instructional  Management  Systems  Project 
(EDUCAUSE,  1999).  Furthermore,  we  used  standard  tools  and  formats  for  both,  the  development  and  the 
implementation.  The  learning  processes  are  modeled  in  ARJS,  a process  modeling  tool  (Scheer,  1 998).  Our 
approach  is  not  limited  to  this  tool,  process  definitions  in  IDEF  (Adelsberger,  Komer,  1995),  UML  (Booch, 
Jacobson,  Rumbaugh,  1998),  or  object  oriented  modeling  languages  can  be  integrated  easily.  We  have 
implemented  a web  based  learning  environment  in  XML.  XML  (Extensible  Markup  Language)  is  a text-based 
language,  describing,  exchanging,  and  presenting  structured  data  objects  (e.g.,  data,  applications)  (W3C,  1999). 
Using  this  approach,  web  based  learning  environments  can  be  combined  and  integrated  with  other  resources. 

In  the  following  I present  the  modules  of  ELM  in  detail  and  a successful  implementation:  a learning  environment 
for  students  of  Business  Computing  at  the  University  of  Essen.  A learning  sequence  on  “Simulation”  was 
implemented  and  is  being  evaluated. 


ELM-C  - Curriculum  Development 

The  ELM-C  module  starts  with  an  analysis  of  the  context.  Educational  and  structural  aspects  are  investigated  in 
this  phase.  First  of  all,  overall  policies  and  strategies  are  investigated.  This  is  necessary  to  determine  the  role  of 
education  for  a specific  company  or  institution.  Then  existing  educational  concepts  and  the  corresponding 
business  processes  are  analyzed.  To  acquire  knowledge  about  the  future  users,  the  organizational  structure  is 
investigated,  determining  roles,  stereotypes,  and  responsibilities  of  the  users.  As  another  important  aspect,  the 
structure  of  existing  technologies  has  to  be  analyzed.  This  leads  to  conclusions  about  which  resources  can  be 
utilized  for  learning  processes.  Furthermore,  potential  changes  of  the  existing  structure  are  identified. 

After  this  first  phase,  the  generic  development  model  is  customized,  resulting  in  a specific  process  model  for  the 
development  process.  The  actual  project  planning,  including  team  building,  scheduling,  and  guidelines  are 
determined.  Furthermore,  the  main  processes  for  the  design,  implementation,  and  evaluation  of  the  curriculum 
(Fig.  4)  are  included  into  the  process  model. 
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Fig.  4:  Main  processes  of  ELM-C 


In  the  design  phase,  learning  goals  are  determined.  Documents  gathered  during  the  analysis  phase,  business 
processes,  and  external  sources  (such  as  reports  on  domain  trends)  are  examined.  Educators  and  domain  experts 
are  questioned,  both  formally  and  in  creativity  sessions.  The  results  are  mapped  into  a scheme,  which  is  a matrix 
of  the  learning  goal  dimension  (Bloom,  1956,  Krathwohl  et  al.,  1964,  Simpson,  1966)  and  the  abstraction  level, 
represented  in  an  ARIS  data  model.  Furthermore,  relations  between  the  learning  goals,  such  as  prerequisites  and 
conceptual  similarities,  are  modeled.  As  mentioned  before,  a learning  sequence  on  “Simulation”  has  been 
developed  based  on  our  approach.  Fig.  5 shows  a simplified  excerpt  of  a network  of  learning  goals.  Based  on  the 
network  of  learning  goals,  a possible  structure  of  learning  sequences  (courses)  is  determined,  serving  as  a 
prototype  for  user  evaluation  and  as  input  for  ELM-D. 
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Fig.  5:  Network  of  learning  goals  for  a course  on  "Simulation" 


ELM-D  - Development  of  Learning  Sequences 

The  focus  of  ELM-D  is  the  development  and  implementation  of  learning  sequences.  Based  on  the  specified 
learning  goals  from  ELM-C,  the  most  important  concepts  of  a certain  topic  are  specified  in  the  phase 
“knowledge  acquisition”.  Those  concepts  are  modeled  as  a data  model  in  ARIS  (or  the  chosen  modeling  tool). 
Furthermore  a user  model  describes  the  proposed  group  of  users.  It  contains  personal  and  job  related  data  (role, 
competencies),  learning  preferences,  characteristics,  and  a history  of  learning  activities.  This  data  is  also 


modeled  in  ARIS  and 

used  for  an  individual  adaptation  of  learning  sequences. 
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Fig.  6:  Main  processes  of  ELM-D 


The  crucial  part  of  this  phase  is  the  selection  of  a didactical  method,  using  educational  technologies.  This 
selection  process  is  rule  based,  depending  on  learning  goals,  contents,  and  user  characteristics.  The  selection 
generates  alternatives,  serving  as  a suggestion  and  support  for  the  author.  The  ARIS  process  model  furthermore 
supports  the  author  in  designing  a specific  learning  sequence.  Fig.  7 shows  a sample  model  from  the 
implemented  learning  sequence  “Simulation”.  In  this  context,  the  method  “Simulation  Game”  has  been  chosen 
based  on  the  complexity  of  the  cognitive  learning  goals,  the  experiences  of  the  user  group,  and  the  goal 
“teamwork”.  Our  methods  base  contains  the  “Goettingen  catalog  of  didactical  models”  (Flechsig,  1991)  and  the 
didactical  models  from  (Paulsen,  1995).  Thereafter,  the  conceptual  and  implementation  design  of  the  specific 
CSLE  is  generated. 


Fig.  7:  Learning  sequence  on  „Simulation“ 


ELM-E  - Development  of  Learning  Units 

Finally,  single  learning  units  are  designed  and  implemented  in  ELM-E.  A pattern  for  the  unit  is  generated,  not 
limiting  the  user  in  the  sense  of  Drill  & Practice  approaches,  but  serving  as  a guideline.  Depending  on  the 
learning  content,  user  group,  and  the  selected  method,  structures  for  presentation  and  interaction/communication 
are  suggested  and  implemented. 
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Fig.  8:  Main  processes  of  ELM-E 


A process  model  of  each  unit  is  generated,  serving  as  an  orientation  for  the  author  and  the  users.  Furthermore, 
this  model  can  be  integrated  in  other  systems  or,  as  suggested  in  (Adelsberger,  Komer,  Pawlowski,  1998) 
combined  with  business  processes.  The  author  is  responsible  for  the  consistency  and  final  evaluation.  Finally,  the 
CSLE  is  implemented  using  standard  authoring  tools,  or,  in  our  university,  in  XML. 


Conclusion 

The  multi-level  approach  of  the  Essen  Learning  Model  leads  to  a variety  of  supporting  features  for  both, 
developer  and  user.  All  phases  of  the  development  process  are  supported,  using  standard  methods  and  formats 
(ARIS,  LTSA,  XML).  Tliis  approach  leads  to  several  synergy  effects:  reusability,  integration,  and  coupling  with 
other  resources.  These  effects  are  essential  for  future  developments  in  the  educational  sector,  helping  project 
managers,  designers,  and  authors  of  cooperative  educational  programs.  The  use  of  a development  model  will 
ensure  the  quality  of  educational  technologies,  helping  companies  and  institutions  becoming  learning 
organizations  and  meeting  the  requirements  of  the  future. 
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Abstract:  Technical  developments  in  the  compression  and  streaming  of  audio  and 
video  over  networks  commonly  available  to  instructors  is  leading  to  a new  cycle  of 
interest  in  the  educational  applications  of  audio  and  video.  In  this  new  cycle,  the 
traditional  benefits  of  audio  and  video  are  given  a wider  audience  through  their 
access  via  WWW  browsers,  but  also  more  forms  of  learning  resources  and  activities 
are  emerging.  In  this  paper  we  first  discuss  some  of  the  educational  possibilities  that 
come  from  the  combination  of  asynchronous  audio/video  and  WWW  functional ites, 
particularly  with  respect  to  a construed vi Stic  learning  context.  Secondly,  we  discuss 
a theoretical  framework  based  on  a constructivist  learning  approach  for  the  use  of 
asynchronous  audio  and  video  in  WWW-based  learning  environments.  Following 
this  we  illustrate  how  these  sort  of  environments  have  been  realised  in  the  course 
Tele-Learning  at  the  University  of  Twente. 


WWW-Based  Learning  Environments  and  Streaming  Audio  and  Video 

WWW-based  learning  environments  allow  the  integration  of  hyped  inked  information 
systems,  databases,  interactive  learning  resources,  communication  tools,  publication  tools,  and  tools  to 
support  collaborative  and  individualised  learning.  Each  of  these  types  of  activities  can  be  supported  in 
a variety  of  modalities,  including  text,  images,  animations,  audio,  and  video.  New  technical 
developments  in  WWW-based  technologies  allow  the  use  of  video  and  audio  on  demand  (as  opposed  to 
the  two-way  video  and  audio  communication  streams  that  occur  in  conferencing  situations).  These 
developments  mean  that  asynchronous  digital  video  and  audio  sequences,  even  of  considerable  size, 
can  now  be  integrated  into  WWW-based  learning  environments,  and  used,  in  whatever  segments  that 
are  wished  and  whenever  desired,  for  appropriate  learning  goals  (Collis  & Peters,  1999).  The  technical 
advances  that  allow  video  and  audio  to  be  integrated  within  WWW  environments  and  to  be  available 
on  demand  to  learners  are  of  educational  consequence  if  they  lead  to  learning  experiences  that  are 
perceived  as  valuable  and  appropriate  for  particular  learners,  subject  areas,  and  learning  conditions. 

In  particular,  with  the  integration  of  asynchronous  audio  and  video  in  an  WWW-based 
learning  environment  different  types  of  constructivist  learning  activities  can  take  place.  Simons  (1998) 
describes  constructivistic  learning  environments  as  environments  where  students  are  being  challenged 
to  learn  actively  together,  with  a clear  focus  on  the  functionality  of  learning  and  the  learning  content. 
Within  such  environments  the  learning  is  situated  in  a concrete  context  where  the  authenticity  of  that 
context  is  as  high  as  possible.  For  example,  a didactical  model  based  on  principles  of  situational 
cognition  (Brown  & Duguid,  1994)  could  be  realised  through  the  capacity  of  video  and  audio  learning 
resources  to  connect  a learning  task  with  a realistic  context.  A didactic  approach  based  on  anchored 
instruction  (Ward,  1998)  could  offer  information-rich  simulations  and  case  studies  with  which  the 
learner  could  interact.  A didactic  model  reflecting  principles  of  cognitive  apprenticeship  (Collins, 
Brown,  & Newman,  1989)  could  be  facilitated  by  presenting  groups  of  learners  a variety  of 
information-rich  resources  that  show  an  expert  in  action  in  related  problem  areas.  A didactic  approach 
representing  crisscrossing  landscapes  (Simons,  1998)  could  be  enriched  by  making  available  a large 
variety  of  audio  and  visual  views  of  a problem  domain.  A didactic  approach  reflecting  reciprocal 
teaching  (Scardamalia  & Bereiter,  1989)  could  be  supported  by  students  creating  their  own  audio  and 
video  resources  and  making  them  available  as  learning  materials  for  their  peers. 

In  each  of  these  approaches,  students  are  expected  to  steer  and  control  their  learning  as 
independently  as  possible.  The  teacher  is  more  a coach  and  guide  than  a transmitter  of  information. 


Audiovisual  and  simulated  realities  offer  the  student  the  possibility  to  come  closer  to  the  concrete 
reality  and  thus  to  learn  in  a practical  and  experiences-centered  perspective.  We  believe  that  these 
learning  approaches  can  be  attractively  supported  in  WWW  environments  that  combine  the  strengths  of 
streaming  video  with  the  capabilities  of  the  WWW  to  support  communication,  offer  links  to  associated 
resources,  and  allow  re-use  of  video  segments  in  other  learning  contexts.  In  Table  1 we  indicate  five 
forms  of  learning  activities  which  can  reflect  constructivist  approaches  and  the  possibilities  that  come 
from  the  combination  of  asynchronous  audio  and  video  with  WWW  functionalities  in  those  learning 
contexts.  Following  Table  1,  each  of  these  forms  will  be  further  discussed  and  their  relationship  to 
constructivist  learning  illustrated. 


Video  aspects 

WWW  aspects: 

Learning 

activities 

Support  communication 

Link  to  other 
information 

Re-use  in  other 
contexts 

/.  Capturing  & 
re-using  a 
comm  uni  cati  on 
event  (such  as  a 
lecture  or  student 
presentation  or 
discussion) 

Can  be  locally  captured 
during  a presentation 

Add  forms  to  send 
questions  to  the  presenter; 
collect  frequently  asked 
questions 

Add  links  to  course 
material,  to  external 
resources 

(students/users  can 
also  add  the  links) 

Portions  of  the 
material  can  be 
ULMs  (see  below) 
for  use  in  other 
settings 

2.  Giving  access 
and  depth  to  real 
events 

Specialised  capture  of 
original  material 

Add  forms  to  send 
questions  and  e>mail  to 
experts;  participate  in 
discussion  groups 

Add  links  to  external 
and  up-to-date 
information  (students/ 
users  can  also  add 
links) 

Re-use  in  courses, 
for  professional 
development,  for 
personal  learning 

3.  Supporting  a 
learning  process 

Small  tutorial  sequences, 
documentary  type 
sequences,  interviews  with 
specialists,  etc.,  can  be 
available 

Add  forms  for  contacts 
with  others  involved  in  the 
process,  with  experts,  with 
resources  persons;  Add 
tools  to  support 
collaborative  learning  if 
process  involves  a group 

Add  links  to  examples, 
to  support  materials; 
Products  resulting 
from  the  process  can 
be  also  entered  as 
objects  of  study  in  the 
database 

Re-use  products 
made  previously 
as  examples  or 
models  for  those 
starting  out  on  a 
process 

4.  Developing  and 
accessing  a 
library  of  cases  or 
units  of  learning 
material  (ULMs) 

Cases  or  ULMs  including 
video  segments  can  be 
captured  in  a variety  of 
ways,  such  as  action  with 
commentary  (Goodyear  & 
Steeples,  1999) 

Add  forms  for  e-mail 
contacts 

Store  in  a database; 
provide  links  on  the 
ULMs  to  relevant 
supporting  materials 
and/or  to  other  items 
in  the  database 

Re-use  in  courses, 
for  self-study,  for 
professional 
development,  for 
personal  learning 

5.  Constructing 
and  sharing  one 's 
own  resources 
(ULMs) 

Video  capture  and 
handling  is  now  available 
to  the  layman 

Add  forms  for  e-mail 
contacts 

Link  products  made  to 
a WWW  site,  so  that 
accessible  and  useful 
to  others 

If  added  to  a 
database,  re-use 
by  others 

Table  1:  New  forms  of  learning  activities,  related  to  video  aspects  and  WWW  functionalities  (Collis  & 
Peters,  1999) 


Capturing  & re-using  a presentation 

There  are  many  examples  now  of  how  a classroom  or  lecture-hall  presentation  can  be  captured 
as  it  is  delivered  (usually  digitally),  stored  as  digital  video  on  a video  server,  and  then  made  available 
for  re-view  or  re-use  via  a WWW  site.  The  intended  users  include:  students  and  non-students  who 
could  not  be  present  at  the  actual  presentation  and  students  who  were  present  but  wish  to  review 
portions  of  the  presentation.  The  presentation  may  be  made  available  in  linear  form  (with  a controller 
to  slide  forward  or  backward  within  the  video  sequence),  but  most  usually  is  also  synchronised  with 
slides  or  presented  along  with  an  index  relating  to  different  portions  in  the  video  sequence,  so  that  the 
user  can  jump  immediately  to  the  portion  desired.  While  the  captured  sequences  can  be  planned  and 
executed  by  the  instructor,  they  can  also  be  captured  in  less-formal  communication  settings,  such  as 
when  students  discuss  different  points  of  view  relating  to  an  issue  or  articulate  their  ideas  and  reactions 
to  a complex  learning  situation.  Students  can  easily  do  such  capturing  themselves  with  a digital  video 
camera.  These  reflections  can  then  be  revisited,  commented  upon,  and  extended  in  asynchronous  print 
communications  and  added  to  course  resources  available  via  a course  WWW  site  for  reuse,  such  as  in 
Categories  4 and  5 in  Table  1 (see  examples  to  follow). 


Giving  access  and  depth  to  real  events 
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There  are  many  examples  of  WWW  sites  which  focus  on  particular  real  events,  as  they  are 
happening  or  extended  in  time  before  or  after  they  happen,  in  which  video  segments  play  a major  role. 
One  example  is  the  is  the  site  of  America's  Health  Network  (http://ahn.com/liveevents/default.asp), 
aimed  primarily  at  health  professionals  or  medical  students,  but  also  for  the  layman  with  a particular 
medical  interest.  At  regularly  scheduled  intervals,  via  real-time  video  streaming,  users  can  watch  an 
operation  as  it  is  occurring  and  hear  the  comments  being  made  by  the  medical  team  as  they  carry  out 
the  operation.  Following  this,  the  operation  segments  in  edited  form  are  available  as  asynchronous 
streaming  video  segments  via  the  WWW  site,  as  well  as  other  video  segments  including  interviews 
with  the  patients  and  their  doctors,  discussions  among  medical  specialists,  and  interviews.  Any  event 
can  be  captured  as  a digital  video  stream,  and  then  used  as  basis  for  students  to  add  elaborative 
materials  and  reflective  comments  via  a WWW  page.  Such  a page  can  start  as  a template  that  students 
then  fill  with  appropriate  extension  resources,  or  students  can  construct  such  a page  themselves  with  an 
html  editor. 


Supporting  a learning  process 

When  the  focus  of  a learning  activity  is  a process,  rather  than  the  learning  of  pre-defined 
content,  WWW  sites  integrated  with  video  segments  can  play  a major  role  in  learning  support.  An 
example  is  the  site  Biological  Timing  Online  Science  Experiment 
(http://www.cbt.virginia.edu/01h/exD.html)  from  the  University  of  Virginia.  This  site  is  designed  to 
help  students  in  the  process  of  carrying  out  scientific  experiments  by  showing  them,  when  they  have 
need  of  the  information,  (via  video  fragments)  many  examples  of  aspects  of  an  experiment,  including 
aspects  relating  to  the  reasons  for  carrying  out  the  process  and  aspects  relating  to  the  setting  up  and 
conducting  of  experiments. 


Developing  and  accessing  a library  of  cases  or  units  of  learning  material  (ULMs) 

The  site  Learn  & Live  (http://www.glef  org/leamlive/toolkit/clearview.html)  is  an  example  of 
how  a collection  of  video  segments,  with  voice  overlay,  can  serve  as  cases,  or  units  of  learning 
materials,  related  to  a particular  theme  or  context.  In  the  Learn  & Live  context,  the  target  group  is 
teachers  and  the  focus  is  on  how  4th-  and  5th- grade  students  conduct  a research  project  when  they  have 
the  opportunity  to  work  collaboratively  with  university  scientists.  In  each  of  the  cases  linked  to  the  site, 
a video  segment  showing  the  children  carrying  out  various  learning  activities  is  available,  with  voice- 
over comments  by  either  the  teacher  or  the  scientists.  The  intention  of  the  site  is  that  others  (teachers, 
teachers-in-training,  scientists,  and  those  who  design  such  special  collaborative  projects)  can  see  what 
actually  occurs  in  the  classroom  during  such  an  interaction. 


Constructing  and  sharing  one*s  own  video  & WWW  resources 

From  a constructivist  perspective  in  which  learning  by  doing  is  emphasised,  the  possibilities 
for  learners  to  create  their  own  video  segments  as  well  as  the  surrounding  WWW  environments 
through  which  those  segments  are  available,  can  be  an  example  of  a new  type  of  collaborative  learning 
activity.  Such  creation  can  relate  to  any  and  all  of  the  other  four  categories  of  learning  activities  in 
Table  1.  In  the  next  section,  an  illustration  is  given  of  how  all  five  of  these  types  of  learning  activities 
have  been  realised  in  the  course  Tele-Learning  at  the  University  of  Twentc. 


Illustration:  The  course  Tele-Learning  at  the  University  of  Twente 

The  course  Tele-Learning  in  the  Faculty  of  Educational  Science  and  Technology  (see 
http://education2.edte.utwente.nl/98193524.nsf/framesform7readform)  at  the  University  of  Twente  in 
The  Netherlands  is  a senior-level  elective  course  focusing  on  the  educational  use  of  network 
applications.  A major  way  in  which  the  learning  takes  place  in  this  course  is  through  the  students 
working  together  in  groups  on  a project  for  a real  context  and  audience  and  in  response  to  a real  need 
of  that  audience.  In  the  1999  cycle  of  the  course,  the  audience  was  instructors  in  our  own  faculty  and 
the  context  was  to  help  them  become  aware  of  opportunities  in  their  own  classes  (all  of  which  make 
use  of  a WWW-based  course-support  system,  called  TeleTOP,  http://telctop.edte.utwente.nl)  in  which 
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video  integrated  with  their  WWW  sites  could  bring  added  value  to  their  courses.  Each  of  four  groups  of 
students  in  the  class  had  to  develop  a different  WWW-based  performance-support  site  for  the 
instructors,  in  which  a form  of  streaming  video  use  integrated  within  an  educational  WWW  site  was 
illustrated  and  described  and  defended  from  an  educational  perspective.  The  forms  chosen  by  the 
groups  included  video  as  part  of  WWW-based  tutorials  for  various  learning  processes  and  video  used 
for  action  with  commentary  to  help  students  work  together  to  handle  a particular  problem  situation. 
Thus  the  students  themselves  participated  in  a learning  experience  as  indicated  in  the  fifth  row  of  Table 
1 (constructing  their  own  uses  of  video),  but  with  a focus  on  showing  examples  of  the  third  and  fourth 
rows  in  the  table  (video  to  supporting  a process,  and  video  to  illustrate  examples  from  practice).  In 
addition,  during  the  final  meeting  of  course,  each  group  of  students  presented  their  WWW  sites  and 
reflected  upon  their  own  impressions  of  the  educational  value  of  video  integrated  in  WWW 
environments  and  were  captured  on  digital  video  during  the  presentation.  These  captured  presentations 
(Category  1 in  Table  I)  will  add  to  the  meaningfulness  of  the  performance-support  sites  for  the 
instructors  who  subsequently  use  the  sites.  The  students  captured  their  own  video  sequences  rather  than 
using  professionally  prepared  video,  using  only  a handheld  digital  video  camera  and  simple  video- 
editing software.  They  learned  the  html  necessary  to  insert  a streaming  video  segment  into  a WWW 
page  and  were  responsible  for  the  design  and  construction  of  the  overall  WWW-based  performance- 
support  environment. 


A video-illustrated  case  as  part  of  WWW-based  tutorials 

An  example  of  one  of  the  performance-support  sites  made  by  the  groups  of  students  in  the 
Tele-Learning  course  for  instructors  in  their  faculty  is  shown  in  Figure  1.  This  group  of  students  chose 
to  develop  a video-illustrated  case  study  as  part  of  WWW-based  tutorial.  They  developed  a WWW- 
based  tutorial  with  video  that  combines  the  benefits  of  video,  self-study  activities,  and  the  WWW.  The 
tutorial  mode  of  instruction  guides  the  learner  throughout  the  material,  the  video  adds  visualisation,  and 
the  WWW  environment  allows  more  flexibility  in  time,  place,  sequence,  and  pace  of  learning  as  well 
as  adding  communication  support  relating  to  the  content  of  the  video.  The  site  has  the  following 
generic  structure  that  all  the  four  groups  in  the  class  used:  (a)  a hyperlinked  set  of  pages  containing  the 
instructional  example  itself  - in  this  case,  the  example  of  a WWW-based  tutorial  integrated  with 
streaming  video;  (b)  educational  issues  - an  essay  on  learning-related  issues  surrounding  the  use  of  this 
sort  of  WWW  environment  for  situated  learning,  anchored  instruction,  and  other  forms  of 
constructivist-oriented  learning  as  well  as  instructivist  learning;  (c)  design  issues  - a discussion  of 
guidelines  for  designing  a WWW  site  that  uses  video  in  a WWW-based  tutorial;  (e)  technical 
procedures  - steps  to  be  taken  when  making  video  for  the  WWW;  (f)  external  examples  - links  to  other, 
external,  examples  of  WWW-based  tutorials  with  video;  and  (g)  group  members  - information  about 
the  students  that  worked  on  the  project.  The  topic  used  for  illustration  purposes  of  the  performance- 
support  site  shown  in  Figure  1 was  ‘Shooting  a good  video!’  The  students  designed  an  environment 
that  contained  resources  for  learners  to  explore  the  characteristics  of  video,  as  well  as,  how  to  take 
these  characteristics  into  account  when  creating  good  educational  videos  for  the  WWW.  The  study 
materials  include  an  option,  called  ‘’Common  problems”  where  instructors  can  find  out  what  students 
usually  don’t  like  about  educational  videos.  A second  option  is  a case  study  where  instructors  are 
prompted  to  think  about  how  to  shoot  a good  video  for  the  WWW,  followed  by  a presentation  where 
the  instructors  have  the  chance  to  study  the  lesson  materials  and  a quiz  to  test  themselves.  Other 
options  available  are  links  to  resources  for  further  learning  and  a forum  where  instructors  can  share 
their  ideas  about  making  video  for  the  web. 


Action  with  commentary 

An  another  example  of  a performance-support  site  made  by  the  groups  of  students  in  the  Tele-Learning 
course  for  instructors  in  their  faculty  is  shown  in  Figure  2.  This  group  of  students  developed  a WWW 
environment  with  asynchronous  video  to  show  how  such  an  environment  can  assist  students  learning  to 
handle  a complex  learning  task.  In  this  case,  the  illustration  chosen  of  such  a task  was  the  experiences 
of  a foreign  student  coming  to  study  at  the  University  of  Twente.  The  environment  includes  the  audio 
of  a moderator,  who  gives  comments  about  the  most  important  things  that  foreign  students  have  to 
know  about  the  campus  in  order  to  handle  problems  that  arise  when  the  student  first  has  to  negotiate 
using  the  library,  the  computing  center,  the  system  for  using  swipe  cards  to  get  in  and  out  of  buildings, 
etc. 
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Figure  1:  An  example  of  a WWW-based  performance-support  site  illustrating  the  use  of  video 
integrated  with  self-study  resources,  made  by  a group  of  students  in  the  Tele-Learning  course 


The  purpose  of  the  embedded  video  segments  in  the  site  is  to  help  students  learn  about  various 
complex  situations  by  having  a voice-over  commentary  guide  the  student  through  visualizations  of 
those  situations.  Links  are  available  for  additional  information,  campus  maps,  contact  names  and  email 
addresses,  and  other  resources  to  help  the  student  in  the  problem  situations.  As  four  of  the  five 
members  of  the  group  were  in  fact  foreign  students  for  whom  the  Tele-Learning  course  was  their  first 
course  at  the  University  of  Twente,  they  had  a personal  context  for  the  problem  situation.  The  video  in 
this  case  captured  the  information  that  had  to  be  communicated  to  the  learner  in  a practical  and 
illustrative  way.  In  their  reflections,  the  students  indicated  that  they  believed  that  this  type  of  learning 
resource  involves  a number  of  educational  benefits  for  the  learner.  The  first  scene  of  the  video 
generates  interest  and  motivation.  The  video  supports  a situated-learning  environment  that  includes 
several  complex  themes:  the  arrival,  procedures  for  using  food  and  drink  automates,  the  computer  lab, 
the  cafeteria,  and  some  general  information  about  the  faculty.  The  video  and  WWW  site  combined 
facilitates  the  possibility  of  exploratory  learning.  The  use  of  knowledge  is  increased  by  the  different 
contexts.  The  students  also  felt  that  the  process  of  making  the  site  themselves  was  a valuable  learning 
experience  for  them  as  well  as  being  of  potential  value  for  others. 

In  addition  to  designing  and  producing  their  own  WWW-sites  integrated  with  video  as 
examples  for  instructors,  the  students  in  the  Tele-Learning  course  also  made  use  of  video  in  many  other 
ways  within  the  course.  Video  integrated  within  educational  WWW  sites  was  the  content  of  the  course 
as  well  as  a medium  for  the  content  of  the  course;  thus  the  students  used  the  course  WWW 
environment  as  a framework  in  which  to  place  their  links  to  examples  of  external  WWW  sites  making 
good  educational  use  of  video  (in  all  of  the  categories  in  Table  I);  their  evaluative  and  reflective 
comments  about  the  use  of  video  in  WWW  sites  for  learning  purposes;  and  for  the  study  of  captured 
presentations  of  the  students  in  the  previous  cycle  of  the  course.  These  presentations  captured  the 
students  during  the  final  class  session  of  the  1998  cycle,  and  made  these  comments  available  to  the 
1999  students  as  asynchronous  video  synchronised  with  presentation  slides  made  by  the  students 
related  to  different  aspects  of  the  course.  The  students  in  the  1999  cycle  of  the  course  could  thus  see 
and  hear  their  peers  in  the  1998  cycle  comment  on  their  own  work  (the  work  was  also  available  to  the 
students  in  the  1999  cycle  as  learning  resources,  another  example  of  re-use  of  video  segments).  During 
the  last  session  of  the  1999  cycle,  this  same  process  of  capturing  on  video  the  presentations  and 
reflections  of  the  students  and  making  these  immediately  available  for  re-view  and  re-use  was 
continued. 
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Figure  2:  An  example  of  video  in  the  form  of  action  with  commentary  site  made  by  a group  of  students 
in  the  Tele-Learning  course 


Conclusion 


Many  of  the  new  technological  developments  in  education  involve  convergences:  of  the 
learner  as  both  consumer  and  producer  of  his  or  her  own  learning  resources;  of  linear  and  interactive 
media  in  a single  WWW  environment;  of  communication,  collaboration,  and  learning  within  contexts 
that  integrate  theory  and  realistic  applications.  These  trends  can  be  seen  in  the  uses  of  asynchronous 
video  within  WWW  environments  that  are  now  emerging,  particularly  when  the  video  is  developed  or 
captured  by  the  learners  themselves  as  a study  resource  and  used,  on  demand  in  an  interactive  manner 
by  other  learners.  These  sorts  of  learner-captured  video  resources  are  now  becoming  easily  available 
with  advances  in  desktop  video  technologies  and  the  WWW;  what  is  only  just  emerging  is  an 
assessment  of  their  educational  potential.  The  promising  technical  developments  are  creating  a strong 
technology  push,  which  needs  to  be  confronted  by  an  even  stronger  educational  rationale. 
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Abstract:  In  this  paper  we  describe  an  effective  way  to  create  a web-based  self  assessment 
tool.  Our  approach  uses  an  exercise  applet  that  reads  and  extracts  questions  from  their 
descriptions  in  XML  format  and  presents  them  to  the  students  depending  on  students’ 
preferences.  A simple  authoring  tool  is  used  for  adding  questions  and  corresponding  hints  to 
the  exercise  environment.  Questions  can  be  accompanied  by  additional  media  such  as  video, 
picture,  sound  or  animation  applets. 


Introduction 

In  the  past,  distance  education  has  always  suffered  from  a lack  of  opportunities  for  hands-on  experimentation 
and  access  to  physical  objects,  models,  simulations  and  instruments  in  computer  and  engineering  labs.  Today, 
computers  in  education  and  networked  learning  environments  offer  new  ways  of  discovery  learning  through 
some  unique  forms  of  interaction  with  virtual  objects,  online  laboratories  at  home,  learner-directed  animation 
of  realistic  processes,  and  light-weight  software  tools.  In  networked  learning  environments  interactivity  allows 
students  to  manipulate  input  data,  parameters  and  thresholds  of  real-time  simulations  or  animations.  Learners 
may  also  influence  the  progress  and  output  of  process  or  algorithm  visualizations.  The  project  group  at 
FemUniversitaet  Hagen  is  exploiting  some  of  these  possibilities  with  the  development  of  a series  of  exercise 
environments  (Mogey  1997)  that  let  students  apply  and  experience  what  they  have  learned  to  predefined 
problems  or  problems  of  their  choice.  Thereby  we  expect  to  raise  the  attractiveness  of  the  learning  material  and 
enhance  the  students’  motivation,  especially  that  of  distance  learners.  In  the  design  of  these  environments  we 
have  attempted  to  associate  learning  objectives  with  appropriate  multimedia  solutions  (Williams  1997)  with 
respect  to  different  teaching  functions  such  as  motivation,  presentation  of  new  structures,  practicing  new 
behavior,  modeling,  exploration  by  trial  use  and  error,  or  application  of  new  techniques.  To  achieve  a high 
degree  of  platform  independence,  ease  of  use  and  portability  and  to  remain  independent  from  proprietary 
formats,  it  was  decided  to  rely  on  HTML,  XML  (Harold  1998,  Idris  1999)  and  Java  (Horstmann  1998,  Sun 
1 998)  as  core  technologies  for  implementation  of  the  exercise  labs. 

Since  the  introduction  of  Java  programming  language  we  can  see  the  use  of  applets  in  traditional  hypertext 
systems  as  well  as  in  an  Intelligent  Tutoring  Systems.  Java  applets  are  employed  very  often  to  explain  complex 
processes  to  deepen  the  understanding  of  difficult  parts  of  educational  documents.  However,  the  use  of  Java 
applets  does  not  have  to  be  restricted  to  simulations  or  visualizations  of  specific  problems.  Many  examples 
show  that  applets  can  be  used  for  exercises  and  tests  and  therefore  offer  more  flexibility  than  common  media. 
Another  advantage  of  applets  is  the  possibility  to  accept  parameters  so  that  they  can  be  tailored  to  different 
learner  types.  It  should  be  possible  to  find  common  learning  oriented  descriptions  for  different  users  of  the 
exercise  applet  and  to  make  the  exercise  applet  itself  adaptable  to  specific  needs.  That  way  only  one  version  of 
the  applet  would  be  necessaiy  for  many  different  kinds  of  users. 

A Generic  Exercise  Environment 

After  having  developed  several  exercise  environments  by  hand,  we  realized  that  we  need  many  more  exercises 
for  the  students  to  make  the  self  assessment  effective.  Developing  an  exercise  applet  for  each  specific  topic  was 


not  very  effective  if  we  want  to  generate  a lot  of  questions  for  different  topics.  Another  possibility  is  to  create 
HTML  based  questions  with  applets  as  companion,  which  are  often  found  in  web-based  exercises.  The 
downside  of  these  HTML  questions  is  that  they  are  static,  which  means  each  time  the  exercise  is  restarted  the 
students  will  receive  the  same  set  of  questions.  What  we  were  looking  for  is  the  ability  to  create  a dynamic 
exercise  environment,  that  can  pick  a random  set  of  questions  from  a question  pool,  adapt  to  the  student’s 
preferences  and  that  is  easy  to  use  for  the  tutors  creating  the  questions.  This  approach  would  also  separate  the 
questions  from  the  applets  and  use  the  applets  in  conjunction  with  the  questions  in  an  exercise  environment. 
The  applets  should  show  an  animation  or  a simulation  of  a certain  topic,  and  the  question  can  be  asked  in 
relation  not  only  to  an  applet  but  can  also  relate  to  other  media  such  as  video,  picture,  sound.  We  chose  to 
develop  an  XML  based  exercise  environment  since  XML  is  ideal  to  describe  the  question  structure,  the  exercise 
structure  and  the  structure  of  a student’s  profile.  The  idea  was  to  describe  each  question  in  XML  and  let  those 
questions  be  parsed  by  the  exercise  environment  applet  and  be  presented  in  an  appropriate  format  depending  on 
the  question  type  and  companion  media  during  the  exercise.  That  way  a large  amount  of  question  pool  can  be 
generated  more  easily.  The  configuration  of  an  exercise,  such  as  how  many  sections  or  how  many  questions, 
what  kind  of  question,  etc.  is  also  described  in  XML  and  parsed  into  an  exercise  specification  object  used  by  the 
exercise  environment  applet  at  the  beginning  of  an  exercise.  The  exercise  environment  covers  both  formative 
and  summative  assessment  types  (Bull  1996,  Williams  1997).  This  approach  provides  flexibility  to  authors, 
since  any  applets  that  the  authors  find  useful  for  the  exercise  can  be  added  any  time  to  the  applets  pool  as  a 
companion  to  the  questions.  The  same  rules  apply  to  pictures  and  video  clips.  With  this  flexibility  comes  a big 
task  of  putting  a lot  of  questions  into  the  exercise  environment.  To  give  a picture  of  it  let’s  take  an  example  of 
an  exercise  for  one  topic  with  one  difficulty  level,  five  question  types  (true/false,  multiple  choice,  text  gap,  set 
ordering,  matching)  with  five  questions  for  each  type.  That  will  result  in  at  least  25  question  files  and  50 
picture  files  and  not  to  forget  the  hints  and  explanations  for  errors  and  companion  applets.  In  order  for  the 
exercise  environment  to  select  a different  set  of  questions  each  time  the  students  restart  the  exercise,  more  than 
25  questions  have  to  be  provided  to  enable  a random  pick  out  of  the  question  pool.  As  soon  as  enough  questions 
are  available,  the  exercise  environment  can  give  the  students  an  adaptive  and  interesting  platform  to  do  self 
assessment.  Currently  for  evaluation  purposes  we  have  created  about  150  question  files,  150  pictures,  a VRML 
scene,  some  companion  applets  and  JavaScript  animations  for  6 topics,  which  in  turn  consists  of  several 
subtopics. 

The  exercise  environment  consists  of  3 modules  or  applications: 
a)  Question  generator 

A Java  application  with  forms  to  create  a question.  Five  of  the  most  common  question  types  in  Computer 

Based  Training  (CBT)  can  be  generated  such  as:  true/false,  multiple  choice,  fill  text,  matching  and  set 

ordering.  An  XML  Document  Type  Definition  (DTD)  was  defined  to  describe  the  structure  of  questions. 

Important  parts  in  a question  description  include: 

> Domain:  defines  to  which  domain  or  topic  the  question  belongs. 

> Create:  defines  the  date  of  creation. 

> Level:  defines  the  difficulty  level  of  the  question.  5 levels  are  currently  supported. 

> Language:  defines  the  language  for  the  question. 

> Subtopic:  defines  the  subtopic  or  subchapter  of  a question. 

> Question  type:  defines  the  question  type  which  has  to  be  one  of  the  5 described  above. 

> Question:  defines  the  elements  that  describe  a question  such  as 

> Problem  statement:  defines  the  question  asked. 

> Answer  options:  defines  the  answer  options  in  question  types  multiple  choice,  matching. 

> Keys:  defines  the  answers  to  the  question. 

> Hints:  defines  the  hints,  each  hint  is  related  to  an  answer  option  and  the  hints  consist  of  3 
levels,  starting  from  a general  hint  to  a detailed  hint.  The  exercise  environment  will  show  the 
hints  depending  on  the  selected  difficulty  level.  All  3 levels  of  hints  will  be  shown  for 
exercises  in  the  first  2 lower  difficulty  levels,  2 levels  of  hints  will  be  shown  to  those  in  the 
medium  difficulty  level  and  1 hint  will  be  shown  to  those  in  the  last  2 higher  difficulty  levels. 
Hints  are  only  available  in  formative  mode  of  the  exercise.  If  the  student  chooses  to  do  the 
exercises  in  exam  mode  (summative),  no  hints  will  be  shown. 
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> Reason:  defines  the  ‘why’  explanation  of  the  answer 

> Show  media:  defines  the  media  related  to  the  question.  The  media  can  be  a picture,  a video 
clip,  audio , applet. 

> Picture  list:  defines  a list  of  picture  needed  for  the  question  type  matching  and  set  ordering. 

Example  of  a question  description  : 

<?xml  version-' 1.0"?> 

<!DOCTYPE  QUESTIONSPEC  SYSTEM  "questionspec.dtd"  > 

<QUESTIONSPEC> 

<ID>MQnetwork_mediumO</ID> 

<TITLE>test  question  chapter  14.2</TITLE> 

<CREATE>25/07/99</CREATE> 

<DOMAIN>network</DOMAIN> 

<LEVEL>medium<7LEVEL> 

<LANGUAGE>de</LANGUAGE> 

<SUBTOPIC>networks</SUBTOPIC> 

<QUESTIONTYPE>MQ</QUESTIONTYPE> 

<QUESTION> 

<PROBLEMSTATEMENT>Match  the  correct  FFDI-components  to  the  pictures.  Click  on  the  pictures 
to  view  a larger  version  of  them.</PROBLEMSTATEMENT> 
<ANSWEROPTIONS>PHY</ANSWEROPTIONS> 
<ANSWEROPTIONS>SMAP</ANSWEROPTIONS> 
<ANSWEROPTIONS>SMT</ANSWEROPTIONS> 
<ANSWEROPTIONS>MAC</ANSWEROPTIONS> 

< AN  S WEROPTIONS>PMD</AN  SWEROPTION  S> 

<K£YS>2</KEYS> 

<K£YS>3</KEYS> 

<K£YS>0</K£YS> 

<K£YS>4</KEYS> 

<K£YS>K/KEYS> 

<HINTS  K£YNUMBER="1"  HLEVEL="1"> 

<HINTTYPE>text</HINTTYPE> 

<HINTTEXT>Physical  Layer  ProtocoK/HINTTEXT> 

<HINTMEDIA>none</HINTMEDIA> 

</HINTS> 

<REASON>The  tasks  of  the  Station  Management  (SMT)  are  control,  monitoring  and  administration  of 
the  connected  stations  and  the  network.  </REASON> 

<SHOWMEDIA  MEDIATYPE="none">none</SHOWMEDIA> 

<PICTLIST> 

<PICTFILE>test/pictures/fddi  1 .gif</PICTFILE> 

<PICTFILE>test/pictures/fddi2.gif</PICTFILE> 

<PICTFILE>test/pictures/fddi3.gif</PICTFILE> 

<PICTFILE>test/pictures/fddi4.gif</PICTFILE> 

<PICTFILE>test/pictures/fddi5.gif</PICTFILE> 

</PICTLIST> 

</QUESTION> 

</QUESTIONSPEC> 

b)  Exercise  generator 

A Java  application  for  creating  the  overall  exercise  specification.  The  exercise  is  described  using 
the  following  fields: 

> Domain:  defines  the  topic  of  exercise. 

> Level:  defines  the  difficulty  level  of  the  exercise. 
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> Number  of  sections:  defines  the  number  of  sections  in  the  exercise,  5 is  the  maximum  since  there  are 
only  5 question  types  available. 

> Section  Description:  specifies  each  section  by  defining: 

> Section  type:  defines  the  question  type  of  this  section. 

> Number  of  question:  defines  the  number  of  questions  in  this  section. 

> Total  question:  defines  the  number  of  available  questions  in  the  question  pool.  If  the  number 
of  questions  needed  in  an  exercise  section  is  smaller  than  the  available  questions  in  the 
question  pool,  the  exercise  tool  will  select  the  questions  randomly  from  the  question  pool.  The 
exercise  is  less  predictable  and  becomes  more  challenging  since  the  students  do  not  get  the 
same  questions  each  time  they  start  the  exercise. 

> Description  of  test  mode: 

> Maximum  error  in  relation  to  minimum  solved  questions.  These  2 numbers  are  needed  to 
define  the  maximum  error  rate  allowed  to  pass  the  exercise  in  exam  mode. 

> Time  limit  can  be  defined  for  exam  mode. 

c)  Exercise  environment  applet 

A Java  applet  which  is  the  environment  (Fig.  1)  that  the  students  will  work  with.  A student  profile  is 
loaded  by  the  applet  at  the  beginning  of  the  exercise.  The  student’s  profile  gives  the  information  of  selected 
exercise  topic,  subtopics,  language  and  difficulty  level.  The  applet  then  loads  the  selected  exercise  scenario 
and  parses  it  using  the  IBM  XML  parser.  Using  the  extracted  information  about  the  exercise,  the  applet 
prepares  the  exercise  and  loads  the  defined  questions  one  after  another.  The  exercise  environment  creates  a 
random  number  for  each  question  category  in  order  to  provide  variation  each  time  the  exercise  is  restarted. 
In  order  to  create  a correct  random  number,  the  exercise  environment  checks  the  number  of  available 
questions  on  the  server.  If  the  number  of  available  questions  is  less  than  requested  in  the  exercise 
specification,  then  the  exercise  environment  adjusts  the  question  number  for  this  exercise  by  reducing  it  to 
the  number  of  available  questions.  Thus  preventing  the  error  of  creating  a random  number  of  a non- 
existent question.  The  XML  file  describing  the  exercise  will  not  be  modified,  since  more  questions  can  be 
added  to  the  question  pool  in  the  future,  so  next  time  the  exercise  is  started  , there  could  be  more  questions 
available  than  requested  in  the  specification.  After  all  random  numbers  for  each  question  category  have 
been  calculated,  each  question  is  loaded,  parsed  and  shown  when  the  student  arrives  at  the  specific 
question.  There  are  two  methods  to  navigate  in  the  exercise;  sequential  navigation  using  the  forward  and 
backward  buttons  and  free  navigation  by  using  the  navigation  tree.  The  navigation  tree  gives  more  freedom 
to  jump  from  one  question  to  any  other  question  in  the  exercise.  If  a certain  media  is  related  to  a question 
such  as  an  applet  showing  an  animation  or  a simulation,  a video  clip,  an  audio  file  or  a picture,  then  it  is 
shown  together  with  the  question.  Questions  can  either  be  answered  or  skipped.  Skipped  questions  are 
counted  as  unanswered  errors.  Hints  are  also  provided  and  can  be  called  during  the  exercise.  Each  possible 
answer  option  has  3 levels  of  hints  starting  from  a general  one  to  a more  detailed  one.  The  number  of  hints 
that  can  be  shown  to  the  student  depends  on  the  difficulty  level  of  the  exercise.  No  hints  are  available  in 
exam  mode.  In  exam  mode  a time  limit  can  be  activated  which  limits  the  time  given  to  the  student  to  finish 
the  exercise.  When  the  time  is  up  the  exercise  will  automatically  end  and  no  further  answers  will  be 
accepted.  Students  can  submit  the  exercise  any  time  and  get  the  result  of  the  exercise.  The  result  of  the 
exercise  has  an  HTML  format  and  is  shown  in  a separate  window.  Student’s  answers  are  listed  together 
with  the  correct  answer.  If  the  question  author  provides  the  question  with  reason,  in  case  of  error,  this 
reason  text  will  be  shown.  A summary  is  provided  at  the  end  the  result  report  showing  error  rate  and 
achieved  points.  Each  question  number  on  the  result  window  is  linked  to  the  exercise  environment  applet 
through  an  interface.  That  way  students  can  reload  the  corresponding  question  by  clicking  on  the  question 
number  in  the  result  window,  instructing  the  exercise  environment  applet  to  reload  and  parse  that 
particular  question.  A log  file  is  created  at  the  end  of  the  exercise  and  the  student’s  profile  is  also  updated. 
This  profile  shows  how  many  exercises  a student  has  done  and  how  well. 


Example  of  a student’s  profile: 

<?xml  version^”  1 .0”?> 

<!DOCTYPE  STUDENTPROFILE  SYSTEM  "studentprofile.dtd"> 
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<STUDENTPROFILE> 

<ID>jwl2345</ID> 

<FIRSTNAME>John</FIRSTNAME> 

<N  AM  E>  W ashington</N  AM  E> 

<PROFESSION>teacher</PROFESSION> 

<AGE>35</AGE> 

<LOGIN/> 

<PASSWORD/> 

<MEDIAPREFERENCE>image</MEDIAPREFERENCE> 

<STEREOTYPE>hierarchical</STEREOTYPE> 

<LEARNINGGOAL>consult</LEARNINGGOAL> 

<KNOWLEDGELEVEL>low</KNOWLEDGELEVEL> 

<LANGUAGE>de</LANGUAGE> 

<DOMAIN> 

<TITLE>communication</TITLE> 

<SUBT0PIC>layer3</SUBT0PIC> 

<SUBTOPIC>QOS</SUBTOPIC> 

</DOMAIN> 

<EXERCISEACTIVITY> 

<H0WMANYEX>2</H0WMANYEX> 

<EXRATE>0.8</EXRATE> 

<EXRATE>  1 ,0</EXRATE> 
<EXPOINT>0.2</EXPOINT> 
<EXPOINT>0.0</EXPOINT> 
</EXERCISEACTIVITY> 

<TESTACTIVITY> 

<HOWMANYTEST>  1 </HOWMANYTEST> 

<TE  STRATE>0 ,4</TESTRATE> 
<TESTPOINT>0.6</TESTPOINT> 
</TESTACTIVITY> 

</STUDENTPROFILE> 
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Figure  1 : a screenshot  of  the  exercise  environment  applet 

Conclusion  and  outlook 


In  this  paper  we  described  an  XML  based  exercise  environment  applet,  a simple  question  generator  tool  and  an 
exercise  specification  generator  tool  to  create  interactive  web  based  exercises  for  self  assessment.  Our  approach 
extends  the  usual  static  HTML-based  exercises  and  provides  more  variation  and  a higher  number  of  questions 
in  an  easy  and  efficient  way.  At  the  moment  the  exercise  environment  is  being  integrated  into  a CD-ROM 
based  multimedia  technology  lessons.  This  material  will  be  distributed  to  buyers  of  the  book  and  students  of  the 
course  Multimedia-Technologie  (Steinmetz  1998)  and  will  be  evaluated  for  usability  and  learning  effectiveness. 
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Abstract:  This  paper  describes  some  design  principles,  main  actors  and  functions  of  an  open 
learning  environment  called  A&O.  A&O  is  an  open  learning  environment  which  can  be  used  via 
computer  networks,  especially  over  the  Internet.  It  supports  both  self-paced  and  collaborative 
learning.  A&O  is  open  to  various  learners  and  different  contents  and  it  offers  an  opportunity  to 
study  independently  of  time  and  place.  The  main  goal  of  the  A&O  environment  is  to  be  both 
pedagogically  appropriate  and  technically  functional.  It  is  designed  on  the  basis  of  modem  learning 
theories  that  emphasise  the  constmction  of  new  knowledge  on  the  basis  of  previous  knowledge,  the 
learners'  active  role  and  collaborative  learning.  In  addition  to  these,  learner's  intentions,  meaningful 
contexts,  transfer  and  reflection  play  an  essential  role.  The  system  architecture  of  A&O  consists  of 
data  management  facilities  including  data  acquisition  tools,  tools  for  communication  and 
collaboration,  course-dependent  cognitive  tools  and  authoring  tools  for  producing  hypermedia 
courseware.  In  this  paper  the  first  learning  experiment  in  Matrix  Algebra  with  A&O-leaming 
environment  is  reported. 


1 Introduction 

The  purpose  of  this  article  is  to  present  the  design  principles,  implementation  and  evaluation  of  the  hypermedia- 
based  A&O  learning  environment  (see  also  Pohjolainen,  Ala~Rantala,  Nykanen  &,  Ruokamo  1999).  The  design  of 
the  learning  environment  is  based  on  the  exploitation  of  the  broadband  information  networks’  characteristics,  on 
hypermedia-based  learning  materials  and  modem  learning  theories  (see  Ruokamo-Saari  & Pohjolainen  1997; 
Ruokamo  & Pohjolainen  1998,  1999).  In  order  to  achieve  optimal  exploitation  of  the  possibilities  provided  by 
modem  information  technology,  theories  of  learning  must  be  incorporated  into  planning  of  the  learning 
environment.  In  the  research  on  technology-based  learning  environments  constructivism  has  gained  ascendancy 
among  theories.  According  to  constructivist  theory  learning  is  a constructive  process  in  which  the  learner  has  an 
active  role  and  learning  is  based  on  the  learner’s  own  cognitive  activity.  Students  construct  new  knowledge  on  the 
basis  of  previous  knowledge,  on  their  own  activity  in  constant  interaction  with  the  surrounding  reality.  The  A&O 
learning  environment  was  planned  to  be  applicable  to  different  students  and  different  content  fields  and  to  be  used 
independent  of  time  and  place.  Such  an  open  learning  environment  supports  lifelong  learning.  Open  and  distance 
learning  (ODL)  based  on  the  information  networks  and  on  hypermedia  can  be  utilised  in  schools  and  universities,  in 
organisations  offering  education  and  in  entrepreneurial  education  - from  childhood  to  mature  students.  This  article 
presents  the  objectives  and  design  premises  of  the  research  and  development  work  of  the  A&O  environment.  The 
article  describes  the  A&O  spaces,  actors  and  tools,  and  partly  its  technical  realisation.  Finally  the  first  learning 
experiment  in  A&O  environment  is  described  and  the  results  are  presented. 


2 Objectives 

The  A&O  open  learning  environment  refers  here  to  a hypermedia-based  environment  which  can  be  used  via  the 
information  networks,  especially  the  Internet,  and  which  supports  self-directed  and  collaborative  learning.  The 
learning  environment  is  open  to  different  students  and  different  content  areas.  It  moreover  provides  the  opportunity 
for  study  via  the  information  networks,  independent  of  time  and  place  and  for  lifelong  learning.  In  the  learning 


environment  an  attempt  has  been  made  to  cater  for  mental,  physical  and  social  activity  and  it  is  intended  to  be  both 
pedagogically  appropriate  and  technically  functional. 


3 Premises  in  Planning  an  Open  Learning  Environment 

This  section  presents  the  basic  structure  of  A&O:  actors,  spaces  and  tools  and  offers  a description  of  the  functions  of 
the  Office  and  Study. 

3.1  Actors 

In  planning  A&O  an  effort  was  made  to  cater  for  all  those  actors  who  operate  in  a conventional  learning 
environment.  Thus  it  is  possible  to  ensure  that  the  environment  enables  and  supports  the  contribution  of  the  various 
actors  in  the  learning  process.  The  main  actors  in  the  open  learning  environment  are:  the  student,  the  student  tutor, 
the  teacher,  the  teacher  tutor,  the  expert,  the  content  provider,  the  director  or  rector,  the  secretary,  the  researcher, 
the  system  administrator  and  the  visitor.  The  role  of  the  student  is  to  function  as  a learner  on  the  courses.  S/he  is  a 
main  actor  in  the  learning  environment  and  is  also  responsible  for  his/her  own  learning.  The  role  of  the  teacher  and 
other  actors  is  to  support  the  student  in  his/her  learning  process.  The  teacher  on  the  course  functions  as  the  teacher 
responsible,  as  supervisor  and  in  the  role  of  evaluator.  The  student  tutor  is  a student  who  has  already  completed  the 
course  in  question.  S/he  may  counsel  the  students  and  advise  them  in  matters  pertaining  both  to  study  and  practical 
problems.  The  teacher  tutor  is  an  assistant  teacher  whose  function  is  to  assist  the  teacher  responsible  for  the  course 
and  to  supervise,  and  further  to  support  and  guide  the  student  in  his/her  studies.  The  expert  is  a person  who  has 
particular  knowledge  or  skills  related  to  the  content  area  of  the  course.  The  task  of  the  expert  may  be  to  offer  advice, 
to  participate  in  discussion  groups,  to  propose  subjects  for  project  work  or  to  give  special  supervision  for  doing 
these.  The  researcher  is  a person  possibly  along  on  the  course  who  examines  the  students’  learning  and  learning 
processes.  The  content  provider  is  a person  who  produces  hypermedia-based  learning  materials  for  the  learning 
environment.  In  principle  a person  external  to  the  A&O  environment  may  also  act  as  a content  provider.  The 
administrative  personnel  of  the  learning  environment  includes  the  director/rector  and  the  secretary.  They  are 
responsible  for  decision-making  and  practical  measures  related  to  the  courses  of  the  open  learning  environment, 
eligibility  to  study,  registration,  and  entering  credit  points  in  the  academic  record.  The  person  maintaining  the 
system  is  a person  who  is  responsible  for  maintenance  and  the  updating  of  the  system  and  for  administration  at 
system  level.  There  may  also  be  visitors  to  the  learning  environment:  a visitor  may  be  anyone  using  the  Internet  who 
has  browsing  access  and  can  get  to  know  the  courses  at  a certain  level. 

3.2  Spaces 

The  A&O  learning  environment  structure  is  based  on  a space  model  comprising  four  spaces.  These  are  1)  Office,  2) 
Study)  3)  Media  Centre  and  4)  Gallery.  Figure  1 presents  the  front  page  of  the  learning  environment.  There  is  a 
noticeboard  on  the  front  page  of  the  A&O  learning  environment  where  Internet  users  can  obtain  information  about 
the  courses  included  in  the  environment.  All  matters  pertaining  to  administration  of  studies  are  taken  care  of  through 
the  A&O  Office.  When  beginning  and  ending  their  studies  students  are  in  contact  with  the  Office,  where  they  enrol 
on  courses  and  have  entries  made  for  courses  completed.  The  Study  is  a working  space  for  students,  teachers,  tutors 
and  experts.  Its  functions  vary  depending  on  the  role  of  the  actor.  Via  the  Study  students  can  study  courses 
independently  or  coiiaboratively  with  other  students:  they  can  obtain  supervision  from  teachers,  tutors  or  other 
students  and  experts  and  also  feedback.  The  students’  log  data  accumulated  in  the  A&O  environment  is  accessible  to 
the  student,  the  teacher  and  possibly  also  to  the  researcher  for  the  duration  of  the  course.  Via  the  Study  all  actors  can 
collaborate  with  one  another.  The  Media  Centre  serves  all  actors  in  the  learning  environment.  In  addition  to  courses 
this  also  contains  materials  related  to  study.  The  material  in  the  Media  Centre  may,  for  example,  include  text,  sound, 
image,  video  and  animation  material.  In  future  it  will  be  possible  to  locate  video-on-demand  services  in  the  Media 
Centre,  too.  The  implementation  of  the  Media  Centre  is  comparable  to  a database,  which  makes  structural  access 
possible.  There  is  also  a Gallery  in  the  A&O  environment,  and  here  the  practical  assignments  and  final  projects  of 
the  students  are  displayed  complete  with  evaluations.  The  Gallery  is  to  have  an  option  to  vote  such  that  Internet 
users  will  be  able  to  elect  their  own  favourite.  The  technical  realisation  of  the  Gallery  is  also  as  in  a database. 
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Figure  1:  Front  page  of  the  A&O  learning  environment 


3.3  Tools 

The  actors  in  the  A&O  learning  environment  have  tools  at  their  disposal  for  the  creation  of  content,  cognitive  tools 
for  each  course,  communication  tools,  tools  to  support  collaborative  learning  and  management  tools.  The  tools  for 
creating  content  are  important,  as  course  content  is  primarily  created  using  these.  The  course  material  may  include 
hypertext-based  material,  images,  animations,  video,  links  to  external  sources  and  readymade  programmes  and 
elements  supporting  self-evaluation.  The  structure  of  the  course  material  to  be  produced  is  an  important  delineating 
factor  in  learning,  its  effects  extending  to  the  organisation  of  the  A&O  communication  and  collaboration  tools. 
Naturally  the  tools  for  creating  content  are  also  available  for  students’  use  as  in  the  open  learning  environment 
students,  too,  are  producers  of  content.  Constructing  knowledge  is  accomplished  through  the  cognitive  tools  among 
others.  Such  cognitive  tools  include  various  tools  for  information  seeking  and  ordering,  navigation  tools,  question- 
making tools,  problem  solving  tools,  and  special  tools  for  use  on  certain  courses.  The  present  article  describes  the 
Maple  program  in  the  field  of  symbolic  mathematics  as  a cognitive  tool.  The  A&O  communication  tools  comprise  e- 
mail,  discussion  groups  and  real-time  discussion.  E-mail  can  be  used  by  the  actors  to  send  information  in  the  form  of 
correspondence  in  their  own  words.  The  discussion  groups  render  possible  supervised  discussion  group  by  group. 
The  real-time  discussion  supports  collaboration  between  students  on  the  course  at  the  time  of  studying.  The 
administrative  tools  are  used  to  administrate  the  environment.  Administrative  measures  include  the  granting  of  the 
right  to  study,  acceptance  as  an  actor,  acceptance  of  the  course  and  entering  the  course  or  parts  of  it  in  the  academic 
register, 

3.4  Functions 

Students  enrol  as  actors  in  the  A&O  learning  environment  through  the  Office,  The  Office  noticeboard  carries 
general  information  on  courses  and  details  of  all  actors  on  the  courses,  A system-specific  ID  card  is  made  for  every 
student,  teacher,  tutor  and  expert.  This  ID  card  contains  personal  details,  role  on  the  course,  contact  information, 
photograph,  link  to  the  individual’s  homepage  and  an  open  space  for  the  individual  to  introduce  him/herself  and 
his/her  hobbies  to  other  participants.  The  aim  of  the  ID  cards  is  to  help  course  participants  to  get  to  know  each  other 
and  to  support  collaborative  activity.  The  Office  also  has  course  curricula,  which  include  course  objectives, 
schedules,  contents,  information  on  course  materials,  teaching  methods,  evaluation,  teachers,  tutors,  experts  and 
researchers  possibly  involved  with  the  course.  In  A&O  all  these  actors  have  their  own  Study.  Figure  2 presents  such 
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a student  Study.  It  contains  the  course  to  be  studied  and  also  a whole  lot  of  tools  for  planning  studies,  study 
management,  communication,  constructing  knowledge,  monitoring  studies  and  resource  management. 
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Figure  2:  A student  Study  and  part  of  its  functions 


In  A&O  the  student  may  participate  in  several  courses,  and  the  selection  of  each  of  these  is  made  in  the  Study.  The 
Study  has  a noticeboard  for  the  course  and  a curriculum  on  the  basis  of  which  students  can  make  their  own  personal 
curricula.  The  A&O  learning  environment  also  collects  log  data  about  students’  activities  in  the  course  of  study. 
This  data  can  be  utilised  in  students’  self-evaluation,  in  teachers’  evaluation  work  and  as  a support  for  researchers’ 
analyses.  A student  can  take  a look  at  the  matters  to  be  dealt  with  on  the  course  in  a given  week.  This  information  is 
obtained  from  the  noticeboard  and  the  course  discussion  group.  S/he  selects  a subject  for  his/her  practical  exercise 
and  seeks  for  further  information  in  the  Media  Centre.  The  selection  of  the  subject  can  be  narrowed  down  by  e- 
mailing  the  teacher.  After  selecting  the  subject  s/he  forms  a workgroup.  For  this  purpose  s/he  goes  through  the  ID 
cards  and  puts  a notice  on  the  course  noticeboard.  The  workgroup  presents  the  plan  for  the  practical  exercise  to  the 
teacher  in  a public  discussion  group,  accomplishes  the  work  using  as  aids  the  e-mail,  the  real-time  discussion  group, 
the  discussion  group  set  up  for  the  workgroup.  Finally  the  workgroup  submits  the  work  to  the  teacher  by  e-mail  and 
comments  and  further  develops  it  as  necessary.  At  the  end  of  the  course  some  of  the  group  work  may  be  displayed  in 
the  Gallery. 


4 A Learning  Experiment  on  Matrix  Algebra  in  A&O 

In  order  to  test  the  A&O  learning  environment  a learning  experiment  was  organised  at  Tampere  University  of 
Technology.  The  students  studied  using  A&O  environment  an  introductory  part  of  the  course  of  Matrix  Algebra  at 
TUT  during  three  weeks  in  September  1999.  The  A&O-leaming  environment  contained:  Introduction  to  Matrix 
Algebra  as  hypertext,  interactive  exercises  generated  and  checked  by  the  symbolic  algebra  program  Maple, 
noticeboard,  internal  E-mail,  real  time  discussion  (chat). 

In  the  beginning  all  the  students  filled  in  their  ID-cards.  The  theory  to  be  studied  as  well  as  well  as  weekly  exercises 
were  announced  weekly  on  noticeboard.  The  students  were  asked  to  study  the  theory,  to  study  the  interactive 
exercises,  to  solve  easier  theoretical  exercises  by  themselves,  and  to  solve  more  demanding  problems  in  groups.  The 
results  of  group  work  were  presented  in  the  weekly  noticeboard  so  that  other  students  could  study  them.  As  all  the 
proposed  solutions  were  not  correct,  it  turned  out  that  some  students  started  to  make  comments  about  weaknesses  in 
the  proofs  and  to  present  ideas  how  to  improve  the  solutions.  This  was  interesting  to  the  teachers,  as  they  had  not 
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planned  to  use  noticeboards  as  a media  to  constructing  the  correct  solutions.  At  the  end  of  the  experiment  the 
students’  opinions  about  the  learning  experiment  were  collected.  The  feedback  consisted  about  pedagogical,  content- 
based  and  technical  matters.  Part  of  the  results  is  shown  in  Table  1 . 

Pedagogical  parts  of  the  learning  experiment  emphasized  self-directed  learning  and  the  learner’s  own  activity.  That 
is  partly  because  teachers  chose  consciously  the  passive  role  in  the  students  learning  processes.  In  the  results  the 
students  seemed  to  be  motivated  to  take  responsibility  for  their  own  learning.  They  preferred  the  A&O  learning 
environment  because  it  gave  freedom  to  learning  for  example  by  delivering  course  from  timetables  and  from  the 
structures  of  the  course  material.  The  students  were  responsible  for  setting  the  goals  to  their  learning  process  and 
also  for  achieving  them. 

Students  felt  that  A&O  made  self-directedness  in  learning  possible  by  providing  cognitive  tools  needed  in  learning 
processes.  Learning  environment  supported  contextuality  by  offering  exercises  helping  to  understand  the  theory. 
Examples  given  in  A&O  helped  students  to  apply  the  theory  into  practice.  Partly  caused  by  the  teachers’  passive  role 
in  the  students*  learning  processes,  interaction  between  the  participants  of  the  learning  environment  wasn’t  too 
active.  Only  few  students  actively  participated  in  discussions  in  the  noticeboards.  That  is  why  students  felt  that  the 
learning  environment  supported  only  slightly  collaboration  between  them.  Students  followed  noticeboards  actively, 
but  they  seldom  commented  conversations. 

One  main  problem  which  arose  in  results  is  that  there  are  still  conservative  attitudes  concerning  studying.  Students 
are  used  to  conventional,  teacher  oriented  learning  and  also  this  was  seen  in  results.  Students  preferred  conventional 
lectures  to  self-directed  learning.  Many  of  them  told  that  they  studied  mainly  using  printed  lecture  notes. 


Pedagogical  issues 

mean 

I felt  myself  to  be  responsible  for  learning  instead  of  course  instructors 

4,39 

I enjoyed  the  freedom  in  learning  provided  the  environment 

4,04 

The  knowledge  and  skills  that  1 learned  can  be  transferred  to  other  fields 

4,00 

The  A&O  environment  provided  the  necessary  tools  to  support  learning 

3,50 

Interactive  exercises  helped  to  understand  concepts 

3,46 

Exercises,  solved  by  myself,  helped  in  understanding  the  material 

4,36 

Self-directed  learning  is  more  useful  than  traditional  lectures 

2,71 

I mainly  used  printed  lecture  notes  in  studying 

4,04 

I studied  groupworks,  made  by  other  students  in  the  net 

2,53 

The  learning  environment  encouraged  to  collaborate  and  to  study  with  others 

2,11 

Technical  issues  / User  Interface 

I think  graphics  were  good,  clear  and  there  was  a good  amount  of  it 

3,68 

The  system’s  performance  was  acceptable  and  the  loading  times  weren't  too  long. 

2,75 

The  system  didn’t  make  errors  and  was  reliable 

2,71 

The  functions  were  placed  logically  to  A&O’s  taskbar  and  they  were  easy  to  find 

3,43 

Utilities  (email,  news  client ..  ) were  clear  and  using  them  was  easy 

3,39 

Table  1:  The  students  feedback  from  pedagogical  and  technical  issues  from  the  learning  experiment.  The  scale  was 
1 = strongly  disagree,  2 = disagree,  3 = do  not  know,  4 = agree,  and  5 = strongly  agree. 

From  the  technical  point  of  view  the  results  of  the  experiment  were  good.  The  server  program  didn’t  break  down 
during  the  experiment.  The  server's  performance  was  a bottleneck,  but  it  will  be  significantly  boosted  by  fine  tuning 
the  program  code.  Users  liked  the  graphics  of  the  environment.  The  placement  of  functions  was  considered  to  be 
fine  and  the  user  interface  of  utilities  was  thought  to  be  acceptable  too.  Altogether,  the  design  of  the  user  interface 
was  good,  but  there’s  still  need  to  improve  it.  The  performance  and  the  reliability  of  the  client  were  known  to  be 
problematic  since  thorough  testing  of  the  environment  wasn’t  done  in  forehand.  The  results  of  the  test  confirmed 
some  of  our  doubts  although  the  problems  weren’t  too  bad.  The  environment  was  updated  three  times  during  the 
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experiment.  The  updates  improved  performance  and  reliability.  The  performance  in  general  is  not  bad  but  some 
functions  are  bit  too  slow  (e.g.  drawing  the  tree  structure  of  noticeboards).  As  a result,  the  problematic  functions 
have  been  pinpointed.  They  will  be  improved  to  an  acceptable  speed.  Reliability  has  already  been  improved  much, 
thanks  to  the  feedback  from  the  students  during  experiment. 

5 Conclusions 

The  present  article  presented  the  A&O  learning  environment.  The  open  learning  environment  is  based  on  a space 
model  which  is  divided  into  Office,  Study,  Media  Centre,  and  Gallery.  The  article  presented  the  main  actors  of  the 
learning  environment,  the  functions  and  the  production  of  content  for  the  open  learning  environment.  The  planning 
and  realisation  of  the  A&O  learning  environment  proved  to  be  a very  challenging  task,  considering  the  learning 
theories  and  rapid  development  in  the  field  of  information  technology.  In  the  learning  environment  the  student  is  the 
most  important  actor  learning  is  the  most  important  process  - every  effort  has  been  made  to  take  this  into  account  in 
the  planning  and  realisation  of  the  A&O  learning  environment.  However,  there  is  so  far  no  empirical  knowledge 
available  as  to  how  well  the  open  learning  environment,  the  structure  of  the  learning  material  and  the  tools 
developed  do  in  practice  support  learning  and  the  activity  of  the  learners,  their  collaboration  and  objectives, 
contextual ity,  transfer  and  reflectivity.  The  first  learning  experiment  in  Matrix  Algebra,  was  carried  out  and  the 
results  were  presented.  Further  experiments  and  data  collection  during  the  ongoing  Open  Learning  Environment- 
research  project  will  be  performed. 
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Abstract:  Models  of  multimedia  communication  attempt  to  classify  the  numerous  types  of  media 
objects  available,  and  to  provide  a basis  for  unambiguous  terminology  in  a new  and  expanding 
field.  Many  of  these  models  are  products  of  theory,  rather  then  practical  investigation,  and  few 
have  been  empirically  studied  to  assess  suitability.  This  paper  reports  on  an  experiment  which 
investigated  the  understandability  of  a recently  proposed  multimedia  model,  TOMUS.  The 
experiment  entailed  subjects  classifying  various  media  objects  according  to  the  TOMUS  model. 
Error  and  perceived  difficulty  data  were  collected;  analysis  revealed  which  of  the  categories  are 
the  most  difficult  to  comprehend.  Suggestions  are  made  as  to  the  causes  of  these  difficulties,  and 
recommendations  as  to  how  the  model  might  be  correspondingly  altered  are  proposed. 


Introduction 

The  concept  of  “multimedia”  suffers  from  inadequate  standardisation  of  terminology,  and  the  lack  of  commonly  agreed 
definitions  results  in  much  ambiguity  both  within  and  between  articles  written  on  the  topic.  Recognising  this  lack  of 
consistency,  many  authors  have  provided  classifications,  definitions  and  models,  attempting  to  create  a theoretical  basis 
for  the  study  of  multimedia  communication.  The  fine-grained  model  proposed  by  Arens  et  al  (1993)  is  based  on  the 
characteristics  of  perceiver,  producer,  information  and  media,  and  has  a set  of  rules  associated  with  it  which  determine 
matchings  between  information  and  media.  The  taxonomy  used  by  Heller  and  Martin  (1995)  is  based  around  common 
(potentially  ambiguous)  terms  “text”,  “graphics”,  “sound”  and  “motion”.  Its  aim  is  to  enhance  students’  understanding 
of  the  range  of  possible  media  types  in  their  production  and  evaluation  of  multimedia  software.  Vetere’s  model  (1997) 
includes  the  notion  of  interaction,  relating  to  the  goals  of  the  user  and  the  goals  of  the  system,  and  uses  3D  vectors  to 
represent  complex  media  types.  Bemsen’s  (1995)  input  and  output  modalities  define  forty-eight  low-level 
“unimodalities”  in  a hierarchical  model. 

Empirical  studies  are  rare  in  the  area  of  multimedia  definition:  the  afore-mentioned  models  are  products  of  thought, 
rather  than  practical  experiment.  Lohse’s  model  (1994)  is  based  on  data  taken  from  subjects’  grouping  of  objects,  but 
only  in  the  visual  domain.  While  Heller  and  Martin  (1995)  tested  their  model  by  observing  its  use  in  student  projects, 
the  outcomes  were  as  much  indicative  of  student  skill  as  of  model  suitability.  Theoretical  multimedia  models  need  to  be 
empirically  tested  for  their  suitability.  This  suitability  may  be  with  respect  to  suitability  for  use  for  a particular 
multimedia  task,  ease  of  understanding  of  the  model  definition,  or  correspondence  between  the  model  and  human 
cognition  and  perception.  The  aim  of  this  paper  is  to  present  the  results  of  an  empirical  investigation  of  the  TOMUS 
multimedia  model  (Purchase,  1 998),  from  the  perspective  of  its  understandability.  A theoretical  model  that  is  based  on 
definitions  that  are  difficult  to  understand  is  not  useful  for  any  task. 


The  TOMUS  model 


The  TOMUS  model  was  devised  as  an  attempt  to  categorise  the  many  types  of  simple  media  objects  that  may  comprise 
multimedia  texts.  Its  intention  was  to  provide  an  unambiguous  theoretical  model  that  could  be  used  to  avoid  vague  and 


inconsistent  terminology:  it  offers  a means  for  producing  a consistent  definition  of  the  terms  associated  with 
multimedia  communication  (multimodality,  hypermedia,  etc.). 

This  paper  presents  results  of  an  investigation  into  the  understandability  of  the  TOMUS  dimensions.  The  aim  of  this 
paper  is  not  to  present  the  TOMUS  model  in  detail,  nor  to  propose  an  unambiguous  definition  of  multimedia 
communication:  this  has  been  done  more  completely  elsewhere  (Purchase,  1998). 

The  TOMUS  model  consists  of  three  dimensions:  sign,  syntax,  and  modality. 

The  sign  dimension  determines  the  nature  of  objects  used  in  multimedia  communication,  and  is  based  on  semiotic 
categories  of  sign:  symbol  and  icon  (Bruner,  1 966).  (The  enactive  category  included  by  Bruner  does  not  form  part  of 
the  TOMUS  model,  as  it  is  concerned  with  physical  movement,  rather  than  communication.)  An  iconic  object  is 
perceptually  similar  to  the  concept  it  represents  (eg.  a “no  smoking”  sign),  while  a symbolic  object  has  no  perceivable 
similarity  to  the  concept  it  represents  (eg.  the  Olympic  rings).  In  the  TOMUS  model,  the  iconic  category  is  divided  into 
two:  concrete-iconic  (the  object  is  perceptually  identical  to  the  represented  concept,  as  in  a photograph),  and  abstract- 
iconic  (the  object  is  perceptually  similar  to  the  represented  concept,  as  in  a cartoon  character).  This  distinction  follows 
Amheim’s  proposal  of  a continuum  of  abstraction  for  visual  signs  (Amheim,  1969).  The  sign  dimension  thus  has  three 
values:  concrete-iconic,  abstract-iconic  and  symbolic. 

The  syntax  dimension  determines  the  manner  in  which  single  media  objects  may  be  arranged  in  a multimedia 
document.  There  are  five  syntax  categories  in  the  basic  TOMUS  model;  individual,  augmented,  temporal,  linear  and 
schematic.  (The  complete  TOMUS  model  includes  a network  syntax  value,  used  in  order  to  distinguish  between 
multimedia  and  hypermedia;  its  inclusion  in  this  TOMUS  description  is  beyond  the  scope  of  this  paper.) 

• The  individual  syntax  describes  documents  comprising  a single  object  in  a single  moment  of  time  (eg.  a road  sign). 

• The  augmented  syntax  describes  documents  that  comprise  a single  object,  where  the  object  is  augmented  with 
additional  information  (eg.  colour  or  font)  essential  to  its  interpretation  (eg.  a red  traffic  light). 

• The  temporal  syntax  describes  documents  where  the  same  object  is  repeated  over  time,  and  only  a single  concept  is 
represented  (eg.  a fire  alarm,  a pendulum).  The  concept  cannot  be  comprehended  without  transmission  over  time. 

• The  linear  syntax  describes  documents  which,  like  the  temporal  syntax,  are  transmitted  over  time.  Unlike  the 
temporal  syntax,  the  concepts  change  over  time,  as  part  of  a narrative  (eg.  a novel,  a film). 

• The  schematic  syntax  describes  documents  where  the  relationship  of  the  objects  to  each  other  in  an  additional 
spatial  dimension  is  important  (eg.  a map  or  graph,  a family  tree).  In  the  aural  dimension,  this  additional  dimension 
is  the  frequency  dimension,  representing  complexity  of  sound. 

The  modality  dimension  distinguishes  between  the  senses  used  for  perceiving  the  multimedia  document.  Currently,  this 
dimension  has  two  values:  visual  and  aural. 

Within  these  three  dimensions,  thirty  different  media  objects  can  be  classified  (Figure  1 ). 


(syntax) 


visual 
aural  ^ 


Figure  1:  The  TOMUS  model  of  multimedia 


symbolic 
^ abstract 
concrete 


An  important  attribute  of  the  TOMUS  dimensions  is  that  they  are  all  defined  independently  of  technology,  and  are 
based  on  the  nature  of  the  text,  rather  than  the  device  that  communicates  it.  This  approach  recognises  that  linking  the 
model  to  technology  may  mean  that  it  will  soon  be  out  of  date,  and  acknowledges  that  multimedia  need  not  be  the 
exclusive  domain  of  the  computer.  This  technology-free  approach  sets  TOMUS  apart  from  other  multimedia 
classifications. 
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Empirical  investigation 


While  TOMUS  was  primarily  intended  to  provide  unambiguous  terminology,  it  also  aimed  to  be  a useful  model  for 
multimedia  tasks,  by  clearly  defining  the  range  of  multimedia  objects.  However,  it  is  a purely  theoretical  model,  the 
product  of  thought  rather  than  practical  investigation.  While  the  sign  dimension  is  based  on  semiotic  definitions,  and 
the  modality  dimension  is  based  on  the  senses,  the  syntax  dimension  is  merely  based  on  the  experiences  of  TOMUS’ 
designer.  There  is  no  guarantee  that  the  model  will  be  easily  comprehensible  to  other  people.  It  was  therefore  important 
that  the  model  be  evaluated  empirically,  to  determine  its  understandability,  and  to  dictate  any  essential  changes. 

To  this  end,  an  experiment  was  conducted  where  twenty  subjects,  having  knowledge  of  TOMUS,  were  each  presented 
with  a single  example  of  each  of  the  TOMUS  categories,  and  were  asked  to  identify  the  correct  TOMUS  category.  The 
aim  was  to  determine  whether  there  were  any  categories  that  subjects  were  having  particular  difficulty  understanding. 


The  research  questions 

A desirable  property  of  a theoretical  model  of  multimedia,  is  that  the  dimensions,  and  the  values  on  the  dimensions,  are 

easy  to  understand..  Subjects  were  asked  to  identify  the  TOMUS  category  of  each  of  thirty  different  media  objects. 

Two  types  of  understandability  were  measured  in  terms  of  misclassification  of  given  media  objects: 

• Dimension  understandability:  the  extent  of  correct  classification  of  media  objects  according  to  their  value  on  each 
dimension.  Thus,  if  the  [concrete,linear, aural]  object  were  classified  as  [concrete,schematic, aural],  a linear- 
dimension  error  would  be  counted,  but  not  a concrete-dimension  error  nor  an  aural-dimension  error. 

• Composite  understandability:  the  extent  of  correct  classification  of  media  objects  within  their  values  on  each 
dimension.  Thus,  if  the  [concrete, linear,aural]  object  were  classified  as  [concrete,schematic,aural],  a linear- 
composite  error,  a concrete-composite  error  and  an  aural-composite  error  would  all  be  counted. 

In  addition,  the  “perceived  difficulty”  of  the  values  along  each  dimension  was  collected  by  way  of  a questionnaire. 

Three  questions  were  therefore  considered  for  each  of  the  three  TOMUS  dimensions  (sign,  syntax  and  modality): 

• Are  the  values  along  the  three  dimensions  equally  easy  to  understand?  (dimension  errors) 

• Are  the  values  along  the  three  dimensions  equally  easy  to  classify  within?  (composite  errors) 

• Are  the  values  along  the  three  dimensions  perceived  as  equally  as  difficult  to  understand?  (perceived  difficulty) 


Methodology 

The  subjects  interacted  with  an  online  system,  the  display  of  which  comprised  buttons  labelled  with  the  values  of  each 
of  the  thirty  TOMUS  categories,  and  a display  area.  For  each  of  thirty  media  objects  that  were  presented,  subjects  were 
required  to  indicate  to  which  TOMUS  category  the  object  belonged. 

A within-subject  experimental  design  was  used,  with  all  subjects  being  shown  all  thirty  media  objects.  To  reduce  any 
learning  effect,  the  order  of  presentation  of  the  objects  was  randomised  for  each  subject,  and  six  “practise  objects”  were 
presented  at  the  start  of  each  session:  the  subjects  were  not  aware  that  these  six  initial  objects  were  not  experimental. 

The  online  system  recorded  the  TOMUS  categories  selected  by  the  subjects  for  each  media  object,  and  identified 
misclassifi cations.  The  independent  variable  for  the  experiment  was  the  TOMUS  category;  the  dependent  variable  was 
errors.  Twenty  third  year  computer  science  students  participated  voluntarily. 

After  the  classification  task,  subjects  completed  a questionnaire.  For  each  TOMUS  dimension  they  were  asked  to  “rate 
the  difficulty  [they]  had  in  deciding  the  [sign/syntax/modality]  of  the  examples”,  1 representing  the  most  difficult  value 
on  each  dimension.  This  data  was  collected  in  the  form  of  rankings. 

The  media  objects 

Each  media  object  was  accompanied  by  a sentence  defining  its  meaning  (Figure  2).  These  concept  definitions  were 
necessary  in  providing  a context  for  the  categorisation:  without  a context,  the  TOMUS  category  of  an  object  may  be 
ambiguous.  For  example,  a picture  of  a lightbulb  might  mean  “lightbulbs”  (abstract-iconic)  if  seen  on  a packing  box, 
but  may  mean  “an  idea”  (symbolic)  if  seen  in  a cartoon. 
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Media  object 

Concept  definition 

CIV 

A photograph  of  an  airplane 

An  airplane 

CAV 

A photograph  of  a male  and  female  parrot 

A male  and  female  bird  distinguished  by  colour 

CTV 

Animation  of  a bouncing  ball 

A bouncing  ball 

CLV 

Film  clip  of  a rally  car  racing 

A rally  car  racing 

CSV 

An  aerial  photograph  of  a city 

A city 

AIV 

A drawing  of  a cartoon  character 

A person 

AAV 

A picture  of  a weather  map 

A weather  map  with  clouds  distinguished  by  colour 

ATV 

Animation  of  a computer  generated  insect  walking 

A walking  insect 

ALV 

An  animated  file  moving  from  one  folder  to  another 

Moving  a file 

ASV 

An  iconic  chart  representing  birth  rates 

The  relative  birth  rates  of  males  and  females 

SIV 

A picture  of  a peace  symbol 

Peace 

SAV 

The  words  “express”  and  “slow”,  in  different  fonts 

Words  denoting  speed,  distinguished  by  font 

STV 

Animation  of  a circular  spinning  cursor 

Indication  of  a computer  processing 

SLV 

Words  forming  a sentence 

A sentence 

SSV 

A picture  of  a parse  tree  of  a sentence 

A parse  tree 

CIA 

The  sound  of  a glass  breaking 

Breaking  glass 

CAA 

The  loud  sound  of  a slamming  door 

A door  slammed  in  anger 

CTA 

The  sound  of  running  water 

Running  water 

CLA 

The  sound  of  a car  driving  past 

A car 

CSA 

The  sounds  of  a military  attack 

A war  time  battle 

AIA 

The  sound  of  a cartoon  “boinnnngggg” 

A spring 

AAA 

A cartoon  sound  of  being  hit  by  an  anvil 

Being  hit  by  a metal  object 

ATA 

The  sound  of  laser  gunfire 

A laser  gun 

ALA 

The  sound  of  a synthesised  heartbeat  changing  pace 

A heartbeat 

ASA 

The  sound  of  a synthesised  car  crash 

A car  crash 

SIA 

The  sound  of  a doorbell  ring 

Someone  requesting  entry  to  a house 

SAA 

The  sound  of  a high  pitched  error  beep 

A severe  error 

STA 

The  sound  of  an  air-raid  siren 

Warning  of  an  air-raid 

SLA 

The  sound  of  a tone  that  changes  pitch 

The  amount  of  traffic  on  a computer  network 

SSA 

The  sound  of  a fire  alarm  with  two  tones 

A fire  alarm  indicating  severity  and  location 

Figure  2:  The  experimental  media  objects.  The  first  column  is  the  TOMUS  category:  sign  (Concrete-iconic,  Abstract- 
iconic,  Symbolic),  syntax  (Individual,  Augmented,  Temporal,  Linear,  Schematic),  and  modality  (Visual,  Aural). 


Data  Analysis 

The  error  data  was  analysed  using  a standard  ANOVA  test  based  on  critical  values  of  the  F distribution;  in  the  case  of 
significance,  a Tukey  pairwise  analysis  was  performed,  to  determine  which  values  on  the  dimensions  proved  most 
difficult.  The  perceived  difficulty  data  was  collected  by  way  of  ranks;  thus,  a non-parametric  statistical  analysis  was 
required.  The  Friedman  analysis  of  variance  test  was  used;  in  the  case  of  significance,  the  Kruskal-Wallis  test  identified 
where  the  differences  lay.  Figure  3 shows  the  outcome  of  this  statistical  analysis,  and  figures  4,  5 and  6 show  graphs  of 
the  average  dimension  errors,  composite  errors  and  perceived  difficulty  rankings.  Figure  7 shows  where  pairwise 
differences  lay.  (NS  indicates  a non-significant  result.) 


dimension  errors  (ANOVA) 

composite  errors  (ANOVA) 

perceived  difficulty  (Friedman) 

sign 

F(2,38,  alpha=0.05)=6.44 

F(2,38,alpha=0.05)=4.01 

F(2,38,  alpha=0.05)=  7.03 

syntax 

F(4,76,  alpha=0.05)=12.37 

F(4,76,  alpha=0.05)=13.80 

F(4,76,  alpha=0.05)=  8.52 

modality 

F(1,19,NS)=0.32 

F(l,19,  alpha=0.05)=36.51 

F(l,19,  alpha=0.05)=  6.33 

Figure  3:  Analysis  of  the  data  for  dimension  and  composite  errors,  and  perceptions  of  difficulty 


Discussion 

The  sign  dimension 

The  data  is  significant  for  all  three  tests,  and  the  pairwise  analysis  shows  significant  differences  between  abstract  and 
the  other  two  categories  in  almost  all  cases. 

The  abstract-iconic  category  was  added  on  to  Bruner’s  semiotic  definitions  (Bruner,  1966).  It  produces  a “grey  area” 
that  the  subjects  may  have  had  difficulty  distinguishing.  Informal  preparatory  experiments  revealed  that  symbolic 
objects  tend  to  be  classified  with  respect  to  function  (e.g.  # has  the  function  of  sharpening  a note  in  music)  while 
concrete-iconic  objects  tend  to  be  classified  according  to  form  (e.g.  a photograph  of  a pie  on  a menu  merely  shows 
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what  the  pie  looks  like,  and  is  therefore  classified  according  to  its  visual  form  as  a pie.)  On  the  other  hand,  many 
abstract-iconic  objects  may  be  classified  according  to  function  or  form,  depending  on  their  context,  and  thus  may  have 
ambiguous  TOMUS  classification.  For  example,  a diagram  of  a pair  of  scissors  may  represent  scissors  in  a pictorial 
representation  of  items  to  be  included  in  a first  aid  kit,  but  may  mean  “cut  here”  if  found  on  a dress  pattern.  It  is  this 
ambiguity  that  requires  the  association  of  concept  definitions  with  media  objects  when  they  are  categorised  within 
TOMUS.  And  it  is  perhaps  for  this  reason  that  the  abstract-iconic  category  produced  the  worst  performance. 

The  aural  distinction  between  concrete  and  abstract  is  awkward  (as  the  difference  between  synthesised  sounds  and  real 
recordings),  making  it  a less  useful  distinction  than  in  the  visual  modality.  Further  analysis  of  the  data  may  reveal 
whether  a significant  number  of  abstract  errors  were  made  within  the  aural  category. 

Following  Amheim  (1969),  we  believe  that  a continuum  between  symbolic  and  iconic  is  necessary,  particularly  in 
multimedia  texts  where  photographs  (which  are  concrete-iconic)  are  used  extensively.  Removal  of  the  abstract-iconic 
category  due  to  bad  performance  is  not  appropriate.  A more  suitable  proposal  would  be  for  the  definition  of  abstract- 
iconic  to  be  clarified,  or  for  a form/function  distinction  to  be  introduced. 
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Figure  5.  Average  values:  syntax  Figure  6.  Average  values:  modality 


□ Dimension  errors 
[g  Composite  errors 
Perceived  difficulty 


abstract 


concrete 


symbolic 


abstract 

NS 

linear 

temporal 

augment. 

individ. 


schematic 


Figure  7.  Pairwise  differences.  Each  cell  shows  whether  significant  differences  were  found  in  the  three  tests.  Thus,  eg., 
the  differences  between  the  responses  for  schematic  and  individual  were  significant  for  dimension  errors  and  composite 

errors,  but  not  for  perceived  difficulty. 


The  syntax  dimension 

The  data  is  significant  for  all  three  tests,  with  best  performance  in  the  individual  and  temporal  categories.  The  pairwise 
analysis  confirms  that,  for  the  error  tests,  augmented,  linear  and  schematic  are  more  difficult  than  individual,  and  that 
linear  and  schematic  are  more  difficult  than  temporal. 

It  is  not  surprising  that  individual  and  temporal  are  easy  to  understand  and  distinguish,  as  they  both  represent  only  a 
single  concept.  But  it  is  a surprise  that  the  linear  category  proved  difficult,  as  the  notion  of  narrative  is  common.  The 
TOMUS  stated  definition  of  this  category  needs  to  be  reassessed,  and  clearly  distinguished  from  temporal. 

The  schematic  category  is  awkward  in  the  aural  modality:  more  tests  are  required  to  see  whether  most  schematic  errors 
were  aural.  Most  surprising  is  the  fact  that  the  subjects  did  not  perceive  the  schematic  category  to  be  difficult,  despite 
their  poor  performance.  The  subjects  seemed  to  understand  the  concept  of  schematic,  but  were  less  sure  when  to  apply 
it.  The  schematic  definition  needs  to  be  reassessed  so  that  it  more  closely  relates  to  its  use  in  the  model. 
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The  augmented  syntax  is  concerned  with  comparisons:  the  font  or  colour  of  an  augmented  object  is  only  important 
when  compared  with  a different  font  or  colour.  This  comparative  property  of  augmented  objects  adds  to  the  complexity 
of  their  TOMUS  definition,  and  to  their  classification.  It  is  recommended  that  the  augmented  category  be  removed 
from  the  model.  The  process  of  adding  extra  information  (e.g.  font,  colour)  to  a media  object  can  be  acknowledged,  but 
can  be  applied  outside  the  TOMUS  model  categories,  in  a separate  part  of  the  multimedia  process. 


The  modality  dimension 

There  is  no  significance  for  dimension  errors:  the  subjects  understood  the  visual/aural  distinction.  The  composite  error 
data  is  highly  significant,  indicating  that  classifying  within  the  aural  domain  was  difficult:  subjects  had  less  difficulty 
understanding  the  other  dimensions  in  the  visual  domain  than  in  the  aural  domain.  This  is  not  surprising,  as  our  visual 
sense  dominates  the  others. 

What  is  surprising  is  that  subjects  thought  it  more  difficult  to  determine  whether  something  was  aural  than  visual:  we 
would  have  expected  this  data  not  to  be  significant.  It  is  possible  that  the  students  were  projecting  their  difficulty  in 
classifying  within  the  aural  dimension  onto  their  response  to  this  question. 

There  is  no  reason  why  this  dimension  should  be  altered:  a predictable  extension  would  be  a tactile  category. 


Conclusion 

The  problem  with  this  model,  is  the  assumption  of  equal  priority  for  each  of  the  classification  cells.  No  consideration  is 
made  for  the  fact  that  some  types  of  media  objects  may  be  more  common  or  more  important  for  multimedia  tasks.  For 
example,  redefining  “schematic”  so  that  it  better  relates  to  the  aural  modality  assumes  an  equal  priority  of  [schematic- 
visual]  objects  with  [schematic-aural]  objects.  It  does  not  acknowledge  that  the  [schematic-aural]  category  may  not  be 
useful,  nor  that  [schematic-aural]  objects  may  be  rare  in  practise. 

This  observation  does  not  mean  that  it  is  irrelevant  if  less  important  or  less  common  categories  are  less  easy  to 
understand,  but  it  does  suggest  that  tweaking  model  definitions  to  force  media  objects  to  unambiguously  fit  one 
category  may  not  be  a useful  process. 

We  anticipate  that  a well  balanced  model,  with  equal  priority  for  each  value  on  each  dimension,  would  be  difficult  to 
propose,  and  may  not  be  any  more  useful  in  practise  than  TOMUS  as  it  stands.  While  investigating  TOMUS 
understandability  has  been  a useful  start  in  assessing  its  worth,  it  is  more  important  that  its  practical  use  be  investigated: 
how  useful  are  these  classifications  to  designers  and  evaluators  of  multimedia  texts?  Assessing  practical  use  will  avoid 
issues  regarding  the  relative  priority  or  frequency  of  occurrence  of  the  values  on  the  dimensions,  and  will  rather  focus 
on  the  benefits  of  models  like  TOMUS  in  action. 
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Abstract:  This  paper  features  a study  on  the  use  of  electronic  catalogs  and  retail  storefronts  as  a 
concrete  collaborative  class  project  for  the  Electronic  Commerce  Course  in  the  Business  curriculum 
and  a survey  of  the  MBA  students  that  undertook  this  project  in  response  to  the  course 
requirements.  The  survey  results  provide  positive  indications  for  the  achievement  of  the  goals  of 
collaborative  inquiry,  team-based  and  active  learning,  constructivist  pedagogical  approach,  and 
learning  by  doing.  The  technical  details  of  the  project  are  also  described  in  the  study. 


Introduction 

The  Internet  is  providing  an  extraordinary  environment  for  designing  teaching  and  learning  experiences. 
This  paper  will  focus  on  how  the  World  Wide  Web  (WWW)  could  be  utilized  for  teaching  Electronic  Commerce  (E- 
Commerce)  in  Management  Information  Systems  (MIS)  program  of  the  Business  curriculum.  The  opening  of  the 
Internet  to  businesses  in  the  early  nineties  initiated  the  E-Commerce  phenomena  that  has  not  only  revolutionized  the 
conduct  of  business  online  but  also  overhauled  the  way  E-Commerce-related  business  courses  need  to  be  taught. 
The  unusual  aspect  of  these  courses  is  that  the  WWW  is  primarily  the  environment  in  which  online  establishments 
also  conduct  their  business.  This  means  students  can  actually  simulate  the  conduct  of  business  via  the  same 
environment  where  their  learning  takes  place.  Not  only  that — the  instructional  setting  can  also  take  advantage  of 
web  hosting  services  used  to  support  online  businesses  to  supplement  if  not  entirely  replace  inadequate  university 
information  technology  infrastructure  that  needs  to  undergird  the  activities  of  electronic  merchants. 

This  paper  reports  the  experiences  of  the  MBA  students  at  the  School  of  Business,  Montclair  State 
University,  in  building  electronic  storefronts  using  Merchandizer  software,  which  is  a requirement  in  their 
Management  Information  Systems  course  that  has  an  E-Commerce  theme.  Using  the  survey  technique,  student 
feedback  was  gathered  from  two  MBA  classes  that  took  the  course  during  the  summer  and  fall  1999.  Data  was 
conducted  to  measure  students’  perceptions  of  the  quality  of  their  overall  learning  experience  and  their  ability  to 
gain  collaborative  team  work  skills,  discipline-specific  skills  in  building  an  electronic  storefront  and  catalog,  skills 
in  using  the  Merchandizer  software,  and  various  forms  of  thinking  skills. 

The  E-Commerce  course  could  be  taught  by  combining  the  pedagogical  principles  that  surround  a number 
of  appropriate  teaching  strategies:  cooperative  learning,  constructivist  approach,  active  learning,  and  learning  by 
doing.  These  strategies  can  be  used  to  structure  and  coordinate  electronic  learning  spaces.  In  a cooperative  learning 
environment,  the  instructor  organizes  content  and  topical  activities  around  tasks,  problems,  and  projects  that  student 
teams  with  a balanced  mix  of  abilities  and  backgrounds  are  designed  to  undertake  (Adams  and  Hamm,  1996). 
Cooperative  learning  also  provides  valuable  opportunities  for  students  to  work  together  across  gender  and  ethnic 
lines  as  long  as  communication  skills,  prior  achievement,  expectations,  friendships,  and  group  status  are  factored  in 
(Chizhik,  1999).  A meta-analysis  of  university-level  and  adult-learning  courses  found  that  use  of  collaborative 
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learning  concepts  promoted  higher  achievement,  higher-level  reasoning,  more  frequent  generation  of  ideas  and 
solutions,  and  greater  transfer  of  learning  than  did  individualistic  or  competitive  learning  strategies  (Johnson  et  aL, 
1991).  Team  members'  social  skills,  also  called  "interpersonal  intelligence"  according  to  Howard  Gardner, 
encompassing  interpersonal  communication,  task  management,  group  interaction,  and  conflict  resolution  are 
constantly  honed  throughout  the  process. 

The  constructivist  pedagogical  approach  invites  students  to  build  personal  meanings  out  of  the  material 
taught  and  encourages  them  to  reflect  on  how  new  knowledge  builds  on  their  previous  base  of  understanding 
(Pankratius  and  Young,  1995).  In  a group  setting,  the  constructivist  approach  could  lead  to  the  social  creation  of 
knowledge  as  a basis  of  learning. 

"The  yeast  of  knowledge,  openness,  and  enterprise  raises  the  need  for  a multiplicity  of  learning  media  and 
technological  tools  "(Adams  and  Hamm,  1996).  The  very  wealth  of  Web  site  building  tools  that  is  now  available 
allows  students  to  experiment  and  "play"  through  "learning  by  doing"  activities  and  significantly  changes  the  whole 
instructional  experience.  Students  have  more  control  over  the  end  product  via  the  software  tools  and  they  assume 
greater  responsibility  over  their  learning  process  (Adams  and  Hamm,  1996).  MIS  instructors  are  strongly 
encouraged  to  move  towards  the  "learning  by  doing"  mode  which  is,  by  far,  the  most  effective  way  of  teaching 
students  "how  to"  skills.  While  this  is  clear  to  most  teachers,  it  is  not  easy  to  do  unless  certain  prerequisites  are  in 
place:  professional  development,  teacher  preparation,  technological  infrastructure  support  (hardware,  software,  and 
helpline  service),  an  appropriate  incentive  system  that  will  reward  the  countless  hours  that  will  go  into  a more 
technologically-oriented  class  (as  opposed  to  a traditional  lecture-style  one),  and  above  all,  an  overarching  plan  that 
clearly  defines  the  role  of  technology  in  the  pedagogical  process  (Adams  and  Hamm,  1 995). 

Merchandizer  Software  Features 

Students  are  asked  to  build  an  electronic  retail  storefront  for  either  consumer-to-business  or  business-to- 
business  commerce.  What  this  means  is  that  the  store  can  sell  a product  that  individual  consumers  may  be  interested 
in  buying  such  as  an  Amazon.com  for  books  or  a CdNow.com  for  musical  CD-ROMs.  Or,  the  storefront  could  be 
an  electronic  catalog  for  firms  interested  in  buying  materials  they  need  in  running  their  business  such  as  office 
supplies,  for  instance — staplers,  bond  paper,  paper  clips,  etc. — in  other  words,  these  will  be  businesses  buying  from 
another  business.  The  product  used  in  the  project  is  Merchandizer  Software,  which  is  an  online  solution  that  allows 
users  to  build  and  manage  an  electronic  storefront  with  just  a browser  software  such  as  Netscape  Communicator  or 
Microsoft  Internet  Explorer  and  an  Internet  connection  which  an  Internet  Service  Provider  (ISP)  or  web  hosting 
service  can  provide  for  a fee.  In  designing  the  site,  students  could  either  enter  all  product  information  using  a step- 
by-step  wizard  or  they  could  work  directly  with  hypertext  markup  language  (HTML)  forms  to  both  make  and 
modify  their  sites.  In  building  the  electronic  catalog  itself,  students  either  enter  information  manually  using  the 
wizard  templates  or  they  may  import  extensive  information  using  Microsoft  Access  database  tables  which  are  loaded 
onto  the  servers. 

Merchandizer  software  makes  use  of  frames  or  windows  appearing  within  the  browser’s  display  area,  each 
of  which  displays  the  contents  of  a different  HTML  file  (Carey,  1998).  The  software  also  allows  storebuilders  to 
divide  their  product  lines  in  a hierarchical  fashion,  first  into  groups,  then,  into  subgroups,  and  finally,  into  catalog 
items.  The  group  and  subgroup  headers  become  navigational  links  in  one  frame,  displaying  product  pages  or  other 
HTML  pages  or  entire  web  sites  altogether  created  by  the  students.  The  frame  concept  facilitates  the  navigation  of 
the  customer  through  the  storefront. 

Once  groups,  subgroups,  and  catalog  items  have  been  defined,  students  enter  product  information  either  of 
two  ways.  They  can  enter  the  information  using  simple  electronic  forms  that  allow  them  to  submit  information  like 
product  name,  code,  description,  and  price.  Alternatively,  they  could  use  preformatted  Microsoft  Access  tables  that 
essentially  mimic  the  wizard  online  templates.  These  tables  are  more  convenient  to  fill  out  when  merchants  have 
hundreds  or  even  thousands  of  product  items  to  enter.  Since  students  are  building  only  prototype  stores  with  a 
limited  product  line,  they  end  of  using  the  forms,  although  the  use  of  the  Access  table  is  demonstrated  to  them 
should  they  be  interested  in  trying  it. 

Merchandizer  has  features  like  a search  function  that  will  allow  hypothetical  customers  to  search  the 
storefront  using  any  word  or  phrase  related  to  the  products  on  sale.  Merchandizer  also  incorporates  a shipping  table 
for  UPS  directly  into  the  software  so  that  the  students  do  not  have  to  be  concerned  with  this.  They  need  to  specify 
the  appropriate  sales  tax,  though.  Merchandizer  keeps  track  of  all  sales  transactions,  customers,  and  prospects 
(those  who  visit  but  do  not  buy).  A report  generating  facility  gives  students  information  on  the  most  saleable 
products,  customers  who  buy  most  from  the  store,  aggregate  sales  figures  for  the  week  or  the  month,  and  areas  of 
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interest  of  prospects,  among  other  things.  The  software  can  also  autogenerate  email  messages  to  prospects  about 
special  promotional  activities  the  store  is  conducting. 

Version  3.0  of  Merchandizer  offers  students  a wide  range  of  features  to  play  with  to  simulate  a real  online 
store  even  more  realistically:  easy  translation  of  the  web  site  into  any  language;  ability  to  insert  forms  with  special 
buying  instructions  before  the  shopper  checks  out;  offer  global  coupon  numbers  to  shoppers  so  that  they  can 
accumulate  their  purchases  and  eventually  received  discounts  on  items;  ability  to  submit  the  store  site  to  top  search 
engines;  more  managerial  report  forms  that  can  be  generated  from  the  sales  transaction  data;  a personalization 
field  for  each  catalog  item  for  shopper  input  that  will  be  carried  over  to  the  order  tables;  ease  of  uploading  and 
downloading  Access  tables,  among  other  more  advanced  features. 

HipHip  software,  the  company  behind  the  Merchandizer  storefront  building  tool,  is  also  a web  hosting 
service.  A merchant  that  does  not  have  networking  facilities  to  support  the  store  could  subscribe  to  this  service  and 
pay  a one-time  setup  fee  and  monthly  maintenance  fees  that  vary  depending  on  the  number  of  catalog  items 
supported  in  the  store.  HipHip  software  offers  educational  pricing  for  universities  interested  in  using  their  hosting 
services  for  the  purpose  of  instruction.  This  arrangement  is  a boon  for  universities  and  colleges  that  do  not  have 
adequate  information  technology  infrastructure  to  allow  E-Commerce  instructors  to  mount  interesting  student 
projects. 

The  web  hosting  service  is  important  because  of  the  backend  office  component  of  the  online  store.  Sales 
transaction  information  submitted  by  customers  and  personal  information  keyed  in  by  interested  prospects  who  do 
not  purchase  during  their  visit  to  the  store  need  to  be  stored  in  web  servers  so  that  the  order  could  be  processed  and 
sales  prospects  could  be  reached  through  promotional  email  messages  in  the  future.  Backend  office  processing 
requires  the  availability  of  web  servers  that  can  store  order  transaction  data  in  databases  that  can  later  be  accessed 
using  some  form  of  structured  query  language  so  that  reports  and  statistics  could  be  generated  for  managerial 
decision  making.  The  extensive  requirements  of  the  storefront  make  the  storefront  project  too  challenging  for  the 
normal  capabilities  of  a typical  university’s  academic  technology  support  group.  It  is  far  better  to  sign  up  with  the 
web  hosting  service — if  reasonable  fees  could  be  negotiated  with  the  company — and  have  their  technical  support 
staff  provide  the  services  necessary  for  optimizing  all  functionalities  and  features  of  the  storefront. 

Methodology 

This  study  used  the  survey  technique  using  a written  questionnaire,  which  was  administered  to  two  classes 
of  the  MIS  503,  Management  Information  Systems  course  at  the  MBA  level.  Data  was  gathered  during  the  Summer 
and  Fall  of  1999.  Both  classes  had  the  theme  of  “Electronic  Commerce”  and  the  project  assigned  to  the  students 
was  the  construction  of  an  online  retail  storefront  for  either  consumer-to-business  or  business-to-business 
commerce.  Students  were  assigned  in  teams  of  four  to  five  students.  Each  team  was  in  charge  of  a storefront  web 
site,  which  was  maintained  and  supported  by  HipHip  Software.  The  technical  support  staff  that  advises  the  students 
and  oversees  the  web  servers  that  host  the  software  and  product  information  entered  by  the  students  is  located  in 
Miami,  Florida.  Survey  data  was  analyzed  using  descriptive  statistics,  correlations,  and  simple  regressions  among 
key  variables. 

Findings 

The  sample  size  consists  of  43  students  in  the  MBA  program  of  the  School  of  Business,  Montclair  State 
University,  who  all  took  the  INFO  503  class  in  Management  Information  Systems,  with  special  emphasis  on 
Electronic  Commerce.  The  students  were  at  various  stages  in  the  MBA  program  and  majority  of  them  hold  full-time 
jobs  in  industry. 

Students  were  asked  to  respond  to  a five-point  Likert  scale  for  different  items  where  1 =Strongly  disagree, 
2=Disagree  to  some  extent,  3=Uncertain,  4=Agree  to  some  extent,  and  5=Strongly  agree.  Most  of  the  students  felt 
that  the  retail  storefront  project  was  an  important  learning  experience  overall  (mean=4.44).  Students  found  the 
project  instructive  in  terms  of  teaching  them  new  ways  of  doing  business  over  the  World  Wide  Web  (mean=4.35). 
They  also  thought  that  the  project  taught  them  skills  they  could  use  after  the  completion  of  the  course,  particularly 
the  setting  up  of  an  online  Web  site  for  a product  or  service  (mean=4.30).  They  also  felt  the  project  gave  them  a 
chance  to  create  Web  site  content  that  was  meaningful  to  the  entire  team  (mean=4.30).  In  the  construction  of  the 
storefront  site,  they  thought  that  they  learned  the  following  specific  skill  sets  in  decreasing  order  of  importance: 
design  issues  for  creating  an  online  catalog  (mean=4.30);  overall  planning  issues  for  preparing  an  online  retail  web 


site  (mean=4.14);  web  site  product  information  and  content  issues  (mean=4,07);  advertising  (mean=3.84);  and 
product/service  promotional  activities  (mean=3.74). 

The  next  important  set  of  findings  relate  to  the  students’  perceptions  of  how  this  project  enhanced  their 
ability  to  collaborate  with  peers  in  a team  setting.  Most  students  found  the  project  a good  venue  for  encouraging 
collaborative  teamwork  (mean=4.40).  Communication  skills  were  paramount  in  getting  the  work  done 
(mean=4,33).  While  standards  for  personal  accountability  were  set  within  the  team  setting  (mean=4,30),  members 
made  sure  they  also  helped  each  other  out  (mean=4.28).  The  project  clearly  taught  the  students  a variety  of  group 
process  skills:  (a)  looking  out  for  the  best  interest  of  all  concerned  over  and  above  one’s  own  personal  interests 
(mean=4.26);  (b)  arriving  at  mutually  acceptable  solutions  (mean=4,26);  (c)  dividing  the  work  equitably  among 
members  (mean=4.23);  (d)  practising  interpersonal  social  skills  (mean=4.21);  (e)  learning  from  members’  different 
ways  of  thinking  (mean=4,21);  (0  learning  positive  interdependence  (mean=4.21);  (g)  respecting  a diversity  of 
opinions  (mean=4,16);  and  (h)  striving  for  goals  as  a team  rather  than  as  individuals  (mean=4. 12). 

They  gained  a better  understanding  E-Commerce  as  a subarea  of  Management  Information  Systems  (MIS) 
in  the  School  of  Business  curriculum  (mean=4.07).  The  storefront  project,  which  is  a natural  cross-over  between 
marketing  and  MIS,  made  them  more  aware  of  the  interdisciplinary  connections  within  the  business  curriculum 
(mean=3,95).  They  perceived  the  project  as  a venue  for  learning  about  E-Commerce  vendors  of  hardware  and 
software  (mean=4,28),  consulting  services  (mean=4.12),  professional  associations  (mean=3.79),  and  job  openings 
(mean=3.63). 

It  is  very  likely  that  the  positive  experiences  reported,  thus,  far  would  not  have  been  possible  if  the 
Merchandizer  storefront  software  itself  was  difficult  to  use.  Most  students  found  learning  the  software  a fairly  easy 
task  (mean=4.19)  and  that  they  found  it  was  a reachable  goal  to  be  skillful  in  the  use  of  it  (mean=4.14).  The 
Merchandizer  software  templates  were  easy  to  understand  (mean=4.07)  so  that  students  were  able  to  learn  them  on 
their  own  without  extensive  instruction  (mean=4.05). 

Two  important  attributes  of  students  in  the  sample  were  measured:  frequency  of  computer  use  and  length 
of  time  spent  working  with  the  computer.  In  a simple  regression  procedure,  it  was  found  that  frequency  of  computer 
use  significantly  predicts  students’  perception  of  the  ease  of  learning  the  Merchandizer  software  at  the  significance 
level  of  p<.10  (F=3.24284;  p=.0791).  In  two  other  simple  regression  runs,  it  was  found  that  the  students’  perception 
of  the  project  as  having  enhanced  collaborative  team  work  predicts  their  perception  of  the  storefront  as  being  an 
important  learning  experience  overall  (F=l  1.42868;  p=.0016)  and  their  ability  to  create  a web  site  with  content  that 
is  meaningful  to  the  entire  team  (F=3,59981;  p=.0648).  Then,  students’  perception  of  Merchandizer  as  an 
application  software  that  was  easy  to  use  predicted  their  organizing  skills  in  arranging  web  site  information  so  that 
the  site  could  be  presented  more  effectively  (F=l 2,275 18;  p=.001 1). 

When  taken  together  as  two  independent  variables  in  a regression  equation,  both  the  students’  perception 
of  the  project  as  having  enhanced  collaborative  team  work  and  their  perception  of  Merchandizer  as  an  application 
software  that  was  easy  to  use  predict  a number  of  dependent  variables.  Significance  levels  were  tightest  for  their 
organizing  skills  in  arranging  web  site  information  so  that  the  site  could  be  presented  more  effectively  (F=7.28378; 
p=.0020)  and  their  perception  of  the  storefront  as  being  an  important  learning  experience  overall  (F=7. 14604; 
p=.0022).  Both  independent  variables  in  a regression  equation  also  significantly  predicted  student’s  perception  that 
the  storefront  project  encouraged  creative  thinking  (F=4. 12591;  p=.0235),  their  perception  that  the  project  enhanced 
their  integrating  skills  or  their  ability  to  put  information  or  content  together  (F=3 .40767;  p=.0430),  and  their 
perception  that  the  project  developed  useful  skills  in  web  site  development  they  could  use  even  after  the  course  was 
over  (F=2.62482;  p=.0849). 

The  association  among  selected  variables  using  Pearson’s  correlation  values  was  also  investigated.  Only 
those  variables  that  bonded  very  strongly  at  the  significance  level  ofp<.0001  are  shown  here. 

Learning  important  lessons  in  determining  web  site  product  information  and  content  issues  correlated 
strongly  with  the  following  seven  variables:  learning  positive  interdependence  in  the  team  (.4999);  learning 

different  ways  of  thinking  in  the  team  (.5271);  achieving  the  goals  as  a team  (.5511);  considering  the  storefront 
project  an  important  learning  experience  overall  (.5193);  creating  web  site  content  meaningful  to  the  team  (.4843); 
developing  skills  usable  even  after  the  course  such  as  building  a web  site  (.6225);  learning  overall  planning  issues  in 
a storefront  (.6334);  and  learning  design  issues  for  an  electronic  catalog  (.6854). 

Learning  how  to  achieve  goals  as  a team  correlated  strongly  with  five  other  variables:  perceiving  the 
project  as  encouraging  collaborative  team  work  (.7380);  learning  to  communicate  with  team  members  (.7264); 
learning  positive  interdependence  (.8872);  learning  interpersonal  social  skills  (.8041);  and  learning  how  to  achieve 
goals  as  a team  (.9168). 

Learning  design  issues  for  creating  an  online  catalog  correlated  strongly  with  the  following  four  variables: 
perceiving  the  project  as  encouraging  collaborative  team  work  (.5269);  learning  to  communicate  with  team 
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members  (.5038);  learning  different  ways  of  thinking  in  the  team  (.5784);  and  learning  how  to  achieve  goals  as  a 
team  (.5380). 

Learning  different  ways  of  thinking  in  the  team  correlated  strongly  with  four  other  variables:  perceiving 
the  project  as  encouraging  collaborative  team  work  (.7333);  learning  to  communicate  with  team  members  (.7026); 
learning  positive  interdependence  in  the  team  (.8792);  and  learning  interpersonal  social  skills  (.8245). 

Conclusion 

This  paper  has  shown  concrete  data  supporting  the  promise  of  the  use  of  the  WWW  for  instructional 
purposes  in  the  area  of  E-Commerce  in  the  business  curriculum.  Not  only  does  the  WWW  provide  an  environment 
that  perfectly  simulates  the  subject  of  the  course  itself,  but  the  web  hosting  service  arrangement  also  provides 
information  technology  infrastructure  support  to  universities  that  are  resource  poor  or  unable  to  afford  the  hardware, 
software,  and  manpower  requirements  of  mounting  E-Commerce  projects.  The  survey  results  provide  positive 
indications  for  the  achievement  of  the  goals  of  collaborative  inquiry,  team-based  and  active  learning,  constructivist 
pedagogical  approach,  and  learning  by  doing. 
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Abstract:  New  developments  in  educational  technology  at  the  University  of  Twente  are  providing  instructors  with  a set  of 
tools  and  templates  for  WWW-based  course-support  environments.  Those  templates  can  be  used  by  the  instructors  to  fill  in 
learning  activities  and  study  material  for  all  students,  both  local  and  at  a distance.  The  task  for  the  instructional  designer  is 
to  create  these  general  templates  and  make  them  suitable  for  different  kinds  of  teaching  situations.  To  guide  the  instructors 
in  how  to  use  those  tools  and  templates,  a series  of  design  guidelines  for  the  use  of  the  WWW  have  emerged  for  each  of  the 
following  didactical  activities:  face-to-face  sessions,  self-study  activities,  assignments,  monitoring  and  feedback.  Those 
guidelines  are  based  on  the  experience  of  the  Faculty  of  Educational  Science  and  Technology  at  the  University  of  Twente 
with  the  implementation  of  approximately  44  courses  supported  by  WWW-based  environments  in  the  context  of  the 
TeleTOP  Project  fhttp://teletop.edte.utwente.nlA  as  well  as  on  instructional  principles  that  predate  WWW  use.  The  paper 
concludes  by  noting  the  need  for  a course  (re-)design  model  which  combines  the  guidelines  into  an  overall  perspective. 

A new  role  for  instructional  designers. 

New  developments  in  educational  technology  as  well  as  new  insights  in  the  experiences  of  instructional  design  and 
information  and  communication  technologies  are  stimulating  new  roles  for  instructional  designers.  Their  major  tasks  are  not  to 
transmit  the  educational  content,  but  to  design  activities;  not  to  design  hyperlinked  learning  materials,  but  to  design  and 
develop  templates  which  can  be  used  in  multiple  situations  in  which  students  are  active  and  using  WWW  tools  and 
environments  to  support  and  manage  that  activity.  Their  task  is  to  design  these  tools  and  functionalities,  so  that  instructors  in 
turn  can  design  their  own  activities  and  study  material  with  those  tools. 

At  the  Faculty  of  Educational  Science  and  Technology  a WWW-based  course-management  system  has  been  developed  in 
which  tools  and  functionalities  are  collected  in  a database  which  serves  as  a general  base  of  templates  (This  system  is  called 
TeleTOP;  see  Collis  & De  Boer,  1998;  see  http://teletop.edte.utwente.nn.  The  instructor  chooses  the  templates  or  functions  he 
or  she  desires  to  support  the  course,  and  only  has  to  type  into  various  types  of  fill-in  forms  to  organize  the  notes  and  other 
material  that  is  to  be  put  into  the  WWW  environment.  If  a template  is  not  available  that  meets  the  instructor's  needs,  the 
TeleTOP  team  creates  it. 

As  the  instructors  are  confronted  with  the  task  of  choosing  tools  and  functionalities  for  their  courses,  they  express  a need 
for  educational  and  didactical  guidance  (Evaluatiewerkgroep  TeleTOP  en  C@mpus+,  1999).  In  response  to  this  growing  need, 
the  new-generation  instructional  designers  of  the  TeleTOP  team  have  created  a set  of  design  guidelines  for  instructors 
(Remmers,  1 998).  Those  guidelines  can  be  sorted  around  five  general  types  of  didactical  activities  which  focus  on  a direct 
learning  process:  face-to-face  sessions,  self-study  activities,  assignments,  monitoring  and  feedback  (Collis,  1997). 
Communication  is  a major  aspect  within  all  the  activities.  For  each  didactical  activity  a variety  of  different  types  of  (WWW- 
based)  applications  can  be  used.  In  Table  1 an  overview  of  a sample  of  such  applications  is  listed  for  each  activity,  as  well  as 
the  numbers  of  the  guidelines  that  are  related  to  those  activity-application  combinations  (the  numbered  guidelines  are 
elaborated  later  in  this  paper). 


Table  1 : Didactical  activities,  their  applications  and  the  related  guidelines 


Didactical  activities 

Applications 

Guidelines 

Face-to-face  sessions 

1 . Preparation  of  material 

2.  PowerPoint  presentation  materials 

3.  Discussions 

4.  Small  assignments 

2 

2 

2 

2 

Self-study  activities 

5.  Readings  and  textbooks 

6.  Quizzes 

7.  Simulations 

8.  Uses  of  asynchronous  video 

3,5 

3,5 

3,5 

5.6 

Assignments 

9.  Small  assignments 

1 0.  Follow-up  assignments 

1 1 . Asynchronous  discussions 

1 2.  Quizzes 

13.  Students  contribute  to  the  study  material 

14.  Case  studies 

7 

4,7 

7 

7 

7.8 

7.8 

Monitoring  and  feedback 

15.  Individual  feedback 

1 6 . Group  feedback 

1 7.  Administration  of  submitted  work  of  students 

18.  Peer  evaluation 

19.  Student  digital  portfolio 

20.  Quizzes 

12 

12 

9,  11 

12 

9 

9,  10 

In  the  next  sections  the  didactical  activities  are  discussed  in  more  detail.  Every  section  discusses  the  following:  (a) 
backgound  information;  (b)  the  manner  by  which  TeleTOP  supports  the  activity  and  what  tools  are  available  for  the  TeleTOP 
instructors;  and  (c)  guidelines  derived  from  practice  and  theory. 


Face-to-face  sessions  and  the  WWW  environment 

Facc-to-face  contact  benveen  the  instructor  and  the  student  as  well  as  between  snidents  are  still  not-to-be  neglected 
instructional  activities  even  when  the  WWW  is  extensively  used.  “It  is  to  the  learner's  advantage  to  occasionally  come  to  a 
face-to-face  setting  interacting  directly  with  the  instructor  and  peers.  Some  learning  experiences,  plus  the  more-intangible 
aspects  of  developing  as  an  intellectual  (called  in  Dutch,  academiscli  vovuung)^  are  more  than  the  sum  of  reading  books  and 
completing  assignments  from  the  convenience  of  one's  home  (Carleer  & Collis,  1998).  Moreover  certain  skills  as,  for 
example,  applying  statistical  formulas  are  sometimes  only  learned  by  the  active  discourse  between  instructor  and  smdent,  at 
least  for  some  students. 

During  the  contact  session:  For  students  who  live  at  a long  distance  from  the  educational  institution,  the  amount  of  face-to- 
face  sessions  must  be  kept  to  a minimum  because  of  time  and  money  issues.  Therefore,  the  face-to-face  sessions  should  be 
used  to  address  only  the  most  important  subject  matter  or  activities  (see  Guideline  I and  Figure  I).  The  evaluation  study  of 
TeleTOP  concluded  that  part-time  students,  students  who  carry  out  a profession  elsewhere  and  study  part-time,  and  students 
with  a part-time  job  particularly  wish  to  have  elaborations  of  the  study  material  in  the  face-to-face  sessions,  not  to  have  the 
time  filled  up  with  listening  to  organisational  aspects  of  the  course,  which  they  can  read  in  the  WWW-based  course 
environment  (Evaluatiewerkgroep  TeleTOP  en  C(§mpus-^.  1999).  Thus,  Guideline  1: 


1 . Use  face-to-face  sessions: 

• 

• 

to  provide  the  context  of  the  course  and  the  subject  matter 
to  show  relations  between  topics  in  the  learning  material 

• 

• 

• 

• 

to  explain  difficult  and  essential  issues 

to  show  (good)  examples  of  applications  or  assignments 

to  make. the  students  enthusiastic  and  inspire  them 

to  let  the  students  interact  and  have  discussions  with  one  another 

• 

• 

to  reflect  upon  submitted  work 

to  invite  expert  guest  speakers  on  the  subject  matter 

— 

Do  1101  use  the  face-to-face  sessions: 

• 

to  provide  organisational  aspects  of  the  course 

• 

to  repeat  what  students  can  read 

Figure  1 reflects  Guideline  1,  showing  how  a face-to-face  session  was  used  in  the  course  ’Tele-Learning",  not  as  a lecture,  but 
for  a variety  of  activities,,  capitalising  on  the  presence  of  a guest  visitor  from  another  institution  whose  work  relates  to  the 
study  material  for  the  week. 


Also,  it  is  necessary  to  prepare  a contact  session  so  that  students  who  were  not  able  to  be  physically  present  (especially  part- 
time  working  students)  can  also  participate,  either  at  the  same  time  but  at  a distance  or  at  a later  time.  This  typically  has  three 
aspects:  (a)  any  content  material  to  be  elaborated  in  the  contact  session  by  the  instructor  is  prepared  in  note  form  in  advance 
and  made  available  on  the  WWW  environment  for  students  who  will  not  be  present:  (b)  an  interactive  activity  (a  discussion,  a 
short  analysis  task,  a problem  to  work  out,  etc.)  is  devised  that  can  involve  the  students  attending  the  contact  session  at  the 
same  time  (and  perhaps  some  of  them  in  different  locations)  so  that  the  students  are  actively  engaged  in  some  sort  of 
articulation  and  collaborative  learning:  (c)  support  materials  to  guide  this  interactive  work,  plus  to  be  used  by  the  students  to 
report  on  their  work,  are  created  and  put  in  the  WWW  course  environment  (see  Guideline  2). 

2.  Prepare  the  face-to-face  session  so  that  students  not  physically  present  can  also  participate  (synchronously  or  asynchro- 
nously), by  providing  the  content  material  to  be  elaborated,  an  interactive  activity,  and  support  materials  to  guide  this 
interactivity. 


Students'  preparation  before  the  session:  For  the  snidents  to  carry  out  an  assignment  during  a face-to-face  session,  to  have 
discussions,  and  to  reflect  on  assignments  it  is  necessary  for  them  to  have  a basic  understandnig  of  the  subject  matter  before 
they  come  to  the  session.  For  this  reason  the  TeleTOP  team  has  desired  a specific  kind  of  matrix-like  strucnire  called  a 
roster,  containing  self-study  activities  to  be  carried  out  before  the  session  (see  Guideline  3 and  Figure  2)  that  are  integrated 
(via  being  in  the  same  row)  with  notes  and  activities  for  the  face-to-face  sessions. 
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3.  To  provide  students  with  the  necessaiy  basic  understanding  of  the  subject  matter,  give  them  self-study  activities  before 
the  session,  integrated  in  the  WWW  environment. 


Figure  2:  Example  of  the  roster  of  a WWW-bMed  course-support  environment  in  which  the  students  have  to  study  certain 
literature  before  the  class  session;  Principles  of  Learning  and  Instructional  Design  CC3.  1999 


Follow-up  after  the  session:  After  the  session  a follow-up  activity  to  stnicture  students'  reflections  oi^e  contact  session  md 
how  it  integrates  with  their  self-study  materials  and  their  other  course  activities  can  be  planned.  The  instructions  for 
activity  and  the  reporting  form  for  the  students  to  use  are  entered  into  the  WWW  environment  before  the  contact 
cut-off  time  should  be  indicated  for  the  submission  of  the  follow-up  activities  from  the  students,  and  after  t^is  time  ^e 
instructor  should  look  at  their  submitted  comments  and  post  a feedback  response  in  the  WWW  environment  or  by  email  ( or 
example,  a different  comment  may  be  appropriate  to  the  part-time  students  than  to  the  local  students).  See  Guideline  4. 

4 To  structure  students’  reflection  on  the  contact  session  and  to  reUte  it  to  the  study-materials,  provide  studj^wi^  a 
■ follow-up  activity  and  a WWW-based  template  through  which  they  submit  their  work  relating  to  this  activity,  and 

indicate  a cut-ofF time  for  this  submission. 


The  orofiie  of  the  ideal  professional  practitioner  requires  that  students  should  possess  a solid  knowledge  base  wth  regard  to 
the  profession  (Verhagen  et  al.  1999).  In  general,  self-study  could  be  used  for  rehearsal  of  Imowledge  and  for  reflection  on^is 
knowledge.  Reflection  is  oriented  towards  realising  learning  tasks:  to  process  and  to  refine  insight  and  understanding  (Oe 

Sshi?y  Activities  are  mostly  used  in  TeleTOP  as  a preparation  for  face-to-face  sessions  and  assignments  and  for  the 
reolacement  of  face-to-face  sessions  if  necessary.  For  self-study  activities,  as  well  as  for  assignments,  it  is  veiy  important  ttat 
they  are  not  a stand-alone  and  independent  activity.  They  should  be  implemented  in  the  course  WW  enviroi^t  wtth  for 
example  an  explanation  of  the  relevant  issues  of  the  self-study  activity,  the  active  involvement  of  the  learner,  fee^wk  from 
the  instructor  and  repetition  (Westera,  1999,  see  Guideline  5);  otherwise  students  will  not  feel  much  value  m the  activity. 

1 For  self-study  activities  to  be  more  effective,  provide  students  with  the  relevant  issues  to  interpret  in  tenna  o^e  stutty 
materials,  provide  WWW-based  templates  for  *e  students  that  structure  their  active  involvement,  feedback  anfl^ 

repetition. 

The  following  applications  could  be  used  in  TeleTOP  for  self-study  activities:  study  guides,  quizzes,  simulations,  and 
asynchronous  audio  and  video.  Quizzes  will  be  discussed  later  in  this  paper. 

Asynchronous  audio  and  video:  For  snidents  who  were  not  participating  in  the  contact  session  as  it  occured.  the  instructor  hM 
the  possibility  to  capture  via  digitised  audio  and/or  video  anything  needed  to  make  the  notes  more  efTcctive  for  those  studente 
who  will  study  the  materials  later.  In  this  way  the  communication  event  is  captured  and  can  be  re-used  (Peters  & Loiiis,  lyyv). 
Other  reasons  for  using  asynchronous  video  are  described  in  more  detail  in  Peters  and  Collis  (1999)  and  are  summarized  in 

Guideline  6. 


6.  Use  asynchronous  audio  and  video: 

• to  capture  presentations  or  lectures 

• to  give  access  and  depdt  to  real  events 

• to  support  the  learning  process  in  situations  involving  procedures  or  social  situations 

• to  develop  a library  of  cases  or  units  of  learning  material 

• to  capture  and  share  one's  own  video  materials 

Self-study  activities  are  primarily  oriented  around  the  acquiring,  rehearsal,  and  reflective  consideration  of  knowledge,  whereas 
assignments  are  more  focussed  on  the  application  of  the  students’  knowledge  base  and  insights  into  practice  (sec  the  next 

sextion). 


o 

ERIC 


It  is  ucnerally  accepted  that  if  you  “put  students  to  work”  they  Icam  more  (Dc  Jong,  1995:  Collis,  1998).  For  this,  the  term 
action  learning  is  used.  Meyers  and  Jones  (1993)  identify  four  major  elements  associated  with  action  learning  which  students 
use  to  create  new  mental  structures:  speaking,  listening,  reading,  and  writing  and  reflecting.  Those  elements  refer  to  cognitive 
activities  which  enable  students  to  clearly  investigate,  consolidate,  and  acquire  new  knowledge.  For  example,  when 
conducting  a discussion,  the  process  of  telling  clears  the  thoughts  of  students,  whereas  listeners  have  the  possbility  to  retlect 
on  alternatives  to  one’s  own  perspectives  (Peters,  1998) 
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Within  TeleTOP  there  are  several  ways  to  put  the  students  to  action:  small  assignments  (before,  dunng,  after  or  in-between 
contact  sessions)  working  on  case  studies,  making  contributions  to  the  course  study  matenal.  developing  their  own  research 
questions  and  discussions,  and  contributing  materials  that  can  be  used  as  model  answers  or  help  resources.  For  assignments  to 
be  effective  some  of  the  issues  mentioned  in  Guideline  5 are  also  important  for  the  implementation  of  assignments.  These  are 

summarized  in  Guideline  7. 

7.  To  help  make  assignments  be  more  effective,  provide  students  with  relevant  issues  relating  to  the  topics  to  be  u« 

WWW-based  templates  that  help  structure  and  manage  the  active  involvement  of  the  students  and  that  integrate  feeabacK 
with  the  assignments;  when  possible,  use  smdent  submissions  as  a basis  for  shidy  and  reflection  in  the  course  itself. 

Small  assignments:  One  of  the  major  orientations  of  the  TeleTOP  method  is  to  reduce  the  number  of  lectures  ^d  increase  the 
number  of  "in-between  assignments”,  relatively  small  assignments  to  which  feedback  is  quickly  given.  The  process  for 
handling  this  is  supported  within  the  Roster  of  the  course-support  environment  (see  Figure  2).  The  tnck  is  to  make  the 
management  of  this  process  as  automatic  as  possible,  while  still  maintaining  instnictor  contact  when  useful.  This  becomes  the 
major  task  of  the  instructor:  to  think  of  these  assignments,  and  of  how  they  can  be  carried  out  efficiently  and  effectively  (l  e., 

both  instructor  and  student  getting  regular  insights  into  how  the  learning  process  IS  going  on).  ,,nxnxr 

In  TeleTOP  the  instructors  have  the  possibility  to  let  the  students  submit  their  work  on  (small)  assignments  in  the  www 
environment  during  a face-to-face  session,  but  also  before  or  after  a session.  Before  the  session,  to  help  the  students  become 
familiar  with  the  subject  matter.  After  the  session,  to  let  the  students  reflect  on  the  presentation  of  the  instructor,  on  the 
answers  to  the  assignments,  or  on  ideas  that  emerged  during  the  face-to-face  session,  etc.  However,  assignments  need  to  be 
assessed  or  else  students  tend  not  to  do  them  and  the  learning  potential  is  not  realized.  "Although  students  appiwiated  the 
follow-up  assignments,  they  often  did  not  have  time  to  carry  them  out  if  they  were  not  given  points  but  only  offered  on  a 
voluntary  basis”  (Dijkstra,  CoIIis  & Eseryel,  1999). 


Students  ‘ contributions  to  study  ma/eria/:  Another  major  orientation  of  TeleTOP  is  that  smdents  can  contribute  smdy 

to  the  course-support  environment.  This  can  occur  in  many  ways:  for  example,  through  adding  to  a FAQ  (frequently  asked 

questions)  collection,  adding  to  a database  of  examples,  supplying  model  answers,  adding  links  to 

definitions  to  a course  glossary,  and  constructing  and  sharing  one’s  own  video  & WWW  resources  (Collis,  1998).  The  most 
important  function  of  the  students  contributing  to  the  course  study  material  is  to  stimulate  a more-active  learning  process 
amone  the  smdents,  by  providing  valuable  exercises  and  letting  them  participate  in  building  the  course  environment  by  finding 
or  creating  resources  and  sharing  them  with  others  (Dijkstra,  Collis  & Eseryel,  1999).  For  example.  “S^nts  appreciated 
being  able  to  read  the  contributions  of  other  students  in  the  shared  workspace”  Dijkstra,  Collis  & Eseryel  (1999). 


IVorking  together  on  case  studies:  If  correctly  applied,  case  smdies  are  suitable  for  developing  skills  for  problem-solvirig,  for 
critical  thinking,  and  for  creativity  (Romiszowski,  1995).  For  part-time  smdents  who  carry  out  a profession  as  weft  follow  a 
smdy  exercises  with  case  smdies  can  be  done  by  placing  the  case  smdy  in  the  contexts  of  their  work.  The  TeleTOP  system 
allows  different  assignments  to  be  described  for  different  groups  of  smdents;  the  smdents  only  see  the  descnptions  for  the 

croups  to  which  they  belong.  . . • i 

For  both  the  smdents’  contributions  to  smdy  material  and  for  the  case  smdies,  TeleTOP  offers  the  tools  workspace  and 
presentation  At  the  workspace  smdents  can  work  together  on  certain  assignments  and  only  the  group  members  can  enter  the 
croup  workspace.  They  can  work  for  example  in  an  on-going  way  on  their  case-smdy  exercises  or  their  self-develo^d  smdy 
material  (see  Figure  3).  To  make  the  workspace  not  too  complicated,  overly  fulled  with  files  and  comments,  the  smdents  can 
present  their  final  work  in  the  section  presentation.  This  function  could  present  in  a well-organised  overview  the  final 

products  of  the  groups  for  all  smdents  to  access  (see  Guideline  8). 

8.  To  let  the  smdents  actively  learn  or  develop  skills  for  problem-solving,  critical  thinking  and  creativity,  use  smdents 
contributions  to  smdy  material  and  case  smdy  exercises  as  resources  in  the  course  WWW  environment: 

• Let  the  smdents  work  together  in  a shared  workspace  and  present  their  final  products  in  a separate  part  of  the  course 
environment 

• For  part-time  smdents  use  the  context  of  their  work  experiences  for  their  case  smdies. 


Figure  3:  Students  working  together  in  the  Workspace  (in  Dutch:  Werkplaats).  fCT^projcct  2000  course.  1999. 


Monitoring  and  feedback  j . j r 

One  of  the  reasons  for  monitoring  which  Dodman  and  Coxhead  (1994)  have  put  forward  is:  monitonng  students  and  groups  of 

smdents  to  see  which  parts  the  smdents  do  not  master  and  which  they  do  not  master  very  well.  In  classical  simations 


monitoring  takes  place  on  the  basis  of  the  experience  and  knowledge  of  the  expert,  the  instructor  (Winnips,  1998).  In 
teleleaming  situations  the  instructor  may  not  always  be  present  physically,  which  could  result  in  limited  feedback  on  the 
progress  of  the  individual  students  or  on  a group  of  students.  By  applying  monitoring  strategies,  this  progress  can  be  made 
clear  and  actions  can  be  undertaken  if  problems  should  occur. 

TeleTOP  offers  the  instructors  a number  of  tools  which  could  make  the  monitoring  of  students  and  group  of  students  easier 
(see  Guideline  9).  Quizzes  (with  multiple-choice  questions,  short  answers,  true-false,  ete.)  and  clear  overviews  of  the 
submilled  work  of  students  are  all  being  used  by  the  instructors  of  the  faculty  of  Educational  Science  and  Technology.  In  the 

beginning  of  August  1999  the  faculty  initiated  a digiial  pori/olio  project  (see  Guideline  9): 

9.  To  monitor  the  work  and  progress  of  the  students,  use  WWW-based  quizzes,  overviews  via  the  WWW-site  of  the 
submitted  work  of  students,  and  digital  portfolios  accesible  via  the  WWW-site. 

Quiz-es-  When  using  quizzes  in  his  or  her  course,  the  instructor  can  get  a glance  at  the  understanding  of  the  students  relating 
to  the  topics  which  are  covered  in  the  quizzes  (see  Guideline  10  and  Figure  4).  The  answers  to  the  questions  in  the  quiz  are 
sent  to  the  instructor  and  if  a student  answered  (several)  incorrect  answers,  the  instructor  could  take  action  and  give 
appropriate  feedback  to  the  student.  Students  can  also  submit  quiz  questions  themselves,  as  small  assignments  dunng  the 

course. 

10  Use  WWW-based  quizzes  to  monitor  the  understanding  of  the  students  of  the  subject  matter.  Let  the  answers  be  sent  to 
you  via  the  WWW-site.  Consider  the  strategy  of  having  smdents  contribute  quiz  questions  via  the  WWW-site  as  an 
assignment,  which  are  then  used  by  all  the  students  for  self-study. 


a 


Figure  4:  Questions  created  by  the  instructor  in  the  course  Instructional  Theory  I , / 999 

Overview  of  submitted  work:  As  the  instructor  has  chosen  that  the  students  will  submit  assignments  through  the  roster,  the 
instructor  is  able  to  get  an  overview  of  each  item  of  submitted  work  of  each  student  and  can  give  the  student  immediate 
feedback  or  remedial  help  and  perhaps  a mark  or  some  number  of  points  (see  Guideline  1 1 and  Figure  5).  If  the  instructor 
decides  to  give  the  students  points  for  the  assignments,  the  overview  will  also  show  the  total  amount  of  points  per  student  at 
each  moment  in  time.  The  instructor  should  keep  track  of  the  student's  submissions,  and  send  a message  to  students  who  did 
not  submit  to  find  out  if  they  are  having  particular  problems  (this  message  can  be  pre-structured,  so  the  time  and  effort  needed 
to  send  it  can  be  small). 


1 1 To  monitor  the  comprehension  of  the  students  as  to  how  to  apply  their  recently  acquired  knowledge  into  practice,  use  an 
overview  of  the  submitted  work  of  students  in  which  ditect  links  are  available  to  the  work  of  the  student,  the  submitted 
date,  the  feedback  to  the  smdents,  the  points  given;  all  ordeted  by  smdent 


Figure  S:  Overview  of  the  submitted  work  of  the  TiudenTs  and  of  the  given  feedback  (checkmarks)  and  points.  Telelearning 
course,  1999, 

Feedback:  When  feedback  is  needed,  as  a consequence  of  monitoring  or  as  a reflection  on  the  work  of  smdents.  the  instructor 
has  the  possibility  to  (a)  provide  individual  feedback  via  the  course  environment  (this  feedback  can  only  be  seen  by  the 
instructor  and  the  particular  smdent).  (b)  provide  public  feedback  (feedback  that  is  meant  for  one  smdent,  but  can  be  read  by 
all  smdents).  (c)  provide  feedback  to  a group  of  smdents,  (d)  give  one  key-answer  to  all  smdents.  and/or  (e)  let  smdents  give 
feedback  to  themselves  and  each  other  (self-  and  peer  evaluation).  See  Guideline  12. 
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12.  Give  the  students  via  the  WWW-site: 

• personal  feedback  when  they  all  have  to  cany  out  the  same  assignment. 

• public  feedback  when  you  want  them  to  leam  from  each  other’s  answers  or  maybe  use  each  other’s  answers  for  a 

new  assignment  . r ju  u 

• group  feedback  when  the  numbers  of  students  of  the  course  is  large  and  could  not  be  handled  with  personal  feedback 

with  regard  to  time  constraints 

• one  answer  key  to  all  students  when  many  students  make  the  same  mistakes  in  their  answers. 

• peer  feedback  when  you  want  to  guide  the  students  in  learning  how  to  evaluate  and  give  feedback 

Digital  portfolios:  As  a current  development  for  the  Faculty  of  Educational  Science  and  Technology,  the  students'  overall 
progress  within  all  their  courses  is  being  monitored  by  the  use  of  students’  digital  portfolios.  Individual  assessment  will  «ill 
take  place  by  the  instructors  of  the  particulair  courses,  but  overall  monitoring  and  support  will  be  done  by  faculty  members 
serving  as  mentors.  This  type  of  support  is  designed  to  stimulate  the  professional  growth  of  the  student  by  examining  his  or 
her  added  products  and  documentary  evidence  relating  to  his  or  her  work. 

How  to  re-design  your  course  for  the  use  of  a WWW-based  course  support  environment? 

The  above-mentioned  guidelines  and  applications  are  a handle  for  the  instructors  when  they  are  (re-)designing  their  courses  to 
make  them  suitable  for  WWW-based  course  support  or  tele-learning  purposes.  Although  the  instructional  principles 
underlying  the  guidelines  may  not  be  new  in  themselves,  what  is  new  are  increased  possibilities  for  putting  these  principles 
into  practice  through  the  support  of  WWW-based  tools  and  environments.  Also,  WWW  tools  and  environments  arc  being  used 
by  many  instructors  (i.e.,  in  technical  faculties)  who  are  not  familiar  with  instructional  design  theories.  The  guidelines  are 

written  in  a pracical  way  to  steer  the  use  ofthe  WWW  for  good  pedagogical  practice.  . ^ , 

The  set  of  guidelines  can  be  elaborated  with  new  experiences  of  instructors  and  students,  as  well  as  with  new  developments  in 
educational  concepts.  On  a regular  basis,  new  guidelines  are  being  designed  and  added  to  the  set  of  guidelines  that  we  "ow 
have.  However,  there  is  also  a need  to  put  the  guidelines  in  a broader  perspective  in  such  a manner  that  instructors  will  know 
when  to  choose  certain  applications  in  certain  didactical  sihiations  (especially  important  for  novices).  This  could  result  in  an 
instructional  (re)design  model  for  WWW-based  course-support  environments. 
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Abstract:  New  search  tools  and  techniques  that  can  improve  the  efficiency  and 
effectiveness  of  web  searches  are  highlighted,  including  metasearch  tools,  improving 
search  relevance  with  search  engines  (backward  searching  and  other  advanced  search 
features),  and  searching  the  “invisible  web”. 


Comprehensive  Searching  with  Metasearch  Tools 

A recent  article  in  Nature  (July  8,  1999)  by  Steve  Lawrence  and  C.  Lee  Giles  received  wide  attention  in 
the  press.  Lawrence  and  Giles  examined  the  World  Wide  Web  in  February  1999  and  found  that  publicly 
indexed  web  pages  had  increased  to  800  million  (up  from  an  estimated  320  million  m December  1997). 
Their  article  analyzed  how  well  eleven  search  engines/directories  performed  in  accessing  the  information 
contained  in  those  800  million  pages.  Lawrence  and  Giles  estimated  that  the  combined  coverage  of  the  1 1 
tools  was  only  42%  of  the  total  web  and  that  the  overlap  among  the  search  engines  was  surprisingly  low. 
This  research  implies  that  using  multiple  search  engines  is  a necessity  today.  Metasearch  tools  allow  us  to 
do  just  that  --  quickly  and  effectively.  Popular  metasearch  tools  include  ProFusion 
(http://www.profusion.com)>  Metacrawler  (http://www.metacrawler.com)  and  SavvySearch 
(http://www.savvvsearch.com). 


Improving  Search  Relevance  with  Search  Engines 
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Most  of  us  probably  have  a favorite  search  tool,  such  as  Yahoo!,  Infoseek,  or  Excite,  and  there  is  certainly 
something  to  be  said  for  developing  a high  degree  of  familiarity  with  the  search  features  of  a single  search 
engine  or  directory.  However,  in  the  rapidly  changing  world  of  the  web,  it  is  important  to  be  aware  of 
new  search  tools  and  the  features  that  they  use  to  try  to  improve  the  relevance  of  results.  New  search 
algorithms  are  developed  regularly,  usually  in  conjunction  with  a new  search  service. 

Google  (http://www.google.com)  is  one  of  the  most  exciting  search  tools  available  today.  Google’s 
search  algorithm  is  based  on  an  approach  called  PageRank.  The  FAQ  states  that,  “PageRank  capitalizes 
on  the  uniquely  democratic  characteristic  of  the  web  by  using  its  vast  link  structure  as  an  organizational 
tool.  In  essence,  Google  interprets  a link  from  page  A to  page  B as  a vote,  by  page  A,  for  page  B.  Google 
assesses  a page’s  importance  by  the  votes  it  receives.  But  Google  looks  at  more  than  sheer  volume  of 
votes,  or  links;  it  also  analyzes  the  page  that  casts  the  vote.  Votes  cast  by  pages  that  are  themselves 
"important"  weigh  more  heavily  and  help  to  make  other  pages  "important" 

(available  online  at  http://www.google.com/whv  use.html).  Google  allows  users  to  capitalize  on  this  by 
choosing  an  “I’m  feeling  lucky”  button.  When  this  button  is  chosen  the  user  is  directly  linked  to  the  page 
Google  identifies  as  the  most  relevant.  A search  limited  to  United  States  government  sites  is  also 
available  through  Google.  A recent  addition  to  Google  is  a directory,  featuring  a hierarchical  list  of  sites 
organized  by  category.  Unlike  the  more  familiar  Yahoo  directory,  the  Google  directory  is  based  on  the 
same  page  ranking  algorithm  used  by  the  search  engine. 

Oingo  (http://www.oingo.com/)  suggests  that  it  is  the  first  meaning-based  search  engine.  When  a search 
term  or  phrase  is  entered,  Oingo  returns  with  one  or  more  dialog  boxes  that  allow  the  user  to  specify  the 
exact  meaning  of  the  word  or  phrase  in  the  context  of  the  search.  For  example,  when  the  search  string 
“travel  to  Montreal”  is  entered,  Oingo  asks  the  searchers  to  specify  the  meaning  of  two  terms,  travel  and 
Montreal.  For  travel,  the  user  is  asked  to  choose  between  several  shades  of  meaning  (from  commercial 
travel  to  locomotion)  and  for  Montreal,  the  searcher  is  asked  to  specify  one  of  several  different  locations 
for  towns  or  cities  named  Montreal.  After  the  meanings  are  predefined  the  search  is  narrowed  and  results 
are  presented.  Results  include  categories  (from  Oingo)  and  individual  web  sites  (from  Alta  Vista).  Those 
results  that  match  the  predefined  meanings  are  indicated  using  a lightbulb  icon.  Another  interesting 
feature  of  Oingo  is  that  it  asks  for  feedback  from  the  searcher. 

While  Surf  Fast  (http://www.surffast.com)  isn’t  based  on  a new  searching  algorithm,  it  does  provide  “one 
stop  shopping”  for  web  searchers.  This  well  organized  page  provides  keyword  searching  for  all  of  the 
major  search  engines,  plus  quick  access  to  major  news  organizations  and  other  useful  web  resources  such 
as  Mapquest  and  Travelocity.  Surf  Fast  shows  how  important  it  is  to  match  the  kind  of  information  needed 
to  the  search  tool.  Making  this  match  is  crucial  in  improving  search  relevance. 

Chris  Sherman  suggests  the  term  “backward  searching”  for  the  technique  of  using  the  “link”  operator  to 
identify  pages  linked  to  the  URL  that  you  have  entered 
(http://websearch.about.com/librarv/weeklv/aa082499.htm).  The  basic  idea  here  is  that  people  only  add 
links  to  sites  that  they  consider  highly  relevant,  although  searchers  still  need  to  determine  whether  or  not 
popularity  actually  equals  quality.  Backward  searching  is  done  using  a link  operator.  Queries  are  usually 
structured  like  this:  +link:URL.  To  assess  quality,  look  over  the  list  of  “linking”  URLs.  Are  all  of  the 
links  from  individual  pages  or  are  the  links  from  well  respected  sites  like  Kathy  Schrock’s  Guide  for 
Educators  (http://school.discoverv.com/schrockguide/index.html)  or  the  Librarian’s  Index  to  the  Internet 
(http://sunsite.berkclev.edu/lntemetlndex/)?  If  you  find  a promising  page  and  you  follow  these 
“backward”  links,  you  get  a sense  of  whether  others  find  the  page  so  useful  that  they  link  to  it.  Backward 
searching  is  just  one  of  the  many  advanced  search  features  included  in  different  search  engines. 
Unfortunately  it  can  be  a challenge  to  locate  information  about  using  advanced  search  features.  At 
Infoseek  you  have  to  choose  advanced  search  then  choose  help  before  you  come  to  this  information 
(http://infoseek.go.com/Find?pg=advanced  www.htipl&ud9=advanced  www). 


Searching  the  “Invisible  Web” 
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Many  useful  and  relevant  resources  cannot  be  retrieved  using  standard  web  search  tools.  ERIC,  online 
library  catalogs,  and  archival  resources  are  examples  of  resources  that  are  simply  not  retrieved  by  the 
spiders  and  bots  used  to  build  search  engine  databases.  If  you  have  ever  spent  hours  fruitlessly  searching 
for  information  you’re  sure  is  accessible  electronically,  the  information  is  probably  “hidden”  on  the 
invisible  web.  And  if  students  tell  you  that  they  couldn’t  find  even  one  article  on  a common  topic,  they 
have  probably  been  doing  a web  search,  not  searching  in  a specialized  database.  Students  often  do  not 
understand  the  difference  between  information  on  web  pages  and  information  found  in  databases  and 
indexes.  Identifying  places  to  search  for  these  “invisible”  resources  is  one  step  toward  developing  this 
understanding. 

Direct  Search  (http ://gwis2.circ.gwu.edu/'^gprice/di rect.htm)  is  a comprehensive  index  maintained  by 
Gary  Price  at  George  Washington  University.  Direct  Search  provides  links  to  a variety  of  education 
specific  resources,  plus  general  interest  resources  including  Your  Nation  (which  uses  data  from  the  CIA 
World  Fact  Book  to  create  customized,  comparative  statistical  profiles  for  different  countries),  Termium 
Plus  (billed  as  the  definitive  French-English  dictionary),  and  the  Writer’s  Guidelines  Database.  Lycos 
provides  access  to  over  300  searchable  education  databases  at 
http://dir.lvcos.com/Reference/Searchable  Databases/Education/.  Infomine  (http://infomine.ucr.edu/) 
permits  multiple  database  searching,  which  is  a significant  time  saver,  and  it  also  has  two  long  lists  of 
instructional  resources  databases  (for  K-12  and  for  higher  education).  The  Invisible  Web 
(http://www.invisibleweb.com/)  includes  both  quick  and  advanced  search  features.  Like  Direct  Search, 
The  Invisible  Web  was  specifically  designed  to  provide  a well  organized  interface  between  end-users  and 
information  on  the  invisible  web.  Categories  listed  under  “education”  range  from  “Alumni  and  Reunions” 
to  “Distance  Learning”  but  the  category  structure  is  simple  to  follow  and  provides  an  excellent  approach 
to  each  topic.  A Hot  List  of  commonly  searched  topics  is  clearly  presented.  Currently  the  only  search 
enhancements  available  through  advanced  search  are  Boolean  and  phrase  searching. 

Research  by  Hertzberg  and  Rudner  (1999;  “The  Quality  of  Researchers’  Searches  of  the  ERIC  Database’^ 
available  online  at  http://epaa.asu.edu/epaa/v7n25.html))  indicates  that  even  experienced  web  searchers 
rarely  use  advanced  searching  techniques  to  improve  the  quality  of  their  searching.  Citing  results  from  a 
study  of  users  of  the  ERIC  database,  Hertzberg  and  Rudner  conclude  that  “Patrons  are  not  using  effective 
search  strategies  and  cannot  possibly  find  the  best  and  most  relevant  articles  in  the  database  being 
searched.  We  have  reason  to  believe  that  most  end-users  are  satisfied  with  any  somewhat-relevant  hit.”  (p. 
8)  If  this  is  the  case  in  a controlled  searching  environment  like  ERIC,  it  is  even  more  of  an  issue  on  the  web,  where 
search  tools  and  techniques  are  much  less  obvious  to  end-users.  Web  searching  is  a complex  activity  that 
requires  a considerable  amount  of  knowledge,  critical  thinking,  and  practice.  The  good  news  is  that  it  is  a 
skill  that  can  be  perfected,  but  the  bad  news  is  that  the  strategies  and  tools  available  change  daily.  A recent 
visit  to  the  Search  Engine  Guide  (http ://www. searchengineguide. com/)  found  a categorized  listing  of  over 
3200  search  tools,  which  implies  that  there  is  a specialized  search  tool  available  to  meet  virtually  any 
information  need.  Useful  resources  for  keeping  up  with  the  latest  trends  in  searching  include  the  Search 
Engine  Showdown  (http://www.notess.com/search/)  and  Search  Engine  Watch 
(http://searchenginewatch.com/).  Both  sites  specialize  in  providing  current  news  about  web  searching  and 
both  also  include  critical  evaluations  and  comparisons  of  search  tools. 
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Abstract:  This  paper  discusses  an  educational  initiative,  VSQOnline  which  has  enabled  the 
University  of  Southern  Queensland  (USQ),  Australia,  to  deliver  multiple  courses  via  the  Internet  to 
students  worldwide.  The  paper  briefly  outlines  the  underlying  structure  and  philosophy  of  distance 
education  at  USQ  and  then  describes  how  the  online  initiative  has  evolved  from  this  existing 
distance  education  infrastructure.  The  paper  reflects  on  the  conceptualisation  and  initiation  of  the 
VSQOnline  project  and  the  consequences  of  an  apparent  shift  in  the  pattern  of  teaching  and 
learning  in  a higher  education  institution  following  the  introduction  of  online  teaching.  An  interim 
evaluation  conducted  by  the  authors  makes  the  following  recommendations:  open  communication, 
consultation  and  collaboration  should  form  the  basis  of  such  a major  institutional  initiative;  roles 
and  responsibilities  of  all  stakeholders  should  be  clearly  defined;  synergies  and  alliances  with 
strong  partners  are  essential;  and  there  should  be  sound  technological  support  at  all  levels. 


Background 

In  recent  years,  higher  education  worldwide  has  been  faced  with  significant  change  which  has  been  in  response  to  a 
number  of  influences  (Armstrong  et  al.,  1997,  Askew  & Camell,  1998).  These  influences  have  been  the  greater 
emphasis  on  lifelong  learning  and  the  advent  of  the  Information  Age,  resulting  in  globalisation  and  a move  to  a 
knowledge  society.  Also,  economic  rationalism  has  required  universities  to  enter  the  commercial  arena  (Coaldrake 
& Stedman,  1999;  Dearing,  1997).  Student  cohorts  have  also  made  further  demands  on  universities  for  greater 
flexibility  in  the  ways  they  are  able  to  access  programs  and  services.  The  adoption  of  flexible  learning  methodology 
has  been  a key  initiative  in  many  institutions  in  an  attempt  to  adapt  to  this  changing  environment.  In  addition, 
information  and  communication  technologies  have  created  new  educational  opportunities  and  challenges,  which 
impact  upon  distance  education  models  and  theory  and  challenge  the  traditional  roles  of  teachers  and  learners.  The 
paper  will  now  examine  the  impact  of  these  changes  within  the  context  of  an  Australian  institution  of  higher 
education. 

USQ  is  an  internationally  recognised  leader  in  the  provision  of  flexible  learning  opportunities  and  has  a well- 
established  Distance  Education  Centre.  In  1999,  the  Executive  Committee  of  the  International  Council  for  Open  and 
Distance  Education  (ICDE),  based  in  Oslo,  Norway,  awarded  its  inaugural  Prize  of  Excellence  to  USQ  in 
recognition  of  the  University's  significant  contribution  to  providing  global  education.  In  1967,  USQ  was  first 
established  as  a face-to-face  Institute  of  Advanced  Education,  moving  to  full  university  status  in  1992.  In  1977,  it 
presented  itself  as  a viable  alternative  to  traditional  universities  by  offering  distance  education.  Distance  education 
has  been  defined  according  to  several  characteristics:  separation  of  teacher  and  learner;  the  existence  and  influence 
of  an  educational  institution;  use  of  various  media;  provision  of  two-way  communication;  and  an  absence  of  group 
learning  with  the  focus  on  independent  learning  (Keegan,  1986;  Rumble,  1986).  In  Australia,  a typical  distance 


944 


education  learning  package  consists  of  print-based  materials  supported  by  audio,  video  and  computer-based 
resources  and  is  designed  to  enable  learners  to  interact  independently  with  the  materials. 

USQ  delivered  distance  education  while  continuing  with  classroom-based  teaching,  earning  it  a ‘dual  mode’  label. 
This  experience  in  dual  mode  delivery  has  strategically  positioned  USQ  to  take  advantage  of  possibilities  created  by 
the  advent  of  information  and  communication  technologies  and  the  move  to  online  delivery.  The  university  services 
over  14,000  distance  students  from  every  state  in  Australia,  and  many  overseas  countries.  It  also  provides  on- 
campus,  face-to-face  delivery  for  5,000  Australian  and  international  students.  USQ  is  a regional  university  with  an 
international  mission.  Almost  seventy-five  percent  of  the  students  studying  with  USQ  are  off-campus  and  more 
students  study  offshore  through  flexible  learning  modes  at  USQ  than  with  any  other  Australian  university.  Providing 
learning  experiences  at  a distance  using  an  electronic  delivery  mechanism  presents  further  challenges  but  also  opens 
up  a whole  new  world  of  opportunities  and  possibilities.  For  one,  the  approach  questions  the  traditional  definition  of 
distance  education  as  having  an  independent  learning  focus  and  lacking  in  group  interaction. 

USQ  delivered  its  first  course  solely  online  in  1997.  In  1999,  a major  online  initiative  called  VSQOnline 
(http://www.usqonline.com.au)  was  introduced.  This  initiative  enables  the  delivery  of  multiple  courses  via  the 
Internet  to  students  worldwide.  VSQOnline  presently  offers  29  award  courses  and  has  involved  all  six  Faculties 
within  the  university  and  approximately  one  quarter  of  the  400  academic  staff.  It  involves  a contractual  arrangement 
with  an  independent  commercial  provider,  a company  in  which  USQ  is  a shareholder.  Aiming  at  a mass  global 
market,  it  provides  a complete  delivery  package  - the  administrative  structures  (including  enrolment),  delivery 
platform,  and  mirror  sites  around  Australia,  Asia  and  North  America. 

Change  in  Higher  Education 

The  introduction  of  the  online  initiative  has  caused  major  cultural  and  administrative  change  within  USQ.  The 
institution  has  responded  to  the  current  influences  and  demands  and  has  made  some  substantial  changes  to  the  way  it 
offers  its  programs  and  services.  Many  of  the  institutional  systems  have  had  to  undergo  major  reform  and  the 
pedagogical  framework  of  "traditionar  distance  education  has  been  challenged  by  the  unique  possibilities  of  online 
delivery.  There  has  also  been  quite  significant  organisational  restructuring  to  enable  the  implementation  of  a more 
corporate  approach  to  management  at  the  University.  Teaching  online  has  had  an  enormous  impact  on  academic 
staff  roles  and  workloads,  and  on  the  conceptualisation  of  models  of  online  teaching  and  learning. 

The  literature  supports  the  belief  that  the  most  critical  barriers  to  change  in  educational  processes  are  "personal”  ones. 
Kefford  (1980)  maintain  that  the  most  decisive  factor  in  ensuring  the  success  of  change  in  educational  organisations  is 
the  positive  reaction  and  commitment  of  the  personnel  affected  by  that  change.  According  to  Waugh  and  Punch  (1987), 
there  are  basic  generalisations  common  to  all  educational  change  and  these  can  be  incorporated  into  ideal-type  models  for 
implementing  change.  Waugh  and  Punch  have  extracted  general  variables  that  should  be  included  in  such  models,  identifying 
the  variables  as: 

• overall  feelings  and  attitudes  towards  the  previous  system,  particularly  so  with  regard  to  teachers  who  have  been  in  the 
system  for  a long  period  of  time.  They  may  not  be  receptive  to  change  that  challenges  their  traditional  values  and  those  of 
the  organisation  and  previous  system; 

• the  extent  of  alleviation  of  fears  and  uncertainties  associated  with  the  change.  The  lack  of  knowledge  about  the  proposed 
change  may  hinder  receptivity  to  that  change  and  teachers  should  be  encouraged  to  be  actively  involved  in  the  decision 
making  process; 

• the  practicality  and  importance  of  the  new  education  system;  and 

• the  personal  cost  appraisal  for  the  change.  Teachers  will  tend  to  appraise  the  value  of  the  proposed  change  in  terms  of  the 
amount  of  incentive  and  support  received  to  implement  change;  the  amount  of  energy,  time  and  difficulty  involved  in 
having  to  learn  new  skills;  and  the  effects  on  students,  on  home  life,  and  on  personal  satisfaction. 

Coaldrake  and  Stedman  (1999)  observe,  "universities  more  so  than  most  organisations  are  built  on  a culture  of 
individualism  and  academic  personal  autonomy"  (p.  1).  The  roles  of  staff  and  students  in  the  traditional  tertiary 
education  framework  have  been  established  over  hundreds  of  years.  The  traditional  academic  culture  has  been 
labelled  a "person"  culture  by  Handy  (1993,  in  Laurillard  & Margetson  1997),  with  an  emphasis  on  individualism 
for  both  staff  and  learners.  In  this  culture,  academics  work  autonomously  on  the  preparation  of  face-to-face  teaching 
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materials.  It  is  very  much  an  individual  task,  with  the  control  of  content  an  individual  responsibility.  The  Distance 
Education  Centre  at  USQ  was  established  to  facilitate  design,  production  and  delivery  of  material  for  students 
separated  from  a tertiary  institution  by  distance,  creating  what  Laurillard  & Margetson  (1997,  p.  3)  call  a ’’role 
culture”  or  industrial  model  of  education.  The  introduction  of  distance  education  caused  considerable  tension 
between  the  ’’person”  culture  and  the  emerging  ’’role”  culture.  The  role  culture  of  traditional  distance  delivery  means 
that  course  material  is  delivered  in  print,  is  therefore  available  for  peer  scrutiny  and  usually  prepared  in  a team 
environment.  The  team  approach  is  deemed  essential  because  there  is  no  one  person  who  can  design  and  deliver  the 
teaching/leaming  programs  that  are  epitomised  by  distance  education,  suggesting  that  perhaps 
’’educators... must...  drop  the  arrogance  of  academic  isolation”  (Limerick  et  al.  1998,  p.l  1)  within  this  culture. 


The  Study 

In  order  to  reflect  on  the  initiation  and  conceptualisation  of  the  VSQOnline  project  within  the  context  of 
organisational  change,  the  authors  conducted  an  interim  evaluation  with  a representative  group  of  academic  staff  at 
the  university.  This  was  a preliminary  investigation  and  the  authors  are  not  suggesting  that  the  initiative  has  only 
affected  academic  staff  Through  an  analysis  of  data  collected  from  interviews,  the  authors  were  able  to  reflect  on 
the  staff  perceptions  of  the  online  initiative.  The  interviews  were  conducted  over  a one  week  period  with  staff  from  a 
cross-section  of  Faculties  and  levels.  This  was  to  gain  a snapshot  view  of  the  perceptions  of  the  initiative,  its 
conceptualisation,  initiation  and  its  communication  to  the  university  community.  The  interviewees  were  sent  the 
questions  (see  appendix  1 , Interview  Questions)  prior  to  the  interview  to  allow  time  for  consideration.  The  structure 
of  the  interview  was  followed  closely  to  give  consistency  in  the  data  collection,  although  the  opportunity  for 
extended  exploration  of  the  themes  and  more  free-ranging  discussion  was  available  within  the  interview  context.  As 
a result  of  analysis  of  the  interview  data,  the  paper  identifies  key  challenges  and  experiences  that  have  emerged  and 
offers  proposals  from  which  others  interested  in  institutional  change  and  transformation  might  benefit. 

Findings  and  Recommendations 

Communication,  Consultation  and  Collaboration 

According  to  the  interview  data,  there  was  a concerted  attempt  to  inform  and  consult  with  staff  about  the  online 
initiative  in  a structured  way  through  a variety  of  channels.  Communication  occurred  via  written  means  and  verbal 
presentations  such  as  university-wide  assemblies  (including  the  ’’launch”),  demonstrations,  meetings  within 
Faculties,  focus  groups,  and  through  small  committees  (see  appendix  2 for  a diagram  of  the  VSQOnline 
management  structure).  Throughout  the  interviews,  a persistent  recommendation  was  the  need  for  open  and  ongoing 
communication  involving  all  stakeholders  during  the  process  of  conceptualising  and  communicating  of  such  a major 
initiative.  Interviewees  articulated  that  lack  of  consultation  and  free  exchange  of  information  can  cause  feelings  of 
disempowerment  and  stressed  that  the  implementation  process,  and  the  following  impact  at  grassroots  level  needs  to 
be  made  very  explicit.  One  recommendation  was  that  an  outside  consultant  could  facilitate  focus  group  processes  to 
ensure  a professional,  unbiased  approach  with  a free  and  fair  exchange  of  information.  It  was  also  articulated  that 
academics  have  much  to  offer  in  the  way  of  educational  and  management  expertise,  and  that  they  should  contribute 
to  both  the  conceptualisation  and  implementation  of  such  an  initiative.  A consensus  of  opinion  from  the  interview 
data  indicated  a need  for  shared  ’’vision”  and  consultation  and  collaboration  at  all  stages  of  the  developmental 
process  in  order  to  facilitate  shared  meaning  and  a sense  of  ownership  of  the  decision-making  process  amongst  those 
who  have  to  implement  the  project. 

The  literature  supports  the  findings  of  the  interim  evaluation.  Dolence  and  Norris  (1995)  observe:  ’’Campus 
leadership  must  find  ways  to  stimulate  discussion,  debate,  and  dialogue  on  the  need  for  a transformation  vision. 
Existing  processes  and/or  initiatives  can  be  reshaped  as  agents  of  transformation.  Given  the  ethos  of  Academe,  the 
development  of  shared  values  must  be  a process  of  co-creation,  consultation  and  testing  of  ideas”  fpp.  87-8).  This 
supports  one  of  the  main  variables  discussed  by  Waugh  and  Punch  (1987)  - that  a lack  of  knowledge  about  a 
proposed  change  may  hinder  receptivity  to  that  change.  Yukl  (1999)  emphasises  the  importance  of  empowerment 
and  discusses  how  ’’resistance  can  be  a source  of  energy  that  enables  people  to  collectively  make  better  decisions 
about  what  type  of  change  is  needed”  (p.40.  These  points  were  strongly  supported  by  all  interviewees  in  the  study. 


Roles,  Responsibilities  and  Workload  Implications 


As  previously  mentioned,  there  have  been  some  significant  changes  at  USQ  in  the  way  the  members  of  the  academic 
community  perform  the  tasks  required  to  develop  and  implement  flexible  learning  programs  and  services.  A key 
pedagogical  feature  of  the  online  teaching/leaming  environment  is  that  the  technology  facilitates  interaction  through 
electronic  means  (discussion  groups,  email,  virtual  chats).  This  capability  impacts  on  the  design  and  delivery  of 
course  material  and  has  caused  a significant  shift  from  the  independent  learner  mode  of  traditional  distance 
education  courses  to  interaction  between  participants  in  online  courses.  Distance  education  also  has  meant  staff  have 
a larger,  more  varied  cohort  of  students  to  service  and  track.  Furthermore,  teaching  online  has  had  a significant 
impact  on  student  learning  and  staff  workloads,  as  technology  allows  opportunity  for  increased  online 
communication,  changes  in  traditional  teaching  roles,  and  changes  in  the  way  information  is  published  and 
resourced.  A more  commercial  emphasis,  changes  in  course  design  and  delivery  and  a shift  to  a greater  client  focus 
(giving  learners  what  they  want,  when  and  where  they  want  it)  has  meant  that  courses  are  now  being  offered 
throughout  the  year,  and  in  multiple  modes.  Students  have  been  provided  with  opportunities  to  extend  or  shorten  the 
time  taken  to  complete  particular  units  of  work.  University  administrative  systems  have  been  put  in  place  to 
accommodate  this  added  flexibility  but  staff  workload  issues  remain  a major  concern.  One  interviewee  observed  that 
"for  a long  time  the  university  has  operated  on  the  goodwill  of  its  staff,  but  when  a job  grows  from  that  which  is 
manageable  to  just  another  ten  percent,  things  get  a bit  fractious  about  the  edges".  Several  of  the  interviewees 
observed  that  USQ  staff  have  already  undergone  role  changes  as  a result  of  distance  education  initiatives  so  perhaps 
it  is  considered  by  management  that  staff  are  ready  to  undergo  a further  evolutionary  move  building  on  this  history 
of  organisational  change. 

Interview  responses  provided  suggestions  for  ways  to  assist  the  workload  situation.  These  include  a reduction  in 
course  offerings  with  a focus  on  particular  disciplines  or  courses  that  have  already  proved  popular  to  learners. 
Reward/award  systems  might  be  established  as  an  incentive  for  staff  having  to  do  the  additional  work  involved 
which  is  often  on  top  of  existing  workloads.  However,  the  conflict  still  exists  in  institutions  of  higher  education  as  to 
what  is  considered  worthy  of  reward  - research,  teaching  or  a combination  of  both.  The  cost-saving  benefits  of  major 
online  initiatives  are  also  still  to  be  proven.  An  investigation  conducted  by  the  University  of  Illinois  (1999)  states: 
“Because  high  quality  online  teaching  is  time  and  labour  intensive,  it  is  not  likely  to  be  the  income  source 
envisioned  by  some  administrators”  (p.  1). 

Loosely  Coupled  Networks/  Synergies  and  Alliances 

One  of  the  key  themes  emerging  in  the  literature  is  the  need  for  an  organisation  to  enter  into  loosely  coupled 
networks  or  strategic  alliances  in  order  to  operate  within  the  emerging  global  environment.  Alliances  with  other 
knowledge  producers,  such  as  a partnership  with  an  independent  commercial  provider  in  the  case  of  USQ  has  been  a 
strategy  aimed  at  capitalising  on  the  strengths  of  both  partners  and  increasing  student  access  to  higher  education. 
Kanter  (1989)  argues  that  "a  major  component  of  post-entrepreneurial  strategy  involves  developing  close  working 
relationships  with  other  organizations,  extending  the  company’s  reach  without  increasing  its  size.  Strategic  alliances 
and  partnerships  are  a potent  way  to  do  more  with  less"  (p.  347).  One  interviewee  observed  that  the  online  initiative 
at  USQ  "builds  on  existing  expertise,  positions  USQ  as  an  initiator  and  leader  in  full-scale,  online  deliveiy,  draws  on 
the  capabilities  of  a commercial  partner  and  also  enables  opportunities  for  joint  partnerships  with  other  academic 
institutions".  The  alliance  enables  USQ  to  offer  its  students  global  access  by  using  the  technology  network. 
According  to  Higgins  and  Vincze  (1989,  in  Limerick  et  al.,  1998),  "corporations  with  ambitions  must  turn  to  a new 
strategy  of  agreements,  alliances,  and  mergers  with  other  companies  if  they  hope  to  survive"  (p.  78).  However,  as 
raised  in  the  interview  data,  the  importance  of  a clear  allocation  of  roles  and  responsibilities  and  a clear 
understanding  of  the  capabilities  of  each  partner  is  essential  and  there  is  also  a need  to  choose  strong  partners,  as 
synergies  do  not  come  out  of  combined  weaknesses. 

Technological  Difficulties 

An  important  consideration  in  the  use  of  electronic  deliveiy  and  communication  systems  is  that  of  technical  support. 
The  interview  data  revealed  that  people  who  are  unfamiliar  with  using  technology  spend  a large  amount  of  time 
coming  to  grips  with  it.  Staff  skills  in  using  the  Internet  vaiy  immensely,  as  do  their  learning  styles.  A Staff  and 
Student  Support  (SSS)  Focus  Group  has  emerged  out  of  the  new  committee  structure.  This  group  has  been  charged 
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with  the  responsibility  of  developing  interactive,  online  education  and  training  materials  for  staff  The  group  has 
also  established  a Staff  Development  Gateway  web  site  which  contains  links  to  other  courses  available,  Help  Desk 
facilities,  multiple  resources,  exemplary  materials,  teaching  and  learning  advice  and  peer  mentoring  support.  All 
interviewees  acknowledged  that  these  developments  have  been  well  received  by  staff  However,  it  was  evident  from 
the  responses  that  there  is  still  a need  for  face-to-face  education  and  training  support  which  is  also  managed  by  the 
SSS  Focus  Group.  It  was  also  acknowledged  that  this  preparation  may  not  be  enough  to  alleviate  all  the  concerns  of 
academic  staff  and  will  have  to  be  closely  monitored  and  addressed.  The  advice  through  the  interviews  has  been  to 
slow  the  developmental  process  down  in  order  to  reflect  on  events  and  evaluate  the  initiative  and  make  sure  the 
operational  components  of  the  system  are  working  well.  It  was  observed  that  there  is  always  more  to  be  done  but 
there  is  still  a need  for  time  out  and  reflection. 


Conclusion 

The  use  of  an  electronic  environment  such  as  \}SQPnline  to  provide  learning  experiences  opens  up  new  challenges 
in  terms  of  teaching  and  learning.  The  interim  evaluation  conducted  by  the  authors  at  USQ  makes  the  following 
recommendations:  open  communication,  consultation  and  collaboration  should  form  the  basis  of  all  stages  of  such  a 
major  institutional  initiative;  roles  and  responsibilities  of  all  stakeholders  should  be  clearly  defined;  synergies  and 
alliances  with  strong  partners  are  essential;  and  there  should  be  sound  technological  support  at  all  levels.  The  key  to 
organisational  change  and  sustainability  is  shared  vision,  shared  values  and  shared  decision  making.  The  trends  in 
and  impacts  of  the  use  of  communication  and  information  technologies  in  the  higher  education  sector  mean  that 
change  is  an  ongoing  factor  of  tertiary  education.  This  is  not  necessarily  something  to  be  feared  as  it  also  promises 
to  offer  some  exciting  challenges.  As  Eckel  et  al.  (1999)  observe,  change  is  "an  ongoing,  organic  process"  and  "there 
is  no  point  in  time  at  which  everyone  can  declare  a victory  and  go  back  to  normal  life"  (p.  1).  The  challenge  facing 
all  education  providers  in  the  21st  century  is  to  meet  the  needs  of  lifelong  learners  in  the  Information  Age  and 
distance  education  providers  are  uniquely  positioned  to  service  the  needs  of  these  learners. 
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Appendix  1:  Interview  Questions 

How  would  you  describe  the  XJ^Qpnline  initiative  to  other  academics  unfamiliar  with  the  project? 

What  do  you  believe  was/is  the  leaders’  vision  for  USQ  when  initiating  this  project? 

How  was  the  vision  communicated  to  USQ  academic  staff? 

Reactions/personal  reflections  to/on  the  concept? 

Reactions/personal  reflections  to/on  the  methods  of  communicating  the  concept. 

Commendations  and  recommendations. 


Appendix  2:  Management  Committee  Structure 


I Manage ment  Structu r e U SQpnlin e 


Faculties 


BEST  COPY  AVAILABLE 


949 


Using  Directed  World  Wide  Web  Navigation  Guidance:  An  empirical 

investigation 


David  Ribeiro  Lamas 


Jennifer  Jerrams-Smith 

(University  of  Portsmouth,  UK,  jenny.jerrams-smith@port.ac.uk) 
David  Heathcote 

(Bournemouth  University,  UK,  dheathco@boumemouth.ac.uk) 


Feliz  Ribeiro  Gouveia 

Universidade  Fernando  Pessoa,  Multimedia  Resource  Center 
Pra^a  9 de  Abril,  349,  Porto,  Portugal 
+351  22  5071306,  +351  22  5508269  (fax) 

{dlamas,  fribeiro}@ufp.pt 


Abstract:  The  Computer  Aided  Information  Navigation  (CAIN)  project  provides  a new  method 
of  adaptive  navigation  support  that  aims  to  increase  the  World  Wide  Web's  value  as  a pedagogical 
tool.  To  examine  whether  the  system’s  approach  improves  comprehension  under  specific 
conditions,  an  experiment  compared  CAIN  with  conventional  web  tools  on  three  measures  of 
performance:  comprehension  scores,  task  completion  times  and  user  satisfaction.  Results  revealed 
that  comprehension  scores  were  30%  greater  with  CAIN  than  with  the  conventional  tools  and 
indicated  that  task  completion  times  can  also  be  reduced  without  compromising  user  satisfaction. 


Introduction 

The  Computer  Aided  Information  Navigation  project  (CAIN)  aims  to  provide  adaptive  navigation  support  as  a 
way  of  increasing  the  Web's  value  as  a pedagogical  tool.  CAIN's  approach  is  a reactive  one  as  it  seeks  to  provide  a 
way  of  dealing  with  the  vast  amount  of  useful  information  available  on  the  Web  as  it  currently  exists  rather  than 
trying  to  improve  or  alter  the  Web's  infrastructure. 

In  order  to  help  users  reach  their  goals,  CAIN  provides  direct  guidance  navigation  support  as  a form  of  non- 
obtrusive  weak  hypertext  linearization  enabling  the  user  to  follow  a context  specific  ranked  sequence  of  selected 
Web  pages  without  ever  needing  to  perform  any  search  or  follow  any  link  if  they  do  not  wish  to.  This  approach 
does  not  seek  to  prevent  goal-oriented  exploration  but  rather  to  provide  a sound  thread  or  guideline  to  help  users  to 
retain  their  focus. 

The  system's  representation  of  the  Web,  crucial  to  the  success  of  its  adaptive  features,  builds  on  the  Dublin  Core 
(Weibel  & Lagoze  1997)  and  on  the  Resource  Description  Framework  (Lassila  & Swick  1998). 

The  users'  representation  combines  three  user  modeling  techniques  resulting  in  a hybrid  solution  that  uses 
stereotypes,  overlays  and  attribute-value  pairs  (Benyon  & Murray  1993;  Brusilovsky  1996). 

Navigation  support  is  accomplished  by  a basic  route-finding  heuristic  which  selects  context  specific  Web  model 
items,  sorts  them  using  associated  qualitative  ratings,  and  presents  them  to  the  user,  one  at  the  time,  based  on  the 
attributes  of  the  user's  model  (Lamas  1998). 

This  paper  presents  and  discusses  the  results  of  an  experiment  which  examined  whether  the  directed  Web 
navigation  guidance  provided  by  CAIN  can  improve  the  performance  of  Web  users  under  appropriate  conditions. 
Section  2 describes  the  methodology  used  and  section  3 presents  and  discusses  the  results.  Section  4 discusses 
related  empirical  work  and  section  5 evaluates  the  current  study  and  suggests  directions  for  future  research. 

Method 

It  was  hypothesised  that  contextualized  direct  guidance  of  the  information  available  on  the  World  Wide  Web  may 
improve  comprehension  under  the  specific  conditions  that: 

• Users  are  either  inexperienced  in  Web  use  or  have  little  expertise  in  the  target  subject  topic,  or  both. 
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• Users  use  the  system  pursuing  learning  or  initial  research  goals. 

For  this  purpose  a standard  task  was  devised  in  which  volunteers  were  required  to  use  the  WWW  to  learn  as  much  as 
they  could  about  two  given  subject  topics  — cryptography  and  poetry  — using  either  CAIN  or  alternative  tools  that 
the  users  chose  to  employ. 

The  experiment  compared  the  use  of  CAIN  with  conventional  tools  on  three  dependent  measures  of  performance: 

• Comprehension  as  measured  by  a multiple-choice  comprehension  questionnaire; 

• Time  taken  to  complete  the  tasks;  and 

• System  usage  appreciation  assessed  by  a like/dislike  rating  completed  at  the  end  of  each  task. 

Performance  was  examined  as  function  of  the  following  independent  variables: 

• Tool  used,  either  CAIN  or  Free\ 

• Web  expertise,  the  Web  experience  of  the  users,  either  Low  or  High\  and 

• Subject  expertise,  the  users*  expertise  in  the  target  subject  areas,  either  Low  or  High, 

In  order  to  implement  the  experiment,  volunteers  were  recruited  from  the  university's  population*,  specialist  help 
was  required  to  build  the  domain  model  and  to  provide  the  subject  abstracts  and  questionnaires  and  a specially 
designed  environment  was  prepared  to  implement  the  experiment. 

Design  and  procedure 

The  devised  task  consisted  of  carrying  out  two  learning  assignments  performed  in  sequence  and  assessed  at  each 
one’s  completion. 

The  task  could  be  accomplished  using  either  CAIN  or  any  other  Web  navigation  tool. 

Forty  volunteers  were  used  as  test  subjects,  20  of  whom  performed  the  test  using  CAIN  while  the  remaining  20 
performed  the  task  using  the  tools  of  their  choice.  To  control  for  order  effects,  the  CAIN  and  non  CAIN  groups 
were  each  divided  into  two  sub-groups  of  10  subjects  so  that  the  learning  assignments  were  counterbalanced 
between  groups. 

Each  sub-group  was  assigned  to  a task  permutation,  either  CAIN  AB,,  CAIN  BA,  FreeAB  or  Free  BA. 

• Before  the  test  began  — Self-assessment  of  Web  expertise  along  a scale  of  1 to  7 was  provided  by  each 
participant. 

• Before  each  of  the  learning  tasks  began  — Self-assessment  of  expertise  in  the  subject  area  of  1 to  7 was 
provided  by  each  participant. 

• Following  completion  of  each  of  the  learning  tasks  — A ten  question  multiple  choice  comprehension 
questionnaire  was  completed  by  each  participant  together  with  a 7 point  rating  of  system  satisfaction. 

Participants  were  required  to  complete  the  tasks  within  time  boundaries  and  each  participant’s  actual  completion 
time  was  recorded  with  a resolution  of  1 minuted 

Volunteers  undertook  the  tasks  in  a specially  prepared  and  isolated  environment. 

Materials  and  apparatus 

In  order  to  provide  valid  learning  assignments  and  comprehension  questions,  two  specialists^  in  the  domains 
provided  all  relevant  data  from  CAIN's  domain  model  to  the  assignment  descriptions  and  the  comprehension 
assessment  multiple  choice  questionnaires. 

The  questionnaires 

Task  scripts,  learning  assignment  descriptions  and  the  various  questions  were  specially  prepared  to  cater  for  each  of 
the  four  task  variations. 

Everything  was  written  in  Portuguese  as  the  experiment  was  conducted  in  Portugal. 

Apart  from  ensuring  the  specific  multiple  choice  questionnaires,  the  subject  specialists  also  provided  marking  grids 
so  that  data  could  be  gathered  as  soon  as  the  tests  were  accomplished. 

The  test  site 

The  test  site  was  designed  so  that  volunteers  could  enjoy  suitable  silence  and  isolation. 

A pilot  experiment  was  run  before  the  main  with  four  volunteers  to  establish  the  appropriate  experimental 
parameters  such  as  the  time  boundaries,  the  learning  assignment  descriptions,  the  number  of  MCQ  questions. 

Participants 

An  opportunity  sample  of  40  volunteers  was  used  in  the  experiment.  All  volunteers  were  members  of  the  university 
and  were  the  first  40  who  responded  to  the  call  for  volunteers  broadcast  throughout  the  university. 

The  sample  consisted  of  24  undergraduate  students,  12  postgraduate  students  and  4 staff  members.  There  were  28 
males  and  12  females  in  the  sample. 


1 Universidade  Fernando  Pessoa,  Porto,  Portugal 

2 these  are  the  Time  A and  Time  B variables 

3 professors  Estela  Pinto  Ribeiro  Lamas  and  Feliz  Ribeiro  Gouveia,  both  from  Universidade  Fernando  Pessoa 
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All  had  English  reading  skills.  No  financial  inducements  were  offered  and  written  consent  was  obtained  from  each 
of  the  volunteers. 

Results  and  discussion 

The  participants  were  grouped  to  produce  three  independent  variables  as  described  below. 

• Given  that  the  design  of  the  experiment  was  counterbalanced,  the  four  permutations,  CAIN  AB^  CAIN  CA,  Free 
AB  and  Free  BA,  results  in  the  Tool  grouping  variable  with  two  levels,  CAIN  and  Free. 

• In  order  to  class  volunteers  as  either  High  or  Low  in  Web  expertise,  a mean  was  taken  of  overall  self-assessed 
Web  expertise.  Volunteers  who  scored  above  the  mean  were  assigned  to  the  high  Web  expertise  group  and 
volunteers  who  scored  below  the  mean  were  assigned  to  the  low  Web  expertise  group  resulting  in  the  Web 
expertise  grouping  variable. 

• The  same  was  done  with  subject  (i.e.  domain)  expertise  resulting  in  the  Subject  expertise  (High/Low)  grouping 
variable. 

Finally,  comprehension  assessment  scores  were  collapsed  across  subject  A and  subject  B multiple  choice 
questionnaires  scores  resulting  in  a mean  comprehension  score  dependent  measure. 

The  same  technique  was  applied  to  the  time  taken  to  accomplish  the  learning  assignments  and  to  the  system 
appreciation/satisfaction  ratings  producing  two  further  dependent  measures:  mean  completion  time  and  mean  user 
satisfaction. 

The  new  variables  meanings  and  usage  conventions  are  explained  in  table  1. 


Variable 

Meaning 

Tool 

The  too!  used  by  the  volunteer  to  perform  the  task 
classified  as  either  CAIN  or  Free 

Web  expertise 

The  volunteer's  self  assessed  Web  expertise  classified 
as  either  Low  or  High 

Subject  expertise 

The  volunteer’s  self  assessed  subject  (domain)  expertise 
classified  as  either  Low  or  High 

Mean  score 

Average  comprehension  score  achieved  by  a volunteer 
after  completing  a learning  task  ranging  from  0 — 
minimum  — to  1 0 — maximum 

Mean  time 

Average  time  taken  by  a volunteer  to  complete  a learning 
task  measured  in  minutes 

Mean  system 

Average  used  system  appreciation  by  volunteer  rated 
from  1 — minimum  — to  7 — maximum 

Table  J:  Definition  of  the  independent  and  dependent  variables  used  in  the  experiment 

Comprehension 

The  results  of  the  self  assessments  conducted  before  each  learning  task  began  showed  that  subject  expertise  was 
equivalent  for  the  CAIN  and  Free  groups  (/(38)  = 0.276,  p = 0.784),  a difference  of  only  2.5%  (approximately). 
Therefore  a meaningful  comparison  between  the  comprehension  performance  of  the  CAIN  and  Free  groups 
following  a Web  learning  session  was  possible. 

Following  Web  learning,  a Mest  comparing  comprehension  scores  showed  that  comprehension  was  significantly 
greater  in  the  C4/V  group  than  in  the  Free  group  (/(38)  = 2.9,  p ~ 0.006). 

The  comprehension  score  group  statistics  presented  in  table  2 together  with  the  mean  subject  expertise  ratings 
presented  in  table  3,  indicate  that  CAIN  produces  an  improvement  in  comprehension  scores  of  nearly  30% 
compared  with  the  other  methods  used  by  the  volunteers.  


Tool 

Volunteers 

Mean 

Standard 

deviation 

Standard 

error 

mean 

Free 

20 

3.8750 

1.2126 

0.2711 

CAIN 

20 

5.0500 

1.3367 

0.2989 

Table  2:  Comprehension  score  group  statistics 


Mean 

Tool 

Free 

CAIN 

Subject  expertise 

3.03 

3.10 

Average  score 

3.88 

5.05 

Table  3:  Mean  subject  expertise  and  comprehension  score  contrast 
In  order  to  explore  these  results  further,  an  analysis  of  variance  of  comprehension  scores,  with  Subject  expertise. 
Tool  and  Web  expertise  as  the  three  independent  factors,  was  conducted. 

The  analysis  showed  a significant  main  effect  of  Tool  such  that  comprehension  scores  were  higher  after  completing 
a learning  assignment  with  C4/Vthan  after  completing  a learning  assignment  with  Free  (F(l  ,32)  = 4.85,/?  = 0.035). 
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This  repeats  the  result  from  the  /-test  and  in  this  case,  no  other  variable  or  interaction  between  variables  has  enough 
statistical  significance  (all  p > 0.05)  to  be  regarded  as  influential. 

Separate  analyses  of  variance  were  conducted  on  comprehension  scores  for  subjects  A and  B which  revealed  that  the 
main  effect  of  Tool  was  highly  significant  on  comprehension  scores  for  subject  B {F  (1,32)  = 8.554,  p = 0.006)  but 
not  for  subject  A (F  (1,32)  = 0.709,  p > 0.05). 

Subject  A was  cryptography  and  subject  B topic  was  poetry.  These  results  may  reflect  the  fact  that  cryptography 
Web  sites  have  better  structure  or  are  easier  to  find  and  use  than  poetry  Web  sites  and  so  benefit  little  from  the  use 
of  CAIN. 

The  separate  analysis  of  variance  also  revealed  a significant  main  effect  of  Web  expertise  for  subject  A {F  (1,32)  = 
5.979, p = 0.020)  and  subject  B (F(l,32)  = 5.353,  p = 0.027)  which  did  not  appear  in  analysis  of  variance  collapsed 
across  domain. 

This  situation  results  from  a crossover  interaction  between  Web  expertise  and  domain  (subjects  (see  table  4). 


Web 

expertise 

Overall 

mean 

Subject  A 
mean 

Subject  B 
mean 

Low 

4.38 

4.29 

4.47 

High 

4.52 

5.83 

3.22 

Table  4:  Comprehension  scores  means  by  Web  expertise  level 

Although  unexpected,  these  results  suggest  that  the  subject  topics  used  — cryptography  and  poetry  — are  at 
opposite  ends  of  a qualitative  range  for  Web  sites  and  that  by  using  them  both,  this  experiment  neutralised  their 
differences  and  provided  a fair  testing  environment. 

Finally,  table  5 provides  a straightforward  comparison  between  the  mean  comprehension  scores  under  every 
condition  combination,  


Web  expertise 

Tool 

Subject  expertise 

Mean 

Low 

High 

Low 

Free 

3.80 

3.50 

3.65 

CAIN 

4.25 

5.33 

4.79 

High 

Free 

2.66 

4.25 

3.46 

CAIN 

5.87 

4.75 

5.31 

Mean 

4.15 

4.48 

Table  5:  Web  expertise  A" Tool  X Subject  expertise  comprehension  scores. 

Time  taken 

Next,  an  analysis  of  variance  of  completion  time  performance  was  conducted  with  the  same  Subject  expertise.  Tool 
and  Web  expertise  independent  variables. 

Although  overall  learning  assignment  completion  times  are  quicker  with  CAIN  than  with  Free,  this  difference  is  not 
statistically  significant  (F(l,32)  = 2.05, p > 0,05). 

Nevertheless,  several  interesting  remarks  can  be  made: 

• There  is  a significant  main  effect  of  Web  expertise  such  that  highly  experienced  Web  users  performed  faster 
than  the  low  experienced  Web  users  (F(l,32)=9.02,  p=0.05).  As  all  of  the  experiment  was  conducted  over  the 
Web,  it  is  not  surprising  that  highly  experienced  Web  users  performed  faster. 

• There  is  an  interesting  3 Way  interaction  between  Subject  expertise.  Tool  and  Web  expertise  (F(l,32)  = 4,61,  p 
= 0.039). 

Table  6 illustrate  this  last  result  highlighting  the  fact  that  CAIN  produces  faster  performance  than  Free  under  all 
combinations  of  levels  of  the  independent  variables  except  when  the  volunteer  has  High  Web  expertise  and  High 
subject  expertise,  i.e.  it  suggests  that: 

• Time  taken  may  be  slower  with  CAIN  when  the  user  is  a highly  experienced  Web  user  and  a an  expert  in  the 
learning  assignment  field,  but; 

• CAIN  is  faster  when  users  are  inexperienced  Web  users  and  inexperienced  in  the  study  subject  or  inexperienced 
in  the  Web  and  experienced  in  the  subject  or  experienced  in  the  Web  and  inexperienced  in  the  subject. 


Web  expertise 

Tool 

Subject  expertise 

Mean 

Low 

High 

Low 

Free 

91.70 

86.50 

89.10 

CAIN 

86.17 

68.50 

77.34 

High 

Free 

74.67 

61.50 

68.09 

CAIN 

50.50 

75.44 

62.97 

Mean 

75.76 

72.99 

Table  6:  Web  expertise  A^Tool  X Subject  expertise  performance  times  contrast 
Note  that  Web  expertise  alone  does  not  slow  performance  when  using  CAIN.  In  fact,  highly  experienced  Web  users 
performances  are  slightly  faster  under  CAIN  than  Free. 
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It  is  the  particular  combination  of  High  Web  expertise  and  High  subject  expertise  that  slows  performance  using 
CAIN  compared  to  using  other  tools  on  the  World  Wide  Web, 

Further  analyses  of  variance,  conducted  separately  on  time  performances  for  subjects  A and  B,  revealed  that  the 
main  effect  of  Tool  was  specially  significant  for  subject  B {F  (1,32)  = 5.783,  p = 0.022)  but  not  significant  for 
subject  .4  (F(l,32)  = 0.017,  p > 0.05). 

These  results  provide  further  evidence  that  cryptography  Web  sites  may  differ  in  quality  or  quantity  from  poetry 
Web  sites  and  that  CAIN  may  be  more  effective  when  addressing  topics  whose  Web  sites  have  characteristics 
similar  to  those  of  poetry  Web  sites  (i.e,  less  well  organised). 

The  same  analysis  of  variance  confirmed  the  significance  of  the  main  effect  of  Web  expertise  on  both  subject  A 
(F(\,32)  = 4.819,  p = 0.036)  and  subject  B (F(l  ,32)  = 5.362,  p = 0.027)  performance  times. 

This  only  indicates  that  although  CAIN  improves  time  performance  in  general,  Web  expertise  still  plays  an 
important  role  as  novice  Web  users  spent  more  time  than  experienced  Web  users.  This  does  not  comes  as  a surprise 
since  CAIN  is  itself  Web  based  and  it  does  not  replace  existing  navigation  tools,  it  merely  provides  non-obtrusive 
navigation  support. 

System  usage  appredation/user  satisfaction 

The  /-test  for  system  usage  appreciation  does  not  show  any  significant  difference  between  the  CAIN  and  the  Free 
groups  (/(38)  = 0.698, /?  = 0.489). 

In  fact,  table  7 shows  that  CAlNxdXQd  5.7  out  of  7 while  Free  rated  5.475  out  of  7.  This  is  only  a 4%  increase. 


Average 

system 

Volunteers 

Mean 

Standard 

deviation 

Standard 

error 

mean 

Free 

20 

5.475 

1.0696 

0.2392 

CAIN 

20 

5.700 

0.9652 

0.2158 

Table  7:  System  usage  appreciation  group  statistics 

Table  8 provides  a straightforward  comparison  between  the  mean  like/dislike  scores  under  every  condition 
combination. 


Web  expertise 

Tool 

Subject  expertise 

Mean 

Low 

High 

Low 

Free 

6.00 

5.50 

5.75 

CAIN 

5.75 

6.00 

5.88 

High 

Free 

4.75 

5.75 

5.25 

CAIN 

4.86 

5.92 

5.39 

Mean 

5.34 

5.79 

Table  8:  Web  expertise  A" Tool  X Subject  expertise  system  usage  appreciation  contrast 
As  shown,  satisfaction  was  higher  for  CAIN  under  almost  every  condition  combination  except  for  users  ranking  low 
in  both  Web  and  subject  expertise.  This  was  probably  due  to  the  extra  effort  required  to  cope  with  new  systems,  the 
Web  and  CAIN,  and  a new  subject  at  the  same  time. 

Finally,  in  this  case  there  was  no  significant  difference  between  the  independent  analysis  of  system  usage 
appreciation  for  subjects  A and  B. 

Related  empirical  evidence 

Although  to  date  empirical  studies  on  the  value  of  adaptivity  in  hypertext  based  learning  environments  are  limited, 
they  are  critically  important  to  validate  the  different  approaches  in  this  research  area. 

Although  most  of  the  existing  empirical  evidence  reviewed  by  Eklund  and  Brusilovsky  (1998)  fails  to  show  if 
adaptive  hypertext  navigation  contributes  at  all  to  improve  the  student's  or  researcher's  performance,  it  hints  that 
adaptive  navigation  could  be  valuable  under  appropriate  conditions. 

Nevertheless,  in  experiments  measuring  the  number  of  hypertext  nodes  used  to  accomplish  some  predefined  goal, 
existing  empirical  evidence  favours  adaptii^e  curriculum  sequencing^  versus  adaptive  link  annotation  over  a closed 
information  corpus.  Although  not  directly  related  to  CAIN,  these  results  do  not  contradict  this  project's  approach. 
Further,  on  the  InterBook^  experiment  reported  in  the  same  review,  apart  from  other  results,  it  is  also  stressed  that 
the  direct  guidance  mechanism  was  chosen  by  test  subjects  in  over  90%  of  the  transactions.  Again,  this  system 
relies  on  a closed  information  corpus  but  nonetheless,  the  experiment's  results  provide  added  support  to  CAIN's 
approach. 

This  is  however  an  under-investigated  area  and  there  is  clearly  a need  for  continued  studies  on  the  value  of 
adaptivity  in  hypertext  based  learning  environments. 


4 a direct  guidance  approach 

5 an  adaptive  hypertext  system  supporting  adaptive  link  annotation  and  direct  guidance  in  the  form  of  a continue  link 
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Final  Remarks 

The  experiment  herein  examined  the  nature  of  the  interaction  between  the  users’  Web  expertise.  Subject  expertise 
and  the  used  tool  and  its  results  show  that  CAIN,  as  specified  and  implemented,  does  indeed  improve  user's 
comprehension  by  30%  when  supporting  learning  or  initial  research  tasks  performed  over  the  World  Wide  Web. 

This  is  probably  due  to  the  way  CAIN  guides  the  user  through  a sequence  of  previously  selected  and  relevant  Web 
resources. 

Though  not  in  a very  significant  way,  the  experiment's  results  also  suggest  that  the  time  taken  to  accomplish  such 
tasks  can  also  be  reduced  when  using  CAIN  except  in  circumstances  in  which  it  is  redundant,  i,e.  when  the  user  is 
already  an  expert  using  the  World  Wide  Web  and  an  expert  in  the  subject  domain. 

As  far  as  enjoying  the  system,  the  results  do  not  show  any  significant  difference  between  using  CAIN  or  any  other 
approach  to  navigate  the  World  Wide  Web.  In  fact,  the  results  show  that  users  in  general  enjoy  using  the  Web  with 
whatever  tools  they  use. 

Although  encouraging,  this  experiment’s  results  are  not  sufficient  to  understand  all  the  implications  of  a system  like 
CAIN  and  further  studies  need  to  be  undertaken.  In  fact,  the  current  investigation  only  measures  the  adaptive 
guidance  support  leaving  untouched  issues  related  to  all  the  other  relevant  areas  brought  together  in  this  work  such 
as; 

• The  adequacy  of  the  metadata  records  and  associated  rating  strategy; 

• The  soundness  of  the  implicit  co-operation  model; 

• The  reliability  of  the  relevance  feedback  mechanism;  and 

• The  validity  of  adaptivity  itself 

Specifically,  it  fails  to  understand  other  success  factors  such  as: 

• How  relevant  is  the  order  in  which  Web  resources  are  presented  to  the  user? 

• How  do  users  cope  with  the  domain  building  process? 

• Are  the  rating  criteria  adequate  and  easy  to  apply? 

• Is  such  Web  representation  successful  for  all  its  users? 

• How  does  the  system  cope  with  long  term  users? 

• How  does  physical  distance  affect  the  system's  collaborative  nature? 

This  is  an  under-researched  area  and  there  is  clearly  a need  for  continued  studies  on  the  value  of  adaptivity  in 
hypertext  based  learning  environments. 

In  conclusion,  the  Computer  Aided  Information  Navigation  system  appears  to  be  moving  in  the  right  direction 
and  research  will  continue  in  order  to  achieve  this  collaborative  environment's  long  term  goals:  To  guide  the  user 
while  learning,  understanding,  memorising  and  forgetting  material  in  order  to  increase  motivation,  concentration 
and  comprehension,  reducing  the  environment's  noise  and  improving  the  knowledge  acquisition  process. 
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Abstract:  Many  industries,  including  aerospace,  automotive,  agricultural  and  marine,  use  Fluid 
Power  systems  for  power  transmission.  The  design  of  these  systems  is  a skilful  and  complex 
task  requiring  significant  practical  experience  and  heuristic  knowledge  gained  over  many  years. 
With  few  formal  Fluid  Power  design  courses  being  taught  in  further  and  higher  education,  both 
academia  and  industry  are  seeking  new  methods  of  delivering  this  specialist  knowledge.  The 
Internet  opens  up  low-cost,  low-risk  training  opportunities  for  Small-to-Medium  Enterprises 
that  may  not  be  able  to  afford  the  cost  of  residential  courses,  and  also  for  employees  seeking 
career  development  within  the  context  of  lifelong  learning.  This  paper  describes  the  early 
stages  of  a project  to  develop  an  open,  domain-independent,  Java-based  framework  for  the 
delivery  of  multimedia  training.  This  framework  forms  part  of  a Performance  Support  System 
for  trainees  and  professionals  involved  in  Fluid  Power  systems  design.  Also  described  is  a 
formative  evaluation  of  the  framework. 


Introduction 

The  need  for  lifelong  learning  is  driven  by  the  swiftly  changing  nature  of  global  commerce  which  requires 
unprecedented  levels  of  flexibility  from  a highly  competitive  employment  market.  The  term  itself  refers  to  the 
continuous  renewing,  upgrading  and  updating  of  existing  personal  skills  and  expertise  required  - in  addition  to 
the  acquisition  of  new  capabilities  - if  individuals  are  to  remain  in  gainful  employment  [Davies  1998].  Holt  and 
Radcliffe  [1991]  state  that  the  quality  of  engineering  design  is  largely  determined  by  the  quality  of  the  learning 
processes  that  are  central  to  the  design  experience.  From  this,  it  is  apparent  that,  in  order  to  remain  competitive, 
enterprises  within  engineering  must  become  pro-active  partners  with  education  providers  in  the  lifelong  training 
of  their  employees  [Berkeley  1991].  Despite  this,  many  Small-to-Medium  Enterprises  (SMEs)  often  find 
themselves  unable  to  afford  traditional  training  courses  either  in  terms  of  the  financial  outlay  or  the  time  lost 
through  personnel  being  absent  for  extended  periods. 

The  term  ’’Fluid  Power"  is  used  to  denote  systems  where  mechanical  power  is  transmitted  from  one  point  to 
another  by  means  of  a fluid  such  as  hydraulic  oil.  Examples  are  to  be  found  on  aircraft,  ships,  industrial  plant 
and  mobile  equipment.  The  teaching  of  Fluid  Power  Systems  Design  poses  particular  problems,  principally  due 
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to  the  complex  nature  of  the  task  which  often  requires  extensive  heuristic  knowledge  and  many  years  of  practical 
experience  for  a professional  to  become  proficient.  In  the  context  of  lifelong  learning,  an  overhead  such  as  this 
may  become  prohibitive  for  many  individuals.  It  is  argued  that  by  using  hypermedia  to  facilitate  the  learning 
process  within  the  context  of  an  overall  Web-based  Performance  Support  System  for  Fluid  Power  Systems 
Design,  the  cost  of  Fluid  Power  training  to  industry  in  general  and  SMEs  in  particular  can  be  reduced  to 
acceptable  levels,  and  the  problem  of  learning  overhead  for  individuals  can  also  be  overcome.  It  is  also  argued 
that  the  developed  methods  and  techniques  would  bring  potential  benefits  to  other  domains. 

One  drawback  with  delivering  training  material  over  the  WWW  is  the  very  loose  integration  of  the  Web  itself  In 
order  to  co-ordinate  lessons  and  gather  feedback  from  trainees  a somewhat  tighter  integration  is  required  which 
is  nevertheless  flexible  enough  to  accommodate  changing  content,  changing  users  and  changing  styles  of 
learning.  One  approach  is  to  use  server-side  scripting  to  generate  HTML  pages  and  forms  on  the  fly.  Three 
systems  currently  seeking  to  improve  integration  of  WWW  learning  frameworks  are  Interbook  [Brusilovsky  et 
al.  1996],  IDEALS-MTS  [Graf  & Schnaider  1996]  and  KBS-HYPERBOOK  [Frolich  et  al.  1998].  One 
disadvantage  of  a server-centric  approach  is  that  the  resulting  content  can  be  rather  static  or  slow  in'  its 
interaction  due  to  being  bound  to  the  server,  while  making  little  use  of  the  ever-increasing  client-side  power  and 
multimedia  capabilities  of  modem  computers.  The  framework  under  development  aims  to  provide  an  optimum 
balance  between  integration,  flexibility  and  interactivity  by  linking  generic  client-side  Java  navigational  tools 
and  multimedia  renderers  with  server-side  tracking,  feedback  and  dynamic  content-generation  capabilities. 
Schank  and  Cleary  [1994]  describe  eight  principles  for  designing  educational  software,  which,  they  argue,  also 
pertain  to  teaching  in  general.  These  provide  a checklist  of  desired  tasks  and  outcomes  for  computerised  teaching 
tools.  The  need  to  relate  any  given  information  to  real-life  examples  and  problems  is  crucial,  with  preferably 
simulation  offered  to  enable  trial  and  error  analysis.  Software  should  be  adaptable  to  the  students'  needs  and  a 
good  design  will  enable  self-assessment  and  monitoring  of  progress.  The  framework  under  development  seeks  to 
satisfy  these  principles  by  providing  real-life  examples  via  multimedia  elements  such  as  2-D  and  3-D  interactive 
graphics,  sound  and  video,  and  supporting  user  experimentation  via  simulation.  Each  framework  element  is 
discussed  below  with  reference  to  a prototype  being  developed  at  the  University  of  Bath. 


Figure  1 : Screen  shot  of  prototype  showing  general  screen  layout 


Feasibility  Study 

Richards  et  al.  [1999]  describe  a feasibility  study  into  the  applicability  of  Java  technologies  for  delivery  of 
multimedia  over  the  Internet.  Multimedia  examples  from  the  domain  of  Fluid  Power  were  developed  as 
animations,  2D  and  3D  interactive  simulations,  and  digital  video.  Navigational  tools  were  also  created  to  provide 
a "clickable"  table  of  contents  (TOC)  and  the  facility  to  search  a remote  database.  Java  technologies  were  used. 
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including  "Swing”,  Java  Media  Framework,  Java  3D,  Java  Database  Connectivity  (JDBC)  and  Remote  Method 
Invocation  (RMI).  [Fig.  1]  shows  the  general  screen  layout  of  the  framework.  The  prototype  multimedia 
modules  were  tested  on  a mixture  of  PCs  and  SUM  Workstations.  The  conclusion  was  that  the  Java  technologies 
tested  had  reached  a sufficient  level  of  maturity  and  breadth  of  functionality  to  satisfy  the  requirements  for  the 
system.  Java  was  therefore  selected  as  the  authoring  language,  with  version  1.2  taken  as  the  project  baseline. 


Client  / Server  Architecture 

At  the  beginning  of  each  session,  user  data  is  read  from  a client-side  "cookie”  by  the  TOC  applet  and  passed  to 
the  server.  The  server  creates  a new  session,  retrieves  the  corresponding  user  object  from  the  previous  session 
(or  creates  a new  one),  and  returns  the  URL  of  the  last  page  visited,  enabling  the  TOC  to  load  the  corresponding 
page  in  the  Content  Panel  and  update  the  TOC  node  elements  accordingly.  The  appearance  of  the  nodes  indicates 
which  sections  have  been  consulted  and  (where  applicable)  successfully  completed.  The  tree  structure  of  the 
TOC  itself  provides  one  suggested  path  through  the  course  material,  but  additional  guidance  will  be  provided  by 
modifying  the  structure  and  appearance  of  the  nodes  according  to  user  actions  through  integration  of  the  JESS 
[Friedman-Hill  1999]  inference  engine  on  the  server  side.  Java  RMI  is  used  for  client/server  interaction  because 
it  offers  the  possibility  of  passing  true  Java  objects  between  client  and  server.  The  architecture  also  lends  itself  to 
the  provision  of  expert  design  guidance  as  part  of  the  overall  Performance  Support  System. 


Re-Usability  of  Software  Components 

In  order  to  allow  users  to  construct  circuits  and  observe  their  behaviour  via  simulation.  Fluid  Power  Components 
have  been  modelled  as  JavaBeans,  which  can  be  loaded  dynamically  and  discover  each  other  at  run-time.  The 
JavaBeans  Event  Model  enables  components  to  be  aware  of  flow,  pressure  and  demand  signal  changes  within 
the  circuit  and  react  accordingly.  Circuits  built  up  in  this  way  can  be  saved  and  re-used  as  courseware  using  the 
generic  multimedia  renderer  applet  (see  below).  The  user  can  choose  to  view  each  component  in  the  form  of  a 
realistic  (and,  where  appropriate,  animated)  diagram  or  as  an  internationally-recognised  symbol.  Component 
pop-up  menus  contain  embedded  search  hyperlinks  that  trigger  searches  in  a remote  multimedia  database.  The 
results  of  a search  can  be  browsed  by  the  user  in  a separate  window  using  a generic  multimedia  viewer.  This 
viewer  provides  a single  container  into  which  the  system  can  plug  different  renderers  on  the  fly,  depending  on 
the  type  of  multimedia  content  to  be  displayed.  The  same  generic  viewer  can  also  be  embedded  as  an  applet  in 
an  HTML  page,  maximising  software  re-use  and  simplifying  the  work  of  the  courseware  author. 


User  Testing  of  Prototype 

Selection  of  Test  Users,  Equipment  and  Courseware 

The  test  users  were  attendees  of  a Fluid  Power  course  run  by  the  Centre  for  Power  Transmission  and  Motion 
Control  at  the  University  of  Bath.  The  course  was  entry-level  in  terms  of  its  Fluid  Power  content  and  was 
administered  using  traditional  teaching  methods.  The  delegates  were  divided  into  four  groups  with  6 trainees  per 
group.  The  allocation  of  groups  was  carried  out  by  the  course  organisers  on  a random  basis.  The  experiment  was 
conducted  on  four  PCs,  each  running  either  Windows95  or  NT4.0,  ranging  from  a 200MHz  Pentium  MMX 
laptop  to  a 400MHz  Pentium  II.  Installed  RAM  was  between  64  and  160  MB,  but  available  RAM  dropped  to 
between  -9  and  44  MB  with  both  the  operating  system  and  web  browser  running.  The  web  browser  used  was 
Netscape  Navigator,  though  the  framework  has  also  been  used  successfully  with  Microsoft  Internet  Explorer.  In 
addition,  each  test  PC  was  set  up  with  the  Java  Plug-In  (version  1.2.2,  using  Java  Runtime  Environment  version 
1.2),  Java  Media  Framework  1.1  for  video  rendering  and  Java3D  1.1.1  (DirectX  for  Win95,  OpenGL  for  NT)  in 
conjunction  with  the  Java3D  VRML  97  loader  for  rendering  of  3D  content.  The  existing  course  notes  were 
converted  into  individual  HTML  pages,  each  page  corresponding  to  a section  within  a chapter.  The  formatting 
was  adapted  to  on-line  viewing  using  Cascading  Style  Sheets  and  by  the  inclusion  of  multimedia  elements  where 
it  was  thought  that  this  would  enhance  comprehension. 
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Test  Procedure  and  Method  of  Evaluation 


Each  test  session  lasted  35  minutes.  A short  introduction  to  the  system  was  given  in  which  the  objectives  and 
procedure  of  the  experiment  were  explained.  It  was  left  to  the  users  to  organise  themselves  between  the  available 
workstations.  In  general,  the  less  experienced  computer  users  tended  to  work  together.  The  users  were  allowed  to 
proceed  at  their  own  pace  through  the  chapters  of  the  test  courseware.  A questionnaire  was  used  to  evaluate  the 
users'  reactions,  which  was  divided  into  four  sections.  The  first  section,  consisting  of  14  questions,  concentrated 
on  the  generic  features  of  the  system,  such  as  navigational  means,  screen  layout,  and  feedback  given  to  the  user 
by  the  system.  The  second  section  (three  questions)  focussed  on  the  users'  estimation  of  the  suitability  of  the 
system  and  multimedia  in  general  to  future  training  needs.  The  third  section  (10  questions)  asked  users  to  assess 
the  effectiveness  of  the  multimedia  elements  used  and  also  give  personal  preferences  for  relative  levels  of 
differing  types  of  multimedia  in  future.  The  fourth  and  final  section  collected  data  on  the  users'  individual  levels 
of  experience.  All  questions  used  sets  of  bipolar  semantically  anchored  items,  on  a scale  from  1 to  5,  except  for 
the  questions  on  personal  experience  where  a scale  of  1 to  10  was  used  in  order  to  provide  users  with  greater 
freedom.  Due  to  the  practical  constraints  of  having  to  operate  within  the  framework  of  the  external  course,  it  was 
not  possible  to  administer  pre-  and  post-tests  or  to  divide  the  users  into  control  and  test  groups. 
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Figure  2:  Evaluation  of  generic  features  of  system 
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Figure  3:  Evaluation  of  effectiveness  of  multimedia  content 


Test  Results 

There  was  a broad  range  in  the  users'  experience  with  both  Fluid  Power  and  the  Internet,  but  in  general  the  users 
could  be  said  to  have  "average"  experience  of  these  domains.  In  contrast,  they  considered  themselves  above 
average  in  terms  of  their  computer  skills.  As  can  be  seen  from  [Fig.  2]  the  generic  features  of  the  system  such  as 
TOC,  search  facility,  navigation,  screen  layout  and  overall  impression  were  felt  to  be  mostly  "Good"  or  "Very 
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Good".  Comments  made  indicated  that  users  liked  the  ability  to  simultaneously  view  the  content  and  browse  to 
another  topic  via  the  navigational  tools.  The  screen  layout  was  thought  to  be  clear  and  uncluttered.  The  ability  to 
increase  the  size  of  the  content  panel  by  selectively  hiding  and  revealing  the  navigational  panel  was  welcomed. 

[Fig.  3]  shows  a high  appreciation  by  the  users  of  the  effectiveness  of  the  multimedia  elements  used  in  the 
tutorial,  particularly  the  2D  and  3D  simulations.  Comments  made  indicated  that  the  3D  axial  piston  pump 
simulation  in  particular  [Fig.  4]  was  a good  example  of  interactive  multimedia  techniques  enabling  users  to 
visualise  a Fluid  Power  concept  which  is  extremely  difficult  to  teach  and  visualise  with  traditional  methods.  In 
the  context  of  what  form  of  training  users  would  generally  prefer,  however,  [Fig.  5]  reveals  a relatively  low 
preference  for  multimedia-based  instruction  over  traditional  methods.  This  particular  question  generated  the 
most  comments  by  far.  Many  users  emphasised  the  value  of  human  interaction  - both  with  tutors  and  other 
trainees  - which  is  absent  in  self-paced  individual  multimedia  learning,  and  proposed  the  adoption  of  some  kind 
of  hybrid  approach.  Notwithstanding  this,  the  users  were  very  enthusiastic  about  the  possibility  of  future 
multimedia  courses  being  offered  and  the  suitability  of  the  system  when  fully  developed  for  meeting  future  Fluid 
Power  training  needs. 


Figure  4:  Interactive  3D  simulation  of  axial  piston  pump 
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Figure  5:  Suitability  of  system  for  meeting  future  Fluid  Power  training  needs 


Conclusions 


The  Java-based  hypermedia  framework  described  in  this  paper  has  been  implemented  in  prototype  form  within 
the  context  of  an  overall  Performance  Support  System  for  Fluid  Power  Systems  Design.  The  content  used  was 
drawn  from  the  domain  of  Fluid  Power  Systems,  and  formative  testing  of  the  framework  as  an  educational  tool 
was  carried  out  with  Fluid  Power  Systems  trainees.  The  results  showed  a high  level  of  appreciation  of  the 
navigation  and  search  facilities,  the  multimedia  elements  used  and  the  overall  functionality.  The  application  of 
3D  techniques  to  provide  visualisation  of  complex  concepts  was  particularly  valued.  The  possibility  of  future 
multimedia-based  courses  was  welcomed,  and  it  was  felt  that  the  system  would  ultimately  be  well  suited  for  the 
future  training  needs  of  the  Fluid  Power  industry.  In  future,  summative  testing  will  be  carried  out  to  evaluate  the 
framework's  educational  effectiveness.  Its  use  as  a design  support  tool,  complementing  the  educational  role,  will 
also  be  investigated.  Although  the  content  itself,  together  with  a number  of  the  prototype's  interactive  elements, 
was  specific  to  Fluid  Power  Systems  Design,  the  overall  framework  is  by  no  means  limited  to  this  one  domain, 
but  lends  itself  to  application  in  many  other  fields.  In  the  next  stage  of  development,  the  adoption  of  XML  for 
content  storage  and  encoding  will  further  enhance  its  cross-domain  applicability. 
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Abstract:  This  is  a report  on  a case  study,  which  was  carried  out  by  applying  the  qualities  of 
meaningful  learning  based  on  constructivist  theory  to  the  evaluation  of  the  distance  learning 
environment.  As  a part  of  the  Web  University  -pilot,  the  evaluation  was  directed  to  the  lecture 
series:  “Web  Publishing  Today  and  Tomorrow”  by  H^kon  Lie,  organized  at  CERN  (European 
Organization  for  Nuclear  Research),  in  Geneva,  Switzerland.  Web  University  distributed  the 
lectures  to  distance  audiences  in  Finland  over  a high-speed  network  and  recorded  them  on  a server. 
Originally,  the  lectures  have  been  planned  for  face  to  face  learning  but  modified  suitable  for 
distance  learning  by  using  learning  tasks  of  the  distance  students.  The  distance  learning 
environment,  delivery  technology  complemented  by  distance  tasks,  has  been  evaluated  by 
analyzing  how  the  qualities  of  meaningful  learning  appear.  The  results  of  the  case  study  lend 
support  to  assumptions  that  delivery  technology  could  be  complemented  by  the  student  project 
work  and  obtain  the  qualities  of  meaningful  learning. 


Introduction 

Web  University  (Rinta-Filppula  1999)  is  the  international  pilot  of  the  Finnish  distance  learning  project 
“Open  Learning  Environment”  (1999-2002).  Web  University  is  based  on  the  concept  of  virtual  university  where 
distance  participants  communicate  over  the  network  by  using  applications  such  as  multicast  interactive 
videoconferencing,  video-on-demand.  World  Wide  Web  (Web),  email  and  mailing  lists.  The  project  uses  national 
research  networks  and  European  wide  high  speed  Internet  connections. 

Distance  learners  can  update  their  knowledge  by  participating  in  CERN  seminars  from  their  personal 
workstations  or  conference  rooms  over  the  real-time  broadband  network  based  on  ATM  (Asynchronous  Transfer 
Mode)  and  Internet  technology.  Teaching  is  interactive  and  it  is  in  principle  targeting  researchers  and  postgraduate 
students,  but  some  courses  in  physics  and  information  technology  have  been  additionally  offered  to  undergraduate 
students  and  special  groups,  such  as  journalists  and  general  public. 

The  latest  scientific  knowledge  of  the  researchers  can  be  applied  straight  and  immediately  even  to 
undergraduate  student  studies.  The  technology  enables  to  attend  the  lectures  and  update  knowledge,  no  matter  the 
physical  location  of  the  learner. 

The  aim  of  this  case  study  is  to  evaluate  a realization  of  the  vision  how  CERN  lectures  could  be  integrated 
in  local  studies  of  distance  students  by  using  the  qualities  of  meaningful  learning  based  on  constructivist  theory 
(Rinta-Filppula  1999).  The  evaluation  has  been  carried  out  analyzing  the  learning  tasks  and  the  recorded 
transmissions.  The  selected  case  is  the  lecture  series  “Web  Publishing  Today  and  Tomorrow”  given  by  Hakon  Lie 
from  the  World  Wide  Web  Consortium  in  March  1999  at  CERN  and  a local  implementation  of  distance  student 
tasks  in  Tampere  University  of  Technology  in  Finland. 


The  Qualities  of  Meaningful  Learning 


The  evaluation  was  realized  by  reflecting  the  qualities  of  meaningful  learning  over  distance  learning 
environment.  The  content  of  the  lectures  was  not  included  in  the  analysis.  The  aim  of  the  study  was  to  analyze  how 
the  qualities  appear  in  real-time  sessions,  recorded  transmissions  and  learning  tasks.  Two  alternative  learning  tasks  - 
learning  diary  and  project  work  with  student  evaluation  reports  - were  connected  to  the  lectures  in  Tampere 
University  of  Technology,  where  the  distance  students  participated  in  real-time  lectures. 

Theoretical  background  of  this  evaluation  lies  on  the  theory  of  constructivism.  Based  on  the  assumptions  of 
constructivist  perspective  Jonassen  has  proposed  a list  of  qualities  of  meaningful  learning  (Jonassen  1995), 
Ruokamo  & Pohjolainen  (1998,  293-294;  1999,  7;  1999,  924)  have  merged  conversational  and  collaborative 
qualities  in  Jonassen’s  list  into  one  and  added  the  quality  of  transfer  into  the  list.  On  the  basis  of  Jonassen  and 
Ruokamo  & Pohjolainen,  seven  qualities  will  be  presented  as  follows: 

Active  - Learners  have  an  active  role  during  the  mindful  processing  of  information  and  they  are  responsible 
for  the  result. 

Constructive  - Learners  construct  new  knowledge  by  accommodating  new  ideas  into  prior  knowledge. 

Collaborative  - Learners  cooperate  in  building  new  knowledge  together  exploiting  each  other’s  skills  and 
providing  social  support,  , 

Intentional  - Learners  attempt  actively  and  willingly  to  achieve  a cognitive  object. 

Contextual  - Learning  tasks  are  situated  in  a meaningful  real  world  tasks  or  they  are  introduced  through 
case -based  or  problem-based  real  life  examples. 

Transfer  - Learners  are  able  to  transfer  learning  from  situations  and  contexts,  where  learning  has  taken 
place  and  use  their  knowledge  in  other  situations. 

Reflective  - Learners  articulate  what  they  have  learned  and  reflect  on  the  processes  and  decisions  entailed 
by  the  process. 


Distance  Learning  Environment 

\ 

Interactive  and  Real-time  Videoconferencing 

The  CERN  lectures  were  transmitted  real-time  and  interactive  over  the  high-speed  network  and  recorded 
digitally  on  CERN  server  at  the  same  time.  The  recordings  were  transferred  to  FUNET  (Finnish  University  and 
Research  Network)  server  daily  after  the  real-time  session.  This  gives  an  opportunity  for  distance  participants  to 
study  distributed  lectures  in  case  a temporary  network  problem  in  a real-time  learning  session. 

The  lecturer  and  the  distance  audiences  in  Finland  were  introduced  at  the  beginning  of  CERN’s  face  to  face 

lecture  series.  The  lecturer  Hakon  Lie  welcomed  all  the  participants  at  CERN  and  in  Finland  (Espoo,  Pori  and 

Tampere)  and  the  distance  learners  were  taken  notice  and  accepted  to  attend  the  lectures  as  local  audience. 

The  lecturer  gave  the  distance  audiences  an  opportunity  to  ask  questions  during  the  lecture,  not  only  at  the 

end  of  each  lecture.  Some  questions  were  asked  during  the  lectures.  When  a distance  learner  wanted  to  ask  a 

question,  he/she  showed  the  question  sign  whereafter  the  lecturer  gave  him/her  the  floor.  The  learner  then  took  a 
microphone  and  asked.  The  lecturer  and  all  the  participants  at  CERN  and  in  Finland  heard  the  question  and  saw  the 
speaker. 

Real-time  interaction  between  the  lecturer  and  the  distance  learners  functioned  well.  The  learners  and  the 
lecturer  were  active,  the  situation  was  authentic  and  the  audio  was  clear  and  comprehensible.  Only  movements  of 
video  images  were  a little  bit  slow  because  of  256  kbit/s  limitation  of  the  software  used  for  transmissions.  In  spring 
2000,  the  limitation  has  been  risen  up  to  1 Mbit/s. 


Video-on-demand  Service 

The  whole  transmission  including  the  distance  audiences  has  been  recorded  on  a server.  Successful 
studying  of  the  recordings  requires  PC,  Linux  or  Unix  workstation  with  network  connection,  soundcard  and  loud 
speakers. 

Studying  the  recorded  transmission  presupposes  an  active  role  from  the  learner.  The  control  of  the  recorded 
lecture  is  on  the  learner’s  hands.  From  the  control  panel  of  the  recorder,  the  learner  is  able  to  see  how  long  the 
lecture  is  still  going  on.  It  is  also  possible  to  return  to  a chosen  point  of  the  lecture  again,  e.g.  to  check  some 
information.  The  learner  can  study  the  recordings  from  his/her  own  workstation  at  his/her  convenience.  He/she  can 
study  the  recorded  lectures  together  with  other  local  learners  and/or  a tutor  or  professor  whenever  necessary. 


The  interaction  between  the  lecturer  and  the  learner  who  studies  the  recordings  is  possible  by  using 
applications  such  as  e-mail  or  mailing  list.  The  benefit  of  the  recorded  transmissions  is  that  they  are  available  when 
the  students  have  time  to  study. 

The  quality  of  the  real-time  session  is  as  good  as  the  quality  of  digital  recording  taken  that  the  real-time 
network  functions  without  any  problems.  This  has  been  usually  the  case:  the  sound  has  been  excellent,  the  video 
colored,  clear  and  bright  and  the  images  of  the  lecturer  and  the  distance  audiences  in  Finland  all  visible  and 
recorded.  Actually,  the  whole  transmission  has  been  recorded.  It  is  also  possible  just  to  listen  to  the  recordings 
without  observing  the  video. 


Learning  Tasks 

The  students  in  Tampere  University  of  Technology  included  the  CERN  lectures  to  their  studies  and 
accumulated  credits.  In  addition  to  attending  the  interactive  and  real-time  videoconference,  their  professor  Seppo 
Pohjolainen  integrated  alternative  learning  tasks  to  the  lectures.  The  students  should  write  individual  learning  diaries 
or  prepare  project  works  in  small  groups  and  write  down  student  evaluation  reports  about  their  work  and  learning 
experiences.  Moreover,  the  professor  asked  all  students  to  add  own  opinions  and  criticism  with  arguments. 

The  professor  requested  the  students  to  describe  in  their  dairies  the  content  of  the  lectures  as  well  as  the 
topics  they  had  found  interesting.  Furthermore,  the  students  were  asked  to  analyze  the  way  the  lectures  were 
accomplished  and  suggest  how  the  lectures  should  be  improved. 

The  students  who  chose  to  write  the  diaries  got  one  study  credit,  whereas  the  students  participating  in  the 
lectures  and  the  project  work  in  a small  group  could  get  three.  The  project  work  was  realized,  as  follows:  a team  was 
asked  to  name  the  project  work  after  an  interesting  topic  presented  by  the  lecturer.  In  addition,  the  students  should 
deepen  the  chosen  topic  by  examining  their  notes  and  looking  for  material  from  the  library  and  Web.  The  team  was 
required  to  publish  their  project  work  on  the  Web  and  study  other  team  project  works.  The  teamwork  also  consisted 
of  an  individual  contribution  of  each  student.  Finally,  the  team  was  asked  to  present  their  work  in  a virtual  seminar 
and  prepare  questions  about  other  team  project  works.  The  professor  also  wanted  the  team  to  analyze  the  project 
working  itself  and  both  the  personal  and  team  learning  process.  Nearly  twenty  students  with  their  professor 
participated  in  the  real-time  lectures  in  Tampere  University  of  Technology.  Ten  of  them  made  distance  tasks  and 
passed  the  course,  six  students  wrote  diaries  and  four  students  formed  a group. 


Evaluation  of  the  Learning  Tasks 

Both  learning  tasks  supported  the  student  learning  as  a process,  where  the  knowledge  is  constructed 
through  a personal  and  active  participation.  It  presupposes  not  only  an  active  participation  in  the  lectures  but  also 
successful  activation  of  the  students  to  follow  the  lectures  intensively.  The  abstract  of  the  lectures  published  on  the 
Web  beforehand  facilitated  the  student  orientation  to  the  content  - it  activated  the  prior  knowledge  of  the  learners. 
This  might  also  encourage  the  learner  to  search  information  on  the  topics  independently. 

Learning  Diaries 

Each  learner  writing  a diary  had  to  react  to  the  information  with  a critical  attitude  and  present  his/her 
personal  opinions  about  the  topics.  This  presupposes  that  the  student  understands  the  content  and  facilitates  calling 
in  question  in  the  knowledge.  In  addition,  it  activates  the  student  to  modify  prior  knowledge  with  new  one  and 
requires  him/her  to  give  arguments  for  own  understanding  and  ideas.  The  conversation  with  another  learners  is  seen 
to  improve  the  student  social  skills,  collaboration  and  argumentation. 

Writing  a diary  strengthens  the  student  as  an  active,  responsible  and  self-directed  learner.  The  fact  that  the 
students  were  additionally  asked  to  write  about  the  topics  that  interested  them  personally  supports  the  student  as  an 
intentional  learner. 

The  students  were  asked  to  comment  on  not  only  the  content  of  the  lectures  but  also  the  way  the  lectures 
were  realized  in  practice.  The  feedback  showed  that  they  were  satisfied  with  the  lecturer  and  technology,  which 
functioned  even  surprisingly  well.  The  audio  was  very  clear  but  the  images  of  the  video  could  have  had  a higher 
speed.  The  content  of  the  lectures  was  considered  interesting  and  useful.  Some  students  requested  to  have  more  this 
kind  of  expert  lectures. 
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When  writing  a learning  diary  the  student  reflection  is  internal  if  he/she  has  neither  experience  nor  practice 
in  process  writing.  The  student  could  naturally  reflect  his/her  own  ideas  with  the  professor  who  gave  the  credits.  The 
interaction  with  the  other  students  would  probably  have  deepened  the  learning  process  because  the  learning  and 
knowledge  construction  are  also  collaborative  and  socially  interactive. 


Project  Work 

Technical  help  was  available  for  the  student  throughout  the  learning  period.  This  is  important  when  using 
new  technology.  The  project  work  organized  and  created  in  a small  group  required  both  active  individual  and 
collaborative  learning.  The  team  had  to  select  a jointly  interesting  topic  and  divide  the  work  itself  into  individual 
tasks.  The  group  should  select  work  tools  and  make  a timetable  and  a plan  to  ensure  that  all-essential  information 
would  be  included.  Furthermore,  the  request  for  additional  material  to  the  project  work  encouraged  each  student  to 
take  an  active  role  in  his/her  own  and  group  learning  process.  It  also  deepened  the  understanding  and  knowledge  of 
the  learner. 

The  learning  process  engages  the  students  in  analyzing  information  through  the  project  work.  Each 
individual  student  is  responsible  for  the  whole  team  success  by  contributing  his/her  own  activity,  knowledge  and 
skills  for  the  good  of  the  project.  The  team  was  also  asked  to  report  on  the  division  of  labor  of  the  group.  This  makes 
possible  to  evaluate  the  activity  of  an  individual  member  and  ensures  that  the  student  is  willing  to  take  responsibility 
for  his/her  own  objectives  of  the  teamwork.  The  fact  that  the  group  made  their  project  work  about  the  topic  that 
interested  personally  and  published  it  on  the  Web  gave  some  additional  support  to  intentional  learning. 

The  content  of  the  lectures  was  implemented  by  asking  the  team  to  publish  their  project  work  on  the  Web. 
This  means  that  the  learners  are  able  to  transfer  their  knowledge  to  a new  situation  and  use  it.  The  technology  and 
the  content  of  the  lectures  offered  a fine  learning  environment  not  only  by  providing  the  lectures  but  also  by 
operating  as  a context  of  knowledge  performing.  The  title  of  the  project  work  carried  out  is  Style  Sheets.  Some 
students  who  wrote  the  dairies  had  asked  additional  information  on  and  examples  of  the  style  sheets.  Now  they  and 
others  who  will  later  study  Web  publishing  have  all  necessary  available  on  the  Web. 

The  project  work  enabled  the  student  collaboration.  The  team  members  had  a common  goal  and  they 
constructed  new  knowledge  together.  The  seminar  at  the  end  of  the  project  work  supported  collaboration  as  well  as 
the  development  of  social  skills  and  reflection.  As  only  one  group  was  formed,  the  team  reflected  their  own  project 
work  with  their  professor  in  the  face-to-face  seminar.  The  learners  used  e-mail  and  organized  four  face-to-face 
meetings  during  the  project  because  they  all  worked  in  the  same  office. 

In  addition  to  the  project  work,  the  team  was  asked  to  report  their  learning  experiences  by  writing  down 
what  they  had  learned  individually  and  jointly.  The  student  work  and  learning  evaluation  report  enable  to  analyze 
the  learning  process  as  a whole.  The  students  articulated  the  matters  they  had  learned  about  the  topics.  It  was 
probably  too  difficult  to  describe  the  learning  process  itself  if  they  had  no  previous  experience  in  it.  More  detailed 
questions  about  the  student  learning  process  could  support  the  process. 

The  students  also  gave  statements  about  execution  of  the  course  and  proposed  e.g.  how  to  improve  it.  This 
means  that  the  students  were  accepted  to  participate  in  the  development  work.  The  students  gave  affirmative 
feedback  about  the  lecturer,  the  lectures  and  very  well  functioned  technology.  In  addition,  they  expressed  a wish  that 
the  slides  would  be  published  in  advance  on  the  Web.  In  order  to  help  the  students  to  concentrate  on  making  their 
own  notes  and  comments  to  the  copies  instead  of  wasting  time  for  copying  the  slides.  The  distance  learners 
evaluated  very  carefully  the  technology  used  for  transmissions.  They  even  noticed  that  the  audience  at  CERN  heard 
distance  questions  better  than  the  lecturer. 


Conclusions 

In  this  case  of  distance  education,  the  qualities  of  meaningful  learning  were  realized  at  least  partially 
throughout  the  process.  The  real-time  interactive  videoconferencing  enabled  interaction  between  the  distance 
students  and  the  experts.  The  digital  recordings  and  the  material  on  the  Web  supported  the  student  flexible  learning 
process  and  encouraged  the  individual  and  group  activity.  In  addition,  the  means  and  methods  mentioned  above 
offered  the  students  an  opportunity  to  reflect  their  prior  knowledge  and  notes  taken  when  constructing  and 
deepening  the  knowledge  by  doing  the  tasks.  This  kind  of  distance  education  gave  the  students  even  a possibility  to 
include  the  latest  research  results  to  their  studies  and  reflect  their  own  ideas  with  the  well-known  experts.  This  all 
took  place  thanks  to  well-functioned  technology. 


The  learning  tasks,  learning  diaries  and  project  work  together  with  the  student  evaluation  report  indicated 
the  qualities  of  meaningful  learning  but  on  a different  level  of  student  learning  process.  Both  tasks  were  based  on  the 
student  activity  and  self-learning.  The  intentional  learning  was  realized  through  writing  the  topics  and  selecting  the 
project  work  topic  that  interested  the  students.  The  learning  context,  knowledge  construction,  transfer  and  reflection 
were  achieved  by  creating  and  publishing  the  project  work  about  Web  publishing  on  the  Web  and  by  writing  the 
student  learning  evaluation  report.  The  student  knowledge  construction  and  reflection  seemed  to  happen  by  writing 
the  learning  diary.  These  were  internal  processes  for  the  student  and  more  superficial  in  comparison  with  the  other 
learning  task. 

Jonassen  underlines  (1995)  that  the  most  productive  and  meaningful  uses  of  technology  engage  learners  in 
knowledge  construction,  conversation,  articulation,  collaboration  and  reflection.  Technology  used  in  this  case  was 
delivery  technology  complemented  by  learning  tasks.  The  results  of  the  case  study  lend  support  to  assumptions  that 
delivery  technology  could  be  complemented  by  the  student  project  work  and  could  thus  obtain  the  qualities  of 
meaningful  learning. 
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Abstract:  Graphics-dependent  industries  such  as  advertising  and  the  print  and  electronic 
media,  as  well  as  institutions  which  undertake  in-house  and  on-line  publishing,  require 
computer  professionals  skilled  in  electronic  image  manipulation,  however,  employers  often 
overlook  the  need  for  staff  to  apply  high  ethical  standards  in  the  practice  of  their  technical 
skills.  Computer  technology  allows  ‘seamless’  alteration  of  photographs  and  other  images,  but 
these  technical  advances  have  made  possible  visual  misrepresentation  with  intent  to  deceive, 
as  well  as  difficulty  in  establishing  copyright  for  original  images.  The  challenge  for  educators 
in  teaching  techniques  of  image  manipulation  is  to  instil  in  students  an  ethical  basis  for  their 
practice  as  computer  professionals.  This  paper  describes  the  image/ethics  interface  in  a higher 
education  multimedia  program. 


Introduction 

A 1999  television  advertisement  on  behalf  of  an  Australian  Cancer  Research  Foundation  is  ‘spoken’  by 
the  late  actor  Steve  McQueen,  who  died  of  lung  cancer  some  twenty  years  ago.  He  begins  his  message  by  saying, 
‘1  died  in  1980.  There  wasn’t  a cure  for  cancer  then  and  there  isn’t  now,  but  ...’  and  he  goes  on  to  solicit 
donations  to  assist  cancer  research.  This  advertisement  demonstrates  state-of-the-art  computerised  imaging,  but 
also  brings  into  sharp  relief  ethical  questions  related  to  image  manipulation. 

On  investigation,  no  ethical  problem  is  created  by  this  particular  advertisement,  as  was  explained  in  a 
television  interview  (Kirwan  1999)  when  a spokesman  for  the  Sydney  advertising  agency  which  produced  the 
video  clip  described  its  technical  dimensions,  and,  more  significantly,  how  its  graphic  effects  were  accomplished 
ethically.  Kirwan  told  how  his  advertising  agency  gained  prior  approval  from  the  executors  of  Steve  McQueen’s 
estate  for  the  use  of  a clip  from  McQueen’s  film  The  Cincinnati  Kid,  in  which  the  actor  spoke  directly  to  the 
camera.  To  create  the  impression  that  McQueen  was  ‘speaking’  twenty  years  after  his  death  in  support  of  cancer 
research  required  the  use  of  an  actor  with  a McQueen  look-alike  mouth,  and  another  with  a McQueen  sound- 
alike  voice,  and  these  physical  features  were  ‘seamlessly  ‘ dubbed  into  the  old  film  clip,  using  similar  imaging 
techniques  to  those  which  created  the  ‘talking’  pig  and  other  animals  in  the  Australian  film.  Babe. 

From  its  beginning,  the  McQueen  project  was  ethically  beyond  reproach,  from  the  worthiness  of  its 
intent,  the  seeking  permission  to  use  the  actor’s  image,  the  involvement  of  the  McQueen  executors  during  the 
advertisement’s  production,  and  their  approval  of  its  final  outcome.  However,  the  print  and  electronic  media  do 
not  always  display  such  ethical  behaviour  in  respect  of  the  computer  manipulation  of  photographic  images.  For 
example,  in  Australia,  on  the  day  following  the  tourist  shootings  at  Port  Arthur,  in  Tasmania,  a photograph  of  the 
alleged  gunman’s  face  appeared  on  the  front  page  of  some  Australian  newspapers  with  his  eyes  ‘re-touched’  to 
create  a sinister  facial  expression.  This  unacknowledged  ‘re-touching’  was  evident  when  the  same  (but 
untouched)  photograph  of  the  suspect  appeared  in  other  newspapers. 

Similarly,  in  the  United  States,  at  the  time  of  the  O.J.  Simpson  trial,  a cover  of  Time  magazine  featured  a 
computer-retouched  police  ‘mug  shot’  of  the  former  football  star  and  then  alleged  murderer.  When  challenged 
that  the  photographic  manipulation  was  misleading,  racist,  and  potentially  prejudicial,  the  editor  argued  that  the 
photographic  image  had  been  ‘subtly  smoothed  and  shaped  into  an  icon  of  tragedy’  (Wheeler  & Gleason  1995: 
8),  however,  under  further  pressure,  the  editor  did  in  fact  apologise  to  his  readers  for  this  photographic 
manipulation.  During  this  same  period,  Kobre  (1995:  14)  describes  a National  Enquirer  photograph  under  the 
banner  headline  ‘Battered  Nicole:  Photos  taken  by  her  sister  show  how  O.J.  beat  her  up’,  which  gave,  at  first 


impression,  a photograph  of  Nicole  Simpson’s  bloodied  and  swollen  face,  until  a statement  in  small  type  under 
the  photograph  was  noticed,  'Sister  describes  photos  seized  by  cops  - computer  re-creation  \ 

In  the  face  of  such  blatant,  misleading  manipulation,  Kobre  makes  a case  for  greater  photographic  access 
to  all  situations  and  a wider  distribution  of  photographs,  so  that  fakes  may  be  countered  with  the  truth.  He  argues 
that  had  the  media  had  access  to  the  pictures  tendered  at  the  trial  of  the  beaten  Nicole  Simpson,  which  were  even 
more  horrendous  than  the  Enquirer's  fictional  image,  then  the  fake  would  not  have  been  necessary,  or  it  would, 
at  least,  have  been  countered  by  the  authentic  images.  Unfortunately  for  Kobre’ s argument,  the  control  of  visual 
news  information  increasingly  rests  with  a powerful  few  (Reaves  1995),  and  photographic  manipulation  has  a 
long  history.  Thus  there  appears  little  chance  that  greater  photographic  access  will  control  this  misrepresentation 
by  skilfully  altered  photographs  unless  the  ethics  of  its  practice  are  challenged. 


Photographic  manipulation’s  long  history  of  deception 

The  manipulation  of  photographs  dates  back  to  the  early  nineteenth  century,  when  photography,  while 
basing  its  validity  on  its  technical  capability  for  making  representations  of  reality,  was  always  a ‘combination  of 
artistic,  technical,  theatrical  and  entrepreneurial  achievement’  (Slater  1995:  218-219).  Thus,  from  its  inception 
the  concept  of  photography  as  truth  has  been  set  alongside  artforms  such  as  photomontage,  where  photographic 
manipulation  was  used  for  caricature  (Evans  & Gohl  1986),  but  in  these  early  days,  this  practice  most  often 
stopped  short  at  retouching  to  remove  unflattering  facial  characteristics  such  as  wrinkles  and  pock  marks  in 
middle  class  portraiture,  which  was  popular  at  the  time  (Lester  1991;  Kobre  1995). 

Later  there  was  more  serious  manipulation  with  obvious  intent  to  deceive,  and  this  may  be  seen  in 
historial  war  and  political  photography,  where  the  deceptions  were  not  detected  until  much  later.  For  example, 
some  photographic  manipulations  from  the  American  Civil  War  were  discovered  only  when  similar  photographs 
were  set  alongside  each  other  and  comparisons  revealed,  for  example,  the  rearrangement  of  ‘corpses’  in  different 
photographs  of  the  same  scene.  Even  the  famous  portrait  of  Abraham  Lincoln  was  a composite  of  the  body  of  a 
Southern  statesman,  with  Lincoln’s  head  sandwiched  atop.  (The  same  technique  has  been  used  recently  to  set 
Oprah  Winfrey’s  head  on  the  body  of  actress  Ann-Margret  on  a TV  Guide  magazine  cover.) 

In  Making  People  Disappear,  Jaubert  (1986)  (cited  by  Lester  1991:  101)  chronicled  numerous 
photographic  abuses  by  totalitarian  regimes  which  involved  using  retouching  and  effacement  techniques  to 
doctor  the  historical  record  to  reflect  a political  leader’s  version  of  the  truth.  And,  in  more  recent  times,  charges 
of  manipulation  cloud  famous  historic  photographs.  Lester  (ibid.:  117)  recounts  the  ‘unrelenting  ethical 
controversy’  surrounding  Rosenthal’s  Pulitzer  Prize  photograph  involving  the  US  marines’  flag-raising  at  I wo 
Jima  in  World  War  II.  Subsequently  it  was  revealed  there  were  at  least  four  photographs  taken  at  the  time,  three 
by  Rosenthal  and  another  by  a Naval  photographer,  which  suggests  the  occasion  of  the  famous  photograph  was 
staged,  and  it  was  not  the  eye-witness  representation  of  the  historical  event  as  was  first  believed. 

What  is  significant  about  this  historical  record  of  photographic  manipulation  is  not  that  it  happened,  but 
that  it  was  so  easily  discovered.  Detection  was  easy  then  because  the  clumsy  techniques  of  the  time  allowed  the 
discovery  of  alterations.  As  well,  these  earlier  techniques  took  painstaking  work  which  was  expensive  and  time 
consuming,  and  this  deterred  the  practice  from  becoming  more  common.  On  both  counts,  this  is  no  longer  the 
case,  for  the  application  of  computer  technology  now  allows  photographs  to  be  modified  seamlessly,  that  is,  with 
little  or  no  evidence  of  alteration,  and  with  an  ease  and  speed  which  encourages  the  practice. 

Modem  computerized  photographic  techniques  also  allow  the  quick  synthesis  of  artificial  images  which 
are  not  based  on  reality.  When  a photograph  is  taken  with  a digital  camera,  or  scanned  and  converted  to  digital 
information,  the  entire  image  can  be  modified  in  many  ways,  that  is,  its  colour,  brightness  and  focus  can  be 
changed,  and  elements  of  the  image  (such  as  a background,  or  persons)  can  be  replicated  or  removed  in  a process 
which  takes  only  minutes  and  is  almost  impossible  to  detect.  In  some  cases,  these  techniques  are  revealed 
intentionally  to  warn  readers  of  the  potential  for  deception,  as  in  a Science  84  cover  (cited  in  Ritchin,  1991:  32) 
which  used  three  major  imaging  techniques  to  create  a ‘realistic’  image  from  scratch,  that  is,  a simulated  reality. 
The  editor  did  not  set  out  to  deceive  his  readers,  in  fact  the  cover  carried  a caption  ‘This  Picture  is  a Fake’. 

In  addition  to  issues  of  misrepresentation,  photographic  manipulation  raises  questions  of  both  copyright 
and  ownership  with  respect  to  the  original  photograph.  Should  the  original  photographer  and  the  photographic 
manipulator  share  the  reward  for  the  new  image?  How  might  a manipulated  photograph  be  traced  back  to  its 
source?  Whereas  a book  or  painting  may  be  seen  as  a whole,  and  the  copying  of  all  or  part  may  be  reasonably 
detectable,  the  easy,  and  most  often  undetectable,  manipulation  of  photographs  and  other  images  (including  clip 
art)  renders  the  ownership  of  this  artwork  difficult,  if  not  impossible,  to  protect  (Mitchell  1994:  51). 
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Ethical  problems  related  to  photographic  manipulation  arise  not  only  in  respect  of  photojournalism,  but 
also  in  using  photographs  as  evidence  in  legal  cases  and  for  other  documentary  purposes  such  as  the  compilation 
of  social  and  cultural  histories.  And  wider  still  are  the  ethical  considerations  which  relate  to  the  rights  of  subjects 
in  photographs,  and  the  visual  persuasion  and  deception  which  is  evident  in  the  advertising  industiy. 


Moral  rights  of  subjects  in  photographs 

Abuse  of  the  moral  rights  of  subjects  in  print  media  photographs,  television  and  film  has  been  identified 
by  Gross  et  al.  (1988)  and  Lester  (1991,  1995  & 1996).  One  case  was  the  use  by  the  large  corporation,  Benetton, 
of  the  harrowing  picture  of  a dying  AIDS  patient  in  an  advertisement  of  its  products  (Lester  1995:  78-79,  98-99). 
Other  examples  of  the  intrusive  ness  and  inappropriateness  of  photographic  reporting  of  tragic  and  often  grisly 
events  are  reported  in  Lester  (1991  & 1995)  and  in  his  1996  text,  in  which  he  has  examined  the  use  of 
photographs  in  perpetuating  the  stereotyping  of  minority  groups. 

The  struggles  for  protection  of  the  privacy  of  personalities  from  the  intrusion  of  media  photographers 
have  been  reported  comprehensively,  but  Powell  (1998:  93)  suggests  the  privacy  of  ordinary  individuals  is  also 
now  under  threat  with  developments  in  satellite  technology  which  fast  outstrip  the  development  of  ethical 
protocols  which  might  guide  its  use.  He  reminds  us  that  in  earlier  days  of  U-2  spy  planes  and  Sputnik,  ethical 
issues  centred  on  military  secrecy  and  national  boundaries,  while  at  the  end  of  the  nineties  it  is  satellite  use,  high- 
powered  lenses,  infra-red  sensory  devices,  instant  high-resolution  image  transmission,  and  the  capacity  for 
global  observation  which  magnify  communication  ethics  issues. 

As  satellite  images  reveal  military  forces,  conventional  warfare  becomes  obsolete,  but  civilian  privacy  has 
changed  dramatically  as  well,  as  ‘backyard  sunbathers,  naturalists,  couples,  speeding  vehicles,  and  naked 
paramours  seen  through  bedroom  windows  can  all  be  identified,  photographed,  and  publicized  without  their 
awareness  or  permission’,  (Powell  ibid.:  94). 

Powell  suggests  that  as  the  capacities  of  space  photography  increases  in  parallel  with  its  potential 
commercialism  (already  vendors  in  many  countries  routinely  sell  space  imagery  to  media  outlets),  then 
communication  ethicists  must  consider,  firstly  in  journalism,  the  decisions  of  editors  and  producers  regarding  the 
publishing  of  invasive  photographs,  and  secondly,  in  new  media  ethics,  who  it  is  who  ‘employs,  duplicates, 
regulates,  as  well  as  who  sells  and  buys  satellite  imagery’,  (Powell  ibid.:  95). 


Visual  Persuasion  in  Advertising 

Ethical  issues  regarding  visual  persuasion  in  advertising  have  been  raised  by  Mitchell  (1994)  and 
Messaris  (1997).  Advertisements  which  use  images  to  make  false  claims  or  to  sell  harmful  products  can  be 
readily  judged  as  unethical,  but  a different  form  of  advertising  uses  images  in  a more  subtly  unethical  manner. 
For  example,  Messaris  (1997:  267-8)  points  to  images  which  focus  on  the  discrepancy  between  an  idealized 
vision  of  life  and  the  needs  and  abilities  of  real  people,  for  advertisements  which  use  sex  or  status  for  their 
appeal  may  create  a source  of  dissatisfaction  in  people’s  lives,  such  as  with  their  body  shape,  thereby  creating  an 
anxiety  which  may  be  linked  to  eating  disorders. 

But,  Messaris  (ibid.:  271)  suggests  ethical  judgements  on  specific  advertisements  may  not  always  be 
clearcut,  and  cites  a 1990  Volvo  TV  commercial  in  which  a monster  truck  flattens  other  cars,  but  fails  to  squash 
a Volvo  240,  in  a scene  staged  with  a car  which  had  been  strengthened.  As  the  advertisement  was  based  on  a 
real-life  incident  in  1988  in  which  a Volvo  had  withstood  a monster  truck’s  weight,  why  was  it  necessary  to 
reinforce  a similar  car  for  filming  the  commercial?  An  industiy  spokesman’s  defence  was  that  the  car  was 
reinforced  because  of  the  need  for  repeated  ‘takes’,  but  Messaris  questions  whether  this  explanation  justified  the 
advertisement’s  staging,  or  whether  the  addition  of  a ‘dramatization’  tag  would  be  sufficient  to  excuse  the 
commericial’s  creators. 

And  graphics-based  advertising  may  allow  sellers  to  express  ideas  they  might  be  unwilling  to  put  into 
words,  lest  the  advertiser’s  claims  be  challenged  legally.  Messaris  (ibid.:  273-4)  cites  an  example  of  the 
promotion  of  nutritional  supplements  for  bodybuilders.  Scientific  evidence  had  shown  that  these  were  no  better 
at  inducing  muscle  growth  than  an  ordinary  balanced  diet,  and,  in  fact  the  commercial  did  not  claim  that  they 
did,  verbally.  Instead  its  message  relied  on  a visual  ‘cause-effect  juxtaposition:  that  is,  on  one  hand,  an  image  of 
the  product;  on  the  other,  an  image  of  a champion  body-builder’.  When  challenged,  a spokesman  argued  the 
company  could  not  be  held  responsible  for  what  individual  viewers  concluded  from  these  images,  however,  the 
intent  here  was  obvious,  that  is,  persuasion  by  inference. 
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Towards  a Protocol  for  Graphics  Ethics 


Wheeler  & Gleason  (1995)  suggest  that,  in  respect  of  the  digital  manipulation  of  photographs,  an  ethical 
protocol  should  be  established  which  would  identity  (by  the  use  of  a system  of  icons)  altered  and  unaltered 
photographs,  thus  enabling  a reader  or  consumer  to  ascertain  whether  or  not  the  graphic  had  been  modified. 
However,  whilst  a system  of  labelling  of  photographs  and  other  artwork  is  commendable,  there  is  need  for  a 
more  encompassing  protocol  for  image  ethics  in  view  of  the  myriad  of  new  communications  technologies 
developed  in  the  late  1 990s,  and  others  which  are  anticipated. 

These  technologies  (which  include  satellites,  the  Internet  and  virtual  reality),  each  bring  with  them  social 
effects  and  ethical  issues  which  arise  before  society  has  begun  to  assimilate  these  rapid  developments  in 
technology.  Documents  and  photographs  can  now  be  altered  from  a distance  without  detection,  and  realistic,  yet 
artificial,  images  can  be  quickly  synthesized.  Theoretically,  as  Cooper  (1998:  74)  suggests,  ‘a  presidential 
candidate  may  now  be  computer-generated,  credentialled  online,  and  elected  without  ever  being  publicly  seen 
(other  than  via  a human  stand-in)’.  In  similar  vein,  Powell  (1994)  predicts  totally  synthetic  news  events. 

When  invasion  of  privacy  occurs  with  old  technologies  such  as  a camera  with  telephoto  lens,  such 
invasion  has  very  different  qualities  than  ‘cyvacy’,  (computer  privacy  invasion)  (Powell  1998:  93).  This  new 
invasion  of  privacy  is  remote,  undetectable  and  impersonal  and  amplifies  existing  problems  such  as  obscenity 
and  indecency  ‘by  making  controversial  and  criminal  (in  many  cultures)  images  available  to  much  larger, 
different,  and  younger  audiences’  (Powell  ibid.:  94). 

A protocol  for  image  ethics  thus  must  encompass  the  issues  outlined  above  and  be  proactive  rather  than 
reactive  to  the  ethical  problems  created  by  the  unique  features  of  the  new  technologies,  which  are  exacerbated  by 
their  rapid  rate  of  invention  and  implementation. 


Image  Ethics  in  a Higher  Education  Multimedia  Program 

In  higher  education  the  teaching  of  ethics  is  most  often  left  to  philosophers  or  to  specially  designated 
courses.  This  is  a missed  opportunity,  for  research  shows  that  integrating  ethics  in  discipline-appropriate  ways 
embellishes  the  study  of  ethics  (Roberts  1994  & 1995;  Lisman  1996).  This  view  supports  the  integration  of 
image  ethics  in  the  multimedia  program  of  a Bachelor  of  Arts  award  in  an  Australian  university. 

The  image  ethics  program  relies  on  the  model  of  ethical  decision-making  developed  by  Rest  & Narvaez 
(1994:  23-24)  which  involves  a continuum  of  four  distinctive  stages,  that  is,  moral  sensitivity,  moral  judgement, 
moral  motivation,  and  moral  character.  This  model  is  particularly  useful  in  the  context  of  the  university’s 
‘Graduate  Qualities’  which  are  a set  of  personal  and  professional  qualities  which  the  university  anticipates  as 
outcomes  for  its  graduates,  and  which  underpin  its  educational  programs.  One  Graduate  Quality  involves  the 
development  of  the  graduate  as  an  ethical  citizen  and  professional. 

The  Image  Ethics  program  commences  in  a foundation  (mandatory)  introductory  computing  subject  in  the 
first  semester  of  the  Bachelor  award.  Its  pedagogy  (described  in  Roberts  1994  & 1995)  aims  to  establish  a moral 
sensitivity  to  a range  of  issues  in  Computer  Ethics,  which  includes,  amongst  others,  an  examination  of  computer 
crime,  ‘hacking’,  privacy,  computers  in  critical  systems,  and  the  use  of  imaging  techniques,  with  the  intent  to 
deceive.  Some  650  students  undertake  this  subject.  For  some  of  these,  this  will  represent  their  only  exposure  to 
considerations  of  the  social  and  ethical  implications  of  computing,  but  as  users  and  citizens,  this  first  encounter 
with  Computer  Ethics  may  provide  them  with  the  opportunity  to  make  better  informed  choices,  and  as 
consumers,  they  are  more  likely  to  influence  the  design  and  direction  of  new  technologies  (Pool  1998). 

For  other  students  who  will  extend  their  studies  to  the  computing  major  in  Multimedia  Studies,  in  subjects 
which  have  as  their  main  focus  visual  arts  and  communication,  these  will  engage  in  the  construction  and 
manipulation  of  graphics.  Each  subject  in  the  major  includes  an  image  ethics  component,  appropriate  to  the 
particular  level  of  study  and  its  technical  content,  and  the  students  are  encouraged  to  develop  moral  Judgement  in 
their  creation  and  use  of  images  in  the  artwork  which  they  design,  in  a manner  similar  to  their  Judgement  in  the 
use  of  text,  with  respect  to  copyright  and  the  intellectual  property  of  others. 

As  students  spend  hours  of  painstaking  work  in  creating  their  own  artwork,  their  natural  concern  for  their 
own  rights  to  claim  intellectual  property  for  their  products  is  a timely  and  practical  reminder  of  their 
responsibility  also  to  protect  the  rights  of  other  artisans,  and  to  encourage  their  peers  to  do  likewise.  It  is 
gratifying  to  see  examples  of  this  growth  in  moral  motivation  which  is  a cornerstone  of  the  moral  character 
required  of  the  computing  professional. 
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In  these  laboratory  sessions,  the  integration  of  discussions  of  ethical  issues  in  image  creation  and 
manipulation  is  combined  with  practical  work  to  produce  a natural  interface  between  ethics  and  computing, 
which  avoids  creating  a dichotomy  of  an  abstract  philosophical  theorizing  of  ethical  principles  in  computing,  and 
the  ‘hands-on’  work  of  the  computing  practitioner. 

The  overall  aim  of  this  ethics  program  then  is  that  students  establish  a personal  protocol  for  image  ethics 
which  they  will  apply  in  their  academic,  personal  and  professional  lives.  This  protocol  may  be  briefly 
summarised  as  follows: 

• An  image  should  be  identified  as  ‘original’  or  ‘altered’  and  labelled  accordingly; 

• An  image  should  not  infringe  copyright  or  another’s  intellectual  property; 

• An  image  should  not  be  used  to  deceive  or  persuade; 

• An  image  should  not  infringe  the  moral  rights  of  its  subject(s)  in  relation  to  privacy  and  stereotyping; 

• An  image  should  not  breach  ethical  and  legal  standards  in  relation  to  obscenity  and  decency; 

• A computing  professional  should  anticipate  that  new  communication  technologies  may  carry  with  them 
unpredictable  and  unwanted  effects  for  society,  be  prepared  to  research  and  debate  these  effects,  and  work 
towards  society’s  control  of  technology; 

• A computing  professional  has  an  individual  responsibility  for  vigilance  and  action  to  defend  the  visual  truth 
of  information. 


Conclusion 

Too  often  ethical  issues  in  computing  are  studied  only  after  serious  problems  have  arisen.  With  the 
benefit  of  hindsight  it  can  now  be  anticipated  that  each  new  technology  will  bring  both  benefit  and  disturbance  to 
society,  and  that  in  order  for  society  to  gain  control  over  technology  and  to  be  able  to  influence  its  effects, 
evaluations  of  technologies  must  include  not  only  the  performance  of  their  technical  properties,  but  their 
potential  effects  on  organizations,  societies  and  individuals. 

Such  is  the  case  in  the  teaching  of  imaging  techniques  to  users  and  future  industry  practitioners  who  will 
work  in  a variety  of  contexts  which  use  graphics  to  enhance  communication.  There  is  a vital  task  here  for  all 
educators  to  prompt  awareness  of  the  scope  of  ethical  issues  which  arise  from  the  use  of  this  technology,  and  the 
need  for  moral  responsibility  in  its  use. 

How  do  we  get  these  budding  professionals  to  look  beyond  the  seductive  power  of  computer  technology 
and  the  aesthetic  appeal  of  working  with  graphics?  The  skills/ethics  interface  is  all  important  in  this  task,  an 
interface  which  requires  that  at  the  same  time  that  students  learn  the  skills  of  image  manipulation,  they  are  also 
made  aware  of  the  ethical  dimensions  of  these  skills. 

Students  are  now  fully  aware  of  the  issues  of  plagiarism  of  text  in  their  academic  work,  and  universities 
have  policies  in  place  which  require  students  to  acknowledge  the  intellectual  property  rights  of  authors,  by 
procedures  for  referencing  quotations  and  ideas  from  original  work.  The  same  respect  for  the  artwork  and 
photographs  of  visual  artisans  has  been  ignored,  and  a similar  system  of  referencing  which  applies  to  a scholarly 
essay  should  be  applied  to  the  production  of  graphics  in  a student’s  multimedia  projects. 

But  the  wider  issue  of  an  ethical  approach  to  the  presentation  of  visual  information  has  great  importance 
for  a society  which  turns  increasingly  to  a reliance  on  pictorial  communication.  For  as  Ritchin  (1991 : 37)  warns: 

As  readers  we  must  remain  vigilant.  Otherwise,  what  as  a society,  and  among  societies,  are 
we  going  to  be  left  with  as  a form  of  communication  that  can  be  trusted?  What  information 
will  people  be  able  to  rely  upon  to  make  decisions?  Or  most  precisely,  what  will  the  role  of 
the  press  in  a democracy  be  worth?  ...  Undoubtedly,  we  will  all  be  told  that  what  has 
happened  is  the  computer’s  fault,  and  we  will  then  be  even  more  isolated  in  our  own  media 
bubbles  than  ever  before. 
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Abstract.  Metadata  and  interoperability  standards  are  essential  to  support  the  educational  and 
commercial  effectiveness  of  learning  technology.  A number  of  related  and  cooperating 
organizations  are  working  to  develop  such  standards.  This  paper  briefly  explains  why  such 
standards  are  important,  who  is  involved  in  their  development,  and  what  standards  are  being 
developed.  The  penultimate  section  discusses  the  standards  process  in  general  and  points  out 
that  learning  technology  standards  are  attempting  to  create  new  infrastructure  rather  than 
codify  or  standardize  existing  practice. 


Introduction 


This  paper  reports  on  activities  in  two  areas:  metadata  and  interoperability  standards.  Its  purpose  is  to 
give  a snapshot  of  the  players  and  recent  developments  in  these  areas  and  to  make  some  general  comments  on 
the  standardization  process.  Its  focus  is  work  being  done  by  the  IEEE  Learning  Technology  Standards 
Committee  (LTSC,  2000)  and  related  efforts. 


Metadata  and  Interoperability  Standards 

Metadata  is  data  about  data.  In  this  paper  metadata  refers  specifically  to  relatively  small  amounts  of 
descriptive  data  that  capture  essential  aspects  of  a learning  resource.  An  example  from  the  print  world  is  the 
information  in  a library  card  catalogue.  The  title,  author,  publisher,  publication  data,  copyright,  subject 
classification,  and  call  number  are  metadata  that  cover  important  aspects  of  a book  or  document,  although 
pedagogical  information  is  largely  missing.  The  most  prominent  examples  of  metadata  standards  in  the 
educational  arena  are  Dublin  Core  metadata  and  the  Learning  Object  Metadata  (LOM)  that  is  being  developed 
by  the  IEEE  learning  technology  standards  committee.  Dublin  Core  metadata  consists  of  fifteen  core  elements 
that  as  yet  do  not  address  educational  issues.  This  paper  will  concentrate  on  developments  in  LOM  and 
standards  related  to  it. 

Interoperability  standards  are  standards  that  permit  different  systems  or  different  parts  of  the  same 
system  to  work  together.  Protocols  such  as  Internet  Protocol  (IP)  and  standards  such  as  Postscript  may  be 
thought  of  as  interoperability  standards,  IP  allows  diverse  systems  to  exchange  information  by  standardizing  its 
encoding  for  transmission.  Postscript  provides  an  abstract  and  device-independent  language  for  describing  two- 
dimensional  print  and  graphics;  documents  described  in  Postscript  can  be  rendered  on  any  device  that  can 
interpret  the  language.  An  analogous  effort  in  learning  technology  is  a standardized  scheme  for  describing  the 
quizzes  and  tests  commonly  used  by  learning  management  systems  and  other  online  testing  software.  A quiz 
described  in  this  fashion  could  be  exported  and  imported  by  any  compliant  systems. 

Metadata  can  in  many  instances  play  the  role  of  interoperability  standards.  The  MIME  type  (Hood, 
1998)  is  a piece  of  metadata  associated  with  data  delivered  over  the  Internet  that  tells  the  recipient  the  general 
nature  of  the  data  and  what  software  is  needed  to  process  it.  As  such,  the  MIME  type  enables  interoperability. 
Descriptions  of  the  subject  matter,  intended  grade  level,  student  learning  outcomes,  and  other  educationally 
relevant  data  associated  with  a learning  resource  also  serve  as  interoperability  standards  since  they  can 
potentially  assist  in  assembling  a collection  of  resources  into  a single  cohesive  unit  such  as  a lesson  or  course. 
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Organizations  Involved  in  Learning  Technology  Standards 

A number  of  organizations  are  involved  in  producing  metadata  and  interoperability  standards  specifically  for 
instructional  technology.  A list  of  the  major  ones  includes: 

• ADL:  The  Advanced  Distributed  Learning  Network  was  formed  in  1997  by  the  United  States 
Department  of  Defense  and  the  White  House  Office  of  Science  and  Technology  Policy  to  enable 
anywhere/anytime  access  to  quality  learning. 

• AICC.  The  Airline  Industry  CBT  Consortium  was  formed  in  1988  to  standardize  hardware  used  for 
training  in  the  airline  industry.  Since  that  time  it  moved  into  the  standardization  of  Learning 
Management  Systems  as  well.  (AICC,  2000) 

• ARIADNE.  The  Alliance  for  Remote  Instructional  and  Authoring  and  Distribution  Netv^^orks  for 
Europe  started  in  January  of  1996  and  focuses  on  the  development  of  tools  and  protocols  that  support 
the  production,  storage,  delivery,  and  reuse  of  curricular  components  used  in  educational  training 
(ARIADNE,  2000). 

• CEN/ISSS  WS-LT.  The  CEN  (European  commission  for  standardization)  Information  Society 
Standardization  System  started  a Learning  Technologies  Workshop  in  March,  1 999,  with  the  goal  of 
supporting  the  learning  technology  market  and  ultimately  the  European  information  society  with 
standards-oriented  services  and  products  (CEN/ISSS,  2000). 

• DUBLIN  CORE:  The  Dublin  Core  metadata  initiative  began  in  May,  1995,  as  a metadata  workshop 
sponsored  by  the  Online  Computer  Library  Center  (OCLC)  and  the  National  Center  for 
Supercomputing  Applications  (NCSA)  in  Dublin,  Ohio  (Weibel  et.  al.,  1995).  Tlie  Dublin  core  has 
produced  a stable  set  of  15  elements  that  support  the  storage  and  retrieval  of  general  online  resources. 
An  education  working  group  was  started  in  August,  1999,  to  investigate  extensions  for  pedagogic 
resources  (Dublin  Core,  2000). 

• GESTALT:  Getting  Educational  Systems  Talking  Across  Leading  Edge  Technologies  is  a European 
Community  project  involving  eight  academic  and  corporate  partners.  Projects  include  metadata  and 
architectures  for  learning  technologies. 

• IEEE  LTSC:  The  Learning  Technology  Standards  Committee  of  the  IEEE  was  formed  in  1996  and  is 
in  the  process  of  developing  standards  for  a variety  of  aspects  of  learning  technology  divided  into  five 
groups:  General,  Learner-related,  Content-related,  Data  and  Metadata,  and  Management  Systems  and 
Applications.  Participation  is  open  to  any  individual  or  organization. 

• IMS:  The  IMS  project  is  a coalition  of  corporate,  academic,  and  government  partners  that  grew  out  of 
the  EduCom  National  Learning  Infrastructure  Initiative  in  1997.  Originally  entitled  the  “Instructional 
Management  System  project,”  IMS  has  the  vision  of  creating  a comprehensive  open  architecture  and 
infrastructure  for  learning  technologies  (IMS,  2000). 

• ISO/IEC  JTC1/SC36:  The  International  Standards  Organization  and  International  Engineering 
Consortium  Joint  Technology  Committee  (ISO/IEC  JTCl)  has  started  a Standards  Committee  (SC36) 
on  Information  Technology  for  Learning,  Education,  and  Training.  The  first  plenary  session  was  held 
in  March  2000  (JTC1/SC36,  2000). 

• PROMETEUS:  The  PROMETEUS  {PROmoting  Multimedia  Access  to  Education  and  Training  in 
European  Society)  initiative  was  started  in  late  1998  to  assert  Europe’s  role  in  learning  technologies 
and  also  to  protect  educational  and  multicultural  values  central  to  the  European  culture.  PROMETEUS 
members  sign  a Memorandum  of  Understanding  committing  to  the  goals  of  PROMETEUS,  among 
which  is  interoperability.  PROMETEUS  supports  several  study  groups  on  metadata  an  interoperability 
standards.  Members  of  PROMETEUS  include  a range  of  corporate  and  academic  institutions. 
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Learning  Object  Metadata 


Learning  object  metadata  is  important  for  the  discovery  and  re-use  of  educational  objects.  One  of  the 
exciting  developments  of  the  last  year  and  a half  is  the  agreement  among  many  groups  developing  learning 
object  metadata  (IMS,  ARIADNE,  GESTALT,  AICC)  that  they  would  use  the  IEEE  LTSC  as  the  vehicle  for 
standardization.  All  four  of  these  organizations  have  actively  participated  in  the  development  of  LOM,  as  the 
■ IEEE  LTSC  metadata  standard  is  called.  Conversely,  IMS  meta-data  Version  1,0  is  based  on  LOM  3.5,  which 
was  the  IEEE  metadata  working  group  document  prior  to  September  1999,  and  similar  statements  apply  to 
metadata  implementations  by  ARIADNE  and  GESTALT, 


The  LOM  Data  Model 


The  approach  being  taken  to  LOM  is  based  on  an  extensible  structured  data  model.  Mathematically, 
this  model  is  a tree.  The  object  being  described  is  the  root.  The  leaves  contain  data.  Every  other  node  is  labeled 
as  an  element.  At  any  node  in  the  tree,  the  path  from  the  root  to  that  node  determines  what  is  being  described 
and  the  entire  sub-tree  descending  from  that  node  is  the  description. 


,:[P;ate] [April;1,20q0]' 

: iPerson] [Author] 


^^^[Llfe  Cycle] [Greate](\^ 

ROOT^^GTHER-ELEMENTS];  [Contribute]/^ 

TH ER  ;E UE men TS].-  [R b le]^ ^[ Judy  Jones] 

FIGURE  I.  Part  of  a LOM  Tree 


For  example,  in  Figure  I the  path  to  the  node  [Contribute]  proceeds  from  the  root  object  through  the 
path  LifeCycle  - Create  - Contribute.  This  means  that  the  node  is  describing  a contribution  to  the  creation  of 
the  object  being  described.  The  sub-tree  emanating  from  Contribute  has  two  elements.  Person  and  Role  and 
terminates  in  leaves  containing  the  data  Judy  Jones  and  Author.  Together  (but  not  separately!)  these  specify 
that  the  contribution  being  described  is  that  of  an  author  named  Judy  Jones. 


LOM  Bindings 

As  it  stands,  LOM  specifies  three  things: 

• A set  of  metadata  elements  and  their  meanings  (semantics) 

• Relationships  among  the  elements  (structure) 

• The  type  and  size  of  data  permissible  in  each  leaf  (data  types) 

This  is  abstract  and  does  not  dictate  how  LOM  should  be  implemented  in  a real  learning  environment 
or  digital  library.  To  do  this,  one  needs  a binding  of  LOM,  meaning  a concrete  data  structure  that  meets  the 
LOM  specifications.  When  existing  projects,  such  as  ARIADNE,  GESTALT,  or  CMU  Online  (CMU,  2000)  use 
learning  object  metadata,  they  must  store,  transmit,  and  receive  the  metadata.  In  doing  so,  they  create  bindings 
of  LOM  in  the  form  of  XML  DTD’s  or  database  designs.  These  bindings  are  not  unique,  which  makes  the 
bindings  themselves  candidates  for  standardization.  A recent  move  on  the  part  of  the  IEEE  LTSC  was  to 
separate  LOM  as  defined  by  the  three  bullet  points  above  from  any  of  its  bindings  and  to  view  bindings  as 
separate  projects. 


Interoperability  Standards 

From  an  interoperability  perspective,  it  is  important  to  have  a standardized  definition  of  the 
components  and  architecture  of  learning  systems,  as  well  as  standardized  interfaces  among  them.  Having  this 
would  address  at  least  three  major  frustrations: 

• Content  developed  for  one  Learning  Management  System  is  not  usable  in  another  except  to  the  extent 
that  raw  text,  HTML,  graphics,  or  applets  can  be  extracted  and  uploaded. 

• Large  consumers  of  training  and  educational  materials  (e.g.,  the  Department  of  Defense)  are  limited  in 
their  ability  to  specify  compliance  for  procurement  purposes. 

• If  content  can  be  developed  once  and  used  many  times,  its  value  is  greatly  enhanced.  This  increases  the 
incentive  to  develop  educationally  effective  and  professional  quality  learning  materials  and  is  a pre- 
condition for  a thriving  market  - a precondition  that  has  been  lacking  and  that  has  contributed  to  the 
reluctance  of  content  producers  to  commit  large  development  budgets  to  online  educational  materials. 

Work  on  interoperability  standards  is  being  done  by  the  IEEE  LTSC,  the  IMS  project,  ADL,  the  AICC, 
and  various  European  groups.  Many  of  the  people  and  organizations  involved  are  active  in  two  or  more 
organizations,.  Each  group  is  really  part  of  a larger  community  effort  but  has  its  own  emphasis.  The  AICC  is 
focused  on  a single  industry.  The  ADL  is  focused  on  broad  applications  and  early  on  designated  the  IMS 
project  as  the  means  through  which  at  least  metadata  specifications  would  be  developed.  The  ADL  has  also  set 
up  a “CO-lab”  program  and  is  working  closely  with  corporate  and  academic  institutions.  The  IEEE  LTSC  is 
working  towards  official  standardization,  which  is  a difficult  and  often  political  process  about  which  more  will 
be  said  later.  The  ISO/IEC  SC36  that  has  just  started  will  also  work  towards  standardization,  while  the  various 
ARIADNE  is  focused  on  creating  and  maintaining  a knowledge  pool  of  learner  resources,  PROMETEUS  is 
concerned  with  European  social  and  political  issues,  GESTALT  has  developed  a working  implementation,  and 
CEN/ISSS  is  concerned  with  standardization  and  dissemination  in  Europe. 


Specific  Interoperability  Projects 

Among  the  many  pieces  that  comprise  interoperability,  the  potentially  easiest  to  handle  is  that  for 
questions  and  tests.  The  IMS  project  has  released  a first  version  of  a Question  and  Test  Interoperability 
structure,  together  with  and  XML  binding.  The  AICC  has  done  a LTSC  work  on  Computer  Managed 
Instruction  as  well  as  SCORM  — Shareable  Courseware  Object  Reference  Model  - released  on  January  31, 
2000,  by  the  ADL.  The  people  responsible  for  SCORM  are  from  the  DOD,  the  AICC,  the  IMS,  and  the  IEEE 
LTSC  Computer  Managed  Instruction  working  group.  The  press  release  (SCORM,  2000)  indicates  that  the 
University  of  Wisconsin  System,  Wisconsin  Technical  College  System  and  Carnegie  Mellon  University  are 
academic  partners. 

Another  area  in  which  progress  is  being  made  is  that  of  learner  profiling.  This  includes  not  only 
personal  information  about  the  user  of  a learning  technology  but  might  also  reference  competencies  that  a 
learner  has  demonstrated  or  that  accrue  to  the  learner  as  the  result  of  a particular  experience.  Learner  profiling 
is  educationally  important  in  that  it  is  necessary  to  implement  any  sort  of  situated  learning.  In  other  words,  the 
background,  context,  and  goals  of  the  learner  can  and  should  influence  the  choice  and  method  of  learning 
experience.  Learner  profiles  also  include  electronic  transcripts  (Roberts  & Robson,  1997).  But  learner  profiling 
touches  on  numerous  institutions  external  to  technology  and  education  - such  as  privacy  protection  and  legal 
access  to  information.  Identifying  the  learner  in  a typical  academic  or  corporate  training  setting  also  begs  for 
interoperability  with  information  systems  that  handle  student/employee  data  and  records. 


Schools  Interoperability  Framework 

The  combination  of  these  external  influences  makes  the  standardization  of  learner  profiling  a high 
stakes  and  non-trivial  matter.  To  underscore  this,  Microsoft  Corporation  started  an  initiative  called  the  Schools 
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Interoperability  Framework  (SIF)  that  has  been  taken  over  by  the  Software  and  Information  Industry 
Association  (SIIA,  2000)  and  is  being  developed  there  as  an  open  standard.  The  SIF  Web  Page  (SIF,  2000) 
explains  SIF  as 

The  Schools  Interoperability  Frameworic  (SIF)  is  an  industry  initiative  to  develop  an  open  specification 
for  ensuring  that  K-12  instructional  and  administrative  software  applications  woiic  together  more 
effectively.  SIF  is  not  a product,  but  rather  an  industry-supported  technical  blueprint  for  K-12  software 
that  will  enable  diverse  applications  to  interact  and  share  data  seamlessly;  now  and  in  the  future. 

The  version  1.0  specification  was  released  at  the  end  of  March  2000.  Although  SIF  is  intended  for  K- 
12  use  only,  it  should  be  observed  that  it  is  in  some  sense  in  competition  with  the  work  being  done  by  IMS,  of 
which  Microsoft  is  a member.  SIF  is  an  XML  application  but  is  educationally  very  conservative  and  primarily 
codifies  an  American  K-12  system  based  on  letter  grades  and  seat  time,  not  content  standards  or  outcomes.  SIF 
also  goes  beyond  the  work  of  being  discussed  in  this  paper  since  it  includes  specifications  for  the  food  service, 
HR/financials,  and  transportation. 


Comments  on  Standardization 

Until  now  an  emphasis  has  been  placed  on  what  is  to  be  gained  by  standardization.  However,  there  are 
some  negative  sides  as  well. 

The  industrial  standardization  process  has  traditionally  been  one  of  fossilization  whereby  established 
(and  possibly  competing)  industry  practices  were  made  uniform.  As  the  pace  of  change  in  technology  has 
picked  up,  the  time  lag  between  the  establishment  and  the  institutionalization  of  practice  through  the  standards 
process  has  perhaps  lessened,  but  the  standardization  process  has  remained  driven  by  that  which  exists  as 
opposed  to  that  which  is  envisioned.  The  process  can  be  highly  charged  since  choices  made  by  standards 
bodies  can  pick  winners  and  losers  and  affect  billions  of  dollars  worth  of  commerce,  but  having  standards  is 
also  a condition  for  accelerated  growth  of  an  industry  sector  (Shapiro  & Varian,  1 998). 

In  the  educational  arena,  content  standards  have  traditional  been  instruments  of  social  change.  This 
was  true  of  Carnegie  units  and  standardized  tests  and  remains  true  of  the  current  educational  reform  movement. 
Educational  standards  are  typically  developed  in  response  to  failures  of  existing  institutions  and  in  the  hopes 
that  the  new  institutions  introduced  by  the  standards  will  be  meet  with  more  success.  Thus  it  may  be  said  that 
educational  and  technology  standards  live  on  opposite  sides  of  the  line  between  fossilization  and  innovation. 

Learning  technology  standards  have  purposefully  crossed  over  the  line  from  fossilization  to 
innovation.  The  goal  of  the  IMS  project,  and  indeed  its  parent,  the  National  Learning  Infrastructure  Initiative, 
was  to  develop  non-existent  infrastructure  that  its  architects  envisioned  as  being  needed  in  the  near  future.  The 
same  applies  to  ADL.  Although  the  IEEE  LTSC  is  more  constrained,  its  standards  also  have  an  anticipatory 
quality.  This  choice  of  modus  operandi  was  made  quite  intentionally  at  the  beginning  of  the  work 

Learning  technology  standards  are  consequently  a bold  experiment  in  creating  standards.  The  World 
Wide  Web  Consortium  (W3C,  2000)  has  done  much  the  same  thing  with  the  development  of  its 
recommendations  (the  equivalent  of  standards),  but  W3C  recommendations  operate  for  the  most  part  on  the 
level  of  communication  protocols  as  opposed  to  applications.  A clear  danger  with  developing  anticipatory 
standards  for  learning  technology  is  that  premature  standardization  can  cut  off  desirable  future  development 
paths.  If,  for  example,  the  model  of  learning  systems  being  promoted  by  SCORM  dictates  industry  practice, 
then  new  models  will  be  harder  to  develop.  Since  pedagogic  innovation  is  slower  paced  and  much  less  of  a 
science  than  technological  innovation,  premature  standardization  could  lead  to  the  tragic  result  that  truly  better 
ways  to  teach  and  learn  are  nipped  in  the  bud  because  they  were  not  anticipated. 

Another  difficulty  with  the  approach  being  taken  by  the  standards  community  in  learning  technology  is 
one  illustrated  by  the  emergence  of  SIF  and  the  exaggerated  claims  made  by  academically  oriented  Learning 
Management  System  companies.  In  practice,  formal  standards  do  not  carry  as  much  weight  as  de  facto 
standards,  and  the  information  industry  is  not  in  the  habit  of  waiting  for  formal  standards.  However,  for  sales 
and  marketing  purposes  it  is  often  convenient,  if  not  necessary,  to  claim  conformance  to  formal  standards.  The 
learning  management  system  vendors  have  been  claiming  IMS  compliance  for  well  over  a year,  despite  the  fact 
that  there  is  no  clear  definition  of  what  this  compliance  means.  If  this  situation  persists,  it  could  devalue  all 
formal  learning  technology  standards  and  leave  the  industry  with  a set  of  competing  de  facto  standards.  The 
result  would  be  a lack  of  interoperability  and  a lot  of  finger  pointing. 
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Conclusion 


Only  time  will  adjudicate  the  success  of  learning  technology  standards.  It  seems  clear  that  they  are 
badly  needed  and  that  there  is  a hard-working  community  making  good  progress  on  their  development.  Anyone 
interested  in  developing  learning  technologies,  especially  online  and  networked  learning  technologies,  should 
pay  close  attention  and,  if  possible,  participate  in  this  process.  This  paper  lists  a number  of  references  to 
relevant  organizations  and  is  meant  to  be  a starting  point  for  this  type  of  involvement. 

It  remains  to  be  seen  whether  the  attempt  to  use  the  standardization  process  as  a means  to  anticipate 
and  jump-start  the  learning  technology  industry  will  be  successful.  It  could  spur  growth  yet  not  find  acceptance 
for  the  standards  currently  being  produced  and  thus  succeed  by  failing.  But  given  the  participation  of  the  major 
technology  companies,  government  agencies,  and  academic  institutions,  there  is  a good  chance  that  the 
combination  of  the  partners  listed  earlier  will  succeed  in  creating  accepted  standards  and  that  the  ultimate 
question  will  be  how  to  adjust  and  revise  these  in  response  to  innovative  products  and  pedagogy. 


References 

AlCC.  (2000).  Aviation  industry  CBT committee.  <WWW  Document>  http://www.aicc.org/.  Accessed  April  1 , 2000. 

ARIADNE.  (2000).  Alliance  of  Remote  Instructional  and  Distribution  Networks  for  Europe.  <WWW  Document> 
http://ariadne.unil.ch/.  Accessed  April  1,  2000. 

CEN/ISSS.  (2000).  CEN/ISSS  Learning  Technologies  Workshop.  <WWW  Document>. 
http://www.cenorm.be/isss/Workshop/lt/meetinqs.htm.  Retrieved  April  1 , 2000. 

CMU  (2000).  CMU  Online.  <WWW  Document>.  http://online.web.cmu.edu/.  Retrieved  April  2,  2000. 

Dublin  Core  (2000).  The  Dublin  Core  Metadata  Initiative.  <WWW  Document>  http://purl.org/dc/.  Accessed  April  1 , 2000 

GESTALT.  (2000).  Getting  Educational  Systems  Talking  Across  Leading  Edge  Technologies.  <WWW  Document>. 
http://www.fdqroup.co.uk/qestalt/about.html.  Accessed  April  1 , 2000. 

Graves,  W.  (1994).  Towards  a National  Learning  Infrastructure.  Educom  Review,  29  (2),  1994.  <WWW  Document> 
http://www.educause.edu/pub/er/review/reviewArticles/29232.html.  Accessed  April  1,  2000. 

Hood,  E.  (1998,  July  1 1).  Multipurpose  Internet  Mail  Extensions.  <WWW  Document> 
http://www.oac.uci.edu/indiv/ehood/MIME/MlME.html.  Accessed  April  1 , 2000. 

IMS  (2000).  The  IMS  Project.  <WWW  Document>  http://www.imsprolect.orq/.  Accessed  April  1 , 2000. 

JTC1/SC36  (2000).  Joint  Technology  Committee  Standards  Committee  36.  <WWW  Document>.  http://itc1  sc36.orq/. 
Accessed  April  1 , 2000. 

PROMOTEUS  (2000).  Promoting  Multimedia  access  to  Education  and  Training  in  European  Society.  <WWW  Document>. 
http://prometeus.org/.  Retrieved  April  1 , 2000. 

Roberts  R.  and  Robson,  R.  (1997).  Electronic  Transcripts  in  the  Age  of  Standards.  Pages  729-731  in  S.  Lobadzinski  and  I. 
Tomek,  eds.,  WebNct97:  World  Conference  of  the  World  Wide  Web  Internet  and  Intranet,  Associatioii  for  the  Advancement 
of  Computing  in  Education,  Charlottesville,  VA. 

SCORM  (2000,  Feb  2).  The  Department  of  Defense  Advanced  Distributed  Learning  - ADL  - Initiative  Releases  Version 
1.0  of  the  Sharable  Courseware  Object  Reference  Model  — SCORM  1.0  — <WWW  Document>. 
http://biz.vahQQ.cQm/bw/000202/dc  dept  of  1.html.  Retrieved  April  2,  2000. 

Shapiro,  C.  & Varian,  H.  (1998).  Information  Rules:  A Strategic  Guide  to  the  Network  Economy,  Harvard  Business  School 
Press. 

Weibel,  S.,  Godby,  J.,  Miller,  E.,  Daniel,  R.  (1995).  OCLC/NCSA  Metadata  Workshop  Report,  <WWW  Document> 
http://purl.Qrq/d^workshQPs/dc1  conference/re port.htm.  Accessed  April  1 , 2000. 


BEST  COPY  AVAILABLE 


Instructional  Design  Agents  - An  Integration  of  Artificial  Intelligence  and 

Educational  Technology 


Erika  Rogers 

California  Polytechnic  State  University 
Dept,  of  Computer  Science 
San  Luis  Obispo,  CA 
erogers@csc.calpoly.edu 

Carol  Scheftic 

California  Polytechnic  State  University 
University  Center  for  Teacher  Education 
San  Luis  Obispo,  CA 
schefl:ic@calpoly.edu 

Emilio  Passi  and  Sharon  Lanaghan 
California  Polytechnic  State  University 
Dept,  of  Computer  Science 
San  Luis  Obispo,  CA 
{epassi,slanagha}@falcon.csc. calpoly.edu 


Abstract:  The  purpose  of  this  paper  is  to  introduce  a project  whose  goal  is  to  design  and  develop  a 
productivity  tool  which  helps  academic  instructors  in  their  course  preparation.  This  tool  will  be 
composed  of  a number  of  “instructional  design  agents”,  which  combine  techniques  in  human- 
computer  interaction,  artificial  intelligence  and  educational  theory.  The  first  of  these  agents  is 
based  on  Bloom’s  Taxonomy,  and  a brief  overview  of  this  work  in  progress  is  presented. 


Introduction 

In  today’s  rapidly  changing  academic  environment,  the  role  of  university  professors  is  also  increasing  in  complexity. 
While  the  three  core  areas  of  faculty  evaluation  are  still  typically  teaching,  service  and  professional 
development/research,  in  reality,  there  are  many  additional  responsibilities  that  are  embedded  in  these  core  areas. 

For  example,  faculty  are  not  only  responsible  for  the  more  traditional  tasks  of  preparation  and  delivery  of  course 
materials,  but  they  are  also  expected  to  devise  and  choreograph  appropriate  "active  learning"  activities,  to 
incorporate  new  technologies  such  as  multimedia  and  the  Internet,  and  to  participate  in  seemingly  endless  cycles  of 
self-  and  program  assessment  and  evaluation,  to  name  just  a few.  In  addition  to  the  conduct  of  their  own  professional 
lives,  it  appears  that  faculty  are  also  to  be  held  accountable  for  students'  learning,  even  though  increasing  class  sizes 
and  shrinking  resources  make  it  harder  and  harder  to  monitor  and  adjust  in  such  an  individualized  manner. 

It  is  becoming  more  and  more  difficult  to  lead  a "scholarly"  life,  particularly  at  institutions  where  teaching  is  the 
primary  activity.  In  fact,  even  there,  it  is  almost  impossible  to  find  the  time  to  devote  to  thorough  a priori  course 
design,  development  and  improvement.  Traditionally,  university  faculty  are  trained  to  be  highly  skilled  content 
experts  in  the  process  of  obtaining  a Ph.D.  Training  for  job  duties  in  the  classroom,  however,  has  been  typically  left 
to  chance  (e.g.,  receiving  training  as  a teaching  assistant),  and/or  the  instructor's  own  initiative.  Even  with  some 
amount  of  training  or  experience,  most  university  faculty  are  not  necessarily  fluent  with  either  a process  for 
instructional  design  or  the  many  education-related  theories  of  learning  that  describe  the  impact  of  the  instruction  or 
teaching  style  on  the  students’  learning  experiences. 

Furthermore,  in  this  age  of  technology,  there  is  a surprising  paucity  of  effective  tools  available  to  the  instructor  to 
help  bridge  this  gap  in  knowledge.  Today's  educational  "productivity"  tools  in  the  academic  environment  center 
around  web  and  slide  authoring  tools,  generation  of  automated  tests  and  communication-enabling  technologies  (e.g.. 
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email,  electronic  forums,  video-conferencing,  etc.)  (The  equivalent  of  this  ten  years  ago  might  have  been  to  say  that 
the  typewriter  or  a word -processor  and  a thermal  transparency  machine  were  "productivity"  tools.)  If  faculty  are  to 
keep  abreast  of  the  changing  face  of  education,  it  is  our  contention  that  they  should  also  then  be  provided  with  a 
suite  of  tools,  preferably  on  their  desktop,  which  will  assist  them  in  critical  tasks  such  as:  formulation  and 
organization  of  course  content,  design  of  classroom  activities,  and  delivery  of  course  artifacts. 

Our  research  group  is  therefore  currently  working  on  the  design  and  development  of  a set  of  such  tools  called 
"Instructional  Design  Agents"  (IDA),  and  this  paper  describes  the  work  in  progress  on  this  project.  It  is  an 
interdisciplinary  effort  that  aims  to  produce  intelligent  interactive  support  agents  for  instructional  design  in  an 
academic  environment.  Our  approach  is  to  apply  practical  and  theoretical  aspects  of  human-computer  interaction, 
artificial  intelligence,  cognitive  science,  educational  psychology  and  educational  technology  to  the  problem  of 
helping  faculty  become  more  effective  and  efficient  in  their  roles  as  course  designers,  developers  and  managers.  We 
have  adopted  a user-centered  approach  to  this  project,  and  have  identified  four  initial  categories  of  activity:  1 ) the 
conduct  of  detailed  interviews  with  course  instructors,  and  observations  of  them  in  the  processes  of  both  designing 
and  delivering  courses,  2)  in-depth  cognitive,  task  and  content  analyses  of  a specific  set  of  courses,  3)  rapid 
prototype  development  of  a number  of  small,  dedicated  stand-alone  agents,  each  of  which  focuses  on  some  aspect  of 
instructional  design  theory  (e.g..  Bloom's  Taxonomy,  Learning  Styles  and  Multiple  Intelligences,  to  name  a few), 
and  4)  deployment  of  these  agents  in  the  academic  community  for  feedback  and  iterative  improvement. 

Our  next  goal  is  then  to  combine  these  individual  agents  into  an  integrated  intelligent  desktop  agent  which  will 
interact  with  an  instructor  to  assist  in  development  of  a course  from  the  initial  outline,  through  the  analysis  and 
organization  of  the  content,  to  the  production  of  educational  artifacts  for  each  class  session  (i.e.,  lecture  notes, 
interactive  class  activities,  multimedia  presentations,  web-based  documents  and  animations,  etc.).  With  such  an 
agent  to  provide  basic  support,  particularly  in  the  early  time-consuming  content  development  and  structuring  phases, 
the  instructor  would  then  be  free  to  address  the  more  creative  aspects  of  course  delivery,  and  to  devote  more  time  to 
individual  student  interaction.  Our  emphasis  at  this  time  is  to  provide  collaborative  assistance  for  the  instructor, 
rather  than  the  automatic  generation  of  "ideal"  course  outlines. 

A longer-term  goal  is  to  incorporate  a learning  component  in  which  the  agent  obtains  feedback  on  the  outcomes  of 
its  recommendations,  and  then  provides  improved  suggestions  the  next  time.  If  multiple  instructors  are  using  them, 
the  agents  could  even  learn  from  each  other,  comparing  notes,  and  sharing  successes  and  failures  with  their  human 
counterparts. 


Related  Work 

If  we  turn  to  the  field  of  Instructional  Systems  Design  (ISD),  we  find  a domain  in  which  many  of  these  activities 
have  already  been  formalized  into  models.  They  typically  consist  of  five  basic  processes:  analysis,  design, 
development,  implementation  and  evaluation,  which  are  then  configured  in  a variety  of  ways,  (see  Gagne  et  al.  1992 
or  Seels  & Glasgow  1998).  In  addition  to  the  large  sector  of  professionals  who  have  made  a career  out  of 
implementing  instructional  design,  a number  of  high-powered  computer-based  products  are  also  available  in  the 
marketplace.  However,  the  current  lack  of  such  facilities  (either  human  or  computer-based)  in  the  daily  professional 
lives  of  most  academic  faculty  suggests  at  least  two  major  obstacles:  1 ) access  to  formal  instructional  design 
software  packages  and/or  ISD  professionals  is  very  expensive  — certainly  beyond  the  reach  of  most  academic 
faculty;  (ironically,  this  problem  is  exacerbated  at  institutions  whose  primary  mission  is  teaching,  where  funds  are 
usually  limited  and  there  is  little  opportunity  to  invest  in  such  high  cost  "facilities"  for  faculty);  2)  it  appears  that  the 
most  successful  work  in  the  ISD  field  has  focused  on  development  of  industry-style  training  courses,  rather  than 
academic-style  teaching/1  earning  courses.  Although  many  researchers  in  the  field  of  ISD  also  mention  academic 
content  in  their  descriptions  of  ISD  models,  it  is  often  as  an  aside,  and  ifs  not  clear  that  these  systems-oriented 
process  models  really  fit  the  goals,  objectives  and  desired  outcomes  of  academic  course  delivery.  Seels  and  Glasgow 
discuss  this  in  relation  to  public  school  implementation: 

"Although  ISD  has  been  adopted  by  many  corporations,  the  military,  and  many  government  agencies,  the 
impact  of  ISD  in  public  schools  has  been  low.  Some  of  the  reasons  for  this  low  impact  are  that  (a)  the 
responsibility  and  authority  over  instruction  are  vested  in  the  teacher's  direct  contact  with  students,  rather 
than  in  the  development  of  systematically  designed  materials;  (b)  the  rigidity  of  the  daily  schedules  are 
defined  in  terms  of  hours  spent  in  a classroom  with  a teacher,  rather  than  the  learning  outcomes  achieved. 
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and  (c)  the  amount  of  discretionary  funding  available  to  support  systematically  designed  technology-based 
instruction  is  low  (Seels  & Glasgow  1998).” 

Although  the  above  quotation  only  mentions  public  schools,  it  seems  that  ISD  impact  is  also  poorly  represented  in 
the  higher  education  environment,  possibly  for  similar  reasons.  This  suggests  that  perhaps  these  techniques  and 
models  do  not  fit  the  realities  of  the  academic  world,  although  there  are  some  who  would  say  that  the  academic 
world  should  be  changed  to  be  more  in  line  with  the  ISD  kinds  of  approaches. 

In  industrial  training  settings,  where  learning  objectives  frequently  correspond  to  concrete,  well-defined  job-related 
skills,  computerized  systems,  which  implement  the  linear  configurations  of  the  majority  of  instructional  design 
models,  may  be  extremely  effective.  However,  it  is  our  hypothesis  that  in  the  more  traditional  academic  setting  of 
higher  education,  activities  such  as  course  design  may  be  modeled  more  appropriately  as  opportunistic  problem 
solving,  similar  to  other  complex  domains  such  as  architectural  design  or  medical  diagnosis.  That  is,  the  problem  is 
too  complex,  uncertain  and  unspecified  to  be  easily  tackled  in  a procedural  or  algorithmic  fashion.  Furthermore, 
there  is  no  one  necessarily  "right"  answer  (i.e.,  a perfect  complete  course  specification)  — a partial  or  less  than 
optimal  solution  may  be  the  best  we  can  do,  given  the  various  constraints  which  impact  not  only  the  design  process, 
but  the  final  delivery  as  well.  Rather,  the  instructor  thinks  about  different  aspects  of  the  problem  almost  in  parallel, 
and  "islands  of  partial  solutions"  develop  and  compete  with  each  other  for  the  person's  cognitive  attention  until 
eventually  there  is  a sufficient  amount  of  structure  from  which  to  go  forward.  The  instructional  design  model  most 
closely  related  to  this  approach  is  that  of  (Kemp  et  al.  1998),  which  has  nine  design  elements  that  can  be 
approached  by  different  paths,  and  which  are  surrounded  by  two  outer  ovals,  which  indicate  revisions  occurring 
throughout  the  process  (similar  to  that  of  user-centered  design  in  the  human-computer  interaction  community). 


Bloom^s  Taxonomy  Agent 

The  first  Instructional  Design  Agent  under  development  in  our  project  is  based  primarily  on  Bloom’s  Taxonomy, 
and  we  are  using  this  as  a basis  to  explore  a number  of  design  issues: 

• how  can  theories  related  to  instructional  design  be  used  proactively  by  instructors  to  articulate,  structure,  design 
and  implement  academic  courses? 

• what  kind  of  knowledge  is  needed  by  the  agent  not  only  to  capture  the  concepts  of  the  theory  but  also  to 
understand  and  respond  to  the  problem-solving  behaviors  of  the  instructor  in  the  process  of  design? 

• what  kinds  of  data/knowledge  representations  are  needed  to  structure  the  relationships  between  course  content, 
theoretical  constructs  and  classroom  artifacts? 

• what  kinds  of  information  visualization  techniques  can  be  employed  most  effectively  to  allow  instructors  to:  a) 
navigate  their  design  problem  space,  b)  to  detect  patterns  in  the  relationships  of  concepts,  and  c)  engage  in 
"what  if  scenarios,  where  the  effects  of  changes  in  certain  parameters  (e.g.,  order  of  course  material)  on  other 
course  components  are  immediately  visible? 

• can  we  effect  a noticeable  shift  of  cognitive  load  so  that  the  instructor  can  operate  primarily  as  a content  expert, 
and  have  the  agent  provide  the  infrastructure  and  expertise  which  a human  instructional  design  assistant  might 
otherwise  afford? 

The  Taxonomy  of  Educational  Objectives  was  developed  in  the  1950s  by  a team  of  educational  psychologists, 
headed  by  Benjamin  Bloom,  as  a tool  for  classifying  educational  goals  and  outcomes  (Bloom  et  al.  1956).  Of  the 
three  domains  of  behavior  described  in  the  original  work  , the  cognitive  domain  is  the  one  most  associated  with 
educational  outcomes,  and  it  is  described  as  a hierarchy  of  six  major  components,  each  increasing  in  complexity: 
knowledge,  comprehension,  application,  analysis,  synthesis  and  evaluation.  From  the  literature  it  appears  that  use  of 
this  particular  taxonomy  as  a context  for  designing  instruction  has  focused  primarily  on  classifying  goals,  objectives 
and/or  test  items  once  they  have  been  devised  (see  Jonassen  et  al.  1 989).  This  suggests  that  the  instructor  has 
somehow  been  able  to  break  down  the  components  of  the  course  into  atomic  pieces,  which  then  can  be  labeled. 
However,  it  is  our  contention  that  such  detailed  a priori  analysis  of  course  content  and  delivery  strategies  is  rarely 
accomplished,  at  least  in  the  first  few  years  of  teaching  a course.  This  suggests  that  devising  the  course  outcomes, 
goals  and  objectives  themselves  (i.e.,  developing  the  mental  model  of  the  course)  is  a challenging  task,  and  that 
perhaps  the  intelligent  assistance  agent  should  first  aim  to  help  the  instructor  articulate  these  preliminary  building 
blocks.  In  order  to  accomplish  this,  our  agent  embodies  Bloom's  Taxonomy  as  an  interactive  design  tool,  which 
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should  help  the  instructor  formulate  an  effective  mental  model  of  the  course  concepts,  rather  than  just  as  a diagnostic 
tool  for  a pre-existing  model. 


Initial  System  Design 

In  order  to  further  constrain  the  problem,  we  have  selected  the  freshman  computer  science  course,  CSC  101,  as  the 
focus  for  our  first  prototype  system.  A preliminary  set  of  interviews  conducted  with  two  groups  of  experts 
(educational  theory  experts  and  instructors  of  CSC  101),  provided  the  foundation  for  our  initial  design  decisions: 

• course  objectives,  content  and  outcomes  are  the  three  main  components  to  be  addressed,  and  appropriate 
mappings  among  these  three  kinds  of  elements  must  be  defined; 

• Bloom’s  taxonomy  provides  one  kind  of  hierarchical  approach  to  the  expression  of  the  course  objectives  and/or 
the  outcomes  - therefore  the  tool  should  provide  assistance  in  articulating  the  objectives/outcomes  as  well  as  in 
authoring  the  content  in  the  context  of  those  objectives/outcomes; 

• the  course  design  process  should  not  be  overly  constrained  by  the  taxonomy  itself  - this  suggests  that  the  agent 
should  be  functional  in  at  least  two  modes:  1)  explicit  taxonomy,  where  the  instructor  wishes  to  do  the  course 
development  in  the  context  of  the  formal  objectives  and  outcomes,  or  2)  transparent  design,  where  the  focus  is 
on  the  content  development,  and  the  agent  keeps  track  of  objectives  and  outcomes  “behind  the  scenes”. 

With  these  ideas  in  mind,  a metaphor  of  a “course  design  space”  has  been  developed,  which  we  define  as  the 
environment  that  contains  all  the  resources  an  instructor  needs  to  design  a course.  This  environment  includes 
electronic  resources  such  as  our  software  agent,  content  authoring  tools,  and  the  World  Wide  Web,  as  well  as 
physical  artifacts  such  as  textbooks,  videos,  demonstration  equipment,  etc.  The  user  interface  of  our  agent  reflects 
this  notion  of  course  design  space  by  supporting  asynchronous  problem  solving.  Unlike  the  physical  artifacts  of  the 
course  design  space,  the  agent  is  dynamic,  in  that  it  demonstrates  how  a change  in  one  aspect  of  the  design  is 
reflected  in  other  relevant  components. 

The  presentation  of  this  work  in  progress  will  include  an  overview  of  the  design  of  the  Bloom's  Taxonomy  agent,  a 
demonstration  from  our  first  prototype,  and  preliminary  results  from  our  first  round  of  functionality  and  usability 
testing. 
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Abstract:  This  article  examines  Solver  Learning  Environment,  methods  and  main  results  of  the  study 
dealing  with  mathematical  abilities  and  development  of  mathematical  word  problem-solving  skills  in  a 
technology-based  learning  environment.  To  collect  data  a teaching  experiment  was  arranged  at  Kaukajarvi 
School  in  the  city  of  Tampere,  Finland.  The  participants  of  the  study  were  fifteen  years  old.  In  the  study 
empirical  group  comparison  methods  were  used.  During  the  teaching  experiment  the  group  A studied 
mathematical  word  problem-solving  in  the  technology-based  Solver  Learning  Environment.  The  group  B 
studied  the  same  material  with  conventional  teaching  methods,  using  paper  and  pencil  only.  A control  group 
did  not  take  part  in  the  problem-solving  lessons,  they  just  participated  in  the  tests.  Several  tests  were 
arranged  to  measure  pupils’  mathematical  word  problem-solving  skills  and  to  evaluate  their  mathematical 
abilities.  In  this  paper  we  present  the  study  design  and  main  results  of  the  study,  too. 


1 Introduction 

The  aim  of  this  study  was  to  examine  the  connection  between  different  learning  environments  with  the 
development  of  problem-solving  skills  of  pupils  on  different  levels  of  mathematical  abilities.  In  this  study  we 
compare  the  development  of  problem-solving  skills  achieved  in  a technology-based  Solver  Learning 
Environment  with  learning  environment  of  conventional  paper  and  pencil  tasks.  During  the  teaching 
experiment  we  collected  data  on  Solver’s  suitability  to  develop  pupils’  problem-solving  skills  on  different  levels 
of  mathematical  abilities.  (See  Ruokamo-Saari  1996a,  1996b,  1997.) 

This  study  was  an  independent  part  of  the  Schools  to  Information  Superhighway  (SISU)  Project  (see 
Koskelainen  et  al.  1996a,  1996b).  A group  of  mathematics  teachers  at  Kaukajarvi  School  prepared 
mathematical  word  problems  and  their  model  solutions  for  Solver  Learning  Environment  in  co-operation  with 
the  author.  Solver  was  produced  as  a part  of  the  Naasnetti  Project  of  Multimedia  Services  of  Alexpress  Ltd.  of 
Aamulehti  Group  Ltd.  The  Naasnetti  Network  Multimedia  Project  was  part  of  The  Finnish  Multimedia 
Programme  (FMP).  Later  on  this  study  became  a part  of  Distance  Learning  in  Multimedia-Networks  Project 
(<URL:  http://matwww.ee.tiit.fl/kamu/>)  of  the  Finnish  Multimedia  Programme  (FMP)  (see  Pohjolainen  & 
Ruokamo-Saari  1997;  Pohjolainen  & Ruokamo  1998,  1999;  Ruokamo  & Pohjolainen  1999). 

This  article  describes  the  Solver  Learning  Environment,  design  and  main  results  of  the  study,  too.  Participants 
of  this  study  were  ninth  grade  pupils  (N=66).  The  contents  of  Solver  Learning  Environment  were  designed 
taking  into  account  contextual  approach  and  transfer:  word  problems  were  connected  to  real  life  situations  and 
thus  to  the  real  world.  Problem-solving  was  done  collaboratively  and  the  view  of  learning  was  "moderate” 
constructivistic  (Reusser  1991a).  Solver  was  designed  to  support  pupil’s  own  activity  and  reflection,  (cf 
Ruokamo-Saari  1996a,  1996b,  1997.)  In  this  paper  we  present  the  main  results  achieved  via  the  teaching 
experiment  using  quantitative  research  methods. 


2 Solver  Learning  Environment  for  Solving  Mathematical  Word-Problems 

Solver  is  a technology-based  learning  environment  that  includes  different  presentation  forms  of  hypermedia 
(e.g,  hypertext,  graphics,  animation,  sound,  video)  and  network  properties,  social  interaction,  physical  learning 
activities  and  mental  thinking  processes  in  learning  situation,  (See  Ruokamo-Saari  1996a,  b;  Ruokamo  & 
Pohjolainen  1998.) 

Solver  was  used  to  study  the  connection  between  the  level  of  pupils’  mathematical  abilities  and  development  of 
their  problem-solving  skills  when  working  in  a technology-based  learning  environment  (Ruokamo-Saari  1997). 
Solver  was  designed  on  the  basis  of  an  existing  computer-based  program  developed  for  word  problem-solving 
by  Reusser  et.  al.  HERON  (see  Reusser  1988,  1991a,  1991b;  Staub  et.  al.  1994;  Reusser  1993,  1995).  In  the 
Solver  Learning  Environment  pupils  solved  word  problems  by  constructing  solution  trees  with  cognitive  tools. 
This  process  ends  up  with  final  result  that  solves  the  problem  posed  in  the  text.  (See  Ruokamo  & Pohjolainen 
1998.) 

In  Solver  Learning  Environment  the  properties  of  on-  and  off-line  hypermedia  were  connected  together.  The 
on-line  part  was  designed  to  be  used  via  the  WWW  and  the  off-line,  local  part  was  programmed  with 
Multimedia  ToolBook  3.0.  Figure  1 presents  the  user  interface  of  Solver  Learning  Environment  (in  Finnish): 
the  tools,  solution  tree,  and  equation  in  off-line  ToolBook  application  are  on  the  right  and  the  tools  and 
problem  text  of  an  on-line  HTML  document  are  on  the  left.  (See  Ruokamo  & Pohjolainen  1998.) 


Figure  1:  The  user  interface  of  Solver  with  on-line  and  off-line  tools 

Problem  tasks,  hints,  the  model  solutions,  and  a bulletin  board  were  programmed  using  HTML,  so  that  these 
can  easily  be  distributed  via  network,  and  they  can  be  updated  later  on.  In  creating  the  problem  situations  a 
contextual  approach  was  adapted:  the  topics  chosen  for  the  word  problems  were  based  on  pupil  inquiry:  the 
pupils  were  asked  about  real  life  problems  they  would  like  to  solve.  Video  material  was  produced  to  introduce 
the  problem  situations  for  pupils.  (See  Ruokamo  & Pohjolainen  1998.) 

In  the  problem  text  there  were  links,  by  activating  them  pupils  could  get  definitions,  rules,  formulas,  and  other 
information  about  the  subject.  If  a problem  was  too  complex  to  solve,  the  user  may  select  the  hint  tool  to  obtain 
tips  for  constructing  the  solution.  Pupils  may  ask  hints  from  each  other  in  problematic  situations  when 
constructing  the  solutions.  Using  the  bulletin  board,  pupils  were  able  discuss  with  other  pupils  or  with  their 
teacher  methods  for  solving  the  problems.  On  the  basis  of  these  conversations  they  could  further  develop  their 
own  constructions.  (See  Ruokamo  & Pohjolainen  1998.) 

In  the  Solver  Learning  Environment  the  model  solution  for  a given  mathematical  problem  was  presented  step 
by  step,  and  the  pupils  were  assumed  to  discover  the  solution  procedure.  The  pupils’  solutions  were  saved  to  be 
analysed  later  on.  (See  Ruokamo  & Pohjolainen  1998.) 


Solver  does  not  tutor  pupils  in  the  problem-solving  process,  unless  asked  for  help  or  hints.  This  property  was 
thought  to  be  suitable  for  especially  better  problem-solvers  who  do  not  necessarily  need  any  tutoring  but  are 
able  to  conduct  autonomous  and  original  problem-solving.  Tutoring  was  designed  with  those  pupils’  needs  in 
mind  who  had  difficulties  in  problem-solving.  It  was  also  important  that  pupils  had  a chance  to  notice  by 
themselves  possible  mistakes  they  made  during  problem  solving.  When  a user  asked  for  a hint  during  the 
solution-process,  Solver  tried  to  find  a so  called  context  sensitive  hint  related  to  the  user’s  situation  in  his/her 
solution.  If  the  user’s  way  of  constructing  the  solution  was  very  original  and  Solver  could  not  help  him/her,  it 
asked  the  pupil  to  try  to  find  support  from  other  pupils  via  bulletin  board  or  to  go  on  and  check  the  final  result 
at  the  end  with  the  checking  tool.  (See  Ruokamo  & Pohjolainen  1998.) 

The  user  could  start  modelling  in  any  situation  and  move  on  to  an  open  solution  space.  In  principle,  he/she  was 
able  to  construct  the  whole  solution  without  tutoring  until  the  final  stage,  where  Solver  asked  the  pupil  to 
evaluate  the  final  equation.  If  the  user  made  mistakes  in  constructing  the  solution,  the  checking  tool  informs 
him  or  her  about  wrong  results.  After  checking  the  results,  the  user  could  see  a representation  of  the  model 
solution  and  could  compare  it  with  his/her  own  solution.  Even  if  the  solutions  were  not  similar,  they  could  be 
correct.  All  the  pupils’  solutions  were  saved  to  files,  so  that  the  problem-solving  processes  could  be  examined 
and  compared  later  on.  The  new  solutions  proposed  by  the  pupil  can  be  used  as  examples  of  alternative  ways  to 
construct  solutions.  (See  Ruokamo  & Pohjolainen  1998.) 


3 The  Methodological  Solutions 

In  addition  to  measuring  mathematical  abilities,  the  study  examined  pupils’  word  problem-solving  skills  and 
their  development  using  the  Solver  Learning  Environment  as  opposed  to  a traditional  pencil-and-paper  method. 
A second  goal  was  to  determine  whether  there  is  any  connection  between  the  development  of  word  problem- 
solving skills  and  pupils'  gender,  attitudes  and  experiences  and  mathematical  abilities. 

In  this  study  we  compared  differences  in  the  problem-solving  skills  of  pupils  on  different  levels  of  mathematical 
abilities.  Pupils  studied  in  different  learning  environments:  in  the  technology-based  Solver  Learning 
Environment  and  in  a learning  environment  of  conventional  pencil-and-paper  tasks.  The  study  was  an 
experimental  comparative  study.  Experiment  group  A studied  in  the  technology-based  Solver  Learning 
Environment  and  experiment  group  B studied  in  the  conventional  way.  The  control  group  did  not  study 
problem-solving,  they  only  took  part  in  tests.  (Ruokamo-Saari  1997.) 


3.1  Experimental  Design  and  the  Participants  of  the  Study 

The  research  was  an  empirical  and  experimental  comparative  study  of  randomly  chosen  pupil  groups.  The 
experimental  groups  were  made  by  lottery.  The  experimental  design  of  this  study  is  presented  in  Figure  2. 
below: 


Experiment  group  A 

MR  — 

— Ml/Al  — 

— TBLE/Sl  - 

— M2  /A2  - 

...  mm  - 

--  M3 

Experiment  group  B 

MR  - 

— Ml/Al  — 

PP 

■—  M2/A2  - 

...  mm  — 

- M3 

Control  group 

MR  — 

— Ml/Al  — 

. — M2 

...  mm  - 

- M3 

Figure  2:  Experimental  design  of  the  study 

In  the  experimental  design  MR  (for  all  the  groups)  refers  to  the  matrices  test  of  Raven,  Ml  refers  to  the  pre-test 
of  problem-solving  skills  and  M2  to  the  post- test.  A1  and  A2  are  tests  of  attitude  towards  mathematics,  word 
problem-solving,  information  technology  (IT)  (Al)  and  towards  the  environment  used  in  the  teaching 
experiment  (A2).  MM  is  a test  that  measures  essential  mathematical  knowledge  (Kupari  1 983).  TBLE  and  PP 
are  environments  - TBLE  is  technology-based  Solver  Learning  Environment  and  PP  is  conventional 
environment  of  pencil-and-paper  tasks,  (cf.  Cohen  and  Manion  1987.)  SI  denotes  an  inquiry  associated  to 
Solver.  M3  (for  all  the  groups)  refers  to  delayed  test  of  lasting  quality  of  problem-solving  skills.  The  control 
group  took  part  into  the  pre-,  post-  and  delayed  tests  and  measurements  of  mathematical  abilities.  The  purpose 
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of  these  tests  was  to  find  out  the  differences  between  the  pupils  who  have  practised  and  who  have  not  practised 
problem-solving.  (Ruokamo-Saari  1997.) 

It  was  decided  to  change  the  ordinary  grouping.  The  groups  were  divided  so  that  they  were  as  homogenous  as 
possible.  Pupils  were  selected  randomly.  The  division  into  groups  was  made  randomly  so  that  each  group 
consisted  of  1 1 girls  and  1 1 boys.  Pupils  in  the  technology-based  learning  environment  group  were  grouped 
into  working  pairs  on  the  basis  of  gender  and  of  success  in  the  pre-test.  (Ruokamo-Saari  1997.) 


3.2  Measurements  and  Data  Collection 

Data  was  collected  on  mathematical  abilities  with  two  different  kinds  of  tests:  The  Standard  Progressive 
Matrices  (SPM)  of  J.C.  Raven  (1958)  to  evaluate  reasoning  ability,  and  a test  made  of  tasks  of  mastery  of  core 
mathematics  by  P.  Kupari  (1983).  With  this  test  we  tried  to  evaluate  the  pupils’  mathematical  knowledge.  We 
also  evaluated  the  pupils’  mathematical  abilities  on  the  basis  of  their  average  grades  in  mathematics.  We  made 
pre-,  post-  and  delayed  tests  of  mathematical  word  problem-solving  to  collect  data  of  pupils  problem-solving 
skills.  We  also  made  attitude  tests  in  regard  to  mathematics,  word  problem-solving,  information  technology 
(IT)  and  learning  environments. 


3,3  Teaching  Experiment 

The  teaching  experiment  and  data  collection  lasted  1 1 hours.  In  the  first  lesson  we  measured  the  reasoning 
ability  of  the  pupils  (N=66)  with  the  SPM  by  Raven.  The  Raven  Test  was  the  first  part  of  the  evaluation  of 
pupils’  mathematical  abilities.  In  the  second  lesson  we  arranged  a pre-test  of  pupils’  problem-solving  skills. 
After  that  we  also  arranged  an  attitude  test.  With  that  test  we  evaluated  the  pupils’  motivation  and  their 
attitudes  towards  mathematics,  towards  mathematical  word  problems,  and  towards  IT.  We  collected  data  during 
the  six  consecutive  math  lessons,  simultaneously  with  different  groups.  The  pupils  were  grouped  randomly  into 
an  experiment  group  A (N=22)  and  an  experiment  group  B (N=22)  and  a control  group  (N=22).  Group  A and 
group  B (N=44)  were  studying  mathematical  word  problem-solving.  Experiment  group  A worked  in  pairs 
during  lessons  collaborating  with  other  pupil  pairs,  and  group  B studied  in  a conventional  way.  The  post-test  of 
pupils’  problem-solving  skills  was  arranged  after  the  teaching  lessons,  when  we  also  measured  pupils’  attitudes 
towards  learning  environments  and  also  to  the  experiment.  After  the  post-test  we  arranged  second  test  of 
mathematical  abilities:  a test  of  mastery  of  core  mathematics.  One  and  a half  months  after  the  lessons  we 
arranged  the  last  test  of  the  experiment,  the  delayed  test,  (see  Ruokamo-Saari  1 997.) 


3.4  Methods  of  Analysis 

In  this  study  we  analysed  data  using  various  quantitative  methods,  and  the  research  data  was  handled  by  SPSS 
8.0  for  Windows  (SPSS  Advanced  Statistics  7.5.  for  Windows).  The  data  comprises  the  scores  of  the  tests  of 
mathematical  abilities  and  the  scores  of  the  pre-,  post-  and  delayed  test  and  the  scores  of  the  motivation  and 
attitude  tests.  The  usability  of  the  statistical  methods  was  affected  by  the  number  of  pupils  actually  participating 
in  the  experiment  (N=66).  The  quantitative  analysis  methods  used  include  frequencies,  correlation,  variance 
analysis,  regression  analysis,  factor  analysis  and  covariance  analysis. 


4 Main  results 

The  Solver  group  did  best  in  the  delayed  test.  Problem-solving  skills  also  developed  best  in  this  group  among 
those  pupils  who  scored  highest  in  mathematical  abilities.  Results  are  not  statistically  significant,  possibly  due 
to  the  smallness  of  the  sample.  No  connection  was  found  between  gender  and  mathematical  abilities,  problem- 
solving skills  or  their  development.  A positive  attitude  to  math  correlated  with  success  in  math,  likewise  with 
problem-solving  skills.  Boys  had  a more  positive  attitude  toward  IT  than  girls  and  were  more  likely  to  view 
math  as  a hobby.  Math  enthusiasts  had  better  problem-solving  skills,  and  IT  enthusiasts  showed  the  greatest 
improvement.  Gifted  pupils  opted  for  harder  courses.  Attitudes  to  word  problem-solving  correlated  with 
mathematical  abilities  and  problem-solving  skills,  except  for  success  in  the  Raven  Test.  Girls  reacted  more 
positively  towards  the  situation  model  as  a tool  and  to  the  contextuality  of  the  Solver  Learning  Environment. 


936 


Gender  affected  interest  in  all  six  series  of  tasks.  It  may  be  concluded  that  the  Solver  Learning  Environment 
helped  to  improve  problem-solving  among  mathematically  gifted  pupils.  Further  research  should  undertake  a 
qualitative  analysis  of  pupils'  problem-solving  processes. 


5 Conclusions 

This  article  described  the  Solver  Learning  Environment,  the  methodological  solutions  and  main  results  of  the 
study  dealing  with  mathematical  abilities  and  development  of  mathematical  word  problem-solving  skills  in  a 
technology-based  Solver  Learning  Environment.  In  this  paper  we  described  experimental  design, 
measurements,  and  data  collection  of  the  study.  We  gathered  data  from  the  teaching  experiment  that  was 
organised  in  spring  1996,  in  this  paper  we  also  presented  quantitative  analysis  methods  of  the  data  and  main 
results  of  the  study. 


References 

Cohen,  L.  & Manion,  L.  (1987).  Research  methods  in  education.  London:  Groom  Helm. 

Koskelainen,  P,  Rakkolainen,  I.,  Soininen,  R.,  Suvanto,  J.  (1996a).  ATM-based  Multimedia.  Trial  in  Comprehensive 
Schools.  Telematics  for  Future  Education  and  Training:  Proceedings  of  the  EAEEIE  '96  Seventh  Annual  Conference  June 
12-14,  1996,  Oulu,  Finland,  23-25. 

Koskelainen,  P.,  Rakkolainen,  1.,  Soininen,  R.,  Suvanto,  J.  (1996b).  SISU  - Schools  to  Information  Superhighway.  In:  S. 
Rao  et  al.  (Eds.)  Global  Information  Infrastructure  (GII)  Evolution,  Interworking  Issues.  lOS  Press,  651-654. 

Kupari,  P.  (1983).  Millaista  matematiikkaa  peniskoulun  p^ttyessa  osataan?  Yhdeksasluokkalaisten  oppimistulokset 
keskeisessa  matematiikassa.  Institute  for  Educational  Research.  Publications  342/1983.  University  of  Jyvaskyla. 

Pohjolainen,  S.  & Ruokamo-Saari,  H.  (1997).  Distance  learning  in  multimedia-networks:  A Finnish  national  multimedia 
project.  T.  Muldner  & T.C.  Reeves  (Eds.)  ED-MEDIA/ED-TELECOM  97.  World  Conference  on  Educational 
Multimedia/Hypermedia,  World  Conference  on  Educational  Telecommunications.  Volume  II.  Proceedings,  University  of 
Calgary;  CANADA;  June  14-19,  1997,  853-863. 

Pohjolainen,  S.  & Ruokamo,  H.  (1998).  Distance  Learning  in  Multimedia  Networks  Project:  Current  State  of  the  Art  and 
Perspectives  for  Future.  J.  Gil-Mendieta  & M.H.  Hamza  (eds.)  Computers  and  Advanced  Technology  in  education  (CATE 
’98).  Proceedings  of  the  lASTED  International  Conference.  A Publication  of  the  International  Association  of  Science  and 
Technology  for  Development  - lASTED.  lASTED/ACTA  Press.  Anaheim,  Calgary,  Zurich.  CATE'98  Conference,  May  27- 
30,  1998  Cancun,  MEXICO,  91-95. 

Pohjolainen,  S.  & Ruokamo,  H.  (1999).  Distance  Learning  in  Multimedia  Networks:  Experiences  and  Results.  In  B.  Collis 
& R.  Oliver  (Eds.)  ED-MEDIA  1999.  World  Conference  on  Educational  Multimedia,  Hypermedia  & Telecommunications. 
Proceedings  of  ED-MEDIA  99  - Volume  2.  Seattle,  Washington  USA;  June  19-24,  1999,  922-927. 

Raven,  J.C.  (1958).  Standard  Progressive  Matrices.  Sets  A,  B,  C,  D,  and  E.  Printed  in  University  Printing  House, 
Cambridge,  Great  Britain.  London:  H.K.  Lewis  & Co.  Ltd. 

Reusser,  K.  (1988).  Problem  Solving  beyond  the  Logic  of  Things:  Contextual  Effects  on  Understanding  and  Solving  Word 
Problems.  Instructional  Science,  17  (14),  309-338. 

Reusser,  K.  (1991a  August).  Intelligent  technologies  and  pedagogical  theory:  Computers  as  tools  for  thoughtful  teaching 
and  learning.  Paper  presented  at  the  Fourth  European  Conference  for  Research  on  Learning  and  Instruction  EARL  I,  held  at 
Turku,  Finland,  24-28  August,  1991. 

Reusser,  K.  (1 99 lb  October).  Tutoring  systems  and  pedagogical  theory:  Representational  tools  for  understanding,  planning 
and  reflection  in  problem-solving.  Universitat  Bern.  Abteilung  Padagogische  Psychologie.  Forschungsbericht/Research 
report  No.9. 

Reusser,  K.  (1993).  Tutoring  systems  and  pedagogical  theory:  Representational  tools  for  understanding,  planning,  and 
reflection  in  problem  solving.  In  S.P.  Lajoie  & S.J.  Derry  (Eds.),  Computers  as  cognitive  tools.  Hillsdale,  N.J.:  Lawrence 
Erlbaum  Associates,  143-177. 


Reusser,  K.  (1995).  From  Cognitive  Modeling  to  the  Design  of  Pedagogical  Tools.  In  S.  Vosniadou,  E.  De  Corte,  R.  Glaser 
& H.  Madl  (Eds.).  International  Perspectives  on  the  Design  of  Technology-Supported  Learning  Environments.  Hillsdale, 
NJ:  Lawrence  Erlbaum  Associates,  81-103. 

Ruokamo-Saari,  H.  (1996a).  Hypermedia  Learning  Environment  for  Especially  Mathematically  Gifted  Pupils  for 
Mathematical  Word  Problem-Solving.  In:  P.  Carlson  & F.  Makedon  (eds.)  Educational  Multimedia  and  Hypermedia,  1996. 
Proceedings  of  ED-MEDIA  96- World  Conference  on  Educational  Multimedia  and  Hypermedia.  Boston,  Mass.,  USA;  June 
17-22,  1996,  804. 

Ruokamo-Saari,  H.  (1996b).  Designing  Hypermedia  Learning  Environment  for  Solving  Complex  Mathematical  Word 
Problems.  In  L.  Aho  ^d. ) Learning  in  Multiple  Contexts.  Papers  and  posters  presented  at  the  First  Joensuu  Symposium  on 
Learning  and  Instruction.  lO.-ll.  November,  1995.  Bulletins  of  the  Faculty  of  Education,  University  of  Joensuu,  N:o  62, 
141-158. 

Ruokamo-Saari,  H.  (1997).  Mathematical  Giftedness  and  Development  of  Mathematical  Word  Problem-Solving  Skills  in  A 
Technology-Based  Learning  Environment:  A Teaching  Experiment.  T.  Muldner  & T.C.  Reeves  (Eds.)  ED-MEDIA/ED- 
TELECOM  97.  World  Conference  on  Educational  Multimedia/Hypermedia,  World  Conference  on  Educational 
Telecommunications.  Volume  II.  Proceedings,  University  of  Calgary;  CANADA;  June  14-19,  1997,  923-927. 

Ruokamo,  H.  & Pohjolainen,  S.  (1998).  Pedagogical  Principles  for  Evaluation  of  Hypermedia-Based  Learning 
Environments  in  Mathematics.  In  JUCS  — Journal  of  Universal  Computer  Science,  4 (3),  292-307.  (online  <URL: 
http://www.iicm.edu/iucs  4 3/>) 

Ruokamo,  H.  & Pohjolainen,  S.  (1999).  Distance  Learning  in  Multimedia  Networks  Project:  Main  Results.  In  L.  Baggott  & 
J.  Nichol  (Eds.)  Proceedings  of  the  Ninth  International  PEG  Conference.  Intelligent  Computer  and  Communications 
Technology,  Teaching  and  Learning  for  the  21st  Century.  Exeter,  UK,  July  10-12,  1999,  153-162. 

Staub,  F.C.,  Stebler,  R.,  Reusser,  K.  & Pauli,  C.  (1994  March).  Improving  understanding  and  solving  of  math  story 
problems  through  collaborative  use  of  a computer  tool  (HERON).  Paper  to  be  presented  in  a structured  poster  session  on 
"Children’s  understanding  of  mathematical  knowledge"  at  the  Annual  Meeting  of  the  American  Educational  Research 
Association,  New  Orleans,  4-8  April,  1994. 


Acknowledgements 

Special  thanks  are  due  to  the  Alexpress  Ltd.  of  Aamulehti  Group  Ltd.  for  technical  implementation  and  programming  of  the 
Solver  Learning  Environment  and  to  the  Digital  Media  Institute  of  Tampere  University  of  Technology  for  technical  support. 
Part  of  this  work  was  financed  by  the  Academy  of  Finland  within  the  Information  Research  Programme.  I gratefully 
acknowledge  the  teachers  and  the  pupils  of  the  Kaukajarvi  School,  who  were  so  willing  to  collaborate  and  so  positive 
towards  the  experiment. 


ERIC 


Is  Computer-Mediated  Communication  (CMC)  Really  Open  and  Flexible? 


Malcolm  Ryan,  School  of  Post  Compulsory  Education  and  Training,  University  of  Greenwich,  UK. 

m.ryan@greenwich.ac.uk 


Abstract:  This  paper  considers  the  concepts  of  openness  and  flexibility  and  the  extent  to 
which  computer  mediated  communication  supports  these  approaches  to  learning.  Following 
an  examination  of  aspects  of  open  and  flexible  learning,  the  nature  and  role  of  Computer 
Mediated  Communication  (CMC)  is  considered  within  a context  of  continuing  professional 
development.  Models  of  curriculum  delivery  employing  CMC  and  experiences  of  using 
these  with  adult  learners  are  examined.  Student  feedback  is  used  to  illustrate  positive  and 
negative  aspects  of  our  approach.  Barriers  to  learning  flexibly  when  employing  CMC  are 
identified  and  conclusions  drawn  about  the  extent  to  which  such  strategies  are  open  and 
flexible. 


The  concepts  of  open  and  flexible  learning 

Open  and  flexible  learning  (OFL)  are  characterised  in  the  literature  by  the  extent  to  which  they  enable  students 
to  access  learning  opportunities.  The  literature  reveals  such  learning  to  be  'resource-based',  exploiting  a range 
of  media  from  printed  texts  to  intelligent  tutoring  systems  using  the  latest  information  and  communication 
technology  (ICT).  The  use  of  a wide  range  of  resources  normally  results  in  such  learning  being  'self  directed' 
including  a number  of 'self  assessment'  activities. 

A major  advantage  for  students  of  this  approach  to  learning  has  been  its  'time-location  independence',  removing 
the  need  to  travel  to  a particular  physical  location  in  order  to  study.  This  aspect  of  OFL  has  led  to  the 
development  of  global  education  opportunities,  enabling  students,  irrespective  of  their  geographical  location,  to 
undertake  a programme  anywhere  in  the  world. 

The  availability  of  quality  materials  on  a wide  range  of  subjects  has  enabled  'learning  on  demand'  to  occur, 
independent  of  a tutor  or  educational  institution.  Being  the  recipient  of  a range  of  learning  resources  has 
required  students  to  become  'self  managed'  and  led  to  greater  'autonomy'.  'Guided  discovery'  and  interaction 
with  these  materials  replacing  the  large  amounts  of  information  that  were  often  delivered  in  a didactic  and 
sometimes  authoritarian  manner  in  large  lecture  theatres. 

This  move  towards  OFL  methods  of  delivery  has  resulted  in  a changed  role  for  the  teacher.  No  longer  is  the 
tutor  the  fount  of  all  knowledge,  the  'sage  on  the  stage',  or  a deliverer  of  content;  instead  s/he  has  become  the 
'guide  on  the  side',  a facilitator  of  learning.  Another  significant  element  of  OFL,  has  been  the  extent  to  which 
peer  networking  and  collaboration  are  being  encouraged  in  both  physical  and  virtual  classrooms. 

Rowntree  (1992)  maintains  that  open  learning  is  both  a philosophy  and  a method  concerned  with,  "'reducing 
the  barriers  to  access  and  giving  learners  more  control. " He  claims  that  open  learning  provides  opportunities 
for: 

• Access  - enabling  more  people  to  learn 

• Time-Place  Independent  study  - when  and  where 

• Individual  pace  - to  suit  own  style/circumstance 

• Learner  Autonomy  - greater  individual  responsibility  and  negotiation 

Increased  access  to  higher  education  has  been  a government  target  within  the  UK  in  recent  years  and  student 
numbers  have  increased  dramatically.  This  has  led  to  a shortage  of  accommodation  and  a concomitant  growth 
in  open  learning  in  an  attempt  to  address  overcrowding  in  classrooms.  Many  institutions  have  converted 
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classrooms  into  open  learning  centres  or  invested  large  sums  of  money  in  building  new  centres,  often 
combining  traditional  libraries,  computing  facilities  and  individual  and  small  group  learning  spaces. 

Whilst  being  freed  from  the  constraints  of  timetables  and  physical  spaces  can  have  a liberating  effect  on 
students,  there  is  evidence  to  suggest  that  not  all  learners  either  appreciate  nor  benefit  from  this  apparent 
freedom.  Anecdotally,  having  to  attend  a physical  class  often  provides  students  with  the  necessary  discipline 
and  focus  to  engage  with  learning.  Evidence  from  our  own  post-graduate  students  suggests  they  appreciate  the 
flexibility  of  open  and  especially,  distance  learning  because  they  are  better  able  to  self-manage  studying,  work 
and  personal  (including  family)  commitments.  However,  when  questioned,  many  say  they  would  have  preferred 
to  attend  a traditional  class  with  a tutor  and  peer  group  on  a regular  weekly  basis  but  their  personal 
circumstances  make  this  a non-viable  option. 

Linked  to  this  self-managed  learning  are  issues  of  learning  style  (or  preference)  and  pace.  Given  the 
circumstances  of  many  of  our  adult  learners,  studying  for  a professional  qualification,  working  full-time  and 
participating  in  family  life,  organising  learning  around  these  diverse  demands  is  seen  as  advantageous.  Having 
the  freedom  to  engage  with  materials  and  undertake  activities  at  a time  of  the  day  or  week  that  suits  individual 
circumstances  is,  for  them,  the  most  attractive  aspect  of  flexible  learning.  However,  unless  a student  is  highly 
self-disciplined,  this  freedom  can  be  abused  and  lead  to  irregular  and  unproductive  periods  of  study,  resulting  in 
loss  of  motivation  and  high  rates  of  dropout  from  courses. 

Greater  autonomy  may  enable  an  adult  student  to  manage  their  time  but  there  are  assessment  targets  to  meet 
and  courses  often  start  and  end  at  times  which  are  more  convenient  to  the  host  institution  than  the  individual 
student.  This  can  generate  stress,  frustration  with  the  system  and  result  in  loss  of  motivation,  failure  and 
dropout. 

So,  OFL  provides  students  with  the  opportunity  to,  ''take  greater  responsibility  for  their  learning  and  to  be 
engaged  in  activities  that  meet  their  own  individual  needs,"  (Wade,  1994).  It  is  free  of  time  and  place 
constraints  and  allows  access  to  a wide  range  of  resources  as  and  when  required  but  does  not  suit  all  learners, 
especially  adults. 


The  nature  and  role  of  CMC 

These  philosophies  of  openness  and  flexibility  can  be  realised  through  a variety  of  approaches  and  those  that 
employ  Information  and  Communication  Technology  (ICT)  are  regarded  as  ’new  forms  of  learning’  (Laurillard, 
1994).  According  to  the  UK’s  Further  Education  Funding  Council  (1996),  "exploiting  emerging  (new) 
technologies  will  lead  to  more  open  systems  allowing  students  to  study  away  from  institutions,  at  a time,  place 
and  pace  to  suit  individual  circumstances. " 

The  University  of  Greenwich  has  been  using  CMC  with  adult  students  on  a range  of  courses  for  several  years. 
The  main  reason  for  using  it  is  that  it  facilitates,  ''interaction  between  students  and  tutors  irrespective  of 
geographical  location  or  time  zone.  " (Ryan,  1997)  Exploiting  the  unique  features  of  technology  enhances  the 
student  experience  (Anderson,  1996)  and  leads  to  new  forms  of  learning  and  teaching. 

Our  use  of  the  groupware  product,  Lotus  Notes  with  customised  desktops  being  distributed  to  students,  led  to 
the  creation  of  a Virtual  Integrated  Learning  Environment  (Ryan  and  Culwick,  1997)  and  the  discovery  of  a 
number  of  unique  features  (Fig.l). 
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Figure  1:  Features  of  CMC  and  traditional  distance  modes  compared 

Firstly,  smdents  have  access  to  a learning  community  that  helps  address  the  feelings  of  isolation  so  often 
experienced  by  distance  learners.  Within  a traditional  distance  learning  context,  students  normally  only  have 
access  to  their  peer  group  if  they  attend  a study  day  or  weekend.  The  second  novel  feature  associated  with  the 
use  of  CMC  is  that  all  discussion  is  archived  and  becomes  a much-valued  resource  by  students  and  tutors. 
(Ryan,  1997) 

Our  experience  of  using  CMC  indicates  that  it  has  considerable  potential  for  facilitating  collaboration  between 
smdents  although  they  are  studying  at  a distance.  Within  a conventional  distance  learning  context,  such 
collaboration  is  only  possible  if  smdents  come  together  for  a smdy  day  in  a physical  location.  Though  regarded 
as  invaluable  by  students  themselves,  if  one  is  working  with  a global  cohort  then,  it  is  unlikely  that  students 
will  be  able  fo  attend  such  events.  The  quality  of  collaboration  we  have  experienced,  particularly  with  health 
professionals  engaged  in  paper  writing  activities,  has  been  excellent 

Team  teaching,  now  considered  too  expensive  in  physical  classrooms,  is  both  possible  and  cost-effective  within 
a CMC  programme.  There  is  a steep  learning  curve  when  tutoring  in  a virtual  classroom  for  the  first  time 
(Ryan  and  Woodward.  1998).  Once  tutors  are  accustomed  to  operating  differently,  it  is  possible  for  several 
colleagues  to  provide  multiple  perspectives  on  a topic,  at  no  additional  cost.  Within  a traditional  distance 
learning  context,  team-teaching  is  usually  only  apparent  in  the  writing  of  materials  or  at  study  days.  However, 
the  costs  of  employing  several  tutors  are  considerable  and  something  of  a luxury  in  today's  financially 
constrained  environment.  Print-based  materials  are  commonplace  in  distance  programmes  but  are  expensive  to 
produce,  store  and  revise.  They  quickly  lose  their  currency  and  as  ICT  systems  provide  us  with  up-to-the  minute 
data  and  become  more  widespread,  they  are  likely  to  replace  print-based  materials  in  all  learning  and  teaching 

contexts. 


Traditional  and  CMC  delivered  distance  modes  share  the  facility  for  students  to  reflect  on  content  and  their 
responses  to  activities.  Another  shared  feature  is  that  both  modes  are  independent  of  time  and  place,  though 
traditional  distance  learning  is  more  independent  because  one  does  not  need  access  to  ICT. 


Models  of  Computer  Mediated  Curriculum  Design 

Mason  (1998)  suggests  the  emergence  of  distinct  approaches  or  models  to  CMC  delivered  or  supported  OFL 
programmes. 

Content  and  Support 

The  content  and  support  model  is  often  applied  to  existing  programmes  of  study  where  students  receive  content 
through  a variety  of  media  and  ICT  is  used  as  just  one  of  several  support  mechanisms.  This  may  take  the  form 
of;  simple  e-mail,  a bulletin  board,  bespoke  conferencing  software  or  a web-based  system  such  as  WebCT. 
Because  CMC  is  only  one  of  many  means  of  communicating  between  students  and  tutors,  it  is  often  under 
utilised  and  its  unique  features  go  unexploited.  Many  institutions  begin  to  explore  the  use  ot  ICT  in  an  OFL 


context  by  adopting  this  approach.  It  has  the  advantages  of  being  relatively  cheap  to  set  up  and  does  not 
necessarily  require  any  major  changes  to  either  the  content  of  a course  or  its  delivery. 

Wrap  Around 

The  second  approach  is  known  as  the  'wrap  around'  or  50/50  model.  The  content  of  the  programme  is  provided 
through  a single  medium,  such  as  a published  book  or  multiple  media,  and  some  form  of  conferencing  mediates 
all  interactions  and  communications.  This  has  become  a very  popular  way  of  employing  ICT  in  support  of 
students  operating  at  a distance  because  it  often  uses  existing  materials.  However,  as  all  of  the  interactions 
employ  new  technology,  it  is  vital  that  students  and  tutors  have  regular  and  frequent  access  to  appropriate  ICT 
tools  and  support.  For  many  participants,  interacting  via  a computer  is  a new  and  strange  phenomenon 
requiring  a different  set  of  skills  to  those  normally  employed  when  learning  and  teaching  at  a distance. 

Integrated 

The  third  emerging  model  is  the  integrated  approach  where  both  the  content  and  the  interactions  are  delivered 
and  managed  by  an  ICT  system.  This  is  perhaps  the  most  difficult  to  establish  because  it  often  requires  a 
revision  of  all  materials  and  a complete  review  of  learning  and  teaching  strategies.  Its  major  advantage  is  that 
one  platform  can  be  used  for  both  delivery  and  support,  but  this  can  also  be  a major  shortcoming.  If  access  to 
the  technology  is  restricted  for  any  reason,  for  example,  a technical  breakdown,  then  the  programme  ceases  to 
function  until  connectivity  to  the  system  is  restored.  As  with  the  wrap  around  model,  familiarisation  with  the 
system  and  an  opportunity  to  explore  this  new  way  of  learning  and  teaching  is  vital  to  the  success  of  any 
programme  using  the  integrated  approach. 

As  a long-term  strategy,  the  adoption  of  the  integrated  approach  has  the  potential  to  provide  global  education 
opportunities  using  the  most  up-to-date  materials  and  research.  Before  an  institution  can  embark  upon 
programmes  using  this  model,  a satisfactory  system  for  the  delivery  of  materials  incorporating  sophisticated 
conferencing  tools  and  facilitating  the  full  range  of  interactions  between  participants  must  be  established.  Once 
established,  the  benefits  in  terms  of  cost  and  learning  effectiveness  can  far  outweigh  the  associated  risks. 

These  last  two  models  exploit  the  unique  features  of  a CMC  delivered  OFL  curriculum.  They  allow  students  to 
have  asynchronous  access  to  the  learning  community,  at  any  time,  irrespective  of  their  geographical  location. 
Secondly,  most  conferencing  systems  keep  an  archive  of  interactions  so  any  participant  can  review  and  reflect 
upon  entries  at  their  leisure.  Many  systems  provide  message  threading  and  can  graphically  display  such 
activities  as  brain  storming  and  complex  discursive  interactions.  Opportunities  for  collaboration  are  present 
although  participants  are  geographically  dispersed.  Tutoring  in  a virtual  environment  is  very  different  to  either 
face-to-face  class  teaching  or  supporting  students  on  a traditional  OFL  programme  (Ryan  and  Woodward, 

1 998).  It  is  vital  that  appropriate  staff  development  be  undertaken  to  ensure  that  tutors  have  the  necessary  ICT 
skills  and  are  aware  of  and  can  manage  those  factors  which  influence  the  quality  of  interactions  in  a computer 
mediated  classroom.  (Salmon,  1998) 


Drawing  upon  experience 

Schools  at  the  University  of  Greenwich  employing  CMC  have  found  that  where  it  is  integrated  into  the 
curriculum  process  it  provides  valuable,  additional  learning  and  teaching  experiences  (Lewis,  et  al,  1997; 
Jordan  and  Ryan,  1997;  Hodges  and  Ryan,  1999).  Over  a five  year  period  each  of  these  emerging  models  has 
been  applied  to  different  programmes  of  continuing  professional  development  (CPD). 

As  an  illustration  of  the  support  model,  Lotus  Notes  was  used  to  support  adult  distance  learners  engaged  in  a 
teacher  education  programme.  The  students  received  materials  in  the  form  of  printed  texts  that  included  a range 
of  activities.  Contact  with  the  peer  group  was  minimal  except  at  induction  and  optional  study  days.  Telephone, 
fax,  and  occasionally  e-mail  provided  tutor  support.  The  use  of  this  groupware  allowed  students  to  interact  with 
each  other  and  their  tutors  more  regularly.  However,  the  materials  had  been  designed  for  a traditional  distance 
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programme,  which  resulted  in  some  confusion  over  the  purpose  and  status  of  activities  delivered  on-line,  versus 
those  in  the  printed  materials.  Because  the  use  of  CMC  was  optional,  only  those  students  who  engaged 
regularly  with  the  system  and  recognised  its  potential,  actually  benefited.  The  experience  of  using  CMC  was 
new  to  both  students  and  tutors.  A Notes  desktop  was  provided  for  all  participants  but  there  was  little  evidence 
of  improved  learning  and  it  would  have  been  an  expensive  system  to  operate  with  large  numbers.  However, 
many  students  welcomed  the  opportunities  for  interaction  and  increased  access  to  their  tutors  even  if  they  didn’t 
always  fully  exploit  this  additional  support. 

The  wrap  around  model  is  currently  being  used  by  the  Natural  Resources  Institute  (NRI)  who  are  delivering  a 
grain  storage  management  programme  at  masters  level  to  a world-wide  audience.  The  original  programme 
involved  students  being  resident  in  Britain  for  periods  of  time,  combined  with  study  in  the  host  country  and 
tutors  from  the  NRI  visited  and  assessed  students  abroad.  Building  upon  their  expertise  in  delivering  face-to- 
face  workshops  and  providing  high  quality  materials,  a team  of  subject  specialists  worked  in  conjunction  with 
curriculum  and  ICT  developers  to  produce  a new  distance  version  of  this  existing  programme.  Essential 
elements  that  needed  to  be  retained  included;  diversity  of  subject  specialist  inputs,  opportunities  for  students  to 
apply  new  knowledge  to  their  working  environment  in  their  own  country  and  sharing  of  peer  expertise  and 
experiences.  The  rationale  for  developing  such  a programme  focused  on  the  need  to  reach  new  markets  and  to 
be  more  cost-effective.  Employing  the  wrap  around  model  allowed  the  essential  features  to  be  retained  but  the 
costs  of  international  travel  and  accommodation  and  separation  from  families  on  the  part  of  students  were 
removed.  As  the  programme  is  only  in  its  first  year  of  operation,  it  is  not  yet  possible  to  say  whether  it  has  been 
successful.  Anecdotal  evidence  suggests  that  the  tutors,  who  undertook  considerable  staff  development,  are 
committed  to  this  new  programme  and  believe  that  it  fulfils  all  of  the  outcomes  of  the  earlier  traditional  model. 
Student  feedback  has  been  limited  but  the  indications  are  that  the  experience  of  working  within  a virtual 
environment  has  been  positive. 

Two  schools  currently  employ  the  integrated  model;  one  to  deliver  an  experiential  course  in  computer  mediated 
tutoring  (CMT),  the  other  is  a Masters  in  CPD  for  health  professionals.  These  programmes  fully  exploit  those 
innovative  features  of  CMC  outlined  earlier  in  the  paper  and  include  electronic  submission  of  assignments  and 
on-line  anonymous  evaluation.  In  both  cases,  a face-to-face  event  precedes  the  virtual  delivery,  which  is 
provided  by  customised  Lotus  Notes  desktops.  These  desktops  include  the  virtual  classrooms,  electronic 
resource  and  information  centres  and  a student  common  room.  Apart  from  a programme  handbook,  course 
outlines  and  instructional  materials  to  aid  students  with  the  installation  and  operation  of  Lotus  Notes,  all  other 
materials,  activities  and  interactions  are  delivered  within  this  Virtual  Integrated  Learning  Environment  (VILE). 
An  in-depth  evaluation  of  the  Masters  programme,  which  has  just  had  its  first  graduates,  has  revealed  many 
benefits  to  learners  and  overall  satisfaction.  Within  the  University  of  Greenwich,  this  programme  is  considered 
highly  innovative  and  many  other  schools  are  seeking  to  adopt  the  model. 


Issues  Raised  by  Evaluation 

Whilst  there  have  been  some  negative  criticisms  of  the  use  of  CMC  in  support  of  OFL  programmes  the  majority 
have  been  positive.  ''With  CMC  there  are  a lot  more  advantages  than  with  other  distance  learning  because  you 
have  the  support  of  your  peers  and  their  instant  availability,”  Another  student  said,  "the  asynchronous 
medium  has  been  wonderful  for  me  because  I don  7 think  easily  on  my  feet,  I do  like  to  ...  be  able  to  sit  back 
and  reflect  before  I put  my  thoughts  forward,  ” 

These  comments  serve  to  illustrate  a number  of  important  issues.  Firstly,  those  students  who  are  prepared  to 
make  a commitment  to  the  learning  community,  which  includes  connecting  to  the  system  regularly  and 
frequently,  are  more  likely  to  have  a positive  experience  which  can  result  in  learning  gains.  The  claims  made 
about  the  use  of  CMC  and  its  ability  to  address  issues  of  isolation,  facilitate  collaboration  and  reflection  are 
supported  by  student  evaluation.  However,  critical  comments  suggest  that  there  are  potential  barriers  to 
employing  CMC. 
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A questionnaire  given  to  adult  students  said,  '7  have  no  access  to  e-mail  <Sc  the  Internet  and  if  I did  I would  be 
reluctant  to  use  it,  A second  student  indicated  that  they,  ^'personally  prefer  'old-fashioned*  methods  of 
communication.  " A third  said,  "If  I had  the  time,  I would  have  used  the  On-Line  Campus,"  Finally,  another 
said  it  is  a,  "very  good  idea  if  it  is  not  going  to  be  a substitute  for  (face-to-face)  tutorial  support. " 

Whilst  this  feedback  was  in  connection  with  the  use  of  CMC  as  a support  tool,  some  of  the  comments  echo 
remarks  made  by  students  both  on-line  and  in  end  of  course  evaluations  with  regard  to  the  integrated  model.  I 
stated  earlier  that  access  to  reliable  equipment  and  systems  is  a prerequisite  for  using  CMC  within  an  OFL 
context.  Students  and  tutors  who  have  access  to  a computer  both  at  home  and  work  are  in  a better  position  to 
exploit  the  advantages  of  virtual  learning  systems.  Despite  the  media  hype  that  implies  every  home  in  the  UK 
has  a computer  connected  to  the  Internet,  this  is  far  from  the  truth.  Where  there  is  a computer,  often  the 
children  within  a household  have  the  skills  and  spend  the  most  time  interacting  with  the  technology.  For  some 
adult  students,  even  if  they  have  access  to  a computer,  they  claim  to  lack  the  necessary  range  and  level  of  skills 
required  to  make  the  most  effective  use  of  the  software. 

It  is  not  just  the  apparent  lack  of  time  or  ICT  skills  that  prevents  some  adults  from  engaging  with  a CMC 
programme.  We  need  to  undertake  more  research  into  the  relationship  between  learning  styles  and  ICT,  but 
experience  suggests  that  adult  students  often  encounter  significant  difficulties  in  adapting  to  new  ways  of 
learning  and  teaching.  Both  the  wrap  around  and  integrated  models  require  students  to  engage  with  the 
materials  and  then  interact  with  their  peers,  even  if  their  preferred  style  of  learning  is  to  work  as  an  individual 
at  their  own  pace.  Many  of  our  adult  learners  find  it  difficult  to  commit  regular  amounts  of  time  to  studying 
each  week.  Their  comments  suggest  that  on  occasions,  several  weeks  may  elapse  before  they  undertake 
activities  contained  in  a printed  text.  Within  a virtual  environment  where  tutors  are  extolling  the  virtues  of 
interaction  with  the  peer  group  and  expecting  students  to  participate  in  collaborative  activities,  it  is  essential 
that  everyone  attend  regularly.  This  concept  of  Virtual  attendance’  can  be  a real  barrier  for  many  students. 

Another  feature  of  this  new  way  of  learning  is  the  facility  to  read  the  entries  of  other  students  before  composing 
their  own.  Many  consider  this  to  be  cheating;  though,  several  have  commented  favourably,  suggesting  that  they 
sometimes  learn  a lot  from  the  inputs  of  other  students  and  have  little  of  significance  to  add.  Furthermore, 
traditional  distance  material  can  be  didactic  in  its  approach  partly  because  it  assumes  little  or  no  opportunity  for 
interaction  between  the  learner  and  tutor.  Many  adult  learners,  even  when  pursuing  an  open  or  flexible 
programme,  have  an  expectation  that  they  will  be  ’taughf  by  an  expert.  The  approach  to  learning  embodied 
within  most  of  our  CMC  supported  programmes  is  much  more-student  centred,  partly  because  we  seek  to 
exploit  the  novel  features  afforded  by  the  technology.  This  clash  of  cultures,  students  wanting  to  be  taught, 
whilst  tutors  see  themselves  as  facilitators  of  learning,  is  an  additional  barrier. 


Some  conclusions 

When  employing  CMC  models  within  an  OFL  context,  there  is  a general  shift  towards  activities  that  require 
collaboration  and  peer  exchange.  In  a CMC  environment  students  are  encouraged  to  become  more  dependent 
on  the  learning  community  and  there  may  be  peer  and  tutor  pressure  for  virtual  attendance  on  a regular  and 
frequent  basis.  Traditional  OFL  programmes  do  not  require  students  to  be  so  dependent  upon  the  peer  group  or 
tutor  and  there  is  usually  no  requirement  to  study  within  such  prescribed  time  frames.  This  allows  students  to 
be  more  autonomous,  independent,  self-managed  and  some  might  say,  more  flexible  and  open.  Additionally, 
where  access  to  and  use  of  a computer  is  not  a pre-requisite,  one  might  claim  students  have  access  that  is  more 
open.  Certainly  some  barriers  to  access  are  reduced,  if  not  entirely  removed,  if  students  do  not  need  to  be  ICT 
literate  or  need  to  acquire  the  skills  and  attitudes  needed  to  learn  in  this  new  way. 

Within  our  CMC  programmes,  we  have  witnessed  some  tensions  between  the  autonomy  of  the  individual  and 
the  needs  of  the  learning  community.  Whilst  we  continue  to  advocate  the  benefits  and  novel  features  of  using 
CMC  our  experience  suggests  the  approach  may  be  less  open  and  flexible  than  more  traditional  systems  of 
delivery  and  support. 
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Abstract:  Internet  has  brought  language  learners  great  benefits  such  as  instant  access  to 
global  information  and  many  chances  to  come  in  contact  with  their  target  language.  In 
order  to  improve  language  ability  effectively,  making  good  use  of  these  benefits,  some 
system  of  encouraging  learners*  pre-knowledge  to  develop  and  expand  is  required.  This 
paper  introduces  a distributed  notebook  system  for  vocabulary  learning,  which  integrates 
electronic  dictionaries  and  database  and  is  implemented  by  Java  RMI  technology.  Since 
this  system  can  work  on  different  operating  system,  it  can  be  used  for  not  only  self-learn- 
ing but  also  as  group  ware.  Possible  use  of  this  system  will  be  listed. 


1.  Introduction  and  background  of  this  study 

The  wide  spread  of  internet  has  brought  great  benefits  to  language  learners,  in  particular,  who  study  a target 
language  in  a non-native  environment  (foreign)  such  as  EFL  ( English  as  a foreign  language)  learners.  For  example, 
since  the  internet  enabled  language  learners  to  easily  access  and  to  get  information  written  in  target  language,  they 
became  to  have  many  chances  to  encounter  new  words  and  to  expose  to  the  target  language  in  daily  life.  Since  the 
use  of  the  internet  is  also  not  influenced  by  the  restriction  of  time  and  place,  learners  are  able  to  study  target 
language  with  internet  at  any  time  and  any  where  they  want.  Recent  trend  of  language  learning  and  teaching  makes 
use  of  information  got  from  internet  as  "authentic  materials"  instead  of  ordinary  materials  such  as  textbook.  The 
internet  has  really  influenced  the  way  of  language  learning  and  teaching  which  we  have  been  doing. 

However,  though  internet  has  provided  the  benefits  and  useful  chances  for  language  learners,  these  factors 
neither  directly  encourage  learners  to  deep  understanding  of  words  nor  improve  their  language  ability.  In  other 
words,  only  increase  of  the  browsing  and  exposing  time  can  not  lead  learners  to  improvement  of  language  skills. 
Thus,  we  think  that  some  system  or  tool  of  encouraging  learners'  pre-knowledge  to  develop  and  expand  should  be 
introduced  in  the  now  environment  in  order  to  improve  language  ability  effectively. 

In  this  paper,  we  introduce  a distributed  notebook  system  for  vocabulary  learning.  This  system  focuses  the 
purpose  of  use  on  vocabulary  learning.  There  are  several  reasons  for  this.  First,  the  vocabulary  learning  is  an 
important  key  to  improve  language  skill  such  as  reading  and  speaking.  To  deep  understanding  of  words  means  to 
know  the  concepts  behind  words,  the  types  of  collocation  and  language  use  as  well  as  their  meanings.  To  acquire 
these  knowledge  facilitates  communication,  and  moreover,  leads  improvement  of  language  skill.  Second,  vocabu- 
lary learning  is  suited  to  an  environment  like  internet,  which  has  good  chances  of  encountering  frequently  "un- 
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known  words"  and  repeatedly  "previous  words"  which  learners  have  already  learned.  Hatch  and  Brown  1995 
stated  that  the  important  point  to  deep  understanding  of  words  is  to  encounters  with  the  same  word  in  multiple 
sources  rather  than  in  just  one  source.  Internet  meets  this  requirement.  Third,  vocabulary  learning  need  constant 
practice.  Thus,  learners  should  keep  on  learning  wherever  they  are.  Since  the  use  of  internet  is  not  influenced  by 
the  restriction  of  time  and  place,  it  will  be  helpful  to  learners  for  constant  practice.  Fourth,  vocabulary  learning 
involves  activities  of  repetition  and  confirmation  when  learners  memorize  new  words  and  review  the  previous 
learning.  If  database  is  used  for  these  purposes,  learners  are  able  to  access  the  previous  learning  easily  and  to 
confirm  the  specific  words  repeatedly.  That  encourages  learners  to  give  any  further  attention  the  previous  learning. 
Thus  all  the  effort  given  to  the  previous  learning  will  not  be  wasted. 

Based  on  these  reasons,  we  implemented  a distributed  notebook  system  for  vocabulary  learning.  This  sys- 
tem consists  of  two  parts  functionally:  the  electronic  dictionary  part  working  on  network;  the  notebook  part  con- 
nected with  remote  and  local  database,  which  can  exchange  data  with  each  other.  When  to  encounter  new  words, 
language  learners  are  able  to  look  it  up  easily  with  the  electronic  dictionary  of  this  system,  and  to  write  information, 
the  result  of  learning  such  as  word  meaning  and  its  use  in  the  notebook.  Since  the  information  written  in  the 
notebook  by  learners  are  stored  in  either  remote  or  local  database  (elective),  learners  need  not  to  worry  about  loss 
of  the  data,  different  from  the  data  written  in  paper.  It  is  also  possible  for  learners  to  see  the  stored  information 
again  and  to  add  new  information  to  it.  Thus,  the  past  effort  given  to  the  learning  of  new  words  are  not  wasted.  This 
system  is  also  implemented  in  Java  and  learners  are  able  to  use  this  system  at  any  time  and  anywhere  without 
restriction  of  operating  system,  if  they  use  different  types  of  operating  system  between  home  and  school. 


2.  Distributed  notebook  system  for  vocabulary  learning 

2.1  Concept  of  this  system 

It  is  said  that  vocabulary  learning  is  a difficult  part  of  teaching  in  language  learning.  Because  there  are  a lot 
of  words  that. learners  need  to  know  and  it  takes  much  time  to  teach  them  in  classroom  activities.  In  most  cases, 
vocabulary  learning  is  left  entirely  to  learners'  effort  and  their  self-study. 

We  develop  a distributed  notebook  system  for  vocabulary  learning,  which  is  designed  for  every  learner  to 
learn  vocabulary  effectively  and  is  based  on  Nation's  methodology  (1990)  and  McCarthy's  (1997).  According  to 
their  methodologies,  vocabulary  learning  is  involved  in  two  kind  of  learning  : receptive  learning  and  productive 
learning.  "Receptive  learning  involves  being  able  to  recognize  a word  and  recall  its  meaning  when  it  is 
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met".(Nation  1990)  On  the  other  hand,  "Productive  learning  involves  what  is  needed  for  receptive  learning  plus 
the  ability  to  speak  or  write  needed  vocabulary  at  the  appropriate  time."  (Nation  1990)  They  stated  that  in  order  to 
make  their  receptive  vocabulary  (acquired  by  receptive  learning)  productive,  learners  should  learn  not  only  the 
meaning  of  word,  but  also  how  it  is  used,  its  grammatical  characteristics  and  which  words  it  is  usually  associate 
with  etc..  They  also  stated  the  importance  of  dictionary  use,  of  vocabulary  note  book  for  writing  down  new  words 
and  expression,  and  moreover,  of  reviewing  vocabulary  repeatedly  and  increasing  in  the  frequency  of  encounter. 

Taking  account  of  what  has  been  said  above,  we  designed  a distributed  notebook  system  for  vocabulary 
learning.  This  system  consists  of  three  components: 

1)  WWW  as  information  resources  and  learning-materials 

2)  Electronic  dictionaries  on  network 

3)  Vocabulary  notebook  connected  with  database  on  network. 

This  vocabulary  notebook  (See  Fig.  1)  includes  some  checkbox  of  choice  (noun,verb,adj,adv....)  for  concentrating 
learner's  awareness  at  grammatical  characteristics  of  the  word.  It  also  includes  some  textfields  for  writing  the 
word,  its  meaning,  the  sentence  containing  the  word  the  sentence  found  in  the  dictionary,  new  sentence  containing 
the  word. 

2.2  Learning  process  of  when  using  this  system 


Fig.  2:  Learner's  learning  process  of  when  to  use  the  notebook  system 


Learner’s  learning  process  of  when  using  this  system  (See  Fig.2)  can  be  performed  the  following  three  steps. 
Repeated  these  steps,  you  can  be  getting  acquired  from  receptive  vocabulary  to  productive  vocabulary. 

Stepl : When  you  find  a new  word  in  WWW  and  other  resources  that  you  are  reading. 

(System)  open  the  dictionary,  if  necessary.  Look  carefully  new  word  in  the  sentence  and  judge  whether  it  is  noun, 
or  verb,  adj,  adv...  Mark  the  grammatical  characteristic  of  the  word  on  the  check  box  of  vocabulary  note.  Next, 
write  in  vocabulary  note  the  word,  its  meaning,  the  sentence  containing  the  word  in  WWW  and  resources,  the 
sentence  found  in  the  dictionary,  new  sentence  containing  the  word.  Then,  push  the  save  button  and  these  informa- 
tion are  stored  in  remote  database.  As  URL  of  WWW  and  frame  number  of  the  dictionary  are  also  stored  with  time 
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stamp  into  database,  you  can  easily  see  these  resources  again. 

Step  2:  When  you  find  the  word  again  you  have  seen  before. 

Though  you  know  the  word  as  receptive  vocabulary,  as  you  need  to  learn  other  expressions  of  the  word  and  appro- 
priateness, open  the  vocabulary  note.  Vocabulary  note  automatically  counts  the  frequency  of  encounter  of  the 
word.  The  frequency  can  motivate  you  to  fix  the  word  in  your  memory.  (System)  open  the  dictionary,  if  necessary. 
It  is  also  possible  for  you  to  open  some  dictionaries  at  the  same  time.  Look  carefully  the  word  in  the  sentence, 
concentrating  on  which  words  it  is  usually  associate  with  . You  find  that  the  word  is  another  kind  of  part  of  speech. 
Mark  the  grammatical  characteristic  on  the  check  box  of  vocabulary  note  , and  add  another  meaning,  another 
sentence  containing  the  word  in  resources,  another  sentence  found  in  the  dictionary  and  make  new  sentence  con- 
taining the  word  yourself  Save  them  and  these  information  are  added  to  database. 

Step.3:  When  you  need  to  know  the  expressions  of  the  word 

When  writing  essay,  you  suddenly  become  want  to  know  the  meaning  of  a word  and  expressions.  Look  at  the  list 
presented  from  your  database  and  look  for  an  appropriate  word  and  expressions.  If  necessary,  search  dictionaries 
and  compare  with  them.  You  will  find  the  appropriate  expression  in  the  context. 

If  you  find  a new  expression  in  another  resources,  add  them  to  database  again. 

This  system  provides  learner  with  many  chances  to  remember  the  word:  for  example,  letting  learner  judge 
grammatical  characteristic  of  word,  and  write  a new  expression,  reconfirming  other  expressions  that  have  seen 
before,  and  moreover,  showing  frequency  of  encounter,  showing  window(vocabulary  note  and  dictionary)  as  the 
need  arises. 


3 System  architecture 


Fig.  3:  System  architecture  and  components 

This  distributed  notebook  system  (See.  Fig  3)  consists  of  two  parts  functionally:  (1)  the  Notebook  system 
part,  which  is  connected  with  remote  and  local  database;  (2)  the  Dictionary  system  part.  The  notebook  part  is 
further  divide  into  three  components:  the  database  for  storing  vocabulary  and  data,  the  server,  and  the  client  for 
inputting  new  words  and  showing  data.  As  Fig.  3 shows,  there  are  two  kind  of  database:  SQL  database  and  simple 
Hashtable  database.  These  databases/servers  can  be  selected  with  easy  operation,  according  to  the  user's  purpose 
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and  environment.  On  the  other  hand,  the  client  consists  of  two  parts:  the  notebook  client  and  the  dictionary  client, 
which  are  integrated  with  each  other  (See  Fig.4).  The  interface  of  notebook  client  is  designed  as  card  type  of  GUI 
(See  Fig. 2)  in  order  to  input  and  see  new  words  and  data  easily.  Since  it  also  provides  user  with  flexible  operation 
that  user  are  able  to  open  any  client  window  s at  the  same  time,  user  can  work  efficiently,  comparing  one  window 
with  another.  Dictionary  client  is  connected  to  the  dictionary  server  by  Java  socket  interface.  The  dictionary 
server  system  (Dserver)  is  available  for  CD-ROMs  in  EPWING.  Dictionary  client  can  switch  any  dictionaries  from 
one  to  another  and  also  use  them  at  the  same  time.  The  shown  data  on  the  dictionary  client  can  be  handled  with 
simple  operation:  Copy  and  Paste.  Thus  user  can  paste  a sentence  from  dictionary  into  vocabulary  notebook.  In 
addition  to  these  two  functions  mentioned  above,  this  system  provides  a function  of  accessing  information  stored 
the  database  via  WWW  browser  (See  Fig.  4 below) . 


Fig.  4:  Data  exchange  between  client  cards 


Fig.  5:  Example:  a use  of  notebook  system 


Since  this  notebook  system  is  implemented  in  Java,  all  system  components  (database,  server  and  client)  can 
work  on  different  operating  system  (Mac,  Windows,  and  Unix).  Since  whole  system  is  also  made  by  Java  RMI 
technologies,  it  is  possible  to  arrange  or  rearrange  the  working  position  of  system  components  on  the  network. 
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according  to  user’s  the  purpose/environment  of  use.  For  example,  database  and  server  component  can  be  put  on 
Unix  system.  Notebook  client  can  be  put  on  Macintosh  system,  dictionary  client  on  Windows  (See  Fig  5).  That  is, 
it  is  possible  to  put  each  system  component  on  different  operating  systems.  (Actually,  we  always  use  this  system 
like  this) 

This  distributed  notebook  system  provides  learners  with  the  new  studying  environment,  which  frees  learn- 
ers from  the  restriction  of  studying  location  and  using  systems.  Because  of  the  system  being  unrestricted  from  the 
working  platforms,  learners  can  use  the  system  not  only  computer  in  the  school,  but  also  their  own  computer  in  the 
home.  This  means  that  after  the  classroom  learners  can  continue  to  use  the  same  system  and  continue  to  study  in 
their  own  house. 

This  system  can  be  applied  not  only  for  learner  to  study  vocabulary  as  self-learning,  but  also  for  teachers  to 
use  as  group  ware  in  classroom.  If  user  change  the  interface  of  vocabulary  learning  into  that  of  another  purpose, 
this  system  can  be  applied  to  different  fields.  For  example,  a notebook  for  collecting  information,  glossary  of 
technical  terms,  and  collaborative  tool. 

4.  Future  work 

We  now  develop  a new  system  which  connects  the  distributed  notebook  system  for  vocabulary  learning 
with  small  PDA  terminal.  This  future  system  will  make  it  possible  for  learners  to  study  the  vocabulary  from  the 
mobile  environment.  In  Japan,  it  generally  takes  a lot  of  time,  approximately  one  hour  or  more,  to  commute  from 
home  to  school/company.  Many  students  always  spend  such  long  time  learning  something  (ex.  learning  language) 
in  bus  and  train.  Thus  the  learning  system  with  PDA  will  be  very  useful  for  them. 

On  the  other  hand,  voice  memo  system  with  small  IC  recorder  is  available  on  the  market.  We  think  such 
equipment  with  audio  tones  will  be  also  useful  for  vocabulary  learning.  We  also  now  try  to  integrate  this  voice 
memo  with  our  notebook  system.  Moreover,  in  the  near  future,  the  integrated  system  will  be  combined  the  sound 
tool  for  editing  speech  signal  called  ’’Sound  Cutter”  (Sagisaka  and  Munakata  1999)  which  we  developed  last  year. 
As  the  Sound  Cutter  is  also  written  in  Java.  Our  goal  is  to  realize  the  distributed  environment  for  language  learning, 
which  includes  not  only  text  but  also  sound. 

5.  Conclusion 

In  this  paper,  a distributed  notebook  system  for  vocabulary  learning  was  introduced.  This  system  was 
developed  for  vocabulary  learning,  making  good  use  of  network  and  Java  technologies.  It  also  provided  learners 
with  new  learning  environment,  which  freed  learners  from  restriction  of  place  and  time,  and  moreover  defferent 
platforms.  This  system  can  be  applied  to  different  fields,  if  user-interface  is  changed  for  different  purpose  of  use. 
Further  development  of  this  system  will  bring  language  learners  more  useful  environment. 
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Abstract:  In  this  paper,  we  describe  the  development  of  an  Organizational  Learning  and 
Information  Support  System  (OLISS)  that  serves  a diverse  group  of  users  for  a wide 
variety  of  purposes.  Designed  for  a government  agency,  the  system  is  developed  to 
support  the  application  of  ’’standard”  business  policies  and  practices  across  the 
organization.  The  system  provides  "just  in  time"  capabilities  for  users  of  the  system  to 
apply  these  standard  business  policies  and  practices  within  the  ongoing  context  of  their 
everyday  work.  Users  come  to  the  system  from  different  jobs,  experience  levels  in  their 
jobs,  and  knowledge  of  the  standard  business  policies  and  practices.  All  the  capabilities, 
in  one  form  or  another,  provide  not  only  a mechanism  for  individual  learning  but  also  a 
means  for  capturing  and  disseminating  newly  created  knowledge  in  the  organization. 


Introduction 

The  growth  and  sharing  of  knowledge  is  recognized  as  an  important  element  in  becoming  a “learning  organization” 
(Easterby-Smith,  1997;  Marsick  and  Watkins,  1994;  Senge,  1990).  However,  what  is  missing,  according  to  many 
researchers  and  practitioners  in  the  field,  is  the  development  of  theoretical  frameworks  for  thinking  about  how 
people  learn  and  perform  in  an  organization  (Raybould,  1995).  Furthermore,  the  few  theoretical  frameworks  that 
have  been  developed  and  presented  for  consideration  are  highly  abstract  and  lack  a description  of  a practical 
mechanism  for  the  capture  and  dissemination  of  organizational  learning.  To  address  this  situation,  this  paper  uses  a 
case  study  to  describe  a theoretical  framework  with  a practical  mechanism  for  capturing  and  disseminating  learning 
at  the  organizational  level.  The  theoretical  framework  includes  a knowledge  creation  process  that  can  be  used  to 
grow  and  share  knowledge  -t  an  essential  part  of  becoming  a learning  organization.  The  framework  relies  on 
technology  — a Web-based  Organizational  Learning  and  Information  Support  System  (OLISS)  — to  assist  in  the 
capture  of  expertise  from  individuals  and  make  it  available  to  other  members  of  the  organization. 

In  the  following  paragraphs,  we  describe  how  the  capabilities  of  a Web-based  OLISS  serve  a diverse  group  of  users 
for  a wide  variety  of  purposes.  Designed  for  a government  agency,  the  system  is  developed  to  support  the 
application  of  "standard"  business  policies  and  practices  across  an  organization.  It  is  performer  centered  (Laflfey, 
1995)  and  its  use  applies  the  principles  of  situated  cognition  (Brown,  Collins,  & Duguid,  1989;  Laffey,  1995).  That 
is,  it  supports  "just  in  time"  capabilities  for  end-users  to  apply  standard  business  policies  and  practices  within  the 
ongoing  context  of  their  everyday  work.  All  the  capabilities,  in  one  form  or  another,  provide  not  only  a mechanism 
for  individual  learning  but  also  a means  for  capturing  and  disseminating  newly  created  knowledge  in  the 
organization.  This  continuing  knowledge  creation  and  capture  process  becomes  an  organizational 
performance/leaming  cycle  (Raybould,  1 995)  as  members  access  expertise  in  the  system,  modify  that  expertise, 
apply  it  to  the  problem  at  hand,  and  add  their  learning  to  the  system.  In  this  cycle,  the  Web-based  OLISS  is  an 
enabling  technology  that  provides  the  link  between  individual  learning  and  organizational  learning  (Kim,  1 993). 


The  OLISS  Capabilities 

In  Figure  1,  we  list  the  major  capabilities  of  the  Web-based  OLISS  and  the  categories  of  end-users  for  the  system. 
End-users  are  "members,"  "managers,"  and  "customers."  Members  are  members  of  the  organization  and  are  directly 
responsible  for  correctly  applying  the  business  policies  and  practices  on  an  everyday  basis.  Managers  are 


supervisors  of  the  members  that  apply  the  policies  and  practices  but  managers  do  not  use  them  as  part  of  their  own 
daily  work.  Managers  are,  however,  responsible  for  the  correct  application  of  the  policies  and  practices  by  the 
members  of  the  organization  that  they  supervise.  Customers  take  possession  of  the  products  that  result  from  the 
application  of  the  policies  and  practices  by  the  members  of  the  organization.  Customers  have  a vested  interest  in  the 
correct  application  of  these  policies  and  practices  since  it  affects  the  quality  of  the  product  that  they  receive. 

A checkmark  in  the  intersection  between  the  row  of  a capability  and  the  column  of  an  end-user  category  indicates 
that  it  is  a capability  designed  for  that  type  of  end-user.  For  example,  in  the  second  row  of  the  reference  materials 
capability,  related  materials,  there  is  a checkmark  for  the  members  category  and  no  checkmarks  for  the  categories  of 
managers  or  customers.  This  means  that  members  of  the  organization  will  use  the  related  materials  but  managers 
and  customers  will  probably  not  use  them.  Managers  and  customers  will  probably  get  all  the  reference 
documentation  that  they  will  require  by  accessing  the  procedure  & policy  documentation  on  the  OLISS  Web  site. 

We  believe  that  having  online  reference  materials  is  an  important  capability  for  a Web-based  OLISS  site.  As  Figure 
1 shows,  all  end-users  will  benefit  from  easy  access  to  online  documentation  describing  the  policies  and  practices. 
Organizational  members,  however,  will  also  benefit  from  the  easy  access  and  cross-reference  of  related  online 
materials.  It  will  help  them  to  ensure  that  the  application  of  the  policies  and  practices  are  consistent  with  the  other 
regulations  that  they  must  follow.  Managers  and  customers  do  not  benefit  as  much  from  access  to  these  related 
materials. 

Figure  1 also  shows  that  the  search  capability  provides  an  important  service  to  all  end-users  of  the  OLISS  Web  site. 
It  allows  a free-form  text  search  where  end-users  can  input  keywords  or  phrases  to  the  search  engine  and  receive 
references  to  all  the  pages  on  the  site  that  match  the  search  criteria.  It  provides  quick  access  to  materials  that  relate 
to  specific  topics. 
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Figure  1.  Capabilities  of  a Web-based  Organizational  Learning  and  Information  Support  System 
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The  communication  capability  of  the  OLISS  Web  site  has  three  features.  The  first  one  is  a "points  of  contact"  e- 
mail  feature  - allowing  users  to  contact  experts  within  a subject  area.  The  second  feature  is  a threaded  discussion 
list  that  allows  users  to  follow  ongoing  discussions  of  those  issues  that  concern  them.  The  third  feature  provides  a 
means  to  collect  frequently  asked  questions  into  a "FAQ"  list. 

As  shown  in  Figure  1,  these  communication  features  extend  the  methods  available  for  end-users  of  the  OLISS  Web 
site  to  grow  and  share  knowledge.  Members,  however,  will  benefit  the  most  from  using  the  features.  Previously, 
members  communicated  only  through  face-to-face  meetings,  telephone,  or  mail.  Now,  members  increase  their 
productivity  by  sharing  with  one  another  about  lessons-leamed  in  conversations  that  are  easily  assessed,  organized, 
and  not  bound  to  specific  time  frames. 

Instruction  is  an  important  capability  in  the  design  of  our  OLISS  Web  site.  In  Figure  1,  we  see  that  members  of  the 
organization  will  benefit  from  all  the  instruction  that  is  available  --  instructor  led  and  stand-alone.  One  advantage  of 
instructor  led  instruction  for  members  is  that  it  provides  an  opportunity  to  ask  questions  concerning  specific  details 
about  applying  the  policies  and  practices.  Managers  and  customers,  in  general,  can  meet  their  needs  with  stand- 
alone instruction. 

The  decision  support  capability  for  the  OLISS  Web  site  has  two  types  of  features  that  benefit  organizational 
members  the  most.  The  first  one  provides  a guided  search  option  for  policies  and  practices.  The  second  feature 
provides  guidance  in  the  completion  of  the  steps  for  a policy  or  practice.  These  features  facilitate  knowledge 
transfer  in  the  organization.  Experienced  members  provide  the  expertise  for  the  decision  support  capability. 
Members  with  less  experience  use  the  embedded  expertise  to  find  and  complete  the  policies  and  practices. 

Managers,  with  less  detailed  knowledge  of  the  policies  and  practices  may  also  benefit  from  use  of  the  expertise  in 
their  supervisory  role. 

The  knowledge  acquisition  capability  of  the  OLISS  Web  site  also  has  two  types  of  features  that  mostly  benefit  the 
members  of  the  organization.  One  feature  is  a means  to  organize  and  store  case  studies.  Case  studies  are  general 
examples  that  illustrate  the  correct  application  of  a policy  or  practice.  Most  of  the  time  they  are  created  for  use  as 
examples  during  instruction.  The  other  feature  enables  the  organization  to  store  "real  life"  examples  that  illustrate 
the  correct  application  of  policies  and  practices  in  the  field.  With  these  two  features,  the  knowledge  acquisition 
capability  facilitates  the  capture  and  sharing  of  new  knowledge  as  it  is  created  in  the  organization.  Managers,  who 
are  not  creating  new  knowledge  from  direct  application  of  the  policies  and  practices,  may  improve  their  decision- 
making through  access  to  this  expertise. 


Example  Scenario  with  the  System 

In  the  following  paragraphs,  we  describe  how  organizational  members  may  use  the  capabilities  of  an  OLISS  Web 
site  to  capture  and  disseminate  knowledge  across  the  organization.  In  this  case  study,  other  categories  of  end-users 
(e.g.,  managers,  customers,  etc)  typically  do  not  create  new  knowledge  but  access  the  knowledge  created  by  the 
members  of  organization  that  directly  apply  the  business  policies  and  practices  in  their  daily  work. 

In  the  following  scenario,  shown  in  Figure  2,  the  organizational  member  is  not  sure  exactly  what  policy  or  practice 
they  are  looking  for,  but  the  member  has  a vague  idea  what  it  is  and  where  in  the  larger  work  process  it  is  used.  For 
example,  a member  knows  that  he  or  she  will  need  to  complete  a change  document  that  describes  the  changes  that 
have  been  done  to  a design  document  but  the  end-user  is  not  sure  what  kind  of  change  document  should  be 
completed.  The  organizational  member  begins  looking  for  change  documents  by  clicking  on  a menu  selection  that 
brings  up  the  entire  process  of  producing  a product.  The  end-user  then  selects  the  sub-process  that  contains  the 
design  document.  The  member  clicks  the  design  document  option  and  is  given  a list  of  two  selections  for  design 
change  documents  (change  document  A and  change  document  B).  Here,  the  OLISS  Web  site  prompts  the  member 
with  the  following  guidance  to  assist  in  determining  which  change  document  to  complete:  "If  you  have  not  released 
the  design  document,  then  complete  change  document  A,  else  (you  have  released  the  design  document)  complete 
change  document  B." 

After  selecting  change  document  A,  the  member  now  sees  a description  of  change  document  A on  the  screen.  Along 
the  sides  of  the  screen  are  menu  selections  for  the  other  capabilities:  reference,  instruction,  knowledge  acquisition. 
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and  communication.  The  member  selects  the  reference  materials  option  and  views  all  the  references  in  the  related 
design  documents  that  refer  to  change  document  A.  Afterwards,  the  member  clicks  on  the  instruction  menu  and 
decides  to  take  a five-minute  tutorial  on  change  documents. 

Next,  the  member  clicks  on  the  knowledge  acquisition  menu,  selects  examples,  and  the  OLISS  Web  site  provides  a 
list  of  example  groupings.  The  member  clicks  on  one  of  the  groupings  and  sees  a list  of  examples  (additional  levels 
of  groupings  can  be  used  to  differentiate  examples).  The  metaknowledge,  used  to  group  the  examples  into 
meaningful  categories,  provides  guidance  for  members  to  find  examples  that  closely  relate  to  the  task  at  hand.  The 
member  examines  these  examples  and  finds  an  example  of  change  document  A that  is  very  similar  to  the  one  that 
the  member  will  have  to  create.  Finally,  with  one  question  left  unanswered,  the  member  clicks  on  the 
communication  menu,  selects  "points  of  contact,"  and  posts  the  question  to  another  member  of  the  organization. 
When  the  answer  is  returned,  the  member  has  all  the  knowledge  that  he  or  she  requires  for  developing  a solution  to 
the  current  problem,  i.e.,  create  a change  document  A.  Lastly,  the  member  updates  the  system  by  uploading  the  new 
solution;  thereby,  creating  a new  unique  example  that  will  be  available  during  the  next  time  the  system  is  accessed. 


Discussion 

Does  the  case  study  presented  in  this  paper  provide  any  insight  into  how  organizations  can  use  technology  to  support 
the  process  of  creating  new  knowledge?  Or  how  organizations  can  make  knowledge  explicit  once  it  has  been 
created?  And  finally,  how  they  can  convert  knowledge  into  different  forms  for  effective  dissemination  across  the 
organization? 

What  has  been  retrieved  from  the  OLISS  Web  site  in  the  scenario  is  what  Nonaka  and  Takeuchi  (1995)  describe  as 
“explicit  knowledge”  - a meaningful  set  of  information  articulated  in  clear  language  including  numbers  or 
diagrams.  Note  that  the  member  did  not  find  a complete  solution  from  the  OLISS  Web  site  — requiring  no 
modification  — to  his  or  her  current  problem  in  the  example  scenario.  The  member  found  an  example  with  guidance 
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from  knowledge  supplied  by  the  system,  gained  knowledge  through  reference  materials  and  instruction,  then 
modified  the  example  to  create  a unique  and  new  solution  for  his  or  her  current  problem. 

This  new  solution  is  the  creation  of  knowledge;  however,  this  new  knowledge  is  tacit  and  cannot  be  used  by  anyone 
since  it  resides  in  the  head  of  an  individual.  It  is  knowledge  that  needs  to  be  made  explicit.  Furthermore,  it  needs  to 
be  converted  to  several  types  of  knowledge  before  it  can  be  effectively  disseminated  across  the  organization.  The 
first  step  is  that  the  solution  itself  needs  to  become  an  example  for  the  next  time  the  member,  or  another  member,  is 
confronted  with  a similar  problem.  To  accomplish  this  technically,  an  online  form  is  filled  out  and  the  new  solution 
is  attached,  which  is  then  uploaded  to  the  OLISS  Web  site.  The  solution  becomes  a form  of  explicit  knowledge; 
what  was  implicitly  known  by  only  one  individual  is  now  available  in  an  explicit  form  for  the  other  members  of  the 
organization.  The  process  of  placing  the  new  example  in  the  correct  category  of  examples,  or  creating  a new 
category  for  the  example,  is  the  creation  of  new  metaknowledge  - knowledge  about  how  to  find  the  new  example. 
This  is  the  knowledge  that  is  traditionally  throvsm  away.  Organizations  may  keep  the  new  example  but  later  don’t 
know  under  what  conditions  it  should  be  used. 

The  second  step  is  to  ask  if  the  other  members  of  the  organization  will  use  the  new  knowledge  often.  If  the  answer 
is  yes,  then  it  may  be  analyzed  and  converted  to  other  types  of  knowledge  that  are  represented  in  the  OLISS  Web 
site.  For  example,  the  new  solution  may  be  analyzed  to  determine  what  steps  are  necessary  to  arrive  at  the  new 
solution.  These  steps  are  documented  as  procedural  knowledge  and  placed  in  the  online  reference  materials. 
Furthermore,  if  the  new  solution  will  be  frequently  used,  then  this  procedural  knowledge  may  also  be  placed  in  the 
decision  support  capability  to  guide  members  to  the  new  solution.  Finally,  if  the  new  solution  represents  a new 
approach  to  conceptualize  problems,  then  it  would  be  analyzed  for  the  general  truths  and  associations  that  describe 
this  new  approach.  The  resulting  declarative  knowledge  would  form  the  basis  for  creating  new  online  instructional 
materials  for  inclusion  in  the  OLISS  Web  site.  In  this  way,  the  OLISS  Web  site  supports  the  process  of  creating 
new  knowledge,  making  it  explicit,  and  converting  the  knowledge  into  different  forms  for  effective  dissemination 
across  the  organization.  It  becomes  the  mechanism  that  provides  the  rapid  capture  and  dissemination  of 
organizational  learning  that  Raybould  (1995)  describes  in  the  fifth  phase  of  his  Organizational 
Performance/Leaming  Model. 
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Abstract:  The  development  of  online  coiirses  is  becoming  increasingly  popular.  In  addition  to  the 
multimedia  benefits  that  the  WWW  offers,  institutions  can  enhance  existing  courses  using  WWW 
technologies  such  as  chat  and  discussion  tools.  These  tools  can  allow  students  from  different 
institutions  to  participate  in  groiq)  work  which  is  pedagogical ly  sound  as  well  as  a valuable  cultural 
experience.  On-line  collaborative  courses  are  an  ideal  way  to  expose  students  to  different  cultural 
perspectives. 

This  paper  discusses  the  development  and  implementation  of  an  online  collaborative  course  used  to 
teach  advanced  international  business  at  undergraduate  level  at  Auckland  University  of  Technology. 
The  paper  discusses  the  nature  of  the  collaborative  exercises  undertaken  by  the  students  and 
concludes  with  a reflective  evaluation  of  the  course. 


Introduction 

The  Bachelor  of  Business  (BBus)  is  a four  year  undergraduate  degree  in  business  which  has  seven 
majors,  including  International  Business.  It  is  offered  by  the  Auckland  University  of  Technology, 
Auckland,  New  Zealand. 

The  aim  of  the  BBus  is  to  develop  confident  capable  learners  who  have  a broad  understanding  of 
business  and  of  the  relationships  between  business  disciplines,  specialist  knowledge  in  one  or  more 
fields  of  business,  and  personal  and  professional  capabilities  in  communication,  team  work, 
interpersonal  skills,  critical  thinking,  research  and  use  of  technology. 

The  approach  to  learning  in  the  BBus  is  student-centred  and  reflects  a philosophy  that 
knowledge  and  understanding  are  constructed  by  the  learner,  not  supplied  by  the  teacher.  Students 
learn  through  action,  interaction  and  reflection  (Ramsden  1992).  The  teacher  is  a facilitator  of  learning 
as  students  interact  with  each  other,  develop  ideas  from  their  reading  and  discussion  and  work  with 
peers. 


Computer  Mediated  Communication 

Studies  show  that  classroom  learning  improves  significantly  when  students  participate  in  structured 
learning  activities  within  small  groups  (Brown  & Palincsar  1989).  The  educational  value  of  the 
collaborative  process  has  led  to  the  development  of  computer-supported  collaborative  learning  (CSCL) 
tools  (Lund,  Baker  & Baron  1996;  McManus  & Aiken  1993).  These  tools  enrich  learning  in  a setting 
that  encourages  students  to  communicate  with  each  other  (Goodman  et  al.  1998). 

Studies  of  computer  mediated  communication  and  computer  conferencing  (Harasim  1995),  identify 
benefits  to  students  such  as  convenience  and  flexibility.  Higher  levels  of  student-student  interaction 
and  of  teacher-student  interaction  have  been  reported  and  linked  to  the  electronic  environment.  It  is 
perceived  as  less  intimidating  and  allowing  more  equal  participation.  Other  benefits  include  improved 
learning  through  the  opportunity  to  reflect,  especially  critically,  on  material  and  discussions  and  to 
develop  understanding  through  sharing  diverse  views,  collaborating  in  writing  and  re-thinking  and  re- 
writing one’s  ideas.  Disadvantages  include  technical  difficulties  in  gaining  access,  the  extra  time  it  may 
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take  compared  to  face  to  face  classes,  the  lack  of  the  social,  visual  and  aural  cues  which  assist 
discussion,  lack  of  immediate  feedback  in  asynchronous  communication,  dealing  with  information 
overloads  and  tracking  the  flow  of  discussions. 

The  way  in  which  computer  mediated  communication  is  included  in  the  structure  of  learning  and 
teaching  in  the  course  is  important,  (Savard  1995)  and  (Ruberg  et  al,  1996),  report  that  using  computer 
mediated  communication  does  not  necessarily  lead  to  increased  student  participation  or  interaction 
between  students.  They  argue  for  the  importance  of  integrating  computer  mediated  communication 
activities  with  the  rest  of  the  course,  especially  in  the  assessment  programme  ensuring  that  students 
perceive  its  benefits.  (Tagg  1994)  reports  on  a postgraduate  course  where  a change  to  having  students, 
rather  than  the  teacher,  facilitate  discussions  and  summarise  results  encouraged  students  to  be  more 
active  in  discussions  and  eliminated  some  of  their  reluctance  to  subject  their  ideas  to  critique  and 
questioning  by  other  students. 


International  Business  Major 

The  international  business  major  of  the  BBus  aims  to  provide  the  BBus  graduate  with  a broad  set  of 
cross-cultural  capabilities  developed  through  international  experiences.  Working  with  students  from 
other  countries  challenges  students’  ideas,  values  and  judgements  and  enables  them  to  understand  and 
develop  sensitivity  to  different  cultures,  respect  for  difference  and  flexibility  in  working  with  people 
from  different  cultures. 

Increasingly  international  business  requires  effective  use  of  communications  technologies.  These 
include  email,  for  discussion,  establishing,  negotiating  and  monitoring  contacts,  contracts  and  other 
business  interactions.  The  use  of  computer  mediated  communication  for  collaborative  team  projects 
can  involve  people  from  the  same  or  different  organisations  working  in  different  countries  and  time 
zones.  The  ability  to  use  these  technologies  to  achieve  organisational  goals  in  the  complex  cross- 
cultural  e-business  environment  is  a key  to  successful  international  business  in  the  future. 

Advanced  International  Business  (AIB)  is  the  capstone  course  in  the  BBus  major  in  International 
Business.  The  course  aims  to  examine  theoretical  and  practical  problems  in  international  business 
through  multi-disciplinary  perspectives.  Planning  of  the  online  development  was  carried  out  in 
Semester  1 (February-June),  1999.  The  first  stage  was  then  implemented  in  July  1999, 

During  the  semester  three  forms  of  computer  mediated  communication  were  used.  They  were: 

• Online  participation  in  synchronous  (real  time)  class  chat  discussions, 

• Leading  a threaded  discussion  using  an  asynchronous  discussion  tool  (BSCW)  with  a group  of 
five  other  students. 

• AUT  students  working  asynchronously  in  pairs  with  students  from  International  Business 
Academy,  Kolding,  Denmark  to  jointly  analyse  an  International  Business  case  and  write  a 
report. 

In  addition  to  the  online  work,  AUT  students  had  one  scheduled  class  of  two  hours  per  week  for 
classroom  activities.  These  included  case  analyses,  class  and  group  discussions  and  individual  and 
group  presentations.  Students  are  accustomed  to  and  expect  that  their  learning  in  the  BBus  will  be 
interactive  and  involve  group  work.  What  we  were  adding  was  experience  in  carrying  on  such  activities 
online  using  asynchronous  and  synchronous  communication  tools. 


Synchronous  chat 

During  the  course  there  were  two  online  chat  sessions  to  enable  the  class  to  participate  in  discussions 
on  international  business  issues.  These  were  not  assessed  as  part  of  the  students’  grades  but  intended  to 
assist  their  learning,  develop  skills  in  online  synchronous  communication  and  identify  students  who 
had  difficulty  with  the  electronic  environment  so  that  they  could  be  given  help  before  they  undertook 
assessment  in  online  work. 

Electronic  Discussion  Leadership 


During  the  first  week  of  the  semester,  each  student  identified  an  international  business  issue  relevant  to 
the  course.  The  teacher  clustered  these  into  groups  of  three  to  five  ensuring  that  each  group  had  a 
range  of  different  issues.  Over  five  weeks  students  took  turns  as  online  discussion  leader,  introducing  a 
topic,  managing  the  different  discussion  threads,  motivating  group  members  to  participate,  commenting 
on  group  members’  contributions,  developing  depth  of  analysis  in  the  discussion,  summarising  and 
drawing  conclusions. 

Leadership  of  the  electronic  discussion  comprised  30%  of  the  course  grade.  The  students  submitted  a 
report  of  the  discussion,  included  a transcript  of  the  discussion  and  a statement  in  which  each  student 
reflected  critically  on  the  content  and  process  of  the  discussion  which  they  led. 


Case  Analysis  Report 

The  case  report  was  prepared  in  pairs  - one  student  from  IB  A in  Denmark  and  one  from  AUT.  The 
students  undertook  this  collaborative  assignment  in  the  context  of  their  different  courses,  using  agreed 
case  materials  and  guidelines  for  writing  a case  analysis  report.  It  required  students  to  manage  a cross- 
cultural  group  task,  deal  with  communication  and  technology  issues  and  negotiate  with  another  student 
a report  on  a complex  international  business  case. 

The  case  analysis  report  comprised  35%  of  the  final  course  grade  and,  like  the  e-discussion,  included  a 
reflective  statement  evaluating  the  content  and  process  of  the  collaborative  work. 


Evaluation 

Student  evaluation  of  the  course  and  the  online  components  was  conducted  through  surveys  at  the 
beginning  and  end  of  the  semester,  an  in-class  discussion  on  the  chat  sessions  and  the  reflective 
statements  submitted  with  the  e-discussion  and  case  analysis.  The  results  are  set  out  below. 


Beginning  of  the  semester 

The  AIB  class  in  semester  2 1999  comprised  21  students  in  their  third  or,  more  typically,  fourth  year  of 
undergraduate  study  in  the  BBus.  Nineteen  students  were  aged  20-24  with  two  students  aged  18-19. 
There  were  fourteen  women  and  seven  men.  At  the  beginning  of  the  semester  students  were  asked 
about  their  level  of  comfort  with  electronic  communication  and  expectations  of  the  course.  All  but  two 
of  the  class  reported  that  they  were  either  comfortable  or  very  comfortable  with  the  use  of  email  and 
approximately  one  quarter  had  experience  with  chat  groups.  All  students  but  one  had  used  the  WWW 
for  study  or  leisure. 

All  students  were  excited  by  the  online  collaboration  and  saw  in  it  the  opportunity  to  develop  effective 
communication  and  cross-cultural  skills  relevant  to  e-business  and  the  changes  in  the  international 
business  environment.  Women  were  more  likely  to  identify  the  development  of  communication  skills 
and  cross-cultural  capabilities.  Three  particularly  expressed  interest  in  learning  to  communicate  and 
achieve  group  goals  without  the  usual  visible  cues  such  as  facial  expression,  body  language. 


Online  chat 

The  students  were  positive  about  the  online  chat  sessions.  Students  felt  that  more  of  them  were  able  to 
participate  in  the  discussions  and  as  a result  the  discussions  included  a wider  range  of  ideas  than  a face 
to  face  classroom  discussion.  They  felt  freer  to  put  their  ideas,  there  was  more  input  from  students,  and 
it  there  was  less  guidance  from  the  teacher. 

Concerns  were  expressed  that  the  sessions  went  too  fast,  students  could  not  always  keep  up,  there  was 
no  time  to  reflect  and  at  times  there  were  private  conversations  going  on  as  well  as  the  class  discussion. 
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In  discussion  about  how  to  improve  the  chat  sessions,  students  agreed  that  each  entry  into  the  chat 
should  contribute  to  both  the  content  and  the  process  of  the  discussion.  They  felt  that  this  would  slow 
down  the  discussion,  reduce  casual  chatter  and  lead  to  more  thoughtful  replies. 


E-Discussions 

Students’  comments  on  the  e-discussions  were  similarly  positive.  These  comments  were  made  in  the 
reflective  statements  submitted  by  each  student  and  in  the  end  of  semester  evaluation.  All  students  said 
e-discussions  were  better  than  face  to  face  discussions.  They  found  facilitating  the  discussions  was  fun, 
interesting  and  motivating. 

Students  reported  higher  levels  of  motivation  and  enjoyment  than  with  traditional  group  assignments. 
They  felt  that  they  learned  more.  Some  students  felt  that  the  electronic  format  gave  them  more  control 
over  their  learning  as  they  set  the  direction  of  discussion  and  managed  others.  One  student  however, 
felt  uncomfortable  with  this  and  wanted  some  supervision  by  the  teacher  of  the  discussions. 

There  was  more  time  to  read,  think  and  reflect  on  the  comments  of  others  before  responding.  Students 
felt  they  were  more  careful  in  writing  their  submissions  and  responses  to  ensure  that  they  clearly 
communicated  their  ideas.  They  had  a record  of  the  discussion.  One  student  commented  that  he 
learned  better  this  way  because  he  was  writing  down  his  ideas  more  clearly  so  that  others  could 
understand,  then  he  received  comments  back  from  other  students  which  assisted  his  understanding. 

Students  said  that  the  lack  of  body  language  and  social  cues  meant  it  was  harder  to  check 
understanding  and  move  the  discussion  along.  Students  reported  that  they  could  not  always  get  access 
to  the  discussion  site,  email  messages  were  lost  and  they  sometimes  resorted  to  the  telephone  to  keep 
the  discussion  going.  The  most  commonly  reported  technical  concern  related  to  frequency  of  logging- 
in  and  slowness  of  the  process.  Students  had  to  wait  for  a response  and  did  not  know  when  that  would 
come  so  they  had  to  log  in  frequently.  If  they  checked  in  too  late  other  students  might  have  responded 
and  they  lost  their  opportunity  to  contribute.  Information  overload  was  an  issue  as  responses  got  longer 
and  managing  the  multiple  discussion  threads  became  difficult. 


Case  Analysis 

The  case  analysis  exercise  provided  the  online  cross-cultural  collaboration  we  considered  important  to 
international  business  graduates.  It  had  an  uncertain  start  as  the  first  business  school  partner  we  had 
arranged,  withdrew  at  the  last  minute.  We  were  very  lucky  that  IBA  Kolding,  Denmark  agreed  to 
participate  at  short  notice.  However,  there  were  difficulties  with  this  arrangement  as  the  IBA  students 
were  not  as  advanced  in  their  international  business  studies  as  the  AUT  students  and  consequently  their 
understanding  of  international  business  issues  was  less  sophisticated. 

AUT  students  commented  that  the  main  benefits  of  collaboration  with  the  Danish  students  were 
receiving  ideas  and  critique  from  a different  cultural  perspective  and  learning  to  deal  with  differences 
in  English  language  and  the  misunderstandings  which  occurred.  One  student  said  that  he  and  his 
partner  avoided  this  problem  by  always  providing  examples  to  illustrate  their  ideas.  Another  observed 
that  because  he  had  to  ensure  the  Danish  student  understood  his  messages,  he  felt  that  he  put  more  time 
into  thinking  them  through  and  keeping  them  as  concise  as  possible.  He  felt  that  this  had  enhanced  his 
learning  from  the  case  and  illustrated  the  benefits  of  peer  tutoring.  Other  students  talked  of  the 
increased  understanding  they  achieved  through  having  to  carefully  explain  and  justify  their  ideas  and 
arguments.  Students  commented  that  they  had  to  learn  to  be  more  patient  with  each  other  and  aware  of 
each  other’s  needs. 

AUT  students  found  the  exercise  more  difficult  and  frustrating  than  they  had  expected  yet  still  rated  the 
exercise  highly  in  its  contribution  to  their  learning.  Time  management  and  differing  levels  of 
participation  by  students  were  obstacles  to  easy  completion  of  the  task.  One  student  observed  that  this 
was  an  issue  with  face  to  face  group  work  so  she  was  not  surprised  to  encounter  it  in  the  electronic 
working  environment.  Another  student  said  she  did  not  want  to  appear  too  pushy  and  risk  losing  the 
contribution  of  her  partner  but  felt  that  she  had  a greater  sense  of  urgency  and  commitment  to  the  task 
because  of  its  significance  in  the  AUT  assessment  programme.  Most  AUT  students  reported  difficulty 
in  contacting  their  Danish  partners  and  in  receiving  responses  which  advanced  the  discussion.  They  felt 


that  the  Danish  students  were  not  as  committed  to  the  exercise.  On  the  other  side  the  Danish  teacher 
reported  to  AUT  that  their  students  thought  the  AUT  students  did  not  seem  to  be  as  hardworking  or 
conscientious  in  applying  themselves  to  the  task. 

AUT  students  said  that  responses  which  were  timely,  not  too  long,  provided  critique  and  new  ideas 
rather  than  constant  agreement  were  best  for  advancing  the  discussion.  They  recommended  that  the 
teams  agree,  at  the  beginning,  the  roles  of  each  student  and  expectations  for  the  communication  and 
work.  Also  attention  should  be  paid  to  creating  rapport  and  attending  to  the  social  side  i.e.  getting  to 
know  the  other  student  not  just  attending  to  the  work.  Showing  your  partner  that  you  valued  their 
opinion  assisted  their  motivation  and  prevented  major  misunderstandings  according  to  one  student. 

Students  experienced  lack  of  control  of  the  group  process  in  the  electronic  environment.  They  observed 
the  importance  of  developing  trust  that  your  partner  would  contribute.  This  was  more  difficult  because 
all  communication  was  remote  and  the  students  could  not  meet. 


End  of  Semester  evaluation 

At  the  end  of  the  semester  students  were  asked  to  complete  a detailed  evaluation  questionnaire. 
Responses  were  received  from  8 students  out  of  21.  This  low  rate  of  return  occurred  because  the 
survey  had  to  be  posted  out  after  the  end  of  the  semester  rather  than  being  handed  out  and  completed  in 
class  time.  However,  the  results  of  the  survey  mirror  comments  in  the  reflective  comments  that  were 
received  from  all  students. 

Students  approved  of  the  electronic  environment,  the  way  in  which  it  was  introduced  and  the 
innovative  and  challenging  nature  of  assessments.  Students  indicated  that  they  were  satisfied  or  very 
satisfied  with  the  module.  In  evaluating  the  contribution  made  to  their  learning  by  the  online  strategies, 
students  were  most  positive  about  the  e-discussion  and  case  analysis.  They  were  less  positive  about  the 
chat  sessions. 

All  the  students  said  they  would  like  more  courses  to  use  electronic  discussion  tools  and  to  be  made 
available  online.  Some  commented  on  the  need  for  training  of  students  so  that  they  can  be  successful. 
Internet  access  at  home  was  another  issue.  Students  commented  that  online  communication  means  less 
time  in  class  and  gives  students  more  flexibility.  All  commented  that  familiarity  with  use  of 
synchronous  and  asynchronous  discussion  tools  was  important  for  their  futures,  in  both  social  contact 
and  work.  One  student  was  concerned  not  to  lose  face  to  face  contact  with  lecturers  and  other  students. 

Discussion  software  was  useful  because  it  made  it  easier  to  organise  discussions,  you  could  see  all 
contributions  at  once  and  students  could  contribute  whenever  they  were  ready  rather  than  having  to 
wait  to  receive  an  email.  Many  students  preferred  email  because  of  its  familiarity  and  ease  of  use. 
Email  did  not  require  them  to  go  to  a different  site  and  log-in  and  it  did  not  crash  as  often  as  did  the 
discussion  software. 


Changes  to  the  course 

As  result  of  our  experience  in  1999  and  student  evaluations  we  have  made  changes  to  the  online  course 

for  2000. 

These  include:- 

• an  improved  look  to  the  online  site  and  navigation 

• synchronous  chat  and  asynchronous  discussion  tools  are  integrated  with  the  site  so  that  students 
only  log-in  once  and  have  all  the  communication  tools  available 

• when  students  first  log-in  they  get  a list  of  new  messages  and  the  week’s  programme  for  their 
course.  This  includes  reading,  online  and  other  activities 

• a messaging  system  has  been  incorporated  into  the  site 

• each  student  has  a personal  journal  for  their  notes  and  work 

• teachers  and  students  may  upload  and  download  files  of  course  materials  or  other  work  on  to  the 
site  for  others  to  access 

• online  help  and  tutorials 
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• students  may  choose  a different  look  or  skin  for  the  site 

• dedicated  technical  support  on  site 

• teachers  are  able  to  monitor  the  progress  of  the  students  and  obtain  reports  on  how  long  students 
spend  on-line,  how  they  use  the  course,  information  on  the  use  of  the  chat  and  discussion  groups. 

• Incorporation  of  a dynamic  roster  (study  guide)  containing  class  preparation,  in  class  activities, 
assessment  and  weekly  reviews. 

The  online  collaboration  in  the  case  analysis  is  being  continued.  We  have  arranged  a different  partner 
in  France  where  the  students  will  be  of  a similar  level  to  those  at  AUT  in  international  business.  The 
1999  pilot  demonstrated  the  importance  of  students  having  a similar  commitment  to  participating  and 
seeing  the  analysis  completed  to  a good  standard,  so  we  have  ensured  that  it  forms  part  of  the 
assessment  requirements  for  all  students.  (Collis  & Remmers  1997)  suggest  that,  in  courses  which 
require  this  type  of  cross-cultural  collaboration,  care  should  be  taken  to  ensure  that  students  understand 
the  expectations  of  the  communication,  formality,  who  initiates,  length,  style  and  level  of  debate.  They 
point  out  that  we  should  not  assume  that  more  interaction  is  better  than  less  and  to  be  aware  of  the 
difficulty  of  communicating  across  language  and  cultural  barriers.  While  we  did  provide  guidelines  on 
the  case  analysis  for  both  groups  of  students  and  the  teachers  spent  time  with  the  students  to  assist  their 
understanding,  this  is  an  area  we  will  aim  to  improve  in  2000.  Having  teachers  monitor  the  discussions 
will  be  easier  because  of  the  integrated  site  in  2000.  This  will  also  enable  us  to  address  a concern  of  the 
teacher  that  the  depth  of  analysis  and  critical  thinking  displayed  in  the  online  work  was  not  consistently 
of  the  level  to  be  expected  of  senior  students. 


Next  Steps 

Our  evaluation  of  the  1999  course  consist  mainly  of  survey  data  together  with  examination  of  the 
students’  reflective  statements  submitted  with  the  e-discussions.  The  next  phase  of  evaluation  will  aim 
to  investigate  the  effects  on  student  learning  of  the  use  of  the  collaborative  tools.  This  will  use  online 
evaluation  tools  which  examine  registration  data,  session  log  data  and  analyse  patterns  of  use  and  types 
of  student  interaction.  (Sail is  & Masi  2000)  have  used  text  processing  algorithms  from  computational 
linguistics  research  and  the  application  of  connectionist  methods,  especially  fuzzy  labelling  and  neural 
network  simulation  to  analyse  the  students’  reflective  statements  and  provide  feedback  for  further 
refinements  to  the  course.  This  approach  will  be  developed  further.  Structured  interviews  will  be  used 
to  examine  student  perceptions  in  more  depth.  The  curriculum  development  team  is  also  aiming  to 
develop  evaluation  tools  to  assist  in  assessing  the  extent  to  which  learning  has  been  promoted  and 
particularly  the  effects  on  higher  order  thinking.  (Burge  1994),  (Ruberg  et  al.  1996)  and  (Webb  et  al. 
1995)  provide  examples  of  studies  attempting  to  evaluate  these  more  complex  aspects  of  computer 
mediated  communication  in  learning. 

The  curriculum  team  encountered  difficulties  in  developing  and  delivering  the  course.  It  took  a great 
deal  of  work  to  set  up  and  maintain  the  online  course  compared  to  the  classroom-based  course.  The 
development  team  was,  in  1999,  based  in  different  countries  and  this  placed  a huge  strain  on 
communication  and  teamwork.  This  has  changed  in  2000  and  the  team  is  now  all  based  in  Auckland. 

Our  experience  in  1999  with  Advanced  International  Business  demonstrated  the  value  of  using  online 
forms  of  collaboration  to  enhance  learning  in  international  business.  More  importantly  students  enjoyed 
it  and  appreciated  its  benefits. 
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Abstract:  Learning  is  frequently  assumed  as  a natural  result  of  projects  performed  by  groups, 
and  cooperative  work  tools  availability  within  a computational  environment  is  enough  to 
motivate  or  to  induce  cooperation  among  participants.  However,  many  times  cooperation 
simply  does  not  happen  within  the  environments,  or  it  has  to  be  externally  articulated  by  a 
tutor.  We  claim  that  CSCL  environments  should  be  built  under  an  explicit  cooperation  model 
to  be  efficient.  The  goal  of  this  woric  is  to  present  a Cooperation  Model  for  Learning  which  is 
described  through  a system  of  patterns. 


1.  Introduction 

Learning  is  frequently  assumed  as  a natural  product  of  projects  performed  by  groups,  and  the  availability  of 
cooperative  work  tools  within  a computational  environment  is  enough  to  motivate  or  to  induce  cooperation 
among  participants.  However,  as  literature  tells  (Roussos,  1997),  in  many  situations  the  cooperation  either  is 
non-existent  or  needs  to  be  promoted  out  of  the  environment  by  a tutor.  Even  in  these  cases,  the  role  that  the 
teacher  should  carry  out  to  guarantee  cooperation  is  not  explicit.  Besides,  many  proposals  neither  have  been 
proved  yet,  nor  present  an  explicit  cooperative  process  model  to  clarify  how  they  would  operate. 

We  identify  the  following  reasons  to  the  problem  of  low  cooperation  level  within  CSCL  environments: 
Culture-  one  of  the  difficulties  that  determine  negative  results  in  the  use  of  CSCL  environments  is  that  people 
are  not  used  to  working  in  groups;  Stimulus  - most  CSCL  environments  exclusively  offer  tools  to  support 
group  tasks  execution,  instead  of  mechanisms  that  favor  group  functions  such  as  cognitive  activity,  support  to 
individuals  and  well  being  of  the  group;  Context  - CSCL  environments  usually  have  a specific  educational 
objective  or  an  organizational  practice  training  goal,  but  besides  it,  they  must  be  integrated  within  other 
activities  on  which  students  participate;  Technology  - there  is  no  tool  integration  within  environments,  and  in 
general,  people  have  difficulties  to  deal  with  it. 

We  claim  that  CSCL  environments  must  address  issues  related  to  these  problems  in  order  to  promote  learning 
in  a real  cooperative  way.  So,  we  understand  the  necessity  to  provide  a Cooperation  Model  specific  for 
Learning,  where  solutions  for  identified  problems  should  be  contemplated.  The  goal  of  this  paper  is  to  describe 
a Cooperation  Model  for  Learning  composed  by  a set  of  elements,  which  characterizes  most  CSCL  study  areas. 
For  each  of  these  areas,  the  model  includes  descriptions  of  problems  and  their  corresponding  solutions  in  a 
pattern  format. 

The  model  will  work  as  a repository  of  problems  and  solutions  patterns  found  in  the  literature  or  specifically 
developed  for  the  area.  The  resulting  patterns  are  interrelated,  forming  a system  of  patterns,  which  aims  at 
addressing  several  CSCL  issues.  The  system  of  patterns  can  be  used  as  basis  for  CSCL  environment 
development  process.  The  rest  of  this  paper  is  divided  in  4 sections.  In  section  2,  the  Cooperation  Model  for 
Learning  is  described.  In  Section  3,  some  related  works  are  presented.  Section  4 concludes  the  paper. 

2.  Cooperation  Model  for  Learning 

Kumar  (1996)  emphasizes  that  one  of  the  most  important  fields  for  research  in  CSCL  is  the  cooperative  activity 
modeling.  According  to  him,  modeling  should  involve  among  other  things,  the  representation  of  cooperative 
actions,  such  as  conflicts,  resolutions  and  cooperative  pair  roles,  that  could  be  modeled  according  to 
characteristics  and  deepen  of  the  cooperation  process. 

This  work  is  concerned  to  a representation  of  most  CSCL  issues  in  such  a way  to  make  it  easier  for  those  who 
want  to  develop  an  environment  not  to  skip  important  aspects,  such  as  cultural  ones.  Regarding  this  goal  - to 
facilitate  development  - the  model  is  described  through  data  elements.  Each  of  these  elements  represents  one 
CSCL  study  area,  which  are  expanded  in  deeper  levels  of  details  and  can  be  described  by  a system  of  patterns. 
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The  core  of  this  model  is  the  objective  and  the  cooperative  process  proposed  within  an  environment.  These 
two  elements  will  determine  all  the  characteristics  that  the  CSCL  environment  should  have.  So,  the  first  level  of 
granularity  of  the  model  is  presented  in  Figure  1,  where  issues  Context,  Culture  and  Stimulus  are  addressed: 


Context 

Stimulus  1 

OBJECTIVE 

PROCESS 

7^ 

Previous  Knowledge 

Cooperation  Forms 

Activity  Roles  Memory 

Learning  Theory 

■ 1 

Coordination  Evaluation  Awareness 

Cultural  Aspects 

Figure  1 - Generic  Cooperation  Model  for  Learning 

The  Cooperation  Model  is  based  on  the  Objective  that  one  wants  to  reach,  for  example,  learning  a certain 
concept;  and  on  the  Process,  that  is  related  to  the  cooperative  actions,  for  example,  cooperative  edition  of  a text. 
The  Objective  of  the  proposal  brings  all  the  issues  related  to  group  Context  and  Culture,  which  will  determine 
how  the  process  should  be  implemented  in  order  to  stimulate  cooperation.  It  is  related  to  4 aspects:  Previous 
Knowledge  - background  knowledge  representation  of  the  group;  Learning  Theories  - a learning  theory 
should  give  basis  to  the  learning  environment;  Cooperation  Forms  - the  cooperative  way  chosen  by  the  group 
to  work;  Cultural  Aspects  - the  cultural  factors,  which  determine  the  context  on  which  the  group  is  inserted. 
The  Cooperative  Process  is  the  element  that  describes  “what  people  is  going  to  do”  within  the  environment.  So, 
its  components  must  define  everything  related  to  the  flow  of  work,  such  as  tasks  to  be  performed,  how  to  know 
if  they  are  being  rightly  done,  and  mechanisms  to  support  the  work.  A good  flow  of  work  definition,  in  which 
interdependencies  are  established,  allied  with  mechanisms  to  support  work  in-group,  guarantee  that  cooperation 
process  will  be  stimulated.  The  Process  element  is  related  to  6 issues  and  is  linked  to  the  problem  of  Stimulus: 
Activity  - activities  to  be  developed  by  the  group;  Roles  - functions  that  group  members  can  assume,  which 
can  be  different  according  to  process  diverse  stages;  Memory  - storage  of  everything  related  to  the  way 
activities  happen;  Coordination  - related  to  process  controls  and  help  to  the  learner;  Evaluation  - mechanisms 
to  support  evaluation  of  learning;  Awareness  - elements  responsible  to  guarantee  that  people  understand,  and 
are  conscious  of  the  process. 

Now  deeper  level  elements  of  the  model,  which  correspond  to  expansions  of  the  first  level,  are  described. 

2.1.  Activity 

Activity  refers  to  the  learning  tasks  proposed  within  an  environment,  which  will  be  developed  by  participants 
during  cooperative  learning  process.  Actually,  there  are  moments  to  work  individually  (Individual)  and  other 
ones  to  work  in  group  (Cooperative).  Besides,  activities  can  generate  Products.  Cooperative  activities  own  a 
special  Dynamics,  because  of  the  interaction  among  people,  which  involves  Protocols  and  Shared  Objects. 


Figure  2:  Expansion  of  the  element  Process 


2.2.  Roles 

According  to  the  activity  proposal,  it  can  be  necessary  that  group  members  play  different  Roles.  Defining  roles 
within  an  environment  makes  it  possible  to  designate  specific  tasks  and  to  best  coordinate  the  execution  of 
them.  It  is  also  important  that  people  can  assume  different  roles  during  the  process  in  order  to  try  different 
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responsibilities.  The  model  should  make  a description  of  the  different  possible  roles  (Definition),  provide  ways 
to  nominate  roles  (Nominating),  and  define  how  these  roles  will  stand  along  the  process  (Formation  Rule). 
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Figure  3:  Expansion  of  the  element  Roles 

In  addition,  it  is  necessary  to  implement  Interfaces  appropriated  to  each  role,  and  to  supply  necessary 
awareness  mechanisms  to  the  various  tasks  performed  by  the  group. 


2.3.  Memory 

The  cooperative  activity  development  should  be  traced  and  stored.  It  is  very  important  to  preserve  not  only  the 
products  generated  during  an  interaction,  but  the  way  it  took  place.  Saving  the  memory  of  an  activity  might  be 
very  useful  as  it  makes  the  produced  material  available,  and  shows  how  the  process  of  cooperation  happened  to 
other  groups.  The  Memory  of  the  activity  involves  the  following  aspects:  Elements  related  to  the  Activity  that 
one  wants  to  keep  persistent;  Capture  of  these  elements;  Storage  and  Retrieval,  that  are  related  to  how  to  store 
and  to  retrieve  elements. 


Figure  4:  Expansion  of  the  element  Memory 


2.4.  Coordination 

Coordination  is  related  to  process  controls  and  help  to  the  learner.  This  is  done  using  Guides  or  agents  that 
should  monitor  accomplished  tasks  and  supply  help  or  procedure  indications. 

2.5.  Evaluation 

Evaluation  is  composed  of  important  mechanisms,  that  measure  whether  objectives  are  being  reached  or  not, 
and  because  of  this  it  should  be  done  during  the  whole  process.  It  is  expected  that  each  individual  reach  a 
certain  knowledge  level  about  concepts  presented,  so  Individual  Results  should  be  measured.  Besides,  the 
group  as  a whole  must  produce  some  results,  which  must  also  be  observed  (Group  Results).  The  evaluation  is 
made  by  mechanisms  that  analyze  the  Process  and  the  Products  generated  by  the  group. 

2.6.  Awareness 

Awareness  is  the  element  responsible  to  guarantee  that  people  understand,  and  are  conscious  of  the  process, 
and  of  the  participant’s  interaction  within  the  environment  (Figure  5).  Four  types  of  awareness  must  be 
provided:  Social*  social  connections  within  the  group;  Tasks-  the  way  tasks  should  be  taken;  Concepts-  how 
knowledge  is  distributed  within  group  members;  and  Workspace-  interactions  within  the  shared  environment. 
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2.7.  Previous  Knowledge 

A learning  environment  will  propitiate  a specific  content  knowledge  gain.  Even  so,  this  content  appropriation 
will  only  be  possible  depending  on  previous  knowledge  that  people  have.  This  way,  it  is  very  important  to 
provide  support  to  identification  of  group  members’  background.  This  background  can  be  appraised  under 
appropriated  methods  - Previous  Knowledge  Evaluation.  The  group  as  an  entity  also  owns  characteristics  that 
evidence  its  knowledge  background.  Group  Formation,  which  refers  to  scholarship  level,  group  interest  areas, 
and  homogeneity,  can  configure  three  kinds  of  knowledge:  Acquired  Concepts;  Acquired  Abilities  and  Meta- 
abilities (Figure  6). 


Figure  5:  Expansion  of  the  Q\QmQnt  Awareness 

2.8.  Learning  Theory 

A cooperative  learning  proposal  should  be  based  on  a learning  theory  that  supports  social  interaction  elements. 
Learning  Theory  should  supply  subsidies  for  composition  and  structures  of  learning  activities,  indicating  how 
interaction  can  be  accomplished,  what  should  be  avoided,  and  what  is  or  not  important  in  this  process.  There  are 
many  leaning  theories  that  mention  interaction  and  cooperation  as  means  to  promote.  To  represent  them  in  this 
model,  we  choose  Dillenbourg’s  work  that  indicates  three  basic  approaches.  Dillenbourg  et  al.  (1994)  state  that 
there  are  three  different  theoretical  approaches  able  to  understand  the  nature  of  cooperative  learning:  either 
focusing  on  the  individual’s  vision  (independent  cognitive  systems  that  interact),  or  on  the  group  (cognitive 
system  with  own  characteristics):  Socio-Constructivist  Approach;  Socio-Cultural  Approach;  and  Shared 
Cognition  Approach. 

2.9.  Cooperation  Forms 

A Cooperative  learning  proposal  can  be  centered  in  different  forms  or  models.  This  is  related  to  level  and 
engagement  in  the  cooperation  process.  Depending  on  how  the  group  has  been  working  or  how  a new  proposal 
is  made,  a work  configuration  can  be  defined.  Bma  (1998)  describes  some  of  these  models  and  affirms  that 
being  conscious  of  which  one  is  used  can  improve  computational  support. 

In  the  context  of  this  work,  six  Cooperation  Forms  will  be  considered:  Work  Division  - in  this  case  a division 
of  tasks  is  made,  and  each  member  of  the  group  is  responsible  for  a task;  Cooperation  State  - there  are 
individual  and  in-group  work  moments,  but  the  group  is  sharing  a cooperative  state;  Cooperation  as  Final 
Purpose  - the  work  objective  is  to  learn  how  to  cooperate;  Cooperation  as  Means  - the  objective  is  to  learn 
something,  using  cooperative  techniques;  Formal  Cooperation  - group  members  make  an  agreement  to 
accomplish  a work  cooperatively;  Informal  Cooperation  - group  members  work  cooperatively,  but  there  is  not 
a formal  agreement,  the  cooperation  happens  spontaneously. 

2.10.  Cultural  Factors 

A proposal  that  involves  interaction  among  people  cannot  avoid  considering  cultural  factors  related  to  context 
where  the  group  is  inserted.  Some  problems  that  may  arise  within  the  group  could  be  predicted  if  the  culture 
was  previously  identified.  Besides  this  issue,  the  group  can  be  motivated  according  to  its  necessities  and 
expectations.  The  environment  can  develop  ways  to  integrate  people  using  common  communication  patterns. 
According  to  social  sciences  researchers  (McGrath,  1993)  and  works  on  organizational  culture,  a set  of 
Cultural  Aspects  (Figure  7)  specifically  for  cooperative  learning  environments  was  specified.  These  factors  are 
divided  in  Individual  Properties  (each  person  owns  individual  components  that  will  influence  group 
behavior),  and  Group  Structure  (demonstrate  how  the  group  is  organized,  and  how  people  fit  within  it). 
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CULTURAL  FACTORS 


Figure  7:  Expansion  of  the  element  Cultural  Aspects 


2.11.  Description  of  the  Model  through  Patterns 

For  each  element  of  the  model  described,  the  most  important  problems  are  identified  and  described  as  patterns, 
like  the  example  presented  bellow  Pattern  Workspace  Awareness  in  CSCL  (Figure  8),  related  to  the  element 
Process-Awareness-Workspace  (in  Figure  5). 

According  to  Johnson  (1997)  patterns  are  attempts  to  describe  a problem  to  be  solved,  a solution,  and  the 
context  in  which  it  is  applied,  A pattern  captures,  structures  and  presents  key  information  about  a domain 
familiar  to  specialists.  There  are  some  kinds  of  patterns  (conceptual,  design  and  programming).  The  patterns  of 
the  Cooperation  Model  for  Learning  are  in  conceptual  level.  Conceptual  Patterns  are  standards  whose  forms  are 
described  through  terms  and  concepts  of  an  application  domain;  they  are  set  in  metaphors  and  restricted  to  an 
application  domain. 

The  patterns  are  interrelated,  forming  a system  of  patterns  developed  from  an  analysis  of  several  environments 
described  in  literature,  accomplished  under  the  optics  of  the  Conceptual  Frame  for  Analysis  of  Computer 
Supported  Cooperative  Learning  Environments  proposed  by  Santoro  et  al,  (1999),  Some  patterns  of  the  system 
are  being  directly  transformed  into  design  patterns,  while  others  will  be  used  just  as  a guideline  to  the 
implementation.  As  an  example  for  the  first  case,  the  conceptual  pattern  of  Figure  8 will  have  an  associated 
design  pattern  representing  the  software  design  of  the  solution.  As  an  example  of  the  second  case,  a pattern 
related  to  a learning  theory  approach  will  only  guide  how  to  deal  with  evaluation  techniques. 


Pattern  Name:  Workspace  Awareness  in  CSCL 

Problem:  What  elements  should  be  disposed  in  a CSCL  environment  in  order  to  guarantee  workspace  awareness  among  the  members 
of  the  group? 

Context:  In  a CSCL  environment,  awareness  of  participants  is  one  of  the  key  elements  to  propitiate  an  effective  interaction.  Knowing 
what  someone  is  doing  in  a certain  moment  can  make  people  interact  easily.  The  access  to  information  on  contributions  and  tasks 
already  completed  is  also  an  important  factor,  because  it  can  approach  people  with  common  interests. 

Forces:  Workspace  awareness  reduces  the  overhead  of  the  work  in-group,  allowing  a more  natural  and  effective  interaction. 
Workspace  awareness  facilitates  students'  practices  that  allow  cooperative  learning  to  occur. 

Solution:  The  environment  should  provide  a representation  of  each  member  of  the  group  within  the  workspace,  so  that  the  whole 
group  can  visualize  "where  he  is”,  “what  he  is  doing  ",  and  "what  he  had  already  done  ".  This  representation  can  be  graphic,  iconic, 
through  windows,  or  virtual  reality. 

Related  patterns:  Social  Awareness  in  CSCL,  Task  Awareness  in  CSCL 

Known  uses:  CSILE  implements  workspace  awareness  through  structured  message  notion,  where  an  author  is  represented  by  its 
initials,  and  therefore  other  members  get  to  know  what  he  is  doing  or  have  already  done.  Following  the  same  line,  CSILE, 
Collaboratory  Notebook  and  CaMILE  also  implement  structured  messages  where  the  authors  are  identified  for  its  names. 

In  NICE  (Roussos,  1997),  the  participants  of  a work  section  are  represented  by  avatars  and  everybody  can  visualize  their  movements 
and  actions  inside  of  the  workspace.  CLARE  supplies  a vision  of  all  the  works  accomplished  by  group  members  in  a specific  window. 

Figure  8:  A Pattern  Example 


3.  Related  Works 

In  the  literature  we  can  find  some  works  on  development  of  groupware  for  learning.  Becker  and  Zanella  (1998) 
introduce  a structured  cooperation  model  to  aid  the  teaching/leaming  process  of  modeling  disciplines. 
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The  process  is  defined  through  development  of  exercises,  criticism  and  discussion  of  them.  A general 
framework  defining  a process,  roles,  and  shared  objects  are  provided.  This  work  is  restricted  to  a domain  of 
applications  (modeling)  and  to  a specific  kind  of  process. 

SECAI  Model  is  defined  within  CLARE  Environment,  which  is  a distributed  learning  environment  which  goal 
is  to  facilitate  learning  through  collaborative  knowledge  construction  (Wan  and  Johnson,  1994).  The  SECAI 
Model  (Summarization,  Evaluation,  Comparison,  Argumentation,  and  Integration)  defines  an  explicit  model  of 
process  for  collaborative  learning  of  scientific  texts,  that  metaphorically  "pulls  " the  apprentices  of  an  external, 
isolated  and  individual  position  to  an  internal,  integrated  and  collaborative  perspective.  This  model  is  limited  to 
represent  some  issues  of  the  cooperation  process.  Our  proposal  goes  further  than  these  works  defining  a 
complete  conceptual  model,  addressing  issues,  such  as  cultural  ones,  learning  theories,  roles  and  evaluation  of 
learning. 


4.  Conclusions 

Research  in  the  area  of  CSCL  found  that  many  of  the  developed  environments  do  not  reach  the  intended 
success,  that  is,  to  promote  cooperation  among  members  of  a group  to  improve  learning.  Most  of  the  problems 
are  related  to  stimulus,  context,  culture  and  technological  problems.  We  present  a Cooperation  Model  for 
Learning  as  part  of  a proposal  to  solve  some  problems  in  the  area  of  CSCL.  The  goal  is  to  facilitate  the 
development  of  cooperative  learning  environments,  supporting  the  aspects  mentioned  previously. 

We  claim  that  building  CSCL  environments  based  on  a safe  model  should  produce  effective  learning  by 
cooperation  as  a result.  One  major  contribution  of  our  proposal  is  to  analyze  non  technical  problems-  cultural 
influence,  educational  approach,  environmental  context  - and  conjugate  them  to  cooperative  activities  and  the 
issues  associated  to  them-  objects  sharing,  coordination,  evaluation,  awareness  - providing  guidelines  for 
implementation  of  the  activities  with  a good  theoretical  basis. 

Comparing  related  works  with  our  proposal,  we  conclude  that  few  of  them  are  based  on  an  explicit  model,  and 
also  none  of  them  address  all  the  issues  that  we  intend  to  support,  even  for  a specific  domain.  Despite  the 
number  of  computer  supported  cooperative  learning  environment  proposals,  the  use  of  patterns  was  not  yet 
explored  in  CSCL,  and  specific  frameworks  practically  does  not  exist  for  CSCL  processes. 

Our  model  includes  a series  of  learning  theory  approaches,  cooperation  forms,  cooperative  activities,  and 
context  issues.  The  system  of  conceptual  patterns  obtained  from  the  model  will  be  translated  into  design 
patterns  and  then  a software  framework  can  be  developed.  The  framework  should  be  restricted  by  a set  of 
choices  made  from  those  available  in  the  model  and  should  be  defined  to  a particular  domain  of  tasks  in  order 
to  implement  an  environment.  The  resulting  environment  will  be  used  in  a real  situation  in  order  to  demonstrate 
the  advantages  of  our  approach. 
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Abstract:  We  have  all  felt  movement  of  the  tide  in  education  toward  resurrecting  the  concept 
of  constructivism  in  the  hope  of  increasing  meaningful  learning  for  our  students.  The  question 
of  why  it  has  met  with  so  little  success  in  the  past  has  to  be  pondered,  and  we  have  to  ask  what 
it  is  that  has  to  be  done  if  such  a worthwhile  concept  is  to  prevail  in  the  future.  One 
speculation  as  to  why  constmcted  learning  has  not  met  with  success  in  the  past  is  that  it  may 
have  been  over  sold  as  unstmctured  discovery  learning.  Discovery  learning  may  be  an  ideal, 
but  too  often  the  approach  is  just  too  impractical.  Perhaps  we  need  to  bring  educators  into 
discovery-learning  constmctivist  practices  more  gradually.  This  paper  deals  with  one  attempt 
to  do  so  by  introducing  constmctivist  learning  strategies  more  gradually  through  a guided 
discovery  approach  that  employs  computer-generated  concept  mapping. 


The  constructivist  view  of  learning  holds  that  meaningful  learning  occurs  when  people  use  existing 
knowledge  schemes  and  the  viewpoints  of  others  to  interpret  the  world  around  them  with  the  result  being  the 
active  creation  of  knowledge  structures  from  personal  experiences.  This  is  at  the  very  core  of  constructed 
learning.  A “constructivist  approach”  is  a general  term  used  to  label  all  those  approaches  that  hold  to  the 
assumption  that  students  should  build,  that  is  construct  knowledge  for  themselves.  While  constuctivist 
explanations  of  learning  are  not  new,  since  noted  scholars  such  as  John  Dewey,  Jean  Piaget,  Lev  Vygotsky,  and 
Jerome  Bruner  to  mention  a few,  have  given  support  to  the  approach;  there  has  been  a resurgence  of 
constructivist  thinking  in  recent  years.  More  and  more  contemporary  theorists  and  practitioners  have  been 
holding  to  the  idea  that  children  do  construct  their  own  knowledge  rather  than  simply  learning  what  someone 
tells  them  to  learn  (Ormond  1995).  The  personal  construction  of  knowledge  concept  is  based  upon  the  belief 
that  the  essence  of  one’s  knowledge  can  never  be  totally  transferred  to  another  person,  because  knowledge  is  the 
result  of  a personal  interpretation  of  experience  that  is  influenced  by  a myriad  of  factors  such  as  age,  gender, 
ethnic  background,  and  knowledge  base  (Snowman  2000). 

Since  constuctivist  approaches  are  basically  discovery  oriented  (Lefrancois  1997),  the  question  that  we 
as  professors  and  teachers  have  to  ponder  is,  “How  will  this  affect  our  own  personal  teaching  styles  and 
approaches  to  student  learning?”  Most  of  us  who  teach,  whether  we  care  to  admit  to  it  or  not,  tend  to  fashion 
our  pedagogical  approaches  to  reflect  the  way  in  which  we  were  taught,  namely  via  the  transmission  model. 
However,  even  if  we  were  to  adopt  a more  discovery  oriented  constuctivist  leaning  approach,  we  still  would 
have  to  deal  with  the  fact  that  discovery  learning  is  not  without  a problem.  A faithful  application  of  Piaget’s  and 
Bnruner’s  ideas  suggests  an  unstructured  discovery  approach,  but  research  does  not  support  its  efficacy 
(Resnick  & Klopfer  1989)(Brown  and  Campionel994).  Students  engaging  in  unstructured  discovery  activities 
tend  to  become  lost  and  even  frustrated  which  can  lead  to  misconceptions.  Therein  lies  the  dilemma.  The  basic 
underlying  assumption  of  discovery  learning,  which  is  at  the  core  of  constructivist  learning  philosophy  conflicts 
with  strong  evidence  that  suggests  when  discovery  is  presented  in  the  unstructured  manner  as  suggested  by 
Piaget  and  Bruner,  is  less  than  compelling.  This  paper  suggests  a more  balanced  structured  discovery-based 
approach;  one  that  mitigates  the  problems  of  an  unstructured  discovery  approach  and  yet  still  encourages 
discovery  learning.  It  is  called  guided  discovery.  While  it  is  not  an  unstructured  discovery  approach  it  still 
embodies  the  essence  of  student  centered  discovery  learning  (Branford  1993).  Studies  have  shown  that  when 
utilizing  guided  discovery  approach  students  have  achieved  more  transfer  of  knowledge  and  more  long-term 
retention  than  those  taught  with  a more  traditional  transmission  of  knowledge  model.  (Hillocks  1984) 
(Alexander&Murphy,  1 994). 

One  specific  method  that  seems  to  have  promise  in  promoting  the  basic  tenets  of  guided  discovery 
learning  is  concept  mapping.  Basically,  concept  mapping  is  a technique  that  allows  one  to  understand  the 
relationships  of  ideas  by  creating  a visual  map  of  the  connections  among  these  ideas.  Concept  mapping  allows 
students  to  see  the  relationship  between  the  knowledge  they  already  posses  and  the  knowledge  to  which  they  are 


trying  to  connect,  what  ever  that  source  might  be.  Concept  mapping  can  also  organize  a myriad  of  ideas  in  a 
logical  kind  of  structure  employing  a type  of  visual  language  that  also  encourages  students  to  operate  at  all  six 
levels  of  Bloom’s  Taxonomy  of  the  Cognitive  Domain  (Bloom  1956)  (Gaines,  Shaw,  1999)(Novak  and  Gowin 
1984).  Concept  mapping  focuses  on  a topic  or  an  area  of  interest,  and  involves  input  from  one  or  more 
participants.  It  can  be  utilized  by  an  individual  student  or  any  number  of  people  in  a cooperative  learning 
strategy.  The  result  is  an  interpretable  pictorial  view  of  the  ideas  and  concepts  constructed;  it  graphically  shows 
how  the  ideas  that  have  been  generated  from  one’s  own  knowledge  schemes  are  related  to  the  environment 
attempting  to  be  explained.  (Trochim  1999).  A well-conceived  concept  map  graphically  reveals  the  shape  of 
the  structure,  the  relative  importance  of  the  information  and  ideas,  and  the  way  that  the  infomiation  relates  to 
other  ideas.  They  can  be  used  to  think  through  complex  problems  by  viewing  the  overall  structure  of  the  subject 
and  to  associate  ideas  and  make  connections  that  would  otherwise  appear  to  be  too  unrelated  to  be  linked. 
Another  strong  advantage  of  using  concept  maps  is  that  once  a student  either  individually  or  cooperatively  has 
developed  a concept  map,  the  organization  and  details  of  the  map  tend  to  be  remembered  for  a much  longer 
period  of  time  than  is  usually  the  case  when  trying  to  remember  the  material  in  text  form,  (Mind  Tools,  1995), 
The  concept  maps  when  utilizing  a guided  discovery  format  promote  constructivist  thinking  because  it  is  still 
the  students  themselves  that  build  or  construct  the  map  with  all  of  its  relationships,  but  the  students  are  not  far 
afield  because  they  are  still  receiving  guidance  from  the  instructor. 

In  the  field  of  education,  the  use  of  concept  maps  probably  began  with  Ausubel’s  learning  theory 
treatment  of  advanced  organizers  and  meaningful  learning  which  holds  that  in  order  for  meaningful  learning  to 
take  place,  students  must  relate  new  knowledge  to  relevant  topics  that  they  already  know,  and  that  teaching 
strategy  in  and  of  itself  does  not  necessarily  result  in  meaningful  learning,  (Ausublel963) 

Ausubel’s  work  in  turn  prompted  Novak  (1977)  to  develop  a system  of  concept  maps  which  have  been  applied 
in  a variety  of  ways  within  the  school  setting.  (Lambiotte,  Dansereau,  Cross  and  Reynolds,  1 989) 

However  concept  mapping  did  not  have  the  success  that  it  deserved,  namely  because  they  were  cumbersome 
and  difficult  to  draw  by  hand.  This  was  the  case  in  the  past,  but  fortunately,  today  we  have  software  that  is 
capable  of  generating  concept  maps  easily  and  efficiently.  One  such  program  is  Inspiration. 

Inspiration  (1999)  is  a software  package  that  is  specifically  designed  to  facilitate  the  construction  and 
use  of  concept  maps.  It  is  a powerful  visual-thinking  tool  that  helps  clarify  and  organize  ideas  and  information. 
It  helps  facilitate  brainstorming,  planning  and  the  explanation  of  relationships  between  processes,  variables  and 
events  by  providing  some  basic  tools  in  the  application  that  make  the  process  of  graphing  the  concept  map  a 
relatively  easy  one.  It  can  easily  be  adapted  by  teachers  to  promote  student-centered,  discovery  based, 
constuctivist,  type  learning. 


Some  of  the  Basic  Tools  of  Inspiration  That  Can  be  Used  to  Facilitate  the  Construction  of 

Concepts  Maps 

A.  Diagramming:  Helps  you  to  quickly  record,  link  and  map  out  concepts  within  a symbol,  and  the  rapid  fire 
element  allows  you  to  put  the  ideas  down  as  fast  as  you  can  think  of  them  while  constructing  the  proper 
linkage  among  the  ideas, 

B.  Moving  a symbol:  A symbol  can  be  moved  at  any  time  and  when  it  is  moved,  Inspiration  automatically 
adjusts  the  link  to  clearly  show  the  new  relationship, 

C.  Adding  Unconnected  Ideas:  Sometimes  you  may  have  an  idea,  but  are  not  certain  about  where  it  belongs 
in  the  whole  scheme.  The  program  allows  you  to  add  an  idea  anywhere  within  the  diagram  and  adjust  it  at  a 
later  time, 

D.  Changing  Symbol  Shapes:  Inspiration  provides  a large  menu  of  symbol  shapes  to  represent  any  number  of 
concepts.  These  shapes  can  be  easily  changed  at  any  time  in  order  to  further  clarify  or  emphasize  the 
relationship  of  ideas  or  concepts, 

E,  Drawing  A Link:  Symbols  are  connected  automatically,  but  unconnected  links  can  be  connected  manually 
at  any  time,  and  the  program  will  make  the  link  precise, 

F,  Adding  Text  to  A Link:  When  additional  information  needs  to  be  added  to  a link  in  order  to  further  clarify 
or  emphasize  the  relationship,  as  is  the  case  in  many  concept  maps,  the  program  easily  allows  for  it. 

G,  Scrolling  and  Magnification:  The  application  allows  you  to  move  the  diagram  around  within  the  window 
or  zoom  in  or  out  of  the  diagram. 

H,  Formatting  Symbol  Text:  Text  font,  and  font  styles  can  be  set  for  the  entire  diagram  or  selectively. 
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I.  Printing  to  fit:  The  print  to  fit  option  allows  you  reduce  the  diagram  to  fit  to  one  page  or  to  any  number  of 
pages  specified  before  printing. 

J.  Switching  Between  Diagram  and  Outline  Views:  You  have  the  option  of  working  in  either  outline  view 
or  diagram  view  for  your  text,  and  you  can  alternate  at  any  given  time. 

K.  Importing  and  Installing  Graphics:  Color  or  black  and  white  graphics  created  in  other  applications  can  be 
imported  into  the  Inspiration  diagram  or  installed  in  the  Inspiration  User  Symbol  Menu. 

A thorough  grounding  in  the  techniques  of  concept  mapping  and  the  Inspiration  program  will  provide  all 
the  foundation  necessary  for  utilizing  concept  mapping  successfully  in  the  classroom. 

With  proper  preparation,  concept  maps  that  are  computer  generated  can  be  effectively  used  as  a teaching 
tool  in  classes  from  elementary  through  college  to  promote  more  efficient  learning  through  a constructivist 
guided  discovery  approach. 
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Abstract:  New  technologies  are  regularly  introduced  in  the  home,  community,  workplace,  and 
educational  settings.  How  do  teachers,  in  particular,  learn  to  appropriately  integrate  new 
technologies  into  their  personal,  instructional,  and  professional  activities?  How  does  a current  or 
potential  employer  determine  the  level  of  proficiency  that  a given  teacher  has  achieved  in  a range 
of  technology-based  areas?  This  paper  describes  how  one  local  educational  agency,  the  San  Luis 
Obispo  County  Office  of  Education,  has  approached  those  questions,  and  how  their  answers  fit  into 
guidelines  currently  under  development  by  the  state  of  California. 


The  Context 

Excerpts  from  two  reports  will  be  used  to  describe  the  context  in  which  the  work  reported  in  this  paper  was 
undertaken. 


“Any  nine  year  old  who  has  ever  commandeered  her  parents’  computer  to  surf  the 
Internet,  asked  a NASA  scientist  about  acid  rain,  or  designed  a multi-media  language  arts 
project  knows  what  dozens  of  blue-ribbon  panels  have  concluded  in  vastly  more 
syllables:  ‘Computers  are  cool!’  Technology  has  the  power  to  teach,  to  motivate,  to 
captivate  and  to  transform  an  ordinary  classroom  into  a training  ground  for  the  next 
generation  of  artists,  entrepreneurs,  and  government  leaders....  The  ability  of  teachers  to 
use  technology  to  promote  students’  acquisition  of  basic  skills  and  subject-matter  content 
is  critical  to  education’s  success.  Accomplishing  this  objective  requires  front-end 
preparation,  as  well  as  ongoing  training  opportunities  and  incentives...  to  integrate 
technology  into  the  content  and  performance  standards  that  will  be  used  as  the  basis  for 
setting  policies  for  preparing,  hiring,  evaluating,  and  promoting  teachers.”  (California 
Education  Technology  Task  Force,  1996,  p.l). 

“The  California  Commission  on  Teacher  Credentialing  was  mandated  ...  to  establish 
‘standards  of  program  quality  and  effectiveness  relative  to  the  use  of  computers  in  the 
classroom  for  preliminary  credential  candidates,  and  to  establish  standards  of  program 
quality  and  effectiveness  relative  to  advanced  computer-based  technology  for 
professional  credential  candidates’.  As  provided  by  law  ...  and  practice,  the  Commission 
elected  to  select  a ‘Computer  Education  Advisory  Panel’  which  was  charged  to  make  a 
comprehensive  review  and  make  specific  recommendations  with  regard  to  computer 
competency  standards.”  (Computer  Education  Advisory  Panel,  1998,  in  the  Foreword). 
This  report  was  adopted  by  the  California  Commission  on  Teacher  Credentialing  [CTC] 
in  December,  1998,  and  on  July  12,  1999,  the  CTC  issued  official  guidelines  describing 
the  new  technology  standards  and  the  process  and  timeline  for  implementing  them 
(Swofford,  1999). 
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The  Issue 


Knowing  that  performance  standards  for  educational  uses  of  technology  were  coming,  in  the  Fall  of  1997  the  San 
Luis  Obispo  County  Office  of  Education  [SLOCOE]  put  together  a group  to  look  at  the  potential  impact  this  would 
have  locally,  on  both  pre-service  and  in-service  teachers.  This  group  included  classroom  teachers  and  technology 
mentors  from  the  public  schools,  building  principals  and  district  supervisors,  technology  specialists  from  the  county 
office  and  faculty  from  local  teacher  training  institutions.  This  Advisory  Board  proposed  a “Technology 
Certification  Program”  for  teachers,  site  administrators,  and  other  instructional  staff  (San  Luis  Obispo  County  Office 
of  Education  Technology  Certification  Advisory  Committee,  1999). 

In  developing  its  recommendations,  the  Advisory  Board  first  agreed  on  a number  of  parameters  for  a certification 
program.  It  was  seen  as  important  to  provide  for  multiple  levels  of  certification:  basic-level  personal  computing 
skills,  intermediate-level  skills  specific  to  the  teaching  and  learning  process,  and  advanced-level  skills  relevant  to 
various  types  of  leadership  roles.  Training  would  be  provided  to  help  educators  reach  the  various  levels,  but  the 
times,  locations,  and  designs  for  such  training  had  to  be  highly  flexible.  Entry  to  and  exit  from  training  programs 
had  to  be  as  open  as  possible,  and  it  was  important  to  provide  opportunities  for  people  with  experience  in  an  area  to 
demonstrate  their  proficiencies  in  meaningful  ways  outside  of  a formal  training  program.  Finally,  evaluation  of 
whether  or  not  people  had  acquired  various  skills,  either  on  their  own  or  as  part  of  a formal  training  program,  had  to 
be  based  on  actual  performance. 


The  Process 

Based  on  the  recommendations  of  the  Education  Council  for  Technology  in  Learning  [ECTL],  a three-tier  structure 
was  developed  for  evaluating  the  level  of  proficiency  with  technology  of  both  educators  (California  Technology 
Assistance  Project,  1997a)  and  students  (California  Technology  Assistance  Project,  1997b).  The  educator  levels  are: 

Level  One:  Personal  Proficiency 
Level  Two:  Instructional  Proficiency 

Level  Three:  Leadership  Proficiency 

Educators  at  Level  One  are  able  to  use  a range  of  tools  for  their  own  purposes.  They  are  able  to  send  and  reply  to 
electronic  communications  in  a number  of  contexts  (e.g.,  personal  mail,  threaded  discussion  groups,  online  chat 
rooms),  and  to  use  basic  Internet  tools  (e.g.,  web  browsers,  file  transfer  protocols).  They  can  use  common 
productivity  and  display  tools.  They  need  not,  for  example,  be  true  experts  at  desktop  publishing,  but  they  are  able 
to  put  together  a simple  newsletter  to  send  home  with  their  students  or  to  help  promote  a school  club’s  activities. 
They  need  not  be  able  to  use  all  the  advanced  features  of  spreadsheet  and  database  packages,  but  they  can  use  a 
gradebook  program  that  incorporates  those  kinds  of  features.  They  do  need  to  be  aware  of  legal  and  ethical  issues  in 
computer  use,  and  they  must  know  how  to  use  various  tools  in  a responsible  and  professional  way.  They  also  need 
to  know  how  to  use  a range  of  resources  such  as  manuals,  help  files,  tech  support  lines,  and  how  to  provide  a 
meaningful  bug  report  when  seeking  assistance.  They  should  feel  comfortable  with  technology:  they  don’t  need  to 
know  and  remember  every  trick  for  every  program  at  their  disposal,  but  they  should  have  both  the  skills  and  the 
confidence  to  learn  new  features  or  programs  as  the  need  for  them  arises. 

Educators  at  Level  Two  are  able  to  take  their  Level  One  skills  and  extend  them  to  a range  of  specific  instructional 
settings.  They  can  design  and  implement  activities  where  students  use  desktop  publishing  or  display  tools  to  present 
reports,  where  they  use  the  Internet  to  gather  and  share  information  or  data  in  various  ways,  or  where  the  students 
use  a database  to  organize  information  and/or  a spreadsheet  to  help  analyze  it.  They  know  how  to  appropriately 
integrate  into  their  instruction  and  manage  a carefully  selected  set  of  tutorial  and  practice  software  available  at  their 
site.  At  this  level,  the  educators’  own  basic  technology  skills  are  also  improving:  they  can  install,  configure,  and  run 
more  products,  and  they  can  do  more  basic  hardware  and  software  trouble-shooting  on  their  own  instead  of  typically 
reporting  problems  to  a support  person. 

The  expectations,  both  in  the  original  ECTL  report  and  in  the  local  committee’s  recommendations,  were  that  all 
teachers  should  be  able  to  reach  both  Levels  One  and  Two.  The  SLOCOE  panel  said  nothing  about  how  long  it 
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should  take  everyone  to  achieve  this,  nor  about  whether  schools  could  or  should  require  their  employees  to  do  so.  It 
did,  however,  offer  mechanisms  for  both  helping  schools  to  provide  opportunities  for  their  instructional  staff  to 
improve  their  skills  and  for  certifying  achievement  of  the  proficiencies. 

Educators  at  Level  Three  were  the  keys  to  this  plan.  This  was  perceived  as  a special  category,  one  that  only  some 
educators  would  be  expected  to  reach.  The  people  at  this  level  are  those  who  act  in  a mentorship  or  leadership  role 
with  respect  to  the  uses  of  technology  at  their  individual  schools  or  across  their  district.  Indeed,  it  was  even 
proposed  that  there  would  be  at  least  two  such  types  of  high-end  proficiencies.  To  present  the  differences  via  an 
example:  a mentor  might  develop  and  offer  a workshop  on  personal  or  instructional  uses  of  databases,  while  a 
leader  might  review  a series  of  gradebook  packages  and  recommend  one  for  district-wide  use.  Mentors  would 
model  and  serve  as  advocates  for  effective  and  efficient  uses  of  technology,  and  serve  as  a coach  to  their  peers. 
Leaders  would  have  a major  role  in  development  of  technology  plans,  policies,  budgets,  etc.,  and  would 
systematically  manage  a range  of  technology  resources.  A given  person  who  achieved  Level  Three  might  function 
as  a mentor  or  as  a leader,  or  as  both,  and  there  would  be  distinct  certificates  to  represent  each  role.  But  one  key  to 
achieving  Level  Three  status  involved  being  someone  who  helped  to  support  those  striving  for  Levels  One  or  Two, 
or  even  three,  and  who  understood  the  certification  process  well  enough  to  be  able  to  determine  if  individuals  were 
meeting  the  various  criteria.  Level  Three  status,  furthermore,  would  need  to  be  re-evaluated  on  a regular  basis.  In 
order  to  have  their  certification  at  that  level  renewed,  eveiy  two  years  Level  Three  educators  must  submit  evidence 
of  the  things  they  are  continuing  to  do  to  provide  such  support  and  of  their  growth  in  knowledge  of  appropriate 
emerging  technologies.  On  an  ongoing  basis,  then,  individual  schools  and  districts  could  use  their  Level  Three 
people  to  help  identify  and  meet  the  professional  development  needs  of  those  teachers  wanting  to  learn  the  material 
for  the  various  proficiency  levels,  and  the  regional  certification  team  could  use  the  Level  Three  people  to  do  initial 
screening  of  those  seeking  certification. 

Portfolios  were  chosen  as  the  mechanism  for  documenting  achievement  of  the  various  proficiencies.  For  each  level, 
there  exists  a list  of  competencies,  and  a number  of  suggestions  for  artifacts  that  can  be  used  to  demonstrate  each 
one.  The  artifacts  are  to  be  assembled  into  a binder  that  can  be  viewed  by  a Level  Three  person  doing  the 
preliminary  verification,  by  the  regional  certification  team  that  reviews  portfolios  before  issuing  a certificate,  and  by 
other  appropriate  people  involved  in  the  hiring,  evaluating,  and  promoting  of  the  person.  Considerable  effort  went 
into  developing  the  competencies  so  that  they  were  both  specific  about  the  type  of  skill  to  be  demonstrated,  yet 
open-ended  about  how  achievement  could  be  demonstrated  so  as  to  accommodate  site-specific  variations  in  both 
expectations  and  resources.  Considerable  time  also  went  into  determining  the  possible  formats  for  a portfolio: 
although  they  are  being  used  to  demonstrate  proficiency  with  electronic  media,  it  was  decided  that  there  needed  to 
be  print-based  evidence  of  the  various  skills.  A few  of  the  criteria  can  be  documented  via  letters  of 
recommendation;  several  others,  by  having  a Level  Three  educator  sign  a form  stating  that  the  skills  have  been 
demonstrated  in  person.  The  majority  of  the  proficiencies,  however,  are  to  be  demonstrated  by  hardcopy 
documentation.  Additional  artifacts  in  electronic  form  may  be  included  (e.g.,  on  diskette  or  CD,  via  URL),  but  the 
panel  felt  it  was  important  for  the  portfolios  to  be  able  to  stand  on  their  own  to  avoid  the  restriction  that  a person 
reviewing  a portfolio  might  need  a particular  computer  system  or  software  tool  or  net  connection  to  view  the 
material  being  used  to  document  a given  competency. 

Once  the  proficiency  levels  were  defined,  the  criteria  for  each  level  determined,  and  the  process  for  evaluating 
portfolios  developed,  the  next  step  was  to  start  soliciting  portfolios  and  evaluating  them.  Since  the  program  was  still 
in  the  beginning  stages,  a small  group  of  Level  Three  candidates  was  identified  first:  people  already  clearly 
recognized  as  technology  leaders  in  several  local  districts.  The  panel  worked  with  this  group  as  they  developed  their 
portfolios,  documenting  issues  that  arose  and  revising  the  guidelines  as  needed.  These  people  were  required  to 
submit  portfolios  that  addressed  all  of  the  Levels:  One,  Two,  and  Three.  As  the  advisory  board  evaluated  the 
portfolios,  they  continued  to  look  both  at  how  well  the  person  met  the  requirements  and  at  issues  in  how  the  process 
itself  seemed  to  be  working.  Once  the  panel  refined  the  evaluation  guidelines,  these  educators  were  issued 
certificates  at  all  three  levels.  They  were  also  given  guidance  on  how  they  were  to  evaluate  the  future  applicants. 
People  assembling  a portfolio  do  not  have  to  complete  a level  before  getting  a preliminary  review.  Instead,  they  can 
have  a Level  Three  educator  “sign  off’  on  individual  competencies  to  get  ongoing  feedback  on  their  progress 
towards  the  requirements  at  a given  level.  An  applicant  can,  in  fact,  have  different  Level  Three  people  sign  off  on 
the  various  individual  competencies:  as  part  of  the  classroom  observation  process,  during  in-service  workshops, 
after  independent  study,  and  so  on.  All  complete  portfolios  at  each  level  are  then  reviewed  by  the  program 
certification  team.  This  is  done  to  maintain  quality  control  on  the  process:  if  a certain  Level  Three  person  seems  to 
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be  signing  off  on  competencies  without  documentation  that  is  consistent  with  the  overall  guidelines,  then  action  can 
be  taken  to  make  sure  that  everyone  understands  the  expectations  and  the  need  to  uphold  standards  in  a consistent 
manner.  When  a portfolio  passes  the  certification  team’s  review,  the  educator  is  issued  an  appropriate  certificate 
from  the  SLO  County  Office  of  Education. 


The  Next  Phase 

The  SLOCOE  process  was  begun  barely  two  years  ago.  The  Advisory  Panel  hoped  that  it  would  eventually  grow 
into  a widespread  program,  but  they  did  not  anticipate  the  speed  at  which  it  has  already  spread  both  locally  and 
regionally.  In  the  meantime,  as  stated  in  the  background  section  above,  the  state  of  California  has  recently  come  out 
with  its  official  standards.  The  certification  panel  is  currently  reviewing  the  state  standards  to  determine  how,  if  at 
all,  the  local  proficiency  criteria  must  be  adjusted  to  be  in  alignment  with  those.  Once  that  is  sorted  out,  the  group 
will  return  to  the  development  of  a set  of  recommendations  for  artifacts  that  would  be  appropriate  for  administrators 
to  assemble  into  a portfolio  demonstrating  the  technology  proficiencies  appropriate  to  their  responsibilities. 


Some  Already-Known  Consequences 

Some  Local  Consequences 

Individual  districts  have  adopted  the  certification  process  in  their  own  individual  ways.  For  example,  a few  have 
said  that  Level  One  and  Two  certification  will  be  required  within  a given  timeframe,  regardless  of  any  exceptions  to 
such  requirements  that  the  state  might  eventually  permit.  Others  say  that  a person  need  not  go  through  the  program, 
but  only  those  who  have  a certain  level  of  certification  are  eligible  to  get  new  hardware  or  software.  Some  offer 
actual  salary  increases  to  those  achieving  certification,  while  others  provide  time  or  money  incentives  to  attend 
workshops  for  developing  proficiencies.  Some  have  tried  to  implement  their  own  version  of  the  program  in  a certain 
way,  but  have  had  to  change  their  process  due  to  conflicts  with  things  like  state  formulas  for  how  funds  can  be 
allocated  or  specifics  of  union  contracts.  Nonetheless,  every  district  in  the  region  is  participating  in  the  process  in 
some  way. 

The  San  Luis  Obispo  County  Office  of  Education  has  developed  a Tech  Mentor  Program  that  is  designed  to  give 
additional  help  in  using  technology  for  the  improvement  of  student  learning  to  schools  and  districts  in  its  local 
service  area.  Each  district  (according  to  its  size)  is  allocated  a certain  number  of  Tech  Mentor  service  days  during 
which  support  or  training  can  be  provided  at  no  cost  to  the  district  using  money  from  several  local  funds;  districts 
can  also  buy  additional  days  from  their  own  funds  if  desired.  Tech  Mentors  are  chosen  from  a pool  of  Level  Three 
Educators  who  have  registered  a list  of  specific  workshops  they  can  offer  under  various  general  categories.  A 
school  or  district  person  contacts  a potential  Tech  Mentor  to  discuss  the  local  needs  (e.g.,  dates  and  times,  how  much 
customization  is  needed,  etc.)  and  to  complete  a SLOCOE  service  contract.  Since  all  the  Tech  Mentors  are  certified 
as  Level  Three  Technology  Educators,  they  offer  not  only  the  specific  training  requested  but  also  any  needed 
guidance  on  how  participants  can  demonstrate  relevant  proficiencies  in  their  technology  certification  portfolios. 


Some  Regional  and  State-Wide  Consequences 

In  part  because  of  their  work  in  developing  this  local  process,  one  member  of  the  SLOCOE  Advisoiy^  Panel  was 
invited  to  be  a member  of  the  California  Computer  Education  Advisory  Panel  [CEAP]  that  developed  the  recent 
state-wide  technology  standards,  and  several  others  served  as  reviewers  of  CEAP’s  preliminary  recommendations. 
While  the  new  state-wide  technology  standards  do  not  mandate  a process  of  portfolio  evaluation  such  as  SLOCOE 
developed,  those  standards  are  consistent  with  such  a method. 

Word  of  SLOCOE’s  work  has  spread,  and  the  education  offices  in  several  other  counties  have  either  adopted  a 
similar  program  or  are  looking  into  the  possibility  of  doing  so.  The  new  regional  group  will  typically  set  up  their 
own  certification  team  that  will  meet  with  the  San  Luis  Obispo  group  to  go  over  the  process  and  consider  the  impact 
of  variations  the  new  group  would  like  to  propose.  (Some  of  those  ideas  have  been  good  enough  that  the  SLOCOE 
board  has  revised  its  process  to  match.  Others  have  been  ones  the  SLOCOE  group  already  tried  and  discarded,  and 


they  are  discouraged.  And  a few  have  been  accommodated  as  minor  variations  in  the  programs  across  different 
counties.)  The  new  group  and  the  SLOCOE  panel  also  typically  meet  to  review  the  first  round  of  a region’s  Level 
Three  portfolios  together,  discussing  how  to  handle  any  issues  that  arise. 

So  far,  such  certification  of  in-service  teachers  is  still  being  handled  regionally.  The  state,  through  the  California 
Technology  Assistance  Program  [CTAP],  has  begun  to  look  at  ways  to  handle  it  state-wide.  The  initial  developers 
of  this  process  are  hoping  that  CTAP  will  do  something  along  the  lines  of  SLOCOE’s  design:  CTAP  could  review 
and  approve  a series  of  regional  programs  (as  regional  groups  do  for  Level  Three  personnel),  CTAP  could  let  the 
regional  panels  handle  local  reviews  (as  Level  Three  educators  can  initially  sign  off  on  Level  One  and  Two 
competencies),  and  CTAP  could  still  review  select  procedures  and  artifacts  to  make  sure  the  regional  programs  are 
maintaining  the  common  standards  (as  the  regional  groups  do  across  their  local  districts). 
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Abstract;  This  paper  looks  at  the  connection  between  organisation,  collaboration  and 
learning  in  virtual  learning  environments  (VLEs).  Our  main  points  of  focus  is  investigating 
the  extent  to  which  course  developers  and  course  instructors  need  to  consider  organisational 
measures  and  design  to  trigger  (self-guided)  learning  and  collaboration  of  participants  within 
online  learning  environments  and  how  deeply  individual  learning  processes  are  stimulated  by 
intercultural  collaboration  in  multinational  groups.  The  design  of  these  virtual  learning 
environments  or  learning  networks  involves  an  intricate  balance  between  the  organisation  of 
the  content;  how  the  instructional  activities  are  sequenced;  how  the  interactions  between 
students,  tasks,  and  materials  are  structured;  and  how  the  learning  process  is  evaluated. 
Mentorship  must  be  present  throughout  this  process.  Aspects  of  mentorship  can  manifest 
themselves  in  a variety  of  ways  including:  asking  an  expert,  true  mentoring,  tutoring,  and 
peer  to  peer  support. 


Introduction 

Our  starting  point  and  subject  of  this  study  is  the  "Online  Seminar"  of  the  University  of  Innsbruck  and  the 
Saarland  University,  which  has  run  six  times  over  the  past  few  years,  and  has  dealt  with  various  topics 
(http://seminar.jura.uni-sb.de).  Special  concerns  for  this  course  include: 

It  is  directed  towards  an  international  audience 
It  accommodates  a large  number  of  participants  (100-250  people) 

It  has  diverse  student  profiles,  i.e.  a mixture  of  students  and  people  already  working  in  their  profession  (mixture 
of  university  and  continuing  education) 

It  actively  integrates  all  participants 
It  utilises  collaboration  as  a vital  part  of  the  course  concept 
It  is  transparent  in  all  activities 
It  provides  constant  evaluation 

Keeping  the  course  concept  as  flexible  and  open  as  possible 

One  advantage  of  online  courses  is  the  potential  for  communication,  collaboration,  and  the  constant  exchange 
of  knowledge  between  students  and  educators,  which  brings  them  closer  to  (and  perhaps  supersedes)  "real  life" 
teaching.  The  Online  Seminars  has  opened  up  new  possibilities  for  students  and  teachers,  and  permit  inter- 
disciplinary and  international  collaboration  but  have  also  proven  to  be  far  more  time  and  personnel  intensive 
than  regular  courses.  Computer-Mediated  Communication  (CMC)  therefore  becomes  an  important  research 
area  for  education.  In  this  paper  some  basic  thoughts  about  the  development  of  communication-centred  online 
web-based  courses  will  be  discussed  and  related  to  some  of  our  experiences  concerning  the  development  and 
administration  of  such  courses.  For  this  purpose  the  structure  and  content  of  the  recent  online  course  will  be 


described  identifying  problematic  aspects  specific  to  online  courses  and  how  those  issues  might  be  resolved  in 
future  courses.  Our  investigation  explores  the  role  of  organisation  of  communication  and  moderation,  the 
analysis  of  interactivity  and  working  styles,  and  the  results  of  the  learning  process.  The  questionnaire  filled  out 
by  the  participants  points  out  student  opinions  and  finally  adds  another  perspective  concerning  the  effective- 
ness of  such  kind  of  courses. 

Designing  collaboration  and  learning 

These  issues  are  complex  and  varied.  Some  of  the  requirements  for  designing  and  developing  effective  online 
courses  are,  attention  to  motivation  and  that  it  must  afford  interactivity.  A vast  investment  of  time  and  energy 
during  the  design,  development  and  the  realisation  of  such  online  courses  are  mandated  to  ensure  organisa- 
tional and  support  activities  to  ward  off  potential  problems.  Online  VLEs  present  technical,  transitional  and 
pedagogical  concerns.  As  (Harasim  et  al.  1995)  notes,  "an  online  mentor  is  a professional  in  a particular 
subject  area  who  provides  on-going  feedback  until  the  apprentice  (student)  masters  the  learning  task.  At  that 
point  the  mentor  fades  away,  and  the  apprentice  engages  in  the  exploration  of  expert  practice."  Mentorship 
then  takes  on  several  added  dimensions  within  a virtual  learning  space.  The  VLE  presents  a new  atmosphere 
that  the  student  and  teacher  must  learn  how  to  navigate.  Online  support  must,  on  one  hand,  mentor  technical 
work  and  on  the  other  hand  provide  online  course  support  to  mentor  content  and  guide  the  learning  process. 
This  melding  of  mentorship  and  purpose  begins  to  reshape  the  face  of  learning  and  education.  Without  the 
active  help  of  professors  and  lecturers  (occupied  with  content  support,  management  and  didactics)  as  well  as  a 
number  of  students  who  volunteered  to  assist  as  moderators  during  the  online  course  the  high  goals  set  for  the 
course  could  not  heave  been  reached. 

In  an  online  environment  you  can  expect  mentoring  to  take  on  a new  dimension.  Here  we  are  challenged 
to  mentor  in  ways  that  (Feuerstein  et  al.  1980)  and  others  define  as  mediation.  By  facilitating,  modelling,  and 
coaching  we  begin  to  have  new  insights  into  mediation,  instructional  strategies,  response  to  learner  queries, 
and  facilitating  online  discussion  and  fostering  critical  thinking.  We  anticipate,  through  an  analysis  of  the 
discussion  threads  and  collaboration  threads,  that  we  will  be  able  to  identify  types  of  dialogue  as  well  as  a 
framework  for  guidance  to  support  and  facilitate  a collaborative  online  environment. 

Dialogue  and  inquiry  become  a powerful  medium  for  collaborative  learning.  Discussion  begins  to  carry 
a flow  a meaning.  Threads  of  conversation  can  be  woven  together.  Opinions  and  points  of  view  become  circu- 
lated as  observations,  assumptions  and  interpretations  become  visible.  Effective  guidance  and  mentoring  can 
be  achieved  through  selective  intervention.  A skilled  moderator  is  a mentor  who  can  keep  the  fragile  balance 
between  advocacy  and  inquiry.  The  moderator  must  facilitate  rather  than  dominate  the  discussion.  When  we 
focus  on  (Vygotskys  1978,  1986)  idea  of  the  learner's  zone  of  proximal  development,  scaffolding,  and  dia- 
logue, we  have  to  turn  our  attention  to  fostering  learning  via  virtual  discussions,  using  strategies  that  include 
“voice”,  address  learning  and  writing  styles,  and  utilise  Socratic  dialogue.  These  ideas  require  us  to  tend  to 
both  the  individual  and  group  learning  process.  If  we  want  to  further  online  discourse,  it  is  important  not  to 
forget  that  this  learning  process  is  complex,  social  and  interactive.  Research  in  social  collaboration  and  nego- 
tiation in  the  vein  of  (Slavin  1987)  and  others,  calls  attention  to  the  design  and  support  of  what  these  learning 
environments  require.  Responsibilities  for  discussion  become  shared,  feedback  becomes  integral,  and  all  par- 
ticipants, including  the  moderator,  are  equal.  Both  instructor  and  learner  are  challenged  to  grow  and  develop 
in  this  environment.  Instructors  grow  to  become  moderators  and  guides  and  students  grow  as  learners. 

Our  goals  as  moderators  and  discussion  facilitators  is  to  keep  the  “conversation”  flowing,  focus  and 
refine  ideas,  challenge/ask  learners  to  explore  deeper  into  critical  issues,  and  support  and  scaffold  an  online 
community.  Facilitating  this  critical  thinking  begins  with  the  questioning  process.  Questions  need  to  stimulate 
creative  thinking  and  exploration  of  a topic.  Marrying  strategies  related  to  electronic  communication  and  criti- 
cal thinking  will  help  foster  a community  of  learners  in  a virtual  environment.  If  the  moderator  demonstrates 
skilful  and  thoughtful  questioning  to  engage  discussion  participants,  learners  begin  to  learn  how  to  ask 
questions  of  each  other.  Engaging  questions  will  stimulate  discussion  and  participation,  if  discussion  lags;  the 
attentive  moderator  will  step  in  to  perturb  the  process.  Although  there  are  no  right  or  wrong  answers,  there  are 
right  and  wrong  questions.  The  “right”  questions  provide  the  seeds  for  conversation  that  empowers  the  learner 
to  actively  participate  and  contribute.  Student  ownership  of  information,  their  ability  to  present  and  defend  a 
position,  and  the  exchange  of  ideas  all  enhance  the  learning  process. 
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Online  collaboration  as  well  as  effective  moderating  must  be  designed  if  it  is  to  have  comparable  impact 
to  face-to-face  instruction.  Through  conscious  effort  and  application  ” virtual"  moderators  can  bring  voice  and 
tone  into  an  otherwise  sterile  and  distant  environment,  accompanying  the  learner  as  a reflective  guide  or 
maybe  even  much  like  a personal  muse.  Moderators  are  mediators  and  facilitators;  generative  guides  and  con- 
ceptual facilitators.  The  same  issues  that  can  be  a weakness  in  a virtual  learning  environment  can  be  exploited 
creatively  to  make  learning  come  alive. 

Active  learning  is  at  the  heart  of  the  design  of  the  Online  Seminar.  Designing  activities  that  ensure 
active  participation  is  part  of  the  moderation  and  discussion  process.  Facilitating  the  discussion  is  one  aspect. 
Another  aspect  is  making  the  discussants  responsible  for  their  own  learning  by  sharing  the  responsibility  and 
leading  group  discussion  and  work.  Roles  that  are  scaffolded  and  modelled  by  the  facilitators  become  shared 
then  taken  over  by  the  participants.  All  participants  in  the  course  are  provided  the  opportunity  to  experience 
being  facilitators  in  discussion,  team  leaders,  presenters  of  information,  observers,  and  all  are  responsible  to 
provide  feedback  and  input  into  the  learning  process.  However,  for  any  of  this  to  occur,  a certain  amount  of 
control  needs  to  be  relinquished  and  turned  over  to  the  group.  Moderators  must  stay  in  the  shadows  allowing 
the  learner  to  shine. 

But  perhaps  at  the  heart  of  design  is  the  feedback,  feedback  that  the  moderator  needs  to  provide  in  order 
to  support  the  learning  process.  Feedback  that  promotes  and  encourages  continued  growth  as  a learner  and 
encourages  collaboration.  Substantive  feedback  not  only  needs  to  be  built  into  the  course  design,  but  also  needs 
to  be  modelled  by  the  facilitators  if  we  are  to  expect  quality  feedback  from  students.  Feedback  needs  to  be  con- 
structive. Connections  need  to  form  between  participants;  commonalties  as  well  as  differences  need  to  be 
discussed.  A shared  interdependence  creates  a collaborative  environment. 

Our  goal  was  that  course  design  and  development  would  reflect  a commitment  to  high  quality  related  to 
state  of  the  art  research  for  teaching  and  learning.  Hence,  "active  learning”  and  a focus  on  the  "communica- 
tive process”  between  and  among  students  and  teachers  were  considered  to  be  essential  as  we  investigated 
designing  "rich  online  learning  communities”.  Since  it  is  a virtual  environment  to  be  implemented  based  on 
the  Internet,  it  was  clear  that  quality  could  be  enhanced  if  the  potentials  of  the  Internet  could  be  exploited. 
Those  potentials  include  information  and  communication  potential;  the  flexibility  of  time  and  space  of  learn- 
ing; and  access  for  everyone  without  the  need  of  additional  financial  investments  in  equipment  or  tele-com- 
munication. Therefore  the  design  of  a flexible  learning  environment  needs  to  consider: 

Openness  in  design  towards;  user-groups  to  address  needs  (cultures,  disciplines,  professions,  number)  for 
teaching  and  learning,  media  being  integrated,  implementation  of  pedagogical  features  and  feedback  from 
learners  and  other  participants 

Flexibility  towards;  applied  educational  concepts  and  their  evolving  updates,  contents  to  be  delivered,  applied 
pedagogical  modes  and  methods  for  teaching  and  learning,  duration  of  courses,  application  of  (and  changing) 
technologies  and  evolving  design 

Realising  an  active  participation  of  all  persons  by;  supplying  and  integrating  the  communication  tools  needed 
and  integrating  motivational  features  for  improving/enhancing  interaction 

Easy-to-use  for  learners  and  instructors,  in  form  of  low  technical  barriers,  motivational  atmosphere  and 
providing  administrative  tools 

Taking  into  account  existing  resources  concerning,  technology,  personal  and  finances  with  a clear  perspective  for 
future  continuation  (no  project  status!) 

Integrating  a platform  for  following  research  interests  and  improving  education  (e.g.  via  evaluation). 

There  is  a tension  between  pedagogical  needs  and  resources  and  efforts  in  regard  to  finances,  organisation  and 
technology.  Therefore  a compromise  must  be  found  for  defining  the  requirements  taking  into  account 
economical  and  technical  issues  as  well  as  specific  legal  regulations  in  formal  education  that  define  obligations 
related  to  privacy,  fees,  etc.  when  running  an  online  course. 

As  (McGreal  1998)  notes  that  commercially  available  integrated  distributed  learning  environments  such 
as  Courseinfo  (http://www.blackboard.com/)  are  popular  given:  the  lack  of  sustained  yields  for  individual 
course  development  within  curriculum,  the  high  time  investment  for  instructors,  and  the  lack  of  a supportive 
technology  infrastructure.  Despite  the  administrative  and  implementation  advantages  these  course  manage- 
ment systems  offer,  these  packages  lessen  the  authoring  power  of  the  instructor,  assume  specific  pedagogical 
approaches,  confine  in  some  ways  the  delivery  of  instruction,  and  limit  the  control  of  the  instructor  over  the 
learning  environment.  Given  our  requirements  and  needs,  we  wanted  control  over  the  tools  and  delivery 
method  to  be  able  to  be  responsive  and  adaptive  to  learner  and  instructor  needs  relative  to  the  content  being 
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presented.  We  wanted  the  learning  environment  to  be  able  to  evolve  and  grow.  We  also  wanted  the  environ- 
ment to  be  learner  centered  and  learner  driven  - hence  our  focus  on  creating  community  and  a collaborative 
learning  environment.  Subsequently  design  looked  to  the  aforementioned  needs  for  input  rather  than  limiting 
the  design  and  development  to  adhere  to  a singular  learning  theory.  Rather,  we  focused  on  developing  a set  of 
tools  for  the  learners  and  designed  how  we  asked  that  learner  to  engage  the  information  to  create  the  kind  of 
interactivity  and  the  level  of  ownership  needed  within  the  learning  process.  We  intended  to  create  a "rich” 
learning  environment,  which  Wilson  describes  as  a setting  in  which  the  student  is  ’’engaged  in  multiple 
activities  in  pursuit  of  multiple  learning  goals,  with  a teacher  serving  the  role  of  coach  or  facilitator”  (Wilson 
1996). 

Since  the  idea  is  the  building  of  ’’learning  communities”  to  achieve  better  and  more  active  learning 
processes  by  the  students  a definition  of  constructivist  learning  environments  given  by  (Wilson  1996)  would  fit 
best  to  the  Online  Seminar  environment:  “a  place  where  learners  may  work  together  and  support  each  other  as 
they  use  a variety  of  tools  and  information  resources  in  their  guided  pursuit  of  learning  goals  and  problem- 
solving activities”.  However,  not  all  tasks  performed  within  the  courses  are  constructivist.  It  is  Apparent 
within  the  design  the  influences  of  other  learning  theories  and  pedagogical  assumptions  e.g,  such  as  cognitiv- 
ism and  social  constructivism,  Cunningham  et  al.  (1993)  listed  several  principles  for  the  design  of  a learning 
environment: 

Provide  experience  with  the  knowledge  construction  process, 

Provide  experience  in  and  appreciation  for  multiple  perspectives, 

Embed  learning  in  realistic  and  relevant  contexts, 

Encourage  ownership  and  voice  in  the  learning  process. 

Embed  learning  in  social  experience, 

Encourage  the  use  of  multiple  modes  of  representation 
Encourage  self-awareness  of  the  knowledge  construction  process. 

These  principles  becomes  instruct! onally  designed  focused  and  more  concrete  when  taking  a look  at  (Xiadongs 
1995)  presentation  of  a typical  instruction  design  process: 

Identify  objectives  (e.g.,  what  do  you  want  students  to  be  able  to  do  when  they  have  completed  the  instruction?) 
Assess  students’  prior  knowledge  and  skills  (e.g.,  determine  whether  the  target  students  have  the  prerequisites 
to  benefit  from  the  instruction) 

Specify  the  content  to  be  taught  (e.g,,  what  content  skills  should  be  taught  to  students?) 

Identify  instructional  strategies  (e.g.,  what  instructional  methods  should  be  used?) 

Develop  instruction  (e.g.,  a learner’s  manual,  instruction  materials,  tests,  and  an  instructor’s  guide) 

Test,  evaluate,  and  revise  (e.g,  how  should  students  be  evaluated  to  determine  the  degree  to  which  students  have 
meet  the  performance  objectives?).” 

It  must  nevertheless  be  stated  that  providing  education  in  virtual  learning  environments  requires  a multitude  of 
facets  to  be  considered  as  can  be  seen  from  this  preceding  list. 

Organising  collaboration 

The  online  seminar  is  a co-operative  international  venture.  The  roles  of  each  of  the  co-operating  institutions 
dependent  on  the  focus,  resources,  personnel,  and  disciplines  of  these  institutions.  Currently  (Summer  2000) 
this  collaborative  venture  involves  participation  of  Higher  Education  institutions  in  Germany,  Austria,  Sweden 
and  Kazakhstan.  Each  of  these  institutions  may  in  turn  collaborate  with  other  local  institutions.  Co-operative 
decisions  are  based  on  common  educational  and  research  interests,  competence,  persons  in  charge,  technical 
resources  and  the  cultural  background.  Educational  interests  can  therefore  be  implemented  in  ways  not  known 
before  ICT  (Information  and  Communication  Technology)  and  the  Internet  was  available. 

On  a broad  scale,  collaboration  takes  place  among  students,  between  students  and  teams,  and  among 
team-members.  Student  collaboration  needs  to  be  organised  and  co-ordinated  too.  No  learning  environments 
are  known  in  formal  education,  where  students  meet  voluntarily  in  order  to  collaborate  for  performing  certain 
certified  tasks.  Taking  a closer  look  at  the  role  of  collaboration  will  show  considerations  to  be  globally  viewed 
on  four  levels;  On  the  students  level  within  the  course,  on  the  team  level  within  the  course,  on  a local  level  and 
finally  on  an  institutional  level:  i.e.  collaboration  between  international  universities. 

The  Online  Seminar-environment  different  teams  consist  of  some  30  individuals  from  many  different 
countries.  These  team  members  are  lecturers,  agents,  volunteers,  former  seminar  participants  and  students 
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doing  research  work.  Most  of  today’s  volunteers  in  the  moderation  of  group  discussions  are  former  students  of 
the  seminar.  Several  tightly  bonded  groups  have  been  formed  as  a result  of  the  seminar  on  varying  levels.  For 
example  ’’Cyber  Grannies”  was  formed  by  some  of  the  student  cohort  during  the  1999  seminar.  Consider  too, 
this  paper  is  a result  of  collaboration  between  the  principal  project  investigator,  and  two  former  students  of  the 
seminar  who  are  now  members  of  the  team.  Learning  processes  are  also  encouraged  for  the  team  - currently 
we  are  developing  training  for  moderators.  As  you  can  see,  design  is  complex  and  multi-faceted. 

Learning  within  the  environment  is  structured  to  be  multi-levelled  and  complex.  Learners  are  learning 
from  the  team,  but  also  from  other  learners.  The  team  is  learning  from  learners  and  other  team  members  and 
participating  institutions  are  learning  from  learners,  teams  and  other  institutions.  These  complex  webs  of 
different  learning  levels  are  intimately  related  to  the  collaboration  taking  place  across  all  these  levels.  Meta- 
leaming  is  built  into  the  design.  Feedback  has  shown  that  learners  rate  this  collaboration  structure  highly. 

Learner  Profile  of  participants  in  the  1999  summer  term  Online  Seminar  in  short  are;  224  students  from 
46  different  universities,  19  academic  specialties  were  represented  from  humanities  to  technology,  88  partici- 
pants held  some  form  of  employment,  37  countries  were  represented  an  the  ages  ranged  from  20  to  64 

A questionnaire  was  administered  to  determine  attitudes  towards  the  Seminar  and  the  model  of  online 
collaboration  and  learning.  Seminar  students  where  asked  to  grade  the  seminar  on  a scale  from  1 (very  bad)  to 
1 0 (excellent).  Of  those  grading  the  seminar  (80  respondents),  70, 1 percent  gave  a grade  of  8 or  above,  and 
only  5 percent  gave  a grade  below  5.  Apparently  the  participants  have  high  regard  for  the  Seminar.  Still  the 
really  interesting  question  is  whether  the  participants  perceive  any  improvements  in  their  skills.  The  theme  for 
1999  Online  Seminar  was  ’’Using  the  Internet  in  Professional  Life”,  and  the  participants  where  asked  a few 
questions  on  whether  their  abilities  in  using  the  Internet  had  changed.  Most  of  the  participants  were  active 
students,  and  their  professional  use  of  the  Internet  is  primarily  for  studies.  This  activity  may  or  may  not 
include  using  Internet  in  their  chosen  field  of  future  professional  careers.  Two  question  on  this  subject  where 
given  to  the  participants,  who  using  a 5 point  scale  where,  1 = no  improvement  and  5 = very  much  improved. 
On  the  question  if  the  Seminar  helped  improve  the  participant’s  ability  to  use  Internet  in  studies,  77,6  percent 
gave  a grade  of  4 or  5: 

This  seems  to  indicate  that  a goal  of  these  participants  was  to  learn  new  ways  to  use  the  Internet  to 
facilitate  studying.  The  1999  Seminar  topic  was  in  Legal  work,  participants  were  also  asked  if  their  knowledge 
in  this  area  improved  during  the  Seminar.  Although  the  answers  on  this  question  were  a bit  more  evenly  dis- 
tributed, the  overwhelming  majority  responded  that  their  skills  had  improved. 

Group  work  is  central  to  the  Online  Seminar.  Given  that  the  requirements  are  to  collaborate  with 
“strangers”,  this  requirement  becomes  embedded  in  the  strategy  of  the  course  design  to  promote  active  learn- 
ing. Given  this  social  dimension,  it  is  critical  that  design  supports  this  framework  for  building  learning  com- 
munities. As  already  stated  by  (Hermes  2000)  students  learn  ”to  take  responsibility  in  their  hands  and  work  in 
groups  as  autonomous  learners”.  They  build  ’’virtual  teams”  (Berg  1999)  and  follow  strategies  in  completing 
their  tasks  by  developing  and  transforming  their  ideas. 

A lot  of  planning  and  creativity  has  gone  into  the  design  of  the  tasks  for  participants.  Special  care  has 
been  given  to  developing  questions  that  require  the  active  use  of  Internet,  and  facilitate  extensive  co-operation 
of  group  members.  This  deliberate  design  opens  up  the  possibility  of  meta-leaming,  i.e.  changed  knowledge 
regarding  online  group  work.  And  it  is  interesting  to  find  out  that  the  students,  in  general  terms,  are  grading 
their  improvements  higher  in  these  areas.  Almost  half  of  the  respondents  (44,7  percent)  graded  their 
improvement  as  high  as  5 (on  the  5-point  scale)  and  another  34,2  percent  graded  the  improvement  as  high  as 
4.  Additionally,  when  participants  are  asked  if  the  seminar  should  be  offered  in  the  future,  68%  of  respondents 
have  rated  this  response  with  a 5. 

Conclusions 

We  have  investigated  several  aspects  related  to  creating  a socially  constructed  course  design  model.  We  have 
also  reflected  on,  and  asked  if  teachers  and  learners  are  flexible  enough  to  adapt  to  these  innovative  forms  of 
education.  Active  learning  needs  to  be  encouraged  and  facilitated  through  supportive  design.  This  design 
needs  to  have  structure  and  clearly  demarcated  support,  the  learner,  especially  within  the  current  education 
system,  cannot  solely  construct  it.  But  there  need  to  be  guidance  for  teachers  as  well  in  order  to  get  acquainted 
with  and  used  to  new  methods  and  attitudes  of  learning. 
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The  future  of  learning  is  still  to  be  found  by  exploring  the  connection  between  organisation  and  collabo- 
ration. Organisation  will  be  facilitated  through  carefully  constructed  collaboration.  There  are  still  questions  to 
be  answered.  How  are  media  selections  for  communications  viewed  and  preferred  by  participants  - and  why? 
The  research  related  to  administrative  functions  is  sparse.  Collaboration  needs  to  be  organised  on  three  levels: 
institutional,  conceptual,  and  on  an  individual  level.  Investments  of  time,  money,  and  effort  need  to  be 
considered. 

As  we  consider  the  Online  Seminar  several  things  have  become  apparent  to  the  team  in  the  design, 
development  and  delivery  of  this  course.  Current  work  and  research  has  been  based  on  face-to-face  instruction, 
we  are  only  just  beginning  to  compile  information  and  experience  for  virtual  learning  environments.  Adapting 
pedagogy  from  face  to  face  instruction  to  virtual  environments  is  first  step  in  the  emergence  of  a pedagogy  that 
is  rooted  in  VLB’s.  For  example,  research  in  computer-mediated  discussion  in  quasi-synchronous  environ- 
ments has  found  that  the  learner  assumptions  and  interpretations  are  very  different  in  a VLB  because  their 
assumptions  and  interpretations  are  based  on  face-to-face  interactions.  Continued  efforts  and  investigations 
will  help  refine  the  research  process. 

The  courses  within  this  collaborative  learning  environment  demonstrate  that  there  is  no  need  to  discuss 
whether  there  will  be  a future  for  teacher’s  profession  competing  with  online  courses.  The  concept  of  this  envi- 
ronment foresees  a very  active  role  of  the  teacher  as  an  organiser,  mentor  and/or  moderator.  But  there  are  still 
strides  to  be  made  in  the  research  on  applying  effective  methods  for  teaching  in  collaborative  learning  envi- 
ronments. 

What  has  the  Online  Seminar  accomplished  so  far?  It  is  believed  that  progress  to  date  has  produced  a 
very  good  test  bed  for  investigation  of  the  following  aspects:  the  organisational  methods  for  this  kind  of 
activity,  methods  of  training  teachers  in  online  teaching,  and  how  to  use  available  resources  more  effectively. 
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Abstract:  This  paper,  discusses  group  process  and  development  from  the  perspective  of  language  use  in  the 
networked  interactive  writing  context  over  time.  It  asserts  that  cohesion  is  a significant  factor  in  group 
development  and  that  the  degree  of  peer  pressure  in  the  form  of  ’’writing  behaviors”  is  very  strong  at  the 
beginning  of  a group  but  lessens  over  time  to  bring  about  a cohesive  functioning  of  that  group.  It  begins  with  a 
definition  of  what  a small  group  is,  continues  with  an  overview  of  the  research  on  classrooms,  socio- 
psychological  environments,  and  computer-mediated  versus  face-to-face  groups.  It  provides  an  overview  of 
several  applicable  models  in  the  group  development  literature  as  a conceptual  framework  for  studying  how 
students  communicate  in  electronic  groups,  and  concludes  with  suggestions  for  needed  research  on  electronic 
groups. 


Introduction 

The  advantages  of  working  together  for  common  benefits  are  often  unmistakable.  Collaboration  allows 
students  to  combine  intellectual  expertise  and  material  resources  to  accomplish  a project  or  task  of  mutual  interest 
(Kraut,  Egido,  and  Galegher  1990).  It  also  allows  them  to  undertake  problems  that  they  are  incapable  of  working  on 
alone  because  of  limitations,  resources,  and  time.  When  students  work  together  on  electronic  networks,  teachers  need 
to  view  communication  and  development  as  part  of  a larger  phenomena-small  group  process.  To  successfully  manage 
electronic  groups,  teachers  need  to  understand  both  the  nature  of  the  electronic  classroom  context  and  the  behavior  of 
the  students  in  it.  It  is  through  the  language-based  dynamics  and  variables  that  are  at  work  in  these  contexts  that 
teachers  can  become  aware  (1)  that  language  use  in  the  form  of  ’’writing  behavior”  reflects  the  process  of  group 
development  in  electronic  group  situations  and  (2)  that  time  and  peer  pressure  are  two  key  factors  which  seem  to 
directly  affect  the  development  of  the  electronic  group.  This  understanding  will  require  considerble  study. 


The  Small  Group 

The  term  small  group  applies  to  both  face-to-face  and  electronic  groups.  A small  group  is  usually  considered 
small  if  it  has  2-30  members.  However,  even  larger  groups  can  have  small  group  characteristics.  What  differentiates  a 
collection  of  individuals  from  a group  is  that  "group  members  (1)  have  shared  values  that  maintain  an  overall  pattern 
of  activity,  (2)  acquire  or  develop  resources  or  skills  to  be  used  in  their  activity,  (3)  conform  to  norms  that  define  roles 
to  be  played  and  have  a sufficient  level  of  morale  to  provide  'cohesiveness,’  and  (4)  have  a specific  goal  with  the 
coordination,  usually  through  leadership,  to  combine  the  resources  and  the  roles  to  accomplish  the  goal"  (Hare, 
Blumberg,  Davies,  Kent,  1996).  Most  group  members  also  need  time  to  develop  intimacy. 


The  Nature  of  Classrooms 

Like  networked  environments,  regular  classrooms  are  very  unique  environments  for  learning  in  their  own 
right.  Doyle  (1986)  in  his  work  on  classroom  management  and  organization,  notes  that  there  are  six  features  that 
make  any  classroom  a unique  environment  of  its  own.  These  features  are: 


1.  Multi-dimensionality 

2.  Simultaneity 


- many  different  events  and  tasks  occurring 

- many  things  are  happening  at  the  same  time 


3.  Immediacy 


--  events  are  taking  place  at  a rapid  pace 


4.  Unpredictability  - events  are  taking  unexpected  turns 


5.  Publicness 

6.  History 


- events  are  witnessed  by  many 

- classes  develop  their  own  past-early 
events  shape  subsequent  actions 


In  a sense,  then,  any  classroom  is  a highly  complex  and  potentially  unstable  environment.  This  is  even  more  true  of 
the  computer  classroom,  for  when  we  add  network  technology,  we  create  even  more  complex  types  of  social  forums. 
On  electronic  networks  students  can  enter  a group,  experience  group  process  dynamics,  and  engage  in  social  discourse 
without  ever  having  encountered  another  human  being  in  face-to-face  interaction.  Given  the  complexity  and 
unpredictability  of  such  classroom  environments,  the  teacher  understandably  is  going  to  be  concerned  primarily  with 
classroom  (or  network)  management— taking  the  necessary  steps  to  ensure  that  the  students  are  both  attentive  and  not 
disruptive.  Leadership  and  management  skills,  then,  take  on  a whole  new  dimension  in  these  classrooms,  as  do  the 
underlying  social  philosophies  and  pedagogies. 

Classroom  Environments:  The  Research  on  Socio-Psychological  Environments 

If  most  classrooms  provide  highly  complex  and  potentially  unstable  environments,  requiring  such 
management  skills,  there  would  be  benefit  in  knowing  what  factors  suggested  in  the  literature  contribute  to  a more 
simple  and  stable  environment  for  students.  The  research  on  socio-psychological  environments  suggests  that  the  socio- 
psychological  environment,  for  example,  correlates  highly  with  both  student  attitudinal  and  achievement  outcomes. 
Hattie,  et  al.  (1987),  who  looked  into  perceptions  of  students'  preferred  learning  environments,  concluded  that  students 
preferred  environments  that  provide  more  teacher  friendliness,  less  competition,  and  greater  cohesiveness.  A research 
synthesis  by  Haertel  et  al.  (1981)  found  that  improved  achievement  was  present  in  classes  where  there  was  greater 
satisfaction,  more  goal  direction,  less  disorganization  and  friction,  and  greater  cohesiveness.  Moos  (1980),  who 
evaluated  200  classrooms  using  the  Classroom  Environment  Scale  (CES)(Moos  and  Tricket,  1974)  concluded  among 
other  findings  that  classrooms  that  combined  warm  supportive  relationships  with  a focus  on  academic  tasks  and 
accomplishments  in  well  structured  surroundings  tended  to  lead  to  a gain  in  achievement  and  led  to  personal  and 
creative  growth  as  well  (see  Figure  1).  What  is  suggested  by  the  findings  of  this  research  is  the  general  importance  of 
the  socio-psychological  environment  in  student  learning  and  a general  consensus  that  an  effective  environment 
addresses  both  student  perceptions  and  results  in  better  achievement. 

Supportive  relationships 

+ Well  Structured  Surroundings 

= Achievement,  Personal  Growth  & Creative  Growth 

Figure  1 : Components  of  an  effective  learning  environment. 

Networked  Environments:  The  Research  on  Computer-Mediated  and  Face-To-Face  Groups 

While  classroom  environments  solely  involve  face-to-face  interactions  between  and  among  students  and  the 
teacher  in  the  classroom,  computer-mediated  groups  in  networked  environments  (with  the  exception  of  classroom- 
based  LAN's),  have  primarily  electronic  interactions.  In  addition  to  the  research  on  socio-psychological  environments, 
there  are  findings  in  the  research  on  computer-mediated  and  face-to-face  groups  which  suggest  the  highly  complex 
potentially  unstable  nature  of  the  electronic  environment  (see  Figure  2).  For  example,  studies  using  small  experimental 
groups  of  variable  duration  found  that  computer-mediated  groups  had  a more  equal  participation  rate  (Easton  et  al.; 
Hiltz,  Johnson  and  Turoff  1986;  Siegel  et  al.  1986)  and  took  longer  to  reach  consensus  (Gallupe  et  al.  1991,  Siegel  et 
al.  1986)  than  face-to-face  groups.  They  also  found  that  computer-mediated  groups  engaged  in  more  inflammatory 
communication  (Siegel  et  al.  1986),  task-oriented  behavior  (Hiltz  et  al.  1986),  and  socio-emotional  communication 
(Hollinshead,  McGrath  and  O'Connor  1993).  The  lack  of  consistency  in  these 

findings  in  terms  of  task  and  communication  variances  can  be  explained  in  Walther  and  Burgoon's  (1992)  research 
which  found  an  interaction  effect  of  social  behaviors  and  task  orientation.  They  found  that  in  extended  time 
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asynchronous  computer  interactions,  computer-mediated  groups  became  less  inflammatory,  less  task  oriented,  and 
more  social. 


Over  Time 



/ 

Findings  compared  to  Face-To-Face: 


more  equal  participation  rate 
more  time  to  reach  consensus 
more  inflammatory  communication 
more  task  oriented  behavior 
more  socio-emotional  behavior 


no  comparison 

no  comparison 

lessened 

lessened 

increased 


Figure  2:  Research  findings  on  face-to-face  and  computer-mediated  groups. 


The  Common  Ground 

What  is  apparent  from  these  findings  on  both  classroom  environments  and  computer-mediated  versus  face-to- 
face  groups  is  that  communication  is  at  the  heart  of  any  group  situation,  whether  it  is  a traditional  classroom  or  an 
electronic  group,  particularly  as  it  relates  to  the  human  intimacy  in  the  interaction.  What  is  also  apparent  is  that  all 
types  of  social  interaction  are  an  attempt  by  participants  to  Tit  in',  much  in  the  same  way  that  teenagers  attempt  to  fit 
into  the  peer  group.  According  to  communication  theorists,  all  behavior  is  communication  and  all  effective 
communication  requires  a shared  culture.  If  cohesion  is  a consistent  outcome  in  student  perception  of  what  constitutes 
an  effective  environment  as  well  as  in  achievement  ("a  shared  culture"),  then  time  will  play  a significant  role  in 
achieving  cohesion.  If  behavior  is  communication,  then  the  students'  writing  will  reflect  that  behavior.  Lastly,  if  peer 
pressure  influences  behavior,  it  also  makes  sense  that  students'  "writing"  behavior  will  change  over  time  in  the 
electronic  group  context. 


Working  Within  Established  Models  and  Formulations  on  Group  Development 

Face-to-face  group  development  research  suggests  that  groups  do,  indeed,  move  through  stages  associated 
with  specific  turning  points  in  group  activity.  Many  researchers,  for  example  follow  the  formulation  of  Tuckman 
(1965).  Farrell,  Heinemann,  and  Schmitt  (1986)  following  the  formulation  of  Tuckman,  distinguish  four  phases  which 
identify  the  change  in  salience  of  roles.  In  phase  one,  referred  to  as  testing  and  dependency,  group  members  are 
uncertain  by  what  is  entailed  in  group  membership  and  what  is  to  be  gained  by  it,  while  at  the  same  time  trying  to  gain 
some  sort  of  orientation  and  guidance.  In  phase  two,  or  conflict,  there  is  a great  deal  of  polarization  and  conflict, 
which  is  followed  in  phase  three,  known  as  cohesion  and  consensus,  by  members  beginning  to  trust  one  another  and 
to  master  work  demands.  In  phase  four,  referred  to  as  functional  role  relatedness,  group  members  have  achieved  a 
structure  and  culture  for  the  group  and  fully  cooperate  on  tasks. 

Robbins  (1993),  also  proposed  a model  using  the  terminology  of  Tuckman  (1965)  and  also  the  theoretical 
underpinnings  of  Gersick,  whose  work  addressed  the  factor  of  time.  Tuckman's  stages  of  development  include  (1) 
forming,  (2)  norming,  (3)  storming,  (4)  performing,  (5)  and  adjourning,  and  according  to  Gersick  (1988,1989),  group 
development  is  underscored  by  periods  of  inertia  and  highlighted  by  periods  of  change.  The  model  proposed  by 
Robbins,  therefore,  addresssed  both  the  sequential  and  cyclical  aspects  of  Tuckman  and  Gersick  and  had  five  stages 
which  included  (1)  forming  and  norming,  (2)  low  performing,  (3)  storming,  (4)  high  performing,  (5)  and  adjourning, 
(see  Figure  3) 


Tuckman 

FORMING 

NORMING 

STORMING 

PERFORMING 


IDS': 


ADJOURNING 


Farrell,  Heinemen,  Schmitt  Robbins 

PHASE  ONE:  Testing  and  Dependency  FORMING  AND  NORMING 

LOW  PERFORMING 

PHASE  TWO:  Conflict  STORMING 

PHASE  THREE:  Cohesion  and  Consensus  HIGH  PERFORMING 

PHASE  FOUR:  Functional  Role  Relatedness  ADJOURNING 

Figure  3:  A Comparison  of  Group  Development  models  (Tuckman,  Farrell  et.  al.,  and  Robbins). 

Srivastva,  Obert,  and  Nielsen  (1977)  offer  another  valuable  perspective  on  group  development  by  presenting 
their  five  stages  both  in  terms  of  the  structural  issues  and  in  terms  of  the  basic  issues  that  are  being  dealt  with  by  the 
group  (see  Figure  4).  They  suggest  that  group  members  move  from  wanting  inclusion,  to  dealing  with  influence, 
finally  arriving  at  intimacy. 

Stage  Structure  of  Group  Basic  Issues 

1 Each  person  for  him  or  herself  inclusion 

2 Dyadic  relationship  inclusion->  influence 

3 Coalition  and  clique  formation  influence 

4 Spread  effect  (enlarging  membership  to  entire  group)  influence-->  intimacy 

5 Goal-directed  task  group  formation  intimacy 

Figure  4:  Group  development  from  both  a structual  and  basic  issues  perspective. 


Needed  Research 

Each  of  the  previously  discussed  models,  focuses  on  various  factors  in  group  development.  They  suggest  as 
did  the  literature  on  socio-psychological  environments  and  computer-mediated  groups  the  central  importance  of 
communication  and  time  in  the  group  development  process  and  the  important  role  communication  and  time  play  in 
lessening  peer  pressure  and  thus  creating  cohesion  in  groups.  However,  social  understanding  of  student  behavior  will 
only  come  with  the  study  of  the  interactions  of  students  within  smaller  communities  such  as  the  classroom  (face-to- 
face)  or  within  the  shared  culture  of  a small  electronic  "peer"  group.  One  study  which  began  to  address  this  need, 
Schipke  (1992),  summarized  the  characteristics  of  student  writing  behavior  and  language  use  over  time  on  a computer 
network.  Based  on  a review  of  electronic  transcripts  and  data  it  was  found  that  when  the  group  lacked  cohesion  and 
peer  pressure  was  strong  students  engaged  in  unproductive  writing  behavior  and  language  use.  When  group  cohesion 
was  attained  after  a period  of  time,  peer  pressure  lessened  and  students  began  exhibiting  more  positive  and  productive 
writing  behavior  and  language  use.  These  findings  applied  to  the  revelation  of  thoughts  and  ideas;  to  response  time, 
length,  correctness,  direction  and  focus;  and  to  language  choice  and  use  (see  Figures  5 and  6 for  a presentation  of  the 
specific  characteristics  in  the  context  of  the  Farrell  et  al.  and  the  Robbins  models  respectively). 

Conclusion 

Understanding  what  transpires  in  electronic  groups  will  take  considerable  investigation.  The  literature 
suggests  that  such  investigation  will  need  to  be  conducted  over  time  to  understand  the  growing  pains  of  the  electronic 
group  through  the  kinds  of  communication  taking  place  in  that  context.  It  also  indicates  the  importance  of  such  factors 
as  cohesion,  peer  pressure,  and  writing  behavior  in  furthering  our  understanding.  These  aspects  will  be  important,  but 
will  only  be  a small  piece  of  the  research  understanding  necessary  to  teach  effectively  using  electronic  groups. 
Understanding  will  also  require  the  study  of  assorted  types  and  sizes  of  groups,  different  technological  platforms, 
diverse  communication  processes,  various  roles  and  behavior,  group  tasks,  group  functions,  socio-emotional  variables, 
and  comparisons  to  face-to-face  and/or  classroom  contexts,  just  to  name  a few.  However,  these  kinds  of  longitudinal 
observations  can  reveal  valuable  insights  into  effective  pedagogy  and  management  for  the  electronic  instructional 
context. 


1038 


PHASE  ONE:  Testing  and  Dependency 


Self-conscious  about  revealing  their  thoughts  and  ideas 
Lag  time  in  response  to  posted  comments 
Express  themselves  using  brief  comments  or  short  sentences 
Overly  concerned  with  grammatical  or  spelling  errors 
Discussions  seem  chaotic  and  undirected 
Lack  focus  and  commitment 
Tend  to  play  using  language 

PHASE  TWO:  Conflict 

Engage  in  grandstanding  behaviors  through 
Disruptive  comments 
Obscenities 

Iconoclastic  use  of  language 

PHASE  THREE:  Cohesion  and  Consensus 

Enthusiastic  and  willing  to  share  thoughts  and  ideas 
Quick  response  time  to  posted  comments 

A more  fluid  dialogue  tempo  is  maintained 
Write  longer,  more  reflective  comments 

Less  concerned  with  surface  errors/more  concerned  with  expressing  their 
ideas  correctly 

Discussions  and  interactions  appear  to  have  direction  and  purpose 

Tend  toward  playful  use  of  language/compete  for  originality  of  content  and  form 

PHASE  FOUR:  Functional  Role  Relatedness 

Engage  in  group  and  task  maintenance  procedures  through 

Supportive  and  focused  comments,  Constructive  criticisms,Suggestions 
Engage  in  Closure  through 

Ending  conunents  including  reflections,  congratulations,  goodbyes. 

* Identifies  the  four  phases  in  group  development  (farrell  et.  al)  following  the  formulation  of  Tuckman  (1965),  and  notes  the 
language  behaviors  (Schipke  1982)  that  are  prominent  during  each  phase. 

Figure  5:  Group  development  from  the  perspective  of  student  writing  behavior  over  time  In  the  context  of 
Farrell  et  al.'s  four  categories. 

FORMING  AND  NORMING: 

Self-conscious  about  revealing  their  thoughts  and  ideas 
Lag  time  in  response  to  posted  conunents 

LOW  PERFORMING: 

Express  themselves  using  brief  conrunents  or  short  sentences 
Overly  concerned  with  grammatical  or  spelling  errors 
Discussions  seem  chaotic  and  undirected 
Lack  focus  and  commitment 
Tend  to  play  using  language 

STORMING: 

Engage  in  grandstanding  behaviors  through 
Disruptive  comments 
Obscenities 

Iconoclastic  use  of  language 
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HIGH  PERFORMING: 


Enthusiastic  and  willing  to  share  thoughts  and  ideas 
Quick  response  time  to  posted  comments 

A more  fluid  dialogue  tempo  is  maintained 
Write  longer,  more  reflective  comments 

Less  concerned  with  surface  errors/more  concerned  with  expressing 
their  ideas  correctly 

Discussions  and  interactions  appear  to  have  direction  and  purpose 

Tend  toward  playful  use  of  language/compete  for  originality  of  content  and  form 

Engage  in  group  and  task  maintenance  procedures  through 

Supportive  and  focused  comments,  Constructive  criticisms.  Suggestions 


ADJOURNING: 

Engage  in  Closure  through 

Ending  comments  including  reflections,  congratulations,  and  goodbyes. 

* Identifies  the  Five  Stages  in  group  development  (Robbins  1993)  who  used  the  terminology  Tuckman  (1965)  and  the  theoretical 
underpinnings  of  Gersick  (1988,  1989),  and  notes  the  language  behaviors  (Schipke  1982)  that  are  prominent  during  each  phase. 

Figure  6:  Group  development  from  the  perspective  of  student  writing  behavior  over  time  in  the  context  of  Robbins' 
five  stages. 
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Abstract:  As  faculty  and  institutions  continue  to  adopt  the  World  Wide  Web  for  delivering 
course  materials,  they  are  encountering  a host  of  related  problems.  Institutions  must  from 
everyone  from  novice  users  to  faculty  who  want  to  go  beyond  the  normal  capabilities  of 
HTML.  Sites  must  often  brand  themselves  with  a uniform  “look  and  feel”  and  also  support 
easy  updates  to  that  look  and  feel.  Sites  must  also  support  some  form-based  page  generation. 

In  this  paper,  we  discuss  SATIRIC  (Site  Authoring  Tool  Incorporating  Regular 
Expression  In  Context),  a pattern-based  translation  language  developed  to  support  these  and 
other  activities.  We  aigue  that  a pattern  language  is  an  appropriate  solution  for  many  tasks. 
Novices  can  write  text  files  and  use  preferred  shorthands;  the  transformation  language  can  turn 
those  shorthands  into  HTML,  XHTML,  or  XML.  Experts  can  write  new  tags  and  define 
transformations  from  those  tags.  Institutions  can  define  standard  page  templates  and 
transformations  that  convert  inputs  to  those  templates.  For  a new  look  and  feel,  they  can  just 
change  the  templates. 


1.  Introduction 

Web  authoring  once  was  fairly  simple:  those  who  wanted  to  use  the  Web  created  a few  pages  a favorite 
authoring  tool  (or  “raw  HTML”)  and  put  it  on  the  Web.  The  enormous  growth  of  the  Web  has  changed 
everything,  particularly  in  academia.  Institutions  must  now  worry  about  supporting  a wide  range  of  page 
authors,  from  faculty  reluctant  to  going  online  to  faculty  eager  to  adopt  every  new  technologies.  Similarly,  the 
movement  toward  sites  containing  thousands  of  pages  has  led  many  to  brand  sites  through  uniform  designs. 

Unfortunately,  support  for  these  different  issues  is  often  handled  in  a somewhat  ad-hoc  method,  often 
leading  to  contradictions.  For  example,  novice  faculty  are  often  trained  in  simple  packages  (often  PageMill  or 
“save  as  HTML”  in  newer  versions  of  Microsoft  Word).  But  neither  PageMill  nor  Word's  save  as  feature  make 
it  easy  for  novice  authors  to  adhere  to  a site  style.  (Thimbleby  1997)  has  suggested  that  we  can  support  uniform 
site  styles  through  templates  plus  variables.  That  is,  we  can  extract  information  about  a page  (author,  body, 
title,  etc.)  and  then  reflow  that  information  into  a template  that  provides  the  site  style. 

But  this  is  not  the  only  way  to  transform  pages  to  meet  stylistic  needs.  Pattern-based  page  transformation 
languages  (e.g.,  (Rebelsky  and  de  Beer  1998)  can  permit  site  designers  to  specify  standard  changes  to  pages, 
such  as  “change  all  body  tags  so  that  documents  use  the  following  colors  and  fonts”  or  “change  all  list  items  to 
use  filled  circles”.  By  careful  design  of  rules,  it  is  possible  to  automate  most  important  aspects  of  site  design. 

In  this  paper,  we  describe  a particular  pattern-based  page  transformation  language,  SATIRIC,  the  Site 
Authoring  Tool  Incorporating  Regular  Expressions  in  Context.  Unlike  general-purpose  pattern-based 
languages,  such  as  Perl  and  Awk,  SATIRIC  is  designed  specifically  for  developing  and  transforming  formatted 
text.  SATIRIC  supports  rich  patterns  through  the  use  of  regular  expressions.  It  adds  power  to  these  patterns 
through  the  use  of  variables  and  by  permitting  regular  expressions  to  be  applied  in  only  certain  contexts. 

SATIRIC  is  designed  so  that  a wide  variety  of  users  can  easily  write  rules.  Typically,  authors  will  describe 
their  general  intent  and  programmers  will  develop  the  rules  to  meet  that  intent.  However,  we  are  developing  a 
user  interface  that  permits  authors  to  write  English-like  text  to  describe  their  rules.  For  example,  someone 
might  write  that  “A  blank  line  signals  the  end  of  a paragraph”  or  “At  the  start  of  a section,  increment  the 
heading  level  and  generate  an  appropriate  heading  tag;  at  the  end  of  a section,  decrement  the  heading  level”. 
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In  Section  2,  we  introduce  the  contextual  patterns  of  SATIRIC  through  a number  of  related  examples.  In 
Section  3,  we  delve  more  deeply  into  regular  expressions,  which  serve  as  the  patterns  for  SATIRIC.  In  Section 
4,  we  describe  the  architecture  of  SATIRIC  and  the  SATIRIC  assembly  language.  In  Section  5,  we  suggest 
applications  of  SATIRIC  for  advanced  HTML  authors.  In  Section  6,  we  discuss  future  directions  for  SATIRIC. 


2.  Supporting  Novice  Authors  and  Plain  Text  Authors 

As  institutions  are  moving  toward  online  course  delivery  and  support,  they  are  finding  it  necessary  to  support 
faculty  who  are  less  comfortable  with  computers  than  early  Web  adopters.  Some  of  these  faculty  don't  see  a 
purpose  to  Web-based  delivery,  and  those  faculty  are  a special  problem.  Others  are  interesting  in  going  online, 
but  don't  have  the  time  or  energy  to  devote  to  learning  a new  computer  program,  even  one  as  simple  as 
PageMill  or  Frontpage.  While  it  is  possible  for  such  faculty  to  use  a favorite  word  processor  and  export  as 
HTML,  the  resultant  HTML  is  often  unsatisfactory  to  both  faculty  member  and  site  designer. 

However,  experience  shows  that  most  faculty  (and  most  writers),  use  “shorthands”  to  indicate  common 
typographical  issues.  Consider  the  sample  text  in  (Fig.  1),  which  one  might  type  in  plain  text  to  describe  the 
formatted  text  in  (Fig.  2).  In  (Fig.  1),  a blank  line  indicates  the  end  of  a paragraph,  an  asterisk  at  the  start  of  the 
line  indicates  a list  item,  text  surrounded  by  underscores  is  emphasized,  text  in  brackets  is  a link  (to  a 
bibliography  entry),  indented  text  is  a quotation,  the  underscore  indicates  a subscript,  the  caret  indicates  a 
superscript,  and  so  on.  The  first  goal  of  SATIRIC  and  its  predecessor,  SiteWeaver/SH  (Rebelsky  and  de  Beer 
1998)  is  to  support  the  transformation  of  such  shorthands  to  HTML  or  other  markup  language. 

In  our  analysis  of  [Jones  1998] , we  found  that  it  is  trivial  to  improve 
his  algorithm  from  0(n^2)  to  0(n  log_2  n) , using  a simple  *divide  and 
conquer*  strategy.  Jones  writes, 

The  algorithm  must  visit  each  element  of  the  collection,  one-by-one. 

Although  a few  elements  need  not  be  visited,  because  they  are 

Irrelevant,  the  number  of  visitations  in  one  round  is  clearly  0(n) . 

However,  we  have  found  that  it  is  possible  to  preprocess  the  data  with 
the  following  strategy; 

* Select  a _separator__ 

* Divide  the  collection  into  two  equal  halves  using  that  separator 

* Recurse  on  the  two  halves 

Figure  1:  Shorthands  in  plain  text. 

In  our  analysis  of  [Jones  1998],  we  found  that  it  is  trivial  to  improve  his  algorithm  from  0(«^)  to 
0(rtlog2rt),  using  a simple  divide  and  conquer  strategy.  Jones  writes, 

The  algorithm  must  visit  each  element  of  the  collection,  one-by-one.  Although  a few 
elements  need  not  be  visited,  because  they  are  irrelevant,  the  number  of  visitations  in  one 
round  is  clearly  0(«). 

However,  we  have  found  that  it  is  possible  to  preprocess  the  data  with  the  following  strategy: 

• Select  a separator 

• Divide  the  collection  into  two  equal  halves  using  that  separator 

• Recurse  on  the  two  halves 

Figure  2:  The  same  text,  reformatted. 

How  do  we  translate  these  shorthands  into  HTML?  We  need  patterns  to  describe  the  various  shorthands. 
For  example,  in  the  standard  regular  expression  notation,  "^\*  represents  “an  asterisk  at  the  start  of  the  line”  and 
\n\n  represents  “a  blank  line”.  Yet  these  patterns  are  not  enough.  Observe  that  the  context  of  a pattern  has 
some  effect  on  the  application.  For  example,  any  blank  line  you  encounter  signifies  the  end  of  a paragraph,  as 
long  as  you  are  in  a paragraph.  Similarly,  an  underscore  has  two  possible  meanings:  emphasis  or  subscript. 
How  can  we  tell  which  the  author  means?  Typically,  through  the  context.  For  example,  if  the  underscore  is 
used  as  part  of  a mathematical  formula,  it  is  likely  to  be  a subscript.  (Fig.  3)  shows  the  translation  of  the 
original  text  to  HTML. 
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<p>In  our  analysis  of  [<a  href=" jonesl998"  class= "cite" >Jones 
1998</a>] , we  found  that  it  is  trivial  to  improve  his  algorithm  from 
<span  class=" formula" >0 ( <i  class="var">n</ixsup>2</sup>)  </span>  to 
<span  class=" formula" >0 ( <i  class="var">n</i>log<sub>2</subxi 
class="var">n</i>) </span>,  using  a simple  <em>divide  and  conquer</em> 
strategy.  Jones  writes, </p> 

<blockquotexp> 

The  algorithm  must  visit  each  element  of  the  collection,  one-by-one. 
Although  a few  elements  need  not  be  visited,  because  they  are 
irrelevant,  the  number  of  visitations  in  one  round  is  clearly 
<span  class= " formula " >0 (<i  class= "var" >n</i>) </span> . 

</p>< /blockquote > 

<p>However,  we  have  found  that  it  is  possible  to  preprocess 
the  data  with  the  following  strategy : </p> 

<ul> 

<li>Select  a <strong>separator</strongx/li> 

<li>Divide  the  collection  into  two  equal  halves  using  that 
separator</li> 

<li>Recurse  on  the  two  halves</li> 

</ul> 


The  use  of  shorthands  is  common  enough  that  major  applications,  such  as  Microsoft  word,  now  include  a 
small  set  of  specified  rules  (such  as  the  rule  that  an  asterisk  at  the  start  of  the  line  indicates  a list  item). 
However,  authors  cannot  design  their  own  shorthands;  while  they  can  use  any  shorthands  they  wish,  they  cannot 
make  Word  acknowledge  them.  By  allowing  authors  (or  their  support  staff)  to  write  patterns,  we  allow  each 
writer  to  design  his  or  her  own  shorthands,  or  a group  of  authors  to  agree  on  a common  set  of  shorthands. 

Recent  events  on  the  Web  suggest  another  important  use  for  shorthands.  There  is  a growing  tendency  to 
build  new  pages  from  input  given  on  forms.  For  example,  a student  might  submit  short  notes  on  a form,  or  add 
annotations  to  a page  using  a form.  Unfortunately,  malicious  users  have  now  targeted  form-based  authoring; 
such  users  add  inappropriate  scripts  and  links  to  pages  (CERT  2000).  This  is  because  most  form-based 
authoring  systems  take  the  form  text  “as  is”,  allowing  it  to  include  HTML  markup  for  more  advanced  authors. 
(CERT  2000)  recommends  checking  for  particular  tags  (e.g.,  script,  applet,  and  form)  and  eliminating  those 
tags.  However,  this  strategy  requires  that  site  authors  update  their  “disallowed  tags”  regularly  as  malicious 
users  find  new  techniques.  A better  strategy  is  to  disallow  any  HTML  in  forms.  Instead,  authors  can  use  a 
shorthand  like  that  given  above. 

The  EdMedia  2000  submission  process  suggests  a similar  reason  that  shorthands  are  useful.  Authors  were 
required  to  submit  only  plain  text  files.  Given  that  most  authors  are  now  accustomed  to  using  formatting  to 
highlight  semantic  intent  (e.g.,  italicizing  emphasized  words)  it  is  difficult  and  inappropriate  to  return  to  plain 
text.  A set  of  common  SATIRIC  rules  would  allow  authors  to  mark  their  plain  text  in  such  a way  that  referees 
can  easily  read  formatted  text. 

We  have  used  similar  techniques  to  mark  up  text  for  a composition  course.  One  of  the  lessons  that  students 
need  to  learn  is  to  associate  nouns  with  the  actors  in  a sentence  and  verbs  with  the  actions  of  those  actors. 
SATIRIC  permits  us  to  mark  the  sentence  once,  using  convenient  shorthand  (e.g.,  asterisks  around  verbs, 
brackets  around  actors)  and  then  format  it  in  different  ways  for  presentation  to  students. 


3.  The  Power  of  Regular  Expressions 

As  (Sect.  2)  suggests,  we  have  chosen  to  use  regular  expressions  for  our  patterns  and  then  add  a context  to  those 
expressions.  Why  regular  expressions?  Because  regular  expressions  are  a powerful  but  fast  way  to  describe 
many  different  kinds  of  patterns.  In  particular,  regular  expressions  support 

• Alternation:  Match  “this  pattern  or  that  pattern”.  Takes  the  form  (this)  | (that) . 

• Sets:  Match  “any  one  of  these  characters”  or  “any  character  between  this  character  and  that 
character”.  Takes  forms  like  [aeiou]  (any  vowel)  or  [a-z]  (any  lowercase  letter). 

• Elimination:  Match  “anything  but  this  character”.  Takes  the  form  [^c] . 


Figure  3.  The  HTML  for  (Fig.  2),  created  by  SATIRIC  from  (Fig.  1). 


• 5e^we/ic//ig:  Match  “this  and  then  that”.  Takes  the  form  (this)  (that). 

• Wildcard  repetition:  Match  “as  many  copies  of  this  pattern  as  you  can”.  Takes  the  form 

(this)  *. 

• Special  characters:  Match  “beginning  of  line”  or  “end  of  line”  or  “whitespace”  or  ... . 

Regular  expressions  have  been  used  for  a variety  of  different  applications,  from  improved  search  and  replace  in 
BBEdit  (Bare  Bones  1999)  to  the  sophisticated  analysis  of  student  hypertext  use  logs  (Jones  and  Jones  1997). 

But  regular  expressions  cannot  match  all  of  the  common  patterns.  Hence,  we  increase  the  power  of  regular 
expression  by  adding  contexts  to  the  patterns.  This  addition  makes  it  easier  and  more  natural  to  express  many 
common  patterns.  For  example,  while  a blank  line  typically  indicates  an  end  of  paragraph,  a blank  line  in  a 
preformatted  section  should  be  left  as  is.  Context  also  extends  the  computational  power  of  the  language. 

In  some  cases,  rules  will  need  to  keep  track  of  what  has  been  seen  before  (e.g.,  the  title  of  the  document) 
and  use  that  value.  We  include  rules  that  incorporate  variables  similar  to  those  used  in  most  programming 
languages.  Rules  can  get  and  set  variables,  append  to  variables,  and  increment  and  decrement  variables. 

At  first  glance,  typical  regular  expressions  are  difficult  for  human  beings  to  read,  although  they  are  easy  for 
computers  to  parse.  Some  describe  complex  regular  expressions  as  “write  only”  in  that  you  never  modify  a 
regular  expression  because  it  is  too  hard  to  figure  out  what  is  going  on  inside.  However,  it  is  also  fairly  easy  to 
turn  common  English  pattern  descriptions  into  regular  expressions.  A current  project  is  to  identify  common 
pattern  utterances  and  write  patterns  to  transform  those  utterances  into  regular  expressions. 

Let  us  consider  a few  examples  of  the  relationship  between  English  text  and  the  corresponding  regular 
expression.  The  pattern  utterance  “a  body  tag”  can  be  represented  as  the  regular  expression 
< [Bb]  [Oo]  [Dd]  [Yy]  ( [^>]  *)  >.  This  pattern  matches  text  that  begins  with  a less-than  sign,  then  either  B or 
b,  then  O or  o,  then  D or  d,  then  Y or  y.  Finally,  there  are  an  arbitrary  number  of  non-greater-than  signs 
(representing  optional  parameters),  followed  by  a greater  than  sign.  Other  tags  can  be  handled  similarly.  The 
utterance  “text  surrounded  by  asterisks”  can  be  represented  as  the  regular  expression  \*[^\*]*\*.  The 
asterisks  prefaced  by  slashes  represent  “just  an  asterisk”.  The  asterisk  without  a prior  backslash  represents  “as 
many  copies  of  the  prior  thing  as  you  can”.  Hence,  this  pattern  is  “an  asterisk,  followed  by  as  many  non- 
asterisks as  you  want,  followed  by  an  asterisk”. 


4.  Architecture 


Given  a source  file  and  a set  of  patterns,  SATIRIC  builds  an  HTML  file  using  the  steps  (Fig.  4)  illustrates. 
First,  the  English-language  rules  (typically  entered  in  a Web  page)  are  converted  to  high-level  rules.  The  high- 
level  patterns  are  then  translated  into  simpler  rules.  Finally,  the  rules  are  applied  to  the  file. 


f User 
I Interface 


I 1 

J L 


/ SATIRIC  ^ 
^ Compiler  j 

V ^ 


Figure  4:  The  Components  of  SATIRIC 

What  is  a SATIRIC  assembly  program?  It  is  a collection  of  simplified  rules.  These  simplified  rules 
provide  all  the  power  of  the  higher-level  design  rules,  but  are  easier  to  implement  quickly.  There  are  a small 
number  of  assembly  rules.  Each  rule  has  a pattern,  an  optional  context,  and  a corresponding  action.  The  basic 
SATIRIC  assembly  rules  appear  in  (Fig.  5).  Note  that  these  differ  slightly  from  the  currently  implemented 
version,  which  handles  context  in  a different  manner. 

Replace:  Generate  replacement  text 

Enhanced  Replace:  Generate  replacement  text  that  may  include  part  of  the  text  as  well  as  variables 
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Load  Text: 
Set: 

Update: 

Apply: 

Load  Rules: 
Start  Context: 
End  Context: 
Close: 


Load  replacement  text  from  a file 
Set  the  value  of  a variable 

Update  the  value  of  a variable  (increment,  decrement,  append  to,  etc.) 
Apply  a set  of  rules  (stored  in  a separate  file)  to  a piece  of  text 
Load  and  begin  to  apply  rules  from  a rules  file 
Start  a particular  context 
End  a particular  context 

HTML-specific;  close  all  open  tags  up  to  a particular  tag 
Figure  5:  SATIRIC  assembly  code  instructions. 


The  rules  fall  into  a number  of  different  groups.  The  first  three  types  of  rules  (Replace,  Enhanced  Replace, 
and  Load  text)  all  generate  replacement  text  for  the  matched  text.  The  next  two  (Set,  Update)  modify  the  values 
of  variables.  There  is  no  corresponding  Get  operation  because  variables  can  be  used  in  replace  and  enhanced 
replace.  The  next  four  instructions  provide  control  over  the  operation  of  SATIRIC,  limiting  the  operation  of 
rules  with  contexts  and  adding  rules  to  the  current  rule  set.  The  final  operation  is  HTML-specific  and  was 
added  to  help  support  better  HTML  authoring;  when  authors  close  a tag,  they  can  use  this  operation  to  ensure 
that  all  intervening  tags  are  closed. 

As  this  description  suggests,  the  language  includes  rules  to  start  and  end  contexts.  In  a related  language 
(Rebelsky  and  de  Beer  1998),  HTML  tags  determine  context.  In  that  language  each  start  tag  begins  a context 
and  each  end  tag  ends  it.  The  new  design  possible  to  use  a variety  of  different  mechanisms  for  denoting 
contexts.  However,  we  still  typically  use  HTML  tags  to  indicate  contexts. 

These  rules  are  not  applied  to  the  whole  file  at  once.  Rather,  the  file  is  separated  into  smaller  chunks  and 
each  chunk  is  processed  in  turn,  as  (Fig.  6)  shows.  The  technique  used  to  “chunkify”  the  file  depends  on  the 
type  of  the  input  file.  HTML  files  are  typically  split  at  each  tag.  In  a future  implementation,  we  anticipate 
providing  rules  that  affect  the  technique  used,  so  that  different  parts  can  be  chunkified  differently. 


Figure  6:  Inside  SATIRIC. 


5.  Supporting  Advanced  Authoring 

A key  part  of  this  design  is  that  the  source  file  need  not  be  a plain  text  file.  The  transformations  can  be  equally 
well  applied  to  existing  HTML  files,  and  we  have  successfully  used  SATIRIC  to  reformat  such  files  to  add 
functionality.  Typically,  new  tags  provide  this  functionality.  For  example,  we  have  used  a version  of  SATIRIC 
to  support  footnote,  tables  of  contents,  and  section  tags.  It  is  easiest  to  understand  their  implementation  in  the 
higher-level  design  language,  which  is  less  repetitious  than  the  corresponding  assembly  language. 

A rule  for  the  footnote  tag  (or,  more  precisely,  the  end-footnote  tag)  does  three  basic  steps.  It  increments 
the  footnum  variable.  It  appends  an  anchor  and  the  footnote  text  to  the  footnote  variable.  Finally,  it  replaces 
that  text  in  the  main  body  with  a link  to  the  footnote,  using  footnum.  How  do  we  get  footnotes?  A rule  for  the 
end-body  tag  says  to  insert  the  footnote  variable.  Other  special  tags  are  implemented  similarly. 

While  the  current  implementation  of  SATIRIC  is  targeted  toward  HTML,  we  have  also  used  SATIRIC  in 
selected  experimental  situations  for  other  purposes.  For  example,  we  have  used  an  early  version  of  SATIRIC  to 
turn  text  files  into  Microsoft  Word  RTF  files.  In  this  case,  we  can  use  the  same  patterns  as  we  would  with 
HTML,  but  write  different  result  texts.  For  example,  instead  of  “given  text  surrounded  by  underscores,  generate 
the  same  text  surrounded  by  <i>  and  </i>”,  we  might  write  “given  text  surrounded  by  underscores,  generate 
the  same  text  surrounded  by  {\i  and  }".  Writing  such  rules  requires  some  understanding  of  the  output  markup 
language,  but  the  rules  only  need  to  be  written  once  and  then  applied  to  as  many  files  as  are  available. 
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SATIRIC  is  also  being  used  within  Web  Raveler  (Kensler  and  Rebelsky  2000)  to  provide  a simple 
mechanism  for  transforming  pages.  Web  Raveler  permits  readers  to  specify  modifications  to  Web  pages  that 
should  be  applied  before  the  pages  are  sent  to  the  browser.  The  modifications  include  filtering  of  images  and 
adding  of  interactive  facilities.  One  such  facility,  Blazer  (Glynn  et  al.  2000),  allows  faculty  to  create  annotated 
trails  on  the  Web.  SATIRIC  is  being  used  for  the  annotations. 

SATIRIC  is  also  appropriate  for  transforming  existing  pages  to  meet  new  stylistic  guidelines.  Such 
transformations  will  typically  be  used  with  individual  tags,  although  more  complex  changes  are  possible. 


6.  Summary 

A number  of  common  Web  applications  require  Web  authors  and  Web  designers  to  convert  existing  files,  from 
plain  text  files  to  existing  Web  pages.  SATIRIC,  our  pattern-based  page  transformation  language,  supports  a 
wide  variety  of  transformations.  SATIRIC  has  already  been  used  for  a number  of  simple  but  useful 
applications,  from  providing  a shorthand  for  authors  to  supporting  special  features,  including  footnotes,  on  Web 
pages. 
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Abstract;  The  Maryland  Electronic  Learning  Community  (www.leam.umd.edu)  is  building  a 
multimedia  digital  library  of  educational  resources.  Now  in  the  fourth  year  of  the  project,  we 
evaluate  early  decisions  we  made  about  segmenting  and  indexing  videos.  We  also  discuss  an 
experiment  in  encouraging  collaborative  community  indexing  with  a Quick  Indexing  Tool. 
We  conclude  that  a broader  base  of  users  would  better  support  the  infrastructure 
requirements  and  we  propose  ways  that  such  a broad  base  can  be  developed  while  also 
providing  a framework  for  local  learning  communities.  We  propose  a federated  system  of 
collaborative  indexing  communities. 


Introduction 

The  Maryland  Electronic  Comrnunity  (MELC)  was  designed  to  investigate  how  the  professional 
development  of  teachers  could  be  supported  with  information  technology  (Rose  et  al.  1999).  One  focus  of  this 
project  has  been  the  creation,  maintenance,  and  cultivation  of  a digital  library  for  the  teachers  to  use  as  they 
develop  interactive  lesson  plans  (ILPs).  While  the  original  emphasis  was  on  the  needs  of  social  studies  and 
science  teachers,  in  recent  months  the  community  has  grown  to  include  mathematics  and  language  arts 
teachers. 

The  digital  library  contains  over  3,200  multimedia  resources,  and  by  August  2000  this  number  is 
expected  to  exceed  6,300.  The  collection  is  comprised  of  text  documents,  audio  files,  video  segments,  still 
images,  and  Web  sites.  There  are  public  domain  items  from  the  National  Archives  and  NASA;  proprietary 
items  that  have  been  contributed  by  project  partners  Discovery  Communications  and  Maryland  Public 
Television;  and  personal  contributions  from  MELC  members  themselves.  Each  resource  is  identified  by  a set 
of  standard  metadata  elements  and  then  indexed  according  to  Topic,  Subtopic,  and  Standard  (corresponding  to 
national  standards).  Efforts  at  collecting  and  cataloging  these  resources  was  initially  the  responsibility  of  the 
team  at  the  University  of  Maryland,  but  this  work  is  gradually  being  handed  off  to  the  teachers  as  the  external 
project  funding  ends. 

By  analyzing  the  Web  logs,  we  can  determined  that  the  following  resources  as  shown  in  Table  1.  We 
are  particularly  interested  in  how  the  video  clips  are  being  used.  Currently  there  are  50  videos  in  the  library: 
24  science,  25  social  studies,  and  1 language  arts.  Seven  of  these  are  in  the  public  domain  (from  the  National 
Archives  and  accessible  by  anyone)  and  45  are  private  (from  Discovery  Communications,  Inc.  or  Maryland 
Public  Television  and  accessible  only  by  MELC  members).  Of  the  video  clips  downloaded,  only  1 0 were  public 
domain.  The  average  video  clip  is  about  two  to  three  minutes  in  length  and  approximately  15  MB.  The  Web 
logs  show  evidence  of  teachers  downloading  new  videos  from  the  server  even  before  CD-ROMs  are  available. 
Putting  the  clips  on  the  server  reduces  the  time  it  takes  to  make  the  videos  available  to  the  teachers  since 
creating  and  distributing  the  CD-ROMs  is  time  consuming. 
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In  this  paper,  we  examine  issues  related  to  the  use  and  development  of  the  resource  collection 
including; 

• the  flexibility  of  digitized  video  segments  (as  opposed  to  full-length  film  on  the  VCR), 

• appropriate  indexing  and  metadata  elements,  and 

• collaborative  indexing  and  the  Quick  Indexing  Tool  (QIT). 


Resource  Types 

Unique  Resources  Accessed 

Total  Resources  Accessed 

Video 

158 

457 

Image 

58 

110 

Web  Sites 

37 

104 

Lesson  Plans 

30 

71 

Student  Activities 

10 

42 

Audio 

6 

14 

Other  Libraries 

3 

4 

Text 

3 

4 

Table  1:  Resource  usage  from  September  1999  to  March  2000. 


Segmenting  the  Videos 

While  the  library  contains  images,  audio,  and  video,  we  focus  here  particularly  on  the  video  because  it 
is  attractive  to  students,  adds  a unique  dimension  to  lesson  plans,  and  is  invaluable  to  visual  learners.  The  full- 
length  educational  videos  are  divided  in  one-  or  two-minute  segments  for  a number  of  reasons.  First,  we 
consider  the  video  segments  to  be  building  blocks  for  the  teachers  as  they  construct  their  lessons.  Second,  our 
technology  was  inadequate  for  the  task  of  managing  digitized  full-length  videos,  especially  at  beginning  of  the 
project.  A third  reason  for  using  segments  is  that  they  encourage  cross-curriculum  tie-ins.  For  example,  “The 
Real  Ben  Franklin”  is  a film  about  the  life  of  a great  American  and  as  a whole  is  clearly  within  the  purview  of 
the  social  studies  department.  However,  it  includes  several  fine  illustrations  of  scientific  principles  and 
inventions  that  would  be  useful  in  a science  class.  As  a result  of  the  segmenting,  science  teachers  can  now  use 
resources  that  they  otherwise  might  not  have  been  aware  of  or  able  to  access. 

While  there  has  been  a great  deal  of  research  in  computer  science  on  segmenting  videos  based  on 
scene  changes,  we  felt  that  truly  meaningful  units  could  be  obtained  only  by  manual  segmentation. 
Unfortunately,  while  this  kind  of  segmenting  yields  useful  material,  it  is  very  labor  intensive,  requiring 
approximately  seven  minutes  of  processing  for  every  one  minute  of  video.  In  order  to  standardize  the  process, 
we  have  developed  a set  of  guidelines. 

As  we  have  teased  out  the  threads  of  these  educational  videos  (interviews  with  experts  in  the  field, 
historical  footage,  animations,  demonstrations,  re-enactments,  first-person  narratives,  etc.),  we  have  come  to 
realize  many  ways  that  these  segments  might  be  re-constructed  or  regrouped  by  the  teachers.  Perhaps  they  will 
string  together  the  re-enactments  of  Singbe  Pieh  and  his  fellow  Africans  aboard  the  Amistad,  or  pull  out  the 
animated  graphics  that  show  how  black  holes  warp  time  and  space.  Thus,  we  try  to  segment  along  the  different 
interviews  or  footage  or  speakers. 

There  were  times  when  we  chose  not  to  segment  the  most  discrete  pieces  of  information,  as  when  the 
sum  of  the  whole  was  greater  than  the  sum  of  the  parts.  This  was  most  often  when  the  medium  itself,  video, 
was  providing  something  that  could  not  have  been  captured  and  conveyed  by  other  media.  For  example,  near 
the  end  of  the  video  “The  Real  Thomas  Jefferson,”  the  narrator  stands  on  a balcony  in  Jefferson’s  American 
Room  in  Monticello.  The  camera  pans  around  the  room  and  as  it  passes  over  the  artifacts  the  narrator  briefly 
identifies  them.  Important  though  the  images  and  the  identification  of  these  ancient  Indian  artifacts  are,  of 
greater  importance  is  the  panoramic  view  of  the  room  itself  Similarly,  we  tended  to  give  priority  to  the  video’s 
visual  presentation  of  information  over  the  text  or  narration  that  accompanied  whatever  was  happening  on  the 
screen.  Words  can  be  gleaned  from  books  or  audiotapes,  moving  images  come  only  from  video. 

We  have  developed  several  search  tools  that  allow  teachers  to  view  these  relatively  small  and  discrete 
pieces.  The  most  ambitious  tool,  the  Explorer  (Rose  et  al.  1995),  gives  teachers  a visual  representation  of  the 
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entire  collection,  allowing  them  to  select  the  data  elements  that  will  be  displayed  on  the  axes.  With  this  tool, 
they  can  isolate  all  of  the  photographs  from  the  African  Americans  in  World  War  II  photograph  collection 
from  the  National  Archives,  or  the  resources  that  deal  with  aquatic  habitats  and  that  also  satisfy  the  Personal 
and  Social  Perspectives  objective  of  the  National  Science  Education  Standards  (1995). 


Metadata  and  Indexing  by  Content 

Effective  indexing  is  essential  for  any  digital  library  and,  in  general,  metadata  and  indexing  should 
reflect  the  likely  information  needs  of  the  library’s  users.  Because  this  project  was  started  several  years  ago 
and  was  focused  upon  the  needs  of  a single  school  system,  we  had  to  develop  much  of  our  own  metadata.  The 
original  indexing  system  was  created  by  library-science  students  who  tailored  it  to  the  needs  of  a specific  group 
of  middle  school  teachers.  They  reviewed  science  and  social  studies  curriculum  objects  and  textbook  headings 
to  determine  what  the  list  of  topics  and  subtopics  would  be  and  compiled  a lengthy  set  of  instructions  for  future 
indexers.  The  result  was  a 60-page  manual  that  outlined  a system  for  assigning  metadata  elements  (see 
http://www.leam.umd.edu/blcmanual.html  ) and  specific  details  for  assigning  the  descriptive  elements: 
Description,  Topic,  Subtopic,  and  Standard. 

Although  a 60-minute  video  in  its  entirety  may  clearly  belong  in  the  social  studies  category,  many  of 
the  segments  from  it  could  be  cross-indexed  to  other  topics.  Unfortunately,  many  times  these  topics  are  far 
beyond  the  scope  of  our  index.  While  it  is  probably  a hallmark  of  high-quality  educational  resources  that  they 
use  the  familiar  to  introduce  students  to  the  unfamiliar,  for  the  MELC  community  this  is  at  once  a blessing 
and  a curse.  As  indexers,  it  leaves  us  with  lots  of  segments  for  which  we  have  no  appropriate  Topic  or 
Subtopic,  or  which  we  can  only  index  partially.  The  Discoveiy  video  entitled  “Unearthing  South  America”  is 
full  of  curriculum-related  segments  about  early  Indian  civilizations,  but  many  of  these  segments  also  deal  with 
the  culture’s  traditions  of  mummification  and  the  archeological  digs  that  are  revealing  much  about  these 
ancient  people.  Archeology  is  not  a unit  of  study  in  the  middle  school  curriculum,  and  so  we  have  no  way  of 
capturing  this  essential  data.  It  most  cases  we  have  adopted  a “best  fit”  mentality  and  tried  to  shoehorn  these 
segments  into  related  categories. 

A second  stress  upon  the  indexing  system  has  been  our  growth  as  a community.  As  our  ranks  have 
swelled  in  recent  years,  we  have  embraced  teachers  from  other  areas,  most  notably  mathematics  and  language 
arts.  We  have  a handful  of  alternatives  to  accommodate  this  expansion.  One  would  be  to  slow  down  the 
processing  of  videos  and  take  the  time  to  construct  an  index  that  is  similar  in  scope  and  detail  to  the  indexes 
for  science  and  social  studies.  However,  not  only  is  this  a very  labor-intensive  task,  but  it  would  suggest  that 
we  have  learned  none  of  the  lessons  of  our  own  recent  history  (what  if  the  foreign  language  teachers  join  us 
next  month?).  Our  discussions  centered  on  two  other  viable  options:  (1)  distill  the  descriptive  indexing  down 
to  the  most  basic  and  community-specific  of  categories  and  simply  assign  segments  to  the  curriculum  unit(s) 
where  they  are  most  likely  to  be  used;  or  (2)  broaden  the  descriptive  indexing  structure  to  encompass  all 
possible  subject  areas  by  using  a scheme  such  as  Dewey,  which  in  theory  covers  the  entire  body  of  knowledge. 

In  retrospect,  we  might  have  limited  our  index  to  the  broadest  of  categories  for  the  subject  areas, 
including  many  beyond  the  curriculum,  and  emphasized  techniques  for  keyword  searching  in  the  Description 
field.  Concurrently,  we  would  have  enriched  the  descriptions.  Rather  than  simply  giving  a brief  one-  or  two- 
sentence  summary,  we  would  have  made  a greater  effort  to  get  the  transcripts  for  the  videos  and  simply  copied 
the  relevant  passages  directly  into  the  Description  fields,  thus  providing  the  teachers  with  richer  text  to  mine. 
Regardless,  we  have  come  to  discover  that  having  the  transcripts  for  any  sort  of  video  indexing  is  highly 
desirable.  Short  of  taking  the  time  to  implement  a controlled  vocabulary  or  develop  an  authority  file,  it  is 
important  to  refer  to  the  same  entity  in  a consistent  fashion.  For  example,  a recent  addition  to  the  library  has 
been  a video  on  HIV  and  AIDS.  In  a system  based  on  simple  character  string  recognition,  it  is  imperative  that 
the  teachers  know  if  the  film  presents  information  on  “CD4  T cells,”  “cd4  t cells,”  “C  D 4 T-Cells.”  or  “see,  de 
four  tea  sells”? 

Our  custom-built  indexing  effort  may  not  be  sustainable.  We  feel  our  efforts  would  be  better  spent 
indexing  to  some  national  metadata  standard  such  as  the  Dublin  Core  or  GEM  (Sutton  1999).  While  we  do 
index  to  the  National  Science  Education  Standards  and  the  Curriculum  Standards  for  Social  Studies,  we  do  not 
have  sufficient  data  to  suggest  that  this  field  is  being  used  by  the  community.  Similarly,  we  have  considered 
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searching  for  an  indexing  system  that  would  apply  beyond  our  middle  school  teachers  to  a high  school  or 
elementary  school  community. 


Collaborative  Indexing 

Our  goal  has  been  for  MELC  to  become  a self-sustaining  community  when  the  project  funding  ends. 
At  the  heart  of  this  community  of  educators  would  be  the  digital  library  that  supports  their  work  in  the 
classroom.  In  order  for  this  project  to  be  self-sustaining  within  the  current  framework  of  the  Baltimore  City 
Public  School  System,  we  believe  that  the  indexing  of  new  resources  would  have  to  be  done  collaboratively  by 
members  of  the  community. 

While  our  effort  is  on  a much  smaller  scale  than  the  CORC  project  (OCLC  1999),  we  certainly  share 
their  vision.  Perhaps  there  are  even  some  parallels  between  the  indexing  of  the  multimedia  content  and  the 
manner  in  which  Varian  (1999)  has  suggested  that  quality  control  in  scholarly  publishing  can  become  self- 
regulating. 

Quick  Indexing  Tool  (QIT)  and  Collaboration 

Our  first  effort  at  fostering  a system  of  collaborative  indexing,  the  Quick  Indexing  Tool  (QIT) 
(http://209.48.188.166/blclib/quickindex/),  is  designed  to  let  the  teachers  capture  descriptive  and  bibliographic 
information  (metadata)  about  resources  as  they  add  them  to  the  digital  library.  Having  come  to  know  both  the 
capabilities  and  needs  of  our  users,  we  have  designed  a simple  interface  that  lets  them  capture  the  greatest 
amount  of  information  with  the  least  amount  of  effort. 

Along  with  several  simple  fields  for  each  QIT  submission,  we  have  included  a description  field  that 
encourages  teachers  to  jot  down  a brief  statement  or  vocabulary  from  the  page  itself  (keywords)  as  a way  of 
representing  the  content.  In  the  second  half  of  the  form,  we  ask  the  grade,  subject  area  (social  studies,  math), 
and  unit  of  study  (Baltimore  uses  the  Berri  Curriculum)  for  which  they  envision  this  resource  will  be  useful. 
We  anticipate  that  these  particular  required  fields  will  not  dampen  enthusiasm  for  the  tool  by  implying  that  all 
teachers  should  be  expected  to  know  about  the  specifics  of  all  units  of  study  across  all  grade  levels  and  all 
subjects.  We  anticipate  that  sixth-grade  math  teachers  will  be  entering  Web  sites  applicable  to  the  sixth-grade 
math  curriculum  and  seventh-grade  science  teachers  will  be  entering  Web  sites  applicable  to  the  seventh-grade 
science  curriculum  units,  and  so  forth.  Having  the  resources  selected  by  these  subject  experts  should  greatly 
increase  the  usefulness  and  quality  of  the  library. 

The  QIT  was  introduced  to  the  MELC  teachers  in  August  1999.  As  of  March  2000,  94  resources  have 
been  added  using  this  tool  including  25  of  the  27  math  resources.  It  has  helped  us  expand  the  library  to  include 
the  new  content  areas  of  math  and  language  arts  by  having  the  new  teachers  themselves  make  contributions. 
Each  of  the  27  math  resources  in  the  library  have  been  added  by  math  teachers  who  used  the  QIT. 

Pros  and  Cons  of  Collaborative  Indexing 

In  a system  of  collaborative  indexing,  all  community  members  would  be  responsible  for  resource 
acquisition  and  cataloguing.  Resources  can  be  found  items,  such  as  Web  sites;  proprietary  items,  such  as  the 
Discovery  video  segments;  or  self-generated  items,  such  as  the  photographs  of  Baltimore  and  Europe  taken  by 
various  teachers  who  scanned  them  and  then  entered  them  as  digital  images.  The  benefits  of  such  a system  are 
obvious  and  important: 

1)  In  theory,  it  guarantees  that  each  item  will  be  have  high  relevance  to  some  subset  of  users. 
Only  resources  that  other  members  of  the  group  have  identified  as  being  specifically 
related  to  the  curriculum  will  be  part  of  the  collection. 

2)  Not  only  will  the  items  all  have  high  levels  of  relevance,  but  they  will  also  be  of  high 
quality.  Each  item  will  have  been  judged  useful  by  a subject  expert. 

3)  Moreover,  the  activity  of  creating  a QIT  entry  may  lead  to  reflection  by  the  teacher  about 
the  value  of  the  item.  The  time  involved  in  entering  the  bibliographic  data,  which  includes 
their  name  and  e-mail,  will  serve  as  further  impetus  for  maintaining  quality  control. 

However,  there  are  a few  difficulties  in  expecting  teachers  to  use  the  QIT  on  a regular  basis: 
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1)  Becoming  a contributing  member  of  a digital  library  community  means  more  work  for  a 
group  that  is  habitually  overworked.  Society,  school  administrations,  and  parents  already 
expect  them  to  perform  a myriad  of  functions  within  their  classrooms.  We  are  asking  them 
to  perform  yet  one  more. 

2)  The  community  of  users  probably  will  not  find  as  many  resources  as  a library  media 
specialist  would.  Although  the  community  users  will  be  able  to  index  the  resources  they  do 
find,  they  may  be  overlooking  many  others.  For  example,  the  media  specialist  knows  that 
many  valuable  sites  are  only  available  to  users  who  know  to  execute  a search  on  the  main 
page  or  by  searching  the  site  index,  and  that  these  pages  cannot  be  found  through  the 
major  search  engines.  Unlike  today’s  librarians,  teachers  have  not  necessarily  been  trained 
to  manipulate  the  search  engine  algorithms  as  a way  to  refine  the  search  process. 

3)  The  community  users  probably  will  not  find  resources  of  the  same  quality  as  a media 
specialist.  The  reason  for  this  is  similar  to  the  argument  made  in  the  previous  point. 

4)  The  community  of  users  and  the  QIT  will  not  exist  in  an  equilibrium.  As  we  have 
discovered,  changes  in  the  environment  can  have  messy  implications  for  the  system  as  it 
was  originally  conceived.  If  they  are  not  provided  with  the  technical  skills  or  training  to 
respond  to  these  changes,  then  the  slightest  shift  in  the  equilibrium  may  well  overwhelm 
them. 


Conclusion 

Based  on  our  experience,  we  envision  an  extended  collaborative  indexing  system  that  is  overseen  by  a 
professional  librarian  with  whom  they  have  ready  access  and  easy  communication.  This  professional  librarian 
would  serve  as  a resource  to  be  used  by  the  community  members  on  an  “as  needed”  basis,  although  this 
professional  will  also  be  an  equal  contributor  to  the  endeavor  and  a person  who  will  no  doubt  have  many 
useful  things  to  share  when  it  comes  to  the  selection,  acquisition,  organization,  and  use  of  resources.  We 
envision  an  iterative  process: 

• an  individual  or  subset  of  members  approaches  the  media  specialist  with  a request  for 
information, 

• the  media  specialist  will  interview  them  to  further  understand  the  nature  of  their  information 
need, 

• the  media  specialist  will  provide  the  user  with  a list  of  resources, 

• the  teacher  will  examine  the  resources,  * 

• the  teacher  will  select  and  index  (with  QIT)  the  ones  that  meet  his/her  need,  including  those  that 
he/she  may  have  found  on  their  own. 

This  scenario  strikes  a balance  between  two  extremes.  It  leaves  ultimate  responsibility  for  resource 
selection  to  the  community  members,  thus  ensuring  high  relevance  and  importance  to  the  curriculum,  but  it 
lets  them  have  the  option  of  having  input  from  a professional  whose  intervention  represents  a great  deal  of 
“value  added”  to  the  final  product. 

Finally,  it  also  suggests  a way  that  we  can  provide  a useful  framework  for  the  local  learning 
community,  yet  provide  an  infrastructure  for  a broader  community.  Once  a media  specialist  has  compiled  a 
listing  of  resources  that  would  be  useful  for  an  eighth-grade  lesson  plan  on  meteorology,  it  is  a relatively 
simple  task  to  share  that  list  with  other  teachers  in  other  local  learning  communities. 

Perhaps  the  solution  lies  in  a federated  system  of  collaborative  indexing  communities.  In  this 
arrangement,  many  small  groups  of  collaborative  indexing  communities  join  forces  to  benefit  from  the  sharing 
of  resources,  expertise,  and  support  that  are  inherent  in  a larger  federation.  Suppose  an  individual  member  of 
Community  A has  annotated  and  cataloged  a resource  in  a way  that  has  made  it  useful  to  the  specific  needs, 
parochial  though  they  be,  of  the  members  of  her  immediate  community.  An  individual  in  Community  B,  upon 
perusing  the  digital  library  holdings  of  Community  A,  accesses  this  same  resource  but  represents  the 
descriptive  data  differently  in  his  catalog.  For  his  users,  he  includes  information  about  how  the  resource 
correlates  to  statewide  standards  outlined  by  the  governor’s  task  force.  On  a macro  level,  the  members  of 
Community  B benefit  from  having  access  to  the  resources  in  the  digital  library  of  Community  A;  on  a micro 
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level,  they  benefit  from  the  refining  process  that  defines  collaborative  indexing  systems. 

We  have  plans  for  adding  tools  similar  to  QIT  that  will  allow  teachers  to  index  video  segments, 
materials  from  online  archives,  and  other  multimedia  items.  This  will  be  accomplished  by  redefining  the 
metadata  fields  in  the  top  half  of  the  form  so  as  to  prompt  users  to  give  more  resource-specific  information 
(e.g.,  audio  tapes  often  have  narrators,  computer  images  come  in  varying  file  formats). 

We  continue  to  believe  that  educational  video  can  be  highly  absorbing  and  can  be  an  important 
resource  for  middle  school  teachers.  However,  many  of  the  costs  for  the  digital  library  (indexing,  segmenting) 
would  be  better  borne  by  a very  large  constituency.  The  issue  becomes  maintaining  a sense  of  involvement 
from  the  local  learning  communities.  The  QIT  is  one  example  of  a community-oriented  tool.  Other  such  tools 
might  include  social  collaboration  tools  (see  Allen  1997). 
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Computers  in  Education  should  go  beyond  task  simplification  in  teaching-learning  processes  in 
order  to  become  a strong  and  effective  resource  for  the  construction  of  knowledge.  For  computers 
to  fit  into  a constructivist  approach,  it  is  necessary  to  find  a way  to  develop  computer  tools  in  which 
dialog  and  comparison  with  the  student's  mental  and  formal  models  is  possible,  focusing  on  the 
achievement  of  meaningful  learning. 

Hypermedia  Concept  Maps  (HCMs),  as  a way  of  representing  ideas,  have  a great  deal  to  offer  to 
the  development  of  models  for  diverse  study  phenomena  and  to  the  adoption  of  the  appropriate 
pedagogical  approach  to  include  computer  technology  and  adapt  it  to  such  approach.  Here,  we 
propose  the  use  of  HCMs  to  create,  together  with  a specific  platform  for  working  with  them,  an 
open  computing  learning  environment.  New  learning  development  implies,  in  some  cases,  a review 
of  the  previously  acquired  knowledge  and,  in  all  cases,  an  integration  of  newly  and  previously 
acquired  knowledge.  In  order  to  work  in  this  direction,  we  suggest  an  environment  that  supports 
HCM  interconnection  and  we  define  the  options  required  to  enhance  the  existing  platform. 


Introduction 

Few  activities  of  man  and  society  have  escaped  from  the  impact  of  computer  technology.  In  education,  the 
attraction  of  technical  features  presented  by  new  tools  has  diminished  the  attention  on  how  they  should  be 
incorporated  to  learning  processes.  Technology  in  itself  does  not  give  satisfactory  results  in  education  if  it  is  not 
integrated  to  an  approach  in  which  we  wonder  about  its  real  power  in  this  field.  Computers  in  Education  should  go 
beyond  task  simplification  in  teaching-learning  processes  in  order  to  become  a strong  and  effective  tool  for  the 
construction  of  knowledge. 

Computer  technology  should  be  understood  as  a means  and  not  as  the  exclusive  purpose  of  education.  For  teaching 
and  learning  this  technology  constitutes  an  additional  tool,  often  appearing  to  reinforce  behavioral  and  verbalist 
learning  frameworks.  To  be  in  agreement  with  a constructivist  approach,  it  is  necessary  to  find  a way  to  develop 
computer  tools  in  which  dialog  and  comparison  with  the  student's  mental  and  formal  models  is  possible,  not  forcing 
the  student  to  obtain  a result  but,  instead,  stimulating  his  or  her  knowledge  building  process  and  reflection  on  it.  In 
order  to  achieve  this,  it  is  necessary  to  focus  attention  on  how  these  tools  are  going  to  be  added  to  the  educational 
process  and  not  only  to  the  contents  they  are  going  to  introduce. 

It  is  worth  remembering  what  Fenstermacher  said  about  the  relationship  of  ontological  dependence  existing  between 
the  concepts  of  teaching  and  learning,  and  how  the  central  activity  of  teaching  is  to  facilitate  the  student  to  develop 
learning  tasks.  That  is  to  say,  "to  teach  is  to  make  learning  possible,  to  provoke  dynamics  and  situations  in  which 
the  learning  process  can  take  place  in  the  students".  In  this  sense,  the  skill  of  the  teacher  in  selecting  the  best  tools 
for  the  achievement  of  meaningful  learning  is  very  important,  [Fen86], 

Hypermedia  Concept  Maps  (HCMs),  as  a way  of  representing  ideas,  have  a great  deal  to  offer  to  the  development  of 
models  for  diverse  study  phenomena  and  to  the  adoption  of  the  appropriate  pedagogical  approach  to  include 
computer  technology  and  adapt  it  to  such  approach. 

Neither  computer  technology  in  general  nor  hypermedia  in  particular  can  solve  completely  nor  directly  the  problems 
posed  by  knowledge  acquisition  within  the  teaching-learning  process;  success  depends  exclusively  on  the  way  in 
which  it  is  applied. 
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Learning  Environments 

The  purpose  of  incorporating  the  computer  to  learning  processes  is  identified  with  the  development  of  thinking 
activities,  and  with  the  idea  that  a computer  is  a support  for  the  student  to  build  his  or  her  knowledge  in  his  or  her 
mental  model  structuring  process. 

A learning  environment  can  be  defined  as  an  environment  formed  by  a non-homogenous  set  of  elements  that  are 
able  to  create  o recreate  situations  from  which  students  can  build  knowledge.  Although  it  is  an  expression  that 
appears  linked  to  Computers  in  Education,  it  is  used  in  a wider  sense  and  it  can  be  used  even  without  the  use  of 
computer  technology.  These  environments  are  presented  as  sites  with  a great  pedagogical  value,  extremely  favorable 
for  learning  and  metaleaming  experiences.  Regarding  computer  learning  environments,  we  can  distinguish  two 
large  groups; 

Closed  Environments:  those  that  consist  of  an  integrated  software  and  hardware  system  that  does  not  require  any 
other  system  outside  it  except  for  the  students  that  interact  with  it. 

Open  Environments:  the  computer  system  forms  part  of  the  environment  together  with  other  elements  and  persons, 
and  the  rol  of  the  teacher  clearly  stands  out. 

Within  the  second  group  we  can  include  HCM  work.  In  this  case,  the  environment  will  consist  of  a computer  with 
the  specific  platform  to  develop  and  read  these  maps,  different  kinds  of  bibliographical  material,  map  authors  or 
readers,  and  the  proper  participation  of  the  teacher  to  assess  the  map  semantically. 

HCMs,  as  a way  of  representing  knowledge,  can  cover  the  expectations  of  the  meaningful  learning  model,  and  they 
constitute  an  important  basis  for  computer  technology  introduction  in  new  educational  processes. 

It  has  been  proved  that  work  with  HCMs  is  very  valuable  for  meaningful  learning  building.  Concept  selection  and  its 
correct  hierarchical  organization  are  directly  related  to  the  previous  knowledge  of  the  map’s  author  and  to  the 
structural  organization  of  this  with  the  newly  acquired  knowledge. 

On  Hypermedia  Concept  Maps 

HCMs  preserve  all  the  educational  richness  of  Novak's  CMs  enhanced  with  the  benefits  of  hypermedia  technology: 
easier  operational  handling,  greater  richness  and  versatility  for  information  patterns,  and  motivationally  more 
attractive,  especially  for  young  learners,  [Sen96a]. 

Definition 

An  HCM  is  a hypermedia  document  in  which  each  of  its  nodes  has  a collection  of  no  more  than  seven  concepts 
interrelated  by  linking  words.  Each  of  these  nodes  is  called  a view. 

There  are  two  types  of  concepts:  those  owned  by  the  view  and  those  imported  to  the  view.  The  former  constitute  the 
initial  concepts  of  the  view  and  the  latter  are  the  image  of  concepts  that  belong  to  another  view  that  are  created  in 
order  to  establish  relationships  among  concepts  of  different  views. 

Relationships  among  concepts  of  the  same  view  are  called  internal  and  relationships  among  concepts  of  different 
views  are  called  external. 

In  each  view,  the  owned  concepts  are  represented  by  means  of  labeled  ellipses,  imported  concepts  by  labeled 
rectangles,  both  with  the  name  of  the  concept  that  they  represent,  and  relationships  (internal  or  external)  by  arcs 
labeled  with  linking  words. 

In  order  to  represent  external  relationships,  a labeled  arc  is  established  between  the  owned  and  the  imported  concept. 
Such  relationship  appears  in  both  views. 

A name  and  a color  identify  each  of  the  views.  The  name  corresponds  to  the  most  comprehensive  owned  concept 
and  the  color  is  used  in  all  the  ellipses  representing  owned  concepts.  Imported  concepts  keep  the  color  of  the  view 
where  they  were  originally  defined. 
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Given  a view  VI  and  a concept  C belonging  to  VI  we  can  say  that  C explodes  in  another  view  V2  when  such 
concept  C develops  in  V2,  i.e.  C constitutes  the  root  node  of  the  map  developed  in  V2.  In  this  case  we  can  say  that 
the  map  represented  in  view  V2  is  a submap  of  the  one  corresponding  to  view  VI . 

The  ellipse  representing  an  owned  concept  exploding  in  another  view  is  an  elliptical  button  that  allows  direct  access 
to  that  view.  The  rectangle  representing  an  imported  concept  C is  a rectangular  button  that  allows  direct  access  to 
the  view  where  C is  defined  as  owned. 

We  can  associate  to  a terminal  concept  (that  is  not  a button)  different  features  such  as  graphics,  sound,  animation, 
etc.  Any  concept  can  be  associated  to  a feature  since  there  will  always  be  a CM  view  in  which  said  concept  is 
defined  as  terminal. 

Each  HCM  presents  two  strata:  the  strictly  speaking  HCM  and  the  underlying  hypermedia  database  created  from  the 
resources  that  originated  the  HCM.  Access  to  it  can  be  achieved  during  map  navigation.  Thus,  those  concepts  in  an 
HCM  that,  due  to  their  richness,  can  provide  new  information,  can  be  more  deeply  explored  by  means  of 
hypermedia.  Both  strata  can  be  queried  alternatively.  HCMs  go  beyond  the  educational  possibilities  of  Novak’s 
traditional  CMs.  This  hypermedia  bibliography  is  extremely  useful  for  those  interested  in  reading  further  about  the 
topic,  as  well  as  those  assessing  it,  [Sen96b]. 

Elements  in  an  HCM  Learning  Environment 

In  order  to  focus  the  HCM  author's  attention  in  the  most  important  and  richest  aspects  from  the  educational  point  of 
view  -such  as  concept  selection,  hierarchical  classification  and  relationship  establishment-  and,  in  order  to  avoid 
distractions  in  building  aspects  and  implementation  options  that  lack  any  pedagogical  value,  the  platform  for  HCM 
work  and  a methodology  for  HCM  development  is  introduced  into  the  environment,  in  addition  to  bibliographical 
material  on  the  subject  and  the  corresponding  simulation  systems  for  HCM  study. 

Platform  for  HCM  Development 

The  design  of  a specific  platform  for  dealing  with  HCMs  aims  at  having  exclusively  those  resources  necessary  for 
HCM  development.  Those  characteristics  can  be  found  in  the  specific  platform  for  HCM  development  and  reading 
presented  in  [Mor96a].  This  platform  has  two  work  modes,  one  of  them  corresponds  to  the  author  -map  development 
and  modification-  and  the  other  corresponds  to  the  reader  -map  perusal  and  inspection.  Immediate  interaction 
between  the  two  modes  is  allowed.  The  author  mode  presents  specific  characteristics  for  HCM  development  that 
makes  the  user  completely  free  from  dealing  with  any  aspect  that  goes  beyond  the  understanding  of  the  topic.  It  has 
a graphic  interface  that  offers  a tools  bar  with  the  different  options  available  for  map  development  and  modification 
and  pull-down  menus  for  editing. 

HCM  Learning  Environment  Enhancement 

New  learning  development  implies,  in  some  cases,  a review  of  previously  acquired  knowledge,  and,  in  all  cases,  an 
integration  of  newly  and  previously  acquired  knowledge.  Thus,  we  propose  an  environment  to  develop  HCM 
interconnections.  TTie  main  reason  for  this  task  is  given  by  the  need  to  integrate  new  knowledge  with  previously 
acquired  knowledge  and  the  wish  to  relate  different  points  of  view  of  the  same  topic. 

HCM  Interconnection 

The  interconnection  of  two  HCMs  can  be  carried  out  from  different  situations.  Let's  consider  A and  B the  topics  - 
between  these  there  is  some  semantic  relationship-  and  HCM  A and  HCMB  the  corresponding  maps  already 
developed.  The  idea  is  to  obtain  only  one  HCM  from  the  two  existing  ones,  without  having  to  do  all  the  map  again 
and  leaving  the  corresponding  relationships  established.  This  can  be  done  as  follows: 

i)  Establishing  a relationship  R between  a concept  Cl  of  HCMA  and  a concept  C2  of  HCMB.  From  the  point  of 
view  of  learning  psychology  it  is  an  integrating  action  that  implies  a semantic  review  of  the  newly  learned 
knowledge  in  relation  to  the  including  knowledge.  From  the  point  of  view  of  operation,  it  is  equivalent  to  the 
establishment  of  an  external  relationship  between  two  concepts  of  the  same  HCM,  since  Cl  and  C2  are  in  different 
views. 

ii)  Concept  C is  detected  in  HCMA  and  in  HCMB.  At  first,  we  suppose  it  is  the  same  concept  and  that  it  is 
represented  by  the  same  name;  later  we  shall  deal  with  concepts  with  synonym  and  homonym  names.  Then,  there  is 
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a view  VI  in  HCMA  and  a view  V2  in  HCMB  where  C is  represented  by  a labeled  ellipse.  We  can  also  see  that  it  is 
possible  for  another  view  with  an  elliptical  button  labeled  C to  exist;  but  in  such  case,  C explodes  in  a view  that 
contains  it  as  an  ellipse.  That  is  to  say,  in  all  cases  it  is  possible  to  find  a view  in  the  map  where  concept  C is 
represented  by  an  ellipse. 

What  are  the  possible  solutions  to  keep  the  following  properties  PI  and  P2  of  the  HCM  in  the  map  resulting  from 
the  interconnection? 

PI:  Each  concept  appears  represented  as  an  ellipse  only  once  in  the  map. 

P2:  The  maximum  number  of  concepts  of  a view  does  not  exceed  from  seven. 

A first  approach  could  be  the  following:  Both  views  VI  and  V2  are  reorganized  in  one  or  two  views  according  to  the 
number  of  concepts  that  both  views  have. 

For  example,  if  VI  and  V2  are  the  views  in  Figure  1,  both  could  be  replaced  by  the  view  in  Figure  2. 

If,  instead,  VI  and  V2  were  like  in  Figure  3,  where  the  number  of  concepts  gathered  between  both  views  largely 
exceeds  the  upper  limit  proposed,  they  could  be  replaced  by  the  views  in  Figure  4,  where  the  original  hierarchy 
among  concepts  has  been  taken  into  account  for  its  organization.  We  can  see  that  in  the  first  view  C is  represented 
by  a buttom. 


Figure  1 : First  approach  - first  case 


Figure  2:  Result  - first  case 


Figure  3:  First  approach  - second  case 


Figure  4:  Result  - second  case 
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Problems  arise  with  this  approach  when  in  HCMA  or  in  HCMB  there  are  views  different  from  VI  and  V2  that  have 
an  elliptical  or  rectangular  button  labeled  C.  In  that  case,  two  views  with  the  same  elliptical  button  exploding  in 
another  view  will  remain  in  the  new  map.  Interconnection  would  imply  redoing  an  important  part  of  the  map, 
loosing  the  pedagogical  objective  that  we  have  in  mind.  This  situation  gets  worse  when  the  interconnected  maps  are 
in  turn  the  result  of  previous  interconnections,  since  a cascade  effect  takes  place. 

The  following  is  a second  approach  in  which  the  problem  is  solved:  C is  identified  in  both  views  by  means  of  an 
external  relation  called  ’’equal".  In  this  way,  in  both  views  the  rectangular  button  that  will  allow  scrolling  from  one 
view  to  the  other  will  appear,  being  possible  to  see  C in  both  contexts,  as  we  can  see  in  Figure  5.  The  overlapping  of 
the  rectangle  and  the  ellipse  graphically  reinforces  the  idea  of  referring  to  the  same  concept. 

The  establishment  of  an  "equal"  relationship  between  concept  C of  both  maps  determines  a circular  behavior  in  the 
access  to  the  views  from  the  use  of  rectangular  buttons  labeled  C. 

If  HCMA  and  HCMB  are  connected  and  one  of  them  is  already  the  result  of  an  interconnection  (for  example, 
HCMB)  the  following  situation  may  arise:  concept  C appears  in  one  HCMA  view  and  in  two  HCMB  Views. 

We  can  see  that  access  circularity  through  rectangular  buttons  involves  the  three  views  after  the  interconnection,  as 
shown  in  Figure  6. 


Figure  6:  particular  case 

Now  that  we  have  discussed  the  advantages  of  this  last  approach  over  the  previous  one,  we  consider  this  is  the 
solution  for  item  ii). 

The  following  special  case  may  come  up.  In  HCMA  view  VI  has  a concept  C represented  by  a labeled  ellipse  that  is 
a leaf.  The  same  concept  C appears  as  root  of  view  V2  in  the  HCMB  and  there  it  is  represented  by  a labeled  ellipse. 
In  this  case,  the  interconnection  transforms  the  C labeled  leaf  of  view  VI  in  an  elliptical  button  exploding  in  view 
V2. 

Synonyms:  when  two  maps  are  interconnected,  the  same  concept  may  appear  represented  by  different  names  that 
are  synonyms.  In  this  case,  the  author  may  choose  between: 

- unifying  the  names  and  proceed  according  to  ii). 

- establishing  an  external  relationship  between  both,  such  as  "it  is  a synonym  of 

Homonyms:  if  two  different  concepts  appear  represented  by  the  same  name,  this  does  not  require  any  special  deal 
for  the  interconnection,  the  meaning  of  each  concept  will  be  shown  through  the  concepts  and  relationships  that  form 
part  of  their  respective  contexts. 
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Adding  the  interconnection  option  to  the  existing  platform.  The  current  platform  allows  for  HCM  construction  and 
reading.  It  is  possible  to  open  two  already  created  maps,  for  example  HCMA  and  HCMB,  leaving  all  their  views 
available. 

Platform  Enhancement 

Interconnection:  the  choice  of  this  option  works  on  open  HCMs.  From  there,  the  platform  considers  all  the  views  of 
said  maps  as  views  of  the  recently  created  HCM.  Then,  the  following  interactive  process  begins:  the  platform  asks 
the  author  a name  for  the  new  map.  Once  it  is  entered,  if  there  is  some  pair  of  views  with  the  same  color  associated, 
the  list  with  the  names  of  said  views  and  a dialog  box  to  allow  for  the  color  change  in  one  of  them  automatically 
appear,  so  that  different  views  do  not  have  the  same  color.  Next,  the  list  of  concepts  represented  by  labeled  ellipses 
that  are  present  in  two  or  more  views  of  the  new  map  appears  on  the  screen.  When  selecting  each  of  them,  the 
corresponding  views  appear  and  the  author,  after  analyzing  them  semantically,  will  decide  the  effective  action  to 
take. 

Equal  Relationship:  the  choice  of  this  option  will  allow  for  automatic  creation  of  the  "equal"  external  relationship  in 
each  of  the  views,  pointing  out  the  equal  concepts  of  the  different  views  indicated  and  leaving  the  rectangular 
buttons  with  the  proper  references  for  a circular  visit  of  the  views. 

If,  instead,  VI  and  V2  were  as  in  Figure  3,  where  the  number  of  concepts  gathered  between  both  views  largely 
exceeds  the  upper  limit  proposed,  they  could  be  replaced  by  the  views  in  Figure  4,  where  concept  hierarchy  has  been 
taken  into  account  for  organization. 

Conclusions 

The  learning  environment  proposed  for  HCM  work  that  includes  the  possibility  of  interconnecting  maps  and 
enhancing  the  platform  with  that  objective,  enlarges  the  richness  of  the  work  with  these  maps  as  a valuable  and 
proven  resource  for  the  representation  of  ideas.  It  expands  its  potential  since  it  allows  to  work  expressly  on  the 
correct  relationship  between  recently  learned  and  previously  acquired  knowledge  or  between  different  approaches  of 
the  same  topic.  It  is  a proposal  for  Computers  in  Education  that  goes  beyond  task  simplification  in  teaching-learning 
processes,  and  it  appears  as  an  effective  tool  for  the  construction  of  knowledge  according  to  a constructivist 
approach. 
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EDUCATIONAL  TECHNOLOGY  AND  ITS  TERMINOLOGY: 
NEW  DEVELOPMENTS  IN  THE  END  OF  THE  20™  CENTURY 


Abstract:  This  paper  is  a report  on  the  findings  of  a quantitative  study  of  the 
Educational  Technology  terminology  conducted  on  the  materials  of  several  AACE 
Conferences.  These  findings  allowed  us  to  single  out  different  technologies  used  in 
education,  to  define  the  structure  of  the  ET,  observe  the  changes  that  have  been 
taking  place  in  this  field  over  the  years  and  describe  the  most  important  events.  The 
list  of  the  most  frequent  terms  and  their  analysis  are  also  presented. 


Introduction 

A language  is  like  a living  organism  - incessantly  developing  and  modifying,  losing  some  elements  and 
creating  the  new  ones  while  adapting  to  the  constantly  changing  environment.  Terminology  of  a 
special  subject  field  is  a part  of  this  organism  that  reacts  most  tangibly  to  the  impact  of  the  outside 
changes. 

Educational  Technology  (ET)  is  a field  that  has  been  rapidly  developing  toward  the  end  of  the 
century  due  to  the  outstanding  innovations  in  Information  Technologies  (IT)  and  their  growing 
application  in  general  and  professional  education  and  training.  These  developments  could  not  have 
failed  to  affect  the  ET  terminology  that  reflects  everything  that  is  happening  in  this  field.  Yet,  however 
important  is  the  terminology  itself  for  the  functioning  of  this  field,  by  studying  terminology  we  can 
also  judge  of  the  latter’s  current  state.  Moreover,  observing  the  development  of  terminology  over  the 
years,  we  take  the  prints  of  various  periods  of  time  and  compare  them,  thus  discovering  similarities  and 
picking  out  the  differences,  which  report  of  the  occurring  changes.  This  paper  is  a further  development 
of  our  research  in  ET  terminology  (2,  3). 


Materials  and  Methods  of  Study 

Our  analysis  of  ET  terminology  is  based  on  the  Proceedings  of  the  AACE  EdMedia  World 
Conferences  that  took  place  from  1996  through  1999  (1).  Four  years  is  certainly  too  short  a period  of 
time  to  make  far-reaching  conclusions  - the  last  15  years  would  have  been  much  more  demonstrative, 
still  it  was  sufficient  to  discover  remarkable  changes  in  ET.  The  papers  contained  in  these  Proceedings 
were  processed  by  a concordance  program  to  obtain  alphabetic  and  frequency  lists  of  all  the  words  that 
occur  in  these  texts.  The  lists  were  then  investigated  by  a number  of  parameters  using  expert  and 
statistical  methods.  Some  of  the  findings  are  presented  in  this  paper. 

The  first  observation  was  a sharp  growth  of  the  volume  of  these  Proceedings:  in  1996  the  volume  of 
the  papers  included  in  these  Proceedings  was  579,256  words,  whereas  in  1999  it  reached  887,630 
words  (an  increase  of  53  percent).  This  increase  reflects  a rapidly  growing  activity  in  ET.  The  total 
volume  of  each  of  the  Proceedings  is  statistically  representative  to  make  grounded  conclusions 
regarding  the  issues  of  interest. 


Peter  Serdiukov 
School  of  Education 
University  of  Utah 


U.S.A. 


serdiu_p@gse.utah.edu 


The  Most  Frequent  Terms 

We  focused  our  attention  on  the  terms  specific  for  ET  and  attempted  to  observe  how  this  field  has 
changed  in  this  short  period  of  time.  The  terms  of  this  field  include  both  the  terms  characteristic  of  IT 
only  and  the  terms  pertaining  to  General  Education,  Psychology,  Theory  of  Instruction  and  other 
relevant  areas.  One  of  the  stages  of  the  investigation  was  devoted  to  the  analysis  of  the  most  frequent 
terms  (frequencies  > 1000)  from  the  1999  file  and  comparing  them  to  the  most  frequent  terms  from  the 
1996  file.  This  analysis  yielded  39  words  that  are  arranged  below  in  the  order  of  the  decreasing 
frequencies.  These  words  are  very  important  for  ET  terminology  as  they  reflect  the  most  significant 
objects,  phenomena  and  processes  that  are  developing  in  it,  thus  allowing  us  to  judge  of  the  contents  of 
this  special  subject  field,  its  current  state  and  changes  that  have  taken  place  between  the  years. 


1999 

1996 

Ranks 

1 . Learning 

1 . Student 

1:2 

2.  Student 

2.  Learning 

2:1 

3.  Course 

3.  System 

3:8 

4.  Technology 

4.  Information 

4:9 

5.  System 

5.  Computer 

5:3 

6.  Information 

6.  User 

6:4 

7.  Education 

7.  Multimedia 

7:12 

8.  Web 

8.  Course 

8:36 

9.  Environment 

9.  Technology 

9:11 

10.  University 

10.  Knowledge 

10:14 

11.  User 

1 1 . Environment 

11:6 

12.  Computer 

12.  Education 

12:5 

13.  Educational 

13.  Tool 

13:18 

14.  Teacher 

14.  University 

14:17 

15.  Design 

15.  Design 

15:15 

16.  Project 

16.  Material 

16:26 

17.  Group 

17.  Teacher 

17:27 

18.  Tool 

18.  Educational 

18:13 

19.  Research 

19.  Program 

19:32 

20.  Teaching 

20.  Communication 

20:29 

21.  Multimedia 

2 1 . Learner 

21:7 

22.  Knowledge 

22.  School 

22:10 

23.  Learner 

23.  Software 

23:21 

24.  Support 

24.  Process 

24:30 

25.  Material 

25.  Application 

25:16 

26.  Process 

26.  Project 

26:24 

27.  Activity 

27.  Group 

27:- 

28.  Communication 

28.  Hypermedia 

28:20 

29.  Program 

29.  Teaching 

29:19 

30.  Online 

30.  Support 

30:- 

31.  School 

3 1 . Video 

31:22 

32.  Virtual 

32.  Research 

32:- 

33.  Model 

33.  Model 

33:33 

34.  Study 

34.  Study 

34:34 

35.  Instructional 

35.  Interface 

35:39 

36.  Content 

36.  Web 

36:37 

37.  Internet 

37.  Content 

37:- 

38.  Software 

38.  Interactive 

38:-23 

39.  Distance 

39.  Instructional 

39:- 

Table.  The  most  frequent  words  of  ET  field  and  comparison  of  their  ranks  in  1999  and  1996. 


lOGO 


The  position  of  each  word  in  this  list  is  taken  as  its  rank  that  reflects  the  value  of  the  term  in  a given 
list.  The  ranks  of  the  words  in  the  two  lists  were  then  contrasted  against  each  other  (1999:1996)  to 
demonstrate  the  shift  in  their  relative  values  over  the  three  years. 

Analysis  of  the  most  frequent  terms  and  comparison  of  the  two  groups  of  the  most  frequent  terms 
taken  over  a 3-year  period,  as  we  can  see  from  the  Table,  allowed  us  to  make  the  following 
conclusions: 

1 . The  inventory  of  the  group  changed  over  this  period  by  12.8  percent  (5  words  of  the  1996  list  — 
application,  hypermedia,  video,  interface  and  interactive  were  replaced  with  the  words  activity,  online, 
virtual,  Internet  and  distance  in  1999).  This  change  reflects  the  general  shift  of  the  ET  focus  in  the 
direction  of  the  Telecommunications  Technology  and  Distance  Education.  It  is  interesting  to  note  that 
all  these  changes  concerned  only  the  words  in  the  bottom  part  of  each  list. 

2.  The  most  remarkable  changes  affected  the  value  of  the  terms  Web,  project,  group  and  research 
that  moved  up  in  the  list  during  these  three  years,  whereas  computer,  multimedia,  material, 
communication,  software  and  model  moved  down.  This  observation  supports  the  abovementioned 
conclusion  and  also  shows  an  increased  interest  in  research  and  group  activities  in  the  area  of  ET. 

3.  No  changes  occurred  in  the  value  of  the  terms  design,  mode  and  study,  and  very  small  shift  in 
the  rank  of  the  terms  student  and  learning  that  replaced  each  other  at  the  top  of  each  list,  and  content. 
This  is  the  evidence  of  the  ET  overall  stability  based  on  the  student-oriented  approach  with  attention  to 
research  in  this  field  and  to  the  content  of  education. 

The  terms  from  the  1999  list  were  later  distributed  into  five  topical  subgroups  (some  words  fall  into 
different  subgroups  and  are  counted  in  each  of  them,  e.g.  information,  system,  user,  etc.): 

1.  Education:  learning,  course,  education,  educational,  teaching,  knowledge,  process,  content, 
activity,  material,  information,  system,  design,  environment,  support,  instructional, 
communication,  program,  development,  study,  problem; 

2.  Technology:  Web,  technology,  computer,  multimedia,  virtual,  Internet,  software,  tool, 
distance,  online,  information,  system,  design,  environment,  support,  instructional,  user, 
multimedia,  communication,  program; 

3.  Research:  research,  project,  model,  development,  study,  problem; 

4.  School:  university,  school; 

5.  People:  student,  group,  teacher,  learner,  and  user. 

These  groups  define  the  basic  contents  of  the  ET  field:  there  are  general  education  terms  (21  words), 
IT  terms  (20),  research  (6),  schools  (2)  and  categories  of  people  involved  in  this  field  (5  words).  The 
changes  that  had  taken  place  in  these  subgroups  since  1996  concerned  only  the  first  (one  word),  and 
the  second  (5  words)  subgroups.  The  ratio  in  the  number  of  terms  in  each  list  remained  the  same  thus 
demonstrating  stability  of  the  ET  structure.  The  most  dramatic  changes  occurred  in  the  inventory  of  the 
technological  terms'  subgroup. 


Technologies  in  Education 

This  investigation  also  allowed  us  to  define  different  technologies  used  in  education  (based  on  Ed- 
Media  99  papers):  Academic,  adaptive,  agent,  architecture,  ASP,  automation,  cable  broadband,  CD- 
ROM,  classroom,  CMC,  collaborative,  communication(s),  computer,  computer-based  learning, 
computing,  conferencing,  course,  database,  digital,  domestic,  education,  educational,  educational 
communications,  engineering,  graphical,  high,  hyperleaming,  hypermedia,  imaging,  human  interface, 
information,  instructional,  instructional  performance,  instructional  systems,  integrated,  interactive, 
interactive  digital,  Internet,  Internet  audio,  Internet  voice,  learning,  media,  METNET, 
multidimensional  analysis,  multimedia,  multimedia  communications,  network,  new  media,  NCATE, 
optical  fiber,  participatory,  projection,  push,  satellite,  social,  student-centered,  subjacent,  SUMMIT, 
teacher-centered,  teaching,  telecommunication(s),  training,  UTP,  video,  videoconferencing.  Virtual 
Reality,  voice,  VR,  Web,  Web-based,  Web-related,  WMI,  WWW  - altogether  73. 


We  can  hardly  apply  the  words  "academic”,  ’’classroom”,  "course”,  "social”,  etc.  to  a technology  - 
these  attributes  refer  to  the  area  of  application,  whereas  "multidimensional  analysis",  "collaborative” 
and  "participatory”  technologies  are,  actually,  concrete  uses  of  certain  technologies.  "Learner”  or 
"teacher-centered”  technology  points  out  to  the  approach  used  in  a technologically-based  education. 
The  terms  "adaptive"  and  "subjacent”  reflect  inherent  characteristics  of  the  technologies,  and  "high" 
demonstrates  the  level  of  technology.  But  we  certainly  can  differentiate  between  technologies  on  the 
basis  of  certain  characteristics  that  can  be  utilized  for  their  classification,  namely: 

Field:  communication,  information 
Area:  CMC,  telecommunications 

Application  sphere:  education,  instructional,  learning,  engineering 
Principle  of  action:  digital  (vs.  analogue) 

Major  tool  or  instrument:  computer,  satellite 
System  (organization):  network,  Internet,  Web,  WWW 
Means  of  connection:  cable  broadband,  optical  fiber 
Technique  or  device:  database,  projection,  CD-ROM 
Activity:  computing,  interactive,  videoconferencing 
Media:  media,  multimedia,  video,  voice,  Internet,  audio/voice 
Structuring:  integrated  (vs.  shared) 

Combined:  educational  communications,  instructional  systems. 

There  can  certainly  be  other  approaches  to  classifying  technologies. 

All  ET  terms  can  be  subdivided  into  four  major  groups:  General  Education,  Computer  Technology, 
Telecommunications  Technology  terms  and  terms  belonging  to  other  technologies  (audio,  video  and 
projection).  The  ratio  between  the  number  of  terms  in  each  group  is  40:24:29:7.  This  means  that 
besides  general  terms  related  to  education,  the  largest  group  of  terms  represents  Telecommunications 
Technology. 

It  was  curious  to  find  quite  a few  neologisms  the  majority  of  which  is  based  on  the  most  popular  roots 
net,  tech,  tele  and  web:  Inet,  technofatigue,  teleinstructor,  telepresence,  webagogy,  webber,  webbook, 
etc.  We  noticed  also  a growing  tendency  to  spell  the  composite  terms  without  a dash,  e.g.  teleleaming 
vs.  tele-learning,  website  vs.  web-site,  etc. 


Conclusions 

Technology  of  a specialty  field  can  be  regarded  as  a system  of  units  and  laws  that,  on  the  one  hand, 
describes  the  field  reflecting  its  structure  and  processes  occurring  there  over  time,  and  regulates  the 
correct  use  of  the  terms  belonging  to  this  field  in  the  text,  on  the  other.  Investigation  of  the  ET 
terminology  using  various  methods  gives  us  better  understanding  of  the  field  organization  and  its 
functioning.  The  next  stage  of  this  study  will  be  focused  on  identiiying  the  changes  that  have  taken 
place  in  ET  in  the  last  10  years  of  the  20^^  century  - this  will  help  us  discover  and  describe  the  trends  in 
the  historical  development  of  ET  and  thus  prepare  for  the  future. 
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Abstract:  In  order  to  make  a technology-based  education  cost-effective,  we 
need  to  find  ways  to  intensify  the  use  of  Educational  Technology  tools  in 
various  teaching/leaming  applications.  One  of  the  ways  to  do  so  is  to  utilize  the 
multifunctional  potential  of  the  technological  tools  that  allows  to  use  one  and 
the  same  tool  for  various  educational  goals.  A new  approach  to  classifying 
Educational  Technology  is  also  proposed  that  allows  to  find  different 
applications  for  the  existing  tools. 


Introduction 

A remarkable  growth  of  Educational  Technology’s  (ET)  applications  in  all  forms  of  education  and  training 
is  often  undermined  by  the  limited  utilization  of  its  potential:  few  educators  fully  exploit  versatile 
capabilities  of  ET.  This  leads  to  its  inefficient  use  in  teaching  and  learning,  lesser  effect  on  the  students, 
inadequate  distribution  of  educational  resources  and  extra  expenses.  However,  practically  each  tool  can  be 
used  to  attain  more  than  one  goal  in  teaching  and  learning  that  can  make  ET-based  education  more  cost- 
effective.  Thus,  we  come  to  the  idea  of  multifunctionality  of  ET  according  to  which  one  and  the  same  tool 
can  effectively  serve  a variety  of  purposes.  This  idea  is  the  opposite  of  the  idea  of  integrated  use  of 
technologies  that  advocates  building  up  a mix  of  various  technological  tools  in  one  educational 
environment  - integration  is  certainly  very  important  but  we  suggest  as  well  to  find  ways  of  intensive 
utilization  of  each  ET  tool  for  different  applications  in  teaching  and  learning  to  maximize  its  effectiveness. 


Theoretical  Considerations 

It  is  interesting  to  follow  the  development  of  this  idea  over  the  last  20-25  years.  At  first,  teachers  tried  to 
find  all  possible  uses  for  each  new  tool  that  appeared  in  schools  and  universities.  For  example,  a slide 
projector  was  first  used  in  foreign  language  teaching  for  the  semantization  of  new  words  - by  showing  on 
the  screen  the  objects  these  words  denoted  to  the  students  (in  this  way,  teachers  avoided  the  need  for 
translating  these  words  into  the  students’  native  language).  Then  it  began  to  be  employed  to  develop  verbal 
skills:  the  pictures  served  as  objects  for  oral  or  written  narration.  Later,  the  pictures  on  the  slides  were 
found  to  stimulate  communication  in  the  target  language  by  creating  a communicative  situation.  Similarly, 
a tape  recorder  was  used  firstly  to  train  the  students  to  pronounce  foreign  sounds,  then  - to  understand 
words  and  phrases,  after  - to  develop  listening  skills;  finally  - to  develop  communicative  competence. 

Later,  there  was  an  attempt  to  integrate  several  different  tools  in  one  classroom  and  in  one  lecture  or 
laboratory  work,  when  the  teachers  as  well  as  the  learners  began  to  realize  that  they  needed  a number  of  ET 
capabilities  to  assist  them  in  the  teaching/leaming  process.  Thus,  integral  use  of  ET  began  to  spread  in 
1980-ies  that  invited  sequential  or  simultaneous  use  of  several  ET  tools  in  one  lesson.  The  idea  of  integral 
use  of  ET  was  grounded  both  on  the  necessity  to  use  various  media  in  information  (text,  sound  and  picture) 


presentation  - simultaneous  use  of  a slide  projector  that  produced  static  images  with  or  without  a written 
text  and  a tape  recorder  with  an  audio  text  and/or  music  is  an  example,  and  also  on  the  desire  to  intensify 
the  impact  of  the  teacher's  pedagogical  actions  and  in  this  way  to  increase  students'  perception,  cognition 
and  learning  outcomes  on  the  whole.  This  approach  was  based  on  peculiarities  of  a person's  perception 
supported  by  the  data  given  by  E.  Dale:  we  learn  1 1%  of  information  through  hearing  and  83  through  sight. 
People  remember  10%  of  what  they  read,  20%  of  what  they  hear,  30%  of  what  they  see,  50%  of  what  they 
see  and  hear,  70%  of  what  they  say  during  communication,  and  90%  of  what  they  say  in  the  process  of 
some  activity  (cit.  by  K.  Spencer  1991,  116).  It  has  been  proved  that  education  and  training  based  on  the 
integral  approach  to  information  presentation  and  its  use  in  learning  activities  could  improve  students' 
cognition  and  provide  efficient  knowledge  construction  (Dovgiallo  1990,  Spencer  1991,  et  al.). 

Due  to  the  versatility  of  contemporary  ET  tools,  primarily  a computer,  the  idea  of  multifunctionality 
revived  again  and  replaced  with  the  time  the  idea  of  integral  ET  application,  at  least  partially,  though  later 
it  underwent  modification  and  now  can  be  viewed  from  a different  angle:  PC  capabilities  today  are 
extended  with  a number  of  peripheral  devices  - printer,  scanner,  LCD  plate,  modem,  video  camera, 
loudspeakers  etc.  that  serve  as  its  extenders  and  provide  the  multifunctionality  of  its  application. 

The  use  of  technological  tools  for  different  educational  goals  is  based  on  the  tools’  technical  capabilities, 
on  the  one  hand,  and  on  the  principle  of  multifunctionality,  on  the  other.  This  principle  first  formulated  in 
1984  (Serdiukov  1984)  and  further  developed  in  our  later  works  (1990,  1997)  states  that  one  and  the  same 
tool  can  and  should  be  used  for  various  educational  purposes.  Implementation  of  this  principle  results  in 
greater  effectiveness  of  teaching,  training  and  learning,  and  in  savings  on  technology  expenses. 

Still,  it  is  quite  obvious  that  each  tool  is  intended  for  one  major  purpose  (selective  application),  so  in  a 
systemic  approach  to  the  ET  application  (see  Serdiukov  1997,  410-419)  one  tool  may  not  be  sufficient  to 
solve  a particular  educational  problem.  Then  we  have  to  look  for  other  tools  that  can  supplement  the 
primary  one.  This  approach  is  used  in  integrated  systems.  Supplementing  one  tool  with  other  tools  allows 
designers  of  educational  software  and  practicing  teachers  to  find  the  best  solution  to  instructional  problems. 


Principles  of  ET  Application 

Multifunctionality  of  ET  tools  application  is  a complicated  domain,  and  it  should  be  appropriately 
structured.  We  suggest  that  the  use  of  ET  tools  in  education  and  training  be  based  on  the  following  three 
principles:  selectivity,  multifunctionality  and  suppletivity. 

Selectivity:  Application  of  each  ET  tool  for  specific  educational  goals  is  regulated  by  its  technical  and 
pedagogical  capabilities. 

Multifunctionality:  Practically  any  ET  tool  can  be  used  for  a variety  of  educational  goals;  these  applications 
are  allowed  by  the  tool's  technical  and  pedagogical  capabilities. 

Suppletivity:  If  a given  ET  tool  utilized  for  a particular  application  cannot  offer  the  best  performance, 
another  tool  with  a better  capability  for  this  application  can  be  used  together  with  the  first  one,  supplement 
or  even  replace  it  so  as  to  increase  its  efficiency  and  achieve  the  desired  goals. 


Multifunctional  Use  of  ET  Tools 

There  are  many  instances  of  the  multifunctionality  of  ET  tools.  Let's  consider  a few  examples. 

Word  processor*  s main  use  is,  clearly,  text  generation  and  processing;  however,  it  also  finds  use  as  an 
educational  tool  for  writing  instruction,  composition  skills  formation,  and  project  development  (Scrimshaw 
1993).  Along  with  developing  writing  skills,  it  indirectly  helps  to  develop  reading  skills  as  well  (Serdiukov 
1997).  Work  with  the  text  in  a word  processor  also  stimulates  thinking  (Marcus  1993). 


Textual  Activities 


Text  Generation 


Text  Processing 


Learning  Activities 


Composition  Drill  & Practice  Project  Development 
(Writing  and  reading) 


E-mail  is  used  predominantly  as  a communication  tool,  but,  at  the  same  time,  it  is  a great  instrument  for 
writing  instruction,  individual  and  group  tutoring  and  consultations,  question  and  answer  exercises, 
problem  solving  and  role  play  (Warschauer  1995): 


Communication  Activities 

Dialogue  Group  Discussion 


Writing  Question  & Answer  Problem  Solving  and  Role  Play  Tutoring 
Learning  activities 

Video  can  be  used  not  only  for  the  teaching  material  presentation  (both  visual  and  audio-visual),  but  also 
for  training  (problem  solving,  role-play,  and  simulations),  for  group  discussions,  and  for  information 
database  creation  and  research.  The  Group  Discussion  application  is  particularly  important  for  professional 
training  when  a videotaped  real-life  situation  is  used  for  group  analysis  and  discussion  (Payr  1999).  It  can 
also  be  used  as  an  assessment  instrument  when  the  student’s  real  activity  in  a predesigned  situation  is 
filmed  and  later  used  in  the  class  collaborative  work  and  group  communication  (both  face-to  face  and 
virtual/electronic)  for  analysis,  discussion  and  evaluation  (Serdiukov  et  al.  2000): 

Material  Presentation  Activities 

Visual  Audio-visual 


Learning  Activities 


Training  Group  Discussion  Database  Research 

Creation 


ET  applications  cannot  be  limited  only  to  the  delivery  or  presentation  of  information,  or  to  the  development 
of  certain  skills,  they  also  affect  emotional  state  and  cognitive  processes  of  the  learner.  Thus,  pictures, 
colored  and  moving,  are  able  to  grasp  our  attention  far  better  that  a plain  text  - hence  a dramatic  spread  of 
videos  and  video  games  and  the  ensuing  tragic  decline  in  reading  among  school  age  children  and  many 
adults.  Music  is  not  just  an  acoustic  accompaniment  of  still  images  or  video:  it  can  influence  the  learners’ 
mood  and  stimulate  cognition  - we  found  these  effects  in  our  research  and  practice  of  intensive  foreign 
language  training  (Serdiukov  1 984). 
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Multimedia  with  its  unlimited  capabilities  is  certainly  the  best  example  of  multifunctionality  of  a computer: 
student’s  interaction  with  multimedia  programs  takes  place  at  various  levels  of  perception  and  cognition. 
Hypertext  is  another  example  of  multifunctionality  at  a different  scale  - the  scale  of  information  depth:  it 
offers  a hierarchical  network  of  information  tools  and  databases  that  can  serve  multiple  applications  in 
teaching  and  learning.  Hypermedia  is  a combination  of  both. 

Versatility  of  the  WWW  adds  up  to  the  ET  multifunctionality:  videoconferencing,  streaming  video, 
multimedia  and  various  communication  capabilities  (asynchronous  and  synchronous  communication,  on- 
line and  off-line  dialogues,  and  group  forms  of  interaction  - bulletin/message  board,  chat,  on-line 
conferences,  etc.)  that  support  electronic  communication  between  students  and  their  teachers,  among 
students  and  with  anybody  in  the  world  on  any  topic  are  another  manifestation  of  multifunctionality  of  ET 
tools. 

Suppletivity  can  be  illustrated  by  using  information  activities  as  an  example.  Students  have  traditionally 
obtained  information  for  learning  primarily  from  the  textbooks.  Today  they  also  use  computer-based  and 
web-based  courses  that  provide  all  the  necessary  information  and  also  support  knowledge  construction. 
Besides,  now  they  have  access  to  web-based  databases  and  distributed  Internet  resources,  such  as  electronic 
university  and  web-based  libraries,  information  sites,  databanks,  journals,  etc.  Finally,  they  utilize  e-mail 
for  collecting  information  from  other  people  (fellow  students,  instructors,  experts,  etc.).  All  these  sources 
supplement  each  other. 

Multifunctionality  of  ET  brings  us  before  the  problem  of  choice:  what  tool  and  what  application  is  best  for 
any  particular  teach ing/leaming  case?  The  right  answer  depends  mainly  on  educational  goals,  potential 
characteristics  of  the  available  ET  tools,  ingenuity  of  the  courseware  designer  and  teacher's  pedagogical 
and  technological  expertise  and  skills. 

Table  of  Multifunctional  Applications  of  ET  Tools 

Before  we  deal  with  these  concerns,  we  should  rather  analyze  various  applications  of  ET  tools  in  students’ 
activities  (See  the  Table). 

We  arranged  students’  learning  activities  into  six  groups  and  matched  them  to  the  existing  tools  so  that  we 
could  have  an  opportunity  to  see  which  activity  could  be  performed  with  which  tools.  There  are  the 
following  activities:  information  activities’,  information  search,  processing  and  presentation;  textual 
activities’,  text  processing,  editing  and  generation;  learning  activities  proper:  question-answer,  drill  and 
practice,  problem  solving,  role-play  and  simulation;  research  activities’^  communication  activities:  dialogue, 
group  discussion  and  conference,  and  assessment  activities  (quizzes  and  tests). 

Technologies  are  divided  into  two  major  groups:  computer  technology  that  offers  computer-based  courses, 
automated/computerized  tests,  word  processors,  spreadsheets,  databases,  graphics  and  presentation 
software,  and  telecommunications  technology  that  offers  distance  courses,  distributed  educational 
resources,  email,  videoconferencing,  bulletin  boards,  whiteboards  and  chat.  (We  did  not  include  other 
technologies  into  our  analysis:  audio,  video,  audio-visual,  projection  and  TV,  though  they  play  an 
important  part  in  education.) 

As  we  can  see  from  the  Table,  practically  each  tool  can  be  used  for  multiple  purposes.  The  widest 
multifunctionality  can  be  attributed  to  the  Computer-Based  and  Web-Based  (Distance)  Interactive 
Multimedia  Courses  - they  must  perform  various  functions  in  students’  learning:  from  information  search 
and  processing  to  different  learning  activities  and  assessment.  Telecommunications  tools,  such  as 
Videoconferencing,  Bulletin/Message  Board  and  Whiteboard,  are  also  multifunctional:  they  can  be  used  for 
information  presentation,  several  learning  activities  involving  collaboration  between  students  and 
communication  with  the  instructor  as  well  as  within  the  student  group.  E-mail  can  be  effective  for 
communication  and  in  supporting  group  activities,  such  as  question/answer,  problem  solving,  role-play  and 
simulations,  and  exchange  of  information.  There  are  two  selective,  monofunctional  tools  - presentation 
software  and  automated  tests,  though  we  are  aware  that  the  lesson  presentation  phase  can  be  easily 


converted  into  a learning  activity  phase  by  asking  questions,  posing  problems,  etc;  in  the  same  way  many 
tests  can  be  used  for  learning  purposes  and  not  only  for  the  assessment. 


Applications 
(Students'  activities) 

\ 

Technologies  and  Tools 

1.0.  Information 

1.1.  Search 

1 .2.  Processing 

1.3.  Presentation 

2.0.  Text 

2.1.  Processing 

2.2.  Editing 

2.3.  Generation 

3.0.  Learning 

3.1.  Questions  & Answer 

3.2.  Drill  & Practice 

3.3.  Problem  Solving 

3.4.  Role  Play 

3.5.  Simulation 

4.0.  Research 

e 

.2 

‘c 

a 

£ 

£ 

o 

U 

o 

V) 

5.1.  Dialogue 

5.2.  Group  Discussion 

5.4.  Conference 

6.0.  Assessment 

A.  Computer  Technology 

Al.  Computer-Based  Course 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

A2.  Automated  Test 

+ 

A3.  Word  Processor 

+ 

+ 

+ 

+ 

+ 

A4.  Spreadsheet 

+ 

+ 

+ 

+ 

+ 

A5.  Database 

+ 

+ 

+ 

+ 

+ 

+ 

A6.  Graphics  Software 

+ 

+ 

A7.  Presentation  Software 

+ 

B.  Telecom.  Technology 

B 1 . Distance  Course 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B2.  Distributed  Resources 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B3.  E-mail 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B4.  Videoconferencing 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B5.  Bulletin  Board 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B6.  Whiteboard 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

B7.  Chat 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

+ 

Table.  Multifunctional  Applications  of  ET  Tools  in  Students'  Learning  Activities. 

This  Table  is  actually  an  attempt  at  classifying  contemporary  ET.  It  can  be  used  by  teachers  and  designers 
of  educational  software  as  a guide  in  the  development  and  application  of  teaching/leaming  materials. 

Our  approach  to  classifying  ET  tools  has  semblance  to  the  chemical  elements'  valences  count:  each  tool  has 
certain  technical  and  educational  capabilities  that  allow  to  utilize  this  tool  for  particular  classroom 
applications.  This  approach  is  different  from,  for  instance,  B.  Bruce  and  J.  Levin's  approach  that  suggests  a 
four-part  taxonomy  based  on  J.  Dewey's  division:  inquiry,  communication,  construction  and  expression 
(Bruce  and  Levin  1997,  79).  We  can  find  similarities  between  our  classification  and  this  taxonomy: 
information  and  research  activities  refer  to  the  inquiry,  communication  is,  clearly,  the  same,  learning  and 
textual  activities  relate  to  construction,  and  information  presentation  is  expression.  However,  our 
classification  embraces  all  major  up-to-date  technologies  and  more  of  the  current  applications  of  ET  than 
other  approaches,  including  the  abovementioned  taxonomy. 


Conclusions 

Multifunctionality  is  a property  inherent  in  almost  any  ET  tool.  The  use  of  ET  based  on  the  principle  of 
multifunctionality  allows  to  expand  the  application  sphere  of  ET  tools  and  brings  about  greater 
effectiveness  of  teaching,  training  and  learning,  as  well  as  significant  savings  on  technology  expenses  that 
can  make  technology-based  education  more  practical  and  cost-effective. 
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Abstract:  In  the  fall  of  1999  students  in  the  SUNY  Learning  Network  completed  surveys  regarding 
their  level  of  satisfaction  and  perceived  learning  in  an  entirely  on-line  learning  environment.  The 
goals  of  the  research  were  to  begin  to  build  a profile  of  satisfied,  successful  students  studying  in  this 
environment  and  to  tie  the  research  to  theory.  To  generate  hypotheses  social  learning  theory  was 
used  as  a conceptual  framework.  Data  were  collected  from  1584  students,  representing  the  largest 
study  of  on-line  student  attitudes  to  date.  The  data  was  analyzed  using  univariate  analysis.  Results 
indicated  that  a number  of  variables  were  significantly  correlated  with  high  levels  of  satisfaction  and 
perceived  learning.  These  included  level  of  interaction  with  the  instructor  and  classmates, 
satisfaction  with  the  Help  Desk,  and  satisfaction  with  the  administration  of  the  SUNY  learning 
network.  The  report  includes  an  explanation  of  the  results  relative  to  social  learning  theory. 

Introduction 

The  SUNY  Learning  Network  (SLN)  is  an  Asynchronous  Learning  Network  (ALN)  for  the  64  colleges 
and  universities  and  400,000  students  of  the  State  University  of  New  York.  All  of  the  activities  that  occur 
in  a traditional  college  classroom  occur  on-line.  These  include,  but  are  not  limited  to,  lectures, 
discussions,  written  assignments,  tests,  labs,  simulations,  and  opportunities  to  socialize  with  classmates. 
This  study  presents  some  of  the  results  of  a survey  of  students  including  relationships  between  student 
demographics  and  levels  of  satisfaction  and  perceived  learning  in  the  online  environment. 

Method 

In  the  fall  of  1999  students  enrolled  in  courses  in  the  SUNY  Learning  Network  completed  a survey,  which 
asked  them  to  comment  about  their  satisfaction  with  and  learning  in  this  on-line  learning  environment. 
The  survey  consisted  of  twenty-five  multiple-choice  questions  that  utilized  a Likert-type  scale  to  assess 
degrees  of  satisfaction  and  learning.  Demographic  data  were  collected  on  variables  such  as  student  age, 
gender,  academic  level,  distance  from  campus,  and  previous  computer  skills.  These  demographics  were 
analyzed  against  items  that  assessed  student  attitudes  about  topics  such  as  level  of  interaction  with  the 
instructor  and  classmates,  level  of  learning  compared  to  the  traditional  classroom,  overall  satisfaction  with 
their  specific  course  and  with  on-line  learning  in  general.  The  data  was  analyzed  for  significant 
differences  using  univariate  analysis  of  variance. 

Review  of  Literature 

Frequent  reports  in  the  popular  media  deciy  the  lack  of  hard  data  on  questions  surrounding  student 
attitudes  about  Internet  based  distance  learning.  This  study  was  designed  to  provide  such  data,  to 
determine  how  students  who  studied  online  felt  about  their  courses  and  to  discover  whether  the  most 
satisfied  students  shared  any  characteristics.  These  are  not  new  questions  and  a large  body  of  literature 
points  to  similar  answers.  Most  of  these  studies  conclude  that,  regardless  of  the  technology  used,  distance- 
learning  courses  compare  favorably  with  classroom-based  instruction  and  enjoy  high  student  satisfaction. 
(Phipps,  Merisotis,  and  O’Brien,  1999):  It  has  been  reported  however  that  research  in  the  area  of  distance 
learning  suffers  from  several  weaknesses.  Phipps,  Merisotis,  and  O'Brien  (1999)  summarize  these: 


"...the  methodology  of  many  of  the  research  designs  is  weak,  with  regard  to  such  factors  as  the 
populations  being  compared  or  otherwise  studied;  the  treatments  being  given,  the  statistical 
techniques  being  applied,  and  the  validity,  reliability,  and  generalizability  of  the  data  on  which 
the  conclusions  are  based.”  (Phipps,  Merisotis,  and  O’Brien,  1999;  p.  28) 

Other  areas  of  weakness  identified  by  the  authors  are 

”A.  The  research  has  tended  to  emphasize  student  outcomes  for  individual  courses  rather 
than  for  a total  academic  program.  B.  The  research  does  not  take  into  account  differences 
among  students.  C.  The  research  does  not  adequately  explain  why  the  dropout  rates  of 
distance  learners  are  higher.  D.  The  research  does  not  take  into  consideration  how  the 
different  learning  styles  of  students  relate  to  the  use  of  particular  technologies.  E.  The 
research  focuses  mostly  on  the  impact  of  individual  technologies  rather  than  on  the 
interaction  of  multiple  technologies.  F.  The  research  does  not  include  a theoretical  or 
conceptual  framework."  (Phipps,  Merisotis,  and  O’Brien,  1999;  p.  1 1) 

The  authors  conclude  that  although  a great  deal  of  research  supports  the  notion  that  distance  learning 
compares  favorably  to  classroom  based  learning,  the  research  is  so  weak  that  no  clear  conclusions  can  be 
made.  While  these  criticisms  cannot  all  be  addressed  by  a single  work,  this  study  will  attempt  to  account 
for  as  many  of  these  weaknesses  as  possible.  To  address  Phipps’  et.al.  first  concern  (A)  this  study  will 
emphasize  student  attitudes  not  for  individual  courses  but  for  the  entire  program  represented  by  the  SUNY 
Learning  Network.  In  all  the  data  used  in  this  study  represents  more  than  250  instructors  and  courses  and 
over  1500  students.  As  such  the  data  in  this  study  represents  a very  large  sample  without  the  weaknesses 
identified  by  analyses  of  individual  courses.  With  a sample  of  this  size  it  can  reasonably  be  expected  that 
a wide  variety  of  students  and  learning  styles  will  be  represented  thus  accounting  for  student  differences  in 

(B)  and  (D).  Further,  this  study  will  look  specifically  at  student  differences  such  as  age,  gender,  academic 
level  etc.  in  an  attempt  to  sort  out  how  such  differences  affect  student  satisfaction  and  learning.  Inasmuch 
as  these  courses  utilized  a wide  variety  of  technologies  (computers,  audio,  video,  the  Internet)  it  does  not 
focus  on  a single  technology  as  in  (E).  This  study  will  also  present  a conceptual  framework  for 
understanding  student  satisfaction  with  online  learning,  accounting  for  (F).  Unfortunately  dropout  rates 

(C)  are  not  addressed  by  this  study. 

Conceptual  Framework 

“Learning,  both  outside  and  inside  school,  advances  through  collaborative  social  interaction  and  the  social 
construction  of  knowledge”  (Brown,  Collins  & Duguid,  1989). 

Learning  has  been  defined  very  cogently  as  a social  process  (Vygotsky,  1962,  1978;  Bruner,  1990;  Brown, 
Collins  & Duguid,  1989;  Lave,  1988,  1990)  and  any  learning  environment  that  is  meant  to  foster 
understanding  must  account  for  the  social  nature  of  the  endeavor.  If  an  overarching  framework  for 
understanding  student’s  feelings  about  working  in  the  relative  isolation  of  a virtual  classroom  is  to  make 
sense,  this  framework  must  account  for  mechanisms  that  overcome  such  isolation.  Students  learning  in  a 
virtual  environment  can  be  expected  to  require  several  support  mechanisms  in  order  to  feel  satisfied  and  to 
learn.  These  include  the  support  of  the  faculty,  classmates,  the  administration  of  the  environment,  as  well 
as  the  technology,  and  content.  If  faculty  are  trained  to  recognize  the  affordances  and  constraints  of  virtual 
learning  environments,  if  students  and  instructors  are  offered  a variety  of  support  mechanisms  for 
overcoming  social  isolation,  if  courses  and  course  content  are  designed  with  an  understanding  of  the 
requirements  of  learning  at  a distance,  then  it  can  be  predicted  that  student  satisfaction  and  learning  will 
be  relatively  high.  In  the  absence  of  these  prerequisites  students  can  be  expected  to  experience  confusion, 
frustration  and  isolation.  For  a detailed  description  of  the  planning  and  execution  of  the  overall  program 
of  the  SUNY  Learning  Network,  which  describes  the  strategies  employed  to  ensure  that  these  prerequisites 
were  met,  see  Pickett,  1999. 


When  learning  is  viewed  primarily  as  a social  process  a number  of  predictions  can  be  made  about  student 
affect  in  virtual  learning  domains.  An  obvious  hypothesis  is  that  interaction  between  student  and  teacher 
and  student  and  student  will  be  of  great  importance.  From  this  conceptual  perspective  one  would 
hypothesize  significant  relationships  between  students  satisfaction  with  levels  of  interaction  with  the 
instructor  and  fellow  students  and  their  overall  satisfaction  with  the  learning  experience.  A second 
prediction  - in  as  much  as  resources  such  as  Helpdesk,  technological  tools  and  system  administration 
break  down  barriers  to  collaborative  social  interaction,  student  satisfaction  with  these  supports  will  be 
significantly  related  to  overall  satisfaction  and  learning.  These  predictions,  stemming  from  a conceptual 
framework  in  which  online  learning  is  viewed  primarily  as  a social  process,  are  tested  in  this  study. 

Results 

The  results  of  the  survey  were  quite  positive  overall.  When  asked,  "How  satisfied  are  you  with  your 
course?"  the  most  common  response  was  the  highest  rating  available  (very  satisfied).  Forty  nine  point 
seven  percent  (49.7%)  of  students  gave  this  response.  Another  thirty  eight  point  five  percent  (38.5%) 
gave  the  second  highest  rating  (satisfied).  Thus,  overall,  eighty  eight  point  two  percent  (88.2%)  of 
students  were  at  least  satisfied  with  their  online  course  regardless  of  their  background.  Eight  point  five 
percent  (8.5%)  were  "not  very  satisfied"  and  three  point  three  percent  (3.3%)  were  "not  satisfied  at  all". 
When  asked,  "How  much  did  you  learn  in  your  online  course?"  the  most  common  response,  once  again, 
was  the  highest  rating  (A  great  deal).  Fifty  three  point  nine  percent  (53.9%)  of  students  gave  this 
response.  Another  thirty  eight  point  six  percent  (38.6%)  gave  the  second  highest  rating  (sufficient).  Thus, 
overall  ninety-two  point  four  percent  (92.4%)  of  respondents  felt  that  their  level  of  learning  was  at  least 
sufficient  in  the  online  environment,  regardless  of  student  background.  Six  point  nine  percent  (6.9%)  felt 
that  their  level  of  learning  was  "insufficient"  and  point  six  percent  (.6%)  felt  that  their  level  of  learning 
was  "none". 

Among  these  students  certain  characteristics  are  significantly  related  to  satisfaction  and  reported  learning. 
When  asked  to  rate  their  level  of  interaction  with  the  instructor  the  following  results  were  found.  Thirty 
three  percent  (33%)  of  the  students  rated  their  level  of  interaction  as  "a  great  deal",  the  highest  rating 
available.  Fifty  one  percent  (51%)  felt  that  their  level  of  interaction  was  "sufficient".  Thus  a total  of  84% 
of  the  respondents  felt  that  their  level  of  interaction  was  at  least  sufficient  in  the  online  environment. 
Fourteen  point  six  percent  (14.6%)  felt  that  the  level  of  interaction  was  "insufficient"  and  one  point  four 
percent  (1.4%)  rated  their  level  of  interaction  as  "none".  When  the  students  level  of  satisfaction  and 
learning  are  compared  to  their  level  of  interaction  with  the  instructor  we  find,  as  predicted,  a significant 
relationship  between  these  variables.  Students  who  reported  the  highest  levels  of  interaction  with  the 
instructor  also  reported  the  highest  levels  of  satisfaction  and  learning. 
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Course  Satisfaction  Level  of  Learning  * Interaction  with 
Instructor 


Interaction  with  Instructor 

Course 

Satisfaction 

Level  of 
Learning 

A great  deal 

Mean 

1.2529 

1.2107 

N 

522 

522 

Std.  Deviation 

.4731 

.4266 

Sufficient 

Mean 

1.5978 

1.5347 

N 

808 

808 

Std.  Deviation 

.6240 

.5795 

Insufficient 

Mean 

2.6164 

2.2328 

N 

232 

232 

Std.  Deviation 

.7919 

.6691 

None 

Mean 

3.1364 

2.5000 

N 

22 

22 

Std.  Deviation 

.9409 

.8591 

Total 

Mean 

1.6547 

1.5436 

N 

1584 

1584 

Std.  Deviation 

.7723 

.6520 

Note:  All  responses  are  based  on  a Likert  scale  from  1-5.  A score  of  1 is  a high  rating;  a score  of  5 is  a low  rating. 

Table  1 : Course  Satisfaction  and  Level  of  Learning  by  Level  of  Interaction  with  the  Instructor 


As  demonstrated  in  Table  1 , each  increase  in  students'  level  of  interaction  with  the  instructor  is  related  to 
an  increase  in  both  satisfaction  with  the  course  and  the  student's  reported  level  of  learning.  These 
increases  are  significant  at  the  .000  level  as  demonstrated  in  Table  2 below: 


ANOVA  Table 


Sum  of 
Squares 

df 

Mean  Square 

F 

Siq. 

Course  Satisfaction  * 
Interaction  with  Instructor 

Between  | (Combined) 

349.760 

3 

116.587 

309.933 

.000 

Within  Groups 

594.345 

1580 

.376 

Total 

944.105 

1583 

Level  of  Learning  * Interaction 
with  Instructor 

Between  (Combined) 

188.214 

3 

62.738 

204.476 

.000 

Within  Groups 

484.781 

1580 

.307 

Total 

672.994 

1583 

Table  2:  Analysis  of  Variance  for  Course  Satisfaction  and  Level  of  Learning  by  Level  of  Interaction  with 
the  Instructor 


Additionally,  these  differences  account  a relatively  large  proportion  in  the  variance  between  students' 
ratings  of  their  levels  of  satisfaction  for  all  the  variables  examined  as  seen  in  the  strength  of  association 
test  in  Table  3 below: 
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Measures  of  Association 


Eta 

Eta  Squared 

Course  Satisfaction  * 
Interaction  with  Instructor 

.609 

.370 

Level  of  Learning  * Interaction 
with  Instructor 

.529 

.280 

Table  3:  ETA  Squared  Measure  of  Association  for  Course  Satisfaction  and  Level  of  Learning  by  Level  of 
Interaction  with  the  Instructor 


Ratings  for  interaction  with  the  instructor  account  for  the  largest  proportion  of  the  variance  for  students' 
reports  of  satisfaction  and  learning.  Additionally,  high  levels  of  interaction  with  classmates  and  high 
reports  of  learning  and  satisfaction  were  also  significantly  related.  Students  with  the  highest  levels  of 
interaction  with  classmates  were  also  the  most  likely  to  report  high  levels  of  learning  and  to  be  the  most 
satisfied.  Due  to  space  constraints,  tabular  information  for  these  and  other  variables  cannot  be  provided 
here.  To  see  this  information  go  to  http://cms.suny.edU/research.htm#Results 

Relationship  of  satisfaction  and  learning  with  other  variables 

As  mentioned  above,  a focus  on  the  social  nature  of  learning  leads  to  several  predictions  regarding  online 
learning.  In  addition  to  the  necessity  of  frequent  interaction  with  the  instructor  and  other  students,  online 
learners  can  also  be  expected  to  require  other  kinds  of  support  in  the  potential  isolation  of  the  virtual 
learning  environment.  For  example,  the  technical  requirement  of  participating  in  such  an  environment 
pose  a possible  barrier  to  interaction,  collaboration  and  hence  the  achievement  of  learning  goals  and 
attendant  satisfaction.  It  can,  therefore,  be  predicted  that  satisfaction  with  the  administration  of  the 
environment  will  be  significantly  related  to  satisfaction  and  learning  online  inasmuch  as  the  effective 
administration  assists  students  to  overcome  these  technological  barriers  to  success. 

An  analysis  of  these  variables  does,  in  fact,  show  that  they  are  significantly  related.  Other  significant 
relationships  that  support  the  view  that  design  and  administration  of  online  instruction  should  focus  on 
the  social  aspects  of  learning  include  significant  relationships  between  satisfaction  with  assistance 
provided  by  the  Help  Desk  and  high  reports  of  student  learning  and  satisfaction.  Students  who  reported 
the  highest  levels  of  satisfaction  with  the  Help  Desk  also  reported  the  highest  levels  of  learning  and 
satisfaction.  Similar  relationships  were  found  for  students'  feelings  about  technical  difficulties.  Those 
that  felt  that  technical  difficulties  caused  problems  in  the  environment  were  also  the  most  likely  to  report 
the  lowest  levels  of  learning  and  satisfaction. 

One  of  the  variables  that  do  not  appear  to  inhibit  online  learning  is  students'  computers  skill  prior  to 
taking  the  course.  Students  who  reported  low  levels  of  computer  skills  before  taking  the  course  were  no 
less  likely  to  report  high  levels  of  learning  and  satisfaction  in  the  course.  As  long  as  students  were  able  to 
get  assistance  for  overcoming  technical  difficulties  reports  of  learning  and  satisfaction  did  not  differ  by 
prior  computer  skill  level.  The  same  can  be  said  of  online  course  experience.  Students  who  had  taken 
multiple  courses  through  SLN  were  no  less  likely  to  report  high  levels  of  learning  and  satisfaction  than 
students  who  had  never  taken  an  online  course  before.  There  is  no  evidence  to  suggest  a "novelty  effect" 
for  satisfaction  or  perceived  learning  among  these  online  students. 

Another  variable  of  interest  was  gender.  Small  but  reliable  differences  exist  suggesting  that,  women  feel 
that  they  participate  at  higher  levels  than  in  the  classroom,  that  they  learn  more,  that  technical  difficulties 
are  less  likely  to  impede  their  learning,  that  they  are  more  likely  want  to  continue  taking  on  -line  courses, 
and  finally,  that  they  are  more  satisfied  both  with  their  specific  courses  at  SUNY  Learning  Network  and 
more  satisfied  with  on-line  learning  in  general  than  their  male  classmates.  In  summary,  the  on-line 
classroom  appears  to  be  a very  female  friendly  place. 
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Discussion 


The  results  reported  here  support  the  notion  that  online  learning  is  best  viewed  through  a focus  on  its 
social  nature.  Students  who  reported  the  highest  levels  of  learning  and  satisfaction  also  reported  the 
highest  levels  of  interaction  with  the  instructor  and  with  other  students,  i.e.  they  had  the  greatest 
opportunity  to  collaborate.  Students  who  reported  the  lowest  levels  of  interaction  with  the  instructor  and 
other  students  reported  the  lowest  levels  of  satisfaction  and  learning,  i.e.  they  had  the  least  opportunity  for 
collaboration.  Barriers  to  participation  evidenced  by  dissatisfaction  with  the  help  desk  and  high  levels  of 
technical  difficulties  are  also  significantly  related  to  satisfaction  and  reported  learning.  Students  who 
reported  the  lowest  levels  of  satisfaction  with  the  help  desk  and  high  levels  of  technical  difficulties  also 
reported  the  lowest  levels  of  learning  and  satisfaction.  Inasmuch  as  interaction  with  the  instructor  and 
other  students  accounts  for  the  most  variation  between  students'  ratings  of  their  satisfaction  and  learning, 
it  can  be  hypothesized  that  barriers  to  this  interaction  caused  by  technical  difficulties  and  inability  to  get 
help  with  their  resolution  can  be  viewed  primarily  as  barriers  to  student  collaboration  with  instructors, 
peers  and  course  content.  Administrative  process  and  instructional  design  in  online  learning  should 
emphasize  lowering  barriers  to  and  increasing  opportunities  for  social  collaboration  and  the  social 
construction  of  knowledge. 
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Abstract:  There  are  huge  numbers  of  Web  pages  in  the  Internet.  However,  many  learners  tend  to 
miss  useful  pages,  miss  valuable  information  in  a page,  or  misunderstand  the  contents  because  there 
are  huge  amount  of  irrelevant  information  exist  mingled  with  the  useful  information.  To  deal  with 
this  problem,  we  propose  a "Web  Recorder^’  system  that  provides  functions  for  recording  and 
replaying  Web  browsing  operations  and  annotating  pages.  Instructors  can  create  a Web  browsing 
"scenario"  to  show  learners  Web  information  that  is  relevant  to  the  learning  objectives.  We 
conducted  experiments  in  real  learning  and  teaching  activities.  The  results  indicate  that  the  system 
made  it  easier  for  learners  to  become  aware  of  and  to  understand  important  information  dispersed 
in  the  pages.  Improvements  in  teaching  and  learning  effectiveness  were  observed  in  the  course  of 
these  experiments. 


1.  Introduction 

The  Internet  has  become  a common  medium  for  the  provision  of  information  and  services.  Numerous  studies  have 
been  done  on  how  to  create  materials  to  be  delivered  via  the  Internet  (e.g.  Brooks  1997,  McCormack,  1997)  and 
there  are  projects  where  the  creation  of  materials  is  used  as  a means  for  education  (e.g.  Thinkquest  2000).  In  these 
kinds  of  activities,  the  Internet  is  considered  as  a communication  channel  to  deliver  materials  and  to  interact  without 
regards  to  time  and  distance.  On  the  other  hand,  the  Internet,  especially  the  Web  system,  can  also  serve  as  a huge 
database  system  consisting  of  sites  around  the  world.  We  believe  that  the  Web  can  be  an  ideal  learning  and  teaching 
resource  if  the  information  meets  the  needs  of  the  learners. 

From  the  viewpoint  of  learners  and  instructors,  there  are  several  issues  in  using  Web  for  learning.  First,  the  Web 
contains  a lot  of  information  but  not  all  of  it  was  created  for  learning  or  teaching  purposes.  Even  if  a topic  on  a Web 
page  is  suitable  for  a learning  activity,  the  page  likely  contains  some  useless  information  or  insufficient  information 
for  the  whole  activity.  Second,  learners  themselves  have  to  decide  which  pages  to  browse,  even  though  they  may  be 
unsure  where  to  go.  Third,  it  is  hard  for  an  instructor  to  control  a learner's  browsing  operations.  A learner  in  front  of 
a Web  browser  is  likely  to  click  hyperlinks  regardless  of  the  instructor’s  intentions. 

All  these  issues  create  difficulties  during  Web-based  learning  and  teaching  activities.  We  the  essential  problems  are 
that  (1)  it  is  hard  for  learners  to  extract  valuable  information  from  large  numbers  of  Web  pages  without  any  help  and 
(2)  it  is  hard  for  instructors  to  lead  their  learners  to  valuable  information  on  Web  pages. 


To  address  these  problems,  we  propose  a method  for  showing  Web  pages  by  using  the  “Web  Recorder"  system.  The 
Web  Recorder  enables  an  instructor  to  record  and  play  back  operations  on  a Web  browser.  An  instructor  can  create  a 
series  of  operations  (which  we  call  a "scenario")  corresponding  to  the  teaching  intention. 

This  paper  describes  experiments  to  investigate  how  the  Web  Recorder  can  improve  learning  and  teaching 
effectiveness.  In  the  next  section,  we  give  an  overview  of  the  Web  Recorder  system,  mainly  from  the  functional 
point  of  view.  In  the  third  section  we  describe  our  objectives,  plans,  and  the  results  of  three  experiments.  After  that, 
we  discuss  the  implications  of  the  results,  mainly  as  regards  how  the  system  changes  a learner’s  behavior.  Finally, 
we  present  our  conclusions  and  outline  future  work. 


2.  System  Overview 

In  this  section  we  explain  the  functions  of  the  Web  Recorder. 

The  Web  Recorder  is  a software  system  that  has  functions  for  both  recording  and  playing  back  a user’s  operations 
on  a Web  browser.  To  record  operations,  the  Web  Recorder  gathers  operation  events  on  a Web  browser.  Each 
operation  is  recorded  with  a time  stamp  and  parameters,  and  stored  in  a file  as  a "scenario."  To  play  back  operations, 
the  Web  Recorder  throws  events  to  a Web  browser  according  to  the  scenario.  The  Web  browser  processes  the  events 
in  the  same  way  as  an  actual  user’s  operations.  In  this  way,  the  Web  browser  plays  back  the  recorded  behavior. 

Web  browser  operations  that  can  be  recorded  by  the  system  are  as  follows; 

(1)  Page  loading 

(2)  Page  scrolling 

(3)  Mouse  movement 

(4)  Text,  images,  hyperlink,  and  voice  annotation 

Page  loadings  are  recorded  with  related  URLs  to  show  the  same  pages  during  a playback.  Page  scrollings  are 
recorded  to  show  the  same  part  of  the  page.  Mouse  movements  are  recorded  to  show  the  same  mouse  pointer 
movement.  The  recorded  mouse  pointer  movements  can  mark  the  focus  on  a page.  These  three  operations  are 
available  on  the  usual  Web  browser  and  the  Web  Recorder  also  enables  users  to  make  annotation  operations  (Figure 
1).  While  recording  on  a Web  browser,  an  operator  (usually  an  instructor)  can  add  text,  images  (e.g.,  illustrations  or 
underlines),  hyperlinks,  and  voice  to  Web  pages  without  modifying  the  HTML  files  of  the  pages.  These  annotations 
are  recorded  as  operations  and  stored  in  the  scenario  file.  A graphical  user-interface  for  annotation  is  included  in  the 
Web  Recorder  system.  Thus  an  operator  can  add  annotations  without  any  special  knowledge,  such  as  how  to  use  a 
page  description  language. 

The  Web  Recorder  was  implemented  with  common  Internet  programming  languages  such  as  Java  and  JavaScript 
and  runs  in  a standard  Internet  environment  such  as  a Web  browser.  Thus  the  program  and  operation  scenarios, 
which  are  plain  text  files,  can  be  distributed  via  the  Internet. 

The  details  of  the  system  are  described  in  (Aoki  et.  al.,  1999,  and  Aoki  et.  al.,  2000). 


(a)  A page  view  without  The  Web  Recorder 


vertical  line, 
an  image, 
a horizontal  line, 
text  boxes  are 
annotated  on  the 
page. 


(b)  A page  view  with  annotations  by  the  Web  Recorder 


Figure  1:  Page  views  with  and  without  the  Web  Recorder 


3.  Experiments  in  Class  Activities 

Two  classes  at  Tamagawa  Gakuen  secondary  school  attended  experimental  lessons,  which  we  organized  in 
cooperation  with  the  school’s  teaching  staff. 
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3.1  Objectives  of  Experiments 

Our  main  objective  was  to  examine  how  the  Web  Recorder  reduced  the  difficulties  of  using  the  Web,  and  improved 
learning  and  teaching  effectiveness.  We  planned  and  conducted  three  experiments  to  examine  the  following  points: 

(1)  The  Web  Recorder  system  as  means  of  showing  information. 

Showing  information  is  an  essential  function  of  scenario  playback.  It  was  compared  with  giving  information  on  a 
sheet  of  paper. 

(2)  A method  for  improving  teaching  effectiveness  with  the  Web  Recorder. 

The  system  enables  teachers  to  indicate  a focus  and  to  show  interpretations  on  parts  of  page.  The  effects  of  these 
techniques  were  examined. 

(3)  A method  for  organizing  learners’  net  surfing. 

The  Web  Recorder  can  guide  learners  around  Web  pages.  We  examined  how  it  works  in  a class  activity. 

3.2  Experiment  1:  Feasibility  Study 

First,  we  examined  the  properties  of  the  system's  functions  for  showing  interpretations.  The  Web  Recorder  system 
has  a function  to  show  interpretations  on  the  Web  pages.  The  interpretations  are  presented  as  textual  and/or  graphic 
annotations.  It  can  facilitate  learning  and  teaching  activities  because  the  system  shows  the  interpretations  near  the 
original  information. 

3.2.1  Experimental  Method 

We  selected  a unit  in  the  English  curriculum  on  understanding  different  cultures,  and  gathered  various  Web  pages 
that  were  appropriate  for  the  unit  except  for  the  difficulty  of  the  English  and  the  number  of  topics  on  the  pages.  We 
then  added  abstracts  for  the  pages  and  notes  (The  pages  were  used  after  receiving  the  author’s  permission). 

We  divided  the  class  into  two  groups  (of  13  and  12  students)  and  conducted  experiments  in  two  sessions.  In  the  first 
session  we  gave  one  group  a Web  page  with  annotations  made  with  the  Web  Recorder.  For  the  purpose  of 
comparison,  the  other  group  saw  the  same  Web  page  with  annotations  printed  on  a sheet  of  paper.  In  the  second 
session,  we  prepared  another  topic.  The  group,  that  used  the  Web  Recorder  in  the  previous  session,  saw  the  new 
topic  on  a Web  page  with  a sheet  of  paper,  while  the  other  group  used  the  Web  Recorder. 

3.2.2  Results  of  the  Feasibility  Study 

After  each  session,  we  asked  the  students  to  answer  six  questions: 

(1)  Was  the  session  fun? 

(2)  Was  the  topic  interesting? 

(3)  Was  there  enough  time  for  you? 

(4)  Is  it  useful  to  use  Web  pages  as  we  did  today,  in  comparison  with  looking  at  Web  pages  by  yourself? 

(5)  Did  you  understand  the  topic  in  the  textbook  better  than  usual? 

(6)  Would  you  feel  more  interested  in  English  if  we  often  have  sessions  like  today? 

Each  question  had  five  possible  responses  to  indicate  opinion  ranging  from  the  most  favorable  (e.g..  It  was  really 
fun,  really  interesting,  and  so  on)  to  the  most  unfavorable  answers  (e.g..  It  was  not  fun  at  all,  etc.). 
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We  converted  the  answers  into  scores  ranging  from  1 (most  unfavorable)  to  5 (most  favorable)  and  averaged  the 
results  separately  according  to  whether  the  Web  Recorder  was  used  or  not.  The  scores  were  averaged  to  cancel  the 
effects  of  topics  and  the  competence  of  group  members. 

Figure  2 shows  the  averaged  scores  for  each  of  the  questions. 

These  results  indicate  that;  the  scores  for  recorder  users  exceeded  those  for  browser  and  paper  users  in  the  results  of 
five  questions  out  of  six.  For  Question  3,  the  score  of  the  Web  Recorder  group  did  not  exceed  those  of  the  other 
group.  The  question  was  about  if  the  given  time  had  been  enough  or  not  (a  higher  score  indicates  that  they  felt 
enough  time  was  given).  There  was  a system  trouble  with  the  Web  Recorder  that  forced  students  to  wait  for  a long 
time  (it  was  solved  before  the  second  session).  It  is  likely  that  the  trouble  caused  a negative  influence  on  the  Web 
Recorder  users'  score  for  the  question. 

In  evaluating  the  result,  we  have  to  take  account  of  the  presentation  problems.  Thus  we  believe  we  can  conclude  that 
the  Web  Recorder  had  equally  good  or  better  characteristics  than  paper  as  a medium  for  showing  information. 

3.3  Experiment  2:  Focusing  and  Interpretation 

In  the  second  experiment,  we  examined  the  effect  of  focusing  and  interpreting  on  teaching  activity  in  a classroom. 
We  believe  that  teacher-led  sessions  are  a common  and  valuable  way  of  presenting  ideas.  It  is  meaningful  to  try  to 
improve  teacher-led  sessions  not  only  through  the  various  merits  of  the  Web  such  as  volume,  freshness  of 
information  and  bi-directional  communication,  but  also  through  the  method  of  the  Web  Recorder. 

33.1  Experimental  Method 

To  observe  how  the  system  facilitated  teaching  sessions,  we  planned  a review  session  after  a presentation  session  in 
English  to  a class,  which  consists  of  students  who  have  returned  to  Japan  after  living  abroad.  The  educational 
objective  was  to  give  the  students  some  ideas  on  presentation  skills  and  to  make  them  to  think  about  how  to  improve 
their  presentations.  First,  we  recorded  all  of  their  speeches  and  created  Web  pages  that  reproduced  their 
presentations,  using  conventional  Web  authoring  tools.  The  pages  consisted  of  their  voices  and  scripts  of  their 
presentations.  We  selected  the  best  two  presentations  and  created  the  teaching  scenario  by  using  the  Web  Recorder 
to  present  these  pages  with  annotations  of  major  points.  We  also  recorded  the  teaching  presentation  as  an  operation 
sequence. 

The  experimental  session  consisted  of  self-reviewing  the  pages  created  for  each  student,  then  a lecture  on  a sample 
presentation,  using  the  Web  Recorder,  and  a discussion  of  how  they  felt  about  improving  their  presentations. 

We  gathered  their  opinions  on  their  presentations  and  suggestions  to  improve  them.  We  asked  them  to  categorize 
their  opinions  and  suggestions  into  those  concerning  presentation  technique  and  those  concerning  contents.  These 
opinions  were  compared  with  the  opinions  submitted  at  the  end  of  the  presentation  session  (i.e.,  before  the 
experimental  session).  We  observed  the  class  session  and  evaluated  how  the  recorder  facilitated  it. 

33.2  Results  of  the  Review  Session 

Differences  were  observed  between  the  opinions  received  before  and  after  the  experimental  session,  in  both  number 
and  quality.  Before  the  review  session,  there  were  only  26  opinions  from  1 5 students.  After  the  review  session,  the 
number  increased  significantly  to  63  opinions.  The  system  contributed  to  the  success  of  the  class  by  focusing  on 
many  ideas,  although  this  was  not  entirely  due  to  the  Web  Recorder. 

As  regards  the  quality  of  the  opinions,  these  after  the  review  session  concerned  concrete  methods  for  achieving 
concrete  improvement.  On  the  assumption  that  the  opinions  progress  from  vague  impressions  to  concrete  methods, 
they  were  categorized  into  three  groups: 

(Category  1 ) Abstract  points  for  improvement 
(Category  2)  Concrete  points  for  improvement 
(Category  3)  Concrete  methods  for  achieving  improvement 
Table  1 shows  the  results  of  categorization: 


Counts  of  Opinion 

Before  the  Review 

After  the  Review 

Category  1 (Abstract  Points) 

5 

4 

Category  2 (Concrete  Points) 

21 

55 

Category  3 (Concrete  Methods) 

0 

4 

Total 

26 

63 

Table  1:  Numbers  of  opinion  in  each  category  before  and  after  the  review  session. 


It  is  obvious  that  more  opinions  appeared  in  Categories  2 and  3 after  the  session.  The  system  had  succeeded  in 
focusing  and  strengthening  the  presentation  of  the  teaching  materials,  and  had  contributed  to  the  success  of  the 
teaching  session. 

3.4  Experiment  3:  Organizing  Web  Browsing 

Third,  we  tried  to  organize  class  activities  in  net  surfing  with  the  aim  of  achieving  a learning  objective  set  by  a 
teacher.  It  is  hard  for  students  to  find  and  understand  descriptions  of  a topic  scattered  over  several  pages.  This 
experiment  examined  the  effects  of  the  Web  Recorder  system  for  guiding  students  to  several  paragraphs  scattered 
over  several  Web  pages  and  interpreting  understanding  them. 

3,4J  Experimental  Method 

We  selected  the  topic  "weddings  and  marriage"  from  a learning  unit  titled  "Understanding  different  cultures."  We 
decided  that  the  educational  objective  of  the  teaching  was  to  motivate  English  study  by  giving  the  students  the 
experience  of  finding  an  interesting  world  of  different  cultures  through  English  and  the  Internet.  First,  we  searched 
around  the  Internet,  then  chose  13  pages  containing  descriptions  related  to  the  topic.  We  felt  that  13  pages  would  be 
enough  to  simulate  Internet  surfing  because  this  amount  is  enough  for  a one-hour  class.  We  received  the  authors’ 
permission  to  use  their  pages,  and  created  link  lists  for  these  pages.  We  prepared  translations  for  each  page,  because 
the  English  of  the  pages  is  too  difficult  for  the  students  to  understand  without  spending  considerable  time. 

We  then  chose  8 pages  containing  descriptions  of  customs  significantly  different  from  those  in  Japan.  From  many 
interesting  points  in  those  pages,  we  selected  10  points  for  the  class,  and  a teacher  created  a surfing  scenario  to  see 
these  points  scattered  among  the  pages. 

In  the  experimental  class  hours,  each  student  had  a computer  and  was  able  to  freely  browse  Web  pages  from  the  link 
list  page  that  we  had  created.  We  divided  one  class  of  students  into  three  groups,  and  divided  a class  hour  into  three 
periods.  We  showed  the  scenario  to  each  group  in  a different  period;  for  instance,  we  showed  the  scenario  to  group  1 
in  period  1,  to  group  2 in  period  2 and  to  group  3 in  period  3.  One  period  was  ten  minutes,  which  was  not  enough  to 
see  the  whole  scenario,  and  the  students  could  control  the  Web  Recorder  to  progress  to  the  next  point.  Each  student 
had  a sheet  of  paper  to  write  down  comments  about  what  they  found. 

3,4,2  Results  of  Organizing  the  Web  Browsing 

Table  2 shows  the  averaged  numbers  of  comments  per  person  for  each  group  and  period. 


Group  1 

Group  2 

Group  3 

Period  1 

1.8  n 

1.5 

1.5 

Period  2 

1.6 

2.2  (*) 

1.9 

Period  3 

1.3 

2 

3.0  n 

(*)  Groups  saw  the  scenario  by  Web  Recorder  in  this  period. 

Table  2:  Tbe  averaged  numbers  of  comments  from  the  organized  Web  browsing. 

Each  group  saw  the  scenario  recorded  by  the  Web  Recorder  in  the  period  marked  "(*)."  Obviously,  the  members  of 
each  group  expressed  more  comments  in  the  period  when  they  saw  the  scenario  than  the  other  groups  in  the  same 
period,  and  than  during  their  other  two  periods.  We  believe  that  this  result  indicates  that  the  Web  Recorder  organizes 
the  Web  surfing  to  focus  on  points  of  the  teacher’s  intention  among  large  amounts  of  information. 

We  also  noticed  a decrease  in  the  number  of  comments  during  the  period  after  they  saw  the  scenario  even  though  the 
average  increased.  Figure  3 shows  the  average  number  of  comments  from  all  groups  for  each  period: 


- Comments 
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per 


Figure  3:  Average  number  of  comments  from  all  groups  for  each  period  in  the  organized  Web  browsing 


We  believe  this  tendency  resulted  from  the  students  getting  the  hang  of  the  work.  This  is  why  we  can  see  that  the 
overall  average  increased,  despite  the  previously  mentioned  decrease  after  seeing  the  scenario.  We  observed  the 
following  attitude  among  some  students:  "I  have  seen  the  contents  of  the  scenario.  What  should  I do  now?*’  even 
though  there  were  many  points  they  had  not  focused  on.  This  kind  of  attitude  and  the  decrease  in  the  number  of 
comments  after  seeing  the  scenario  shows  that  the  students  felt  satisfaction  and  had  an  illusion  that  they  had  seen  all 
the  topics. 

Figure  4 shows  the  average  number  of  comments  for  each  group  in  whole  period. 


Figure  4:  Average  number  of  comments  for  each  group  in  the  organized  Web  browsing. 

The  average  numbers  may  have  been  affected  by  the  period  when  they  saw  the  scenario.  We  believe  that  they  were 
not  significantly  affected  by  the  ability  of  each  individual  group,  because  we  took  care  to  create  groups  of  equal 
ability.  To  put  it  simply,  we  feel  these  averages  indicate  that  the  scenario  should  be  shown  at  the  end  of  the  class 
hour.  We  can  conclude  that  the  Web  Recorder  helps  learners  to  focus  on  the  exact  point  selected  by  the  educator,  but 
we  have  to  be  aware  of  the  feeling  of  complacency. 

4.  Discussion 

Besides  presenting  the  results,  we  have  to  consider  some  limitations  of  our  research.  First,  we  conducted  only  one 
experiment  of  each  type  because  we  had  limited  time  for  these  experiments  in  the  regular  teaching  schedule.  Second, 
the  numbers  of  students  are  too  small  to  prove  the  effects  statistically.  Also  there  should  be  a measurement  method 
for  education  results  beside  the  use  of  questionnaires.  In  spite  of  these  limitations,  we  believe  that  the  results 
indicate  the  software  has  some  excellent  features  for  education  and  suggest  directions  for  the  next  experiments, 
which  will  investigate  the  system  while  addressing  these  limitations. 

5.  Conclusion 

The  Web  Recorder  has  functions  for  showing  important  parts  of  Web  pages  without  distraction  from  a huge  number 
of  pages  or  from  a large  number  of  topics  in  a single  page.  It  also  has  functions  for  adding  annotations  to  parts  of 
pages  to  interpret  them  or  provide  the  contextual  knowledge  required  understanding  them.  We  conducted  three  types 
of  experiment  involving  real  class  activities  to  examine  how  these  functions  affected  learning  and  teaching.  The 
results  indicate  that  the  Web  Recorder  can  eliminate  some  of  the  difficulties  faced  by  learners  because  of  specific 
properties  of  the  Internet,  such  as  excessive  numbers  of  pages,  excessive  number  of  topics  on  a page,  and  lack  of 
necessary  background  information.  The  Web  Recorder  improves  learning  and  teaching  effectiveness  with  Web 
browsing  by  improving  the  method  of  showing  pages. 
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Abstract:  A promising  approach  to  support  learning  in  and  about  complex  domains  involves  the  use  of 
system  dynamics  as  a methodology  since  it  supports  both  the  modeling  of  complex  systems  and 
interaction  with  those  models  (Sterman,  1994).  Two  central  disputes  among  advocates  of  system 
dynamics  based  learning  are:  (1)  the  extent  to  which  collaboration  among  learners  facilitates  learning; 
and,  (2)  the  utility  of  making  underlying  models  accessible  to  and  modifiable  by  learners  (Davidsen, 
1996).  Both  of  these  issues  concern  the  degree  of  learner  engagement  and  appear  cmcial  to  overall 
learning  effectiveness.  Little  empirical  evidence  has  been  gathered  to  explore  learning  effects  of 
different  approaches.  This  paper  describes  a methodology  for  evaluation  of  learning  effectiveness  in 
complex  and  dynamic  domains  and  reports  on  its  the  use  in  settings  that  include/exclude  opportunities 
for  collaboration,  which  show/hide  the  underlying  system  dynamics  model,  and  which 
allow/disallow  modification  of  the  underlying  model. 


Introduction 

Many  excellent  uses  of  system  dynamics  to  support  learning  in  complex  domains  exist  and  are  available 
commercially.  Learning  environments  to  support  learning  about  complex,  dynamic  systems  include  SimCity^*^  for 
helping  children  learn  about  factors  influencing  the  growth  of  urban  area  and  Beefeater^*^  for  helping  adults  learn 
about  factors  influencing  the  growth  of  a particular  business,  and  a host  of  other  such  environments  which  are  now 
frequently  used  to  support  various  types  of  learning  activities.  Such  environments  are  consistent  with  a great  deal  of 
learning  research  concerning  active  learning  (Bnmer,  1985),  cognitive  apprenticeship  (Collins,  1991),  cognitive 
flexibility  (Spiro  et  al.,  1987),  and  situated  learning  (Lave,  1988). 

However,  there  is  very  little  evidence  to  demonstrate  the  impact  of  these  environments  and  their  overall 
learning  effectiveness.  Moreover,  there  is  no  established  methodology  to  determine  which  design  approach  might  be 
most  likely  to  lead  to  desired  learning  outcomes  in  various  situations.  Our  long  term  goal  is  to  develop  a 
methodology  which  can  be  used  to  insure  consistent  success  in  the  effectiveness  of  system  dynamics  based  learning 
environments,  and  to  establish  reliable  measures  of  effectiveness  for  such  environments.  We  shall  describe  an 
evaluation  methodology  that  is  particularly  well  suited  to  complex  and  dynamic  domains.  We  shall  then  describe 
how  that  methodology  can  be  used  to  evaluate  the  impact  of  different  key  design  decisions  in  constructing  such 
environments,  including  whether  and  how  to  support  collaborative  learning,  whether  and  when  to  make  the 
underlying  model  visible  to  learners,  and  whether  and  how  to  provide  learners  with  control  of  the  underlying  model. 


There  exists  a well-elaborated  methodology  for  analyzing  a complex,  dynamic  domain  in  terms  of  system 
dynamics  (Davidsen,  1996;  Spector  & Davidsen,  1997).  That  model  proceeds  on  several  assumptions.  First,  the  type 
of  complexity  involved  is  characterized  by  systems  which  involve  nonlinear  relationships,  delays  and  internal 
feedback  among  several  variables,  systemic  behavior  which  may  change  dramatically  over  time,  and  systems  in 
which  human  perceptions  and  actions  based  on  perception  play  an  active  role  in  determining  performance  and 
outcomes.  That  such  systems  can  be  effectively  modeled  using  system  dynamics  is  now  quite  well  established 
(Forrester,  1961,  1985,  1992;  Sterman,  1994).  What  is  not  well  established  is  how  such  models  can  be  optimally 
used  to  facilitate  learning. 

It  has  been  argued  that  collaboration  facilitates  learning  about  new  and  complex  topics  (Salomon,  1993; 
Vygotsky,  1978).  This  belief  in  the  utility  of  collaboration  in  learning,  however,  has  not  been  demonstrated  to  be 
effective  in  the  context  of  system  dynamics  based  learning  environments,  although  nearly  all  such  environments  in 
some  way  attempt  to  integrate  collaboration  with  other  learners  into  various  learning  activities. 

Moreover,  there  is  a clear  distinction  between  a model  of  reality  and  the  process  of  modeling  reality.  Many 
of  those  who  support  the  use  of  system  dynamics  in  learning  about  complex  domains  generally  support  the  notion  of 
constructivists  that  learners  construct  their  own  models  of  reality.  Likewise,  many  who  construct  system  dynamics 
based  learning  environments  believe  that  active  learner  engagement  with  the  underlying  models  is  crucial.  This 
implies  that  learners  should  be  eventually  be  provided  with  access  to  underlyling  models  and  encouraged  to  modify 
and  reconstruct  those  models. 

Rouwette  and  colleagues  (in  press)  further  argue  that  group  model  building  actively  contributes  to  learning 
outcomes.  They  have  produce  some  preliminary  evidence  that  both  the  collaborative  and  the  constructivist  aspects 
of  such  system  dynamics  based  learning  environments  contribute  to  improved  learning.  However,  their  research  is 
largely  based  on  data  gathered  from  self-reports  of  participants  and  is  difficult  to  independently  confirm  and 
validate. 

From  this  brief  review  of  design  trends  and  beliefs  with  regard  to  system  dynamics  based  learning 
environments,  it  should  be  obvious  that  learner  engagement  is  generally  viewed  as  critical  to  the  efficacy  of  a 
learning  improvement,  consistent  with  the  general  instructional  design  and  learning  effectiveness  literature. 
Moreover,  the  general  belief  in  the  system  dynamics  learning  community  is  that  collaboration  with  other  learners 
enhances  learner  engagement  as  does  the  ability  to  manipulate  existing  and  create  alternative  models.  While  we 
share  these  widely  held  beliefs,  it  is  our  intention  to  establish  a methodology  which  can  be  used  to  determine  to  what 
extent  these  beliefs  are  true. 

The  Evaluation  Methodology 

It  is  possible  to  view  learning  from  a number  of  alternative  perspectives.  The  preferred  learning  perspective 
largely  determines  the  appropriate  learning  effectiveness  methodology.  Sfard  (1998)  identifies  a participation 
metaphor  and  contrasts  it  with  an  acquisition  metaphor.  She  argues  that  both  metaphors  should  be  taken  into  account 
when  considering  learning  from  a larger  and  longer-term  perspective.  We  agree  with  this  argument.  The 
implications  for  evaluation  are  twofold.  When  learning  is  viewed  as  acquisition  of  expertise,  it  makes  sense  to 
evaluate  learning  in  terms  of  how  learners  are  performing  and  thinking  in  comparison  with  experts.  When  learning  is 
viewed  as  participation  in  a community  of  practitioners,  it  makes  sense  to  evaluate  how  activities  of  learners  are 
changing  in  various  learning  and  working  situations.  When  the  domains  are  dynamic  and  complex,  evaluation  is 
more  challenging  since  experts  are  likely  to  exhibit  a wider  ranges  of  performances  and  since  activities  are  much 
more  open-ended  than  is  the  case  with  simpler  domains.  Nevertheless,  a scientific  attitude  with  regard  to  the  design 
of  environments  to  support  learning  about  complex  domains  implies  the  need  to  collect  measures  of  outcomes  in  a 
variety  of  settings  in  order  to  determine  what  works  best,  when,  and  why. 

We  begin  with  the  leaming-as-acquisition-of-expertise  perspective  (Ericsson  & Smith,  1991).  With  simpler 
domains,  it  has  been  demonstrated  that  experts  do  in  fact  exhibit  similarities  in  their  thinking,  and,  consequently,  it 
is  possible  to  capture  initial  learner  thinking  patterns  in  the  form  of  concept  maps,  to  see  how  those  maps  differ  from 
concept  maps  of  experts,  and  then  track  changes  over  time  and  after  instruction  and  experience  performing  a variety 
of  tasks  (Schvaneveldt  et  al.,  1985).  The  type  of  concept  maps  which  have  been  used  to  capture  learner  and  expert 
thinking  range  from  association  nets  to  semantic  networks.  Surprisingly,  simple  association  nets  have  proven  to  be 
excellent  predictors  of  performance  in  a variety  of  domains,  including  aircraft  maintenance  as  well  as  formation 
flying  (Schvaneveldt  et  al.,  1985).  These  domains,  however,  are  not  complex  and  dynamic  in  the  sense  described 
earlier,  so  we  still  regard  these  domains  as  simple  when  compared  with  the  domains  modeled  and  supported  with 
system  dynamics  (e.g.,  predicting  the  spread  of  an  epidemic,  macro-economic  planning,  environmental  policy 
formation,  etc.). 
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Figure  1.  Causal  loop  diagram  for  workforce  problems. 

The  type  of  concept  mapping  methodology  which  appears  to  be  the  most  appropriate  for  the  kinds  of 
complex,  dynamic  domains  which  are  well  supported  by  system  dynamics  modeling  and  simulation  techniques  is 
causal  loop  diagramming.  Causal  loops  are  visual  representations  of  the  dynamic  influences  and  interrelationships 
that  exist  among  a collection  of  variables.  Figure  1 depicts  a causal  loop  diagram  for  typical  workforce  problems 
which  project  managers  confront.  This  diagram  consists  of  a number  of  key  factors  which  are  linked.  The  arrows 
and  letters  'S*  (same)  or  ‘O’  (opposite)  indicate  the  direction  and  nature  of  causal  influence.  For  example,  hiring 
leads  to  an  increase  in  the  total  workforce.  An  increase  in  the  total  workforce  leads  to  a decrease  in  the  workforce 
gap  (those  needed  to  fulfill  task  requirements).  The  double  slash  across  a link  indicates  a delay.  Identifying  and 
understanding  the  nature  of  delays  is  especially  problematic  in  complex  systems.  In  this  example,  there  is  a delay 
indicated  between  the  time  an  advertisement  for  a job  is  published  and  when  a person  is  hired.  A second  delay  is 
depicted  between  when  an  increase  in  necessary  team  size  is  identified  and  persons  hired  and  placed  onto  the  team. 

It  should  be  noted  that  this  causal  loop  diagram  is  intended  only  to  represent  the  general  nature  of  causal 
loop  diagrams.  It  docs  not  necessarily  represent  how  experts  view  a particular  workforce  problem.  Much  that  is 
relevant  to  such  problems  has  been  omitted  to  make  the  problem  and  diagram  easily  accessible.  For  example,  there 
is  a significant  delay  between  the  time  when  persons  are  placed  on  a project  and  an  improvement  in  overall  project 
productivity.  Often  productivity  even  declines  initially  when  additional  persons  are  added  because  they  are  typically 
less  experienced  workers,  require  training  and  project  orientation,  and  increase  the  communication  overhead 
associated  with  an  effort.  The  point  here,  however,  is  that  it  is  just  such  subtle  relationships  and  delays  that  expert 
project  managers  are  able  to  anticipate  and  integrate  into  project  planning.  Moreover,  it  is  precisely  such  delays  and 
internal  feedback  that  less  sophisticated  concept  mapping  tools  fail  to  capture. 

Our  general  hypothesis  is  that  within  the  context  of  a problem  area  in  a complex,  dynamic  system  experts 
will  construct  similar  causal  loop  diagrams.  Furthermore,  these  diagrams  will  be  noticeably  different  from  those 
constructed  by  less  experienced  persons.  As  a consequence,  the  level  of  fit  between  a learner’s  causal  loop  diagram 
and  that  of  an  expert’s  will  be  a reasonable  predictor  of  level  of  expertise  in  that  problem  domain.  This  particular 
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methodology  is  partly  quantitative  and  partly  qualitative  in  nature.  A measure  of  fit  between  a learner’s  causal  loop 
diagram  and  an  expert’s  is  derived  based  on  similarities  in  the  set  of  key  concepts  identified  for  a particular  problem 
domain,  and  the  types  and  directions  of  links.  Additionally,  subjects  are  asked  to  insert  open-ended  comments  about 
links  and  key  components.  Protocol  analysis  is  used  to  determine  whether  a subject’s  comments  reflect  an  expert 
level  of  understanding  of  that  problem  domain. 

When  learning  is  considered  as  participating  in  a community  of  practitioners,  a different  evaluation 
methodology  appears  more  appropriate.  In  this  case,  patterns  of  collaboration  with  peers,  places  where  specific 
expertise  is  sought,  how  proposals  are  evaluated,  interactions  with  and  about  various  artifacts,  and  the  like  are  more 
directly  relevant.  These  data  can  be  collected  by  interviews  and  observations  and  changes  noted.  The  evaluation 
methodology  in  this  case  is  primarily  qualitative  in  nature,  drawing  on  the  notion  of  distributed  cognition  (Salomon, 
1993)  and  the  general  approach  found  in  activity  theory  (Nardi,  1996)  from  a learner-centered,  phenomenographic 
perspective.  Specifically,  learners  and  teachers  are  interviewed  prior  to  exposure  to  a learning  experience  with  a 
participatory,  system  dynamics  learning  environment.  That  learning  environment  will  constitute  the  primary 
mediating  object.  Questions  are  designed  to  establish  current  patterns  of  interactions  with  other  learners  and 
teachers,  to  determine  attitudes  about  specific  technology-based  learning  capabilities,  and  to  record  perceived 
learning  value  of  those  capabilities.  During  the  learning  experience  and  subsequent  to  the  experience  with  the 
participatory  learning  environment,  learners  and  teachers  are  interviewed  and  asked  similar  questions.  Changes  are 
analyzed  so  as  to  determine  whether  and  how  patterns  of  interaction  and  attitudes  evolve  as  a result  of  participation 
in  the  continuing  construction  of  a digital  database  or  simulation  on  the  web. 

Discussion  and  Conclusions 

The  first  step  in  developing  the  causal  loop  mapping  tool  was  to  determine  if  it  was  true  that  experts  did 
produce  similar  causal  loop  diagrams  for  particular  problem  domains.  Three  quite  different  problems  were  selected 
for  this  purpose:  (1)  an  inventory  management  problem;  (2)  an  environmental  policy  formulation  problem;  and,  (3) 
a problem  involving  interpersonal  relationships.  In  each  domain,  five  experts  and  five  novices  were  asked  to 
produce  causal  loop  diagrams  on  paper  representing  the  concepts,  factors,  and  causal  relationships  thought  to  be 
associated  with  a short  problem  description  (about  two  pages  each). 

Since  there  were  obvious  similarities  in  the  set  of  expert  causal  loops  diagrams,  and  because  newcomers  to 
a complex  domain  exhibited  noticeably  different  causal  loop  diagrams,  there  is  the  possibility  that  the  nearness  of  fit 
to  an  expert’s  causal  loop  diagram  would  be  a predictor  of  expert  understanding  in  a particular  problem  domain. 
Based  on  this  initial  confirmation,  a tool  was  built  to  facilitate  the  creation  of  problem  scenarios  and  associated 
causal  loop  diagrams.  The  tool  is  designed  to  proceed  in  two  steps,  first  prompting  subjects  to  create  a list  of 
annotated  key  concepts  from  the  scenario,  and  then  prompting  users  to  create  an  annotated  causal  loop  diagram 
based  on  those  key  concepts.  Subjects  are  allowed  to  revise  their  ideas  and  all  entries  are  recorded  in  a user  log  file. 

Automating  the  process  of  performing  the  concept  mapping  is  intended  to  make  it  possible  to  collect  and 
analyze  a great  deal  of  relevant  data  to  determine  how  effective  particular  learning  environments  are  with  regard  to 
promoting  understanding  in  and  about  complex  domains.  This  approach  is  basically  an  extension  of  the  pathfinder 
technology  (Schvaneveldt,  1 990)  to  include  complex,  dynamic  domains  and  a concept  mapping  and  analysis  tool 
appropriate  to  such  domains. 

The  qualitative  methodology  and  interview  techniques  used  for  the  analysis  of  participation  in  communities 
of  practice  has  been  used  in  other  domains  and  independently  demonstrated  to  be  robust  (see,  for  example,  Nardi, 
1996).  Our  findings  from  initial  pilot  tests  indicate  that  both  methods  tend  to  produce  consistent  indications  of 
learning  effects.  If  this  finding  continues  to  hold  as  additional  subjects  are  tested,  and  if  there  is  independent 
confirmation  that  a correlation  between  these  indicators  and  performance  on  complex  tasks,  then  the  advantage  of 
the  causal  loop  diagramming  technique  is  that  it  can  also  support  learning  about  complex  domains  and  not  merely 
evaluation  of  learning  effects. 
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Abstract:  This  paper  explores  the  validity  of  contentions  made  in  the  literature  concerning 
the  advantages  of  computer  conferencing  in  facilitating  learning.  It  asserts  that  much  of  the 
supporting  research  is  lacking  in  conceptual  clarity,  methodological  rigor  and  systematic 
analysis.  The  findings  have  frequently  emerged  from  studies  conducted  under  conditions 
removed  from  everyday  instructional  contexts.  Consequently,  these  can  only  be  generalised 
with  difficulty  to  extra-research  settings,  if  at  all.  The  paper  includes  a report  of  case  studies 
on  the  use  of  computer  conferencing  in  settings  that  probably  approximates  more  closely 
the  typical  instructional  context  in  many  institutions  of  higher  education.  Non-moderated 
computer  conferencing  was  employed  on  a number  of  courses  and  the  results  compared  to 
others  that  employed  traditional,  non-electronically  mediated,  pedagogic  approaches.  The 
results  indicate  that  in  the  absence  of  any  moderator  input,  students  can  achieve  results 
comparable  to  those  attained  by  more  traditional  methods. 


Overview 

The  literature  on  computer  conferencing  is  already  very  extensive  and  has  periodically  been  subjected  to 
extended  reviews  (e.g.  Kaye  91;  Mason  91;  Paulsen  95;  Berge)  Although  a significant  volume  of  work  has  been 
undertaken  to  establish  whether  or  not  the  deployment  and  use  of  electronic  mediating  technologies  enhances 
learning  outcomes,  many  findings  have  been  called  into  question  on  conceptual,  methodological  and  empirical 
grounds,  (e.g.  Schute;  Russell;  Neal;  Cooper  and  Robinson;  Springer,  et  al.)  Moreover,  the  findings  tend  to  be 
somewhat  inconsistent,  as  Kaye  (1991)  has  noted.  Although  there  appears  to  be  a broad  consensus  that  computer 
conferencing  can  enhance  the  learning  experiences  of  students,  especially  as  far  as  distance  learning  is 
concerned,  not  much  evidence  has  been  adduced  that  the  leaming/pedagogic  outcomes  are  in  some  sense 
superior  to  those  attainable  via  more  traditional  face-to-face  interactive  forums. 

Much  of  the  literature  is  comprised  of  case  studies  of  the  ongoing  experiences  of  educators.  Generally,  this  has 
a tendency  to  be  methodologically  unstructured.  It  is  often  non-comparative  in  orientation,  does  not  include 
adequate  control  groups,  and  is  rarely  enlightened  by  appropriate  statistical  analysis.  Consequently,  many  of  the 
findings  are  essentially  anecdotal,  being,  for  the  most  part,  reports  from  the  electronic  frontier.  Although  it  may 
very  well  turn  out  in  the  medium  to  long  term  that  many  of  these  “common-sense”  assertions  are  borne  out  by 
subsequent  research,  at  this  juncture  many  of  the  claims  that  have  been  made  appear  to  be  unproven,  not  a few 
of  which  are,  in  Popperian  terms,  untestable. 

To  illustrate  with  one  example,  the  literature  abounds  with  claims  concerning  the  benefits  of  computer  mediated 
group  work  without  specifying  clearly  what  the  dependent  variables  are,  and  what  would  constitute  reasonably 
adequate  confirmation.  Thus,  Henri  (91;  see  also  Kaye  91;  Harasim  89)  contends  that  group-work  “consistently 
yields  results  of  a higher  calibre  than  those  attained  by  the  average  group  member.  This  is  due  to  the  greater 
amount  of  information  available  within  the  group,  the  greater  diversity  of  interpretations  of  fact  and  the 
opportunity  to  test  individual  ideas.  ...the  combination  of  social  osmosis,  the  circulation  of  ideas,  and  the  links 
established  among  the  participants  all  contribute  to  the  efficiency  of  CMC  group  exchanges.”  Perhaps,  but  a 
very  elaborate  and  problematic  research  design  would  be  required  to  provide  evidence  attesting  to  this  (see 
Berthold  M.R.  et  al.,  97,  on  mailing  list  interactivity).  At  present  the  most  frequently  employed  objective 
evaluative  criterion  is  grade  performance,  whilst  the  subjective  counterpart  is  student  satisfaction.  Detailed 
qualitative  analyses  of  exchanges  are  the  only  means  of  grappling  with  such  wide  reaching  claims.  The  required 
research  designs  are  intensively  resource  demanding,  and  the  findings  are  unlikely  to  meet  requirements  of 
replication  and  objectivity,  however  loosely  defined,  (see  Mason,  1991;  Berthold  et  al.,97.;  Henri  91;  Graham,  et 
al.,  99,  on  the  need  for  qualitative  assessments  of  content) 
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In  these  reports  there  is  all  too  oflen-insufficient  consideration  given  to  the  large  number  of  interacting  variables 
plausibly  associated  with  perceived  outcomes,  learning  and  other.  Some  of  the  most  significant  relate  to 
characteristics  associated  with  participants  in  conferencing  exchanges.  There  is  no  question  that  until  very 
recently  most  case  studies  were  based  on  experiences  with  distance  education  students,  (e.g.,  Kaye  91,  Mason 
91,  McConnell  91)  A disproportionate  number  involve  students  taking  information  technology  related  courses, 
students  who  can  be  expected  to  be  more  favourably  disposed  toward  pedagogic  programs  employing  electronic 
platforms  (e.g.  Hiltz  95).  The  student  subjects  in  many  studies  tend  to  be  older  than  the  typical  campus-based 
student,  and  differ  from  them  in  other  important  respects  which  probably  have  a direct  bearing  on  learning 
related  outcomes.  Their  motivation  and  commitments,  their  social  relations,  the  nature  of  their  social  networks, 
their  economic  situation,  and  their  pedagogic  backgrounds  and  experiences,  all  potentially  relevant  to  learning 
outcomes,  are  probably  significantly  different  from  those  of  the  vast  majority  of  students  undertaking  campus- 
based  higher  education  courses. 

The  list  of  potentially  relevant  variables  is  too  lengthy  to  include  a detailed  exposition  on  here.  They  include, 
but  are  by  no  means  limited  to:  (1)  the  category  of  student  [mature/school  graduate;  graduate/post-graduate; 
experienced  electronic  communication  media  users/inexperienced  electronic  communication  media  users; 
continuing  education/life-long  learning].  (2)  The  conferencing  system  employed  and  its  organisation,  [which 
relate  to  ease  of  use,  difficulty  of  hierarchical  inclusion  and  integration,  feedback  on  activities  engaged  in  by 
participants,  availability  of  private  and  public  fora,  integration  with  other  electronic  applications  and  their 
transparency/ease  of  use];  (3)  The  extent  to  which  the  conferencing  system  is  integrated  with  other  electronic 
applications  [e.g.  use  of  the  Web,  electronic  revision/testing  applications];  (4)  The  role  of  the  conferencing 
component  in  the  overall  structure  of  the  delivery  of  pedagogic  materials.  (5)  The  overall  pedagogic  context  and 
its  structure. 

Two  other  clusters  of  variables  that  I wish  to  draw  attention  to  here  are  the  instructional  context  and  instructor 
moderation.  In  the  research  literature  on  computer  conferencing  there  is  often  a failure  to  emphasise  the 
somewhat  ideal  conditions  under  which  a significant  number  of  these  projects  have  been  undertaken.  Many 
studies,  as  noted  earlier,  have  been  carried  out  on  mature  and  highly  motivated  students,  frequently  undertaking 
courses  at  a distance.  The  sample  sizes  are  frequently  relatively  small  (e.g.,  McConnell  91;  Soby  91;  Warren  and 
Rada  88;  Mason,  91;  Heath  98).  Many  studies  have  been  funded  with  additional  financial  resources  that  have 
enabled  instructors  to  carry  out  such  projects  unburdened  by  the  teaching  and  administrative  constraints  that 
many  instructors  in  higher  education  institutions  have  to  contend  with.  The  instructional  context  in  which  the 
case  studies  described  below  were  carried  out  approximate  more  closely  the  conditions  described  by  Hiltz  as 
obtaining  at  NJIT,  namely,  “first  generation  college  students”,  who  must  work  while  attending  school  and  who 
commute  rather  than  live  on  campus,  who  are  faced  with  overcrowded  classes  and  lecture  halls,  and  who  are 
tutored  by  staff  who  are  themselves  under  severe  time  constraints. 

My  use  of  the  terminology  instructional  context  subsumes  the  whole  complex  of  pedagogical  related 
experiences  and  programmes  that  the  student  encounters  within  the  same  time  frame  that  includes  the  research 
project.  The  performance  of  students  in  a collaborative  group  environment,  whether  mediated  by  electronic 
platforms  or  not,  is  affected  by  the  way  in  which  that  experience  melds  with  the  pedagogical  approaches 
employed  in  other  courses  that  they  are  pursuing.  Many  of  the  presumed  benefits  currently  attributed  to 
collaborative  conferencing  and  other  work  might,  in  considerable  measure,  be  a consequence  of  a high  novelty 
factor,  something  that  I would  expect  to  be  substantially  dissipated  if  all  courses  were  undertaken  employing 
similar  approaches. 

As  far  as  computer  mediated  communications  based  on  conferencing  applications  are  concerned,  the  role  of  the 
moderator  has  frequently  been  taken  to  be  of  critical  importance.  (Brochet  89,  quoted  in  Paulsen  95;  Paulsen  95; 
Feenberg  89;  Mason  91;  Collins  and  Berge  99;  Berge;  Kaye91;  McConnell  91)  Numerous  roles  are  allocated  to 
the  moderator  in  the  literature,  most  of  which  are  deemed  to  be  important  to  the  success  of  conferencing 
outcomes.  The  more  effectively  implemented,  the  greater  the  benefits.  Paulsen,  who  reviews  the  literature,  lists 
15.  There  is  no  reason  to  assume  that  if  these  roles  are  effectively  adopted  and  implemented  that  students  will 
not  benefit.  However,  and  related  to  the  point  that  I made  earlier  concerning  the  relationship  between  the 
research  and  typical  instructional  context,  this  literature  also  stresses  the  amount  of  time  that  is  required  to  carry 
out  such  roles  effectively.  Mason  (91),  in  reviewing  one  such  study,  carried  out  under  the  auspices  of  the  UK 
Open  University,  noted  that  the  300  students  had  the  benefit  of  the  attention  of  16  tutors,  in  addition  to  a “super- 
tutof’  who  moderated  and  organized  the  conference,  “under  whom  the  other  tutors  made  their  own 
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contributions/*  This  ratio  of  1 to  18  is  somewhat  removed  from  that  of  1 to  95-120  that  applies  with  respect  to 
the  case  studies  that  1 discuss  below. 

Case  Studies 

Context: 

The  empirical  study  focuses  on  the  analysis  of  computer  conferencing  by  undergraduate  students  at  the 
University  of  the  West  of  England.  The  students  were  drawn  from  two  programmes.  Those  enrolled  in  the 
Faculty  of  Economics  and  Social  Science  and  undertaking  one  of  the  programmes  of  study  in  Social  Science  that 
it  offers,  and  students  who  were  taking  an  Inter-Faculty  mix  of  programmes,  one  of  the  two  strands  being  a 
programme  offered  by  the  Faculty  of  Economics  and  Social  Science. 

The  University  of  the  West  of  England  is  one  of  the  newer  UK  universities.  Located  in  Bristol,  in  south  west 
England,  it  currently  has  an  enrolment  of  some  28,000  students  undertaking  a wide  variety  of  degrees,  diplomas 
and  certificates,  ranging  form  sub-degree  vocational  courses  to  PhD  programmes.  It  has  a mix  of  full  and  part- 
time  students,  weighted  toward  the  former. 

The  students  who  are  the  subjects  of  this  study  were  all  undertaking  undergraduate  programmes,  the 
overwhelming  majority  of  whom  were  18  to  19  year-olds  who  entered  university  directly  after  the  completion  of 
their  secondary  education.  Hardly  any  of  them  had  had  any  prior  experience  of  using  a conferencing  system.  All 
students  undertaking  programmes  offered  by  the  Faculty  of  Economics  and  Social  Science  are  provided  with  the 
opportunity  to  attend  IT  induction  sessions,  which  include  familiarisation  with  the  use  of  FirstClass 
conferencing,  the  majority  of  whom  take  this  up. 

FirstClass  conferencing  is  widely  used  by  staff  in  the  Faculty  of  Economics  and  Social  Science,  principally  as  a 
broadcast  medium,  providing  information  to  students  on  administrative  matters  relevant  to  courses,  and  as  a 
repository  of  articles  and  lecture  notes  relating  to  specific  modules.  Other  than  in  the  modules  discussed  below, 
FirstClass  conferencing  in  the  Faculty  of  Economics  and  Social  Science  is  not  employed  in  interactive  mode,  nor 
are  conferencing  contributions  used  as  part  of  course  assessments. 

In  the  Faculty  of  Economics  and  Social  Science  the  academic  year  is  divided  into  three  trimesters.  In  practice 
only  the  first  two  are  teaching  terms,  the  summer  term  being  used  principally  for  examinations. 


Case  Studies  Course  Modules: 

The  students  whose  conferencing  work  forms  the  basis  of  the  discussion  and  analysis  below  were  taking  the 
following  course  modules.  The  term  assigned  work  below  refers  to  course  work  other  than  tests  or  examinations. 

(i)  Methods  of  Analysis.  A U*  year  undergraduate  module  that  all  students  studying  Social  Science 
programmes  are  required  to  take.  Its  aim  is  to  provide  students  with  critical  appraisal  of  qualitative  and 
quantitative  research  methods.  The  analysis  and  discussion  below  is  based  on  work  undertaken  by 
students  during  the  first  trimester  of  the  1999/2000  academic  year.  There  were  119  conferences.  The 
total  number  of  students  taking  this  course  in  the  first  term  of  1999/00  =420.  Comparison  was  made 
with  assigned  written,  non-conferencing,  work  carried  out  by  students  taking  this  course  in  1998/99,  the 
number  of  students  =290 

(ii)  Introduction  to  Social  Psychology.  This  is  a compulsory  module  for  those  majoring  in  Sociology 
but  optional  for  all  those  taking  other  Social  Science  programmes.  It  can  be  taken  by  and  3*^  year 
students.  Analysis  is  based  on  assigned  work  undertaken  during  the  first  trimester  of  1998/1999  N=61, 
divided  into  19  conferences,  membership  ranging  from  3 to  6,  1 1 of  which  had  4 members,  one  with  6. 
Comparison  was  made  with  performance  on  assigned  and  examination  work  on  other  modules  taken  by 
the  students,  excluding  (Hi)  below: 

(iii)  The  Holocaust  and  Other  Massacres.  This  is  a compulsoiy  module  for  those  majoring  in 
Sociology  but  optional  for  all  those  taking  other  Social  Science  programmes.  It  can  be  taken  by  and 


3^^^  year  students.  Analysis  based  on  those  taking  this  course  in  1998/99  divided  into  22  conferences  of 
between  2 and  6 members  in  the  first  trimester  with  66  students,  and  22  of  between  1 and  5 in  the 
second,  also  with  66  students.  There  was  only  one  conference  with  1 member.  Most  conferences  had  a 
membership  of  between  3 and  5,  relatively  evenly  distributed  in  both  trimesters.  Comparison  was  made 
with  performance  on  assigned  work  and  overall  results  on  other  modules  taken  by  students,  excluding 
(ii)  above. 

Students  were  posed  essay  type  questions  relating  to  the  subject  matter  of  the  courses  that  they  were  undertaking 
and  were  required  to  discuss  this  through  the  medium  of  the  FirstClass  conferencing  system.  Because  of  the 
large  numbers  of  students  involved,  and  student  commitments  and  time  constraints,  very  few  of  these  students 
met  face  to  face  in  connection  with  the  fulfilment  of  conferencing  course  requirements.  Students  were  allocated 
to  conferences  alphabetically,  or  on  the  basis  of  surname  after  they  had  chosen  an  assignment  topic.  They  were 
assessed  on  the  basis  of  the  quality  of  the  contributions  that  they  made  to  the  conferences,  and  written  work  that 
summarised  their  own  conclusions  concerning  the  essay  questions.  Students  were  given  detailed  information 
concerning  what  was  expected  from  them  as  far  as  the  volume  of  contributions  they  should  make,  when  these 
should  be  made,  the  range  of  issues  that  should  be  covered  (e.g.,  methodological  issues,  relevant  theoretical 
frameworks,  status  of  empirical  data),  and,  in  the  case  of  course  (i)  above,  requirements  concerning  alternating 
turn-taking  in  assuming  a lead  and  reactive/interactive  contributing  role.  All  students  were  given  the 
opportunity  of  having  30  minutes  training  using  the  conferencing  system. 

The  overall  course  grades  for  (ii)  and  (iii)  above  are  made  on  the  basis  of  the  assigned  work  and  a final 
examination,  each  contributing  50  percent  to  the  total.  The  assigned  work  was  a mix  of  conferencing 
contributions  and  written  work  related  to  the  question  posed  for  conferencing  contributions.  All  other  modules 
are  also  assessed  on  a combination  of  written  coursework  and  final  examinations,  each  contributing,  with  few 
exceptions,  fifty  percent  of  the  total.  None  of  these  other  courses  use  conferencing  for  assessment  purposes, 
although  instructors  on  many  courses  employ  the  conferencing  system  as  a broadcast  medium.  Students  taking 
these  courses  take  a total  of  six  courses  per  year,  with  the  exception  of  the  very  few  part-time  students.  The 
performance  of  the  students  on  the  conferencing  courses  was  compared  with  their  performance  on  the  other 
courses  that  they  took  in  the  same  academic  year.  The  comparison  was  with  the  average  performance  on 
coursework  grades,  examination  grades,  and  the  overall  average  of  coursework  and  examination  grades  on  these 
other  courses.  In  most  cases  these  were  averages  of  performance  on  five  courses.  However,  in  some  instances 
the  comparison  was  confined  to  a comparison  of  performance  on  the  conferencing  related  module  with 
performance  on  four  other  courses,  as  some  students  opted  to  study  both  courses  in  the  same  academic  year. 

As  far  as  the  methods  course  was  concerned,  (i)  above,  comparison  was  made  between  the  assessed  work  on  the 
conferencing  system  of  students  in  the  first  term  of  the  1 999/00  academic  year,  with  that  of  written  coursework 
undertaken  by  students  taking  the  same  course  in  the  first  term  of  the  academic  year  1998/99.  These  latter 
students  did  not  use  the  conferencing  system  for  assessed  work  in  any  courses  during  that  year. 

Although  I designed  the  essay  questions,  wrote  the  guidelines  relating  to  conferencing  work  requirements,  and 
administered  the  conferencing  system,  the  assessment  of  conferencing  contributions,  written,  and  examination 
work,  was  undertaken  entirely  independently  by  the  course  seminar  instructors.  The  methods  course  had  10 
seminar  instructors,  whereas  (ii)  and  (iii)  had  one  each.  None  of  the  conferences  were  moderated.  None  of  the 
instructors  made  any  contributions  to  the  conferencing  discussions,  with  a few  exceptions  when  it  was  necessary 
to  intervene  for  administrative  purposes,  for  instance,  informing  students  that  they  were  sending  the  message  to 
the  parent  conference  rather  than  to  the  project  group  conference.  Consequently,  students  undertook  all 
conferencing  work  independently. 


Findings 

With  respect  to  modules,  (ii)  and  (iii)  student  t-Tests,  paired  two  samples  for  means,  were  carried  out  on  a 
number  of  data  series,  the  Null  Hypothesis  being  that  there  was  no  difference  between  the  means  attained  on 
conferencing  work  modules  and  the  averaged  performance  of  students  on  all  other  modules.  For  module  (i)  the  t- 
Test  two  sample  assuming  unequal  variance  was  employed.  This  was  undertaken  separately  for  the  assigned 
coursework  and  examination  grades.  No  significant  difference  was  found  at  the  .01  level  in  respect  of  (iii)  for 
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the  second  term.  There  was,  however,  a significant  difference  for  (iii)  in  the  first  term,  the  mean  for  coursework 
on  the  conferencing  module  being  3%  higher  than  for  the  average  for  all  other  modules.  There  was  a 6.7% 
difference  between  the  final  mark  on  the  conferencing  module  and  the  overall  final  mark,  for  (iii)  in  the  first 
term.  Significant  differences  persisted  when  the  tests  were  carried  out  separately  for  males  and  females. 
However,  for  (iii),  term  two,  no  significant  differences  were  found  respecting  any  of  the  data  series,  although  the 
means  for  performance  on  all  of  the  data  series  relating  to  the  conferencing  modules  were  higher  than  those  with 
which  they  were  compared.  The  differences  between  terms  (i)  and  (ii)  could  conceivably  be  interpreted  as  being 
a product  of  the  wearing  off  of  the  novelty  effect,  but  many  other  explanations  are  equally  plausible. 

For  module  (ii),  the  only  significant  difference  was  between  the  means  for  the  overall  grade  for  the  module  and 
the  average  overall  grade  on  other  modules,  the  mean  for  the  former  being  nearly  6%  higher.  The  means  in  all 
other  comparison  were  higher  for  the  conferencing  module. 

For  the  methods  course,  (i)  above,  the  t-Test  between  the  written  assignment  work  for  the  1998/99  and  the 
1999/00  conferencing  work  was  significantly  different  at  the  .01  level,  the  means  differing  by  4.2%,  being 
higher  for  the  conferencing  work.  The  t-Test  between  the  1998/99  and  the  1999/00  wntten  work  was  not 
significantly  different  at  the  0.05  level,  the  mean  for  the  former  being  higher. 


Case  Studies  Conclusions 

The  analysis,  along  with  mixed  effects  regression  and  correlation  analysis,  not  reported  upon  here,  indicate  that 
conferencing  undertaken  without  the  benefit  of  moderation  is  not  reflected  in  poorer  performance,  as  measured 
by  grades  in  written  and  examination  work,  compared  with  courses  using  traditional  higher  education 
instructional  modes.  In  fact,  students  performed  better  on  average  on  two  of  the  conferencing  modules  (ii,  iii), 
than  they  did  on  other  coursers.  There  are  various  reasons  that  can  be  advanced  for  this.  As  noted  earlier,  as  the 
pedagogic  context  is  one  in  which  conferencing  work  is  a novelty  on  the  case  study  courses,  this  may  impact  on 
student  interest,  perseverance  and  motivation.  Another  possible  explanation  is  the  impact  of  what  social 
psychologists  refer  to  as  social  facilitation.  It  is  also  possible  that  the  subject  matter  of  the  assigned  conferencing 
work  is  perceived  by  students  to  be  more  interesting  than  that  than  which  they  are  required  to  undertake  on  other 
courses.  It  is  also  plausible  to  argue  that  many  of  the  benefits  singled  out  by  other  researchers  as  being 
associated  with  collaborative  work,  are  also  active  in  the  minimalist  intervention  computer  conferencing 
contexts  that  applied  in  the  present  case  studies. 


Summary  and  Conclusion 

Although  there  is  now  a substantial  body  of  literature  dealing  with  varied  matters  relating  to  the  deployment  of 
interactive  communication  technologies  in  educational  settings,  we  are  still  a long  way  from  knowing  or 
understanding  how  to  deploy  them  effectively  in  order  to  achieve  pedagogic  outcomes.  This  is  hardly  surprising 
given  that  disputes  concerning  the  effectiveness  of  more  traditional  pedagogic  mediums,  such  as  lectures  and 
seminars,  are  far  from  being  settled.  Some  of  the  reasons  are  not  difficult  to  track.  Much  research  has  been 
anecdotal,  case  study  oriented,  findings  being  applicable  to  very  specific  contexts,  and  therefore  non-replicable. 
Much  research  has  been  carried  out  under  conditions  that  do  not  approximate  those  of  typical  educational 
settings.  Many  of  the  assertions  deployed  with  missionary  zeal  by  educational  technology  enthusiasts  have  not 
been  systematically  tested.  At  the  same  time,  it  is  possible  to  demonstrate  that  such  technologies  might  be 
effectively  deployed  under  less  than  ideal  conditions  to  achieve  modest  improvements. 
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Abstract:  Recorded  multimedia  presentations  in  the  form  of  lectures,  conference  proceedings, 
commercial  demonstrations  or  tutorials,  are  becoming  increasingly  available  in  multiple,  diverse, 
and  often  heterogeneous  formats.  These  materials  are  stored  in  databases  or  digital  libraries  where 
the  user  can  search  by  a variety  of  mechanisms,  ranging  from  keyword  search  to  content-based  que- 
ries. Tracking  human  expression  actions  is  a valuable  tool  for  mining  information  because  it  can 
add  significance  value  to  the  materials  being  searched.  In  this  paper,  we  focus  on  tracking  the  pre- 
senter’s pointing  activity  during  a lecture,  and  show  how  this  information  can  be  used  to  provide 
the  user  with  an  improved  overview  of  the  lecture  to  quickly  navigate  to  those  points  that  the  user 
might  find  the  most  interesting.  The  strength  of  this  approach  lies  in  the  ability  to  correlate  the 
pointing  actions  of  the  lecturer  with  other  streams  of  data  during  the  recorded  lectures,  such  as 
audio  amplitude,  gesture  motion,  slide  presentation,  or  text.  In  the  absence  of  complete  transcripts 
or  manually  added  annotations,  which  are  expensive  to  generate  given  the  quantity  of  recorded 
lectures,  the  level  and  type  of  interaction  of  the  presenter  with  the  material  can  be  analyzed  to  yield 
likely  points  of  interest. 


1.  Introduction 

As  classrooms  are  increasingly  equipped  with  presenting  and  recording  technology,  more  and  more  recorded 
lecture  material  becomes  digitally  available  for  both  local  and  remote  students.  Central  repositories  of  such  lectures 
are  valuable  assets  for  online  education  as  students  can  re-create  the  environment  of  the  lecture  hall  wherever, 
whenever,  and  however  often  they  need  to.  Yet  as  such  digital  libraries  grow  in  size,  the  problem  of  accessing  or 
retrieving  relevant  information  becomes  increasingly  pronounced.  It  is  helpful  to  locate  individual  segments  of  lec- 
tures dealing  with  a particular  aspect  of  a problem  in  order  to  view  these  segments  side  by  side,  rather  than  review- 
ing entire  lectures  sequentially.  When  reviewing  a single  lecture,  instead  of  providing  uniform  access  to  any  point, 
we  focus  on  providing  “relevance”  by  computing  points  of  interest  within  a lecture.  This  is  done  by  tracking  lecturer 
pointing  activity,  a new  data  stream  that  can  be  combined  with  other  captured  streams,  such  as  slides,  audio,  etc. 
Using  lecturer  pointing  activity  presents  a new  way  for  browsing/searching  large  digital  libraries  of  lectures  and  for 
data  mining  higher  level  information,  such  as  a lecture’s  “hot  spots”. 


2.  Related  Work 

In  computerized  classroom  projects  (e.g..  Classroom  2000  project  at  the  Georgia  Institute  for  Technology 
(Abowd  et  al.  1998)),  the  idea  is  to  capture  and  re-create  the  classroom  experience.  One  pitfall  is  to  store  informa- 
tion graphically,  not  symbolically,  which  makes  it  very  difficult  to  correlate  diverse  streams  of  data  occuring  during 
the  lecture,  and  thus  to  do  data  mining,  and  increases  the  work  necessary  to  analyze  the  content  of  lectures  to  add 
conceptual  structure.  Other  systems  are  similarly  based  on  the  idea  of  providing  sequential  lecture  material.  Barry 
Arons  from  the  MIT  Media  Lab  developed  SpeechSkimmer  (Arons  1997),  which  tries  to  find  the  important  parts  of 
speech  recordings  in  order  to  give  quick  overviews,  saving  reviewing  time.  Arons  evaluated  attributes  in  speech 
patterns  that  indicate  where  the  speaker  is  placing  emphasis.  We  translate  this  idea  to  the  classroom  recording:  in  our 
case,  the  protagonist  places  emphasis  by  interacting  with  the  presented  material,  literally  pointing  out  the  important 
parts.  At  the  Dartmouth  Experimental  Visualization  Laboratory  (DEVLAB),  earlier  projects  focused  on  making 
electronic  lectures  and  conference  proceedings  more  interactive  (Ford  et  al.  1998),  and  to  correlate  data  streams  of 
different  modalities  (Owen  et  al.  1999,  Owen  1998).  Currently,  the  DEVLAB  is  combining  this  previous  work  to 
extend  the  Authoring  on  the  Fly  (AOF)  system  (Mueller  et  al.  to  appear)  developed  at  the  University  of  Freiburg, 
Germany.  Because  of  its  capability  to  store  lecture  data  streams  symbolically,  the  AOF  system  is  used  by  six  major 
universities  in  Germany  sharing  lectures  in  real  time  via  the  Internet,  and  the  University  of  Freiburg  is  creating  an 


online  library  of  these  lectures. 

2.1  Processing  parallel  data  streams  during  lectures 

In  order  to  extract  higher  level  information  about  a lecture,  it  is  necessary  to  correlate  multiple  data  streams 
such  as,  speech,  video,  speaker’s  facial  expression,  speaker’s  head  motion,  and  other  data  streams.  Tracking  the  cor- 
relation between  heterogeneous  data  streams  allows  us  to  perform  data  mining  and  determine,  for  example,  where 
the  highest  pointing  activity  occurred  during  a slide  presentation.  Computing  the  synchronization  between  diverse 
data  streams  is  the  work  described  in  the  book  by  Owen  and  Makedon  (Owen  et  al.  1999).  We  demonstrate  this  ap- 
proach with  the  AOF  (Authoring  on  the  Fly)  System  (Mueller  et  al.  to  appear)  which  already  captures  diverse  data 
streams  in  synchronized  mode.  The  AOF  system  is  used  as  an  intermediate  step  on  which  to  build  the  theory  of 
multiple  data  stream  tracking,  and  has  been  chosen  for  two  major  reasons:  (a)  it  provides  a symbolic  representation 
of  the  data  contained  within  a recorded  presentation;  and  (b)  it  is  easily  extensible  via  helper  applications. 

The  AOF  system  has  a whiteboard  (AOFwb)  which  is  the  electronic  substitute  of  a blackboard.  Lectures  are 
created  in  two  phases:  (a)  In  the  pre-processing  phase,  the  instructor  uses  AOFwb  to  produce  slides  to  present  in 
class.  The  result  of  the  editing  process  is  a presentation  file  with  a page  structure  based  on  SGML,  (b)  In  the  pres- 
entation and  recording  phase,  the  presentation  file  is  loaded  into  AOFwb  and  used  by  the  instructor  to  present,  an- 
notate, and  orally  explain  the  prepared  material,  i.e.  the  presentation  file  together  with  other  material  such  as 
animations,  simulations,  audio  or  video  files,  etc.  Annotations  are  made  on  a transparent  overlay  on  top  of  the 
loaded  page,  using  the  same  graphic  and  text  manipulation  tools  available  during  authoring. 

While  the  instructor  is  presenting  the  material,  the  recording  facility  in  AOFwb  captures  the  important  data 
streams  generated:  the  audio  stream  with  the  instructor’s  voice,  the  graphics  stream  of  time-stamped  graphics  dis- 
played during  the  presentation,  the  external  applications  stream,  any  video  streams  (of  the  instructor  or  audience), 
and  any  others  depending  on  the  presentation.  At  the  end  of  the  presentation,  a post-processing  step  builds  a self- 
contained  lecture  represented  by  an  HTML  overview  with  links  between  the  presented  slides  and  associated  replay 
locations.  The  resulting  AOF  document  can  be  replayed  synchronously,  i.e.  the  relation  between  the  various  cap- 
tured data  streams  is  maintained.  (See  (Mueller  et  al.  to  appear)  for  details.)  The  most  attractive  features  of  the  AOF 
replay  component  are  the  random  visible  scrolling  facility  and  the  case  with  which  AOF  documents  can  be  inte- 
grated into  web-based  teaching  and  learning  environments. 

AOF’s  random  visible  scrolling  facility  permits  fast  browsing  of  a recorded  lecture,  similar  to  flipping  through  a 
book.  Apart  from  this,  however,  the  current  AOF  system  does  not  have  tools  for  retrieval  of  lecture  content  by  im- 
portance of  information  or  by  lecturer  activity.  In  other  words,  it  is  not  possible  for  a user  to  query  a database  of 
AOF  lectures  for,  say,  all  slides  where  certain  phrases  appear  or  where  there  are  “important”  parts  for  a given  topic. 
In  this  paper  we  report  on  ongoing  work  that  aims  to  enable  queries  of  what  is  important  in  a lecture  and  what  is  not. 
We  do  this  by  observing  that  AOF  records  a great  deal  of  information  which  is  not  utilized  to  its  full  extent.  This 
information  is  valuable  for  supporting  the  retrieval  of  what  is  important  content  in  a lecture  and  in  segmenting  the 
lecture  (giving  it  conceptual  structure)  into  parts  of  high,  average,  and  low  importance. 

Using  the  AOF  recording  process,  the  respective  data  streams  have  already  been  aligned  (e.g.,  slide-to-audio, 
application-to-audio,  and  video-to-audio  alignments).  This  helps  us  use  text  queries  to  search  for  lecture  parts  be- 
cause, as  we  showed  in  earlier  work,  we  can  do  “cross  modal  information  retrieval,”  i.e.,  search  for  information 
across  modalities.  For  example,  we  can  find  the  words  spoken  when  high  pointing  activity  takes  place.  Since  the  text 
and  formatting  information  on  all  slides  is  available,  and  we  know  exactly  when  and  where  each  annotation  was 
made  by  the  instructor  during  the  presentation,  we  can  use  this  information  for  retrieval  either  within  one  document 
or  across  many  documents.  Thus,  a large  repository  of  AOF  documents  becomes  accessible  via  text  queries.  The 
HTML  document  automatically  generated  is  not  hierarchically  structured  but  just  a flat  index.  By  slightly  extending 
the  AOF  system  in  the  pre-  and  the  post-processing  phase,  a better  clustering  and  structuring  of  information  that 
enhances  retrieval  is  possible.  This  leads  to  database  and  data  mining  facilities  where  lecture  data  streams  are  seg- 
mented and  stored  accordingly. 


3.  Tracking  Lecture  Activity:  Why  and  How 

Currently,  it  is  very  difficult  to  search  for  certain  parts  of  recorded  presentations  (lectures,  conference  proceed- 
ings, commercial  demonstrations  or  tutorials)  that  are  stored  in  heterogeneous  formats  and  in  distributed  databases. 
Users  can  apply  a variety  of  tools,  ranging  from  keyword  search,  to  content  based  queries  but  this  will  not  give  them 
results  to  “what  is  important.”  We  introduce  a new  “importance  mining”  framework,  which  works  by  tracking  and 
correlating  the  different  data  streams  generated  by  a lecturer  during  a lecture.  We  focus  on  a lecturer’s  pointing  ac- 
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tivity,  such  as  pointing  to  items  on  a slide.  Instead  of  searching  by  keywords,  we  introduce  the  idea  of  searching  a 
digital  library  by  a certain  human  activity.  This  has  significant  implications  in  the  management  of  complex  knowl- 
edge in  future  large-scale  digital  libraries  of  human  presentations.  For  example,  one  can  search  for  all  places  where 
the  lecturer  exhibited  pointing  activity  over  a certain  threshold  and  a certain  set  of  keywords  appeared. 

Assuming  that  locations  of  high  interest  have  high  pointing  activity,  then  tracking  pointing  activity  is  an  effec- 
tive search  “modality  stream”  in  locating  complex  human  information.  If  the  pointing  activity  stream  is  the  primary 
search  stream,  then  we  have  a simple  way  to  derive  correlation  between  lecture  structure  and  content  based  on  the 
human  tracking  of  the  speaker.  Here,  we  employ  the  built-in  synchronization  between  the  audio  and  the  slide-based 
human  actions  provided  by  the  AOF  system.  This  is  a common-sense  mechanism  of  deriving  “hot  spots”  during  a 
lecture.  Our  system  generates  an  activity  profile  for  a slide,  a lecture,  or  a set  of  lectures.  Visualization  graphs  are 
automatically  derived,  thus  providing  an  intuitive  and  easy  to  use  interface  for  the  digital  library  user. 

Capturing  lectures  by  merely  recording  the  video  and  audio  streams  is  sufficient  to  deliver  the  information  pre- 
sented, but  not  for  finding  higher-level  information  about  the  importance  of  a lecture,  especially  if  one  wants  an  an- 
swer in  a practical  amount  of  time.  Video  processing  is  hindered  by  limitations  in  the  image  resolution  and  quality: 
discovering  what  a lecturer  marked  or  what  information  was  presented  on  a slide  is  very  difficult  and  expensive  to 
do  by  computer.  On  the  other  hand,  using  an  authoring/presentation/recording  tool  such  as  AOF,  one  combines 
preparation  of  lecture  material  with  the  recording  facility,  thus  incorporating  “access  markers”  for  finding  lecture 
significance.  AOF  delivers  not  just  the  images  of  the  captured  lecture  but  also  stores  the  lecturer’s  actions  during  the 
presentation.  With  this  information,  it  is  relatively  simple  to  deduce  the  interaction  between  lecturer  and  presented 
material,  and  thus  to  compute  “pointing  activity”  for  each  segment  of  the  lecture. 

3.1  Determining  lecture  importance 

The  basic  premise  is  that  the  level  of  interaction  of  the  lecturer  with  the  material  being  presented  is  an  indica- 
tion as  to  how  relevant  the  current  point  in  time  is  within  the  whole  lecture.  A lot  of  interaction  means,  in  all  likeli- 
hood, that  the  material  is  very  relevant,  as  it  augments  the  point  that  the  lecturer  is  currently  trying  to  convey  to  the 
audience.  The  more  often  or  persistently  a given  element  is,  quite  literally,  pointed  out  to  the  audience,  the  more 
important  this  element  is  for  understanding  the  information.  On  the  other  hand,  no  interaction  is  likely  to  indicate 
that  the  lecturer  is  giving  secondary  information,  such  as  background  or  related  material,  that  has  no  direct  relation 
to  the  current  slide,  or  that  the  lecturer  is  merely  glossing  over  the  material  in  order  to  move  on  to  more  important 
things. 

Our  approach  allows  a user  to  search  for  interesting  points  in  a lecture  or  set  of  lectures  because  it  can  combine 
results  from  the  following:  (a)  the  value  of  lecturer  pointing  (pointing  activity  that  a lecturer  exerts  on  a slide  or 
view  graph);  (b)  the  value  from  additional  parallel  lecturer-generated  data  streams:  voice  pitch,  frequency,  ampli- 
tude, head  motion,  eye  direction,  etc.;  (c)  the  value  derived  from  applying  data  mining  algorithms.  The  results  can  be 
visualized  and  build  what  we  call  a lecture  architecture,  showing  the  peaks  and  valleys  of  lecturer  pointing  activity. 
Once  a lecture  has  been  processed,  a digital  library  user  may  wish  to  search  for  all  the  sorting  lecture  slides  that  have 
high  pointing  activity  and  also  exhibit  audio  amplitude  within  a certain  range. 

The  fact  that  the  system  is  slide-based  is  a conscious  decision:  the  use  of  slides  is  a natural  segmentation  of  a 
multimedia  information  stream  (the  lecture):  it  assumes  that  the  material  is  presented  via  slides  or  similar  media 
which  contain  enough  information  to  automatically  structure  the  lecture.  In  other  words,  the  slides  provide  a form  of 
segmented  presentation,  each  being  the  visual  encapsulation  of  a richer  textual,  audio  or  video  content.  At  the  lowest 
level,  each  new  slide  indicates  a new  segment;  by  adding  section  numbers  to  each  slide,  multiple  slides  can  be 
grouped  into  sections  or  chapters.  Using  slides  as  “visual  information  holders”  is  not  a new  idea  and  is  widely  used 
on  the  web  (thumbnails).  However,  what  we  propose  here  is  an  extension  of  this  idea  to  include  thumbnail  hot  spots. 

The  system  we  are  developing  does  this  automatically  for  AOF,  which  provides  the  infrastructure  for  correlat- 
ing lecture  structure.  Most  importantly,  since  the  correlation  information  is  stored  symbolically,  it  can  be  shared 
over  the  web  and  thus  the  system  has  particular  application  to  distance  learning  systems  using  digital  library  tech- 
nologies. 

Since  the  assumption  is  that  intensive  pointing  indicates  important  information,  we  measure  various  parameters 
including:  duration  of  pointing,  locality  (relative  location  of  pointing  activity  to  previous/following  pointing  activi- 
ties), repetition,  frequency,  related  text  material,  and  attributes  of  the  corresponding  audio.  These  five  parameters  are 
assigned  weights  and  the  pointing  activity  is  then  ranked  and  visualized.  Another  facility  may  collect  statistical  data 
of  these  actions  which  are  then  mined  for  the  benefit  of  the  student. 
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Slide 

Content 

% Time 

% Activity 

1 

Presentation  Outline 

4.7% 

0.0% 

2 

Problem  Statement 

5.3% 

1.6% 

3 

Some  Observations 

7.0% 

2.0% 

4 

A Useful  Theorem 

21.2% 

26.7% 

5 

Next  Fit  Algorithm 

9.4% 

7.2% 

6 

7.4% 

4.0% 

7 

First  Fit  Algorithm 

13.6% 

24.7% 

8 

6.9% 

0.8% 

9 

Best  Fit  Algorithm 

4.4% 

4.4% 

10 

Off-line  Algorithms 

6.9% 

0.0% 

11 

3.8% 

10.0% 

12 

Conclusions 

7.7% 

17.1% 

13 

1.7% 

1.6% 

Table  1.  Slides  of  the  Binpacking  presentation. 


4.  Pointer  Tracking 


□ Percentage  of  Spending  Time 
Q Percentage  of  Pointer  Activity 


Figure  1.  Percentage  comparison  of  spending  time  and  pointer  acti 
among  slides  in  Binpacking  presentation.  The  X-axis  shows  the  sli 
numbers  (cf.  Table  1). 


During  a presentation  involving  slides,  the  lecturer  can  focus  the  attention  of  the  audience  to  a slide  element  by 
placing  emphasis  on  it.  In  AOF,  this  is  usually  done  by  using  a telepointer  or  by  drawing  onto  the  slide,  such  as  un- 
derlining a word,  connecting  items  with  a stroke,  or  similar  marks.  In  either  case,  there  is  interaction  between  the 
lecturer  and  the  presented  material,  and  the  more  activity  the  lecturer  devotes  to  a certain  element,  the  more  relevant 
that  element  becomes  within  the  context  of  the  lecture.  To  measure  the  relevance  of  a slide  element,  we  therefore 
have  to  measure  the  interaction  between  the  lecturer  and  a given  element. 

For  example,  we  have  done  the  telepointer  tracking  to  a lecture  on  Binpacking  by  Prof.  Ottmann.  Its  length  is 
about  44  minutes.  Table  1 shows  the  titles  for  the  slides  in  this  presentation.  Figure  1 shows  the  comparison  result 
among  slides  regarding  their  spending  time  and  telepointer  activity.  From  this  result,  we  can  see  that  slides  4 and  7 
contain  high  levels  of  interaction.  They  turn  out  to  be  key  slides  in  the  presentation.  Clearly,  the  telepointer  tracking 
result  can  help  retrieval  system  find  interesting  information. 

Depending  on  the  note-taking  system,  the  lecturer  can  have  a variety  of  tools  that  can  be  used  to  mark  slide 
elements.  In  the  following,  we  classify  these  tools  by  their  impact  on  the  presentation.  We  then  describe  classes  of 
actions  that  pointer  tools  can  perform.  Next,  we  view  these  actions  in  context  to  one  another,  i.e.  to  distinguish  dif- 
ferent patterns  of  actions.  Finally,  we  consider  how  the  slide  is  structured;  interacting  with  a slide  that  consists  of 
text  needs  to  be  treated  differently  from  interaction  with  slides  consisting  of  graphics. 

4.1  Pointer  types 

One  can  assign  different  weights  of  significance  to  different  pointing  actions  during  a lecture.  We  distinguish 
between  the  following  types  of  pointers  that  a lecturer  can  use  during  a presentation  to  indicate  areas  of  interest. 
These  pointers  are  tools  available  in  AOF  but  are  general  enough  to  apply  to  similar  note-taking  systems.  The  pri- 
mary classification  is  by  impact  on  the  presentation:  the  arrow  (or  telepointer)  is  a non- intrusive  tool,  meaning  that  it 
leaves  no  trace  on  the  slide,  but  also  that  its  action  has  a time  element;  whereas  the  freehand  tool  leaves  a quasi- 
permanent line  on  the  slide,  meaning  that  the  only  available  time  information  is  the  appearing  point  of  the  line. 

Arrow/Telepointer:  a non-intrusive  and  easily  visible  pointer  that  can  either  indicate  a single  locationor  be  used 
to  trace  along  slide  elements,  e.g.  to  underline  a word  as  it  is  spoken.  Freehand  (drawing)  tool:  similar  to  the  arrow, 
but  leaving  a colored  line  on  the  slide  for  a more  permanent  effect.  The  freehand  tool  can  also  be  used  to  add  infor- 
mation. Line  tool:  usually  used  to  underline  or  mark  slide  elements,  or  to  show  direct  connections  between  slide 
elements.  Rectangle  tool:  outlines  a word  or  a block  of  text,  such  as  a definition  of  a term,  or  a more  general  object. 
Move  tool:  this  tool  modifies  the  slide  by  moving  existing  slide  elements  around,  either  to  rearrange  the  slide  (e.g.  to 
simulate  animated  effects),  or  to  move  an  indicator,  such  as  a rectangle. 

What  all  these  tools  have  in  common  is  that  they  are  used  to  direct  the  attention  of  the  viewer  to  a particular 
section  of  slide  in  one  way  or  another,  which  implies  that  the  current  point  of  interest  is  on  the  slide.  Conversely,  by 
locating  a pointer  action  linked  to  a specific  topic  in  a presentation,  such  as  a query  word,  we  have  found  a section 
of  the  presentation  that  is  relevant  to  this  topic.  Analyzing  the  pointer  action  yields  an  estimate  of  the  relevance  of 
the  section:  the  more  the  lecturer  concentrates  on  a slide  element,  the  more  relevant  it  becomes.  In  this  context,  the 
type  of  the  pointer  bears  significance:  (a)  Using  the  arrow  may  indicate  a passing  reference,  as  the  highlighted  ele- 
ment is  no  longer  standing  out  once  the  lecturer  moves  on.  Also,  repeated  highlighting  increases  relevance  (see 
Pointer  actions,  below),  (b)  Drawing  and  moving  tools  have  a more  persisting  effect,  which  translates  into  a higher 
relevance.  Unlike  the  arrow,  however,  it  is  difficult  to  measure  the  amount  of  time  devoted  to  the  highlighted  ele- 


ment,  meaning  that  taking  successive  actions  becomes  more  important.  The  amount  of  drawing  can  also  be  consid- 
ered a factor. 

4.2  Pointer  actions 

As  described  above,  segments  of  a lecture  or  presentation  where  the  lecturer  interacts  with  the  material  can  be 
considered  more  relevant  than  the  segments  where  there  is  no  such  interaction.  Rather  than  simply  assigning  rele- 
vance for  segments  with  interaction,  however,  we  wish  to  refine  the  measurement  of  interaction.  We  thus  classify 
pointing  actions  using  the  following  attributes:  (1)  duration  of  interaction,  (2)  repetition  of  (same  or  similar)  action, 
(3)  location  of  action,  (5)  text  in  vicinity  of  action,  and  (5)  speech  audio  levels  and  speech/facial  motion  attributes. 
Some  of  these  attributes  are  explained  in  detail  below. 

Duration  of  interaction.  TThe  relevance  of  a slide  element  increases  primarily  with  the  time  the  lecturer  spends 
on  this  element.  The  measurement  of  the  time  is  straightforward  in  case  of  the  telepointer,  as  the  lecturer  has  to  keep 
the  telepointer  activated  (e.g.  by  keeping  the  mouse  button  depressed).  Depending  on  whether  and  how  much  the 
pointer  is  moved  during  this  time,  the  attribute  can  be  either  limited  to  a single  object  (on  which  the  lecturer  is  fo- 
cussing the  whole  time),  or  shared  among  the  objects  covered  during  the  action. 

The  Move  tool  can  be  viewed  as  a hybrid  case  between  the  telepointer  and  a drawing  tool:  the  action  is  limited 
by  the  time  the  lecturer  activates  the  tool,  but  its  effects  are  quasi-permanent,  similar  to  the  drawing  tools.  However, 
the  basic  usage  of  the  Move  tool  more  closely  resembles  a drawing  tool  in  that  the  end  result  is  the  desired  effect,  so 
evaluation  for  the  Move  tool  follows  that  of  the  drawing  tools. 

Repetition  of  actions.  As  mentioned  in  the  previous  section,  repeated  focus  on  a single  element  enlarges  that 
element’s  significance  for  the  current  point  in  time.  In  particular,  the  number  of  actions  associated  with  one  element 
on  a slide  can  indicate  whether  it  is  of  central  focus  or  “just”  a member  of  a list  of  items  being  presented. 

Location  of  actions.  (Rus  et  al.  1995)  uses  the  structure  of  a document  to  classify  the  type  of  each  component. 
Thus,  a word  appearing  in  the  title  of  a slide  has  more  weight  than  a word  appearing  in  an  indented  sentence  at 
regular  or  reduced  size.  We  invert  this  indication  when  it  comes  to  activation,  as  a targeted  element  which  was  in  an 
already  elevated  position,  such  as  the  slide  title,  is  not  made  much  more  significant  when  highlighted. 

Neighboring  text.  As  a general  rule,  words  that  are  pointed  out  gain  more  significance  if  they  were  equal  to  their 
neighbors  before.  For  instance,  highlighting  a word  buried  in  a paragraph  increases  its  significance;  whereas  point- 
ing at  a single  word  on  a slide  does  not  change  the  fact  that  it  is  already  the  most  significant  word  on  this  slide! 

Audio.  By  correlating  the  pointing  action  to  the  audio  stream,  we  can  see  whether  emphasized  speech  coincides 
with  pointing  action,  indicating  a higher  relevance  than  for  regular  speech. 

4.3  Patterns  of  actions  and  data  mining 

There  are  two  basic  kinds  of  patterns:  spatial  and  chronological.  The  former  include  basic  movements  of  the 
pointing  action  on  slides,  e.g.  top-to-bottom  (indicating  a list  of  items)  or  a star  pattern  (central  item  with  secondary 
items  around  it).  The  timing  between  pointing  actions  can  similarly  indicate  the  relationship  between  the  elements 
pointed  at.  One  interesting  question  here  is  if  there  are  “typical”  patterns  in  presenting  that  can  be  used  to  build  up  a 
library  of  such  patterns,  either  for  generic  or  for  lecturer-specific  pattern  recognition.  We  can  use  data  mining  tech- 
niques to  find  such  kinds  of  patterns.  Our  objective  is  to  build  a model  for  a classifying  attribute,  using  algorithms 
like  SPRINT  (Shafer  et  al.  1996),  to  classify  interactions,  e.g.  into  spatial  or  chronological  patterns.  Besides  classi- 
fication, there  are  other  interesting  mining  issues  (Fayyad  et  al.  1996)  (e.g.  associations,  sequential  patterns)  for  ex- 
tracting knowledge  for  a large  digital  library  of  presentations  that  remain  to  be  explored  in  depth. 

4.4  Slide  content:  text  vs.  graphics 

In  the  absence  of  a transcript  or  structural  information  on  the  lecture  or  presentation,  the  evaluation  of  the  pre- 
senter’s interaction  is  limited  to  what  elements  are  contained  in  each  slide.  As  words  can  be  viewed  as  forming  indi- 
vidual objects,  assigning  a pointer  action  to  a highlighted  word  is  simpler  and  more  concrete  than  connecting  the 
action  to  an  element  of  a graphic.  Furthermore,  in  the  case  of  automated  retrieval  of  lecture  material  (cf  section  6 
below),  we  need  to  be  able  to  associate  a slide  containing  only  graphics  with  textual  information  in  order  to  compare 
it  to  the  query  terms.  We  plan  on  developing  methods  by  which  we  can  use  the  surrounding  slides  with  text  to  glean 
what  connotation  the  graphics  may  have.  Even  in  the  absence  of  text,  it  is  important  to  give  slides  with  only  graphics 
“credit”  for  interaction,  as  graphics  are  useful  or  used  to  illustrate  points  that  the  lecturer  wants  to  convey. 


5.  Representation  of  Lecture  Activity  and  Retrieval  Applications 

Traditional  video  presentation  libraries  represent  a video  clip  with  an  icon  and  a caption  for  identification.  The 
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icon  is  either  a generic  video  symbol  or  a representative  frame  of  the  video  clip,  while  the  caption  is  usually  the  file- 
name. When  the  user  selects  the  video  clip,  it  is  played  back  from  beginning  to  end.  To  control  playback,  the  user  is 
presented  with  a generic  progress  bar  or  similar  timescale,  and  VCR-like  controls.  While  this  model  is  appropriate 
for  short  clips,  it  is  lacking  for  videos  of  entire  lectures  or  presentations  of  comparable  length  when  the  user  is 
searching  for  specific  information.  A first,  simplistic  segmentation  of  a lecture  would  split  it  along  slide  changes:  the 
user  can  now  select  a slide  and  find  out  whether  the  requested  information  is  contained  on  this  slide.  If  it  is,  the  vid- 
eoclip is  started  at  the  corresponding  position  and  the  user  reviews  the  lecture  from  this  point  on. 

An  even  finer  segmentation  of  the  lecture  takes  into  account  where  the  presenter  interacted  with  the  presented 
material,  using  this  as  an  indication  of  when  main  points  are  discussed.  Presenters  tend  to  indicate  an  element  on  the 
slide  as  they  begin  talking  about  it,  so  by  jumping  to  such  a point  in  the  lecture  gives  users  a faster  access  to  the 
main  points.  Once  such  a main  point  is  found,  the  traditional  controls  can  be  used  to  rewind  if  necessary;  depending 
on  the  style  of  the  presenter,  the  actual  beginning  of  a main  point,  in  relation  to  the  interaction  with  the  slide,  can 
vary  significantly,  and  it  is  not  clearly  understood  how  to  universally  locate  such  beginnings.  A corresponding  user 
interface  can  feature  either  icons  along  a time  scale  to  indicate  overall  or  individual  actions,  or  a graph-based  track 
in  which  interaction  is  quantized,  similar  to  the  volume  of  the  audio  track. 


6.  Conclusion 

We  have  presented  a framework  for  improving  browsing  and  querying  digital  libraries  of  recorded  presentations 
which  are  not  just  audio  and  video  data  streams  but  also  capture  the  interaction  with  the  presented  source  material. 
By  taking  this  additional  information  into  account,  lectures  and  presentations  are  enriched  by  an  evaluation  of  their 
relevance  at  any  given  point,  both  in  form  of  a contour  spanning  the  entire  document  as  well  as  a comparison  to 
other  documents  within  the  context  of  a user  query.  Since  lecturers  and  presenters  have  varying  styles  of  presenta- 
tion and  form  of  interactivity  with  their  material,  our  approach  cannot  universally  provide  good  results:  a lecturer 
who  in  his  presentation  does  not  discriminate  between  important  and  peripheral  material,  may  even  see  skewed  re- 
sults as  peripheral  elements  are  given  undue  significance.  However,  a lecturer  who  takes  care  to  place  emphasis  on 
important  elements  and,  quite  literally,  points  these  out,  will  see  more  focus  on  the  primary  segments,  which  allows 
students  to  be  more  likely  to  get  to  the  heart  of  the  matter  when  reviewing  the  information. 

We  plan  on  using  AOF  in  more  realistic  environments  to  learn  from  other  lecturers’  styles  and  incorporate  these 
findings  in  quantizing  different  kinds  of  interaction.  One  primary  concern  has  to  remain  the  system’s  low  profile 
during  a lecture  so  that  the  presenter  can  concentrate  on  the  material  and  the  reaction  of  the  audience. 
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Abstract:  This  paper  looks  at  two  action  research  projects  supported  by  ozTeacherNet  to  extend  the 
walls  of  three  Queensland  classrooms  to  the  Yasuura  education  district  in  Japan.  The  first  project 
extended  a Book  Rap  to  include  a cultural  comparison  of  the  construct  of  age.  It  involved  students  in 
all  four  communicative  modes  of  language  and  the  development  of  webpages.  The  second  project 
followed  the  visit  of  a Travel  Buddy  to  Japan  and  back  again. 


Introduction 

The  oz-Teachemet  is  an  on-line  community  facilitated  by  the  Research  into  Information  Technology  in 
Education  (RITE)  Group  at  the  Queensland  University  of  Technology,  Australia.  The  purpose  of  oz-Teachemet  is  to 
provide  an  infrastructure  for  using  telecommunications  to  support  teachers  with  professional  development  and 
curriculum  activity.  Williams  (1997)  highlights  the  need  for  teachers  to  be  "connected”  in  a society  that  is  increasingly 
dependent  on  the  Internet  for  communication.  Additionally,  she  stresses  the  importance  of  teacher  support 
organisations  in  helping  teachers  learn  about  and  excel  in  the  new  professional  environments  where  teachers  work. 
Consequently,  the  RITE  Group  offers  teachers  opportunities  to  both  learn  from  and  contribute  to  a dynamic  virtual 
community.  In  doing  so  the  RITE  Group  hopes  to  learn  more  about  models  of  professional  development  and  ways  in 
which  teachers  can  harness  telecommunications  for  powerful  learning  experiences. 

This  paper  will  look  at  two  action  research  projects  which  have  extended  oz-TeacherNet  to  Japan,  meeting, 
and  to  a certain  extent  overcoming,  the  language  (both  written  and  oral)  barriers.  The  first  project  looks  at  how  the 
Book  Rap  project  was  adapted  and  extended  to  develop  awareness  of  different  cultural  aspects  of  the  regard  for  the 
elderly  in  the  International  Year  of  the  Older  Person.  The  second  project  focuses  on  the  Travel  Buddy  Project  and 
reports  on  the  fostering  of  rapport  between  two  school  communities,  one  in  Japan  and  the  other  in  Australia  through  a 
Travel  Buddy  icon. 

Project  one:  Investigating  the  use  of  Information  and  Communication  Technologies  in 
Languages  Other  Than  English 

Within  the  Queensland  curriculum  the  study  of  a Language  Other  Than  English  (LOTE)  is  a requirement  for 
upper  elemcntar>'  classes.  Schools  are  able  to  choose  which  language  they  will  study  but  seven  languages  are  currently 
encouraged:  Japanese,  Indonesian,  Korean,  Mandarin,  German,  French  and  Italian.  The  Romance  languages  show  the 
derivation  of  the  English/Australian  language,  and  the  Asian  languages,  Australia’s  close  proximity  and  economic 
interactions  with  Asia.  Despite  the  concerns  of  some  (Dabblestein,  1997)  that  a socio-cultural  perspective  marginalises 
the  primacy  of  language  acquisition  as  a discipline  goal,  at  the  level  of  elementary  schooling  this  approach  introduces 
students  not  only  to  basic  language  phrases  but  also  to  social  and  cultural  aspects  whereby  Australian  and  other  cultures 
can  be  compared. 

Following  research  taken  by  the  Australian  Catholic  University,  Brisbane  Catholic  Education  developed  a 
position  paper  on  Cultural  Literacy  and  Languages  (Brisbane  Catholic  Education,  1997)  which  identified  five  key 
concepts  to  develop  through  the  teaching  of  this  subject;  culture,  diversity,  interdependence,  identity  and  reconciliation. 
These  five  concepts  formed  the  basis  of  research  conducted  with  a primary  school  during  1 999. 
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The  action  research  project  evolved  from  a request  of  a LOTE  teacher  of  Japanese  to  incorporate  Information 
and  Communication  Technologies  into  the  upper  elementary  curriculum.  The  environmental  aspects  of  the  school  play 
an  important  part  in  the  conduct  of  the  project  and  thus  are  elaborated  upon  at  this  stage. 


The  school 

The  school  is  situated  in  a garden  suburb,  eight  kilometres  from  the  centre  of  Brisbane,  Queensland,  Australia. 
It  is  a systemic  Catholic  school  of  340  students.  The  predominant  cultural  derivation  of  the  families  is  Italian.  The 
school  has  a well-developed  Japanese  program,  which  unlike  most  schools,  involves  students  from  Years  1-7.  The 
school  is  relatively  well  resourced  with  Japanese  resources  with  a LOTE  teacher  who  teaches  at  the  school  three  days  a 
week,  in  a specially  allocated  room.  The  school  has  yearly  exchanges  with  a sister  school  at  Yasuura,  near  Hiroshima, 
in  Japan.  These  activities  are  being  extended  as  the  previous  head-teacher  from  the  Australian  school  has  joined  the 
Education  Board  in  Yasuura. 


The  project 

The  action  research  project  was  planned  in  co-operation  with  the  LOTE  teacher,  the  Year  7 teachers,  as  well 
as  the  Library  teacher,  and  the  Book  Rap  co-ordinator  from  oz-TeacherNet,  based  at  QUT.  It  extended  a Social  Justice 
unit  that  the  Year  7 teachers  conducted  in  first  term,  that  had  as  one  of  its  resources  Wilfrid  Gordon  McDonald 
Partridge  by  Mem  Fox  (1987).  This  book  is  usually  used  in  early  childhood  classrooms  but  a Book  Rap  was  developed 
that  allowed  older  students  to  reflect  on  their  relationships  with  their  grandparents  and  other  older  people  that  they 
came  in  contact  with. 

The  book  was  seen  as  a vehicle  to  explore  cultural  differences  on  the  way  in  which  Australia  and  Japan 
viewed  older  people.  The  storyline  of  the  book  revolves  around  a little  boy  who  lives  next-door  to  an  old  peoples' 
home,  and  the  friendships  that  develop  between  him  and  some  of  those  who  live  there.  Old  peoples'  homes  (or  nursing 
homes)  are  familiar  in  Australia  but  at  present  old  people  are  usually  looked  after  within  the  family  home  in  Japan. 
Thus  it  was  thought  that  the  cultural  differences  would  be  easily  noticed  in  a project  that  connected  a school  in 
Australia  with  a school  in  Japan. 

Building  on  students'  reflection  on  their  own  ideas  about  older  people,  students  in  Australia  conducted  an 
interview  with  an  older  person  that  they  knew.  Most  students  elected  to  interview  one  of  their  grandparents.  They  asked 
them  about  their  hobbies  and  their  favourite  television  program.  They  created  an  image  of  their  'Golden  Oldie's'  (GO) 
hobby  and  were  asked  to  collect  photos  of  them  when  they  were  young  and  a recent  photo.  These  were  to  be  displayed, 
with  GO  permission,  on  a webpage. 

The  LOTE  teacher  required  that  all  four  communicative  modes  were  explored  in  the  program.  At  first,  the 
planned  use  of  written  script  was  a problem,  as  Japanese  word  processing  applications  used  Romaji  which  the  teacher 
did  not  want  the  students  to  use,  preferring  them  to  use  Hiragana.  However,  it  was  decided  that  students  should  create  a 
website  that  included  a video  clip  of  them  talking  about  their  Golden  Oldie.  They  used  Hiragana  to  write  and  learn  their 
script.  It  was  also  anticipated  that  they  would  read  Japanese  script  on  the  website  of  the  participating  Japanese  school. 
In  keeping  with  the  philosophy  of  authentic  use  of  language,  the  visiting  Yasuura  students  were  able  to  help  the 
students  rehearse  their  scripts  before  the  recording  during  their  visit  to  the  school. 

The  use  of  Information  and  Communications  Technology 

The  project  was  staged  so  that  students  could  complete  tasks  either  on  the  classroom  computers  or  with  the  aid 
of  the  Teacher-Librarian  (TL)  in  the  library.  The  students  used  Paintbrush  to  complete  the  image  for  their  GO's  hobby. 
They  scanned  the  photos  with  the  aid  of  the  TL.  A template  was  set  up  on  a laptop  and  students  wrote  in  English  about 
their  GO  on  their  webpage  and  were  recorded  on  a video  clip  using  a Sony  digital  camera  speaking  in  Japanese.  The 
mpg  file  was  linked  to  their  webpage.  The  site,  once  complete,  was  posted  on  the  RITE  server. 

Five  Dimensions  of  Cultural  Literacy 

The  project  to  was  able  to  facilitate  learning  across  the  five  dimensions  of  the  Cultural  Literacy  curriculum  through  the 
use  of  Information  and  Communications  Technology. 


1099 


1 . Culture  - use  of  authentic  photos 

Students  were  able  to  discuss  differences  and  similarities  between  both  the  young  and  old  photos  of  both  Japanese  and 
Australian  'Golden  Oldies'.  The  use  of  authentic  photos  enabled  students  to  look  at  changes  to  other  cultural  aspects 
such  as  transport,  clothing,  hairstyles. 

2.  Diversity  - comparison  of  graphs 

Students  were  able  to  see  a diversity  between  people  by  looking  at  the  hobbies  and  favourite  television  programs  of  the 
Golden  Oldies  and  comparing  them  with  their  own  favourites. 

3.  Interdependence-  development  of  web  sites 

A project  such  as  this  enables  students  to  see  how  it  is  possible  for  different  groups  to  co-operate  across  boundaries  to 
take  part  in  a project. 

Students  were  also  able  to  explore  the  different  relationships  that  they  had  with  their  Golden  Oldies. 

4.  Identity  - production  of  graphic  for  hobby 

Students  were  able  to  focus  on  older  people  as  individuals  by  looking  at  their  hobbies  and  television  programs. 

5.  Reconciliation 

The  teacher  needed  to  be  aware  of  the  possibility  that  members  of  the  age  group  of  the  Golden  Oldies,  both  in  Japan 
and  Australia  would  have  memories  and  may  be  personal  experience  of  the  hostilities  between  the  countries  during  the 
Second  World  War.  In  only  one  case  was  reference  made  to  this. 

Summary 

The  culmination  of  the  event  was  a Grandparents'  day  where  participating  Golden  Oldies  were  invited  to 
morning  tea  and  the  website  was  displayed.  The  classroom  teachers  commented  on  how  involved  the  students  became 
with  the  project  It  strengthened  family  ties,  especially  for  those  grandparents  who  lived  overseas  and  with  whom  the 
students  had  had  minimal  contact.  It  enabled  them  to  see  them  as  a young  person  and  as  a person  with  definite  hobbies 
that  the  students  had  previously  been  unaware  of 

The  LOTE  teacher  noted  how  viewing  the  students'  video  clips  had  enabled  her  to  be  much  more  objective 
about  their  spoken  language  ability.  She  felt  that  it  would  give  the  host  families  an  indication  of  the  students'  skills 
before  they  took  met  the  students  in  the  student  exchange  program.  All  teachers  commented  that  being  in  the  project 
had  increased  their  computing  skills  and  had  made  them  aware  of  how  much  they  could  do  with  their  students  by  using 
computers  with  a curriculum  focus. 

Project  two:  Establishing  real  audience  and  authentic  experience 

Australia,  merely  by  its  physical  location,  is  an  isolated  environment  and  population.  As  a result  of  this, 
Australians  seem  to  have  developed  a strong  sense  of  identity  in  which  we  can  relate  to  our  unique  environment,  flora 
and  fauna  and  lifestyle  as  Australian  cultural  identity.  Inversely  though,  because  of  this  isolation,  there  is  a danger  that 
our  people  can  become  insulated  from  the  rest  of  the  world  and  don't  develop  a sense  of  a wider  community  from  an 
international  perspective. 

As  mentioned  in  the  description  of  the  first  project  in  this  paper.  Languages  Other  Than  English  (LOTE)  is  a 
defined  curriculum  area  in  Australia  and  is  one  of  eight  Key  Learning  Areas  (KLA's)  prescribed  by  Education 
Queensland  (Education  Queensland,  1999),  the  government  body  for  education  in  this  state.  While  this  curriculum  area 
does  involve  a study  of  the  chosen  language,  it  also  involves  study  of  the  cultural  and  physical  perspectives  of  the 
country  or  countries  in  which  the  language  is  spoken.  The  rationale  for  this  is  that  if  children  are  going  to  learn  a 
language  effectively,  there  is  a need  to  establish  an  awareness  of  the  context  in  which  the  language  is  used. 

Of  course,  the  best  way  for  a student  to  immerse  in  the  context  of  a language  is  to  actually  visit  the  country. 
This  option,  ie.  a school  visit  overseas,  is  becoming  more  common  in  Australian  schools  as  the  relative  cost  of  airfares 
has  reduced  and  the  notion  of  travelling  abroad  is  becoming  a realistic  concept  for  many.  Never  the  less,  the  physical 
location  of  Australia  and  therefore  the  cost  of  travel  is  still  a major  impediment  for  schools.  In  cases  where  the  chance 
to  travel  to  a LOTE  country  is  offered,  it  is  usually  every  second  year  on  a voluntary  basis.  It  is  likely  that  a student 
may  contemplate  an  overseas  visit  once  in  their  school  years  and  then  only  if  their  parents  can  afford  to  finance  the  trip. 
It  is  obvious  that  another  mechanism  that  will  provide  "real  experience"  is  required  to  supplement  this  valuable  but 
infrequent  (and  often  inequitable)  venture. 

The  Travel  Buddy  Project 
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The  Travel  Buddies  concept  is  a valid  way  to  meet  this  need.  A Travel  Buddy  is  a soft  toy  or  puppet  that 
travels  the  world  as  a representative  of  a school  or  class.  The  buddy  may  go  on  a cultural  exchange  with  a single  school 
or  travel  to  many  destinations  on  an  itinerary  planned  by  the  travel  buddy's  "family".  The  buddy  becomes  a member  of 
the  host  class  and  participates  in  normal  activities  with  the  children,  both  in  and  out  of  school.  Of  course,  they  need  to 
keep  a record  of  their  trip  (eg.  photos  and  a diary  produced  by  the  children  looking  after  the  buddy)  and  write  emails 
home.  At  the  end  of  the  exchange,  the  buddy  returns  home  with  souvenirs  and  memories  to  share  with  the  class. 

The  oz-Teacher  net  maintains  the  Travel  Buddy  Project  through  the  Travel  Buddy  forum.  The  forum  is  a 
web/email-based  community  that  provides  teachers  with  an  avenue  to  contact  other  teachers  and  establish  exchange 
projects.  While  the  projects  that  emerge  from  this  program  vary  considerable  as  participants  negotiate  their  own  terms, 
the  travel  buddy  character  is  the  common  theme  of  all  projects.  The  role  of  the  oz-Teachemet  in  this  project  is  as  a 
facilitator. 

Keith  Koala  - A Travel  Buddy 

The  Keith  Koala  project  was  designed  as  a simple  action  research  that  would  be  used  to  demonstrate  the 
Travel  Buddy  project  to  educators  around  the  world.  In  particular,  it  was  displayed  in  a poster  session  at  the 
International  Conference  for  Computers  in  Education'99  that  was  held  in  Chiba,  Japan.  Additionally,  this  project  was 
designed  to  inform  future  development  of  the  Travel  Buddy  Project. 

The  "Keith  Koala"  puppet  spent  three  weeks  with  a year  six  class  (1 1 year  olds)  in  suburban  Brisbane, 
Australia.  The  brief  that  the  children  were  given  was  that  they  needed  to  teach  Keith  about  their  lives  in  Australia.  The 
evidence  of  Keith’s  experiences  was  presented  in  a diary,  a collection  of  souvenirs  that  were  placed  in  Keith’s  backpack 
and  a web  page  about  the  project.  The  children  also  composed  a class  email  to  the  school  in  Japan  and  recorded  a 
digital  video-greeting  message  in  Japanese.  At  the  end  of  the  three  weeks  Keith  was  taken  to  Japan  and  the  ICCE’99 
conference.  After  the  conference  the  puppet,  along  with  the  memorabilia  of  his  travels  was  delivered  to  a parallel  class 
in  a Japanese  school.  The  Japanese  children  in  turn  wrote  in  the  Keith  Diary,  collect  souvenirs,  take  photographs  and 
draw  pictures  to  portray  their  own  lives  for  the  Australian  children.  At  the  closure  of  the  project,  the  buddy  returned  to 
the  Australian  class  to  share  stories  and  presents,  just  as  a real  exchange  student  might. 

The  Keith  Koala  project  provided  some  fascinating  insights  into  the  role  that  a travel  buddy  may  take  in  a 
classroom  community.  The  Keith  travel  diary  is  a rich  source  of  memorable  moments  that  captures  the  exchange 
perspective  and  the  experiences  that  Keith  had  with  the  classes.  Additionally  the  Keith  Koala  web  site 
(http://www.fed.qut.edu.au/masters/koala/menu.htm)  provides  an  engaging  and  informative  record  of  a "real" 
adventure. 

The  Role  of  Information  and  Communications  Technology 

As  many  of  the  oz-Teachemet  endeavours  involve  extensive  and  complex  use  of  technologies,  It  was  initially 
disappointing  when  it  became  apparent  that  computers  and  communication  technology  would  play  a very  small  role  in 
this  project.  The  Australian  school  had  one  computer  in  the  classroom  and  an  Internet  connection  in  the  school  library, 
while  the  Japanese  class  had  no  computer  access  in  their  school,  their  only  access  being  through  a laptop  and  email 
account  belonging  to  a parent  and  Internet  access  at  the  local  community  centre.  However,  as  the  conceptualisation  of 
the  project  developed  it  became  obvious  that  the  technologies  would  actually  play  a key  role  in  the  venture. 

The  advantage  of  the  travel  buddy  idea  is  that  it  can  actually  operate  successfully  with  NO  use  of  technology.  As 
such,  it  provides  an  excellent  mechanism  to  gradually  introduce  technology  into  school  environments  under  real  and 
meaningful  context.  In  this  instance,  the  project  gave  a great  opportunity  for  the  project  officer  to  model  and  then 
support  valid  uses  of  technology  for  communication  purposes.  This  included: 

• the  use  of  email  to  negotiate  an  exchange  project 

• the  use  of  the  classroom  computer  to  prepare  text  files  for  later  emailing 

• using  a digital  camera  to  capture  aspects  of  school  community 

• scanning  drawings  and  photographs  for  electronic  display 

• creating  a webpage  to  present  the  project  to  a wider  audience 

• using  a digital  movie  camera  to  record  a video  message  for  a real  audience 

• using  email  as  a fast  communication  device  from  remote  locations  in  the  world 
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Summary 


Teachers  who  have  used  the  Travel  Buddy  Project  with  their  students  often  report  on  the  way  in  which  the 
buddy  acts  as  a catalyst  for  all  types  of  educational  investigation.  (Masters,  1999).  It  seems  that  the  nature  of  this 
project  is  that  it  offers  opportunity  for  any  type  of  focus  a teacher  may  wish  to  pursue.  While  the  Keith  Koala  project 
was  relatively  modest  in  its  aspirations,  it  certainly  demonstrated  the  power  of  a travel  buddy  as  a metaphor  for  real 
travel  experiences. 

Conclusion 

The  oz-Teachemet’s  axiom  is  "teachers  helping  teachers".  It  is  through  projects  such  as  the  two  described  here 
that  the  RITE  Group  can  support  teachers  to  become  confident  at  using  Information  and  Communication  Technology 
for  their  own  curriculum  purposes. 
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Abstract:  In  this  paper  we  argue  that  effective  teaching  is  a multi-method  process  and  propose 
a teachware  management  framework  flexible  enough  to  support  multiple  teaching  strategies. 
We  model  e.g.  behaviouristic  or  construct! vi Stic  strategies  based  on  a metamodeling  approach 
to  learning  material.  As  the  common  implementation  language  we  introduce  the  XML-based 
Learning  Material  Structure  Markup  L^guage  which  also  facilitates  the  interchange  and 
portability  of  learning  material.  Our  teachware  management  framework  has  already  proved  its 
advantages  from  both,  the  author’s  and  the  learner’s  perspective. 


1.  Introduction 

Learning  processes  depend  on  the  individual  properties  of  learners  like  age,  level  of  experience,  knowledge  and 
interest  which  usally  span  a wide  range.  Thus,  there  is  no  best  teaching  strategy  that  applies  to  every  learning 
situation.  In  fact,  even  for  a single  learner  the  best  strategy  may  change  over  time.  Students  taking  an 
introductory  course  on  database  theoiy,  for  instance,  should  be  guided  through  the  material  whereas  later  on  they 
may  want  to  find  out  on  their  own  about  the  SQL  query  language  perhaps  coached  by  a human  tutor.  As  another 
example,  students  taking  a course  on  the  history  of  music  may  at  one  time  wish  to  learn  about  operas  by 
following  a timeline-based  tour  through  the  musical  epochs  while  at  another  time  they  may  prefer  to  have  more 
or  less  unstructured  material  on  composers  and  their  works  available  and  to  construct  an  individual  map  of  the 
world  of  opera. 

Apparently,  teachware  management  systems  or  computer  managed  instruction  (CM I)  systems  should  support 
multiple  teaching  strategies.  Moreover,  as  the  examples  above  show,  they  should  provide  an  appropriate  data 
model,  suitable  abstraction  mechanisms,  and  tools  not  only  to  authors  but  also  to  learners.  It  is  obvious,  that  a 
flexible  multi -strategy  support  and  powerful  application  interface  both  require  system  properties  that  are  beyond 
most  traditional  hypermedia  systems  and  common  techniques  and  tools  for  computer  based  teaching. 

In  this  paper,  we  show  how  the  required  structural  properties  and  multi-strategy  support  can  be  largely  achieved 
in  the  PaKMaS  teachware  management  framework  (PaKMaS  2000)  which  is  based  on  a meta-modeling 
approach  (Siifi  et  al.  1999)  providing  a flexible  and  adaptable  data  model  and  on  the  XML-based  Learning 
Material  Markup  Language  (LMML).  Some  of  the  key  features,  of  course,  are  a simple  yet  powerful  component 
model,  repository  techniques  that  exploit  the  meta-data  available,  the  strict  separation  of  contents  from  structure 
as  well  as  the  capability  of  adaptive  presentation  and  navigation. 

The  rest  of  the  paper  is  organized  as  follows:  In  section  2,  we  have  a look  at  the  three  most  important  learning 
theories  and  emphasize  the  need  to  apply  multiple  teaching  strategies  in  teachware  managment  systems.  In 
section  3,  we  discuss  our  meta-modeling  approach  focussing  on  the  concepts  needed  for  the  modeling  of  learning 
strategies.  We  also  introduce  a sublanguage  of  LMML,  the  Learning  Material  Structure  Markup  Language 
(LMSML).  The  benefits  of  the  teachware  management  framework  based  on  the  meta-modeling  approach  and  the 
LMML  for  authoring  and  application  of  different  strategies  are  discussed  in  section  4.  A comparison  with  related 
work  can  be  found  in  section  5.  The  paper  is  concluded  with  a summary  in  section  6. 


2.  Learning  theories  and  teaching  strategies 

Due  to  the  heterogenerity  of  the  goals  aimed  at,  practical  arrangements  for  teaching  and  learning  are  influenced 
by  various  didactical  and  leam-theoretical  considerations.  Consequently,  teachware  appreciating  this  variety  of 
approaches  and  thus  satisfying  the  requirements  of  practical  teaching  has  to  account  for  and  give  support  to 
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different  leam-theoretical  foundations.  The  latter  can  be  classified  into  behaviouristic,  cognitivistic  and 
constructivistic  theories. 


2,1,  Behaviourism 

The  behaviouristic  approach  is  based  on  the  assumption  that  behaviours  can  be  steered  by  external  stimuli. 
Certain  stimuli  (S)  cause  related  behavioural  responses  (R).  Stimulus-response  pairs  (S-R  pairs)  can  be  chained 
and  become  habits.  Teaching  goals  are  to  be  achieved  by  presenting  selected  information  and  questions  or 
exercises  as  stimuli,  that  are  expected  to  cause  a certain  behaviour,  to  the  learner.  If  the  learner  reacts  in  the 
expected  way  his  or  her  behavioural  response  will  be  confirmed.  The  same  principle  applies  to  both  simple  and 
complex  teaching  goals.  However,  in  the  case  of  complex  goals  the  entire  process  is  broken  into  a collection  of 
simpler  steps  which  are  normally  ordered  sequentially.  The  behaviouristic  learning  theory  is  chiefly  materialized 
as  a learning  strategy  by  guided  tours  or  linear  presentations  using  multiple  choice  questionnaires  or  drill  & 
practice  modules.  Teaching  machines  and  the  model  of  programmed  instruction  are  early  realizations  of 
behaviourism  (Skinner  1958).  The  basic  principle  of  these  approaches  is  the  presentation  of  information 
followed  by  a collection  of  questions  and  according  feed-back  to  the  learner.  Authoring  systems  (Chambers  et  al. 
1980)  support  the  development  of  behaviouristic  teaching  material.  The  overall  teaching  goal  is  the  acquisition 
of  factual  knowledge. 


2,2,  Cognitivism 

Cognitivism  which  relates  to  the  theories  of  Piaget  (Piaget  1977)  and  Bruner  (Bruner  1966)  considers  the  learner 
as  an  individual  that  processes  external  stimuli  actively  and  independently  and  that  can  therefore  not  be  steered 
by  external  impulses.  As  opposed  to  the  behaviouristic  view,  the  cognitivists  believe  that  learning  is  based  on 
cognitive  structures.  It  is  assumed  that  the  learner  perceives,  interprets  and  processes  impressions  in  a selective 
way  according  to  his  or  her  level  of  experience  and  personal  evolution.  The  latter  is  characterized  by  the  total  of 
perception,  comprehension  and  processing  patterns  and  schemata  that  are  at  the  command  of  the  individual  and 
constitute  his  or  her  cognitive  structure. 

Therefore  the  question  what  processes  underly  the  interaction  of  learning  material  as  the  external  condition  of 
learning  on  the  one  hand  and  cognitive  structure  as  the  internal  condition  of  learning  on  the  other  hand  receives 
special  attention.  Within  the  cognition-theoretical  framework  there  exist  various  different  positions  concerning 
(among  others)  the  question  whether  the  overall  goal  should  be  the  construction  of  a knowledge  structure  or 
rather  the  development  of  a general  problem  solving  ability  or  intellectual  and  moral  capabilities,  the  practical 
means  in  both  cases  being  hints  and  problems  in  simulations  or  case  studies  as  appropriate  to  the  individual 
cognitive  structure.  As  another  question  it  has  been  discussed  how  the  learning  material  and  related  information 
is  processed  by  the  human  brain  and  how  content  can  be  adapted  accordingly,  e.g.  by  relating  notions  to  their 
conceptual  environment  or  by  mixing  visual  and  textual  presentations.  The  cognitivistic  learning  theory  together 
with  behaviouristic  ideas  has  influenced  the  Instructional  Design  (Merrill  1991).  However,  essential  aspects 
received  attention  only  in  connection  with  constructivistic  approaches. 


2.3.  Constructivism 

From  the  perspective  of  constructivism  the  potential  of  teaching  media  to  steer  learning  processes  is  considered 
even  lower  than  from  the  cognitivistic  point  of  view.  The  active  processing  of  external  impulses  is  rated  even 
higher.  The  learner  is  assumed  to  construct  reality  individually  based  on  his  or  her  subjective  experience 
structures  (Maturana  et  al.  1984).  Although  constructivism  is  an  offspring  of  cognitivism  it  refuses  the  latter’s 
objectivism  (Jonasson  1991).  Constructivism  considers  knowledge  not  as  a reflection  of  an  external  reality  but 
rather  as  a function  of  perceptual  processes.  From  this  point  of  view  learning  material  is  to  consist  just  of 
collections  of  information  units  and  tools  that  form  the  input  of  the  learning  processes  built  by  the  individual.  It 
is  assumed  that  learning  material  can  neither  control  the  learning  processes  nor  provide  any  reflection  of  reality. 
Every  act  of  perception  creates  a new  world  (Maturana  et  al.  1984).  As  one  possible  realization  of  the 
constructivistic  approach,  learning  environments  can  be  provided  that  support  the  activity-based,  explorative 
interaction  with  the  environment.  Three  approaches  to  computerized  learning  environment  are  rooted  in 
constructivism: 

1 . Cognitive  tools  are  a collection  of  software  programs  that  use  the  control  capabilities  of  the  computer  to 
amplify,  extend,  or  enhance  human  cognition  (Kozma  1987).  Cognitive  tools  support  the  learner  in  the 
formation  of  his  or  her  own  concepts.  The  learning  environment  is  therefore  formed  and  modeled  by  the 
learner. 
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2.  Learning  is  understood  as  a communicative  activity  in  knowledge  communities.  The  construction  of  reality 
requires  the  interaction  with  the  social  environment,  too.  Communication,  e.g.  in  networks,  should  therefore 
be  supported  (Brown  1985). 

3.  Learning  is  viewed  as  coaching  based  on  a master-apprentice  relationship  (cognitive  apprenticeship).  The 
master  offers  advice  but  gives  the  apprentice  more  and  more  independence  when  he  or  she  makes  progress 
(Collins  et  al.  1989) 


2.4.  Multiple  teaching  strategies 

Courses  and  seminars  normally  contain  components  belonging  to  different  leam-theoretical  positions.  We  are 
convinced  that  all  three  approaches  sketched  above  should  be  taken  into  consideration.  However,  which  learning 
theory  is  best  suited  to  support  learning  processes  depends  on  the  teaching  goals  and  the  individual  properties  of 
the  learner,  i.e.  level  of  experience  and  education,  age,  learning  habits.  To  teach  freshmen  the  foundations  of 
database  systems,  for  instance,  could  possibly  be  accomplished  best  by  a guided  tour,  whereas  more  experienced 
students  may  be  better  served  with  an  interface  to  a real  database  system  together  with  hints,  comments, 
documentation,  and  a facility  to  contact  the  responsible  coach,  e.g.  by  email.  Consequently,  a teachware 
management  system  should  have  the  capability  to  support  different  teaching  strategies.  These  can  be 
materialized  in  the  form  of  behaviouristic  components  like  guided  tours  with  multiple-choice  questions. 
Likewise,  a constructivistic  strategy  could  be  provided  by  a collection  of  informative  components  and  tools  that 
allow  the  learner  to  construct  or  change  the  learning  environment,  in  a certain  way  being  an  author  him-  or 
herself.  Besides,  a coaching  support  and  facilities  for  human  communication  should  be  provided.  Most  CMI 
systems  implement  only  one  single  teaching  strategy.  To  overcome  this  restriction  a suitable  modeling  and 
implementation  of  multiple  strategies  are  needed  as  well  as  an  appropriate  toolkit  for  authors  and  learners.  Of 
course,  a CMI  system  can  provide  means  to  develop  and  use  teaching  material.  However,  its  success  is  largely 
determined  by  the  knowledge  and  competence  of  the  author. 


3.  A teachware  management  framework 

3.1  A metamodeling  approach  to  hypermedia  teachware 

We  have  developed  a meta-modeling  approach  to  hypermedia  teachware  (SiiB  et  al.  1999),  which  is  flexible 
enough  to  let  the  teachware  management  system  PaKMaS  support  multiple  teaching  strategies  (see  section  2.4.). 
The  bottom  layer  of  our  meta-modeling  architecture  consists  of  the  subjects  to  be  taught  or  learned.  One  layer 
up,  hypermedia  learning  material  are  describing  the  given  domain  of  application,  e.g.  database  theory  or  the 
world  of  opera.  They  use  domain-specific  means,  which  are  specified  in  domain-specific  models  and  describe 
the  conceptual  content  as  well  as  the  modular  structure  of  learning  material  and  different  teaching  strategies  of  a 
given  domain  of  application.  Finally,  the  common  abstract  meta-model  specifies  learning  material  in  an  abstract 
way.  This  includes  aspects  also  of  teaching  strategy  by  specifying  how  both,  authors  and  learners,  can  form 
learning  material  in  general.  Learning  material  consists  of  learning  modules  that  are  either  basic  modules  or 
structure  modules  containing  other  modules.  The  former  are  the  terminal  nodes  of  a polyhierarchical  hypermedia 
structure  similar  to  directories  or  books,  but  allowing  modules  to  belong  to  more  than  one  super  module.  Indeed, 
each  module  has  to  be  contained  in  at  least  one  structure  module.  By  composition,  they  dynamically  adapt 
learning  material  to  the  needs  of  different  groups  of  learners  at  the  content  level  and  realize  the  different  teaching 
strategies  of  section  2 using  appropriate  meta-data.  Basic  modules,  however,  are  the  smallest  self-contained  units 
or  building  blocks  of  the  modular  structure  of  learning  material.  Representing  conceptual  units,  they  contain 
various  content  objects,  which  can  be  behaviouristic,  e.g.  drill  & practice  instructions  and  multiple  choice 
questions,  or  constructivistic,  e.g.  coaching  remarks,  simulations  or  interfaces  to  real  world  systems. 


3.2.  Learning  Material  Markup  Language 

To  represent  learning  material  incorporating  different  teaching  strategies,  we  need  a data  format,  which  is 
suitable  for  storing  the  content  and  structure  of  the  material  and  which  allows  to  access  the  stored  modules  or 
data,  very  much  like  objects  in  a database  system.  Moreover  also  the  authors  and  learners  should  find  it  easy  to 
read  and  use  the  information  stored  in  the  modules.  As  described  in  (SiiB  et  al.  1999)  we  have  developed  a XML 
representation  for  course  content  and  structure  representation  and  interchange:  the  Learning  Material  Markup 
Language  (LMML).  It  comprises  two  sublanguages,  the  Learning  Material  Content  Markup  Language,  which  is 
a syntactical  representation  of  the  conceptual  content  and  the  Learning  Material  Structure  Markup  Language, 
which  represents  the  modular  structure  model.  To  implement  multiple  teaching  strategies,  it  uses  special 


elements,  e.g.  guidedTour  or  collection,  which  are  declared  in  the  Document  Type  Definition  (DTD)  (PaKMaS 
2000)  and  which  have  strategy  attributes  with  possible  constant  values  beh  (behaviouristic)  or  con 
(constructivistic)  respectively.  On  the  one  hand,  the  use  of  the  XML-based  LMML  faciliates  the  interchange  and 
portability  of  learning  material  (SuB  et  al.  1999).  On  the  other  hand,  using  metadata  like  those  mentionend 
above,  a teachware  management  system  like  PaKMaS  supports  authors  wanting  to  create  learning  material  using 
multiple  teaching  strategies  as  well  as  learners  using  the  presented  material. 

4.  Supporting  multiple  strategies 

4.1.  Behaviouristic  strategies 

Within  our  framework,  teaching  material  can  be  presented  hierarchically  using  a book  metapher  with  chapters 
and  sections  as  structure  modules.  They  can  also  be  presented  sequentially  starting  at  a first  basic  module  using  a 
guided  tour  metapher  (Fig.  3).  Both  represent  behaviouristic  strategies  where  the  author  explicitly  plots  the  route 
through  the  given  learning  material. 


Figure  3:  Behaviouristic  strategy  using  a guided  tour 

If  an  author,  for  example,  wants  to  create  a section  of  a booklike  course  or  a guided  tour,  she  can  create  new 
behaviouristic  basic  modules  using  course  units  defined  in  LMCML  which  can  be  extended  e.g.  by  question 
elements  from  the  Tutorial  Modelling  Language  (TML)  (Brickley  1999).  She  also  can  query  the  systems 
repository  for  suitable  basic  modules,  i.e.  for  modules  with  strategy  attribute  (see  section  3.2.)  set  to  beh 
(behaviouristic).  PaKMaS  uses  these  meta-data  to  answer  questions  like  “Which  basic  units  about  the  SELECT 
statement  are  appropriate  to  be  used  in  a guided  tour  on  SQL?“.  For  the  modules  found,  the  system  can  suggest  a 
linear  ordering  or  check  the  consistency  of  a sequence  created  by  an  author  using  the  modules  prerequesite  and 
objectives  attributes.  As  guided  tours  and  sections  or  chapters  are  structure  modules  which  impose  a linear 
ordering  to  the  children  modules  a teachware  managment  system  can  offer  navigation  facilities  with  which  the 
learner  can  sequentially  walk  step-by-step  through  the  given  basic  modules. 

4.2.  Constructivistic  strategies 

Our  framework  is  flexible  enough  to  support  constructivistic  strategies,  too.  It  provides  structure  modules  like 
indexes,  glossaries,  time  lines  or  even  just  unordered  collections  (see  Fig.  4)  of  learning  material. 


basic  modules 

Figure  4:  Construed vistiv  strategy  using  collections 
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They  all  allow  direct,  non-hierarchical  access.  Special  basic  modules  representing  coaching  units  give  some 
hints  to  the  learner  on  the  appropriate  usage  of  the  available  material.  PaKMaS  supports  cognitive  apprenticeship 
(see  section  2.3.3)  using  these  modules  and  offering  easy  to  use  communication  facilities  with  human  coaches. 
Furthermore  communication  via  email  and  chat  with  other  users  of  PaKMaS  as  well  as  posting  to  appropriate 
newsgroups  managed  by  PaKMaS  support  the  creation  of  knowledge  communities  (see  section  2.3.2.).  A learner, 
for  example,  can  freely  browse  through  basic  modules  on  composers  or  operas  in  any  order,  e.g.  alphabetical  or 
by  date  of  birth.  If  provided  by  the  author,  she  can  use  coaching  remarks  or  hints  in  a coaching  unit. 
Furthermore,  she  can  communicate  with  coaches  or  with  other  learners  in  this  domain  of  application. 

Our  framework  allows  the  wrapping  of  simulations  or  real  life  systems  into  basic  modules  using  LMCML.  This 
way  a learner,  for  example,  in  a database  course  can  find  out  about  SQL  by  herself  using  an  interface  to  a real 
world  database  (Fig.  5).  As  another  example,  she  can  learn  about  harmony  in  a music  course  using  a composing 
system  thus  acquiring  complex  knowledge  by  herself  It  should  be  noted  that  we  do  not  address  the  support  for 
creation  of  constructivistic  programs  like  simulations  etc. 
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Figure  5:  Collections  with  coaching  unit,  SQL  interface  and  descriptions  of  SQL  statements 


In  addition  to  the  simple  possibility  to  annotate  learning  material  with  individual  notes,  PaKMaS  provides 
cognitive  tools  which  are  most  important  for  the  application  of  constructivistic  strategies  (see  section  2.3.1). 
With  these  tools,  learners  can  structure  and  form  their  learning  environment  with  respect  to  the  meta-model.  For 
example,  they  can  use  existing  structure  modules  to  form  own  collections  and  to  create  own  guided  tours.  Note 
that  although  the  latter  represents  a behaviouristic  strategy,  creating  a guided  tour  is  a constructivistic  learning 
step.  To  model  their  own  learning  material,  learners  can  use  existing  content  objects  like  definitions  or  examples. 
Beyond  that  learners  can  create  new  types  of  structure  modules  like  slide  show^  or  new  types  of  conceptual  units 
like  slides.  Thereby  learners  can  continuously  form  their  learning  environment  with  respect  to  the  metamodel. 
Students  in  a music  course,  for  example,  can  construct  their  own  “map”  of  the  world  of  opera  using  own 
concepts. 


5.  Related  work 

The  Instructional  Material  Structure  Description  Language  (IMSDL)  allows  to  specify  instructional  models 
(Silberhom  et  al.  1999).  In  contrast  to  IMSDL,  we  differentiate  between  different  types  of  conceptual  units  or 
different  types  of  content  objects.  Thus,  we  have  the  concepts  and  syntactical  representations  of  coaching  units 
or  coaching  remarks  which  are  necessary  in  constructivistic  strategies.  We  also  differentiate  between  different 
types  of  structure  modules  to  realize  structures  like  books  or  collections.  Finally,  using  a meta-modeling 
approach,  we  distinguish  different  domains  of  application  and  their  specific  needs  with  respect  to  teaching 
strategies.  (IEEE  PI 484  1999)  proposes  a text-based  programming  language  to  describe  courses  which  consist  of 
blocks  and  assignable  units.  The  former  are  the  largest  structural  elements  in  a course  and  are  comparable  to 
structure  modules,  wheras  the  latter  are  programs  or  whole  lessons  launched  by  a CIM  system.  In  contrast  to  our 
work,  the  order  in  which  a student  may  go  through  a course  is  only  implicitly  given  by  the  order  of  the  blocks 
and  assignable  units  or  their  prerequisites  data.  In  the  XML-based  TeachML  (Wehner  et  al.  1999),  a course  is  the 
container  of  learn  blocks,  which  contain  any  number  of  didactical  paths.  The  latter  contain  at  least  one  and  at 
most  four  facts  and  one  optional  exercise.  They  seem  implicitly  to  specify  behaviouristic  guided  tours.  In  our 
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approach,  teaching  strategies  are  explicitly  specified  allowing  the  use  of  learning  strategies  different  from 
behaviouristic  guided  tourSy  too.  Finally,  (Teege  1999)  also  proposes  a XML-based  TeachML.  Here,  the  author 
recursively  composes  learning  material  from  more  or  less  large  modules  using  XSLT.  The  resulting  hierarchical 
structures  implicitly  realize  behaviouristic  strategies. 

6.  Conclusion 

In  this  paper  we  argued  that  effective  teaching  requires  the  use  of  multiple  teaching  strategies.  The  meta- 
modeling approach  to  hypermedia  teachware  which  we  have  proposed  allows  the  domain-specific  modeling 
necessary  for  the  authoring  and  application  of  different  teaching  strategies  by  providing  appropriate  structure 
modules,  conceptual  units,  content  objects  and  meta-data.  We  discussed  how  these  concepts  can  be  implemented 
using  the  XML-based  LMML.  On  the  one  hand  the  Learning  Material  Content  Markup  sub-Language  of  LMML 
syntactically  represents  different  conceptual  units  like  coaching  units  or  drill  & practice  units  and  different 
content  objects  like  coaching  remarks  or  instructions.  The  sublanguage  Learning  Material  Structure  Markup  sub- 
Language  on  the  other  hand  implements  behaviouristic  strategies  using  guided  tours  or  books  as  well  as 
cognitivistic  or  constnictivistic  strategies.  Furthermore,  appropriate  meta-data  are  used  by  a teachware 
management  system  like  PaKMaS  to  support  authoring  as  well  as  usage.  Finally,  relying  on  the  properties  of 
XML,  LMML  can  facilitate  the  portability  and  exchange  of  learning  material. 
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Abstract:  This  paper  describes  the  development  and  exploration  of  a model  for  professional 
development  concerned  with  the  integration  of  computing  technologies  into  classroom  teaching  and 
learning.  Grounded  in  situative  theories  of  knowledge  and  learning,  the  model  adopts  a mentoring 
approach  that  places  technology  experts  in  schools  to  work  with  teachers  to  create  technology 
supported  lessons  that  meet  the  teachers’  curricular  needs. 


Theoretical  Perspectives 

Recent  large  scale  studies  of  computer  usage  in  schools  (Becker,  1994;  Panel  on  Educational  Technology,  1997; 
Educational  Testing  Service,  1 998)  have  precipitated  public  debate  concerning  the  efficacy  of  using  computers  to 
support  instruction,  and  highlighted  the  need  for  professional  development  in  this  area.  While  highlighting  such 
need  and  pointing  to  the  relationship  between  professional  development  and  more  sophisticated  uses  of  technology 
in  schools,  these  and  other  studies  have  contributed  little  to  our  understanding  of  how  professional  development 
impacts  technology  integration,  especially  in  terms  of  how  it  affects  teaching,  learning,  and  technology  integration  at 
the  classroom  level. 

Indeed,  teacher  lore  suggests  that  traditional  inservice  teacher  education  has  little  impact  on  teaching  practices  in 
general.  Smylie  (1989),  for  example,  found  that  teachers  ranked  inservice  training  last  out  of  fourteen  possible 
opportunities  for  learning.  What  teachers  ranked  as  most  important  was  direct  classroom  experience.  Other 
researchers  report  similar  findings  (Little,  1994),  leading  scholars  in  the  field  to  develop  lists  of  the  features 
common  to  effective  staff  development  activities  (Little,  1988;  Abdal-Haqq,  1995;  Ball,  1996;  Wilson  & Berne, 
1999).  Putnam  and  Borko  (1997),  for  example,  reduce  the  essential  features  of  effective  teacher  education  to  four: 

• Teachers  should  be  treated  as  active  learners  who  construct  their  own  understanding. 

• Teachers  should  be  empowered  and  treated  as  professionals. 

• Teacher  education  should  be  situated  in  classroom  practice. 

• Teacher  educators  should  treat  teachers  as  they  expect  teachers  to  treat  students. 

In  a more  recent  article,  Putnam  and  Borko  (2000)  relate  recent  trends  in  research  on  professional  development  to 
new  understandings  of  the  nature  of  learning  and  knowing  that  have  beencollectively  labeled  “situative”  (Greeno, 
1997).  They  identify  three  conceptual  themes  central  to  situative  perspectives  ~ that  cognition  is  situated  in 
particular  physical  and  social  contexts,  that  it  is  social  in  nature,  and  that  cognition  is  distributed  across  the 
individual,  others,  and  tools  (p.  4)  - which  they  believe  have  important  implications  for  professional  development. 
Putnam  and  Borko  argue  that  how  teachers  learn  new  methods  of  teaching  is  no  different  from  any  learning.  If 
knowledge  and  learning  are  indeed  situative,  then  the  most  effective  inservice  education  will  be  situated  in  authentic 
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classroom  practice  and  constructivist  in  focus.  The  situative  perspective  thus  gives  a theoretical  rationale  for 
previous  findings  concerning  effective  inservice  activities. 

To  our  knowledge,  such  perspective  has  not  been  used  to  frame  either  the  development  of,  or  research  on, 
professional  development  programs  aimed  at  technology  integration,  although  a situative  perspective  is  widespread 
in  research  and  development  of  technology-based  educational  programs.  Indeed,  the  Report  to  the  President  on  the 
Use  of  Technology  to  Strengthen  K-12  Education  in  the  United  States  (Panel  on  Educational  Technology,  1997) 
consciously  grounds  its  recommendations  concerning  technology  integration,  professional  development,  and 
educational  technology  research  within  a constructivist  and  situative  theoretical  framework.  The  Panel’s  argument 
for  these  later  approaches  is  analogous  to  Putnam  and  Borko’s  (1997)  finding  that  teacher  educators  should  treat 
teachers  as  they  expect  teachers  to  treat  students;  it  only  makes  sense  to  model  best  technology  integration  practices 
when  attempting  to  get  teachers  to  use  them. 

It  only  makes  sense,  and  yet  most  professional  development  programs  aimed  at  technology  integration  are 
instructionist,  application-driven,  after-school  workshops  or  summer  “institutes”  well  removed  from  classroom 
practice.  Some  have  argued  (Wilson  & Berne,  1999)  that  while  not  optimal,  such  approaches  are  often  the  only 
practical  solutions  to  meeting  large-scale  professional  development  needs  with  limited  resources.  There  has  been 
very  little  investigation  of  the  effects  of  these  traditional  professional  development  activities,  however,  on 
technology  integration  at  the  classroom  level.  In  fact,  what  little  research  there  is  suggests  such  activities  have  little 
impact  on  the  day-to-day  integration  of  computing  technologies  into  classroom  teaching  and  learning  (Becker,  1994; 
Panel  on  Educational  Technology,  1997;  Educational  Testing  Service,  1998). 

It  is  hard  to  see  how  approaches  that  consistently  have  little  or  no  effect  on  classroom-based  technology  integration 
can  bee  deemed  “practical,”  thus  we  decided  to  try  an  “impractical”  approach.  In  the  remainder  of  this  paper,  we 
describe  a professional  development  program  which  has  adopted  an  approach  to  technology  education  that  is  situates 
professional  development  in  day-to-day  classroom  practice. 

The  CATIE  Program 

The  Capital  Area  Technology  and  Inquiry  in  Education  (CATIE)  program  was  established  through  the  Institute  for 
Pre-College  Initiatives  at  Rennselaer  Polytechnic  Institute  (RPI)  as  an  innovative  means  for  addressing  technology- 
based,  constructivist-oriented  staff  development  in  elementary  schools  in  the  greater  Albany  (NY)  area.  This  unique 
program  places  technology  experts  in  school  buildings  where  they  serve  as  mentors  to  teachers  interested  in 
integrating  the  use  of  technology  into  their  day-to-day  classroom  activities.  The  school-based  mentors  provide 
training  to  teachers  on  technology  utilization,  but,  more  importantly,  do  so  in  the  context  of  jointly  designing 
computer-supported  lessons  that  incorporate  technology  into  existing  classroom  curricula.  Teacher  learning  about 
technology  integration  is  thus  situated  in  authentic  technology  integration  activities. 

Typically,  the  mentors  first  meet  with  teachers,  both  individually  and  in  groups,  to  discuss  how  technology  might  be 
used  to  enhance  learning  in  planned  units  on  particular  topics.  Mentors  try  to  avoid  planning  that  is  either  artificial 
or  focused  on  specific  software  applications.  They  then  work  with  teachers  to  design  computer-supported  lessons 
that  are  integral  parts  of  larger,  classroom-based  learning  units.  They  encourage  inquiry-based,  student-centered, 
constructivist  uses  of  computing  technologies,  but  do  not  insist  on  them.  Often,  mentors  model  best  practices  in 
computer-based  teaching  and  learning  by  taking  the  lead  in  implementing  jointly  created  lessons.  They  then  guide 
teachers  in  designing  and  implementing  their  own  computer-based  lessons,  finally  fading  their  support  as  teachers 
become  more  confident  in  the  use  of  computer  technologies. 

Mentor  support,  however,  does  not  just  disappear.  Each  mentor  structures  his  or  her  schedule  according  to  their 
school’s  individual  needs,  but  is  generally  available  two  or  three  days  each  week  for  a period  of  two  years  or  more 
to  work  with  teachers  and  their  students  on  a continuing,  as-needed  basis.  Many  teachers,  having  mastered  a 
particular  technology  tool,  return  to  their  mentor  for  help  in  utilizing  other  applications  in  their  teaching.  Some 
teachers  just  come  to  share  with  their  mentors  the  ways  in  which  they  are  using  technology  on  their  own,  and  some 
mentors  meet  regularly  with  groups  of  teachers  to  discuss  technology  integration.  As  mentors  become  a part  of  the 
culture  of  the  school,  formal  and  informal  conversations  of  this  sort  become  more  common  and  ongoing,  and  a 
discourse  community  grows  up  around  technology  integration. 
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The  major  goals  of  the  CATIE  program  are  to  foster  individual  teacher  and  student  development  of  technological 
skills,  to  assist  teachers’  with  the  infusion  of  technology  into  existing  curricula,  to  broaden  the  use  of  computing 
technologies  within  the  elementary  school  setting,  and  to  foster  constructivist  teaching  and  learning  around 
electronic  technologies.  It  is  an  ongoing  program  and  has  grown  from  a single  mentor  in  one  school  to  more  than 
ten  mentors  working  in  fourteen  elementary  schools  in  four  school  districts.  Indeed,  the  model  is  currently  being 
institutionalized  in  at  least  one  of  these  districts  as  the  preferred  means  of  technology  training. 

Research  Methodologies 

In  keeping  with  its  constructivist  goals,  research  on  the  CATIE  program  is  being  conducted  using  an  action  research 
approach.  Mentors  work  with  the  researchers  to  set  an  evolving  research  agenda,  collecting  data  and  sharing  it  in 
weekly  meetings  with  the  researchers  and  each  other.  Of  course,  as  with  mentors  work  in  the  schools,  constructivist 
approaches  can  be  slow  going,  and  this  process  has  taken  time.  Data  being  collected  to  date  centers  on  narratives 
structured  to  tell  the  stories  of  the  mentors’  experiences  and  the  themes  that  emerge  from  them.  Other  data  being 
regularly  collected  include:  time  spent  on  particular  mentoring  activities,  teachers  worked  with,  lessons  planned, 
subject  areas  and  learning  standards  addressed,  technologies  used,  and  grading  rubrics.  The  mentors  have  also  been 
collecting  student  work,  and  project  researchers  are  conducting  observations  in  representative  classrooms.  In 
addition,  outside  evaluators  have  conducted  interviews  with  each  of  the  mentors  and  a sample  of  participating 
teachers. 

Data  analyses  for  the  first  year  of  CATIE’s  operation  concentrated  on  four  objectives: 

• documenting  how  mentor  services  were  utilized 

• identifying  best  practices  in  mentoring 

• assessing  stakeholder  perceptions  of  the  CATIE  program 

• identifying  areas  in  need  of  change  or  additional  resources. 

Data  analyses  for  this,  the  second  year  of  CATIE’s  operation  will  focus  on: 

• evidence  of  teacher  change 

• student  learning 

• mentoring  themes 

• categories  of  technology  integration 


Findings 

Clearly,  the  CATIE  program  exhibits  the  features  Putnam  and  Borko  (1997)  identify  as  essential  to  effective  teacher 
education: 

• Teachers  involved  in  the  CATIE  program  are  treated  as  active  learners  who  construct  their 
own  learning  to  meet  their  own  specific  professional  needs.  CATIE  mentors  serve  as 
facilitators  for  that  learning  but  follow  and  respect  the  directions  it  takes. 

• Thus,  teachers  in  the  CATIE  program  are  empowered  to  use  technology  in  their  teaching 
(through  the  on-site  support  of  the  mentors)  and  treated  as  the  professionals  they  are. 

• In  the  CATIE  program,  teacher  learning  about  technology  integration  is  situated  in  classroom 
practice. 

• CATIE  mentors  model  constructivist,  student-centered  teaching  with  technology  in  their  work 
with  both  students  and  teachers.  Mentors  thus  treat  teachers  the  way  they  would  have  them 
treat  students. 

Most  mentors  spend  the  majority  of  their  time  working  with  teachers  and  students  in  the  computer  labs  of  their 
participating  schools,  although  they  also  assisted  some  classes  within  their  own  classrooms.  The  latter  is  the  goal  of 
the  program  and  something  to  work  towards  in  the  future.  A related  goal  is  to  encourage  teachers  to  look  beyond 
their  current  uses  of  technology.  This  seems  to  be  happening  with  the  more  technology-experienced  participating 
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teachers,  and  with  teachers  involved  in  their  second  year  with  the  program.  The  CATIE  program  has  also  made 
significant  gains  in  helping  teachers  less  familiar  with  educational  technology  become  comfortable  in  its  use. 

Flexibility  and  adaptability  were  found  to  be  central  to  best  practices  in  mentoring.  Mentors’  ability  to  work  with 
variations  in  teacher  learning  styles,  pedagogical  approaches,  and  prior  experiences,  as  well  as  with  existing  school 
technology  resources  were  found  to  significantly  influence  technology  integration  in  schools  and  classrooms. 

Teacher  perceptions  of  the  CATIE  program  were  overwhelmingly  positive.  Teachers  uniformly  reported  increased 
knowledge  of  computing  technologies,  greater  confidence  in  using  them,  and  more  creative  teaching  with 
computers.  Positive  outcomes  for  students,  including  greater  independence,  heightened  self-efficacy,  and  increased 
motivation,  were  also  noted  by  participating  teachers.  In  addition,  the  CATIE  program  has  been  a positive 
experience  for  mentors  as  well,  giving  them  the  opportunity  to  become  part  of  school  communities  while  learning 
more  about  technology  infusion  therein. 

Finally,  additional  resources  needed  to  improve  the  mentoring  program  included: 

• Time  - More  time  is  needed  for  teachers  to  work  with  the  mentors  and  experiment  with  the 
technologies.  Teachers  would  also  like  mentors  to  be  available  more  times  in  the  schools. 

Mentors  would  like  more  time  to  meet  together  to  share  strategies  and  projects. 

• Technology  Resources  - Many  schools  and/or  classrooms  had  outdated  hardware  and 
software;  access  to  better  technology  resources  would  make  it  easier  to  implement  best 
practices. 

• Administrative  Support  - Administrative  support  can  lead  to  changes  in  the  culture  of  a 
school  such  that  it  supports  technology  integration.  Administrative  practices  identified 
identified  as  most  supportive  of  technology  integration  included:  defining  mentors  roles  early 
in  the  school  year,  discussing  technology  integration  with  teachers,  frequent  visits  to  the 
computer  labs,  spotlighting  teachers’  technology-based  projects  and  other  systems  of  rewards 
for  technology  using  teachers,  and  support  for  technology  needs. 

• Additional  Opportunities  for  Teacher  Learning  - Additional  opportunities  for  teacher  learning 
requested  included:  summer  technology  workshops,  planning  meetings  with  mentors  before 
the  beginning  of  school,  and  more  one-on-one  learning  opportunities  during  the  school  year. 

Educational  Significance 

Although  early  on  in  its  development,  the  CATIE  program  seems  to  be  positively  affecting  technology  integration  in 
the  schools  in  which  it  is  operating.  Indeed,  in  this  its  second  year  of  operation,  its  effects  are  becoming  clearer. 
Perhaps  the  best  measure  of  its  success  is  that  one  of  the  districts  in  which  it  has  been  implementing  is  adopting 
mentoring  as  the  preferred  means  of  technology  training.  Common  sense  indicates  that  the  CATIE  program  is 
successful  because  it  is  reaching  out  to  teachers  in  the  physical  and  social  context  of  their  practice,  because  it 
provides  ongoing,  long-term  support  for  technology  integration,  and  because  of  the  personal  relationships  mentors 
are  forging  with  participating  teachers  and  within  the  culture  of  the  schools  in  which  they  work.  Its  success  thus 
supports  a situative  perspective  on  teacher  learning,  especially  teacher  learning  about  technology  and  technology 
integration. 
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Abstract:  This  paper  explores  the  notion  that  the  desired  outcome  of  current  technology  initiatives 
is  technology  integration  across  the  curriculum.  It  argues  that  technology  integration  across  the 
curriculum  can  best  be  understood  in  terms  of  an  expanded  notion  of  literacy;  that  technology 
integration  across  subject  areas  and  grade  levels  can  best  be  guided  and  assessed  in  terms  of  the 
critical  and  creative  uses  made  of  nonprint  media  and  computing  and  communications 
technologies.  To  such  ends,  it  presents  a possible  set  of  performance-based,  nonprint  media  and 
technology  literacy  standards  for  guiding  and  assessing  technology  integration  across  the  K-12 
curriculum. 


Perspectives 

For  the  past  several  centuries,  the  dominance  of  print  over  other  forms  of  communication  media  has  been 
overwhelming  and  largely  unchallenged.  Recent  decades,  however,  have  witnessed  rapid  changes  in  how  we 
communicate  with  one  another,  entertain  ourselves,  conduct  business,  get  information,  create  knowledge,  and 
generally  make  sense  of  the  larger  world.  Electronic  texts  are  everywhere  replacing  printed  ones  as  the  media  of 
choice  in  a wide  range  of  human  endeavors.  Our  notions  of  what  it  means  to  be  literate  are  correspondingly 
expanding. 

For  example,  the  National  Council  of  Teachers  of  English  (NCTE)  and  the  International  Reading  Association  (IRA) 
state  in  their  summary  of  the  national  Standards  for  the  English  Language  Arts, 

“.  . . being  literate  in  contemporary  society  means  being  active,  critical,  and  creative  users  of 
print  and  spoken  language,  as  well  as  the  visual  language  of  film  and  television,  commercial  and 
political  advertising,  and  more.  It  also  means  being  able  to  use  an  array  of  technologies  to  gather 
information  and  communicate  with  others.  ” (NCTE/IRA,  1996a,  p.  2). 

Renee  Hobbs  (1988)  similarly  argues  for  a new  definition  of  literacy.  “Literacy,  ” she  writes  (p.  7),  “is  the  ability  to 
access,  analyze,  evaluate,  and  communicate  messages  in  a variety  of  forms.  ” 

Indeed,  as  we  find  ourselves  on  the  eve  of  a new  millenium,  the  question  is  no  longer  whether  or  not  we  should  be 
using  new  media  technologies  in  our  nation’s  classrooms  but  how  we  can  best  integrate  their  use  across  the 
curriculum  (Panel  on  Educational  Technology,  1997,  p.  90).  The  Panel  on  Technology  (1997),  for  example,  notes 
that  the  use  of  computers  in  American  schools  mostly  focuses  on  either  learning  about  technology  or  drill  and 
practice  in  basic  skills.  Other  national  studies  have  reported  similar  findings  (Becker,  1994;  Educational  Testing 
Service,  1998). 

Researchers  agree  that  the  biggest  reason  for  such  under-utilization  is  lack  of  understanding.  Our  own  research 
(Swan,  et.  al.,  1997)  suggests  that  teachers  just  don’t  know  how  to  make  nonprint  media  and/or  computers  and 
communication  technologies  an  integral  part  of  day  to  day  learning  in  their  classrooms.  Teachers  need  to  learn  to 
treat  computing  technologies  and  other  nonprint  media  in  the  same  ways  they  treat  technologies  of  print.  In  the  rest 
of  this  paper,  I describe  a set  of  nonprint  media  and  technology  literacy  standards  for  guiding  and  assessing 
technology  integration  across  the  K-12  curriculum  which,  I believe,  at  least  begins  a discussion  that  accepts  the  use 
of  such  media  as  literate  behaviors. 
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Methodology 


Diane  Ravitch  writes,  “ . . . those  who  develop  standards  should  recognize  their  role  is  to  discover  and  explain  the 
very  best  existing  standards,  not  to  invent  new  and  untried  ones'^  (1995,  xxvi).  In  the  case  of  nonprint  literacies, 
three  more  than  adequate  sets  of  national  standards  collectively  identity  the  competencies  in  the  use  of  nonprint 
media  and  communications  technologies  experts  and  practitioners  believe  students  should  have  by  the  completion  of 
elementary,  middle,  and  high  school  These  are: 

• National  Educational  Technology  Standards  for  Students  (International  Society  for  Technology  in 
Education,  1998), 

• Information  Literacy  Standards  for  Student  Learning  (American  Association  of  School  Librarians,  1998) 

• Standards  for  the  English  Language  Arts  (National  Council  of  Teachers  of  English/Intemational  Reading 
Association,  1996b). 

These  three  sets  of  standards  were  chosen  because  they  are  national  in  scope,  because  they  are  constructivist  in 
approach  (Panel  on  Educational  Technology,  1997),  and  because  they  all  address  media  and  technological 
competencies  as  part  of  a larger  notion  of  literacy.  The  three  sets  taken  together  address  technological  literacy, 
information  literacy,  and  literacy  in  general.  All  three  sets  of  standards  were  developed  by  the  leading  professional 
organizations  in  their  specific  areas  of  interest,  and  revised  through  extended,  open  processes  of  review  and  revision 
that  included  a variety  of  stakeholders  as  well  as  educational  technology  experts. 

The  nonprint  media  and  technology  competencies  found  in  these  three  sets  of  standards  were  reworked  to  a list  of 
simple  performances  identifying  the  competencies  students  should  have  at  the  completion  of  elementary,  middle, 
and  high  school,  and  redundancies  among  them  were  eliminated.  The  resulting  performance  standards  were  then 
sorted  into  three  categories  - basic  skills,  critical  literacies,  and  construction  skills  --  to  highlight  the  kinds  of 
competencies  many  experts  believe  students  should  be  developing  around  nonprint  media  and  computing 
technologies  (Panel  on  Educational  Technology,  1997). 

According  to  Tyner  (1998),  two  kinds  of  literacy  are  taught  in  schools.  Tool  literacies  focus  more  on  simple 
meanings  and  basic  functionality  - what  Hobbs  (1998)  calls  “access”.  In  our  list,  these  fall  in  the  basic  skills 
category.  Literacies  of  representation,  on  the  other  hand,  stress  the  need  to  analyze  information  and  understand  how 
meaning  is  created.  Traditional  (print-based)  literacy  addresses  both.  Too  often  nonprint  media  and  technological 
competencies  are  only  addressed  at  the  tool  level.  We  therefore  have  highlighted  literacies  of  representation  in  the 
critical  literacies  and  construction  skills  categories. 

Results 

Basic  skills,  critical  literacies,  and  construction  skills  are  described  in  the  sections  that  follow,  and  specific  competencies  for 
each  category  are  given  at  the  elementary,  middle,  and  high  school  levels.  The  lists  of  standards  should  be  understood  as 
tentative.  It  should  also  be  kept  in  mind  that  the  standards  lists  were  derived  from  existing  national  standards,  and  were  not 
constructed  by  us.  Thus,  while  we  agree  with  most  of  the  individual  competencies,  we  do  not  necessarily  agree  with  all  of 
them.  We  think  there  may  be  competencies  missing  that  represent  literate  behaviors  which  are  unique  to  electronic  media. 
We  feel  confident  in  arguing,  however,  that  these  lists  provide  a good  starting  point  for  discussing  nonprint  literacy  and  for 
guiding  and  assessing  technology  integration  in  our  schools.  In  particular,  the  sorting  of  the  standards  into  basic,  critical,  and 
construction  skills  suggests  more  sophisticated  and  literate  ways  of  understanding  nonprint  media  and  technology  literacy 
and  provides  a way  of  assessing  not  just  the  extent  but  the  quality  of  technology  integration  efforts. 

Basic  skills  are  competencies  involving  the  use  and  simple  manipulation  of  nonprint  media  and  the  recognition  of  common 
conventions  used  by  them.  They  include  competencies  related  to  accessing,  decoding,  encoding,  locating,  etc.  In  the  list 
which  follows,  the  basic  skills  students  should  have  by  completion  of  elementary,  middle,  and  high  school  are  given,  in  that 
order.  These  skills  should  be  understood  as  cumulative;  that  is,  by  the  completion  of  high  school,  students  should  have 
acquired  all  of  them. 


elementary 

• Use  a mouse  to  successfully  operate  a computer 

• Use  a keyboard  to  successf  ully  operate  a computer 
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• Use  a computer  monitor 

• Use  a computer  printer 

• Use  a scanner 

• Use  a digital  camera 

• Operate  a VCR 

• Operate  an  audio  tape  player 

• Use  interactive  books 

• Use  developmental ly  appropriate  multimedia  encyclopedias 

• Use  content  specific  educational  software  to  support  learning 

• Use  a variety  of  nonprint  media  resources  for  directed  & independent  learning  activities 

• Use  a word  processor 

• Use  computers  to  compose  texts 

• Use  computers  to  compose  graphical  representations 

• Use  multimedia  authoring  tools 

• Use  computers  to  search  a variety  of  databases 

• Use  data  collection  probes 

• Use  calculators 

• Use  email 

• Participate  in  online  discussions 

• Use  a browser  to  navigate  the  WWW 

• Use  search  engines  to  locate  & access  remote  information 

• Use  productivity  tools  & peripherals  to  support  personal  productivity 

• Communicate  about  technology  using  accurate  terminology 

• Demonstrate  knowledge  of  video  conventions 

• Demonstrate  knowledge  of  computing  conventions 

• Give  examples  of  situations  where  more  information  is  needed  to  solve  a problem 

• List  ideas  for  identifying  A finding  needed  information 

• Describe  several  ways  to  organize  information 

• Name  a variety  of  formats  for  presenting  different  kinds  of  information 

• Practice  responsible  use  of  technology  systems  A software 

• Work  cooperatively  using  technology 

middle  school 

• Use  content  specific  computer  simulations  to  support  learning 

• Use  exploratory  environments  to  support  learning 

• Use  graphing  calculators 

• Use  computers  to  search  the  Internet 

• Use  communications  & computing  technologies  to  locate  information  efficiently 

• Brainstorm  a range  of  information  sources  to  meet  a specific  information  need 

• Use  productivity  tools  & peripherals  to  support  group  collaboration 

high  school 

• Take  notes  & gather  data  from  nonprint  sources 

• Use  online  information  resources  for  research 

• Use  technology  tools  Y resources  for  managing  personal/professional  information 

• Use  technology  tools  & resources  for  communicating  personal/professional  information 

• Use  online  resources  to  enhance  personal/professional  productivity 

• Discuss  real  world  applications  of  expert  systems 

• Discuss  real  world  applications  of  intelligent  agents 

• Discuss  real  world  applications  of  simulations 

• Explore  a range  of  sources  to  find  information  of  personal/professional  interest 

• Use  & cite  others'  work  appropriately  & correctly 

Critical  literacies  are  competencies  concerned  with  the  ability  to  interpret,  critique,  and  evaluate  nonprint  texts,  to  synthesize 
information  found  within  them,  and  to  apply  them  in  solving  problems  and  increasing  personal  understandings.  They  include 
such  abilities  as  making  sense,  analyzing,  evaluating,  applying,  etc.  The  cumulative  critical  literacies  students  should  have  by 
completion  of  elementary,  middle,  and  high  school  are  listed  below. 

elementary 

• Make  sense  of  a variety  of  graphical  representations 
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• Demonstrate  listening  skills 

• Make  sense  of  films  A videos 

• Make  sense  of  simple  computer  programs 

• Make  sense  of  WWW  pages 

• Use  computer-based  puzzles  A logical  thinking  software  to  support  problem  solving  activities 

• Use  a variety  of  graphical  representation  to  acquire  Information 

• Use  audio  presentations  to  acquire  information 

• Use  films  A videos  to  acquire  information 

• Use  a variety  of  computer-based  resources  to  acquire  information 

• Discuss  A critique  audio  presentations 

• Discuss  A critique  films  A videos 

• Discuss  A critique  computer  programs 

• Discuss  A critique  WWW  sites 

• Use  a variety  of  nonprint  n^edla  to  build  self-knowledge 

• Use  a variety  of  nonprint  media  to  understand  cultures 

• Distinguish  between  accurate  A Inaccurate,  complete  A incomplete  information 

• Define  A/or  give  examples  of  accuracy,  relevance,  comprehensiveness,  appropriateness  A bias  in  non-print  media  and/or 
electronic  resources 

• Distinguish  between  fact,  opinion  A point  of  view  in  a variety  non-print  media  and/or  electronic  resources 

• Evaluate  the  accuracy,  relevance,  appropriateness  A bias  of  a variety  of  nonprint  resources 

• Recognize  A make  sense  of  new  information  in  nonprint  sources 

• Define  A/or  give  examples  of  equity  in  access  to  information  resources  A technologies 

• Define  A/or  give  examples  of  intellectual  freedom 

• Define  A/or  give  examples  of  respecting  intellectual  property  rights 

• State  the  main  points  of  school  policy  concerning  computing  A communications  technologies 

• Discuss  basic  issues  related  to  the  responsible  use  of  technologies 

• Demonstrate  positive  social  A ethical  behaviors  when  using  technology 

• Discuss  the  advantages  A disadvantages  of  common  uses  of  technologies 

• Select  and  use  appropriate  technology  tools  to  complete  a variety  of  tasks 

middle  school 

• Analyze  & explain  graphical  representations 

• Analyze  & explain  audio  presentations 

• Analyze  & explain  films  & videos 

• Distinguish  the  uses  of  graphical  representations 

• Distinguish  the  uses  of  audio  presentations 

• Distinguish  the  uses  of  films  & videos 

• Distinguish  the  uses  of  WWW  sites 

• Evaiuate  nonprint  media 

• Synthesize  information  from  nonprint  media 

• Select  & use  appropriate  technology  tools  & resources  to  complete  a variety  of  tasks 

• Explain  how  Inaccurate  or  incomplete  information  can  lead  to  faulty  conclusions 

• Compare  & contrast  nonprint  information  sources  for  accuracy,  relevancy,  comprehensiveness  & bias 

• Create  a plan  to  access  information  that  meets  a particular  need 

• Analyze  information  from  a variety  of  sources  to  determine  its  applicability  to  a specific  problem 

• Assess  both  the  process  & the  product  of  a specific  information  search 

• Use  multiple  & diverse  information  sources  to  answer  questions  or  resolve  problems 

• Demonstrate  positive  ethical  & legal  behaviors  when  using  information  & technology  resources 

• Demonstrate  knowledge  of  current  information  technologies 

• Demonstrate  an  understanding  of  the  effects  technological  changes  have  on  society  & the  workplace 

• Discuss  the  consequences  of  the  misuse  of  information  &/or  technologies 

• Demonstrate  an  understanding  of  the  concepts  underlying  hardware,  software  & connectivity  tools 

• Apply  strategies  for  identifying  & solving  routine  hardware  & software  problems 

high  school 

• Choose  the  most  appropriate  formats  for  presenting  a range  of  information 

• Critique  & evaluate  advertising  campaigns  for  a variety  of  products 

• Recognize  & compare  different  media  genres 

• Evaluate  the  strengths  & weaknesses  of  various  creative  presentations 

• Judge  the  quality  of  one's  own  information  products  & solutions 

• Judge  the  accuracy  & completeness  of  information  & support  those  conclusions 

• Appropriately  distinguish  between  fact,  opinion  & point  of  view  in  one's  own  nonprint  work 
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• Evaluate  the  electronic  information  seeking  process  as  it  evolves  & make  appropriate  adjustments 

• Discuss  & evaluate  technology-based  options  for  lifelong  learning 

a Identify  capabilities  & limitations  of  current  & emerging  technologies 

a Assess  the  potential  of  current  & emerging  technologies  to  address  personal  & workplace  needs 

a Make  informed  choices  among  technology  systems,  resources  & services 
a Analyze  the  advantages  & disadvantages  of  the  widespread  use  of  technology  in  society 
a Advocate  for  ethical  & legal  behaviors  when  using  information  technology 

Construction  skills  are  competencies  involving  the  creation  and  use  of  nonprint  texts  for  developing  ideas  and  opinions,  for 
communicating  and  collaborating  with  others,  and  for  enhancing  problem  solving  and  personal  fulfillment.  Construction 
skills  focus  on  capabilities  for  composing,  developing,  integrating,  presenting,  etc.  The  construction  skills  students  should 
have  by  completion  of  elementary,  middle,  and  high  school  are  presented  in  the  list  which  follows.  The  skills  should  be 
understood  as  cumulative. 


elementory 

• Use  computer-based  writing  tools  to  communicate  thoughts,  ideas  A stories 

• Use  computer-based  drawing  tools  to  illustrate  thoughts,  ideas  A stories 

• Use  digital  cameras  to  illustrate  thoughts,  ideas  A stories 

• Use  multimedia  authoring  tools  in  the  creation  of  knowledge  products 

• Use  presentation  software  in  the  creation  of  knowledge  products 

• Use  WWW  authoring  tools  in  the  creation  of  knowledge  products 

• Use  audio  tapes  for  self -directed  A/or  extended  learning 

• Use  videos  for  self-directed  A/or  extended  learning 

• Use  technology  resources  for  self-directed  A/or  extended  learning 

• Use  technology  resources  for  problem  solving 

• Create  nonprint  media  for  personal  fulfillment 

• Explain  basic  strategies  for  revising,  improving  A updating  nonprint  media 

• Use  telecommunications  technologies  to  participate  in  collaborative  projects 

• Work  collaboratively  to  seek  A/or  communicate  information  in  nonprint  formats 

• Work  collaboratively  to  create  simple  nonprint  information  products 

midclle  school 

• Use  nonprint  media  to  create  knowledge 

• Use  nonprint  media  to  communicate 

• Differentially  organize  information  according  to  differing  problems 

• Choose  appropriate  media  formats  for  presenting  a variety  of  information 

• Use  nonprint  media  to  create  information  products  related  to  topics  of  personal  interest 

• Express  information  & ideas  creatively  In  nonprint  formats 

• Design,  develop  & present  videotapes  that  communicate  curriculum  concepts 

• Use  telecommunications  & collaborative  tools  to  collaborate  with  peers,  experts  & others  on  curriculum  related  problems 

• Work  collaboratively  over  distance  to  create  & evaluate  complex  information 

• Select  & apply  appropriate  strategies  for  revising,  improving  & updating  work 

high  school 

• Use  technology  to  collaborate  with  others  to  contribute  to  a content-related  data  base 

• Select  & apply  technology  tools  to  support  research  in  content  learning 

• Select  & apply  technology  tools  for  decision  making  in  content  learning 

• Select  & apply  technology  tools  for  problem  solving 

• Select  & apply  technology  tools  for  information  analysis 

• Differentially  organize  information  so  that  it  is  effectively  presented  in  a single  nonprint  product 

• Express  Ideas  creatively  &/or  uniquely  in  integrative  nonprint  formats 

• Collaboratively  create  complex  information  over  distance 

• Collaboratively  evaluate  complex  information  over  distance 

• Devise  creative  ways  to  use  information  to  resolve  problems  &/or  answer  questions 


Educational  Significance 

To  reiterate,  the  above  lists  of  nonprint  media  and  technology  literacy  competencies  are  offered  as  a starting  point.  We  hope 
they  will  begin  a discussion  of  what  one  should  know  and  what  one  should  be  able  to  do  to  be  considered  literate  in  the  21** 
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century.  Thus,  they  are  also  intended  to  precipitate  discussions  of  how  we  should  be  teaching  and  learning  about  literacy  in 
our  schools  today. 

Most  importantly,  we  hope  the  lists  will  encourage  educators  to  extend  traditional  notions  of  literacy  to  include 
critical  and  creative  uses  of  images,  video,  and  electronic  texts  of  all  sorts.  We  hope  they  will  encourage  educators 
to  stop  treating  technology  as  a separate  subject  to  be  taught,  often  as  a “pull-out,”  and  to  start  treating  the  use  of 
electronic  media  as  a form  of  literacy  to  be  integrated  across  the  curriculum.  In  particular,  we  hope  that  educators 
will  take  special  note  of  the  critical  literacies  and  construction  skills  categories  and  make  mindful  efforts  to  include 
these  whenever  they  incorporate  technology  usage  in  teaching  and  learning. 

Changes  in  communications  technologies  over  the  past  century  have  created  a world  culture  that  has  extended  and 
reshaped  our  symbolic  environment.  Today,  most  Americans  receive  the  majority  of  their  news,  information,  and 
entertainment  through  electronic  sources.  It  only  makes  sense  that  we  should  teach  our  children  to  use  those  sources 
well,  and  that  we  make  the  use  of  electronic  media  an  integral  part  of  the  daily  activities  in  every  classroom  in  this 
country.  The  nonprint  media  and  technology  literacy  standards  herein  presented  offer  a starting  point  for  discussion 
towards  such  ends. 
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Abstract:  This  paper  presents  the  quantitative  results  of  a study  that  examined  the  effects  of 
the  structure  of  learning  activities  and  the  learner  characteristics  of  prior  knowledge,  ability 
and  attitudes  on  students*  acceptance  of  control  opportunities  while  using  hypermedia 
software  and  consequent  learning  outcomes.  Participants  in  the  study  were  101  year  8 
students  in  two  Australian  schools.  A confirmatory  factor  analysis  of  a model  that  theorized 
the  relationships  between  the  variables  did  not  fit  the  data  well,  but  careful  substantively  based 
amendment  of  the  model  saw  the  introduction  of  a new  factor  that  resulted  in  an  improved  fit. 
The  new  factor  was  that  of  the  school  which  students  attended,  a factor  that  proved  to  have 
greater  influence  than  the  hypothesized  variables.  This  paper  is  supported  by  a related  article 
which  reports  the  results  of  qualitative  data  collected  for  the  same  study. 


Introduction 

Catering  for  the  needs  of  individual  students  by  offering  some  control  over  learning  is  a commendable  ideal  that 
has  been  seen  by  many  as  offering  opportunities  to  help  students  reach  their  potential.  Although  research  in  the 
area  of  learner  control  has  often  been  surrounded  by  a great  deal  of  enthusiasm,  the  reality  of  the  situation  is 
that  research  findings  in  learner  control  are  mixed  and  inconclusive.  There  are  those  who  claim  that  there  are 
no  positive  findings  to  support  the  enthusiasm  (Nienmiec,  Sikorski  & Walberg,  1996;  Reeves,  1993;  Snow, 
1980)  and  others  who  suggest  the  benefits  are  limited  by  contexts  in  which  control  is  offered  and  by  the 
characteristics  of  learners  (Dillon  and  Gabbard,  1 998). 

One  of  the  main  problems  in  the  area  of  learner  control  is  the  complexity  of  interactions  that  exist  between 
individual  learners’  characteristics  and  the  environments  in  which  learning  takes  place.  The  number  of  learner 
characteristics  is  substantial,  as  is  the  variety  of  learning  environments  created.  Many  studies  have  been 
undertaken,  but  they  are  difficult  to  compare  because  they  use  different  combinations  of  variables.  For  this 
reason  alone  it  is  not  surprising  that  results  are  mixed,  inconclusive  and  contradictory. 

There  are  individual  characteristics  that  have  been  studied  to  a greater  degree  than  others  that  do  offer  some 
more  consistent,  positive  findings  and  which  suggest  promising  trends  or  directions  for  classroom  practitioners 
to  cater  for  needs  of  individual  learners.  Some  studies  have  found  that  learners  with  varied  abilities  (Gray, 
1987;  Hooper,  Temiyakam  & Williams,  1993;  Kinzie  et  al.,  1988;  Singhanayok  & Hooper,  1998),  learning 
styles  (Ferguson,  1988;  Klein  & Keller,  1990;  Santiago  & Okey,  1992),  preferences  (Carrier  & Williams,  1988; 
Hannafin  & Sullivan,  1995;  Hannafin  & Sullivan,  1996;  Hicken,  Sullivan  & Klein,  1992)  and  levels  of  prior 
knowledge((Gay,  1986;  Gay,  Trumbull  & Mazur,  1991;  Kinzie  & Sullivan,  1989;  Ross,  Morrison  & O’Dell, 
1 989;  Shin,  Schallert  & Savenye,  1 994;  Stephenson,  1 992)  differently  accept  levels  of  control. 


1120 


The  importance  of  learner  control  research  is  amplified  by  technological  developments  allowing  the  creation  of 
interactive  learning  environments  that  are  being  adopted  rapidly  and  enthusiastically,  with  considerable 
investment  of  financial  resources  and  practitioner  time  and  effort.  Hypertext,  hypermedia  and  multimedia 
environments  offer  masses  of  information  that  can  be  navigated  according  to  the  interests  and  needs  of  users. 
Multiple  links  and  reference  points  provide  a network  of  possible  paths  for  investigation.  The  problem 
associated  with  the  use  of  these  systems  is  that  although  learners  are  provided  a high  degree  of  control,  they  can 
become  overwhelmed  with  the  number  and  scope  of  choices  available  to  them.  Rather  than  accepting  the 
myriad  of  opportunities  offered  by  hypermedia  learners  may  utilize  only  a small  number  of  the  choices 
available  to  them,  failing  to  make  effective  use  of  the  software.  It  is  apparent  that  if  we  are  to  continue  to  adopt 
the  use  of  hypertext,  hypermedia  and  multimedia  products  in  education  then  there  is  a need  to  identify  how  we 
might  support  students’  use  of  the  software  and  encourage  them  to  make  effective,  productive  and  independent 
use  of  such  systems. 

One  approach  to  supporting  students  is  to  attend  to  the  design  of  the  user-interface  of  software  packages, 
finding  ways  to  provide  help  for  students  within  the  software.  This  may  be  done  by  developing  adaptive 
systems  that  respond  to  user  input,  limiting  available  options  by  providing  a smaller  number  of  options  to 
novices  and  increasing  available  navigational  choices  as  expertise  and  learning  in  a domain  develop.  Another 
approach  to  support  student  use  of  complex  software  packages  effectively  is  to  consider  the  learning 
environment  in  which  the  software  is  used  and  identify  classroom  practices  and  develop  support  materials  that 
will  encourage  and  support  effective  learning.  Rather  than  developing  software  that  attempts  to  cater  for  the 
needs  of  all  students,  it  would  be  better  to  utilize  the  talents  of  teachers  in  catering  for  individual  needs  while 
making  powerful  resources  available  to  them  and  their  students.  The  main  aim  of  the  research  presented  in  this 
article  is  to  utilize  existing  hypermedia  products  available  on  the  educational  software  market  and  find 
procedures  and  strategies  that  can  be  used  in  the  classroom  that  will  encourage  students  to  make  effective, 
productive  and  independent  use  of  such  systems. 

Research  Model  and  Study  Description 

There  are  a number  of  studies  that  have  produced  some  arguably  consistent  findings  regarding  particular 
variables  concerned  with  provision  of  learner  control.  These  include  the  amount  of  control  afforded  to  learners, 
the  level  of  prior  domain  knowledge  and  prior  system  knowledge,  ability,  locus-of-control,  and  students’ 
learning  goals  or  task  orientation.  The  findings  for  these  variables  have  been  reported  in  terms  of  achievement 
outcomes,  motivational  and  attitudinal  outcomes  and  learning  efficiency.  As  it  is  not  possible  to  effectively 
address  all  of  the  above  mentioned  variables  in  a single  study,  a subset  of  variables  form  the  focus  of  this 
research.  Those  examined  were  selected  because  they  have  a history  of  research  findings  that  provide  a sound 
theoretical  base. 

A theoretical  model  based  on  the  variables  of  interest  and  their  hypothesized  effects  on  each  other  can  be 
expressed  in  the  form  of  a covariance  structure  model.  Such  a model  is  presented  in  Figure  1.  The  model 
illustrates  the  theory  that  prior  domain  knowledge,  learning  activity  structure,  ability  and  attitude  causally  affect 
the  acquisition  of  knowledge  and  students’  post  attitudes  indirectly  through  a learner's  acceptance  of  control 
opportunities.  Also,  prior  domain  knowledge  directly  affects  acquisition  of  task-specific  and  non-task-specific 
domain  knowledge  and  software  structure  knowledge;  ability  directly  affects  the  acquisition  of  task-specific  and 
non-task-specific  domain  knowledge  and  acquisition  of  knowledge  of  software  structure;  and,  prior  attitude 
directly  affects  post  attitude.  While  it  was  expected  that  the  weights  of  the  direct  effects  would  be  fairly  strong, 
for  example  prior  attitude  would  have  a strong  effect  on  post  attitude,  it  was  also  predicted  that  the  weights  of 
the  indirect  effects  mediated  by  acceptance  of  control  opportunity  would  not  be  insubstantial. 
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Figure  1:  Covariance  structure  model  for  the  effects  of  attitudes,  prior  knowledge,  ability  and  experimental 
grouping  on  acceptance  of  control  opportunity,  attitudes  and  acquisition  of  three  types  of  knowledge 

To  investigate  the  variables  of  interest  identified  by  the  research  questions  and  hypotheses,  a 3x2  factorial 
design  with  three  levels  of  structure  (worksheets  of  high,  medium  and  low  structure)  and  two  levels  of  prior 
knowledge  (high  and  low)  was  utilized.  So  that  an  authentic  educational  situation  could  be  the  focus  of 
research  intact  classes  were  used  for  grouping.  In  addition,  as  the  research  literature  indicates  that  boys  tend  to 
dominate  activity  in  coeducational  computer-based  activities  (Culley,  1990;  Fatouros,  Downes  & Blackwell, 
1994;  Hattie  & Fitzgerald,  1987),  single  gender  schools  were  used  so  as  to  remove  the  effects  of  gender 
interaction  and  increase  the  generalizability  of  the  findings.  Rather  than  fully  replicating  the  study  with  six 
groups  in  both  boys’  and  girls’  schools,  the  strategy  of  fractional  replication  was  employed  in  this  design. 
This  was  done  to  remove  the  effects  of  gender  interactions  as  an  uncontrolled  variable,  while  allowing  the 
practicalities  of  conducting  the  study  for  each  gender  to  be  met.  Participants  in  the  study  were  101  year  8 
students  in  two  Australian  schools  who  examined  software  on  the  topic  of  the  Australian  federal  parliament. 

Measures  for  quantitative  data  are  shown  in  the  rectangles  in  (Fig.l).  Data  were  collected  by  asking  students  to 
complete  questionnaires  for  attitude  measures  and  written  tests  of  knowledge  to  help  determine  levels  of 
domain  specific,  non-domain-specific  and  software-structure  knowledge.  In  addition  student  interactions  were 
recorded  by  the  software  in  the  form  of  audit-trails  that  stored  information  about  every  active  choice  made  in 
the  software  during  five  40  minute  lessons.  To  supplement  the  quantitative  data  that  aimed  to  identify  ‘what’ 
students  did,  qualitative  data  in  the  forms  of  student  interviews  and  analyses  of  navigational  choices  aimed  to 
identify  ‘why’  students  made  particular  choices  and  adopted  particular  navigational  strategies.  Qualitative 
results  are  discussed  in  a related  article. 

Results 

Quantitative 

A confirmatory  factor  analysis  of  the  covariance  structure  model  shown  in  (Fig.l)  produced  goodness-of-fit 
indices  that  were  very  good  for  the  left-hand  side  measurement  model  (Goodness  of  Fit  (GFI)=.96,  Comparative 
Fit  Index  (CFI)=1.0,  Standardized  Root  Mean-square  Residual  (SRMR)=.05 1),  but  not  acceptable  for  the  right- 
hand  side  measurement  model  (GFI=.85,  CFI=.91,  SRMR=.099).  With  the  poor  fit  of  the  right-hand  side 
measurement  model  it  was  not  surprising  that  the  structural  model,  whose  analysis  is  based  on  the  two 
measurement  models  was  less  than  ideal  (GFI=.72,  CFI=.74,  SRMR=.  12). 

Careful,  substantively  based  post-hoc  analysis  resulted  in  the  identification  of  a modified  model  that  produced  a 
better  fit  than  the  originally  proposed  model  (GFI=87,  CFI=.91,  SRMR=.07l),  and  which  could  be  described  as 
a marginally  good  fit.  This  alternative  model  is  shown  in  (Fig.  2),  including  the  standardized  parameter 
estimates  and  error  terms.  The  most  important  feature  of  the  alternative  model  produced  was  that  a new  factor 
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was  introduced,  and  that  this  factor  produced  the  highest  parameter  estimate  between  an  exogenous  variable  and 
acceptance  of  control  opportunity  (.62,  p<.001).  This  variable  was  introduced  due  to  observed  differences 
between  the  navigational  approaches  of  students  at  the  two  schools  when  the  study  was  being  undertaken, 
supported  by  statistically  significant  differences  on  the  measures  of  acceptance  of  control  opportunity  between 
the  two  school  groups.  The  implications  of  this  introduced  factor  are  addressed  in  the  later  discussion. 


Figure  2:  Alternative  model  showing  completely  standardized  parameter  estimates  and  error  terms. 

Note:  # indicates  the  y-value  for  each  factor  that  was  set  to  1.0  to  establish  the  scale  of  each  factor  resulting  in 
no  test  of  statistical  significance  calculated  by  LISREL.  * p<.05,  **  p<.01,  ***  p<.001 

Qualitative 

While  the  quantitative  results  identified  that  the  school  students  attended  was  an  important  factor  influencing 
choices  made  in  the  use  of  the  software,  this  methodology  can  not  identity  why  this  may  have  been  the  case. 
An  analysis  of  the  qualitative  data  for  this  same  study  identified  three  possible  factors  that  may  have  influenced 
the  strength  of  the  school  factor:  specifically  these  were,  the  gender  differences  of  students  at  the  two  schools, 
the  epistemological  beliefs  of  students  and  the  different  cultures  and  learning  environments  that  existed  at  the 
two  schools.  A more  comprehensive  report  of  the  qualitative  findings  is  presented  and  discussed  in  detail  in  the 
supportive  short  paper  ‘The  Emergence  of 'School'  as  a Factor  Influencing  Patterns  of  Navigational  Choices  in 
Hypermedia’  in  this  ED-Media  2000  conference  proceedings. 

Discussion 

The  research  reported  in  this  article  identifies  an  unexpected  factor,  that  of  the  school  which  students  attend,  as 
having  a greater  influence  on  students’  software  use  than  those  identified  in  the  literature  as  being  potentially 
influential.  Unfortunately  the  adoption  of  a fractional  replication  in  the  3x2  factorial  design,  based  on  the 
gender  of  students,  did  not  allow  the  identification  of  specific  variables  that  may  have  caused  the  strength  of  the 
factor.  Fractional  replication  was  used  in  this  study  to  remove  the  effects  of  gender  domination  in  software-use, 
but  in-fact  caused  complications  in  the  light  of  the  unexpected  outcomes. 

The  unanticipated  finding  that  the  school  which  students  attend  can  have  an  effect  on  navigational  choices  in 
software  does  have  important  implications  however.  The  identification  of  this  variable  as  being  potentially 
influential  on  students’  software  choices  is  important  in  the  design  of  any  learning  environment  and  to  the 
policy  decisions  made  about  education  and  technology.  The  obvious  question  that  arises  is  that  of  ‘...  what  are 
the  important  factors  in  the  schools  that  influenced  students  use  of  software?’  The  qualitative  results  from  this 
study  suggest  three  possible  contributing  factors,  specifically  those  of  the  genders  of  the  students,  the 
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epistemological  beliefs  of  the  students  and  the  learning  environments  created  by  the  cultures  of  the  schools. 
These  potentially  contributing  factors  are  discussed  in  greater  detail  in  the  supportive  article  reporting  the 
qualitative  results  of  this  study,  and  are  also  >vorthy  of  further  attention  in  subsequent  research  studies. 
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Abstract:  Using  Web  Pages  for  supplemental  materials  in  the  classroom  is  very  popular.  However, 
some  Web  Pages  are  not  suitable  for  educational  use  as  their  stuffy  description.  Our  research  aims 
at  translating  these  formal  web  pages  into  plainer  Japanese  sentences.  We  provide  the  system  as  a 
proxy  program  running  on  UNIX  workstation.  The  main  part  of  this  system  consists  of  three  phases 
- morphological  analysis,  transformation,  and  mark-up.  It  is  expected  that  students  can  read  web 
pages  easily  by  using  our  system.  At  the  beginning  of  this  paper,  we  describe  the  overview  of  our 
proxy  program  and  implementation  of  it.  Then  we  explain  many  functions  of  our  proxy  program. 
The  functions  are  divided  into  two  categories  - “linguistic  functions”  and  “multimedia  and 
appearance  functions.”  At  last,  a brief  experiment  result  is  also  mentioned. 


Introduction 

As  the  Internet  has  come  into  wide  use,  the  WWW  is  rapidly  becoming  a great  knowledge  resource  in  the 
educational  domain.  Many  teachers  use  a variety  of  web  pages  in  their  classroom.  There  are  many  culturally 
oriented  web  pages,  for  instance,  those  pages  produced  by  museums,  galleries,  libraries,  etc.  They  are  suitable  for 
supplemental  materials  in  the  classroom.  However,  the  purpose  of  these  pages  is  not  always  for  educational  use 
only.  Often,  an  overly  formal  or  “stuffy”  language-style  is  used  in  conveying  information.  Students  sometimes 
hesitate  to  read  these  formal  language  descriptions.  Our  research  aims  at  translating  these  formal  web  pages  into 
plainer  language  in  order  to  better  suit  students'  abilities  and  understanding. 

We  have  implemented  this  translation  program  as  a proxy  program  on  our  proxy  server.  The  reason  is  that  we  do  not 
have  to  distribute  our  program  nor  install  it  on  students’  PCs.  Another  merit  of  this  implementation  is  that  we  can 
administer  our  proxy  program. 

The  following  sections  in  this  article,  we  will  discuss  many  Japanese-specific  problems,  as  our  target  language  is 
Japanese  and  we  assume  the  input  of  our  proxy  program  is  the  web  pages  created  by  Japanese  cultural  sites  and  the 
output  web  pages  would  be  read  by  Japanese  students.  However,  we  also  explain  the  language- in  dependent 
functions  of  our  proxy  program,  such  as  appearance  functions. 

In  the  following  sections,  we  explain  our  proxy  program  overview,  functions,  a brief  experiment  and  its  result.  Then 
we  discuss  the  merits  and  the  demerits  of  our  proxy  program  in  the  considerations  section.  Finally,  we  describe  our 
future  plans  in  the  conclusions  section. 
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A Proxy  Program  Overview 


Figure  1 shows  the  overall  system  of  our 
proxy  program.  When  students  access  any 
web  pages  that  they  have  difficulty  with,  they 
can  use  our  proxy  server  to  translate  the  web 
pages  into  plainer  language.  Once  they  have 
set  the  proxy  server  to  the  URL  of  our  site  in 
their  web  browser  setting,  they  don’t  have  to 
be  conscious  of  our  proxy  program. 

Figure  2 shows  the  flowchart  of  our  proxy 
program.  As  shown,  the  program  consists  of 
following  three  phases. 

(1)  Sentence  resolution 

This  phase  resolves  an  input  HTML 
document  into  sentences  and  HTML  tags. 

(2)  Translation 

This  phase  is  composed  of  “linguistic 
translation”  and  “multimedia  and  appearance 
functions.”  We  will  describe  these  functions 
in  the  following  sections. 


Our  Proxy  Server 


Fig.  IrOverall  System 


(3)  Document  reconstruction 

In  this  phase,  translated  sentences  and  translated  HTML  tags  are  reconstructed  as  original  documents. 


Linguistic  Translation 

Linguistic  translation  process  is  composed  of  following  three  steps. 

(1)  Morphological  analysis: 

In  Japanese  sentences,  we  can  determine  the  part-of-speech  of  each  word  by  morphological  analysis. 

(2)  Transformation: 

This  is  the  main  step  of  the  linguistic  translation  process.  The  syntax  of  the  input  sentence  will  be  changed  according 
to  transformation  rules  (described  in  the  next  section). 

(3)  Mark-up: 

There  are  many  technical  terms  in  web  pages.  Our  approach  is  to  mark-up  difficult  terms  with  HTML  tags.  The  user 
can  then  open  a link  to  yahoo.com,  or  an  on-line  dictionary,  by  clicking  on  these  terms.  Optionally,  we  can  replace 
difficult  terms  with  easier  expressions.  However,  these  expressions  may  contain  the  main  theme  of  the  web  page. 
Also,  too  much  added  description  may  cause  the  reader  to  become  frustrated.  Utilizing  our  proxy  system,  users  can 
specify  the  difficulty-level  for  the  program  to  automatically  mark  up  text,  thus  giving  the  user  a number  of  options 
to  selectively  control  the  proxy  program’s  functioning.  Currently,  we  have  catalogued  approximately  6,000  Kanji 
characters,  divided  into  various  levels  of  difficulty,  in  order  to  create  optional  levels  of  automatic  selection  and 
descriptive  narration  for  the  user. 


Transformation  rules 

(1)  Compound  noun  phrase  transformation 


1127 


Fig.  2:  Program  Flowchart 


Similar  to  English,  Japanese  sometimes  uses  long  compound  noun  phrases.  Long  noun  phrases  give  readers 
serious  and  formal  impression.  This  category  of  rules  inserts  “no’’  (similar  to  “of’  in  English)  at  appropriate 
positions  in  order  to  make  a phrase  easier  to  read.  The  search  algorithm  for  the  appropriate  position  is  based  on 
[Tsutsumi  89].  Figure  3(a)  shows  an  example  of  compound  noun  phrase  transformation. 

(2) D Formal  expression  replacement 

There  are  several  formal  expressions  in  Japanese  sentences.  "Formal  expression  replacement"  replaces 
those  expressions  which  contain  adverbs  and  auxiliary  words,  as  they  are  function  words.  Function  words  create  a 
"stuffier"  sentence  than  content  words  do.  This  rule  replaces  the  formal  syntax,  which  is  determined  by  adverbs  and 
auxiliary  words,  with  a plainer  syntax. 

(3) D  Sentence  length  adjustment 

If  a sentence  has  more  than  ten  phrases,  the  sentence  will  be  divided  at  a conjunctive  auxiliary  into  two 
sentences.  This  rule  will  supply  a conjunction  between  the  sentences. □ Figure  3(b)  shows  an  example  of  sentence 
length  adjustment. 


Multimedia  and  appearance  functions 

(1)  Iconization  of  hidden  image 


Web  pages  have  many  links  to  image  with  “here”  or  “this.”  For  example,  “You  can  see  the  castle  by  clicking 
HERE,”  This  type  of  web  pages  is  often  dull.  Therefore,  we  replace  “HERE”  with  “FIG.  1”  and  place  the  iconized 


(a)  Compound  noun  phrase 
transformation 
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(b)  Sentence  length  adjustment 


Fig.  3 Examples  of  linguistic  functions 
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(a)  Original  web  page(s) 


(b)  Result  web  page 


Fig.  4 An  example  of  appearance  functions 


image  of  linked  one.  Students  will  be  interested  in  the  graphical  pages.  Figure  4 illustrates  an  example  of 
iconization  of  hidden  image.  If  the  classroom  computers  are  connected  to  the  Internet  via  low  speed  line,  loading  the 
graphical  pages  needs  students’  patience.  We  provide  an  option  for  this  problem.  After  students  choose  this  option, 
hidden  image  iconization  function  will  not  be  effective. 


(2)  Font  size  replacement 
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We  often  use  small  font  in  order  to  show  much  information  in  one  page.  However,  too  small  fonts  make  readers 
be  tired  of 

(3)  Reformation  of  list-items 

Students  must  push  a scroll  button  many  times,  if  there  are  too  many  items  in  a web  page.  This  function  divides 
list-items  into  several  pages,  and  students  can  get  next  items  by  clicking  “forward  button”  created  by  this  function. 


Pedagogical  Evaluation 

We  have  tested  our  proxy  program  briefly.  The  examinees  are  five  junior  college  students.  We  first  showed  them 
translated  web  pages  and  asked  whether  the  web  pages  are  easy  to  read  or  not.  Then  we  showed  them  the  original 
web  pages  and  got  their  impressions. 


Table  1 

'The  translated  web  pages  are  easier  to  read.” 

Agree 

Disagree 

Not  answered 

Web  Page  A 

3 

1 

1 

Web  Page  B 

4 

1 

0 

Web  Page  C 

5 

0 

0 

Web  Page  D 

3 

2 

0 

Some  of  the  impressions  are; 

“It  is  clear  to  read.” 

“Iconized  images  are  good.  If  a web  page  is  full  of  characters,  I would  hesitate  to  read  it.” 

“Too  much  HI  RAG  AN  A characters  are  messy  to  read.” 

Considerations 

In  this  section,  we  discuss  the  advantages  and  aspects  of  our  approach. 

(1)  Usage 

Students  can  easily  use  our  program  because  our  program  is  developed  as  a proxy  program.  Once  students  have 
specified  our  server  as  a proxy  server,  they  will  get  translated  web  pages  without  any  operation.  This  is  very 
important  in  elementary  school  students.  Most  of  them  do  not  have  computer  operation  skill  yet. 

If  the  classroom  PCs  are  protected  by  a firewall  and  they  must  use  the  proxy  server  inside  the  organization,  a 
way  to  use  our  proxy  program  is  the  proxy  chain  from  the  proxy  server  inside  the  firewall  to  our  proxy  server. 
However,  you  have  to  ask  your  system  administrator  to  fix  it. 

(2)  Performance 

Our  proxy  program’s  process  is  light  because  we  only  use  morphological  analysis.  Unlike  semantic  analysis  or 
syntax  analysis,  the  morphological  analysis  is  very  fast  and  reliable.  As  our  program  is  running  on  our  server,  we  do 
not  need  high-speed  client  PCs.  On  the  other  hand,  when  many  students  use  our  proxy  program  at  the  same  time,  the 
tum-around-time  would  be  very  long. 

(3)  Transformation  Rules 

We  have  defined  several  transformation  rules.  However,  we  have  to  add  some  rules  in  order  to  improve  the 
readability  of  the  translated  web  pages. 

(4)  Dictionaries 

We  provide  three  dictionaries,  analysis  dictionary,  Chinese  character  dictionary  and  Japanese-language 
dictionary.  The  analysis  dictionary  cannot  be  viewed  from  students.  It  is  used  for  morphological  analysis.  Other 
dictionaries  can  be  accessed  via  WWW.  Chinese  character  dictionary  is  also  used  for  translation  level  setting. 
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(5)  Adaptability  to  the  multimedia  technology  progress 

The  multimedia  technology  has  been  rapidly  progressing  these  days.  We  have  incorporated  an  adaptive 
mechanism  for  future  changes  in  multimedia  technologies  into  our  proxy  program.  To  be  concrete,  our  proxy 
program  has  an  action  table  in  which  each  file  type  is  registered  as  an  index  of  table  with  a program  name.  When  the 
proxy  program  detects  any  file  type  in  the  table,  a corresponding  program  will  be  activated.  When  your  web  browser 
cannot  handle  Quicktime  movies,  you  can  see  the  movies  in  MPEG  format,  if  you  have  a format  conversion 
program. 

(6)  Mark-up 

To  mark  up  difficult  words  is  for  searching  for  words  in  a dictionary  or  searching  for  related  items  in  web 
pages.  It  is  necessary  to  build  a web-based  dictionary  for  users  of  this  system  in  order  to  use  this  easily. 


Conclusions 

In  this  paper,  we  have  described  our  proxy  program  to  translate  web  pages  into  plainer  language.  Our  program  is 
designed  to  help  students  utilize  cyberspace  as  an  educational  tool  for  sourcing  out  information  related  to  classroom 
presentations  and  learning.  Using  our  proxy  program,  students  will  not  have  to  hesitate  at  formal  language,  as  words 
that  are  difficult  to  understand  are  each  marked-up  to  search  for  in  a dictionary,  or  selectively  replaced  with  a plainer 
syntax.  Also  our  program  rebuilds  messy  web  pages  to  tidy  web  pages.  These  translations  allow  young  students  to 
start  to  read  the  web  pages.  We  believe  that  our  proxy  program  can  be  an  effective  tool  for  learners  of  Japanese  as 
well  as  for  young  Japanese  students. 

Since  our  target  language  is  Japanese,  we  have  discussed  some  Japanese  specific  problems.  For  example,  difficult- 
to-read  characters  are  rarely  appeared  in  English  sentences.  However,  some  functions  of  our  program  are  language 
independent,  such  as  multimedia  and  appearance  functions. 

Our  future  plans  include  further  experiment  and  evaluation  of  our  program  in  the  classroom.  We  also  plan  to  apply 
our  proxy  program  to  other  languages. 
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Abstract:  This  paper  analyses  traditional  methods  of  teaching  Electronics  to  non-el ectronic 
engineers  and  proposes  an  alternative  educational  system  involving  theoretical  and  practical 
learning.  The  proposed  system  is  divided  into  hardware  and  software  resources.  Hardware 
resources  comprise  a PC  and  external  circuits  corresponding  to  representative  electronic 
systems.  Software  resources  provides  students  with  a unique  environment  containing 
hypermedia  theoretical  lessons,  simulation  tools  and  virtual  instrumentation.  Through 
theoretical  lessons  students  access  to  simulation  and  practical  exercises  where  they  verify  the 
behaviour  of  a real  circuits  using  virtual  instrumentation. 


Introduction 

Continuous  training  of  engineers  and  technicians  is  one  of  the  most  urgent  tasks  of  modem  industry. 
This  is  mainly  due  to  the  vertiginous  evolution  of  technology  imposing  very  demanding  competitive  levels.  That 
is  the  reason  why  both  companies  and  governmental  institutions  promote  applied  research  projects  developing 
new  teaching  methods,  contents  and  resources. 

At  present  our  research  group  is  working  on  a project  whose  objective  is  the  development  of  a system 
for  non-electronic  engineers  training  about  the  fundamentals  and  applications  of  Electronics.  Even  though  the 
project  contents  are  focused  on  a specific  technological  area  (Electronics)  the  methods  we  propose  can  be 
extended  to  other  educational  levels  and  technology  fields. 

This  project  is  a joint  venture  between  the  Institute  of  Applied  Electronics  “Pedro  Barrie  de  la  Maza”  of 
the  University  of  Vigo  and  TEFASA  S.L\  and  is  co-financed  by  the  European  Regional  Development  Fund 
(ERDF). 


Background 

Traditionally  Electronics  education  of  non-electronics  engineers  has  been  earned  out  combining 
theoretical  education  with  the  assembly  of  circuits  using  specific  measurement  instruments  like  power  supplies, 
voltmeters,  oscilloscopes,  and  signal  generators  (LAGO  93)(MIRI  93).  But  this  method  is  very  costly  and  it  is 
not  really  useful  for  non-electronics  engineers. 

An  alternative  method  is  the  system  analysis  using  kits  (COEL  96)(MICR  97)(SALE  96).  The  kits 
advantages  are  the  low  cost  and  the  possibility  of  implementing  them  when  and  where  convenient  favouring  the 
independent  learning.  Nevertheless  from  an  educational  point  of  view  the  role  of  the  kits  is  very  limited  because 
they  are  only  designed  to  show  specific  systems  running  and  not  to  carry  out  measurements  nor  explain  the 
concepts  of  system  functionality. 

The  previously  mentioned  disadvantages  can  be  overcome  using  computer  assisted  learning  (CAL) 
methods.  The  potentiality  of  computer  hypermedia  resources,  and  the  low  cost  and  widespread  diffusion  of 
computers,  convert  CAL  into  an  essential  teaching  tool.  In  the  case  of  Electronics  the  computer  can  be  used  as 
support  both  for  theoretical  and  practical  teaching.  In  this  sense  three  kinds  of  applications  can  be  pointed  out: 

- Tutorial  systems:  Mainly  oriented  toward  the  explanation  of  theoretical  concepts  and  self-evaluation 
allowing  individual  and  non-presential  learning  (CUET  94). 


[1]  Spanish  company  dedicated  to  the  development  and  marketing  of  didactic  systems. 


- Simulation  programs:  Oriented  toward  the  analysis  of  circuits  operation  modes  and  fundamentals 
avoiding  their  practical  implementation.  Using  simulation  programs  system  design  and  faults  detection  is  carried 
out  before  its  implementation  (MICR  96)(RASH  90). 

- Virtual  Instrumentation  systems  (VIS):  Made  up  of  a data  acquisition  board  connected  to  the  computer 
and  a program  controlling  the  data  acquisition  and  the  information  processing.  The  results  are  shown  on  the 
computer  screen  using  a format  that  resembles  traditional  measurement  instruments  (oscilloscopes,  voltmeters, 
signal  generators,  etc.).  From  an  economic  perspective  VIS  represent  a very  appealing  alternative  in  the  cases 
where  bandwidth  and  precision  are  not  very  important  constraints.  From  a teaching/leaming  point  of  view  VIS 
are  low  cost  laboratories  affordable  to  small  and  medium  sized  enterprises  or  to  engineers  interested  in  a 
continuous  training  at  home  (KEIT  97)(LABW  94). 

CAL  tools  can  be  used  for  courseware  development.  Courseware  combines  computers  with  real  circuit 
modules  where  students  are  able  to  measure  some  parameters  without  modifying  their  design  or  structure.  These 
systems  are  principally  based  on  methods  based  on  proposing  questions  and  analysing  several  solutions.  In  most 
cases  they  consist  of  lessons  guiding  the  analysis  of  a specific  circuit.  These  lessons  are  tutorials  linked  to 
simulation  programs  or  virtual  instrumentation  systems  to  carry  out  measures  on  pre-established  test  points  of 
the  analysed  circuit  to  verify  its  behaviour  when  some  variations  or  faults  are  introduced  (ALEC  95)(EURO  94). 

Nevertheless,  many  research  works  point  out  that  the  methods  based  on  problems  solution  and  practical 
experimentation  are  the  most  adequate  approach  for  teaching  technologies  like  Electronics  (BARK  95)(KLEE 
94)(VIAU  94).  From  this  perspective  we  considered  the  development  of  a new  product  oriented  not  only  to  the 
analysis  of  basic  circuits  as  an  introduction  to  Electronics  (What  they  are  and  how  they  work?),  but  also  showing 
different  electronic  systems  applications. 


Computer 


Figure  1:  A general  view  of  the  proposed  system. 


The  proposed  system 

The  features  of  a system  oriented  to  the  education  of  non-electronic  engineers  of  small  and  medium 
sized  enterprises  must  take  the  following  aspects  into  account: 

- Consistent  theoretical  contents  allowing  progressive  and  individual  learning. 

- Availability  of  resources  for  simulation  and  analysis  of  real  circuits,  including  faults  detection  and  changes. 

- Possibility  of  individual  (at  home)  or  co-operative  learning. 

- Low  cost. 

- Portability. 

To  meet  these  requirements  we  developed  a CAL  system  combining  theoretical  lessons  with  simulation 
and  experiments  using  adequately  chosen  electronic  modules.  The  objective  of  our  system  is  to  achieve  a 
consistent  base  of  basic  knowledge  making  the  comprehension  of  present  and  futures  technologies  easier  as  well 
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as  the  development  of  skills  for  the  verification  and  maintenance  of  electronic  equipment.  From  the  point  of 
view  of  the  system  structure,  it  can  be  divided  into  a hardware  system  and  a software  one. 

Hardware  system 

The  hardware  system  consists  of  three  basic  elements  (see  Figure  1): 

- A set  of  real  circuit  modules  related  to  each  lesson. 

- A base  to  support  the  modules. 

- A personal  computer. 

The  modules  are  circuits  oriented  to  the  introduction  of  simple  electronic  devices  or  systems  that  can  be 
interconnected  to  obtain  more  complex  ones.  Our  proposal  envisages  the  possibility  that  some  modules  be  closed 
systems  oriented  to  the  analysis  of  circuits  where  modifications  cannot  be  carried  out.  Other  modules  are  open 
systems  oriented  to  the  design  of  simple  circuits  that  engineers  themselves  can  connect  and  modify  if  applicable. 

The  base  to  support  the  modules  is  the  equipment  where  the  circuit  modules  related  to  each  lesson  are 
connected.  They  are  made  up  of: 

a)  A control  system  that  establishes  the  communication  protocol  with  the  computer,  acquires  data  when  a test 
must  be  carried  out,  or  induces  faults  in  specifics  points  of  the  analysed  circuits. 

b)  Power  supplies  and  common  circuits  (signal  conditioning  circuits,  protective  circuits,  etc.)  are  placed. 

The  personal  computer  sends  information  to  the  control  system  for  generating  test  signals  or  inducing 
faults.  It  also  receives  information  from  selected  test  points. 

Software  system 

The  software  system  is  made  up  of: 

a)  A hypermedia  application^  oriented  toward  the  presentation  and  evaluation  of  contents. 

b)  Simulation  tools  to  facilitate  the  analysis  of  circuits. 

c)  Virtual  instrumentation  (oscilloscope,  logic  analyser,  voltmeter  and  functions  generator)  for  data  acquisition 
from  different  test  points. 

Access  to  these  resources  is  carried  out  from  a main  program  (user  interface)  that  resembles  a Windows 
browser.  In  this  way  the  user  is  provided  with  a familiar  environment  avoiding  additional  learning.  Besides,  a 
multiplatform  system  (Java  interface)  is  used  to  guaranty  its  diffusion  and  the  implementation  of  new  functions 
not  foreseeable  in  the  first  phase  of  the  project,  such  as  the  use  of  networks  like  Internet. 

Hypermedia  application  is  the  core  of  the  system.  It  is  made  up  of  theoretical  lessons  complemented 
with  simulation  exercises  and  experiments.  Each  lesson  has  hyperlinks  permitting  access  to  other  lessons,  to  the 
simulator  or  to  the  virtual  instrumentation  system. 

At  present  the  contents  of  the  system  are  oriented  toward  the  following  areas  of  Electronics: 

- Basic  electronic  devices  and  circuits. 

- Analog  circuits  and  systems  and  their  applications. 

- Digital  circuits  and  systems  and  their  applications. 

- Introduction  to  embedded  electronic  systems:  Principles  and  applications  of  microcontrollers. 

Future  versions  will  extend  the  system  to  other  Electronic  fields. 

To  guarantee  progressive  learning  information  is  structured  from  general  to  more  specific  concepts 
using  an  original  method  for  complex  technologies  education  (VALD  97)(VALD  99).  Users  can  navigate 
through  the  application  linking  related  information  which  broadens  their  previous  knowledge.  Figure  2 shows 
some  screens  of  the  application  that  illustrate  this  methodology. 

The  first  screen  defines  Electronics  using  a pyramidal  structure  with  four  levels  corresponding  to  the 
main  topics  of  the  application:  solid  state  electric  phenomena,  electronic  components,  electronic  circuits  and 
electronic  systems.  Each  part  of  the  pyramid  is  an  active  zone  permitting  access  to  the  corresponded  topic.  For 
example  clicking  on  "Electronic  components"  the  second  screen  appears. 


[2]  The  information  in  a hypermedia  application  is  structured  in  nodes  containing  different  concepts.  Links  can  be  established 
between  related  nodes  (related  information)  providing  users  with  different  paths  to  analyse  and  understand  a concept.  Each 
user  looks  up  the  application  in  an  intuitive  way,  pathing  through  those  nodes  that  allow  him/her  to  extend  the  information  or 
introduce  himself/herself  in  a new  concept  (LESK  91)(RAMI  97)(SHNE  89)(TERR  94).  Besides,  hypermedia  applications 
include  audio-visual  resources  (texts,  images,  animations,  videos,  etc.)  especially  useful  to  present  many  concepts  which  are 
very  difficult  to  explain  by  means  of  words. 
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The  second  screen  is  the  first  one  of  the  topic  ’’Electronic  components"  and  classifies  them  into  linear, 
non-linear,  passive,  active  and  integrated  circuits.  This  screen  links  with  others  where  concepts  related  with  each 
classification  are  explained.  Navigating  through  the  link  "passive"  of  this  screen  users  can  arrive  to  the  third 
screen  of  Figure  2 where  the  electrical  characteristics  and  a typical  application  of  a photodiode  are  presented  as 
an  example  of  a passive  device. 
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Figure  2:  The  hypermedia  application. 


Conclusions 

The  system  being  developed  is  based  on  the  new  information  technologies  to  achieve  non-electronic 
engineers  education  about  the  analysis  of  basic  circuits  as  well  as  electronic  systems.  To  carry  out  the 
educational  training  a computer  is  used  as  theoretical  and  practical  instruction  tool  (using  hypermedia  techniques 
and  virtual  instrumentation  systems).  As  a result  we  develop  a training  system  including  the  following  features: 

- Guided  instruction  using  hypermedia  applications. 

- Circuits  analysis  using  simulation  programs. 

- Testing  of  real  systems  by  means  of  virtual  instrumentation. 

- Individual  or  co-operative  learning. 

- Self-evaluation. 

- Possibility  of  expansion  (contents  and  functions). 

- Competitive  cost. 
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Abstract:  Changes  in  the  educational  methods  of  nursing  students  created  great  possibilities  for  the 
use  of  computer  technologies.  With  the  help  of  computer-based  multimedia  technologies  we  can 
build  libraries  of  simulated  clinical  problems  (patients  encounters)  that  can  serve  to  fill  some  of  the 
gaps  in  the  current  educational  systems.  This  paper  covers  the  structure  of  a patient  simulation 
system  in  the  early  stages  of  development,  called  CompuPatient.  Feedback  from  a questionnaire 
sent  out  to  collaborators  on  the  project  is  being  used  as  a foundation  for  the  design  of  a system 
customized  to  fit  the  requirements  of  the  collaborators.  The  outcome  of  the  questionnaire  will  be 
discussed.  This  will  be  followed  by  a discussion  on  the  users  involved  in  a patient  simulation 
system.  The  final  section  and  also  the  focus  of  the  paper  is  the  basic  structure  of  CompuPatient 
covering  the  primary  modules  involved  and  communication  between  them. 


Introduction 

The  traditional  way  of  teaching  nursing  and  medical  students  using  lectures  can  be  enhanced  by  a more  active  and 
enjoyable  approach,  namely  Problem-based  Learning  (PBL).  According  to  Albanese  and  Michell  (1993)  as  quoted 
by  Finucance  et  al.  (1998)  PBL  is  "an  educational  method  characterized  by  the  use  of  patient  problems  as  a context 
for  students  to  learn  problem-solving  skills  and  acquire  knowledge  about  the  basic  and  clinical  sciences".  It  seems 
that  more  and  more  nursing  and  medical  educational  institutions  are  integrating  this  approach  in  their  curriculums 
(Finucance  et  al.  1998). 

One  way  of  presenting  PBL  to  the  students  is  by  using  sessions.  A session,  usually  two  to  three  times  a week,  will 
start  with  a presentation  of  a clinical  problem  in  a small  group  setting.  The  group  discusses  possible  causes  and 
develops  hypotheses  and  strategies  to  test  these  hypotheses.  They  are  then  presented  with  further  information. 
Some  information  is  omitted  for  self-study  between  sessions.  The  information  gathered  from  self-study  as  well  as 
the  information  presented  to  them  is  used  to  refine  their  hypotheses,  and  to  finally  reach  a conclusion.  A facilitator 
guides  the  students  in  this  group-learning  process.  The  student  and  facilitator  evaluate  the  session  at  the  end 
(Finucance  et  al.  1998). 

A further  possibility  when  using  PBL  is  the  use  of  a standardized  patient,  which  is  an  actor  that  plays  the  role  of  the 
patient  in  the  clinical  setting  or  an  actual  patient.  The  students  can  interact  with  this  patient.  The  use  of 
standardized  patients  can  enhance  the  presentation  of  the  clinical  problem. 

Although  PBL  is  widely  used  by  educational  institutions,  disadvantages  do  exist.  These  disadvantages  include  a 
greater  demand  for  teacher  time,  stress  experienced  by  students  and  teachers,  less  knowledge  acquired  of  basic 
sciences,  the  difficulty  in  implementing  PBL  with  large  class  sizes,  and  finally  the  costs  involved  (Finucance  et  al. 
1998).  Standardized  patients  augment  this  list  of  disadvantages  with  the  lack  of  appropriate  patients,  difficulty  in 
scheduling  a standardized  patient  for  a PBL  session  and  the  inconvenience  of  using  the  same  patients  over  and  over 
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again  (Roberts  et  al.  1995).  When  using  an  actor  as  a standardized  patient  another  problem  becomes  apparent:  an 
actor  cannot  have  all  the  attributes  of  an  actual  patient. 


Personal  computers  with  multimedia  support  is  more  accessible  than  in  the  past  making  it  feasible  to  implement 
PBL  on  them.  The  computer  can  simulate  the  clinical  problem  or  patient,  allowing  interaction  between  the  users 
(e.g.  students)  and  the  simulated  patient.  The  user  can  ask  the  patient  questions,  examine  the  patient,  request 
laboratory  tests,  etc.  These  systems  are  called  patient  simulation  systems. 

Patient  simulation  systems  used  to  enhance  or  replace  PBL  sessions  have  some  advantages  over  paper-based 
presentations  and  standardized  patients.  In  the  next  paragraph  these  advantages  are  listed. 

The  faculty  should  see  a reduction  in  labor  cost  from  both  teachers  and  facilitators  (Gleydura  et  al.  1995).  Patient 
simulation  systems  let  the  students  analyze  a clinical  problem  at  their  own  pace,  in  their  own  way.  Because  the 
computer  is  consistent  there  is  no  fear  that  some  group  of  students  might  receive  better  help  from  their  facilitators 
than  others.  This  is  a potential  problem  when  using  'human'  facilitators  (Vincelette  et  al.  1997).  It  is  also  much 
easier  for  the  teacher  to  monitor  progress  of  an  individual  student  and  groups  of  students.  Students  can  receive 
detailed  self-evaluation  feedback  on  their  abilities  and  mistakes  made  by  the  student  can  be  simulated  on  the  patient. 

This  paper  covers  the  structure  of  a patient  simulation  system,  called  CompuPatient,  in  the  early  stages  of 
development.  We  are  working  in  collaboration  with  six  Nursing  Departments  at  South  African  Universities.  A 
questionnaire  consisting  of  nine  categories  covering  over  125  features  of  patient  simulation  systems  was  sent  out  to 
the  collaborators.  These  categories  are  applicability  and  scope,  computing  environment,  human-computer 
interaction,  simulation  enhancement,  patient-nurse  encounter,  additional  assistance  available  to  student,  student 
assessment,  feedback  and  the  facilities  provided  to  the  teacher.  The  feedback  from  this  questionnaire  is  being  used 
as  a foundation  for  the  design  of  a system  customized  to  fit  the  requirements  of  the  collaborators.  The  outcome  of 
the  questionnaire  will  be  discussed  in  the  next  section.  Not  all  but  some  of  the  more  important  features  affecting  the 
basic  structure  of  the  system  will  be  covered.  The  users  involved  in  a patient  simulation  system  were  identified 
using  the  feedback  from  the  questionnaire  and  used  to  refine  the  design  of  the  system.  They  will  be  discussed  in 
section  3.  The  focus  of  the  paper  is  the  basic  structure  of  CompuPatient  covering  the  primary  modules  involved  and 
communication  between  them.  The  structure  is  discussed  in  section  4.  Although  the  structure  was  thoroughly’ 
thought  through,  it  is  possible  that  the  first  prototype  might  have  an  influence  on  the  structure. 


Outcome  of  Questionnaire 

The  collaborators  rated  more  than  125  features,  indicating  how  important  the  inclusion  of  these  features  are  to  them. 
Although  the  questionnaire  is  being  used  as  a foundation  for  the  structure,  the  system  developer  could  decide  not  to 
include  certain  features  because  of  their  complexity  and  present  non-viability.  In  the  following  paragraphs  the 
outcome  of  the  questionnaire  is  discussed  and  the  influence  it  has  on  the  structure  of  CompuPatient. 

When  looking  at  the  scope  of  the  system  it  should  not  only  support  the  training  of  nurses  but  also  the  training  of 
medical  doctors.  The  focus  of  the  system  should  not  be  on  certain  areas  in  the  medical  field  but  accommodate  a 
wide  area  of  disciplines.  It  is  important  that  the  system  is  customized  for  healthcare  in  South  Africa.  These  features 
will  not  influence  the  basic  structure  of  the  system. 

A student  could  use  the  system  in  a laboratory  environment  and  should  be  able  to  access  the  system  over  a local  area 
network  and  from  home  using  the  Internet.  Both  of  these  are  addressed  in  the  design  by  creating  a module 
responsible  for  network  communication.  The  other  modules  do  not  have  to  know  how  communication  take  place 
over  the  network  but  only  know  how  to  communicate  with  this  module.  The  system  should  run  on  a personal 
computer  using  Microsoft's  Windows  operating  system.  A decision  was  made  to  support  Windows  95  or  later,  with 
the  possibility  of  supporting  other  platforms  using  Java. 

When  looking  at  the  interaction  by  the  student  with  the  system  it  seems  that  the  collaborators  were  in  favor  of  direct 
manipulation  for  patient  examination  and  interaction  with  instruments.  Natural  language  was  the  preferred  choice 
for  the  implementation  of  other  interaction  options,  followed  by  predefined  menu  choices.  Because  of  complexity 


of  natural  language  it  was  ruled  out  for  CompuPatient,  however  the  design  of  the  structure  gives  the  possibility  of 
accommodating  this  in  the  future.  Predefined  menu  choices  can  be  accommodated  by  the  structure  of  the  system. 

The  simulation  should  and  will  support  the  use  of  text,  graphics,  animation,  audio  and  video.  Although  audio  and 
video  were  seen  as  essential  by  some  collaborators  support  for  it  over  an  Internet  connection  might  not  be  viable  at 
the  present  time  because  of  bandwidth  constrains.  Some  collaborators  prefer  virtual  reality  to  simulate  a patient  or 
environment.  Virtual  reality  simulations  can  have  a great  influence  on  the  structure  of  the  patient  simulation  system 
because  of  the  tight  integration  of  the  student  interface  and  the  simulator  itself  and  will  not  be  supported  in 
CompuPatient. 

The  system  should  monitor  and  give  intelligent  hints  to  the  student.  Additional  assistance  should  also  be  available 
that  can  be  activated  by  the  student  that  could  give  background  information  concerning  the  patient’s  symptoms, 
information  concerning  the  physical  examination,  information  concerning  the  interpretation  of  the  physical 
examination  and  pointers  to  resources  (e.g.  Internet  references,  books,  etc.).  A module  responsible  for  help  and 
hints  will  do  this.  Although  electronic  communication  with  fellow  students  and  the  teacher  was  rated  as 
unimportant  by  some,  provision  was  made  for  this  in  the  structure  because  of  the  additional  benefits  it  could  have. 
This  feature  can  easily  be  integrated  into  the  system  and  be  disabled  when  not  used. 

Evaluation  of  the  student  during  the  simulation  of  a problem,  also  called  the  case,  is  important  and  a "goodness 
measure"  should  be  assigned  to  the  outcome  of  the  session.  The  teacher  should  also  be  able  to  specify  the 
particulars  of  the  scheme  used  to  determine  the  "goodness  measure"  of  an  outcome.  He  should  also  decide  if  the 
evaluation  result  should  be  available  to  the  student  (e.g.  after  the  simulation).  An  assessment  module  is  responsible 
for  the  evaluation  in  the  structure. 

When  looking  at  the  facilities  provided  to  the  teacher  it  is  clear  that  the  system  should  analyze  the  evaluation  results 
discussed  in  the  previous  paragraph  giving  suggestions  on  improvement  of  the  students.  An  essential  feature  is 
support  for  the  teacher  to  develop  new  cases  and  share  cases  with  other  institutions.  Some  collaborators  need 
access  management  facilities  while  others  do  not.  This  facility  can  easily  be  disabled  when  not  used  and  therefore 
included  in  the  structure  of  CompuPatient. 


Users  of  a Patient  Simulation  System 

In  this  section  the  users  identified  from  the  questionnaire  and  the  facilities  that  should  be  available  to  them  are 
discussed.  There  are  two  primary  users  of  the  system,  the  student  and  the  teacher.  Two  secondary  users  were 
identified,  the  administrator  and  the  case  developer.  The  teacher  can  also  function  as  the  administrator  and  case 
developer  depending  on  the  set  up  at  the  education  institution. 


Student 

The  first  and  most  important  user  of  the  patient  simulation  system  will  be  the  student.  According  to  Thomas  (1997) 
the  objectives  of  PEL  are  to  "motivate  learning;  developing  effective  clinical  reasoning;  structuring  knowledge  in 
clinical  contexts;  and  developing  self-learning  skills".  These  objectives  are  all  focused  around  the  student.  She  will 
interact  with  the  simulated  patient  that  responds  to  the  requests  and  reacts  on  changes  in  the  simulated  environment 
(e.g.  time).  Actions  available  to  the  student  might  include  reviewing  the  patient  history,  interviewing  the  patient, 
doing  a physical  examination,  requesting  laboratory  tests,  making  a diagnosis  and  when  necessary  referring  the 
patient  to  a specialist  or  other  professional.  The  system  might  give  intelligent  hints  when  she  seems  stuck.  Help 
should  be  available  to  the  student  that  provides  a connection  or  reference  to  related  material  that  she  can  consult 
when  more  information  is  needed.  Assessment  of  the  student  should  be  done  by  the  system  using  an  assessment 
technique  defined  by  the  system  developer  that  uses  case  specific  detail  supplied  by  the  case  developer.  The  result 
of  the  assessment  or  evaluation  should  be  available  to  the  student  after  the  session  depending  on  the  preferences  of 
the  teacher.  The  student  could  communicate  with  fellow  students  using  a discussion  group  or  with  the  teacher  using 
private  messages. 
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Teacher 


Another  user  of  a patient  simulation  system  will  be  the  teacher  The  teacher’s  task  in  the  patient  simulation  system 
will  be  to  monitor  and  oversee  the  learning  progress  of  an  individual  student  or  group  of  students.  One  of  the 
benefits  of  computers  is  the  large  amount  of  detailed  information  that  can  be  collected  by  the  computer  on  a given 
student  or  group.  This  information  could  be  analyzed  by  the  computer  and  reports  could  be  generated  that  can  help 
the  teacher  to  concentrate  on  problem  areas  in  the  field  of  study. 


Administrator 

The  administrator  will  be  another  user  of  a patient  simulation  system.  He  will  be  responsible  for  administrating 
access  to  students  and  other  users  for  particular  cases  and  information.  A typical  patient  simulation  system  will 
consist  of  more  than  one  case  or  clinical  problem.  Depending  on  the  current  progress  of  the  student  the  student  has 
access  to  certain  cases.  The  access  restrictions  can  be  automated  to  a certain  degree,  for  example  when  the  student 
finished  case  A successfully,  access  is  automatically  granted  for  case  B.  Another  administrator  task  will  be  the 
management  of  student  groups.  It  could  be  easier  to  grant  access  to  certain  cases  by  selecting  a student  group  (e.g. 
first  year  students)  than  to  select  each  student  individually.  The  necessity  of  groups  also  becomes  apparent  when 
working  with  discussion  groups  where  students  can  only  communicate  with  other  students  within  their  group. 


Case  Developer 

The  final  user  of  a patient  simulation  system  will  be  the  case  developer.  The  case  developer's  main  purpose  will  be 
to  develop  new  clinical  problems  or  patients  for  the  patient  simulation  system.  It  is  important  to  remember  that  case 
developers  should  decide  which  variables  should  be  included  and  which  variables  should  be  purposefully  excluded, 
and  how  those  included  should  be  presented  to  the  student.  The  case  developer  will  also  be  responsible  for 
specifying  any  help  and  hints  as  well  as  reference  material  available  to  the  student  for  a particular  case.  He  should 
also  define  case  specific  assessment  variables  used  for  evaluating  the  students'  performance  for  a particular  case. 


A Patient  Simulation  System  Structure 

Using  the  users  and  feedback  of  the  questionnaire  as  a foundation,  a structure  for  CompuPatient  was  designed.  In 
this  section  the  primary  and  sub-modules  of  this  structure  and  the  communication  between  them  are  discussed. 


Modules 

Five  primary  modules  were  identified  (Fig.  1).  The  first  module  is  the  student's  environment  module.  The  purpose 
of  this  module  is  to  provide  an  environment  for  the  student  to  interact  with  the  simulated  clinical  problem  and 
consists  of  six  sub-modules.  The  first  sub-module  is  the  student  interface  that  defines  the  information  presentation 
and  is  customizable  to  suit  the  preferences  of  the  student.  The  case  simulator  is  the  second  sub-module  and  is 
responsible  for  the  simulation  of  the  clinical  problem.  The  case  simulator  is  an  essential  part  of  the  patient 
simulation  system.  This  module  takes  interaction  commands  from  the  student  interface  and  reacts  on  them  by 
changing  the  presentation  of  the  simulated  patient.  When  using  natural  language  another  sub-module  can  fit  between 
the  case  simulator  and  the  student  interface.  It  translates  the  natural  language  into  a format  understandable  by  the 
case  simulator.  The  monitor  module  keeps  a log  of  data  received  from  the  case  simulator  relating  to  the  state  of  the 
simulation  as  well  as  interaction  commands  received  from  the  student.  The  results  are  stored  for  analysis  by  the 
teacher's  environment.  The  assessment  module  uses  the  data  of  the  monitor  to  evaluate  the  student's  progress  during 
the  session  using  case  specific  assessment  detail  defined  by  the  case  developer.  The  evaluation  is  available  to  the 
student  after  the  session  if  enabled  by  the  case  developer.  The  intelligent  help  and  hints  module  gives  help  and  hints 
to  the  student,  customized  to  suit  the  current  progress  of  the  student  by  analyzing  the  log  from  the  monitor.  The  help 
and  hints  can  consist  of  additional  material  or  references  to  additional  material  available  to  the  student.  The  final 
sub-module  of  the  student's  environment  module  is  the  communication  interface.  The  communication  interface  is 
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responsible  for  the  communication  between  the  student's  environment  and  the  other  primary  modules.  Although  the 
primary  modules  can  be  on  the  same  machine  it  is  possible  that  they  can  be  distributed  over  several  machines 
connected  to  a network. 


Figure  1 : Primary  Modules  of  a Patient  Simulation  System 

The  second  primary  module  is  the  storage  area  and  stores  all  data  used  by  the  other  primary  modules  lasting  more 
than  one  session  with  a user.  This  module  is  split  into  five  sub-modules.  The  first  sub-module  called  the  case 
library  stores  all  data  of  the  cases  available  to  the  system.  This  data  includes  multimedia  used  to  simulate  the  case, 
reference  material,  help  and  hints,  as  well  as  case  specific  variables  used  by  the  case  simulator  and  assessment 
module.  Maintenance  of  the  users  involved  in  the  system  and  the  cases  and  information  they  have  access  to  are 
done  by  the  access  rights  module.  The  next  sub-module  called  the  student  records  module  maintains  information 
about  the  student's  case  log  received  from  the  monitor  and  evaluation  results  received  from  the  assessment  module 
and  is  available  to  the  teacher's  environment  for  analysis.  The  communication  message  centre  deals  with  the  storage 
and  distribution  of  messages  sent  between  the  teacher  and  students  or  between  the  students  in  a group.  The  final 
sub-module  is  the  communication  interface  and  is  responsible  for  the  communication  between  the  storage  area  and 
the  other  primary  modules. 

The  next  primary  module  is  the  administration  environment  module  that  provides  an  environment  for  the 
administrator  to  add  and  remove  users,  maintain  student  groups  and  also  grant  and  restrict  access  to  cases  and  other 
information  (e.g.  student  records). 

The  teacher's  environment  module  is  the  next  primary  module  in  the  system.  The  purpose  of  this  module  is  to 
provide  an  environment  for  the  teacher  to  retrieve  and  analyze  student  records.  The  teacher  can  view  an  individual 
student  or  group  of  students’  records,  view  information  about  their  evaluation,  as  well  as  problem  areas  identified  by 
the  system.  This  makes  it  possible  for  the  teacher  to  concentrate  on  these  problem  areas  in  class  or  when  developing 
new  patients  for  the  system. 

The  final  primary  module  of  the  system  is  the  case  development  environment  that  presents  the  case  developer  with 
an  environment  to  develop  new  cases,  alter  old  cases  and  import  and  export  cases  from  and  to  other  similar  systems. 


Communication  between  Modules 


Communication  between  all  the  primary  modules  go  through  the  storage  area  using  a client-server  approach.  The 
storage  area  operates  as  the  server  and  the  other  primary  modules  as  the  clients.  An  advantage  of  this  approach  is 
that  the  system  can  be  updated  to  work  not  only  on  the  local  machine  but  also  over  a local  area  network  or  even  the 
Internet  without  changing  the  structure  of  the  modules.  Only  the  communication  interface  of  all  the  primary 
modules  will  change  to  accommodate  the  new  communication  protocol. 

When  the  student  signs  on  for  a session  with  the  system,  communication  is  established  by  the  student’s  environment 
with  the  storage  area.  The  rights  of  that  particular  student  for  the  case  are  verified  with  the  access  rights  module  in 
the  storage  area.  If  access  is  granted  to  the  student,  the  contents  in  the  case  library  for  the  selected  case  are 
transferred  to  the  case  simulator,  not  necessary  all  at  once  but  can  also  be  transferred  on  demand.  This  data  includes 
the  case  specific  variables  and  multimedia.  The  reference  material,  help  and  hints  are  transferred  from  the  case 
library  to  the  intelligent  help  and  hints  module.  The  monitor  loads  case  specific  assessment  detail  into  the 
assessment  module  from  the  case  library.  The  monitor  queries  the  student  record  to  see  if  the  student  previously 
suspended  the  selected  case.  If  she  did,  the  simulation  starts  with  the  state  of  the  simulation  just  before  the 
suspension.  Otherwise  it  starts  at  the  beginning  of  the  case.  Interaction  between  the  student  and  case  simulator  is 
sent  to  the  monitor  by  the  case  simulator.  The  monitor  keeps  a log  of  all  interactions.  The  assessment  and 
intelligent  help  and  hints  modules  can  use  the  log  of  interaction  to  make  an  evaluation  or  giving  hints  to  the  student. 
If  the  session  is  suspended  or  terminated  the  data  in  the  monitor  is  stored  in  the  student  record.  Communication 
messages  sent  by  the  student  to  the  teacher  or  other  students  are  sent  by  the  student's  environment  to  the 
communication  message  centre.  The  storage  area  keeps  record  of  all  open  sessions  and  if  a message  arrives  for  a 
student  currently  using  the  system,  the  system  sends  the  message  through  to  the  student's  environment  of  that 
particular  student. 

This  paper  will  not  include  a discussion  on  the  communication  of  the  case  development  environment,  administration 
environment  and  the  teacher's  environment  because  of  the  similarities  in  communication  with  the  other  modules 
already  discussed. 


Conclusion 

At  the  time  of  writing,  CompuPatient  was  still  in  the  phase  leading  to  the  implementation  of  the  first  prototype. 
When  looking  at  the  transition  of  education  towards  distant  learning  the  possibilities  of  patient  simulations  are 
endless.  Patient  simulation  might  in  the  future  revolutionize  the  nursing  and  medical  education  process. 
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Abstract:  Synchronized  Multimedia  Integration  Language  (SMIL)  enables  web  developers  to 
create  multimedia  content  for  efficient  delivery  over  the  Internet.  The  objective  of  the  SMIL 
Presentation  Generator  that  this  p^er  discusses  is  to  make  the  capabilities  and  advantages  of 
SMIL  available  to  non-technical  users.  The  SMIL  Presentation  Generator  provides  templates 
for  the  generation  of  RealText,  RealPix  and  SMIL  files.  This  enables  non-technical  users  to 
create  RealText  and  streaming  media  files  which  they  can  then  synchronize  using  the  SMIL 
generator  templates  for  the  creation  of  a web-based  multimedia  applications. 


Introduction 

Synchronized  Multimedia  Integration  Language  (SMIL)  is  based  on  the  extensible  Markup  Language 
(XML).  SMIL  was  developed  by  the  World  Wide  Web  Consortium  (W3C)  and  enables  Web  developers  to 
create  multimedia  content  for  delivery  over  the  Internet.  SMIL  components  consist  of  files  and  streams  of 
audio,  video,  text,  or  image  content.  These  components  are  transmitted  separately  over  the  Internet  thus 
providing  more  economical  and  efficient  use  of  resources.  However,  the  components  are  then  displayed 
together  as  if  they  were  a single  multimedia  stream.  This  gives  a more  reasonable  download  time  as  compared 
to  that  required  by  traditional  monolithic  multimedia  files. 

The  SMIL  specification  consists  of  XML  tags  that  allow  the  definition  of  windows,  multimedia 
elements,  and  their  synchronization  in  time  when  displayed  in  a SMIL  capable  browser.  In  order  to  use  SMIL 
for  creating  presentations,  the  developer  must  devote  the  time  to  learn  the  language.  In  addition,  the  developer 
must  understand  and  be  able  to  encode  specifications  regarding  specific  details  about  size,  placement  and 
structure  of  the  windows  and  the  placement  of  the  multimedia  elements  in  time  for  proper  synchronization. 

This  detail  work  is  very  exacting  and  time-consuming  for  the  developer.  Thus  it  is  difficult  for  the  non- 
technical user  to  take  advantage  of  the  capabilities  provided  by  SMIL. 

In  order  to  make  the  advantages  of  SMIL  available  to  potential  non-technical  users  such  as  educators, 
tools  with  enabling  interfaces  are  needed.  The  objective  of  the  SMIL  Presentation  Generator  that  this  paper 
discusses  is  to  make  the  capabilities  and  advantages  gained  by  using  SMIL  available  to  non-technical  users. 
SMIL  files  incorporate  RealText,  RealPix,  real  audio  and  real  video  files.  The  SMIL  Presentation  Generator 
consists  of  editors  for  RealText,  RealPix  and  SMIL.  Many  faculty  are  attempting  to  improve  the  instruction 
content  of  course  material  through  the  use  of  multimedia  and  also  improve  the  availability  of  the  material  by 
deploying  it  on  the  Internet.  The  intent  in  developing  this  tool  was  to  provide  a user  friendly  front-end  with 
general  templates  that  could  be  used  for  the  generation  of  SMIL  web-based  multimedia  instructional  modules. 
Through  the  use  of  templates  that  guide  the  developer  in  creating  the  files  necessary  for  a SMIL  presentation, 
more  faculty  members  would  have  the  ability  to  create  their  own  SMIL  presentations. 
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Use  of  SMIL  Presentation  Generator 


The  SMIL  Presentation  Generator  provides  an  interface  that  enables  a non-technical  user  with  an 
understanding  of  the  terminology  and  limited  knowledge  of  the  technical  aspects  of  SMIL  to  generate  files  in 
RealText,  RealPix  and  SMIL. 


The  interface  for  the  RealText  Generator  looks  like  a usual  HTML  editor.  The  interface  is  divided  into 
3 major  sections  (See  Figure  1).  One  section  allows  the  user  to  select  the  window  type  from  a list  of  possible 
types  that  s/he  wants  to  include  in  the  presentation.  Another  section  allows  the  user  to  set  the  specific  window 
properties  of  the  type  of  window  that  the  user  has  selected.  For  example,  for  a Ticker  Tape  window  type,  the 
properties  are  duration,  width  and  loop.  The  working  section  allows  the  user  to  type  the  contents  that  are  to  be 
included  in  the  RealText  file  or  preview  either  the  RealText  file  generated  or  the  code.  Icon  buttons  open 
dialog  boxes  for  the  user  to  do  other  tasks  such  as  set  the  time  for  the  display  of  the  text,  link  to  a player  or  to  a 
browser,  select  font  style  and  size,  and  specify  background  color. 
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Figure  1:  Interface  for  the  RealText  Generator 

The  interface  for  the  RealPix  Generator  has  four  sections  (See  Figure  2).  The  Image  Window  allows 
the  user  to  view  images  and  add  them  to  the  presentation.  The  Images  Window  lists  all  images  that  have  been 
added  to  the  presentation.  The  Transition  Window  previews  the  source  image  and  lets  the  user  select  the  type 
of  transition  (crossfade,  fill,  fadein,  fadeout  or  wipe)  to  be  used.  The  Time  Window  lets  the  user  sequence  and 
time  the  use  of  images  and  transitions. 
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Figure  2:  Interface  for  The  RealPix  Generator 


The  interface  for  the  SMIL  Generator  has  four  windows:  Object  Toolbox,  Properties,  Layout  and 
Sequence  (See  Figure  3).  Real  time  files  can  be  added  to  the  presentation  by  clicking  on  the  appropriate  button 
in  the  Object  Toolbox  Window.  A dialog  box  opens  and  lists  available  files  of  the  type  specified.  The  user  can 
then  select  the  file  to  be  added  to  the  presentation.  The  added  files  can  be  viewed  in  the  sequence  window.  The 
Layout  Window  is  used  to  define  the  size  of  the  real  player  screen  and  to  create  regions. 
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Figure  3:  Interface  for  the  SMIL  Generator 

Technical  Description  of  the  SMIL  Presentation  Generator 

The  SMIL  Presentation  Generator  is  an  application  which  facilitates  the  production  of  SMIL 
presentations  by  providing  templates  for  the  developer  to  use  in  order  to  generate  these  web-based  multimedia 
presentations.  The  tool  is  composed  of  three  programs:  the  SMIL  Generator,  the  RealText  Generator  and  the 
RealPix  Generator.  The  SMIL  Presentation  Generator  was  implemented  using  Visual  Basic.  In  addition  to 
generating  SMIL,  RealPix  and  RealText  files,  each  of  the  generators  is  capable  of  loading  and  interpreting  the 
native  formats  and  reverse  engineering  the  output  file  type  into  the  generator  format.  Thus,  a user  can  create, 
load  and  save  SMILs  for  deploying,  editing  and  updating. 

The  RealText  Generator  helps  the  users  generate  their  own  RealText  files  in  a format  that  meets 
RealText  specifications.  Using  the  generator,  the  user  can  either  create  an  original  RealText  file  or  convert  an 
existing  text  document  into  a RealText  file.  The  users  can  select  the  type  of  window  style  and  the  rate  at  which 
the  text  would  flow.  They  can  also  add  links  to  web  sites  and  presentations.  The  RealText  application  presents 
text  in  synchronization  with  other  media  elements.  All  contents  of  a RealText  document  are  contained  within 
the  <window>  tag  and  are  stored  in  files  with  the  .rt  extension.  There  are  four  window  types:  generic  (the 
default  mode),  ScrollingNews,  TickerTape  and  TelePrompter.  The  markup  is  basically  the  same  for  all  window 
types. 
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The  RealPix  Generator  helps  the  users  generate  their  own  stream  images  in  the  RealPix  format  which 
meets  SMIL  specifications.  The  user  can  select  images,  can  specify  the  order  of  the  selected  images,  the  time 
for  which  each  image  stays  on  the  screen  and  the  transition  effects.  Graphics  in  this  format  can  be  streamed 
over  the  web  as  part  of  a SMIL  application.  The  RealPix  file  is  defined  between  two  tags  which  define  the 
beginning  and  end  of  the  RealPix  file.  RealPix  can  stream  images  in  the  following  formats:  GIF87,  GIF89 
(♦.gif),  and  JPEG  (*jpg).  Both  interlaced  and  noninterlaced  GIFs  will  work,  but  RealPix  does  not  take 
advantage  of  any  features  of  interlaced  GIFs.  RealPix  does  not  support  animated  GIFs.  RealPix  can  use  RGB 
baseline  JPEGs;  however,  progressive  and  grayscale  JPEGs  are  not  supported. 

The  SMIL  Generator  is  a SMIL  authoring  tool.  It  can  use  the  files  produced  by  the  RealText  and  the 
RealPix  Generators  plus  additional  types  of  media.  As  an  authoring  tool,  it  lets  the  user  partition  the 
presentation  screen  into  a number  of  regions.  The  user  can  then  define  the  type  of  media  (video,  audio, 
RealText,  RealPix)  associated  with  these  regions.  The  user  can  also  specify  the  time  for  which  each  of  these 
media  items  would  play  in  the  specified  region.  The  output  from  the  SMIL  Generator  is  a SMIL  file  that  can  be 
played  using  a SMIL  capable  browser  such  as  the  RealPlayer. 


Summary 

The  emerging  SMIL  specifications  provide  new  avenues  for  delivering  multimedia  applications  that 
can  be  used  in  academia  for  web-based  instruction.  However,  the  use  of  SMIL  requires  technical  knowledge 
and  understanding.  The  SMIL  Presentation  Generator  can  be  used  by  instructors  who  do  not  have  this  expertise 
but  do  have  an  interest  in  developing  web-based  instructional  modules  with  ease.  The  information  captured  by 
means  of  the  Generator's  templates  will  be  translated  into  SMIL  presentations  that  can  be  made  available  over 
the  Internet  to  students. 
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Abstract 


The  paper  first  reviews  the  authors’  recent  comparative  studies  of  existing  online  learning  environments  and  makes  the  case  that 
no  current  commercially  available  system  effectively  supports  intensive  collaborative  and  creative  discussion  in  a way  that  may 
compete  with  facilitated  small-group  methodologies  such  as  for  example  the  Harvard  case  study  method.  Second,  it  reviews 
earlier  work,  spanning  some  thirty  years,  on  a methodology  named  Structural  Communication  that  achieved  some  success  in 
automating  and  individualizing  such  methodologies.  Then,  the  paper  describes  a methodology  that  combines  some  aspects  of 
Structural  Communication  with  interactive  Web-based  discussion  to  create  an  effective  and  efficient  alternative  to  classroom- 
based  case  study  discussions.  Finally,  some  key  research  findings  are  summarized,  suggesting  that  this  hybrid  methodology  may 
be  more  effective  than  the  facilitated  small  group  for  promoting  creative  ideas,  as  well  as  being  more  efficient  and  multipliable. 


Introduction:  Key  Issues  in  Use  of  CMC  in  Higher  Education 


The  variety  of  Internet-based  synchronous  and  asynchronous  communication  systems  keeps  growing.  In  addition  to 
the  already  well  known  forms  of  asynchronous  Computer-Mediated-Communication  (CMC)  systems,  such  as  email, 
listserv  and  threaded  discussion  lists,  we  now  use  a variety  of  new  synchronous  communication  alternatives,  such  as 
electronic  whiteboards,  Internet  Relay  Chat,  Web-based  audio  and  video  conferencing  and  a growing  variety  of 
“groupware”  packages.  As  the  power  of  the  Internet  grows,  so  does  the  complexity  of  the  material  posted.  Ever 
more  ambitious  examples  of  interactive  multimedia  are  launched  on  the  Web  every  day.  A number  of  novel 
research  questions  and  issues  arise  in  relation  to  the  design  and  use  of  these  new  systems. 

Much  existing  research  is  related  to  earlier  forms  of  text-based  CBT.  Some  of  these  results  may  be  equally  valid 
within  the  context  of  multimedia  distance  education/training  systems.  However,  we  may  expect  many  new  issues 
and  questions  to  emerge  as  these  broad  band  multimedia,  multimodal  communication  systems  link  both  people  and 
remote  databases  into  one  seamless  information  and  communication  environment.  One  recurrent  problem  is  that  we 
hop  from  one  recently-emerged  technology  to  another  currently-emerging  technology  that  promises  some  new 
potential,  without  ever  learning  to  fully  exploit  the  potential  of  the  old.  It  is  a sobering  thought  that  in  all  the 
centuries  since  the  Gutemberg  print  technology  facilitated  the  mass  dissemination  of  text,  we  are  still  struggling  with 
the  issues  of  mediocre  textbooks,  instructional  manuals  that  fail  to  instruct  and  communications  (including  on-line 
texts  and  hypertexts)  that  just  do  not  communicate. 

In  addition  to  the  communication-technology  and  instructional  design  variables,  another  aspect  to  consider  for 
improvement  of  existing  on-line  learning  environments  is  the  promotion  of  effective  “conversational”  interaction 
between  groups  of  students  (and  instructors)  engaged  on  a joint  project.  There  is  a growing  need  for  the 
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implementation  of  learning  exercises  that  prepare  students  for  the  new  profession  of  “Knowledge  Work”.  These 
exercises  should  allow  students  to  work  creatively,  collaboratively  and  at  a distance  on  complex,  leading-edge 
problems  that  impact  their  life  and  work.  Teaching  methods  such  as  seminars  or  case  studies  are  traditionally 
employed  for  developing  creative  thinking  skills  through  collaborative  effort.  They  are  typically  implemented  in 
small  or  medium  sized  groups,  led  by  skilled  and  experienced  "facilitators".  The  success  of  these  methods  depends 
much  on  the  facilitators  and  the  skill  with  which  they  perform  their  roles:  focus  the  discussion;  guide  the  approaches 
adopted  by  the  participants;  use  the  natural  group  dynamics  to  stimulate  interest;  promote  and  support  participation 
and  deep  involvement  by  all;  and  pull  together  what  has  been  learned  in  the  final  debriefing  discussion.  Can  such 
participatory  discussion  methods  be  effectively  orchestrated  at  a distance?  How  might  this  be  done?  And,  most 
importantly,  how  might  we  do  it  so  as  to  create  practical  and  sustainable  WBT  systems  that  will  survive  the  test  of 
time  as  the  initial  enthusiastic  “early  adopters”  move  on  to  other  projects  and  their  place  is  taken  by  the  rank  and  file 
of  the  teaching/training  profession? 

There  are  also  other  pressures,  both  organizational  and  philosophical,  that  are  increasing  the  amount  of  autonomy, 
self-direction  and  responsibility  that  learners  have  in  respect  of  their  own  education  and  development.  Given  the 
increasingly  competitive  nature  of  business  in  the  international  marketplace  and  the  critical  importance  that  access  to 
and  use  of  up-to-date  information  and  methods  plays  in  a company's  competitiveness,  it  is  not  surprising  that  the 
concept  of  human  resources  development  as  "self-development"  is  taking  root.  This  concept  sees  keeping  up-to- 
date  and  employable  as  the  responsibility  of  eveiy  employee.  The  employer's  responsibility  is  to  make  this  possible, 
by  helping  to  identify  the  needs  of  the  individual  and  by  facilitating  access  to  the  resources  necessary  to  satisfy  those 
needs.  This  will  ever  less  frequently  call  for  lengthy  courses  organized  either  within  the  company  or  by  outside 
providers,  but  will  instead  make  much  more  use  of  networking,  access  to  external  databases  and  electronic  libraries, 
small  specialist  group  tele-training  and  self-instruction  in  all  its  forms  (Eurich,  1990). 


An  integrated  research  agenda 

The  research  focus  adopted  by  the  authors  is  on  the  effective  implementation  of  group  discussion,  or 
"conversational",  methodologies  on  electronic  telecommunications  networks.  This  focus  is  particularly  important, 
as  we  know  much  less  about  how  to  converse  effectively  on  electronic  networks,  than  we  do  about  electronic  self- 
instruction.  There  is  a long  history  and  fairly  developed  technology  of  the  design,  development  and  deliveiy-at-a- 
distance  of  self-study  materials.  There  is  much  less  known  about  the  running  of  effective  group-discussion  sessions 
at  a distance  (Chang,  1 994). 

In  a recent  study  reported  elsewhere  (Villalba  and  Romiszowski,  1999)  the  authors  performed  a comparative 
analysis  of  typical  on-line  learning  environments  currently  used  in  higher  education  and  the  typical  ways  in  which 
these  environments  are  used  to  implement  collaborative  group  learning  activities.  The  findings  indicated  that  few 
currently  implemented  on-line  courses  actually  include  a strong  emphasis  on  collaborative  small-group  learning  and, 
when  such  activities  are  implemented,  this  is  generally  as  a relatively  unstructured  on-line  group  discussion,  using 
either  synchronous  “chat”  sessions  or,  more  frequently,  asynchronous  email  driven  discussion  lists.  There  is  little  if 
any  research,  however,  indicating  that  such  environments  are  conducive  to  in-depth  reflective  discussions  of  the  type 
required  to  develop  critical  and  creative  thinking  skills.  And  there  are  some  studies  (e.g.  Romiszowski  and  DeHaas, 
1989,  Romiszowski  and  Chang,  1994)  that  suggest  they  are  singularly  ineffective  in  this  respect.  As  a means  of 
verifying  these  suggestions,  the  authors  selected  one  of  the  previously  evaluated  on-line  learning  environments, 
Aulanet,  for  further  in-depth  study. 


Aulanet  is  a web  based  instruction  environment,  developed  in  Brazil  at  the  Catholic  University  of  Rio  de  Janeiro 
(Lucena  et.al.  1998)  which  is  also  available  in  an  English  language  version.  It  was  selected  as  it  offered  a wider 
variety  of  on-line  discussion  environments  than  most  other  currently  available  systems.  In  addition  to  the  regular  e- 
mail,  both  threaded  and  unthreaded  asynchronous  discussion  environments  and  text-based  synchronous  “chat” 
rooms,  options  are  available  for  audio,  audio-graphic  and  full  video-conference  sessions  in  small  or  large  groups.  In 
addition,  the  creators  of  Aulanet  claim  the  system  is  based  on  or  influenced  by  contemporary  theories  of  cognition 
and  constructivism.  The  present  authors  have  observed  and  analyzed  the  use  of  Aulanet  as  a delivery  system  for  four 
courses  running  through  the  spring  and  the  fall  semesters  of  1999.  These  courses  form  part  of  a graduate  (Masters 
and  Doctorate)  program  in  Educational  Technology  in  a leading  North  American  Research  University.  The  study 
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involved  both  the  observation  of  student  use  of  different  collaborative  learning  environments  provided  within 
Aulanet  and  the  analysis  of  student  questionnaire  responses  and  user-evaluations  administered  during  the  course  of 
the  1999  academic  year. 


In  that  study  the  students  made  some  quite  significant  suggestions  for  enhancement  of  the  learning  environment.  A 
major  observation  is  concerned  with  the  structure  of  facilities  for  constructive  educational  “conversations”.  The 
many  and  various  components  of  Aulanet  that  permit  both  synchronous  and  asynchronous  student/teacher  and 
student/student  interaction  are  seen  to  be  no  different  from  the  facilities  that  exist  in  many  other  online  learning 
packages  currently  on  the  market.  Both  faculty  and  students  have  come  across  limitations  in  the  available  group 
communication  facilities  that  limit  what  they  can  implement  in  the  way  of  “creative  group  work  at  a distance”. 


In  the  present  study  we  are  seeking  ways  around  these  limitations.  In  order  to  perform  this  research  we  have 
developed  a web  based  automated  case  discussion  system.  To  do  this,  we  turned  to  a methodology  called  Structural 
Communication,  originally  developed  in  the  United  Kingdom  in  the  late  60’s  and  early  70*s  as  a methodology  for 
writing  and  presenting  interactive  print-based  exercises.  An  adaptation  of  this  methodology  had  shown  itself  to  be 
effective  for  automating  and  individualizing  small-group  learning  methods  such  as  the  Harvard  Business  Case 
methodology  (Hodgson  et.  al.  1971).  Later  studies  demonstrated  the  effectiveness  of  the  methodology  in  reducing 
the  amount  of  human  facilitator  or  monitor  interaction  necessary  in  order  to  lead  the  exercise  to  a satisfactory 
conclusion  (Romiszowski,  1990;  Romiszowski  & Chang,  1992;  Chang,  1994). 


Structural  Communication  is  an  instructional  approach  that  provides  a simulated  dialogue  between  an  author  of 
instructional  materials  and  the  students.  It  has  been  called  ”an  interactive  technique  for  communicating 
understanding”  (Egan,  1976).  Understanding  is  "inferred  if  a student  shows  the  ability  to  use  knowledge 
appropriately  in  different  contexts,  and  to  organize  knowledge  elements  in  accordance  with  specified  organizing 
principles”  (Egan,  1972,  p.  66).  The  technique  was  designed  to  encourage  creative  thinking  in  learners,  allowing 
them  to  develop  an  understanding  of  a topic,  not  simply  to  memorize  facts.  Furthermore,  Structural  Communication 
was  designed  to  promote  learning  for  social  action.  Hodgson,  in  line  with  many  current  constructivists,  viewed  the 
social  contexts  of  the  learning  activity  to  be  critical  for  the  transfer  of  learning  to  practical  situations  (Egan,  1976). 
The  distinctions  between  the  learning  of  knowledge  and  the  learning  for  social  action  are  evident  in  the  actual 
components  Hodgson  designed  into  the  Structural  Communication  technique.  The  typical  components  of  a 
Structural  Communication  unit  are  described  below. 


Intention 


Presentation 


Investigation 


Response  Matrix 


Discussion 


The  opening  statement,  which  defines  what  is  to  be  studied,  provides  an  overview, 
possibly  an  "advance  organizer”,  and  sometimes  a rationale.  It  is  used  to  provide  a 
context  for  the  content  of  the  study  unit. 

The  material,  experience,  exercise,  case  study,  etc.  which  supplies  the  essential  facts  and 
concepts  of  the  domain  being  studied.  This  may  be  an  existing  text,  a video,  a case  study, 
a simulation,  or  real-life  experience,  depending  on  the  overall  strategy  of  the  exercise. 
This  could  also  be  any  sort  of  computer-based  instruction,  including  simulations. 

A set  of  problems  for  solution,  which  are  designed  to  present  the  "intellectual  challenge” 
that  is  an  essential  part  of  the  Structural  Communication  methodology.  These  problems 
are  interrelated  and  are  open-ended  to  allow  multiple  responses  and  viewpoints.  The 
purpose  of  the  investigation  section  is  for  the  learner  to  interact  with  the  subject  matter. 

A randomized  array  of  items  which  summarize  key  parts,  concepts  or  principles  from  the 
knowledge  base  that  is  being  used  and  studied  in  the  exercise.  Often  it  resembles  a "key 
point  summary”  of  the  Presentation.  The  student  composes  a response  (outlines  an  essay) 
by  selecting  any  number  of  these  items  as  a "best”  response  to  a given  problem. 

The  Discussion  has  two  parts:  a Discussion  Guide  and  a set  of  Discussion  Comments. 
The  Guide  is  a set  of  if-then  rules,  which  test  the  student's  response  for  omission  or 
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inclusion  of  certain  significant  items,  or  combinations  of  items.  The  Comments  are 
constructive  statements  that  discuss  in  depth  the  rationale  for  including  or  excluding 
certain  items. 

Viewpoints  An  outline  of  the  author’s,  and  other  alternative  viewpoints;  this  may  review  some  aspects 

stated  in  the  Intention,  make  explicit  some  biases  or  standpoints  held  dear  by  the  author, 
draw  attention  to  other  views  in  the  literature,  etc.  Ideally,  the  viewpoint  section  plays  a 
final,  interactive  role  between  author  and  learner. 

An  additional  aspect  of  the  Structural  Communication  study  unit  is  the  assessment  of  the  learner’s  responses  to  the 
questions  posed  by  the  study  unit.  Hodgson  developed  a numerical  measure  called  the  coherence  index.  According 
to  Egan  (1976),  the  coherence  index  is  an  objective  measure  of  how  well  the  student  organizes  knowledge  about  the 
specific  topic  in  the  study  unit.  However,  the  constructivist  may  reject  this  interpretation,  since  it  implies  a correct 
set  of  responses  based  upon  the  ideal  organization  and  analysis  of  the  presented  data.  Constructivists  are  more  likely 
to  view  the  coherence  index  as  an  objective  measure  of  the  degree  of  agreement  of  the  student  with  the  author  of  the 
Structural  Communication  unit.  They  may  well  be  more  interested  in  measures  of  divergence  of  viewpoints  rather 
than  convergence  on  a uniform,  or  coherent,  viewpoint  on  the  issues  being  discussed. 


A Revised  Structural  Communication  Methodology 

The  original  SC  methodology  was  redesigned  in  order  to  create  a somewhat  more  “constructivist”  and  collaborative 
learning  environment.  Some  of  the  most  important  features  of  the  SC  method,  for  example  individualized  learning 
of  basic  content  and  access  to  expert  opinions,  were  maintained.  However,  the  revised  form  provides  open-ended 
discussion  environments  for  students  to  share,  argue,  persuade,  and  negotiate  their  perspectives  on  the  expert’s 
feedback  comments  as  well  as  on  their  own  opinions.  In  the  original  method,  the  planned  learning  activity  ends  with 
reading  the  expert’s  comments.  Since  the  revised  method  is  designed  for  small  group  collaborative  learning,  from 
this  point  on,  further  learning  activity  occurs  collaboratively  with  group  members.  After  completing  their  work 
individually,  students  browse  what  each  group  member  did.  A summary  table  tells  the  group  members  who  selected 
which  items  in  the  response  matrix  during  individual  study.  With  the  summary  table  as  a guide,  group  members  can 
access  any  item  and  read  the  viewpoints  and  justifications  of  other  group  members.  They  are  asked  to  react  to  each 
other’s  justifications  stating  whether  they  agree  or  disagree  with  each  other. 

As  the  process  continues,  the  pool  of  the  students’  discussion  log  grows.  The  software  is  designed  to  save  every 
justification  and  discussion  statement  that  students  make,  and  to  enable  students  to  browse  those  reactions  and 
opinions  in  a cumulative  mode  as  the  process  continues.  While  they  are  performing  these  activities,  the  nature  of 
their  interaction  may  vary  along  several  dimensions,  which  are  under  the  control  of  the  software.  For  example,  they 
may,  or  may  not,  access  expert’s  feedback  comments  to  inform  them  on  “what  the  experts  say”.  Similarly,  they  may 
have  to  construct  their  opinions  and  justifications  in  their  own  words,  or  merely  select  them  from  a long  list.  Four 
different  versions  of  the  revised  Structural  Communication  software  were  developed  for  experimental  purposes. 
These  vary  along  “simple-to-complex  interaction  structures”  and  “more  or  less  constructivist  philosophy” 
dimensions  (Chang,  1994). 


Practical  Conclusions  and  their  Implications  for  Conversational  WBT 

Several  conclusions  were  drawn  fi*om  this  study.  One  practical  conclusion  was  that  the  discussion  environments 
based  more  closely  on  the  original  SC  exercise  model,  offer  an  appropriate  environment  for  intensive  small  group 
conversational  interaction  via  CMC  networks,  even  if  these  seem  to  be  less  in  line  with  currently  popular 
philosophical  positions.  However,  the  environments  must  support  the  discussion  process  in  an  effective  and  efficient 
manner.  This  implies,  among  other  things,  ensuring  that  the  discussion  process  is  motivating  and  not  over-taxing  in 
terms  of  workload.  The  researchers  found  no  trace  of  the  supposed  “perils”  of  limiting  the  creative  and  critical 
thinking  of  the  discussion  participants  by  exposing  them  to  “expert”  viewpoints.  The  more  tightly  structured  SC 
environments  produced  more  frequent  creative  insights  than  the  supposedly  more  “constructivist”  environments. 


As  regards  student  motivation,  there  is  little  doubt  that  exercises  organized  in  the  form  of  Structural  Communication 
are  most  effective.  The  several  hundred  students  who  participated  in  the  above-mentioned  4-treatment  study  worked 
voluntarily  and  without  recompense  in  the  form  of  grade  points  throughout  a whole  semester  with  very  low  dropout 
rates.  The  students  in  earlier  studies  showed  similar  levels  of  motivation.  This  finding  is  important  in  the  context  of 
conversational  CMC-type  WBT  in  that  the  study  materials  placed  in  a publicly  “open”  WBT  environment  will 
survive  and  prosper,  or  alternatively  wither  and  die,  as  a function  of  their  ability  to  attract  voluntary  participation 
from  a significant  number  of  WWW  navigators. 

It  may  well  be  that  the  reasons  for  the  high  levels  of  motivation  encountered  in  these  studies  are  not  entirely  and 
uniquely  a consequence  of  the  SC  format.  As  mentioned  before,  the  quality  of  design  of  the  exercises  themselves  is 
probably  a most  critical  factor.  In  Khan's  recent  collection  of  essays  on  Web-Based  Instruction  (Khan,  1997),  two 
chapters  are  devoted  to  issues  of  motivation  in  WBI  (Cornell  & Martin,  1997;  Duchastel,  1997).  Both  chapters 
review  several  well-known  approaches  and  models  for  the  design  of  motivational  learning  materials,  such  as  ARCS 
(Keller,  1983).  However,  Structural  Communication’s  unique  mechanism  of  the  response  matrix  is  an  approach  that 
has  been  largely  overlooked.  The  response  matrix  allows  for  easy  and  rapid  responding  to  complex  multi-faceted 
problems,  at  the  same  time  allowing  one  to  explore  one's  own  and  each  others'  cognitive  structures  in  some  detail. 
This  is  the  inherent  attractiveness  of  the  methodology  to  the  interested  and  reflective  student. 

We  should  not  leave  the  question  of  motivation  without  also  considering  the  motivation  of  the  instructor  or 
discussion  facilitator.  Cornell  & Martin  (1997)  address  the  question  of  instructor  motivation  in  Web-based  learning. 
The  authors  mention  seven  reasons  for  lack  of  instructor  motivation  (as  compared  to  twelve  for  student  motivation) 
and,  interestingly,  this  list  of  seven  reasons  does  not  include  the  avoidance  of  extra  workload.  However,  experience 
and  research  suggest  that  CMC-delivered  courses  almost  always  involve  significantly  more  instructor  time  than 
conventional  courses.  We  therefore  feel  that  one  should  add  this  eighth  reason  to  the  list.  Furthermore,  we  believe 
that  Structural  Communication  offers,  at  least  in  part,  a solution  to  this  eighth  reason. 

In  discussions  organized  within  a SC-based  CMC  environment,  the  problem  of  instructor  overload  is  much  reduced. 
The  automatic  generation  of  extensive  and  constructive  feedback  to  students  on  the  basis  of  the  pattern  of  responses 
they  select  from  the  response  matrix  acts  like  a first-round  live  reaction  from  the  instructor  to  specific  aspects  of  the 
students’  overall  response.  In  practice,  these  automatically  generated  feedback  messages  satisfy  well  over  70%  of 
the  students’  needs  for  clarification  or  orientation.  The  remaining  20%  to  30%  of  feedback  interactions  generate 
second-round  student  responses  in  the  form  of  electronic  messages  that  pose  supplementary  questions  or  comments, 
usually  at  a deeper  and  more  reflective  level  of  discourse.  The  effects  of  this  on  the  motivation  of  the  instructor  are 
twofold.  First,  the  overall  workload  of  responding  to  students  (measured  in  terms  of  the  number  of  messages  to 
compose)  is  reduced  by  some  70%.  Second,  the  messages  to  be  responded  are  on  issues  that  are  often  more 
interesting  and  always  relatively  original  and  non-repetitive. 

Having  touched  on  the  issue  of  workload  as  it  relates  to  the  instructor/facilitator,  we  may  mention  our  experience 
with  the  SC  response-matrix  mechanism  as  a device  for  the  reduction  of  student  workload.  In  selecting  a subset  of 
response  elements  from  the  universe  represented  by  the  items  in  the  response-matrix,  the  student  is  in  effect 
composing  a complex,  structured  response  to  the  problem  under  analysis.  This  is  analogous  to  the  preparation  of  an 
outline  for  an  essay  or  live  seminar  presentation.  The  intellectual  effort  necessary  to  evaluate  the  relevance  of  each 
item  of  the  matrix  to  the  problem  being  studied  is  equivalent  to  that  expended  in  planning  an  essay  or  presentation. 
However,  the  time  and  effort  involved  in  writing  an  essay  or  preparing  and  delivering  a presentation  are  saved. 

We  have  also  observed  that  students  spend  more  time,  more  productively,  in  the  intellectual  restructuring  of  ideas 
and  the  creation  of  their  own  knowledge  structures  than  is  often  the  case  in  conventional  instruction.  To  be  sure, 
students  do  restructure  their  ideas  when  they  engage  in  free,  unstructured,  small-group  discussion  with  their  peers 
and  teachers,  but  not  with  the  frequency  and  depth  of  insight  that  seems  to  be  generated  in  the  SC  environment.  And, 
in  our  view,  the  relatively  unstructured  discussion  lists  that  abound  in  current  WBT  environments,  even  the  popular 
“threaded”  discussions,  are  less  effective  in  promoting  creative  idea-generation.  Furthermore,  they  are  more 
wasteful  of  student  time  than  both  the  SC  environment  and  the  conventional  small-group  seminar  presentation,  in 
that  every  single  contribution  must  be  produced  as  an  original  typed  message. 
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In  conclusion,  it  is  not  our  argument  to  eliminate  written  communication  from  the  ideal  WBT  environment.  There 
are  strong  learning  benefits  to  “putting  pen  to  paper”  (or  fingers  to  keyboard).  As  Albert  Einstein  replied  when  asked 
what  he  thought  of  a novel  scientific  issue:  “I  do  not  know  what  I think  until  I have  v^itten  it  down”.  However,  as  in 
real  life  time  is  always  limited,  why  not  use  it  most  efficiently?  In  the  SC-based  form  of  WBT  environment,  the 
student  is  saved  the  effort  of  writing  out  surface-level  arguments  that  mainly  serve  to  “get  the  ball  rolling”  by  acting 
as  stimuli  for  comments  from  instructor/facilitators  or  peers.  The  discussion  is  set  in  motion  by  means  of  the 
response  matrix.  The  writing  of  “what  I think”  then  does  occur  at  the  second  and  subsequent  rounds  of  in-depth 
discussion.  We  close  by  stressing  the  importance  of  implementing  effective,  efficient  and  multipliable  small  group 
collaborative  learning  environments  online.  Otherwise,  due  to  the  difficulties  and  costs  of  arranging  sufficient 
opportunities  for  facilitated  small-group  collaborative  learning  by  conventional  means,  tomorrow’s  educational 
systems  will  offer  “less  and  less  of  what  learners  need  more  and  more”. 
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Abstract:  This  paper  describes  an  empirical  study  conducted  within  users  of  Graphical  User 
Interfaces  that  deal  with  files  and  folders  in  a similar  way  as  the  Windows  98/NT  Explorer.  10 
human  experts  analysed  user  protocols,  which  were  collected  using  computer  logging,  so  that 
the  most  common  user  errors  could  be  identified.  The  comments  made  by  the  human  experts 
served  as  a basis  for  the  requirements  specification  of  a second  executable  release  of  a GUI 
called  IFM  that  incorporates  intelligence.  IFM  reasons  about  users'  actions  so  that  it  can 
provide  spontaneous  help  to  users’  errors.  The  user  protocols  collected  during  the  empirical 
analysis  were  used  in  the  formative  evaluation  of  the  second  executable  release  of  IFM.  The 
reactions  of  IFM  were  compared  to  the  comments  of  the  human  experts  that  had  taken  part  in 
the  empirical  study. 


Overview 

Graphical  user  interfaces  are  user  friendlier  than  others,  such  as  command  language  interfaces. 
However,  as  McGraw  (1994)  points  out,  they  may  prove  difficult  to  traverse  and  use  if  they  are  poorly 
designed;  hence  in  addition  to  basic  user  interface  guidelines,  developers  may  also  want  to  consider  the 
feasibility  and  benefits  of  using  AI  to  enhance  the  user  interface. 

Intelligent  File  Manipulator  (IFM)  is  a graphical  user  interface  that  incorporates  intelligence.  It  is  used 
for  a file  manipulation  program  similar  to  the  Windows  98/NT  Explorer.  However,  it  can  also  reason  about 
users’  actions.  IFM’s  main  aim  is  to  provide  spontaneous  help  and  advice  to  users  who  have  made  an  error  with 
respect  to  the  hypothesised  intentions.  IFM  is  meant  to  operate  like  a human  expert  who  watches  a user  over  the 
shoulder  and  provides  spontaneous  advice.  This  advice  may,  in  time,  enhance  novice  users’  skills  in  operating 
file  manipulation  programs.  Therefore,  IFM  may  also  be  used  as  a learning  environment  for  novice  users  of  a 
GUI  (Virvou  & Kabassi  2000). 

Indeed  IFM  may  provide  an  environment  where  novice  users  would  be  more  protected  from  their 
errors  and  would  have  more  informative  advice  from  the  system.  In  this  way  they  could  become  familiar  with 
file  manipulation  programs  and  thus  learn  how  to  use  them.  IFM  can  be  used  for  teaching  purposes  in  student 
computer  labs.  Each  student  can  give  a password  so  that  s/he  may  get  individualised  support. 

IFM  has  been  developed  based  on  an  object  oriented  life  cycle  model,  the  Rational  Objectory  Process 
(Quantrani  1998;  Kruchten  1999),  which  supports  many  iterations  of  the  development  phases.  A first  prototype 
of  IFM  has  been  completed  (Virvou  & Stavrianou  1999).  Its  design  originated  from  another  system  called 
RESCUER  that  was  meant  to  provide  assistance  to  users  of  command  language  interfaces,  such  as  the  interface 
of  UNIX  (Virvou  1998;  Virvou  & Du  Boulay  1999). 

In  order  to  enhance  the  design,  the  first  prototype  of  IFM  was  evaluated.  As  (Dix  et  al.  1993)  point  out, 
evaluation  is  an  integral  part  of  the  design  process  and  should  take  place  throughout  the  design  phase  of  the  life 
cycle;  most  techniques  for  evaluation  at  this  stage  are  analytic  and  involve  using  an  expert  to  assess  the  design 
against  cognitive  and  usability  principles.  Evaluation  of  the  design  of  IFM,  at  this  stage,  was  conducted  through 
an  empirical  study.  Ten  human  experts  commented  on  the  actions  made  by  users  of  a standard  manipulation 
program  (Windows  98/NT  Explorer).  The  human  expert’s  reactions  were  then  compared  to  IFM’s 
recommendations.  All  of  the  human  experts  who  commented  on  user  actions  had  a first  and/or  higher  degree  in 
Computer  Science  and  had  useif-extensively  used  programs  like  a standard  file  manipulation  program.  Many  of  | 
them  had  also  teaching  experience  related  to  the  use  of  such  programs  in  computer  labs. 
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The  users  who  participated  in  the  empirical  study  were  both  expert  and  novice.  The  experts  who  were 
involved  as  users  were  different  from  the  experts  who  were  used  to  comment  on  users.  All  users  were  asked  to 
work  with  a standard  file  manipulation  program,  as  they  would  normally  do.  Computer  logging  was  used  to 
record  all  users’  actions  during  their  interactions  with  the  system.  Computer  logging  has  been  used  in  evaluating 
user  interface  designs  for  a long  time  (Bradford  et  al.  1990).  The  major  advantages  of  computer  logging  are  that 
it  automatically  and  continuously  collect  objective  data  for  further  analysis  and  interpretation  and  does  not 
interfere  with  users  during  their  interactions  with  a system  (Chou  1999). 

The  collected  user  protocols  were  passed  on  to  the  10  human  experts  so  as  to  analyse  them.  All  of  the 
human  experts  gave  their  comments  on  every  user’s  action.  One  of  the  aims  of  this  kind  of  analysis  was  to  find 
usability  problems  of  a standard  Explorer,  through  heuristic  evaluation  (Nielsen  1994).  In  particular,  human 
experts  were  asked  to  focus  on  two  usability  heuristics  specified  by  Nielsen: 

1 . Error  Prevention. 

2.  Help  users  recognise,  diagnose  and  recover  from  errors. 

Therefore,  the  human  experts’  comments  should  include  whether  they  thought  that  the  user  had  made  an 
error  and  what  sort  of  advice  they  would  give  to  these  users  to  recover  from  their  errors.  Hence,  the  empirical 
study  revealed  a lot  of  usability  problems  of  a standard  file  manipulation  program  in  terms  of  user  errors. 

Half  of  the  user  protocols  were  also  given  as  input  to  IFM  at  that  stage.  IFM’s  reactions  were 
compared  to  the  human  experts’  comments.  The  conclusions  from  this  comparison  were  used  for  the 
construction  of  a second  executable  release  of  IFM.  In  the  second  release,  IFM’s  knowledge  base  was  enriched 
with  categories  of  error  that  IFM  could  not  recognise  in  the  previous  release. 

A formative  evaluation  of  the  second  executable  release  of  IFM  involved  the  rest  of  the  user  protocols, 
which  had  been  collected  during  the  empirical  study,  were  commented  by  the  human  experts  but  were  not  used 
for  the  requirements  specification  of  the  second  executable  release  of  IFM.  In  most  cases,  IFM’s  reactions  were 
found-similar  to  the  comments  made  by  the  majority  of  human  experts  at  this  stage.  However,  some  limitations 
were  also  identified.  These  limitations  are  meant  to  be  addressed  in  the  third  executable  release  of  IFM. 


Life  Cycle  of  IFM 

In  the  development  of  IFM,  the  iteration  of  several  phases  of  the  development  was  considered  crucial. 
Therefore  the  Rational  Objectory  Process  formed  the  basis  for  the  life  cycle  of  the  system.  Using  the  Rational 
Objectory  Process,  an  iterative  development  is  advocated,  in  which  the  architecture  is  actually  prototyped, 
tested,  measured,  and  analysed,  and  then  refined  in  subsequent  iterations  (Kruchten  1995).  Rational  Objectory 
Process  divides  the  software  life  cycle  into  cycles,  each  cycle  working  on  a new  generation  of  the  product.  The 
process  divides  one  development  cycle  in  four  consecutive  phases:  the  inception,  the  elaboration,  the 
construction  and  the  transition  phase  (Quantrani  1998;  Kruchten  1999). 

In  the  inception  phase,  a primary  executable  release  of  IFM  was  developed  (Virvou  & Stavrianou 
1999).  In  the  phase  of  the  elaboration  an  empirical  study  was  conducted  involving  a number  of  expert  and 
novice  users.  All  users  were  asked  to  work  with  the  regular  Windows  98/NT  Explorer  for  two  hours.  While 
working,  the  system  was  monitoring  users’  actions  without  interrupting  them.  The  user  actions  were  recorded 
using  the  computer  logging  method  and  then  were  analysed  in  order  to  identify  the  errors  that  users  make  while 
completing  certain  tasks.  Human  evaluators,  using  the  video  recording  method  observed  the  commands  that  the 
users  issued  during  their  interaction  with  the  regular  explorer  and  wrote  down  their  comments  about  each 
action.  The  empirical  study  showed  the  misconceptions  users  have  while  navigating  through  the  Windows 
98/NT  Explorer  and  which  were  not  identified  by  the  previous  version  of  IFM.  Based  on  the  results  of  the 
empirical  study,  the  extension  of  the  design  of  IFM  was  described. 

In  the  construction  phase,  the  primary  executable  release  of  IFM  was  expanded  based  on  the 
limitations  of  IFM  that  were  identified  during  the  empirical  analysis.  In  the  phase  of  elaboration,  a formative 
evaluation  of  the  second  executable  release  of  IFM  was  conducted.  The  outcome  of  the  construction  phase  is  a 
product  ready  to  be  used  by  its  end-users  (Quatrani  1998;  Kruchten  1999). 


Design  of  IFM’s  Primary  Executable  Release 

IFM  constantly  reasons  about  users’  actions,  so  that  it  can  diagnose  problematic  situations  and  give 
advice  concerning  users’  errors.  As  Cerri  and  Loia  (1997)  point  out,  if  error  diagnosis  is  to  be  performed,  a user 
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modelling  component  should  be  incorporated  into  the  system.  Therefore,  IFM  has  a user  modelling  component 
that  generates  hypotheseis  about  users’  intentions  and  possible  errors.  However,  there  may  be  ambiguities  about 
errors,  since  there  may  be  different  explanations  of  observed  incorrect  users’  actions  (Mitrovic  et  al.  1996). 
Ambiguity  resolution  about  errors  is  done  in  IFM  by  using  two  different  reasoning  mechanisms:  a simulator  of 
human  error  generations  and  a limited  plan  recognition  mechanism. 

The  simulator  of  human  error  generations  is  based  on  a cognitive  theory  that  attempts  to  formalise  the 
reasoning  that  people  use.  This  theory  is  called  Human  Plausible  Reasoning  (Collins  & Michalski,  1989). 
Intelligent  File  Manipulator  uses  an  adaptation  of  this  theory  to  simulate  possible  errors  of  a user.  For  example, 
if  a user  has  clicked  on  a certain  file  having  a certain  name  s/he  could  be  considered  to  have  intended  to  click  on 
a different,  similarly  named,  file. 

The  limited  plan  recognition  mechanism  is  used  by  the  system  to  improve  control  and  is  based  on  what 
we  call  'instabilities”.  Every  user  action  is  categorised  in  one  of  four  categories,  namely  expected,  neutral, 
suspect,  erroneous,  depending  on  its  degree  of  compatibility  of  the  users’  hypothesised  intentions.  For  example, 
an  action  is  considered  “expected”  if  it  deletes  instabilities.  Instabilities  are  added  and/or  deleted  from  a list  as  a 
result  of  user  actions.  Instabilities  are  deleted  when  an  action  results  in  a file  store  state  that  should  not  contain 
them.  In  this  sense  the  deletion  of  an  instability  represents  the  continuation  of  a user  plan  that  started  earlier.  In 
the  current  implementation,  a file  store  is  considered  to  be  absolutely  stable  if  it  does  not  contain: 

1 . empty  directories 

2.  directories  with  only  one  child 

3.  multiple  copies  of  a certain  file 

4.  folders  with  the  name  ‘New  Folder’ 

5.  files  with  the  name  ‘New  File’ 

6.  a file  placed  in  clipboard  by  a cut  or  copy  command  and  not  have  further  use. 


Empirical  Study 

The  empirical  study  involved  15  users.  Among  them,  7 users  were  novice  and  8 users  were  experts. 
These  users  were  given  a program  that  was  very  similar  to  a standard  Explorer  and  were  asked  to  use  it,  as  they 
would  normally  do.  Computer  logging  was  used  to  capture  their  actions  in  video,  so  that  the  human  experts 
would  be  able  to  watch  and  comment  them.  10  human  experts  were  involved  in  the  empirical  study,  in  order  to 
comment  on  the  15  users.  All  10  experts  commented  on  all  15  users.  In  some  cases  all  of  the  experts  made  the 
same  comment  and  suggested  the  same  replacement  command.  In  these  cases  we  have  100%  agreement  of 
experts.  In  other  cases  there  was  a diversity  of  opinions.  (Tab.  1)  illustrates  an  example  of  a user  protocol, 
among  the  ones  collected,  together  with  the  comments  that  the  majority  of  experts  provided.  The  experts  had 
actually  seen  the  exact  user  actions,  e.g.  clicking  on  a certain  object,  dragging  a selected  object,  and  so  on. 
However,  for  the  purposes  of  illustration  of  the  example  in  limited  space,  we  use  the  meaning  of  each  action 
rather  than  showing  the  exact  screenshots. 


USER  ACTIONS 

HUMAN  EXPERTS  COMMENTS 

1 . Copy(A:\House2\Room2\) 

S/he  wants  to  copy  the  folder  A:\House2\Room2\ 

2.  Copy(A:\Housel\) 

100%  suggested  the  replacement  command  Paste(A:\Housel\) 

3 . Paste(  A :\House  1 \) 

With  this  action,  the  following  error  message  has  occurred  ‘The 
destination  folder  is  the  same  as  the  source  folder’ 

4.  Copy  (A:\House2\Room2\) 

S/he  wants  to  copy  the  folder  A:\House2\Room2\ 

5.  Paste(A:\Housel\) 

S/he  wants  to  paste  the  copied  folder  in  A:\House  1\ 

6.  Delete(  A:\House2\Room2\) 

S/he  wants  to  delete  the  folder  A:\House2\Room2\ 

7.  Delete(A:\House  l\book.bmp) 

S/he  wants  to  delete  the  file  A:\Housel\book.bmp 

8 . Delete(  A:\House  1 \Room  1 \) 

89%  suggested  the  replacement  command 
Delete(A:\House2\Room  1 \) 

9.  Mkdir(A:\House3\) 

100%  suggested  the  replacement  command  Mkdir(A:\House2\) 

1 0.  Del(A:\House3\New  Folder\) 

S/he  wants  to  delete  the  new  folder  that  was  created  by  mistake 

Table  1:  A real  Explorer-user  interaction  with  the  comments  of  human  experts 


One  of  the  aims  of  the  empirical  study  was  to  identify  the  most  frequent  errors  that  expert  and  novice 
users  may  make  while  interacting  with  a standard  Explorer.  Another  aim  of  the  empirical  study  was  to  compare 
the  human  expert  comments  to  IFM’s  reactions,  in  order  to  identify  IFM’s  limitations.  These  would  lead  to 
further  requirements  specifications  for  a second  version  of  IFM.  Therefore,  half  of  the  protocols  were  given  as 
input  to  IFM,  in  order  to  evaluate  IFM’s  reactions,  in  comparison  to  the  human  expert  comments. 

Some  conclusions  drawn  from  the  empirical  study,  were  the  following: 

• Users,  expert  or  novice,  tend  to  reproduce  the  same  mistakes  over  and  over.  Therefore,  user  profiles 
can  be  very  useful  for  providing  individualised  help. 

• Evaluation  of  user  auditing’s  results  showed  that  expert  users  generally  make  different  types  of 
mistakes  than  novice  users.  Consequently,  we  propose  a different  user  approach,  taking  in 
consideration  a user’s  level  of  experience.  Empirical  study  has  revealed  that  novice  users  are  usually 
very  careful  but  make  mistakes  due  to  lack  of  knowledge,  whereas  expert  users  tend  to  be  very 
confident  but  careless.  This  results  in  different  categories  of  errors.  For  example  novice  users  are 
prone  to  command  selection  and/or  execution  mistakes  and  experts  usually  confuse  neighbouring 
objects  with  similar  names. 

• Frequent  error  categories,  among  the  ones  identified,  are  the  following: 

1.  Users  tend  to  forget  to  rename  a new  object  (e.g.,  a folder  or  a text  file)  from  the  default 
Windows  name  (e.g.  ‘New  Folder’,  ‘New  Text  Document’).  This  results  in  a mess  of 
identically  named  objects  in  the  file  store. 

2.  Another  error  is  that  users  often  copy  or  cut  one  or  more  files  but  they  do  not  know  how  to 
complete  the  move  or  copy  operation.  Finally,  users  tend  to  paste  the  same  group  of  files  to 
more  than  one  location. 

3.  Some  of  the  recognised  errors  are  believed  to  result  from  the  novice  users’  lack  of  knowledge 
of  the  regular  Explorer’s  structure.  For  example,  they  tend  to  confuse  the  parent  folder  at  the 
left  part  of  the  Explorer  with  the  folder  shown  at  the  right  part  of  the  program. 

4.  Another  significant  error  category,  committed  by  both  novice  and  expert  users,  concerns  the 
deletion  of  objects.  This  error  category  is  considered  dangerous  because  the  results  of  such  an 
error  can  be  devastating.  Users  often  delete  a folder  whose  content  they  are  not  aware  of  A 
subset  of  this  category  concerns  users  that  delete  an  object  after  they  have  copied  it,  though 
'have  not  pasted  it. 

5.  Another  conclusion  derived  from  user  auditing  concerns  object  similarities,  namely,  the  case 
when  a user  confuses  an  object  with  another.  Users  may  mistake  an  object  for  another  if  they 
are  neighbouring  objects,  or  in  case  of  objects  with  similar  names.  In  the  latter  case, 
similarities  in  terms  of  names  may  be  categorised. 

The  comparison  of  the  human  expert  comments  to  IFM’s  reactions  showed  that  in  cases  where  IFM 
had  diagnosed  an  error,  it  had  also  been  successful  at  generating  advice,  which  was  similar  to  the  majority  of 
the  experts’  advice.  However,  a lot  of  limitations  of  IFM  were  also  identified.  These  resulted  in  the 
enhancement  of  IFM’s  design,  as  explained  in  the  following  section. 


Design  Extension  after  the  Empirical  Study 

The  conclusions  drawn  from  the  empirical  study  were  used  for  the  extension  of  IFM’s  design.  The  results  are 
described  below. 

1 . New  Instabilities  added 

A.  Observation:  Users  often  neglect  to  provide  a name  for  a new  object. 

Addition:  An  instability  is  added  in  case  of  the  creation  of  a new  object.  This  instability  is  removed 
when  the  user  renames  that  object. 

B.  Observation:  Users  often  tend  to  leave  a copy/cut  - paste  task  incomplete. 

Addition:  An  instability  is  added  when  a user  performs  the  copy  or  the  cut  part  of  the  task  that  places 
something  new  in  the  clipboard.  In  order  to  delete  such  an  instability,  the  user  has  to  perform  a paste 
command. 

C.  Observation:  The  user  performs  the  paste  part  of  a copy/cut-paste  action  more  than  one  time. 

Addition:  A new  instability  is  added  every  time  a user  performs  a paste  command  for  the  same  object. 
Such  an  instability  is  removed  when  one  of  the  objects  being  pasted  has  been  modified. 


2.  Potential  Deletion  Errors  Detected 

A.  Observation:  Users  often  do  not  pay  great  attention  to  the  contents  of  a folder  that  is  placed  deeply  in  a 
path  and  this  results  in  deleting  objects  without  realising. 

Addition:  IFM  is  constantly  monitoring  the  user  throughout  his/her  navigation.  If  the  user  has  not 
visited  the  folder  containing  an  object  to  be  deleted,  then  the. system  considers  the  action  suspect  and 
informs  the  user.  Otherwise,  the  deletion  action  is  being  performed  without  further  notice. 

3.  Users’  misconceptions 

A.  Observation:  Users  often  make  mistakes  due  to  neighbouring  location  of  objects 

Addition:  A new  category  of  error  was  introduced,  to  detect  erroneous  actions  that  are  caused  by  users’ 
misconceptions  about  either  neighbouring  objects  or  objects  with  similar  names. 

4.  Users’  behavior 

A.  Observation:  Users  reproduce  the  same  mistakes. 

Addition:  In  order  to  have  more  individualised  user  advice,  IFM  keeps  a long  term  user  model,  which 
consists  of  long  term  information  about  a user  (Rich  1979;  Jones  & Virvou  1991).  The  long  term  user 
model  holds  the  individual  features  of  the  user,  that  have  been  recorded  in  previous  interactions,  so  that 
IFM  can  diagnose  and  provide  advice  in  a more  precise  way. 

B.  Observation:  Users  issued  more  commands  than  those  included  in  the  domain  representation  of  IFM’s 
primary  design. 

Addition:  The  domain  representation  was  extended  to  include  several  new  commands. 


Evaluation  of  IFM 

IFM’s  second  executable  release  was  evaluated  using  half  of  the  protocols  collected  during  the  | 
empirical  study.  Again  the  human  experts’  comments  were  compared  to  IFM’s  reactions.  (Tab.  2)  illustrates  the 
protocol  example  that  was  presented  in  the  previous  section.  However,  this  time  in  the  second  column  we  show 
how  IFM  categorised  each  command  of  the  protocol.  In  case  a command  was  characterised  as  suspect  or 
erroneous,  IFM  generates  alternative  commands  and  suggests  the  user  to  replace  the  issued  one  with  the  new 
one.  In  the  third  column,  we  give  the  degree  of  compatibility  of  IFM’s  suggestions  to  the  human  experts’ 
suggestions  for  a particular  command. 


USER  ACTIONS 

IFM  SUGGESTION 

DEGREE  OF  COMPATIBILITY 
WITH  THE  COMMENT  OF 
THE  MAJORITY  OF  EXPERTS 

1 . Copy(A:\House2\Room2) 

A neutral  command 

2 . Copy(A:\House  1\) 

A suspect  command,  IFM 
suggested  the  alternative 
command  Paste(A:\Housel\) 

100% 

3 . Copy(A:\House2\Room2\) 

A neutral  command 

4 . Paste(A:\House  1 \) 

An  expected  command 

5.  Delete(A:\Housel\Rooml\) 

A suspect  command,  IFM 
suggested  the  alternative 
command 

Delete(A:\House2\Room  1\) 

100% 

6.  Delete(A:\House2\Room2\) 

An  expected  command 

7.  Delete(A:\House  l\book.bmp) 

A neutral  command 

8 . Delete(A:\House2\Rooml\) 

An  expected  command 

9.  Cut(A:\Housel\Rooml\) 

A neutral  command 

10.  Paste(A:\House2\) 

An  expected  command 

1 1 . Mkdir(A;\House3\) 

A suspect  command,  IFM 
suggested  the  alternative 
command  Mkdir(A:\House2\) 

1 00% 

Table  2:  IFM’s  reasoning  on  a sample  of  a real  Explorers’  user  interaction 


IFM  was  quite  successful  at  achieving  a high  degree  of  compatibility  with  the  experts’  advice. 
However,  in  many  cases  there  were  a lot  of  alternative  commands  that  IFM  generated.  In  most  of  these  cases, 
one  of  the  alternative  commands  was  compatible  with  the  suggestion  of  the  majority  of  the  experts.  Still,  the 
limitation  of  the  generation  of  many  alternatives  needs  to  be  addressed  in  a subsequent  executable  release  of 
IFM. 


Conclusions 

The  empirical  study  conducted  within  expert  and  novice  users  of  a standard  file  manipulation  program 
has  revealed  a lot  of  usability  problems  of  such  programs.  The  incorporation  of  intelligence  can  indeed  enhance 
a GUI  in  terms  of  user  error  prevention  and  recovery.  IFM  incorporates  intelligence  and  provides  spontaneous 
assistance  to  users. 

IFM’s  reactions  as  compared  to  the  human  experts’  comments  who  took  part  in  the  empirical  study 
showed  that  IFM  can  reproduce  a great  deal  of  the  human  expert  advice.  However,  there  are  still  some 
limitations  that  need  to  be  addressed  in  a subsequent  version  of  IFM.  One  important  limitation  is  that  IFM 
generates  too  many  alternative  commands  to  be  shown  to  the  user  in  case  it  diagnoses  that  there  has  been  a 
problem.  This  problem  can  be  addressed  by  extending  IFM  to  include  a subsystem  that  would  calculate 
certainty  degrees  about  alternatives  so  that  these  can  be  ranked  before  being  presented  to  the  user. 
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learning  in  team  oriented,  project  based  university  level  courses.  It  was  developed  to  meet  the 
needs  of  student  teams  and  instructors  efficiently  and  at  low  cost.  TeamCMU  provides  most 
of  the  electronic  support  that  work  groups  expect  and  need,  but  is  highly  configurable,  easy  to 
use,  flexible,  low  cost,  low  maintenance,  and  portable. 


Introduction 

TeamCMU  was  developed  at  Carnegie  Mellon  University  to  meet  the  need  for  an  enabling,  electronic 
environment  that  supports  learning  through  both  theoretical  and  experiential  exposure  to  complex  team 
assignments  and  team  processes.  TeamCMU  is  an  electronic,  web  based,  integrated,  easy-to-leam  and  easy-to- 
use  software  environment  that  supports  collaboration  and  communication  within  and  among  student  project 
teams.  It  is  intended  to  provide  students  with  the  electronic  support  they  need  to  develop  complex  systems 
projects.  It  was  developed  to  meet  the  needs  of  student  teams  and  instructors  efficiently  and  at  low  cost. 
TeamCMU  provides  most  of  the  electronic  support  that  work  groups  expect  and  need,  but  is  highly 
configurable,  easy  to  use,  flexible,  low  cost,  low  maintenance,  and  very  portable. 

Today’s  workplace  increasingly  emphasizes  teamwork,  especially  in  technical  fields  such  as 
information  systems.  Employers  of  new  college  graduates  in  information  systems,  software  engineering, 
technical  communications  and  related  areas  value  students'  experiences  working  as  members  of  a team. 
Academic  classroom  settings,  however,  have  traditionally  been  successful  in  providing  students  with  the  theory 
and  practical  skills  needed  for  small  individually  completed  development  projects.  Students  often  have  not  had 
appropriate  learning  opportunities  and  skill-building  support  for  larger  team-based  projects  and,  as  a result,  do 
not  usually  bring  practical  collaboration  skills  into  the  workplace. 

At  Carnegie  Mellon  University,  students  majoring  in  Information  and  Decision  Systems,  Technical 
Writing,  Public  Administration  and  many  other  disciplines  complete  substantial,  live-client  team  projects  as  part 
of  their  courses.  Students  in  these  project  courses  deal  with  all  of  the  challenges  common  to  student  project 
teams  - difficulty  coordinating  schedules,  synchronizing  and  stabilizing  their  work  products,  and 
communicating  effectively  with  each  other  (Phan  & Weinberg  1995).  To  complete  their  work  successfully, 
however,  members  of  software  development  teams  must  learn  to  successfully  manage  their  time,  their  work 
products,  and  their  interactions  with  fellow  team  members  and  their  sponsoring  clients.  Effective  coordination, 
communication,  and  use  of  shared  resources  are  crucial  skills  for  team  members  to  master.  The  use  of  high 
quality  electronic  tools  to  support  development  team  efforts  can  play  an  important  and  useful  role  in  facilitating 
successful  team  collaboration  and  project  development  as  discussed  in  (Humphrey  1997).  With  few 
commercial  toolsets  readily  available,  however,  most  project  teams  have  had  to  rely  on  standalone  products  and 
email  for  effective  coordination  and  communication  of  their  work  activities  and  work  products. 
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The  TeamCMU  Project 


Using  TeamCMU,  students  gain  familiarity  and  experience  with  the  types  of  electronic  support  tools 
they  will  use  on  the  job.  Because  the  tools  included  in  the  TeamCMU  environment  are  easily  mastered, 
students  can  be  productive  with  a minimum  of  training  or  faculty  support  services.  Students  gain  valuable 
exposure  to  the  types  of  team  support  environments  they  will  find  in  industrial  and  governmental  software 
development  practice. 

Additionally,  TeamCMU  includes  facilities  to  support  faculty  needs  in  teaching  project  based  courses. 
Faculty  who  use  TeamCMU  in  their  courses  also  gain  experience  in  managing  electronic  communications  in  an 
organized  way,  reducing  the  need  for  email  and  other  forms  of  communications  with  class  members.  The 
emphasis  in  the  first  version  of  the  toolset  is  on  team  organization,  communications,  and  process  for  software 
development  project  teams.  Later  versions  will  provide  support  for  project  management  and  project  tracking. 

The  toolset  is  rich  in  useful  features,  flexible  and  extensible  and  provides  support  for  both  student 
teams  and  faculty.  For  student  team  members,  the  initial  version  of  TeamCMU  includes 

- personal  calendars 

- team  calendars 

- course  calendars 

- secure  server  space  for  team  work  products 

- version  and  access  control  with  check-out  and  check-in  protection 

- time  sheets  for  recording  work  hours 

- meeting  agendas  and  minutes 

- discussion  boards  for  course,  team  and  client  communications. 

All  facilities  of  TeamCMU  are  accessible  to  students  from  a homepage  configured  by  the  faculty  in 
charge  of  the  course.  Because  all  communications  and  artifacts  relating  to  a project  are  maintained  and 
organized  in  a logical,  cohesive  way,  TeamCMU  reduces  the  need  for  members  to  communicate  and  share 
documents  and  electronic  artifacts  via  email  and  other  means.  The  TeamCMU  project  URL  is  : 
http://www.cmu.edu/acs/telab/teamcmu/. 

TeamCMU  helps  keep  students  in  team-based  project  courses  focused  on  accomplishing  educational 
objectives.  Students  easily  learn  the  tools  and  apply  them  to  their  group  project  assignments.  We  expect 
students  to  benefit  from  use  of  the  toolset  in  several  ways:  reduced  process  time,  better  coordination  of  team 
resources,  enhanced  communication  among  themselves  and  with  their  instructors,  and  reduced  rework.  Further, 
in  the  case  of  the  Information  and  Decision  Systems  curriculum,  it  is  expected  that  the  students  will  mature  in 
their  use  of  the  toolset  during  their  junior  and  senior  years,  showing  increased  team  integration,  effectiveness 
and  competence. 

For  faculty.  Team  CMU  includes  configurable  links  to  supporting  course  documents  including 
syllabus,  course  description,  and  faculty  information.  It  also  includes  an  announcement  board  for  posting 
important  course  related  information  and  assignment  boards  which  enable  students  to  submit  time-stamped 
assignments  electronically  and  securely.  Faculty  are  permitted  to  access  each  team's  public  workspaces  and 
discussion  boards  to  view  project  progress  and  assess  toolset  usage.  Further,  the  toolset  includes  background, 
passive  instrumentation  for  faculty  assessment  and  evaluation  of  toolset  usage  and  team  effectiveness.  With 
minimal  modification,  individual  faculty  can  tailor  the  toolset  by  adding  additional  features  as  needed. 

To  promote  communications  among  faculty,  two  special  teams  exist  - a faculty  team  consisting  of  all 
faculty  members  established  in  the  toolset's  installation  and  course-faculty  teams  consisting  of  all  faculty 
members  jointly  teaching  an  individual  course.  Faculty  in  these  teams  may  share  documents  and  communicate 
using  the  same  facilities  as  the  student  team  members. 

An  important  design  objective  has  been  to  make  the  toolset  very  convenient  and  easy  to  learn  and  use 
for  both  faculty  and  students.  The  interface  to  the  toolset  requires  only  a web  browser  and  a single  login  to 
provide  authenticated  access.  Consistent  with  our  design  objectives,  we  have  selected  customizable 
components  that  are  affordable,  portable,  extensible,  and  easily  maintained.  We  have  selected  these 
components  with  an  eye  toward  easy  substitution  as  future  technologies  evolve  and  become  available.  Thus,  the 
toolset  itself  is  based  on  established  web  based,  nonproprietary  technologies  - HTML,  cgi  scripting  via  perl, 
and  SQL  databases.  To  keep  the  toolset  development  costs  and  complexity  down,  some  of  its  internal 
components  are  low-cost  licensed  software  components  and  custom  created  databases. 
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TeamCMU  can  be  customized  or  extended  to  meet  the  needs  of  a wide  variety  of  individual  instructors 
and  individual  courses.  The  tools  will  support  the  needs  of  faculty  in  evaluating  student  learning  of  content  and 
process  and  in  providing  appropriate  feedback  grounded  in  the  student’s  work.  For  faculty  assessment  and 
evaluation  of  student  teamwork,  special  modules  provide  administrative  capabilities  as  well  as  online  support 
for  administering  and  collecting  assignments,  work  products,  periodic  surveys,  weekly  hours-worked  time 
sheets,  peer  assessments,  and  other  research  instruments.  Faculty  will  be  able  to  track  project  status  and 
progress  and  follow  team  communications,  as  the  majority  of  the  team  communications  will  be  posted 
electronically.  As  students  learn  to  use  the  tools  and  follow  established  team  management  procedures  such  as 
structured  meeting  management  principles,  more  information  will  be  accessible  to  the  faculty.  Grading  for  team 
projects  will  be  enhanced  based  on  factual  information  gathered  from  and  via  the  toolset.  Instrumentation  built 
into  the  toolset  will  allow  researchers  to  learn  more  about  patterns  of  usage. 

Over  time,  TeamCMU  will  be  incorporated  into  numerous  courses  that  teach  or  require  skills  in 
process  management,  team  management  and  team  organizational  skills.  We  hope  to  use  TeamCMU  as  a 
vehicle  for  better  understanding  how  electronic  tool  support  can  enhance  the  teaching  and  learning  process. 
While  our  initial  efforts  have  focussed  on  supporting  the  needs  of  software  development  teams,  the  toolset  was 
developed  with  a view  toward  more  general  applications  for  project  teams,  curricula,  and  content  of  various 
disciplines.  Additional  development  in  the  future  will  support  differing  team-based  methods  used  in  other 
project  courses. 

Two  departments  at  Carnegie  Mellon  University  introduced  TeamCMU  into  their  courses  as  a pilot  test 
during  the  Fall  Semester,  1999:  Information  and  Decision  Systems  (IDS)  with  two  courses  and  Master  of 
Professional  Writing  Program  (English)  with  one  course.  Target  courses  are  Fundamentals  of  Systems 
Development  (a  junior-level  IDS  course  with  90  students).  Information  System  Applications  (a  senior-level  IDS 
course  with  70  students),  and  Writing  for  Multimedia  (a  masters-level  English  course  with  25  students).  Other 
departments  will  be  using  the  toolset  during  Spring  Semester,  2000.  After  successful  completion  of  the  pilot 
testing  during  AY  1999-2000,  the  toolset  will  be  made  freely  available  to  the  Carnegie  Mellon  academic 
community  with  an  eye  toward  wider  dissemination  in  the  future. 
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Abstract:  In  this  paper  we  present  the  design  of  a course  on  multimedia-based 
learning  and  a case  study  carried  out  at  the  Darmstadt  University  of  Technology. 
We  describe  the  course,  the  interdisciplinary  audience  and  the  employment  of 
computer-based  communication  and  cooperation.  In  more  detail  we  introduce 
VITAL,  a computer-supported  cooperative  learning  environment.  We  state  the 
results  of  the  evaluation  of  the  course.  Finally,  we  come  to  the  conclusion  that  by 
using  the  computer  as  a communication  and  cooperation  medium  new  forms  of 
discussion  arise  which  have  more  of  a brainstorming  character  than  a continuous 
coherent  discussion. 


1.  Introduction 

Currently,  we  are  experiencing  a boom  in  demand  for  the  use  of  multimedia  technology  to  support 
teaching  and  learning  at  various  levels  of  education  and  training.  Multimedia  technology  (we  use  this 
term  in  short  to  refer  to  multimedia,  hypermedia  and  telemedia  technologies)  is  a basis  for  enabling  new 
forms  of  teaching  and  learning  in  domains  such  as  virtual  universities,  distributed  organisations  and  life- 
long learning. 

In  order  to  qualify  students  for  designing  multimedia  learning  units  and  environments  we  developed  a 
seminar  on  ’’Learning  with  Multimedia"  which  is  based  on  the  following  principles: 

Learning  about  new  technologies  in  education  should  take  place  not  only  by  hearing  and  reading 
about  these  technologies  but  by  using  learning  software  based  on  these  technologies. 

Knowledge  about  learning  with  multimedia  should  not  be  learned  isolated  but  embedded  in  an 
authentic  context. 

The  task  of  designing  and  realizing  multimedia  learning  units  and  environments  is  complex  and 
usually  done  not  by  a single  person  but  by  a team. 

These  teams  should  consist  of  experts  of  various  disciplines,  e.g.  computer  science,  pedagogy,  and 
media  design. 

The  course  on  "Learning  with  Multimedia"  addresses  these  principles: 
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The  course  includes  lectures  about  educational  technology  as  well  as  hands-on  learning  experiences. 
The  learning  software  is  used  to  learn  about  the  content  of  the  course  - ’’Learning  with  Multimedia” 
During  a larger  part  of  the  course  the  students  work  in  small  groups  (2-5  members)  on  the  task  of 
designing  a lesson  about  a specific  subtopic  of ’’Learning  with  Multimedia”. 

Students  participating  in  the  course  should  have  a background  in  a related  discipline,  such  as 
pedagogy,  computer  science,  psychology,  or  media  design. 

In  addition,  seminar  tutors  should  (re)present  multiple  perspectives  on  the  topic,  in  order  to  foster 
interdisciplinary  work  in  the  teams. 

In  sum,  our  goal  was  to  create  a harmony  between  the  content  of  the  course  and  the  methods  used  to 
deliver  it.  The  ideal  course  should  provide  hands-on  learning  experiences  punctuated  with  short 
demonstration  and  discussion  sessions.  At  the  end  of  the  course,  participants  should  be  able  to  judge  and 
design  multimedia-based  instruction. 

The  remainder  of  this  paper  is  structured  as  follows:  In  the  next  section  we  describe  the  course 
’’Learning  with  Multimedia”.  Then  we  sketch  the  virtual  learning  environment  VITAL  and  the  ways  we 
used  VITAL  during  the  course.  In  section  four  we  present  some  results  of  our  evaluation  based  on 
interviews  and  questionnaires. 


2.  Course  Description 

The  seminar  took  place  in  the  summer  term  of  1999  at  the  Darmstadt  University  of  Technology, 
Gennany,  lasting  three  months  and  taking  two  hours  per  week.  The  aim  of  the  course  is  to  learn  about 
and  discuss  the  basics  of  multimedia-based  learning.  The  students  are  supposed  to  actively  contribute 
to  the  seminar.  The  curriculum  comprises  the  topics  of  learning  with  multimedia,  web-based  learning 
systems,  tele-teaching,  computer-supported  cooperative  learning,  learning  processes  supported  by 
computers,  and  new  forms  of  teaching  and  learning.  For  this  specific  course  the  audience  consisted  of 
15  students,  ten  with  a more  technical  background  (computer  science  and  electrical  engineering)  and 
five  with  a pedagogical  background.  The  lecturers  of  this  course  were  also  interdisciplinary:  one  with 
both  a computer  science  and  a pedagogical  degree  and  one  with  a degree  in  Mathematics  and 
Linguistics.  The  intention  of  the  lecturers  of  this  course  was  to  teach  the  topics  not  only  in  a theoretical 
way  but  to  give  the  students  real  experiences  with  these  programs  etc.  The  seminar  was  supposed  to 
integrate  the  content  with  the  way  in  which  it  was  taught. 

The  students  were  divided  into  five  interdisciplinary  teams  of  three  students  each.  Three  of  the  teams 
included  pedagogic  students.  These  teams  had  two  tasks:  They  each  had  to  develop  a lesson  about  one 
of  the  following  topics: 

• Hypertext  learning  system 

• Cooperative  learning  methods 

• Characteristics  of  geographically  distributed  learning 

• Communication  in  CSCW  (computer-supported  cooperative  work)  / CSCL  (computer-supported 
cooperative  learning)  systems 

• Comparison  of  commercially  available  CD-ROM  learning  systems  for  Mathematics  in  elementary 
school 

• Visualization  with  interactive  simulations 

The  second  task  was  to  generate  these  lessons  with  computers  and  to  present  them  as  a computer 
program.  To  support  communication  and  cooperation  both  within  the  teams  and  within  the  whole  group 


the  students  were  asked  to  use  the  virtual  learning  environment  VITAL  which  is  described  below  in 
detail. 

(The  lecturers  were  aware  of  the  self  referencing:  A seminar  about  multimedia-based  learning  were  the 
students  are  supposed  to  work  in  a multimedia-based  way  on  producing  lessons  about  multimedia-based 
learning.) 

The  course  started  with  four  group  sessions.  The  students  were  given  an  overview  about  the  technical 
and  pedagogical  problems  concerning  multimedia-based  learning,  and  an  introduction  to  the  VITAL 
system.  This  period  was  followed  by  three  weeks  of  team  work.  During  this  time,  VITAL  served  as  a 
blackboard  for  announcements  and  references  to  literature.  The  students  were  invited  to  come  to  the 
course  room  and,  if  needed,  seek  for  advice  from  the  lecturers,  but  had  to  find  their  own  way  of  working 
within  the  groups.  After  this  period,  the  intermediary  results  of  each  team  were  presented  to  the  other 
groups.  Here,  problems  concerning  information  retrieval,  using  VITAL,  finding  a common  language 
between  the  disciplines,  selecting  the  suitable  sections  to  present  in  a lesson  etc.  were  discussed.  After 
another  week  of  team-work  the  final  presentations  were  made. 

During  the  period  of  working  in  a team,  the  students  had  diverse  possibilities  to  communicate.  They 
could  meet  face-to-face  either  with  or  without  a lecturer,  they  could  telephone,  email,  use  the  chat  tool 
of  VITAL,  or  employ  the  learning  repository  provided  also  by  VITAL.  Some  of  the  groups 
communicated  nearly  exclusively  via  the  computer  environment,  and  some  groups  used  more  the 
traditional  means  including  email. 

Also  the  final  presentations  were  performed  differently  by  the  various  groups.  One  team,  having  used 
exclusively  VITAL  all  the  time,  used  also  VITAL  as  a presentation  tool.  Others  did  not  use  a computer 
for  the  presentation  at  all.  The  team  preparing  the  lesson  about  learning  with  hypertext  presented  their 
results  as  hypertext.  The  discussion  about  this  took  place  as  an  asynchronous  session  within  VITAL. 
For  ten  days  this  hypertext  system  was  part  of  the  World  Wide  Web  and  each  student  had  to  read  it  and 
had  to  take  part  of  the  on-line  discussion. 


3.  The  Virtual  Learning  Environment  VITAL 

The  course  design  included  the  provision  of  rich  opportunities  for  the  students  and  the  lecturers  to 
communicate  and  cooperate  during  the  course.  In  addition  to  standard  communication  means  such  as 
phone,  email  and  the  world-wide  web,  we  used  the  virtual  cooperative  learning  environment  VITAL  (for 
Virtual  Teaching  And  Learning)  developed  at  GMD-IPSI  (Pfister  et  al.  1998).  VITAL  aims  to  support 
small  and  medium  sized  teams  of  adult  learners.  Its  main  objective  is  to  enable  users  to  learn  about  a 
large  range  of  topics  (i.e.,  VITAL  is  domain-independent)  by  providing  a virtual  environment  and  a set 
of  tools  that  are  intuitive  to  use  and  conducive  to  the  coordination,  communication  and  cooperation 
processes  that  are  typical  for  learning. 

In  VITAL,  all  users  work  with  cooperative  hypermedia  documents.  They  can  view  documents  and 
create  new  ones  of  arbitrary  complexity  by  means  of  introducing  new  links.  Users  live  in  so-called 
virtual  rooms,  which  make  up  the  learning  world.  Virtual  rooms  provide  a metaphor  that  serves  the 
purpose  of  supporting  orientation  and  group-awareness  in  the  learning  environment.  Users  who  occupy 
the  same  room  have  the  same  view  on  the  presented  material,  they  are  aware  of  each  other,  can 
communicate  with  each  other,  and  they  are  able  to  cooperatively  manipulate  documents.  Virtual  rooms 
are  especially  useful  for  providing  smooth  transitions  between  synchronous  and  asynchronous  modes  of 
learning,  since  persons  in  the  same  room  have  full  group-awareness  for  synchronous  work,  but  objects 


(texts,  pictures,  etc.)  remain  persistent  in  a room  for  later  asynchronous  work.  Figure  1 shows  some 
major  components  and  flmctions  of  VITAL. 
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Figure  1:  A VITAL  auditorium  window,  a chat  window,  and  the  world  browser 

As  can  be  seen  in  Figure  1 , most  of  a virtual  room  consists  of  a shared  whiteboard,  where  users  can 
cooperatively  view  and  create  hypermedia  documents.  Group-awareness  is  supported  b'^  showing 
images  of  all  persons  currently  in  a room,  and  by  using  personalized  telepointers.  Synchronous 
communication  is  performed  via  a chat-tool,  or  by  an  audio  connection.  (The  latter  was  not  used  in  the 
case  study  because  of  technological  constraints).  Asynchronous  communication  is  performed  either  by 
sending  emails  or  by  leaving  text-messages  on  the  shared  whiteboard. 

VITAL  provides  three  types  of  virtual  rooms:  (ij  private  homes  for  individual  study,  (ii)  group  rooms 
for  discussion  and  self-organized  cooperative  learning,  (iii)  auditoriums  for  presentation  and  teacher- 
guided  learning.  In  an  auditorium,  two  roles  are  distinguished,  that  of  a learner  and  that  of  a trainer.  The 
trainer  controls  the  learners'  access  to  the  material  presented  as  well  as  to  the  cooperation  tools. 

In  the  remainder  we  focus  on  the  following  three  usage  scenarios  of  VITAL  in  the  course: 

• Synchronous  distributed  group  work: 

Students  meet  in  virtual  group  rooms  and  communicate  via  the  VITAL  chat  tool.  The  chat  tool 
provides  only  text-based  communication.  In  addition  learners  can  refer  to  material  on  the  shared 
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whiteboard  in  the  virtual  room.  The  room  also  provides  awareness  of  the  other  inhabitants  of  the 
room  by  the  way  of  small  pictures  for  each  learner  (see  Figure  1). 

• Asynchronous  presentational  discussion: 

The  students  use  the  learning  environment  for  discussion  by  sticking  their  contribution  at  an 
arbitrary  position  on  the  whiteboard  in  the  virtual  group  room  used  for  the  discussion.  Technically 
students  add  a link  to  another  hypermedia  page.  This  link  can  be  labeled  to  express  the  core  idea  of 
the  contribution.  The  referred  hypermedia  page  can  contain  information  in  various  formats  such  as 
text,  pictures,  and  tables. 

• Learning  repository: 

In  this  scenario  VITAL  is  used  as  a persistent  storage  of  arbitrary  information.  Thus,  teams  can 
deposit  (intermediary)  results  for  their  colleagues  or  the  tutors  in  their  group  room,  the  tutors  can 
announce  up-to-date  information,  provide  references  to  the  literature,  e.g.,  in  an  auditorium,  and 
monitor  the  progress  of  the  group. 


4.  Evaluation 

The  evaluation  of  the  course  consisted  of  student  interviews  at  the  beginning  and  at  the  end  of  the 
course,  and  questionnaires  in  the  middle  of  and  at  the  end  of  the  course. 

An  initial  interview  about  their  motivation  to  select  this  course  discovered  two  main  reasons  for  course 
selection: 

The  reason  which  was  given  by  most  of  the  participants  was  the  interdisciplinary  character  of  the 
seminar.  The  participants  were  curious  about  learning  and  working  with  students  from  another 
academic  background.  The  second-most  common  reason  was  their  interest  in  learning  more  about  the 
topic  of  computers  and  learning. 

In  the  middle  and  at  the  end  of  the  course  questionnaires  were  used  to  gather  data  from  the  students.  The 
questionnaire  consisted  of  14  questions  concerning  the  individual  work  and  the  team-work,  the  usage 
and  usefulness  of  the  communication  and  cooperation  tools  as  well  as  of  the  tools  for  the  realization  of 
the  team  projects.  Another  set  of  questions  addressed  the  satisfaction  of  the  students  with  their  group, 
the  course,  and  the  results  of  the  individual  work  and  the  team-work.  Due  to  the  small  sample  size,  all 
quantitative  results  should  be  taken  as  purely  descriptive  information  (no  statistical  test  were 
performed).  Selected  results  are  presented  in  Figure  2. 
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Figure  2:  Selected  questionnaire  results 

The  general' acceptance  of  the  course,  the  groups  and  the  results  was  rated  as  fairly  high.  With  respect 
to  the  usage  of  the  tools,  results  are  more  mixed.  Though  students  generally  had  a positive  attitude 
towards  the  tools  a few  students  had  serious  technical  problems. 

In  addition  to  the  quantitative  questionnaire  interviews  with  the  students  were  conducted.  The  issues  can 
be  grouped  in  the  following  way; 

• The  interdisciplinary  constitution  of  both  the  complete  group  and  the  teams  provided  some 
difficulties  but  very  positively  effected  the  methods  and  the  results  of  the  team-work  and  the  course 
as  a whole. 

• The  sequence  and  the  proportion  of  sessions  with  the  whole  group  and  periods  of  team-work  was 
perceived  differently  by  various  students.  A trade-off  between  the  depth  and  quality  of  team-work 
and  of  the  work  in  the  course  with  the  whole  group  was  detected. 

• The  computer-supported  communication  and  cooperation  was  different  from  earlier  e.xpenences  of 
the  students. 

In  the  following,  we  concentrate  on  the  last  issue: 

The  students  described  their  experiences  with  the  computer-based  communication  and  cooperation 
during  the  phases  of  team  working.  Three  kinds  of  such  a communication  were  used; 

• Synchronous  distributed  group  work;  Two  groups  never  used  this  possibility.  One  team  used  it  quite 
a lot  for  social  chat  but  not  for  issues  concerning  the  course.  Two  groups  communicated  via  the 
chat  tool  occasionally.  Both  complained  that  it  is  laborious  not  only  to  type  everything  instead  of 
just  speaking,  but  to  have  to  make  explicit  that  which  in  a face-to-face  discussion  is  expressed  by 
mime  or  gestures.  There  was  also  a problem  of  sequencing  since  there  were  three  students  in  each 
team  and  it  was  not  always  clear  who  answered  whom.  Nevertheless,  the  students  judged  this  kind 
of  communication  of  high  value,  when  used  for  organizational  matters. 

• Asynchronous  presentational  discussion;  Many  of  the  students  were  disappointed  by  the  standard 
of  the  discussion.  Whereas  they  liked  the  fact  that  each  student  and  both  of  the  two  lecturers  joined 
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the  discussion,  they  remarked  that  the  contributions  are  not  intertwined,  but  rather  disjointed 
statements. 

• Learning  repository:  The  usage  of  this  component  was  not  uniform.  One  team  split  their  task  into 
three  subtasks.  Each  student  solved  one  of  the  subtasks  and  placed  her/his  results  into  the 
repository.  The  others  read  and  debated  this  contribution.  They  met  only  occasionally.  Another 
group  used  it  to  exchange  literature.  However,  the  other  three  groups  did  not  employ  the  learning 
repository.  One  reason  for  this  was  the  limited  word  processing  facilities  as  well  as  the  limited 
number  of  import/export  facilities  provided  by  the  system. 

Based  on  the  results  of  the  quantitative  and  qualitative  evaluation  of  the  case  study  we  come  to  the 
following  conclusions: 

• Traditional  forms  of  discussion  cannot  be  transferred  easily  into  virtual  learning  environments. 

• Synchronous  creative,  productive  discussions  are  quite  hard  and  should  be  avoided  for  groups  of 
more  than  two  or  three  members.  However,  arrangements  are  made  much  easier  than  by  telephone 
as  soon  as  more  than  two  people  are  involved. 

• Asynchronous  discussion  of  a bigger  group  based  on  a thesis  or  a work  known  to  everybody,  has 
more  of  a brainstorming  character  and  should  only  be  used  for  this  purpose. 

• Repositories  make  sense  when  they  are  integrated  in  the  existing  tools.  Then  they  are  very  helpful 
for  exchanging  documents  and  for  giving  external  users  an  impression  about  the  status  of  the 
ongoing  work.  They  also  serve  very  well  as  blackboards  for  announcements  and  background 
information. 

The  results  match  to  evaluation  results  of  other  courses  in  which  the  learning  environment  VITAL  was 
used  in  a different  setting  during  a synchronous  lecture  (Pfister,  1999).  For  the  next  run  of  the  course, 
we  plan  to  replace  VITAL  by  its  successor,  the  CROCODILE  system.  The  CROCODILE  system 
provides  so-called  learning  protocols  (Wessner  et  al.,  1999)  which  help  the  learners  to  structure  their 
cooperative  learning  process. 
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Abstract:  As  technology  changes  so  many  parts  of  life  also  education  and  the  academic 
society  cannot  esc^e.  Because  of  the  many  different  perspectives  of  people  involved  in  higher 
education  predicting  success  of  innovations  such  as  the  integration  of  ICT  is  considered  a 
difficult  problem.  The  paper  intends  to  structure  the  perspectives  of  students,  teachers  and 
management  in  higher  education  related  to  change.  Survey  data  illustrate  the  conclusion  that 
there  is  potential  for  change  but  potential  success  is  often  characterized  by  uncertainty  such  as 
attitudes,  individual  key  experiences  and  support  infrastructures. 


Introduction 

The  quality  of  higher  education  is  both  very  difficult  to  assess  and  very  important  as  one  of  the  indicators  for 
decision-makers  to  allocate  substantial  resources  for  universities  and  colleges  to  maintain  their  academic 
education  infrastructure.  But  how  to  measure  the  quality  of  higher  education?  And  how  to  relate  quality  to 
efforts  needed  to  achieve  and  maintain  quality? 

A term  related  to  the  improvement  of  quality  in  higher  education  which  is  used  in  the  Netherlands  is 
‘studyability’  (in  Dutch:  studeerbaarheid)  (VSNU,  1998).  This  term  means  ‘the  characterization  of  the  optimal 
context  for  supporting  students  in  achieving  learning  goals’.  Avoidable  obstacles  for  students  are  to  be  taken 
away  as  much  as  possible.  It  is  expected  that,  when  the  ‘studyability’  of  a curriculum  is  improved,  also  the 
efficiency  will  improve.  Improvement  of  studyability  at  first  sight  concentrates  on  student  characteristics  and 
perspectives.  But  the  teacher  and  the  university  perspective  are  also  important  to  consider  in  the  context.  A 
research  project  has  been  started  to  investigate  ways  to  improve  studyability  at  the  University  of  Twente  by 
integrating  information  and  communication  technology  in  the  curriculum  (Wetterling  & Moonen,  1998).  The 
project  called  IDYLLE  (IDYLLE  stands  for  ‘Innovative  Distributed  Learning  Environments’)  is  situated  in  the 
Center  for  Telematics  and  Information  Technology  (CTIT),  a research  institute  at  the  University  of  Twente.  In 
the  CTIT  research  is  done  on  telematics  and  information  technology  systems,  including  the  application  of  these 
systems  in  user  environments.  The  IDYLLE  project  is  a multidisciplinary  project  containing  components  from 
educational  technology,  computer  science  and  ergonomics. 

The  paper  concentrates  on  one  of  the  parts  of  the  IDYLLE  project  that  investigates  the  implementation  process 
for  university  teaching  staff  as  Information  and  Communication  Technology  (further  to  be  referred  to  as  ‘ICT’) 
is  concerned  and  how  to  deal  with  problems  occurring  in  this  process.  Accents  are  on  the  different  perspectives 
of  analysis,  the  uncertainty  of  decision-making  and  attempts  to  structure  support  mechanisms  to  improve 
effectiveness  and  efficiency  of  using  ICT  in  higher  education. 


Scenarios  for  ICT  Integration  in  Higher  Education:  Different  Perspectives 

The  success  of  implementing  ICT  in  higher  education  is  determined  by  a variety  of  parameters  such  as 
educational,  technological  and  organizational  (Harmon  & Jones,  1999).  These  parameters  are  operational  at  a 
diversity  of  levels  such  as  the  student  level,  the  instructor  level  and  the  faculty  / university  level.  Additional 
factors  that  make  the  implementation  of  ICT  in  universities  and  the  prediction  of  success  a complex  issue  are  the 
rapid  developments  in  the  ICT  and  education  sector  and  the  difficulties  experienced  in  research  on  evaluating 
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the  impact  of  ICT  on  educational  effectiveness  (Moonen,  1997).  The  implementation  of  ICT  in  university 
teaching  can  be  analyzed  from  the  perspective  of  the  student,  the  university  or  the  instructor.  Even  tough  all  of 
these  are  relevant  to  the  success  of  integration  of  ICT  this  paper  concentrates  on  the  teacher  perspective. 

As  the  student  perspective  is  concerned,  three  aspects  are  to  be  acknowledged  in  the  context  of  ICT  in 
universities.  The  first  two  are  related  to  the  input  and  output  of  the  university.  Input  means  implementation 
targets  are  on  serving  a greater  flexibility  in  student  enrolment  such  as  more  part  time  students,  continuing 
education  target  groups  and  distance  students.  ICT  is  integrated  to  optimize  quality  for  a greater  variety  in 
student  characteristics.  Output  means  ICT  is  integrated  to  deal  with  more  specialized  requirements  for  graduates 
in  their  future  work  environment.  As  the  measurement  of  input  (enrolment  statistics)  and  output  (graduation 
rates  and  salaries  of  graduates)  indicators  are  relatively  easy  the  quality  of  the  education  process  and  the 
influence  of  ICT  integration  in  the  curriculum  is  more  difficult.  One  of  the  indicators  in  this  context  if  the  time 
students  spend  on  study  tasks.  Concerning  this  problem  a survey  was  carried  out  within  the  IDYLLE  project 
resulting  in  a model  for  determining  factors  that  influence  study  time  and  study  habits  of  students  in  university 
(Kamp,  1997a,  1997b).  The  University  perspective  concentrates  on  the  macro  level  of  analysis,  which  means 
preparing  a situation  in  which  ICT  can  be  implemented  successfully  in  parts  or  all  of  the  university.  At  the 
University  of  Twente  successful  developments  are  on  their  way  in  this  direction  (Collis,  1997,  1998). 

The  remainder  of  this  section  concentrates  on  the  teacher  perspective,  as  the  survey  results  to  be  discussed  in  the 
paper  are  directly  related  to  this  perspective.  If  the  teacher  perspective  is  compared  to  the  student  perspective 
one  of  the  main  differences  is  that  teaching  staff  in  higher  education  often  has  not  such  a varied  background  as 
students.  Even  tough  there  are  old  and  young  teachers,  teachers  with  more  or  less  experience  with  ICT  and 
teachers  in  technical,  social  or  behavioral  sciences,  they  all  have  a common  task  in  design,  delivery  and 
maintenance  of  courses  in  a specific  domain  area  where  learning  and  research  should  be  integrated.  And  by 
integration  of  ICT  roles  and  tasks  for  teachers  change  and  there  are  differences  in  how  teachers  deal  with  these 
changes  (Shotsberger,  1998).  The  teacher  perspective  in  this  paper  is  differentiated  in  three  characteristics:  first 
the  educational  scenarios,  in  which  teachers  deliver  their  courses,  second  the  personal  factors  of  the  teacher  and 
third  the  infrastructures  of  the  faculty  in  which  they  deliver  their  courses. 

A series  of  educational  scenarios  can  be  identified  in  higher  education  settings.  First  the  traditional  scenario  as 
higher  education  has  persisted  for  decades  almost  during  the  whole  century.  In  this  scenario  students  attend 
lectures  and  read  their  study  material,  mostly  books  or  readers  at  home  and  sometimes  they  go  to  workshops 
where  they  discuss  assignments  or  have  practical  lab  work  assignments.  Then  there  is  the  scenario  of  distance 
education.  This  scenario  is  characterized  by  students  who  do  not  live  near  or  on  the  campus,  the  (almost) 
absence  of  instructional  activities  where  students  and  teachers  meet  at  the  same  location  and  the  accentuation  of 
high  quality  study  material  to  structure  learning  and  instruction  for  students.  A third  and  final  scenario  is 
project-based  learning.  In  this  scenario  students  are  usually  located  near  the  campus.  Traditional  lectures  and 
workshops  are  replaced  by  group  discussions  with  a teacher  who  performs  roles  of  coach,  tutor  or  guide. 
Students  also  do  a lot  of  literature  study,  assignments  and  practical  work  either  individually  or  within  the  project 
group. 

The  use  of  the  World  Wide  Web  and  other  forms  of  ICT  can  be  integrated  more  or  less  in  all  three  scenarios.  In 
the  traditional  scenario  students  can  retrieve  lectures  from  the  web  for  later  review  and  assignments  can  be 
structured  on  the  web  thus  saving  time  at  workshops  for  discussion.  In  the  distance  scenario  written  material  can 
be  enriched  by  multimedia  and  communication  facilities  on  the  web  and  it  would  also  be  easier  to  communicate 
between  teacher  and  students.  The  work  of  project  groups  can  be  structured  on  the  web  using  workflow 
management  environments  and  offering  discussion  platforms  for  efficient  interaction  among  group  members 
and  active  learning  opportunities.  A really  useful  categorization  of  educational  scenarios  related  to  integration  of 
teleleaming  was  presented  by  Kerres  (Kerres,  1998). 

The  personal  characteristics  of  the  teacher  are  one  the  most  important  factors  to  determine  the  success  of  the 
integration  of  ICT  in  higher  education.  Let’s  differentiate  four  indicators  for  the  personal  profile  of  the  teacher. 
These  indicators  are  (1)  the  workload  of  the  teacher,  (2)  the  expertise  of  the  teacher  related  to  ICT,  (3)  the 
resources  available  to  the  teacher  and  (4)  the  attitudes  of  the  teacher  towards  integration  of  ICT.  These  four 
indicators  can  be  used  to  predict  the  success  for  a specific  teacher  to  integrate  ICT  in  his  or  her  courses.  The 
problem  (and  also  interesting  challenge)  is  how  to  estimate  the  parameters  for  the  four  indicators.  As  addition  to 
these  characteristics  incentives  are  important  to  teachers.  Change  means  more  often  than  not  extra  efforts  to  be 
invested  and  extra  efforts  need  incentives.  The  teacher  perspective  on  ICT  is  to  a large  extent  depending  on 
external  influences  such  as  incentives  and  disincentives  (Williams  & Peters,  1997). 
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The  past,  the  present  and  the  future 


The  university  of  today  is  seen  as  a component  of  society.  Today’s  society  is  rapidly  changing  due  to 
technological,  political,  sociological  and  organizational  developments.  One  of  the  main  problems  to  keep 
universities  op  to  standard  in  the  curricula  is  that  these  curricula  prepare  students  for  work  after  graduation  and 
thus  that  these  students  usually  learn  to  work  with  technologies  that  are  outdated  when  the  are  ready  to  apply 
them  in  their  work  environments.  This  means  that  universities  either  should  come  closer  to  requirements  of 
business  and  society  as  technology  and  their  applications  are  concerned;  or  a basic  curriculum  in  ICT  is  offered 
followed  by  an  advanced  program  for  those  needing  more  specialized  skills  and  knowledge  in  the  context  of 
their  working  area  after  graduation.  Both  of  these  approaches  show  a need  for  a more  flexible  and  individual 
learning  program.  Integrating  ICT  should  be  considered  as  one  of  the  solutions  to  improve  higher  education  and 
meet  the  goals  of  flexibility  and  individualization.  But  it  is  also  acknowledged  that  it  could  be  better  to  build 
new  ideas  on  existing  proven  foundations  instead  of  building  new  concepts  from  scratch  with  all  pre- 
programmed uncertainty  to  consider  (Holtham  e.a.  1998), 


Survey 

How  is  the  integration  of  ICT  in  the  university  curriculum  perceived  by  those  who  have  to  deal  with  the 
innovation  in  their  everyday  job?  A survey  has  been  conducted  among  teachers  at  the  University  of  Twente  to 
achieve  data  on  the  perspectives  of  teaching  staff  on  integration  of  ICT  and  teleleaming.  For  the  survey  a sample 
was  drawn  of  100  instructors  in  10  faculties  from  those  staff  members  involved  in  one  or  more  student  courses. 
Subjects  were  sent  an  e-mail  message  with  information  about  the  survey.  The  survey  questionnaire  was  attached 
to  the  e-mail  message.  A response  rate  of  28  was  achieved  for  the  survey.  The  results  unveil  two  major 
characteristics  of  the  analysis  of  success  of  potential  ICT  in  higher  education:  these  characteristics  are 
uncertainty  and  trends.  Uncertainty  because  of  the  difficulty  to  predict  effects  of  decisions  made  today  on 
tomorrow’s  situations  and  trends  because  of  the  preference  at  teacher  and  institution  level  to  some  ICT 
application  over  others.  Another  problem  with  interpreting  results  of  the  survey  is  the  background  of  the 
respondents.  Subjects  that  already  either  have  a stronger  attitude  to  ICT  or  use  ICT  in  their  courses  can  be  more 
motivated  to  return  the  survey  which  can  bias  results.  The  questionnaire  contained  12  closed  questions  about  the 
following  topics:  Opportunities  for  ICT  in  university  courses,  instructors  experience  with  different  ICT  tools, 
the  ICT-infrastructure  in  the  university,  teacher  support  and  reasons  for  using  ICT. 

The  survey  started  with  two  general  statements.  These  were  used  as  indicators  for  the  expectations  of  instructors 
from  ICT  on  student  learning  and  quality  of  education.  The  first  statement  was  “ICT  has  a positive  impact  on 
student  learning”.  On  this  statement  11  respondents  either  agreed  or  strongly  agreed.  Three  respondents 
disagreed  and  only  one  strongly  disagreed.  The  second  statement  was  “ICT  will  improve  higher  education 
quality”.  On  this  statement  18  respondents  either  agreed  or  strongly  agreed.  One  respondent  disagreed  and  there 
were  no  ‘strongly  disagree’  responses.  Another  item  asking  for  longer-term  expectations  of  ICT  in  university 
teaching  confirmed  these  data  as  14  respondents  rated  this  high  against  6 low  and  7 had  no  opinion. 

One  item  asked  for  the  ICT  tools  from  which  instructors  expected  the  most  for  improvement  of  the 
communication  between  instructors  and  students  and  among  students.  In  this  item  e-mail  and  the  World  Wide 
Web  were  rated  highest  whereas  chat  and  videoconferencing  were  rated  lowest.  This  has  a high  correlation  with 
item  4,  asking  for  instructor’s  experience  with  ICT  tools.  Here  also  e-mail  and  WWW  were  rated  highest  and 
chat  lowest.  Another  item  later  in  the  questionnaire  asked  what  tools  respondents  used  most.  Here  also  e-mail 
and  the  WWW  scored  highest.  The  correlation  between  these  two  items  is  not  unexpected.  It  also  indicates  that 
future  developments  should  consider  carefully  the  prior  knowledge  and  experience  of  the  teachers  in  the  target 
group  to  optimize  the  impact.  On  the  other  hand  low  instructor  experience  should  not  be  considered  too  much  a 
reason  to  avoid  investigating  the  potential  of  these  technologies  in  the  university. 

Another  item  asked  for  the  support  instructors  receive  from  the  faculty  or  university  level.  Response  shows  a 
high  perception  of  motivational  (15  out  of  28  (strongly)  agree)  and  financial  (12  out  of  28  (strongly)  agree) 
support  from  the  side  of  the  management  but  instructors  do  not  often  perceive  they  have  sufficient  time  to 
integrate  ICT  in  their  courses  (8  out  of  28  (strongly)  agree).  The  University  of  Twente  has  three  major  support 
institutes  for  the  faculties.  These  are  the  libraries,  the  computing  center  and  the  educational  support  center. 
Respondents  were  asked  how  important  they  find  these  three  institutes  on  a 5-point  scale.  16  out  of  28 
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respondents  rated  the  educational  support  center  as  important  or  very  important.  10  out  of  28  gave  this  rating  to 
the  computing  center  and  1 1 out  of  28  for  the  library.  There  is  a trend  to  combine  these  types  of  support 
institutes  in  one  large  organization  in  order  to  make  service  more  efficient  and  more  effective  within  the 
university. 

What  factors  influence  the  quality  of  ICT  in  university  courses?  Respondents  were  given  five  factors,  which 
they  rated  on  a five-point  scale.  These  factors  were:  ‘instructor  capacities’,  ‘student  motivation’,  ‘quality  of 
learning  material’,  ‘technology  policy  of  the  university’  and  ‘reliability  of  the  hardware  and  software’.  Four  out 
of  five  of  these  factors  scored  very  high  in  the  item  (21  or  more  out  of  28  respondents  found  the  factor  important 
or  very  important).  Only  ‘technology  policy  of  the  university’  was  rated  a little  lower,  even  tough  almost  half  of 
respondents  rated  this  as  (highly)  important.  ICT  is  expected  to  have  more  impact  in  individualized, 
personalized  education  activities  rather  than  in  group  teaching.  Respondents  were  asked  to  indicate  for  a series 
of  educational  activities  whether  they  think  ICT  could  have  a positive  impact  on  the  quality  of  these  types  of 
activity  in  the  future.  Activities  in  which  the  most  respondents  expected  a positive  ICT  impact  were  the 
presentation  of  learning  material,  the  coaching  of  project  groups  and  the  individual  coaching  of  students. 

The  last  item  asked  respondents  about  the  reason  why  they  use  ICT  in  their  courses.  Respondents  were  given 
eight  ‘reasons’  on  which  they  could  answer  ‘yes’,  ‘no’  or  ‘don’t  know’.  The  two  reasons  that  scored  highest  on 
this  item  (18  and  19  times  ‘yes’)  were  ‘the  course  content  ha§  opportunities  for  ICT  use’  and  ‘the  innovation- 
mindedness  of  the  instructor’.  About  eight  or  nine  respondents  circled  ‘yes’  on  the  reasons  ‘too  many  students 
fail  exams  on  my  course’,  ‘the  university  wants  me  to  use  ICT’  and  ‘students  complain  about  the  course’.  A 
very  small  majority  indicates  that  they  use  ICT  to  decrease  their  workload  or  to  increase  student  numbers.  No 
respondent  uses  ICT  to  decrease  student  workload  in  the  course. 


Conclusion 

Getting  ICT  into  the  university  and  especially  realize  that  staff  uses  ICT  in  a way  that  improves  quality  and 
efficiency  of  the  university  product  such  as  education  and  research  is  perceived  as  a long  an  enduring  process. 
All  phases  in  this  process  have  their  own  characteristics  influencing  the  success  or  failure  of  the  integration 
process.  This  includes  situational  characteristics  such  as  the  availability  of  resources  or  infrastructure  but  also 
personal  characteristics  such  as  the  attitude  of  teaching  staff  and  students  or  expertise  of  instructors.  A large 
variety  of  these  factors  are  expected  to  have  impact  on  the  probability  of  successful  integration  of  technology. 
So  what  can  be  said  about  the  way  ICT  is  integrated  in  the  university  and  the  way  instructors  deal  with  this 
process  on  the  basis  of  the  data  collected  and  analyzed  in  the  survey  in  this  study?  First  there  seem  to  be  high 
expectations  and  positive  perspectives  on  the  side  of  the  instructors  in  higher  education.  But  on  the  other  hand 
many  instructors  have  some  resistance  to  ICT  because  of  the  initial  efforts  needed  to  get  things  started.  As  the 
ICT  tools  are  concerned,  results  show  that  especially  e-mail  and  the  World  Wide  Web  is  used  frequently.  It 
could  be  interesting  to  find  out  how  other  areas  such  as  videoconferencing  or  groupware  can  be  stimulated 
among  university  staff  Another  aspect  in  the  survey  is  the  role  of  the  support  institutions  such  as  the  library,  the 
computing  center  and  the  educational  center.  There  is  a trend  to  integrate  these  support  institutions  to  increase 
efficiency  in  universities.  The  rather  equal  distribution  of  how  respondents  perceive  the  value  of  the  three 
institutions  seem  to  support  integration  efforts  but  more  research  is  needed  to  validate  the  impact.  A final 
conclusion  deals  with  the  value  of  ICT  for  specific  instructional  activities.  Results  show  a high  score  for  ICT  as 
to  improve  communication  and  interaction  scenarios  in  course  delivery.  Especially  the  support  of  student  groups 
by  the  use  of  ICT  is  perceived  as  being  a valuable  contribution  and  addition  to  traditional  instruction. 
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Abstract:  This  paper  will  report  on  a pilot  study  which  surveyed  faculty  to  determine  their 

attitudes  towards  WebCT  during  its  first  year  of  use  at  George  Mason  University.  Faculty  who 
responded  to  the  survey  seemed  to  be  generally  satisfied  with  both  the  WebCT  tool  and  the  WebCT 
support  available  on  campus.  However,  faculty  also  reported  barriers  to  WebCT  usage,  including 
difficulties  using  the  designer  interface  and  the  amount  of  time  involved  in  learning  WebCT  and 
setting  up  a WebCT  folder. 


Introduction 

In  January  1999,  George  Mason  University  began  use  of  WebCT,  an  online  course  management  system  designed  to 
facilitate  the  creation  of  web-based  instructional  material  and  instructional  supports.  Currently,  WebCT  is  being 
used  by  some  faculty  and  is  considered  to  be  in  its  early  stages  of  implementation.  Up  to  the  present  time,  no 
systematic  assessment  of  WebCT’ s implementation  has  taken  place.  This  paper  presents  a pilot  study  attempting 
such  an  assessment  by  looking  at  faculty  satisfaction  with  WebCT.  Data  for  the  current  study  was  gathered  through 
surveys,  and  a large  student  including  both  surveys  and  interviews  with  selected  faculty  is  planned.  Before 
beginning  a discussion  of  the  data,  a definition  of  an  online  course  management  system  is  provided. 

What  is  WebCT? 

WebCT  is  an  on-line  course  management  system  that  assists  faculty  in  the  creation  of  complex  educational  websites. 
It  provides  faculty  with  three  types  of  tools:  (1)  an  interface  to  help  with  aspects  of  page  design  and  layout,  (2) 
educational  features  such  as  a quiz-creation  tool,  a glossary,  and  a student  presentation  tool,  and  (3)  a course 
management  system  to  help  instructors  manage  and  distribute  grades,  and  release  course  content  according  to 
grades,  dates,  and  other  related  criteria.  (Goldberg,  1997b) 

Related  Research 

According  to  Phipps,  R.,  & Merisotis,  J.  (1999)  “There  is  a relative  paucity  of  true,  original  research  dedicated  to 
explaining  or  predicting  phenomena  related  to  distance  learning.”  (Page  2).  Much  of  the  body  of  research  that  does 
exist  dealing  with  on-line  learning  “are  more  anecdotal  than  systematically  empirical  or  critical.”  (Hara  & Kling 
1999).  In  addition  Windschilt  (1998,  as  quoted  by  Hara  & Kling  1999)  also  notes  that  much  of  the  published 
literature  dealing  with  computer-mediated  on-line  instruction  consists  of  descriptions  of  software  or  anecdotal 
accounts  of  how  technology  was  integrated  in  a course  or  classroom. 

To  date,  there  has  been  virtual  no  research  on  faculty  reaction  to  WebCT  or  other  on-line  course  management 
software.  The  only  article  published  dealing  with  faculty  attitudes  have  been  “testimonials”  published  by  faculty  on 
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their  individual  experiences  with  teaching  on  now  (Ruth,  Foreman,  & Tschudy  1999,  Goldberg  1996).  Although 
these  articles  are  usually  positive,  they  are  usually  written  by  faculty  who  were  previous  technology  (Phipps  & 
Merisotisl999)  or  even  the  developer  of  the  software  they  are  using  in  the  course  (Goldberg,  1996).  Although  their 
research  focused  primarily  on  students,  Wermet  & Oliges  (1999)  noted  that  learning  curve  was  high  for  faculty 
learning  WebCT.  Pauley  & Cunningham  (1999)  noted  that  faculty  emphasized  the  importance  of  developing  an 
understanding  of  the  effects  & implications  of  distance  learning  & needs  of  adult  learner.  They  also  noted  that 
faculty  emphasized  the  importance  of  the  instructional  support  staff 

An  investigation  and  review 

The  investigations  were  design  to  explore  and  gather  information  that  could  possibly  be  used  to  formulate  a 
hypothesis  to  be  tested.  The  following  sections  discuss  the  methodology  used  and  threats  to  the  validity  and 
reliability  of  the  data  collected. 

Faculty  Survey  Process 

The  faculty  reaction  survey  consisted  on  19  questions  dealing  with  faculty  reaction  to  the  WebCT  software  as  well 
as  their  judgment  as  to  the  quality  of  support  provided  by  the  university’s  Instructional  Resource  Center.  Questions 
also  included  asking  if  they  had  ever  used  the  Internet  for  instruction  previously  and  how  many  WebCT  workshops 
they  participated  in.  The  survey  was  then  sent  out  on  the  university’s  WebCT_users  e-mail  list,  to  which  all  faculty 
are  added  when  they  request  their  first  WebCT  course.  The  list  usually  serves  as  a forum  for  announcements 
concerning  WebCT,  discussion  of  problems  with  WebCT,  and  posts  of  upcoming  WebCT  related  campus  events.  In 
addition  to  the  WebCT_users  list,  the  survey  was  also  sent  to  all  faculty  and  staff  who  had  attended  a WebCT 
workshop  since  March.  An  e-mail  was  sent  out  to  all  the  above-mentioned  faculty  and  staff  containing  the  required 
cover  letter  for  research  projects  exempt  from  Human  Subjects  Review.  In  the  body  of  the  e-mail  below  the  cover 
letter  information  was  the  survey.  Participants  could  complete  the  survey  in  one  of  three  ways:  (1)  print  out  the  e- 
mail,  fill  it  out  by  hand,  and  send  it  back  via  intercampus  mail,  (2)  reply  to  the  e-mail  and  type  their  responses  right 
in  their  reply,  and  (3)  completing  an  on-line  survey  at  a URL  provided.  Faculty  were  given  10  days  to  complete  the 
survey.  The  multiple  modes  of  completing  the  survey  were  designed  to  encourage  participation  and  also  to  prevent 
the  possibility  of  difficulty  filling  out  the  survey  affecting  the  outcome  of  the  study. 

Threats  to  validity  and  reliability 

As  all  faculty  at  the  university  involved  in  WebCT  were  surveyed,  threats  to  validity  due  to  selection  were  not 
relevant.  However,  as  with  all  survey  studies,  results  are  based  on  those  subjects  who  choose  to  participate  and 
return  the  survey  and  who  are  therefore  self-selected.  Due  to  the  desire  of  the  Instructional  Resource  Center  to 
gather  data  from  as  many  faculty  as  possible  and  due  to  the  short  amount  of  time  available  for  this  study,  this  threat 
to  internal  validity  was  not  compensated  for  and  must  be  taken  into  account  in  our  discussion  of  results.  In  addition, 
WebCT  has  only  been  used  at  George  Mason  University  since  January  1999,  and  most  faculty  who  have  been 
trained  in  WebCT  and/  or  decided  to  use  this  tool  began  only  this  past  summer.  Consequently,  faculty  attitudes  may 
be  tainted  by  a novelty  affect,  and  a follow-up  study  after  several  more  semesters  of  WebCT  usage  is  planned. 

Since  this  is  the  first  known  study  of  faculty  reaction  to  WebCT,  there  currently  exists  no  other  surveys  or  research 
with  which  to  compare  either  the  instrument  or  results  of  this  study.  However,  the  survey  was  constructed  in  such  a 
way  that  it  would  be  easily  adaptable  to  other  institutions  for  future  research.  Most  questions  in  the  study  were 
multiple  choice,  precluding  the  possibility  that  researcher  bias  could  affect  the  results.  In  addition,  there  were  a 
number  of  free  response  questions  which  will  be  discussed  below. 

Preliminary  Survey  Analysis 

Survey  response  rate: 

One  hundred  seven  people  received  the  request  to  complete  the  survey.  Of  those  eleven  were  eliminated  because 
they  were  staff  members  in  the  department  running  the  survey  rather  than  faculty.  Of  the  96  remaining,  twenty-two 
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had  completed  the  survey  during  the  time  period  requested,  a 23%  response  rate.  A few  additional  surveys  have 
been  returned  since  the  deadline,  but  they  have  not  yet  been  added  to  the  database  yet. 

Overall  level  of  satisfaction: 

As  for  WebCT  itself,  faculty  rated  it  at  an  average  of  2.05  on  a scale  of  one  (excellent)  to  four  (very  poor),  with  only 
2 respondents  saying  that  WebCT  was  either  poor  (3)  or  very  poor  (4).  A large  majority  gave  WebCT  a rating  of  2 
(good).  Faculty  who  have  actually  used  WebCT  with  their  students  generally  reported  that  their  students  level  of 
satisfaction  was  good.  In  fact,  faculty  rated  students  as  slightly  more  satisfied  than  the  faculty,  with  an  average 
rating  of  1 .9375.  Faculty  were  then  asked  to  rate  the  usefulness  of  particular  WebCT  tools  on  a scale  of  one  (very 
useful)  to  four  (very  unuseful).  Few  faculty  rated  all  tools,  since  they  were  asked  to  only  rate  the  tools  they  had 
experience  with.  Ratings  ranged  from  an  average  of  1.43  for  the  bulletin  board  to  2.22  for  the  White  Board.  Very 
few  tools  received  scores  of  very  unuseful,  although  ratings  of  unuseful  were  somewhat  common. 

Level  of  satisfaction  w/  training  & support: 

Faculty  who  responded  to  the  survey  seemed  to  be  generally  satisfied  with  both  the  WebCT  tool  and  the  WebCT 
support  available  on  campus.  On  a scale  of  one  to  four,  with  one  being  the  best,  faculty  rated  WebCT  workshops  as 
1 .47.  While  only  approximately  half  of  the  respondents  took  advantage  of  opportunities  for  one-on-one  help,  those 
who  did  rated  it  very  highly,  with  eleven  respondents  saying  this  help  with  excellent  (1)  and  two  replying  that  the 
one-on-one  help  was  good  (2)  for  an  rating  average  of  1 .15. 

Best  / Worst  aspects  of  WebCT: 

Faculty  expressed  the  most  satisfaction  with  the  things  that  WebCT  did  to  make  their  lives  easier.  These  included 
the  ability  to  post  grades  for  students  and  to  cut  down  on  the  number  of  paper  copies  that  needed  to  be  distributed, 
and  the  self-grading  quizzes  with  automatic  feedback. 

Faculty  thought  the  biggest  drawback  to  WebCT  was  the  “clunky  interface.”  It  is  “over-complication”  with  a 
“somewhat  scattered  and  cumbersome  array  of  elements.”  Another  big  drawback  was  the  time  involved  in  learning 
WebCT  and  setting  up  a WebCT  folder.  “I  still  don't  know  how  to  use  most  of  the  applications,”  complained  one, 
“That  is,  it  is  tremendously  time-consuming  to  prepare  course  materials  for  on-line  use  and  I just  don't  have  much 
time.” 

In  fact,  most  of  the  faculty  who  are  not  using  WebCT  cited  either  the  clumsy  interface  or  their  lack  of  time  as  their 
reason.  “I  haven't  decided  NOT  to  use  it,  but  haven't  had  the  time  to  get  geared  up  yet,”  said  one.  Another 
responded,  “a  program  should  not  have  to  require  so  [many]  workshops.  We  ought  to  be  able  to  figure  [it]  out 
ourselves. . .it  made  things  a lot  harder  to  do.” 

Conclusions  and  future  directions: 

Although  the  data  collected  so  far  represent  one  of  the  few  attempts  to  collect  information  on  faculty  reaction  to 
WebCT,  it  is  currently  only  skimming  the  surface.  While  faculty  seem  to  be,  by  and  large,  satisfied  with  WebCT 
and  with  the  WebCT  support  offered  at  George  Mason  University,  it  is  also  clear  that  they  find  some  aspects  of 
WebCT  confusing  and  that  many  also  find  it  difficult  to  set  aside  the  time  that  they  need  to  learn  WebCT  and  design 
their  WebCT  folder.  In  order  to  get  a clearer  understanding  of  how  faculty  are  using  WebCT,  how  successful  they 
feel  that  that  use  is,  and  how  the  university  could  best  support  WebCT  course  development,  it  will  be  necessary  to 
do  further  research.  Interviews  with  randomly  selected  WebCT  faculty  are  planned.  In  addition,  we  plan  to  send  a 
reminder  to  all  faculty  on  the  webct_users  distribution  list  to  attempt  to  get  surveys  from  those  who  did  not  complete 
them  by  the  initial  deadline. 
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Abstract:  The  paper  reviews  the  past  uses  of  Virtual  Reality  (VR)  applications  for 
students  with  special  needs.  Although  VR  technology  shows  promise  for  helping  these 
groups,  applications  have  been  slow  to  emerge.  Reasons  for  the  lag  in  development 
include  the  expense  of  VR  components,  special  skills  required  to  develop  learning 
environments,  and  special  challenges  of  educational  applications.  Research  in  the  use  of 
VR  in  physical  rehabilitation  and  remediation  for  cognitive  deficits  are  discussed. 
Design  issues  and  variables  are  reviewed  together  with  a description  of  new 
developments  in  the  use  of  low-end  VR  systems  and  the  Internet. 


Background  on  Virtual  Reality  Applications  for  Special  Needs  Students:  Potential 
vs.  Performance 

It  is  likely  that  no  technology  has  created  greater  hope  for  improving  the  quality  of  life  and  learning  for 
students  with  cognitive  or  physical  disabilities  than  Virtual  Reality  (VR).  Although  VR  environments  for 
military  training  were  first  developed  over  20  years  ago,  and  applications  in  education  have  been  in 
development  since  the  early  1 990’s,  VR  applications  to  remedy  or  compensate  for  cognitive  or  physical 
disabilities  have  been  much  slower  to  emerge.  Several  reasons  seem  to  account  for  this  lag. 

• Expense  of  VR  components  - Until  recently,  the  software  and  devices  required  to  develop  and 
use  VR  environments  were  so  expensive  as  to  be  prohibitive  for  schools,  since  they  requried  well- 
funded  research  and  development  settings. 

• Special  skills  required  - Because  of  the  expense  and  complexity  of  the  systems,  few  educators 
have  skills  required  to  design  virtual  reality  programs.  Most  environments  have  been  developed 
in  university  labs  or  by  special  projects.  Long  term  research  and  development  efforts  to  address 
the  needs  of  special  populations  have  been  rare. 

• Special  challenges  of  educational  applications  - Students  with  special  needs  usually  cope  with 
a combination  of  physical  and  cognitive  problems  that  make  learning  in  any  environment 
problematic.  VR  environments  can  add  another  layer  of  required  adaptation. 
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• Difficulty  of  demonstrating  need  - The  expense  of  VR  systems  in  comparison  to  the  small 
number  of  people  in  various  special  needs  populations  has  made  it  difficult  to  justify  extensive 
development  in  this  area. 

It  is  apparent  that  the  relatively  high  costs  of  VR  systems  has  been  the  single  greatest  obstacle  to  past 
progress  in  this  area.  However,  recent  developments  in  low-end  VR  systems  may  decrease  the  impact  of 
all  these  factors.  This  paper  describes  examples  of  past  work  in  this  area  and  work  currently  under 
development  by  the  authors.  The  presentation  accompanying  this  paper  will  demonstrate  sample  VR 
systems  that  use  either  lower-end  software  or  the  Internet  as  a means  of  accessing  virtual  environments. 

Research  To  Date 

Descriptions  of  VR  applications  for  students  with  special  needs  have  appeared  both  in  medical  and 
education  literature  since  around  1992.  Environments  in  experimental  phases  have  been  described  in  two 
major  areas: 

• Physical  rehabilitation  - VR  environments  have  been  designed  to  restore  skilled  movement  in 
students  with  physical  disabilities  resulting  from  various  traumas  and  diseases  (Andrae,  1996; 
Ira,  1997;  Kuhlen  & Doyle,  1994;  Latash,  1998;  Rose,  Johnson,  & Attree,  1997).  Some  systems 
have  focused  on  the  needs  of  people  with  motor  problems  due  to  cognitive  processing  difficulties. 
Krais,  Kuhlen,  and  Friedewald  (1993)  reported  on  the  use  of  systems  to  assess  apraxias,  loss  of 
ability  to  organize  and  coordinate  movement  sequences  spatially  and/or  chronologically,  Molendi 
and  Patriarca  (1992)  explored  environments  for  helping  people  coping  with  ataxia,  a condition  in 
which  visual  stimuli  are  highly  distorted,  Stanton,  Foreman,  & Wilson,  1996;  Stanton,  Foreman, 
Martin,  & Wilson,  1997)  used  VR  to  teach  building  navigation  to  students  in  wheelchairs, 
teaching  them  to  locate  and  activate  fire  alarms  and  hoses  and  find  exits  from  the  building, 

• Remediation  for  cognitive  deficits  - Cromby,  Standen,  Newman,  and  Tasker  (1996)  assessed 
the  utility  of  ’Virtual  shopping  trips"  on  the  life  skill  development  of  students  with  moderate  to 
severe  degrees  of  mental  retardation.  Christiansen,  Abreu,  Ottenbacher,  Huffman,  Masel,  and 
Culpepper  (1998)  evaluated  the  feasibility  of  virtual  reality  meal  preparation  procedure  with 
thirty  subjects  who  had  mental  deficits  due  to  Traumatic  Brain  Injury  (TBI).  Strickland,  Marcus, 
Mesibov,  and  Hogan  (1996)  worked  with  children  exhibiting  autistic  behaviors,  placing  them  in 
immersive  environments  that  simulated  safe  procedures  for  crossing  a street. 

As  recent  reviews  have  noted  (Cass  & Roblyer,  1999;  Roblyer  & Cass,  1999),  few  of  these  reports  include 
any  actual  research  and  data  on  the  design  problems  and  impact  of  VR  environments.  Interestingly,  most 
of  the  research  results  come  from  two  groups  in  England:  research  teams  at  the  University  of  Nottingham 
and  the  University  of  Leicester.  It  is  evident  from  the  limited  data  available  that  students  vary 
considerably  in  the  length  of  time  it  takes  them  to  adapt  to  this  kind  of  environment.  However,  each  of  the 
R&D  groups  experimenting  with  this  technology  feel  that  their  results  to  date  encourage  them  to  be 
optimistic  about  VR’s  future  impact. 

Selecting  VR  Environments  to  Design 

Even  though  VR  costs  are  coming  down,  the  complexity  of  these  systems  and  the  time  and  expense  of 
designing  them  make  it  imperative  that  we  assess  carefully  the  reasons  for  choosing  VR  as  an  educational 
environment  and  the  feasibility  of  implementing  it  in  schools  and  classrooms.  Two  areas  of  criteria  seem 
to  emerge  from  past  applications  that  can  help  us  determine  the  future  usefulness  of  VR. 

• Safety  concerns  - When  students  need  to  acquire  skills  in  areas  that  help  them  become 
functional,  independent  human  beings  (e.  g.,  cooking,  crossing  streets),  but  learning  these  skills 
involves  a degree  of  risk  to  their  personal  safety,  VR  seems  to  be  the  learning  medium  of  choice. 


• Mobility  issues  - Because  of  its  capability  to  simulate  real  environments  using  varying  degrees 
of  real-life  complexity,  VR  also  seems  the  best  choice  for  students  who  need  to  re-acquire 
physical  or  mental  abilities  lost  due  to  illness  or  accident  or  to  make  up  for  physical  limitations. 
Though  it  is  not  often  recognized  as  a priority,  some  educators  who  work  with  students  with 
physical  disabilities  have  high  hopes  for  improving  the  quality  of  life  of  these  students  through 
VR  simulated  participation  in  sports  and  games. 

Design  Issues  and  Variables 

From  the  limited  experience  educators  and  trainers  have  had  with  designing  VR  learning  environments,  it 
is  evident  that  this  already-complex  endeavor  carries  even  more  problems  and  issues  when  the  target 
population  is  students  with  special  needs.  It  is  critically  important  that  research  studies  help  us  address  the 
design  and  implementation  issues  that  have  emerged  from  prior  work. 

• Side  effects  from  immersion  environments  - Some  studies  have  observed  symptoms  similar  to 
motion  sickness  (Latash,  1998),  and  others  (Strickland,  Marcus,  Mesibov,  & Hogan,  1996)  have 
noted  difficulties  adapting  to  headgear.  These  reports  are  rare,  but  cannot  be  ignored. 

• Transfer  issues  - Several  different  kinds  of  challenges  emerge  in  developing  systems  to  teach 
skills  that  will  transfer  successfully  to  real  life  activities.  Even  fully  immersive  environments  with 
goggles  and  data  gloves  do  not  provide  perceptual  experiences  identical  to  real  environments. 
There  are  differences,  and  the  impact  of  differences  on  skill  transfer  are  largely  unknown. 

• False  safety  issues  - Another  problem  related  to  transfer  of  skills  has  been  hypothesized  but 
remains  unexplored.  Some  students  with  mental  deficits  may  learn  to  feel  such  confidence  in 
their  ability  to  operate  in  this  less  complex,  simulated  world  that  they  achieve  a "learned 
recklessness”  in  real  environments.  If  this  aspect  is  ignored,  it  has  the  potential  to  negate  the 
benefits  offered  by  VR  systems. 

New  VR  Development  Resources 

Recently,  VR  software  systems  have  been  developed  that  do  not  require  the  high-end  computer  systems  of 
prior  VR  generations.  Some  of  the  groups  that  have  developed  these  systems  have  found  that  completely 
immersive  environments  do  not  appear  to  be  necessary  to  achieve  the  desire  impact.  Thus,  rather  than 
requiring  headsets  and  data  gloves,  students  can  manipulate  the  environment  using  a 3D  mouse  and  a 
standard  monitor. 

An  added  capability  that  makes  these  systems  more  feasible  for  implementation  in  schools  is  being  able  to 
access  them  over  the  Internet.  One  current  example,  SuperScape  software  (developed  by  SuperScape, 
Inc.),  which  was  used  by  several  of  the  R&D  groups  described  in  this  paper,  provides  a plug-in  that  can  be 
downloaded  from  their  website  and  would  allow  schools  to  use  the  environments  with  a browser  and  a 
desktop  computer.  More  such  Internet-accessible  VR  environments  are  sure  to  follow  in  the  near  future. 

Conclusion 

As  development  continues,  other  issues  may  emerge  that  must  be  addressed  before  VR  environments  can 
be  made  functional  components  of  educational  plans  for  students  with  special  needs.  However,  as  one 
recent  reviewer  noted,  VR  represents  "the  new  frontier  for  a field  that  represents  perhaps  the  most 
exciting  pioneering  work  in  educational  technology"  (Roblyer  & Cass,  1999,  p.  53).  Recent  developments 
in  VR  software  and  Internet-access  resources  also  may  mean  we  are  closer  to  closing  the  gap  between 
potential  and  performance  of  VR  for  special  needs  students. 


References 


Andrae,  M,  (1996),  Virtual  reality  in  rehabilitation,  British  Medical  Journal  International,  5/2(7022),  4-5, 

Botella,  C,,  Banos,  R,,  Perpina,  C,,  Villa,  H,,  Alcanez,  M,,  & Rey,  A.  (1998),  Virtual  reality  tretament  of 
claustrophobia:  A case  report.  Behavior  Research  and  Therapy,  36,  239-246. 

Brown,  D,,  Kerr,  S,,  & Wilson,  J,  (1997),  Virtual  environments  in  special-needs  education.  Communications  of  the 
ACM,  40(8),  72-75. 

Cass,  M,,  & Rob  Iyer,  M.  D.  (1999).  A review  of  experimental  research  on  virtual  reality  in  special 
education:  The  status  of  the  dream.  VR  in  the  Schools,4(2)  [Online],  Available  at: 
http://soe.eastnet.ecu.edu/vr/vrits/4-2cass.htm 

Christiansen,  C,,  Abreu,  B.,  Ottenbacher,  K.,  Huffman,  K.,  Masel,  B.,  and  Culpepper,  R,  (1998),  Task 
performance  in  virtual  environments  used  for  cognitive  rehabilitation  after  traumatic  brain  injury.  Archives  of 
Physical  Medicine  and  Rehabilitation  79,  888-892. 

Cromby,  J,,  Standen,  P.,  & Brown,  D.  (1995),  Using  virtual  environments  in  special  education.  VR  in  the  Schools, 
/(3)  [online].  Available  at:  http://eastnet.educ.ecu.edu/vr/vrits/l-3Cromb.htm 

Cromby,  J.,  Standen,  P.,  Newman,  J.,  & Tasker.  H.  (1996).  Successful  transfer  to  the  real  world  of  skills  practised 
in  a virtual  environment  by  students  with  severe  learning  disabilities.  In  Proceedings  of  the  First  European 
Conference  on  Disability,  Virtual  Reality,  and  Associated  Technologies,  Maindenhead,  Berkshire,  UK,  pp.  93-101. 

Darrow,  M.  (1995).  Increasing  research  and  development  of  virtual  reality  in  special  education:  What  about  mental 
retardation?  VR  in  the  Schools,  /(3)  [online].  Available  at;  http://eastnct.educ.ecu.edu/vr/vrits/l-3Darro.htm 

Dickson,  S.  (1996).  Scientific  concretization:  Relevance  to  the  visually  impaired  student.  VR  in  the  Schools,  /(4). 
[online  ].  Available  at:  http://eastnet.educ.ecu.edu/vr/vrits/l-3Darro.htm 

Ira,  V.  (1997).  Virtual  reality  and  mobility  skills.  Exceptional  Parent,  27(1 1),  50. 

Kuhlen,  T,,  & Doyle,  C.  (1994).  Virtual  reality  for  physically  disabled  people.  Computing  in  Biological  Medicine, 
25(2),  205-211. 

Krais,  K.,  Kuhlen,  T.,  & Friedewald,  M.  (1993).  Einsatz  von  multimedia-und  virtual  reality-techniken  fur  die 
diagnostik  und  therapie  motorischer  storungen.  Technical  report.  Institute  of  Technical  Computer  Science,  Aachen 
Technical  University. 

Latash,  M.  (1998).  Virtual  reality;  A fascinating  tools  for  motor  rehabilitation:  To  be  used  with  caution.  Disability 
and  Rehabilitation,  20(3),  104-105. 

Middleton,  T.  (1992).  Applications  of  virtual  reality  to  learning.  Interactive  Learning  International,  5,  253-257. 

Molendi,  G.,  & Patriarca,  M.  (1992).  Virtual  reality,  medical  researches.  Technical  Report  1/92,  Universita  degli 
Studi  di  Milano. 

Muscott,  H.,  & Gifford,  T.  (1994).  Virtual  reality  and  social  skills  training  for  students  with  behavioral  disorders: 
Applications,  challenges,  and  promising  practices.  Education  and  Treatment  of  Children,  77(4),  417-434. 

Powers,  D.,  & Darrow,  M.  (1994).  Special  education  and  virtual  reality:  Challenges  and  possibilities.  Journal  of 
Research  on  Computing  in  Education,  27,  111-121. 


11S2 


Rose,  F.,  Johnson,  D.,  & Attree,  E.  (1997).  Rehabilitation  of  the  head-injured  child:  Basic  research  and  new 
technology.  Pediatric  Rehabilitation,  7(1),  3-7. 

Roblyer,  M.  D.,  & Cass,  M.  (1999).  Virtual  reality  in  special  education:  Still  more  potential  than  performance. 
Learning  and  Leading  With  Technology,  2d(8),  51-53. 

Stanton,  D.  (1997).  Enhancing  spatial  cognition  in  disabled  children  using  virtual  reality.  Unpublished  dissertation 
thesis.  University  of  Leicester. 

Stanton,  D.,  Foreman,  N.,  & Wilson,  P.  (1996,  July).  Uses  of  virtual  reality  in  training:  Developing  the  spatial 
skills  of  children  with  mobility  impairments.  In  Shakey,  P.  (Ed.).  Proceedings  of  the  First  European  Conference 
on  Disability:  Virtual  Reality  and  Associated  Technologies,  Maidenhead,  England. 

Stanton,  D.,  Foreman,  N.,  Martin,  & Wilson  (1997).  Successful  transfer  of  spatial  knowledge  from  a virtual  to  a 
real  school  environment  in  physically  disabled  children.  Unpublished  report.  University  of  Leicester. 

Strickland,  D.,  Marcus,  L.,  Mesibov,  G.,  & Hogan,  K.  (1996).  Brief  report:  Two  case  studies  using  virtual  reality  as 
a learning  tool  for  autistic  children.  Journal  of  Autism  and  Developmental  Disorders,  26{6),  651-659. 

Taylor,  W.  (1997).  Student  responses  to  their  immersion  in  a virtual  environment.  Paper  presented  at  the  Annual 
Meeting  of  the  American  Educational  Research  Association,  Chicago,  Illinois,  March  24-28  (ERIC  Document  No. 
407  931) 

Wilson,  P.,  Foremen,  N.,  & Tlauka,  M.  (1996).  Transfer  of  spatial  information  from  a virtual  to  a real  environment 
in  physically  disabled  children.  Rehabilitation  Engineering,  18(12),  633-637. 


11S3 


Implementing  a 'Scaffolding  by  Design'  Model  in  a WWW-based  Course  Con- 
sidering Cost  and  BeneHts 


Koos  Winnips 
Betty  ColHs 
Jef  Moonen 

Faculty  of  Educational  Science  & Technology,  University  of  Twente,  The  Netherlands 
Contact  author:  winnips@edte.utwente.nl 


Abstract:  A model  for  'scaffolding  by  design'  is  implemented  in  a course  (called  ISM-2)  about  the 
fundamentals  of  media  design  for  education.  This  model  tries  to  combine  the  focus  on  task- 
performance  of  cognitive-apprenticeship  with  the  fading  of  support  of  scaffolding,  and  the  flexibility 
of  teleleaming.  To  explore  the  impact  of  this  model  it  is  applied  in  the  ISM-2  course.  A scaffolded 
group  (creators)  is  compared  with  a non-scaffolded  group  (reflectors).  On  a multiple-choice  test  for  a 
knowledge  and  a transfer  type  of  questions  no  significant  difference  was  found  between  the  two 
groups.  The  dropout  rate  for  the  scaffolded  group  was  lower  and  their  overall  time-investment  for  the 
course  was  higher.  It  is  concluded  that  the  extra  cost  of  using  this  approach  of  scaffolding  can  - at  this 
moment  - not  be  justified  by  the  measured  learning  results. 


Introduction 

A scaffold  is  a support,  such  as  the  temporary  framework  that  supports  a building  when  it  is  built.  When  the  building 
is  finished,  the  support  is  removed  and  the  building  will  stand  on  its  own.  In  instruction  scaffolding  serves  the  same 
role  as  the  scaffold  for  a builder  (Wood,  Bruner,  & Ross,  1976): 

1 . It  helps  to  ensure  the  learner’s  success. 

2.  It  extends  the  learner’s  competency  in  a new  territory. 

3.  It  can  be  taken  away  as  the  learner  becomes  more  responsible. 

Scaffolding  is  part  of  the  cognitive-apprenticeship  theory  (Brown,  Collins  & Duguid,  1989),  in  which  a student  is 
modeled  and  coached  to  perform  a specific  task.  Reflection,  and  articulation  of  reflection  takes  place,  after  which  a - 
more  free-  phase  may  follow  where  students  can  self-reliantly  explore,  set  their  own  subgoals  and  frame  problems. 
In  this  way  the  student  gradually  becomes  an  expert  in  the  performance  of  a specific  task. 

Nowadays,  cognitive  apprenticeship  is  being  used  in  a number  of  teleleaming  settings  (see  for  example:  Guzdial  & 
Kehoe,  1998;  McLouglin  & Oliver,  1998,  Suthers,  1998).  Part  of  the  appeal  of  using  cognitive  apprenticeship  and 
thereby  scaffolding  in  teleleaming  situations  is  its  promise  of  flexibility.  As  the  performance  of  the  student,  and  not 
that  of  a lecturer  is  key  (Pressley,  Hogan,  Wharton-McDonald,  Mistretta,  & Ettenberger,  1996),  a lecturer  does  not 
always  have  to  be  present  to  support  students.  Students  can  work  in  their  own  time  and  place,  and  receive  and  ask 
for  support  from  peers,  or  use  support  that  is  received  in  an  a-synchronous  fashion  from  an  instructor. 

Some  other  appealing  factors  of  scaffolding  are: 

- Support  at  the  level  of  the  student,  as  stressed  by  Vygotsky  (1978),  attention  is  paid  to  the  specific  difficulties  of  a 
student  when  solving  a problem. 

- It's  focus  on  the  self-reliance  of  the  student.  When  the  level  of  the  student  rises,  support  is  faded,  so  that  the  student 
learns  to  self-reliantly  perform  a task.  This  again  is  an  advantage  for  teleleaming  situations  where  the  continuous 
presence  of  the  lecturer  during  learning  is  not  possible. 

When  the  aforementioned  focuses  on  task-performance  of  cognitive  apprenticeship,  the  fading  of  support,  and  the 
flexibility  of  teleleaming  are  combined,  the  model  seen  in  Figure  1 can  be  constructed.  Performance  has  a main  role 
in  the  model,  fading  is  visualized  by  the  decreasing  part  of  the  task  that  is  performed  by  the  instructor,  and  the  in- 
creasing performance  of  the  student.  Using  prototypes  as  a means  of  monitoring  progress  of  students  provides  flexi- 
bility. In  this  model  the  prototypes  provide  proof  of  performance  via  which  communication  can  be  structured.  The 
initial  support  provided  is  support  that  is  already  available  in  a learning  environment.  They  can  can  take  the  form  of 
explanations  provided,  videoclips,  process  worksheets,  or  parts  of  the  solution  to  a problem. 
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Scaffolding  is  a time-consuming  and  demanding  form  of  instruction  (Pressley,  et.  al.,  1996).  In  order  to  scaffold 
students  on  their  own  level  of  development,  and  to  scaffold  students  based  on  the  progress  they  have  made,  much 
effort  needs  to  be  invested  in  monitoring  or  tracking  of  the  progress  of  students.  Therefore  a part  of  the  challenge  of 
implementing  this  approach  of  scaffolding  is  a cost-effective  application  in  teleleaming  environments. 

In  this  paper  an  experiment  is  described  whereby  the  model  for  'scaffolding  by  design'  is  implemented  in  a course 
(called  ISM-2)  about  the  fundamentals  of  media  design  for  education.  More  specifically  the  research  questions  for 
this  experiment  were: 

- What  impact  does  the  implementation  of  this  model  for  scaffolding  have  on  students  learning,  measured  by  stu- 
dents course  completion,  study  behaviors  during  the  course,  and  student  achievement  on  the  final  examination  of  the 
course? 

- How  much  extra  time  is  involved  for  students  and  instructors,  for  a scaffolded  variant  of  the  course  compared  to  a 
variant  of  the  course  without  WWW-supported  scaffolding?  If  there  is  a difference  in  time  investment,  is  this  extra 
cost  balanced  by  the  benefits  received? 


Context:  The  course  ISM-2,  fundamentals  of  media  design  for  education 

The  course  ISM-2  is  an  obligatory  course  for  second-year  students  in  the  Faculty  of  Educational  Science  and  Tech- 
nology. It  is  the  second  of  a number  of  courses  that  introduce  the  student  to  the  field  of  work  of  the  Department  of 
Educational  Instrumentation  (abbreviated  ISM).  The  Department  ISM  focuses  on  the  design,  development,  and  use 
of  technological  tools  and  resources  for  learning-related  purposes.  While  the  first  year  course  (ISM-1)  provides  a 
practical  introduction  to  the  work  of  the  department,  ISM-2  provides  a more-theoretical  perspective,  especially  fo- 
cusing on  human-computer  interaction  and  the  design  base  of  the  field,  ISM-2  is  worth  2 points  of  credit,  represent- 
ing 80  hours  of  study  by  the  students. 


Experimental  design 

To  address  the  research  questions,  two  experimental  groups  were  formed.  One  group  participated  in  a variety  of 
required  learning  activities  involving  group  collaboration  and  discussion,  using  a variety  of  WWW -based  resources, 
participating  in  decision  making  and  preparation  of  a summary  of  an  article  as  an  addition  to  the  course  study  mate- 
rials (therefore  called  creators),  and  interacting  with  the  instructors  in  personalized  and  substantial  scaffolding  ac- 
tivities. The  other  group  (called  reflectors)  had  access  to  the  same  resources  and  activities,  but  not  in  a required  way. 
Both  groups  attended  the  same  four  lectures;  were  responsible  for  reading  the  same  set  of  study  materials,  and  wrote 
the  same  final  examination  (worth  70%  of  the  course  mark). 

A statement  often  referred  to  is  that  'conditions  of  learning  determine  the  learning  effect'  (Aronson  & Briggs,  1983). 
In  this  case,  the  learning  effect  of  the  groups  will  be  different,  as  the  conditions,  the  WWW  based  learning  environ- 
ment and  the  interaction  with  instructors  and  peer-students  are  designed  differently.  As  the  creator  group  was  mak- 
ing links  outside  of  the  initial  course  materials,  by  searching  for  articles  for  their  summary,  they  were  expected  to 
better  be  able  to  use  their  knowledge  in  different  contexts.  So,  at  the  final  examination,  they  could  also  have  a better 
score  on  a transfer  type  of  questions,  asking  for  application  of  a known  concept  in  a new  domain.  The  Reflectors 
were  presented  with  the  content  of  the  course  and  did  not  have  to  find  or  contribute  new  materials  to  the  course.  So, 
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it  was  expected  that  they  could  spend  more  time  studying,  and  reflecting  on  the  materials  presented,  and  have  a bet- 
ter score  on  a retention  type  of  questions,  asking  for  definitions  and  direct  recall  of  information. 

Participants  were  29  second-year  university  students  participating  in  the  course  ISM-2.  From  the  students  that  were 
present  in  the  first  lecture  of  this  course  the  creator  and  reflector  groups  were  formed.  Because  there  were  six  topics 
available  (in  three  cycles)  for  contributing  a summary  to  the  course,  students  were  assigned  to  the  creator  group  first. 
The  remaining  students  were  assigned  to  the  reflector  group.  The  creator  group  consisted  of  1 8 students,  the  reflec- 
tor group  consisted  of  1 1 students.  The  creator  group  was  divided  into  6 subgroups  of  three  students  each. 

The  course  made  use  of  the  C@mpus+  course  support  environment  developed  at  the  Faculty  of  Educational  Science 
and  Technology  (Tielemans  & Collis,  1999).  The  two  groups  had  access  to  their  own  version  of  this  environment. 
They  had  however  the  same  communication  tools,  instructor  notes,  and  course  information  available,  with  only  the 
roster  (used  for  structuring  information  and  providing  forms  and  deadlines  relating  to  the  assignments  in  the  course) 
being  different  for  the  groups. 

To  measure  the  time  investment  of  both  students  and  instructors,  a time  investment  form  was  designed  to  be  filled  in 
with  an  estimate  of  the  number  of  minutes  spent  on  the  different  activities  of  the  course. 

The  final  examination  of  the  course  consisted  of  a multiple-choice  test,  with  a total  of  36  questions  covering  all  the 
objectives  of  the  course.  In  this  multiple  choice  test  18  knowledge  type  of  questions  were  included,  and  18  transfer 
type  of  questions.  As  a pretest,  the  results  from  the  first  year  course  ISM-1  were  used  in  order  to  compensate  for 
possible  differences  in  ability  between  the  two  groups.  The  courses  ISM-1  and  ISM-2  can  be  compared,  as  there  is 
continuity  between  them  in  content,  nature  of  the  examinations,  and  instructors. 


Procedure 

During  the  course  the  creators  have  actively  constructed  the  knowledge  that  was  used  as  study  materials  for  the  re- 
flector group,  thereby  scaffolded  by  the  instructors.  As  initial  introduction  to  the  topics  of  the  course,  three  introduc- 
tory lectures  were  given.  In  the  week  after  the  lectures  the  creators  were  asked  to  work  with  the  materials  presented 
in  the  lectures  (and  in  the  reader  of  the  course),  and  find  additional  sources  of  information.  This  could  be  informa- 
tion from  several  sources,  such  as  the  WWW,  staff  members,  CD-ROMs,  etc.  Each  student  had  to  find  one  source 
and  submit  some  information  about  it,  suitable  for  the  topic.  The  sources  were  discussed  in  the  groups,  and  one  of 
the  sources  was  selected.  Of  this  source  a summary  was  prepared  as  study  materials  for  all  students.  The  sources 
found,  the  choice  for  one  of  three  sources,  the  group's  motivation  for  this  choice,  and  a draft  and  final  version  of  the 
summary  were  all  included  in  the  WWW  environment. 

The  instructors  each  gave  careful  feedback  to  the  groups.  The  feedback  was  related  to  the  choice  of  articles  and  the 
quality  of  the  (draft)  summary.  The  instructors  gave  personal  comments  to  the  group  about  the  quality  of  the  chosen 
articles  and  the  suitability  for  their  topic.  The  instructor  could  add  personal  feelings  about  a source,  add  some  back- 
ground information  about  the  author  of  the  source,  suggest  a different  article,  or  add  some  information  about  the 
impact  the  chosen  article  had  on  the  field.  Typically,  each  of  the  feedback  comments  by  the  instructors  were  about 
one  or  two  pages  of  typed  text.  The  comments  were  uploaded  to  the  WWW  site  and  were  only  visible  by  the  stu- 
dents of  the  creator  groups. 

In  a face-to-face  session  following  the  posting  of  these  comments,  each  group  discussed  the  comments  with  an  in- 
structor, and  revised  their  summary.  When  the  final  version  of  the  summary  was  approved  by  one  of  the  instructors, 
it  was  posted  to  the  course  WWW  site  as  part  of  the  study  material.  Also,  the  summaries  were  now  available  to  the 
reflectors.  By  the  end  of  the  face-to-face  session,  the  summaries  should  be  posted,  and  the  reflectors  would  have  the 
whole  week  to  read  and  study  the  materials  posted.  All  of  the  creators'  work,  including  draft  submissions,  and  in- 
structor feedback,  can  be  viewed  at  the  course  WWW  site,  at 
http://education2.edte.utwente.niyi93032.nsf/guest7readform.  (Most  of  the  work  and  communication  is  in  Dutch.) 
Students  in  the  reflector  group  followed  the  same  lectures  as  the  students  in  the  creator  group.  They  studied  the  ma- 
terials provided  in  the  reader,  and  the  summaries  provided  by  the  creator  groups.  Further,  they  individually  prepared 
three  essays  around  three  topics  of  the  course.  They  received  feedback  on  these  essays  from  the  instructors,  but  did 
not  have  to  improve  the  essays,  nor  were  they  used  as  study  materials. 


Results 


All  students  in  the  creator  group  participating  fully  in  the  activities  designed  for  them.  All  groups  submitted  the 
sources,  draft  summary,  and  final  summary.  On  average  the  grade  for  the  summaries  was  8.25  on  a 10-point  scale. 
All  students  in  the  creator  group  participated  in,  and  passed  for  the  final  exam. 

Of  the  students  in  the  reflector  group  4 of  the  original  1 1 students  failed  to  submit  their  essays  and  4 out  of  1 1 were 
not  present  at  the  final  exam.  The  average  mark  for  the  essays  was  7.55  on  a 10-point  scale. 

Creators  were  more  often  present  at  the  lectures  than  students  in  the  reflector  group.  Of  the  creator  group,  4 out  of 
1 8 creators  attended  all  four  lectures,  8 out  of  1 8 attended  three,  and  6 out  of  1 8 attended  two  or  less.  Of  the  reflector 
group,  none  of  them  attended  all  lectures,  2 attended  three  lectures,  and  4 out  of  7 attended  two  or  less  lectures. 


Results  on  the  Final  Examination 


The  scores  for  the  final  examinations  of  the 
ISM-1  and  ISM-2  are  given  in  Figure  2.  The 
scores  for  the  creator  and  reflector  groups  are 
under  the  graph,  with  the  standard  deviation 
brackets. 

There  was  no  difference  found  between  the 
based  on  the  score  of  the  previous  course  ISM- 
0.352,  p=0.730).  No  significant  difference  was 
between  the  groups  on  the  score  of  ISM-2 
overall  grade  for  the  final  examination  is  con- 
(t(22)=1.270,  p=0.222).  When  the  knowledge 
and  the  transfer  items  on  the  final  examination 
considered  separately,  there  remains  no 
difference  between  the  groups  (t(22)0.103, 
for  knowledge  questions,  and  (t(22)2.083, 
for  transfer  questions). 


Time-investment  by  Students  and 
Instructors 
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Figure  2.  Student  performance  on  final  examina- 
tion, average  and  standard  deviations. 


Table  1 gives  on  overview  of  the  time-investment  of  students  for  the  course,  split  into  creator  and  reflector  groups. 
When  an  asterisk  (*)  is  printed  there  was  a significant  difference  between  the  groups. 


Activity 

Minutes,  Creators  (means 
and  standard 
deviations.ro  unded) 

Minutes,  Reflectors  (means 
and  standard  deviations, 
rounded) 

Searching  on  the  WWW 

79  (93) 

56  (35) 

Searching  in  books  and  journals* 

337  (1  86) 

7(13) 

Inserting  materials  into  the  course  WWW 
site 

144  (1 35) 

N/A 

Learning  to  work  with  the  course  WWW  site 

25  (23) 

1 9 (28) 

Dealing  with  computer  problems 

1 0 (1  5) 

69  (1  56) 

Creation  of  essays 

N/A 

592  (425) 

Use  of  a shared  workspace 

No  use 

No  use 

Use  of  a course  mailing  list 

1 .53  (4.9) 

0.43(1.13) 

Personal  discussions  with  the  instructors 
(face  to  face  or  via  the  WWW)* 

3 1(21) 

9(15) 

Personal  discussions  with  fellow  students 

95  (82) 

44  (56) 

Studying  the  instructor-provided  materials* 

1,611  (953) 

872  (563) 

Attending  the  lectures* 

265  (67) 

167  (81) 

Other  activities 

1 02  (288) 

0.7(1.89) 

Writing  the  final  examination 

85  (22) 

91  (3.78) 

Total  time  spent  on  course 

2,802  (1  077) 

1 .932  (933) 

Table  1.  Participation  and  time  of  participation  in  course-related  activities  (*  Significant,  p<.05). 
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The  time  invested  by  the  instructors  of  the  course  can  be  found  in  Table  2.  Times  are  gives  in  minutes. 


Task 

Primary 

instructor 

Instructor  2 

Instructor  3 

Total  (minutes) 

preparation  of  lectures 

3,770 

840 

960 

5,570 

Preparation  of  assignments  for  students 

190 

300 

150 

640 

Giving  feedback  to  assignments 

1.230 

1,500 

680 

3,410 

Preparing  and  entering  materials  on 
WWW  site 

10 

495 

875 

1,380 

Personal  discussions  with  students 

360 

270 

435 

1,065 

Attendance  at  lectures 

450 

550 

500 

950 

Maintain  WWW  site;  ongoing 
administration  of  student  data 

30 

0 

705 

735 

Other 

510 

30 

900 

1,125 

Total  per  instructor 

6,580  minutes 

3,655  minutes 

5,1 80  minutes 

15,41 5 minutes 

(110  hours) 

(61  hours) 

(86  hours) 

(257  hours) 

Table  2.  Tasks  and  time  on  task  of  instructors  (in  minutes) 


Discussion 

No  significant  differences  were  found  when  comparing  the  performance  of  the  creator  and  reflector  groups  on  the 
final  examination  of  the  course.  The  creator  group  however  spent  considerably  more  time  on  the  course  than  did  the 
reflector  group  (2,802  minutes  compared  to  1,932  minutes).  So,  the  extra  time  spent  by  the  creators  in  ISM-2  did  not 
result  in  overall  better  performance  on  the  final  examination.  However,  the  nearly  significant  difference  on  the  trans- 
fer type  of  questions  (p=0.054)  suggests  that  there  may  be  a significant  difference  when  more -sensitive  test  ques- 
tions are  used.  Further,  the  payoff  maybe  more  related  to  higher  level  insights  than  to  knowledge  level  retention. 
There  are  some  other  indicators  of  differences  between  the  groups.  There  were  fewer  dropouts  in  the  creator  group 
of  the  course,  and  as  a result  of  this  more  students  of  the  creator  group  finally  passed  the  course. 

With  regard  to  the  time-investment  of  students  there  were  some  interesting  differences  (Table  1).  The  time  spent 
searching  in  journals  and  books  was  337  minutes  for  the  creators,  while  only  7 minutes  for  the  reflectors.  With  the 
time  spent  searching  for  sources  on  the  WWW  there  was  less  difference:  79  minutes  for  the  creators,  and  56  minutes 
for  the  reflectors.  These  results  suggest  that  the  WWW  seems  to  invite  students  to  search  for  additional  materials  to 
the  course,  much  more  than  the  library  seems  to  invite  our  students  to  read  additional  materials. 

Further,  the  creators  spent  nearly  twice  as  much  time  studying  the  provided  materials  than  did  the  reflectors  (1,610 
minutes  compared  to  872  minutes).  It  seems  that  the  scaffolded  approach  used  with  the  creators  stimulated  them  not 
only  to  spend  more  time  on  the  activities  required  for  them,  but  also  to  spend  more  time  on  activities  that  were  ex- 
pected to  be  more  important  for  the  reflector  group.  This  point  is  further  illustrated  by  the  total  time  investment  of 
both  groups,  which  was  higher  for  the  creators  than  for  the  reflectors  (2,802  compared  to  1,932  minutes).  Time- 
investment  overall  was  lower  than  the  expected  4,800  minutes  (80  hours). 

The  instructor  time  spent  was  considerably  higher  than  the  time  spent  by  the  students  of  the  course.  A part  of  this 
invested  time  may  be  reduced  next  year,  such  as  the  time  for  redesign  of  the  course  and  the  course  WWW  site,  time 
for  discussing  the  new  approach  followed,  time  for  designing  the  final  examination  questions,  and  time  for  redesign 
of  the  lectures.  Another  part  of  this  invested  time  will  remain,  for  example  for  giving  elaborate  feedback  to  students 
(total:  3,410  minutes).  This  time  could  however  be  reduced  by  using  other  forms  of  giving  feedback,  such  as  peer- 
feedback,  and  using  other  media,  such  as  audio  for  giving  feedback. 


Conclusions 

So,  what  has  been  the  impact  of  applying  this  model  for  scaffolding  on  this  course?  The  first  research  question 
(about  the  impact  of  implementing  this  model  for  scaffolding)  is  answered  as  follows:  the  impact  of  the  model  has 
not  been  expressed  in  a significant  difference  on  the  final  examination  favoring  the  creator,  or  reflector  group.  There 
was  an  impact  however  on  the  attendance  at  lectures,  overall  time  spent  for  the  course,  and  completion  rate  of  the 
course. 
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The  second  research  question  (about  the  balance  of  extra  cost  with  benefits)  can  now  also  be  answered.  Due  to  the 
lack  of  measureable  differences  found  in  the  results,  the  extra  cost,  in  terms  of  time-investment  of  both  students  and 
instructors,  can  at  this  moment  not  be  justified. 

Research  on  scaffolding  does  not  stop  here.  First,  the  measurement  of  the  results  could  be  greatly  improved.  While 
the  results  point  in  the  direction  of  a gain  in  transfer-type  of  knowledge.  These  skills  may  be  more-sensitively  meas- 
ured with  a different  instrument  than  multiple-choice  questions. 

A second  point  is  the  improvement  of  efficiency.  Not  all  aspects  in  a course  need  to  be  scaffolded.  When  an  instruc- 
tor knows  what  are  the  most  important  or  hardest  parts  for  the  students,  the  scaffolding  can  be  focussed  on  this  part 
of  the  course.  Further,  scaffolding  does  not  necessarily  need  to  be  done  by  writing  elaborate  feedback,  but  could  also 
be  done  in  the  form  of  impersonal  or  general  feedback  remarks,  as  suggested  in  studies  of  Visser  (1998)  and  Gordijn 
(1998). 

The  last  point  is  related  to  the  preference  of  the  instructor.  Instructors  may  believe  that  the  changes  occurring  in  the 
division  of  roles  between  student  and  teacher,  with  the  student  taking  over  the  learning  process  as  much  as  possible, 
collaborating  with  an  instructor,  and  learning  to  regulate  their  own  learning,  may  be  important  developments  for  the 
future.  If  instructors  believe  this,  it  is  worth  exploring  scaffolding,  and  further  improving  the  efficiency  of  scaffold- 
ing. 
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Abstract:  The  development  of  a Statistics  221  course  at  Brigham  Young  University  required  that  a 
large  number  of  multimedia  objects  (>800)  be  created  in  a relatively  short  period  of  time  (<8 
months)  by  over  20  employees,  most  of  whom  were  students  with  varying  schedules  and  abilities. 
As  a means  of  providing  a standard  communication  device  in  the  development  of  the  course,  a 
storyboard  was  created  in  Microsoft  PowerPoint.  The  effectiveness  of  the  storyboard  design  was 
then  evaluated  by  surveying  all  project  participants.  The  results  of  the  evaluation  were  used  in 
redesigning  the  storyboard  for  use  in  the  development  of  additional  online  courses. 


Need  Statement 

The  development  of  a Statistics  221  course  at  Brigham  Young  University  required  that  a large  number  of 
multimedia  objects  (>800)  be  created  in  a relatively  short  period  of  time  (<8  months)  by  over  20  employees,  most  of 
whom  were  students  with  varying  schedules  and  abilities.  These  objects  then  needed  to  be  inserted  into  web  pages 
with  other  instructional  elements  (assignments  and  activities)  to  form  the  basis  of  the  course.  For  the  most  part, 
these  media  objects  are  discrete  multimedia  objects  embedded  in  instructional  modules  programmed  in  Macromedia 
Flash. 

In  a previous  project,  written  and  verbal  descriptions  of  multimedia  objects  had  been  used  to  manage  their 
production.  However,  our  experience  with  that  project  provided  insight  about  using  written  and  verbal  descriptions 
of  media  objects.  The  following  weaknesses  were  observed: 

1 . Text  descriptions  of  media  objects  are  very  time  consuming  to  write. 

2.  Text  descriptions  never  adequately  convey  what  the  designers  want  created. 

3.  Verbal  descriptions  required  arranging  expensive  meeting  times  between  designers,  artists,  and 
programmers. 

4.  Because  verbal  descriptions  are  not  stored,  they  can  be  easily  forgotten,  misinterpreted  or  changed. 

5.  Verbal  descriptions  are  also  difficult  to  track. 

the  nature  of  these  objects  was  part  of  the  problem.  Many  of  the  objects,  in  addition  to  being  very  visual,  were  also 
designed  to  take  advantage  of  Flash’s  highly  interactive  capability,  making  them  difficult  to  describe. 

Search  for  Existing  Solutions 

Various  solutions  to  managing  the  output  and  quality  of  these  kinds  of  multimedia  objects  were  explored,  but  neither 
instructional  design  software  (like  Designer’s  Edge  from  Allen  Communications)  nor  movie  storyboarding  software 
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(such  as  Storyboard  Artists  from  PowerProduction  Software)  filled  the  need.  Problems  encountered  included  the 
inability  to  print  text  and  visual  instructions  on  the  same  page  and  the  inability  to  recognize  a broad  range  of 
standard  files  types.  A standard  tool  that  content  experts  could  learn  quickly,  and  that  adapted  to  our  specific  need 
was  needed  and  existing  solutions  did  not  meet  our  needs. 


Our  Solution 

The  development  of  multimedia  objects  in  Flash  was  actually  the  second  phase  of  the  course  development  project. 
An  earlier  version  of  the  course  had  been  created  using  PowerPoint.  The  purpose  of  migrating  from  PowerPoint  to 
Flash  was  to  take  advantage  of  Flash’s  more  robust  animation  and  interactive  capabilities  and  to  make  the  discrete 
objects  more  suitable  for  the  web.  Yet,  it  was  determined  that  the  older,  PowerPoint  versions  of  the  lessons  could 
serve  as  the  basis  for  the  storyboards. 

Because  the  objects  were  going  to  be  animated  in  Flash,  it  was  decided  that  the  content  experts  would  not  spend 
additional  development  time  animating  multimedia  objects  in  PowerPoint.  Rather,  the  slide  layout  feature  of 
PowerPoint  would  be  used  to  design  the  visual  content  of  the  course  (see  Fig.  1 Feature  2).  Then,  the  “Notes” 
feature  of  PowerPoint  was  used  to  provide  additional  art  and  animation  instructions  to  the  artists  and  programmers 
developing  the  content  (See  Fig.  1,  Feature  6). 
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Fig.  1 Page  from  a Statistics  Storyboard 


This  solution  lent  itself  particularly  well  to  the  statistics  course  because  so  much  of  the  preliminary  work  had 
already  been  done  in  PowerPoint  and  because  there  are  many  visual  elements  of  the  course  that  required  statistical 
precision.  These  elements  had  to  be  specified  by  a content  expert.  The  artist  could  then  modify  the  statistical 
information  to  improve  the  aesthetic  presentation  to  the  degree  that  the  statistical  content  remained  accurate. 

A design  process  was  developed  in  which  the  content  experts  and  instructional  designers  used  PowerPoint  as  the 
medium  through  which  the  course  content  was  developed.  Statistical  software  such  as  Minitab  was  used  to  produce 
charts  and  graphs  (see  Fig.  1,  Feature  4),  which  were  then  imported  into  PowerPoint.  The  content  experts  added 
textual  instructions  to  the  artists  and  programmers  (see  Fig.  1,  Feature  6). 

Once  the  storyboards  were  approved  by  the  content  experts,  the  Notes  Template  feature  in  PowerPoint  was  used  to 
create  a header  on  every  page  where  the  art  director  could  indicate  which  art  template  would  be  used  and  which 
artist  would  be  assigned  to  the  project  (see  Fig.  1,  Feature  1).  Space  was  also  provided  at  the  bottom  of  the  notes 
section  for  the  art  director  to  draw  the  layout  that  should  be  used  and  give  other  instructions  to  the  artists  (see  Fig.  1, 
Feature  7).  The  ability  to  embed  objects  and  add  textual  instructions  in  PowerPoint  was  useful.  But  allowing  the  art 
director  to  add  freehand  sketches  allowed  us  to  compensate  for  the  limitations  of  the  tool. 
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BEST  COPY  AVAILABLE 


After  the  art  director  finished  making  comments  on  the  storyboard,  copies  of  the  storyboard  were  made  for  each 
department  participating  in  the  project.  These  copies  were  bound  and  distributed  to  the  Art  Department, 
Instructional  Designer,  Statistics  Department,  Programming/Authoring  Department,  and  Video  Department. 
Additionally,  the  final  electronic  copy  of  the  storyboard  was  distributed  to  the  Art  Department  so  that  the  Minitab 
graphs  could  be  imported  into  Flash.  The  Art  Department  would  create  specific  media  objects  while  the 
programmers  created  the  Flash  file  in  which  the  text  of  the  presentations  and  the  media  objects  were  embedded  (see 
Fig.  I,  Features  3 and  5).  Once  a produced,  the  instructional  designer  and  content  team  reviewed  the  objects  and 
requested  revisions  if  necessary  (compare  Fig.  I and  Fig.  2). 


Fig.  2 Screenshot  of  a statistics  lesson.  End-product  of  the  media  objects  requested  in  the  storyboard  above. 
The  hardcopy  of  the  storyboard  then  served  as  the  design  document  that  all  members  of  the  party  could  reference. 


Results 

After  using  the  storyboards  for  the  statistics  project,  we  conducted  an  evaluation  of  the  storyboarding  process  as  a 
means  of  determining  whether  or  not  it  would  be  useful  in  additional  course  development  projects  and  how  it  could 
be  improved. 

All  the  storyboard  users  who  participated  in  the  project  were  identified.  This  included  the  instructional  designer,  6 
members  of  the  content  team  (I  faculty  member  content  expert  and  5 student  content  experts),  19  members  of  the  art 
team  (I  full-time  senior  production  designer,  16  student  artists,  and  2 photographers),  and  17  programmers  (1  full- 
time lead  programmer  and  16  students). 

Each  type  of  storyboard  user  was  asked  how  the  storyboard  was  used,  what  elements  of  the  storyboard  were  useful, 
and  what  elements  of  the  storyboard  were  not  useful.  All  storyboard  users  were  contacted  via  email  with  the 
exception  of  the  lead  faculty  content  expert  and  the  instructional  designer.  These  two  participants  were  interviewed 
in  more  detail. 

The  summary  of  the  results  follows  in  the  detail  given  below. 

Content  Team 
How  it  was  used 

The  content  experts  used  the  storyboards  as  a basis  of  communication  in  the  design  process  and  during  production. 
They  compared  the  final  product  with  what  was  requested  in  the  storyboard  to  determine  whether  or  not  the  media 
objects  met  the  original  specifications. 
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What  worked 

1.  The  storyboards  were  useful  as  a tool  to  compare  the  final  product  with  the  original  production 
requests, 

2.  The  storyboards  provided  a tool  to  help  the  content  experts  organize  the  logical  flow  of  the  lesson.  As 
one  student  content  expert  reported,  “The  storyboards  helped  me  organize  the  lesson  in  a logical  flow, 
without  being  bogged  down  in  the  details  of  artistic  content.” 

3.  The  storyboards  a single  point  of  reference  for  all  participants  of  the  project  to  communicate  with. 

What  didn’t  work 

1.  The  content  experts  felt  that  the  animated  nature  of  many  of  the  objects  was  not  adequately  described  on 
the  static  storyboards.  He  felt  that  when  objects  were  designed,  he  could  not  visualize  what  they  would 
look  like  when  animated.  This  led  him  to  desire  revisions  to  the  media  object  design  after  it  was  produced 
because  he  felt  if  reworked,  they  would  work  better  in  the  classroom. 

2,  The  content  experts  felt  that  artists  took  artistic  license  in  revising  objects  that  needed  to  be  copied 
precisely.  As  a result,  the  statistical  accuracy  of  some  objects  was  altered,  requiring  additional  revisions. 

Instructional  Designer 

How  it  was  used 

The  instructional  designer  used  the  storyboard  as  a means  of  communication  while  designing  the  content  of  the 
course  with  the  content  expert  and  his  assistants.  She  then  used  it  as  tool  to  communicate  production  specifications 
to  the  artists  and  programmers.  It  was  also  used  as  the  standard  against  which  produced  objects  were  checked  for 
accuracy. 

What  worked 

1 , Useful  as  a single  point  of  reference  for  all  project  participants. 

2.  Because  of  the  large  number  of  objects,  it  was  helpful  way  of  documenting  design  specifications. 


What  didn’t  work 

1,  The  storyboard  was  intended  to  be  used  as  a design  document.  It  became  obsolete  when  design 
changes  were  requested  after  objects  were  in  production. 

2.  Many  of  the  statistically  accurate  objects  were  embedded  in  the  electronic  version  of  the  PowerPoint 
file  that  was  distributed  to  the  artists.  When  artists  created  objects  from  scratch  instead  of  using  the 
statistically  accurate  charts  and  graphs  provided  them,  it  became  problematic. 

Artists 

How  it  was  used 

The  artists  used  the  storyboard  as  the  design  document.  Essentially,  it  was  their  sole  set  of  instructions  for 

producing  an  object. 

What  worked 

1 . Useful  as  a single  point  of  reference  for  all  project  participants. 

2.  The  storyboards  helped  the  artists  understand  what  the  content  team  wanted  designed.  For  example, 
when  student  artist  said,  “They  provided  detailed  directions  of  how  to  present  the  concept,  making  it 
possible  for  animators  who  have  not  taken  the  course  yet  to  still  know  how  to  teach  the  concept.” 

What  didn’t  work 

1.  Visual  decisions  were  made  by  content  experts  rather  than  by  artists.  The  artists  viewed  this  as  a 
problem  for  two  reasons.  First,  the  artists  felt  that  many  of  the  layout  decisions  were  artistically 
inferior  to  what  the  artists  could  suggest.  The  result  was  decreased  job  satisfaction  because  artists 
were  not  allowed  to  be  as  creative  as  they  would  like. 
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2.  Storyboard  authors  assumed  their  audience  had  a greater  understanding  of  statistical  audience  than 
they  really  did.  Problems  incurred  included  confusing  directions  (because  artists  did  not  understand 
the  terms);  difficulties  with  accuracies  of  graphs  because  artists  would  make  artistic  changes  to  graphs, 
not  realizing  the  changes  made  them  statistically  inaccurate;  references  to  non-ascii  characters  were 
unclear  (authors  would  refer  to  a spelled  out  version  of  a non-ascii  character  (i.e.  “alpha”  for  oc);  the 
text  references  were  spelled  out  and  the  visual  storyboard  used  symbols;  artists  lacked  the  necessary 
knowledge  to  correctly  identify  them  as  the  same  character,  making  the  instructions  confusing. 

3.  Content  experts  requested  additional  design  changes  after  storyboards  were  in  production. 

4.  Storyboards  were  not  thought-through  enough.  The  artists  felt  that  requests  for  design  changes  and 
misspellings  in  the  storyboards  were  evidence  of  the  content  experts  not  thinking  through  their 
production  requests  thoroughly  enough  in  the  pre-production  process. 

By  using  PowerPoint  to  storyboard  a Flash  object,  the  capabilities  of  Flash  were  not  maximized.  The  artists  felt  that 
more  innovative  animations  could  have  been  created  in  Flash.  By  using  PowerPoint  to  design  the  storyboards,  it 
limited  the  content  experts  to  the  technologies  they  were  designing  with. 

Programmers 

How  it  was  used 

The  programmers  used  the  storyboards  for  several  purposes.  First,  the  referenced  the  storyboard  layouts  to  create 
the  Flash  file  which  contained  the  lecture  and  all  embedded  media  objects.  Second,  once  the  media  were  produced, 
they  would  reference  it  to  see  where  all  the  media  objects  needed  to  be  inserted  and  how  they  should  work.  Finally, 
they  referenced  the  storyboard  for  instructions  on  programming  java  applets. 

What  worked 

1 . Detailed  instructions  were  useful. 

2.  A single  point  of  reference  for  all  participants  in  the  project.  For  example,  one  programmer  reported 
that,  “The  storyboards  gave  a common  ground  for  everyone  working  on  the  stats  lessons  to  work 
from.” 

What  didn’t  work 

1 . Revisions  made  in  the  art  process  were  not  communicated  effectively  to  programming.  As  a result,  by 
the  time  the  programmers  were  ready  to  insert  media  objects,  their  storyboard  were  often  out-of-date. 

2.  Degree  of  detail  and  instructions  led  to  decreased  job  satisfaction.  Like  the  artists,  the  programmers 
felt  that  they  were  not  able  to  fully  utilize  their  programming  skills. 

3.  The  amount  of  revisions  occurring  in  production  was  problematic.  As  one  programmer  stated,  “We 
would  make  changes  and  then  take  them  to  the  professor  to  approve  and  then  we  would  make  more 
changes  and  this  would  be  repeated  many  times.” 

4.  Storyboarded  objects  sometimes  didn’t  take  into  account  programming  issues  in  Flash.  Essentially,  the 
design  instructions  presented  to  the  artists  and  programmers  did  not  reflect  a consideration  of  how  to 
best  develop  the  objects  given  the  selected  technologies. 

Conclusions  and  Recommendations 

All  project  participants  responded  positively  to  having  a single  point  of  reference  for  all  the  objects  being  designed 
in  the  course.  Most  of  the  frustration  identified  by  all  participants  of  the  project  stemmed  from  the  amount  of 
revisions  implemented  after  objects  were  produced.  This  frustration  can  be  significantly  reduced  by  helping  content 
experts  in  the  design  process  think  understand  that  once  the  design  stage  ends  and  production  begins,  revisions  can 
only  be  made  when  produced  objects  do  not  meet  specifications  rather  than  when  a new  idea  for  improving  the 
object  occurs.  Such  adherence  to  feature  lock  be  understood  by  all  parties  in  the  project  as  well  as  the  decision- 
making level  of  the  organization. 

Assuming  that  the  revision  process  is  decreased,  the  storyboard  seemed  to  be  quite  useful.  The  storyboard  is  being 
used  in  the  creation  of  an  introductory  biology  course.  In  this  case,  a word  processing  tool  is  being  used  in  place  of 


1134 


PowerPoint.  The  decision  to  not  use  PowerPoint  was  made  for  several  reasons.  First,  the  biology  project  is  a self- 
contained,  fully-online  course  rather  than  an  in-class  presentation.  As  a result,  PowerPoint  does  not  lend  itself  as 
well  to  the  format  of  the  course.  Second,  PowerPoint  is  a very  time-consuming  tool  to  storyboard  in.  It  was 
selected  for  statistics  since  most  of  the  content  was  in  PowerPoint  to  begin  with.  But,  in  the  case  of  the  biology 
course,  it  is  more  time-efficient  to  use  a word  processor. 

The  biology  course  storyboard  is  divided  into  two  sections,  in  attempt  to  address  the  needs  of  programmer  and 
artists  more  specifically.  The  programming  section  includes  a hardcopy  of  the  entire  script  of  a given  biology  lesson 
with  instructions  as  to  where  to  insert  specific  media  objects.  The  art  section  details  the  instructional  objective  and 
production  instructions  for  each  object.  Once  again,  space  is  provided  for  the  art  director  to  add  addition  comments, 
sketch  layouts,  etc.  In  addition  to  the  programming  and  art  sections,  the  front  of  the  storyboard  includes  the 
specification  for  the  navigation  of  a particular  lesson  as  well  as  a Table  of  Contents  listing  each  media  object,  the 
outline  of  the  lesson,  and  the  file  names  for  each  object  in  the  lesson.  As  with  the  statistics  storyboard,  the  biology 
storyboards  are  bound  and  distributed  to  all  project  participants. 

The  intent  of  the  biology  storyboard  is  to  improve  upon  the  features  that  worked  in  the  statistics  storyboard  such  as 
providing  a single  set  of  instructions  to  al  participants.  At  this  early  stage,  participants  have  indicated  that  the  new- 
and-improved  storyboard  addresses  most  of  the  problems  of  the  first  type  of  storyboard  and  is  much  for  useful. 
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Abstract:  The  purpose  of  this  case  study  is  to  investigate  language  teachers’  experiences 
in  developing  multimedia  instructional  projects.  This  paper  reports  two  language 
teachers’  project  development  experiences,  difficulties,  successes,  and  concerns,  and 
reflections.  Implications  and  suggestions  for  technology  training  programs  for  language 
teachers  are  discussed. 


INTRODUCTION 

Media  and  technologies  have  had  an  inevitable  impact  on  education.  Multimedia  software  programs  have 
been  made  available  to  teachers  and  students  by  commercial  companies.  Multimedia  hardware  and  software 
are  tools  that  offer  possibilities  for  improving  the  learning  process,  but  the  teacher  will  have  to  integrate  them 
into  the  curriculum  in  a way  to  facilitate  teaching  and  learning  (Heinich,  Molenda,  Russell,  & Smaldino, 
1999).  In  such  integration  process  in  language  education,  finding  useful  software  applications  can  be  a 
difficult  task  due  to  the  following  factors:  (1)  A large  number  of  commercial  applications  are  designed  for 
personal  use,  not  for  classroom  use.  (2)  Some  applications  might  be  useful,  but  too  expensive  to  buy  enough 
licenses  for  all  the  computers  in  a language  learning  lab  (Aoki,  1996).  Therefore,  language  teachers  often 
need  to  create  their  own  multimedia  instructional  materials  customized  for  their  instructional  objectives  and 
their  students'  learning  goals,  in  their  specific  teaching  and  learning  contexts,  and  at  their  students'  language 
proficiency  levels. 

The  purpose  of  this  case  study  is  to  investigate  language  teachers’  experiences  in  developing  multimedia 
instructional  projects.  The  results  of  the  study  are  expected  to  help  teacher  trainers  discover  and  gain  insights 
about  language  teachers’  experiences  (such  as  successes,  difficulties,  and  concerns)  in  creating  multimedia 
projects.  The  understanding  of  such  experiences  is  intended  to  (1)  enhance  teacher  trainers’  facilitation  in 
preparing  language  teachers  (both  pre-service  and  in-service)  to  develop  multimedia  instructional  projects, 
(2)  helps  language  teachers  be  informed  of  the  possible  difficulties  they  might  encounter  before  attempting  to 
start  the  multimedia  development  process,  and  (3)  allow  language  teachers  to  reflect  upon  their  own 
experiences  in  developing  multimedia  instructional  materials  and  thus  improve  the  development  process  in  a 
more  effective  and  efficient  fashion. 


BACKGROUND 

More  and  more  teacher  education  and  professional  development  programs  offer  educational  technology 
courses  such  as  development  of  multimedia  instructional  programs  to  both  pre-service  and  in-service 
teachers.  Especially  in  the  fields  of  science  and  math  education,  many  teacher  educators  have  been 
incorporating  multimedia  development  training  into  their  curriculum.  In  the  meantime,  this  trend  is  being  or 
will  be  soon  adopted  by  language  education  programs.  As  Feyten,  Taylor,  and  Nutta  (1996)  noted,  "It  is 
imperative,  then,  that  our  teacher  preparation  programs  integrate  instructional  technology  into  their 
curriculum,  to  begin  to  correct  this  deficiency,  and  to  produce  future  foreign  language  teachers  who  are 
comfortable  with  and  adept  at  the  implication  of  the  media  into  their  classrooms." 
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In  the  process  of  developing  multimedia  instructional  projects,  teachers’  experiences  might  vary  and  depend 
upon  a number  of  possible  (or  assumed)  factors  such  as  computer  anxiety,  previous  computer  experience, 
training,  subject  area,  and  creativity.  However,  these  factors  may  or  may  not  be  applicable  to  every  teacher, 
especially  a language  teacher.  Thus,  an  understanding  of  language  teachers’  experiences  in  creating 
multimeia  instructional  projects  may  help  teacher  educators  improve  their  educational  technology  programs 
based  on  various  learner’s  needs  and  their  individual  differences.  However,  only  have  a few  studies  (Mitchell 
& Hunt,  1997;  Anderson  et  al.,  1993;  Tucker  et  al.,  1990)  been  conducted  to  explore  the  experiences  of 
teachers  in  developing  multimedia  instructional  materials.  Research  studies  of  such  focusing  on  language 
teachers  in  particular  are  even  rare. 


LITERATURE  REVIEW 

In  the  research  literature,  studies  on  language  teachers’  experiences  in  developing  multimedia  instructional 
programs  are  rare.  However,  there  are  a few  studies  on  multimedia  development  experiences  of  teachers 
of  other  subject  matters. 

Mitchell  and  Hunt  (1997)  state  that  higher  education  must  train  pre-service  physical  education  (PE) 
teachers  based  on  the  requirements  of  more  technologically  literate  students.  In  their  study  conducted  at 
the  University  of  Central  Florida,  an  innovative  way  to  develop  lesson  plans  has  been  discovered  by  using 
multimedia  authoring  software,  such  as  HyperStudio,  that  lets  students  integrate  text,  graphics,  video, 
audio  and  digitized  images  into  their  lesson  plans.  They  reported  that  a multimedia  lesson  plan  can 
become  a three-dimensional  lesson  plan  that  is  advantageous  in  PE  as  it  can  show  physical  movements. 

Hatfield  (1996)  describes  a project  that  combines  technology,  teacher  education,  and  the  use  of 
manipulatives  for  mathematics  instruction  to  enhance  the  professional  development  of  prospective 
teachers.  The  multimedia  developed  in  the  project  are  means  of  bringing  the  classroom  environment  to 
the  university  to  facilitate  discussions  about  prospective  teachers’  beliefs  about  mathematics  teaching. 

Another  study  (Anderson  et  al.,  1993)  was  conducted  in  Scotland  to  determine  whether  untrained 
classroom  teachers  could  learn  HyperCard  fi*om  a minimal  manual  and  then  create  their  own  stacks.  The 
development  of  the  minimal  manual  is  discussed,  use  of  the  manual  and  software  is  analyzed,  and 
teachers'  attitudes  are  examined. 

In  a 3-year  project  started  in  1989  to  train  faculty  in  the  College  of  Education  to  incorporate  hypermedia 
into  their  curriculum  at  the  University  of  South  Alabama,  Tucker  et  al.  (1990)  reported  that  evaluation  of 
the  program  concentrated  on  program  context,  process,  and  product.  Data  sources  included  review  of 
project  proposals,  analysis  of  internal  evaluation  data,  visitation  to  field  applications,  interviews,  and 
photo-interviews.  Photography  was  chosen  as  an  evaluation  method  because  of  its  holistic  quality,  the 
volume  of  detail  it  provides,  and  the  role  it  plays  as  a source  of  discussion.  Results  of  the  program 
evaluation  reveal  that:  (1)  most  early  adopters  had  prior  experiences  with  technology;  (2)  HyperCard  has 
been  used  by  program  participants  in  presentations,  interactive  video  and  animation  projects,  and  reading 
analysis;  (3)  the  biggest  need  cited  for  skills  training  was  "time  to  practice";  (4)  photo-interviews  yielded 
rich  data  and  provided  a means  to  describe  and  explain  the  program  and  its  consequences;  and  (5)  photo- 
interviewing can  serve  as  an  instructional  as  well  as  an  evaluative  strategy  given  the  projective  nature  of 
visual  images. 

According  to  another  line  of  foreground  research  studies  which  concentrate  on  attitudinal  effects,  teachers' 
attitudes  toward  computers  have  significant  impact  on  their  decision  to  use  specific  instructional  media 
including  computers  in  the  classroom  and.the  curriculum  (Dupagne  & Krendl,  1992;  Kluever,  1994;  Pina  & 


Harris,  1993;  Savenye,  1992).  For  instance,  in  a review  of  literature  on  teachers*  attitudes  toward  computers, 
Dupagne  and  Krendl  (1992)  described  that  ’’early  studies  reported  some  resistance  to  computer 
implementation”  (pp.  420-421),  and  this  statement  correlated  with  teachers’  ambivalent  attitudes  toward 
computer  technology  revealed  in  the  studies  conducted  in  the  1970s  and  1980s  (Dupagne  & Krendl,  1992). 
When  teachers  are  found  holding  negative  attitudes  toward  computer  technology,  their  concerns  about 
computer  use  in  the  school  environment  must  be  considered  and  pinpointed  since  there  are  a number  of 
factors  contributing  to  such  attitudes.  Some  of  the  factors  identified  include  computer  anxiety,  efficiency, 
liking,  and  instructional  usefulness  (Kluever,  1994).  Furthermore,  knowledge  of  and  training  on  applications 
of  computer  in  education  and  the  actual  limitations  of  the  computer  hardware  and/or  software  may  also  be 
concerns  of  teachers  (Dupagne  & Krendl,  1992). 

Due  to  the  lack  of  research  literature  similar  to  the  study,  the  literature  review  provided  in  this  section  might 
not  be  directly  associated  with  the  phenomenon  to  be  discovered  and  addressed  in  the  study.  First,  the 
research  studies  discussed  above  dealt  with  contexts  different  from  those  in  the  study,  for  example,  the 
content  areas  of  teaching  in  the  literature  review  differ  from  that  in  the  study.  Second,  the  participants  in  the 
studies  outlined  above  might  also  differ  from  those  in  the  study  who  are  teacher  education  students  ranging 
from  undergraduate  to  doctoral  levels  and  who  have  or  have  not  served  as  language  teachers.  Third,  the  most 
important  aspect  of  the  phenomenon  considered  as  background,  which  is  the  whole  phenomenon  understood 
as  a complex  system  that  is  more  than  the  sum  of  its  parts,  is  the  whole  experience  of  language  teachers  in 
developing  multimedia  instructional  programs  from  a holistic  perspective;  nevertheless,  no  research  evidence 
exists  to  help  understand  this  aspect  of  the  phenomenon. 


METHOD 

A semi-structured  questionnaire  was  utilized  in  the  study.  With  the  voluntary  permission  of  the  participants, 
the  responses  were  stored  electronically  and  then  analyzed.  The  participants  were  language  teachers  enrolled 
in  a semester-long  course.  Seminar  in  Curriculum  & Instruction:  Introduction  to  Computer-Assisted 
Language  Learning  (CALL)  at  the  University  of  Missouri-Columbia.  Their  participation  was  optional.  Two 
individuals  participated  in  the  case  study  by  completing  a semi-structured  questionnaire  with  a large  number 
of  open-ended  questions.  They  were  then  asked  to  respond  to  follow-up  questions  via  e-mail  to  clarify  and 
confirm  the  accuracy  of  the  interview  interpretation  made  by  the  researcher,  and  the  researcher  also  collected 
copies  of  their  multimedia  instructional  project  proposal,  class  reading  journals,  and  other  documents  which 
illustrate  their  multimedia  instructional  project  development  activities. 


RESULTS 

The  Chinese  Language  Teacher*s  Case 

Minjuan,  a doctoral  student,  has  taught  three  levels  of  Chinese  to  undergraduate  students  for  a year  at  a 
major  public  university  in  the  United  States.  She  rated  her  computer  anxiety  as  ’’high”  on  a five-point 
scale.  She  has  experience  in  ’’CALL  courseware  authoring”  and  HTML  editing,  also  familiar  with  other 
’’basic  tools”  such  as  word  processing,  spreadsheet,  Internet  applications,  image  manipulation,  and  audio 
editing.  Minjuan  hasn’t  utilized  computers  in  her  language  classroom  before  but  did  provide  a one-on-one 
tutorial  with  the  use  of  a CD-ROM  software.  She  has  developed  about  four  multimedia  programs 

The  Development  Experience:  Minjuan  developed  the  ’’Practical  Chinese  Reader,”  a 

courseware  designed  for  university  level  Elementary  Chinese  course.  She  stated  that  the  guidelines 
provided  by  the  instructor  who  is  teaching  this  course  set  up  an  excellent  beginning  for  this  project.  It 
took  her  about  4 weeks  to  develop  a set  of  multimedia  instructional  programs  supplementary  to  six 
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chapters  of  the  textbook  students  are  using.  "The  work  was  really  time-consuming,  but  lots  of  fun,"  she 
summarized.  "The  final  project  works  well"  though  there  are  "some  problems  in  font-displaying"  and  in 
one  of  the  chapters,  "audio  is  not  played  well,  due  to  the  quality  of  the  audio." 

Difflculties,  Successes,  and  Concerns:  As  Minjuan  described,  "It  took  too  much  time.  The  biggest 
challenge  is  creating  the  glossary.  I need  to  look  up  in  the  dictionary,  to  make  sure  all  the  words  and 
phrases  are  correctly  annotated."  However,  she  exclaimed  that  "the  success  is  to  see  the  program  works 
well  after  it  is  finished.  All  links  are  good,  audio  is  well  played,  pictures  are  shown  up."  As  a language 
teacher.  Miss  Chan's  concern  with  developing  her  multimedia  program  is  "how  to  integrate  it  into  the 
classroom  instruction." 

Reflections:  "If  s a long  process.  But  I enjoyed  the  creativity,"  Minjuan  commented.  She  followed  Philip 
Hubbard's  design  and  development  modules  which  have  "given  very  good  guidance  on  methodology." 
Both  knowledge  on  the  targeted  language  and  skills  in  using  the  authoring  software  are  most  important. 
Her  extrinsic  motivation  was  to  finish  this  project  as  a class  assignment,  and  the  intrinsic  one  was  to 
present  it  at  a conference  related  to  language  education. 

The  ESL  Teacher's  Case 

Sheila,  a master's  degree  program  student,  has  taught  English  as  a second  language  (ESL)  for  three  years 
in  junior  high  and  senior  high  schools  in  Japan.  She  rated  her  computer  anxiety  as  "none"  on  a five-point 
scale.  Spending  about  two  years  using  computers,  she  has  learned  word  processing  and  basic  HTML, 
computer  art  tools,  and  Macromedia  Director  at  a major  public  university.  She  has  used  software  such  as 
word  processing,  spreadsheet,  Internet  applications,  audio  and  video  editing,  image  manipulation, 
HyperStudio,  and  HyperCard.  Sheila  has  not  utilized  computers  in  a language  classroom  before.  She 
developed  two  simple  multimedia  programs  and  thought  that  "both  were  interesting  experiences,  but  a 
little  frustrating  given  the  amount  of  effort  involved  to  produce  a not  very  professional  product." 

The  Development  Experience:  In  commenting  her  experience  in  developing  her  multimedia 
instructional  program,  a web-based  program  for  people  learning  Japanese,  Sheila  described  it  a 
"sometimes  frustrating,  but  on  the  whole,  a really  enjoyable  experience.  I enjoyed  practicing  using  new 
technology  and  being  creative."  She  had  problems  with  putting  video  on  the  web  because  the  file  size 
was  too  huge,  and  the  end  result  was  very  disappointing.  As  she  explained,  "Planning  first  and  keeping  it 
simple  (i.e.,  without  too  many  bells  and  whistles)  is  what  works.  Aiming  for  too  much  (taking  advantage 
of  everything  you  can  offer  students  on  a web  site)  doesn't." 

Difflculties,  Successes,  and  Concerns:  Sheila  complained  that  her  web  space  provided  by  the  university 
was  too  small  to  accommodate  all  the  sound  files  and  movies  she  had,  "things  seemed  to  change  a lot 
overnight,  without  having  even  touched  them,  and  different  browsers  (even  different  computers)  give 
different  results  in  spite  of  following  what  a web  development  book  said  were  standards."  However, 
Sheila  had  some  positive  comments:  "the  sound  files  are  great.  I was  really  pleased  to  be  able  to  put 
something  worthwhile  for  Japanese  learners  on  the  web.  Also,  the  quizzes  (housed  at  the  University  of 
Hawaii)  are  a valuable,  and  enjoyable,  in  addition  to  the  lesson.  These  features  (along  with  the  fact  that 
anybody  can  access  it  free  at  any  time),  really  made  the  web-based  lesson  far  more  valuable  than  a text 
version  would  have  been."  "Though  worth  it  in  the  end,"  as  a language  teacher,  Sheila's  only  concern  was 
that  the  development  process  was  "extremely  time  consuming." 

Reflections:  Sheila  felt  excited  and  challenged  by  the  development  process  in  which  planning 
(storyboarding)  was  an  essential  stage  because  she  had  to  "think  first  in  terms  of  educational  value  rather 
than  what's  technologically  possible."  The  "ability  to  honestly  assess  educational  value  of  the  multimedia 
project  and  a good  sense  of  what's  really  practical  and  useful  for  students"  were  important,  she  explained. 
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"On  the  technical  side,  knowledge  of  how  to  create  web  pages  is  a must."  Intrinsic  motivation  for  Sheila 
was  to  get  a good  job  that  she  enjoys  doing,  and  she  described,  "personal  work  satisfaction  which  would 
be  achieved  by  helping  students  (particularly  disaffected  ones)  get  motivated  and  interested  in  learning 
by/for  themselves.  The  desire  to  create  something  worthwhile,  be  useful  and  productive,  and  make  a 
difference  is  strong."  She  would  not  mind  if  someone  is  willing  to  pay  her  "lots  of  money  for  this." 


DISCUSSION 

Previous  Computer  Experience 

The  participants  in  this  study  are  considered  as  intermediate  to  advanced  users  of  the  computer  and 
multimedia  production  software.  In  the  future,  beginning  users  and  non-users  would  be  included  in  order  to 
reveal  their  experiences  which  might  be  different  from  those  described  above,  so  the  assumption  about  novice 
users  tending  to  report  more  technical  difficulties  and  higher  frustration  as  well  as  anxiety  level  than 
experienced  users  will  then  be  discovered. 

Both  participants  emphasized  that  having  the  technical  skills  is  crucial.  An  issue  related  to  this  is  whether 
pre-service  and  in-service  language  teachers  should  take  a series  of  technology  courses  prior  to  taking  the 
CALL  course  which  focuses  on  both  theory  and  practice,  or  they  should  learn  most  of  the  technical  skills  like 
using  multimedia  production  software  in  the  CALL  course.  For  this  issue,  two  sides  of  the  coin  need  to  be 
examined.  If  novice  users  take  a certain  number  of  technology  courses  that  are  of  interest  to  them  first,  they 
will  have  at  least  some  good  working  knowledge  about  computers  when  they  are  taking  the  CALL  course 
later.  However,  if  they  first  take  the  CALL  course  partly  as  an  introduction  to  software  available  to  them; 
once  they  have  a basic  understanding  of  the  technical  functionality  and  feasibility  of  the  software,  they  then 
can  decide  later  which  technology  course(s)  they  would  like  to  take  in  the  future  and  utilize  those  skills 
acquired  from  such  course(s)  to  create  their  multimedia  instructional  programs. 

Difficulties  and  Concerns 

Both  participants  reported  that  the  development  of  their  multimedia  instructional  programs  required  a 
tremendous  amount  of  time  although  approximately  one  third  of  the  class  time  was  devoted  to  developing.  In 
math  and  science  education,  lab  and  tutorial  hours  are  supplementary  to  regular  class  meetings.  Likewise,  for 
a course  in  language  education  that  has  a multimedia  development  component,  it  might  be  adapted  to  the 
model  used  in  math  and  science  courses. 

Technical  difficulties  are  no  surprises:  just  to  name  a few,  limitation  of  web  space  and  poor  audio  clip. 
Participants  should  be  able  to  request  a larger  web  space  if  they  intend  to  create  a complex  multimedia  web- 
based  program.  The  computer  lab  should  be  equipped  with  acceptable  quality  of  multimedia  production  and 
editing  software  and  hardware.  In  terms  of  dealing  file  content  instability  on  the  web  server,  participants  in  a 
CALL  course  are  advised  to  keep  a master  copy  of  the  entire  web-based  program  for  recovery  and  a working 
copy  for  revision.  Similarly,  such  dual  copy  approach  can  be  adapted  and  adopted  by  participants  creating 
other  multimedia  programs  than  a web  site. 

One  of  the  participant's  concerns  was  how  to  incorporate  her  multimedia  program  into  the  classroom.  During 
the  planning  stage,  it  is  essential  to  invite  participants  in  a CALL  course  to  conduct  a user  analysis  of  their 
multimedia  program.  A review  of  the  target  audience  will  help  developers  to  focus  on  the  content  and 
interface  of  tihe  multimedia  program.  Philip  Hubbard’s  (1996)  design  module  provides  a concrete  framework 
to  guide  developers  building  the  ideas  theoretically  and  practically.  Besides,  participants  in  a CALL  course 
should  strongly  be  encouraged  to  create  something  they  will  use  in  their  own  classrooms  as  an  authentic 
learning  and  production  experience. 


Positive  Comments 


Though  both  participants  encountered  frustration,  difficulties  and  challenges  in  the  process  of  creating  their 
multimedia  programs,  they  agreed  that  it  was  an  enjoyable  experience  after  all,  particularly  the  opportunity 
for  them  to  express  their  creativity  though  the  final  product.  \^en  things  have  come  to  work,  the  degree  of 
satisfaction  and  a sense  of  achievement  are  positive  driving  forces  for  these  amateur  multimedia  developers. 


IMPLICATIONS 

More  participants,  especially  those  with  various  levels  of  computer  literacy,  will  be  needed  to  share  their 
experiences  for  this  on-going  research  study,  so  a larger  scope  of  experiences  can  be  documented  and 
revealed.  The  researcher  of  the  study  has  found  the  data  collected  so  far  very  useful  in  refining  the  current 
CALL  program  designed  for  language  teachers. 
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Abstract:  Interactivity  is  a popular,  although  little  understood  term.  This  paper  provides 
a short  definition  of  interactivity  in  instruction.  It  then  proposes  a framework  to  study 
interactivity  that  addresses  both  content  learning  and  affective  benefits  that  arise  from 
instructional  interactivity. 


The  Nature  of  Instructional  Interactivity 

The  proclaimed  educational  benefits  of  the  "new  media"  are  their  potential  for  increased  and  improved 
interactivity.  However,  the  nature  of  interactivity  is  a conceptual  area  that  has  historically  been  poorly 
developed.  This  paper  presents  a cognitive-affective  framework  for  instructional  interactivity  (CAFI^),  and 
describes  the  general  rationale  for  the  framework.  The  CAFI^  is  currently  being  used  as  part  of  an  ongoing 
cataloging  and  evaluation  project,  described  later  in  the  paper. 

In  his  keynote  address  at  AACE,  Thomas  Reeves  (1999)  suggested  that  we  do  not  have  a very  good 
definition  of  interactivity  in  instruction,  suggesting  that  "teacher-in-a  box"  has  been  the  prevailing 
approach.  The  Institute  for  Higher  Education  Policy's  recent  review  on  distance  learning  (1999)  cited  the 
lack  of  a conceptual  or  theoretical  framework  as  a major  shortcoming  in  studying  distance  learning. 

Research  on  various  aspects  of  interactivity  seems  to  focus  on  either  the  cognitive  or  affective  aspects  of 
interaction,  but  not  both.  For  example,  human-computer  interaction  studies  generally  concern  themselves 
with  the  way  that  users  understand  computer  systems,  the  mistakes  that  they  make,  and  the  previous 
knowledge  that  they  bring  into  play,  among  other  issues.  These  types  of  studies  are  generally  cognitive  in 
their  concerns  with  a user's  understanding  of  the  principles  and  concepts  of  a system.  On  the  other  hand, 
affective  issues  are  frequently  brought  to  light  in  teacher  training  and  educational  literature  as  issues  of 
motivation,  group  support,  participation,  collaboration,  and  attribution  among  others. 

Therefore,  a framework  that  describes  both  cognitive  and  affective  interactivity  within  a single  framework 
provides  a mechanism  for  integrating  past  and  future  research.  This  framework  would  also  become  a 
structure  by  which  evaluators  could  more  thoroughly  study  distance  delivered  courses. 

This  paper  describes  the  rationale  behind  the  Cognitive-Affective  Framework  for  Instructional  Interactivity 
(CAFI^),  an  attempt  to  link  previous  research  across  the  cognitive  and  affective  domains  into  a 
contemporary  framework  for  analyzing  instruction.  This  framework  has  been  initially  intended  to  provide  a 
map  and  a terminology  by  which  designers  can  catalog  and  describe  the  components  of  distance  learning 
courses.  It  is  hoped  that  this  framework  can  provide  a mechanism  for  analysis  of  current  instructional 
designs  so  that  teachers  and  designers  can  ensure  that  courses  are  appropriately  providing  for  cognitive 
aspects  of  instruction  as  well  as  affective  aspects. 


Interactivity  Defined 

The  term  interactivity  is  itself  troubling.  While  the  term  is  common  in  the  literature,  the  exact  meaning  of 
interactivity  is  generally  quite  vague.  We  attempted  to  create  a structural  definition  of  interactivity  that 
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The  term  affective  benefits  was  selected  because  learning  and  growth  in  the  affective  domain  is  not  as 
straight-forward  as  growth  in  the  cognitive  domain,  where  the  principle  of  complexity  (Krathwohl  et.al., 
1 964)  is  the  primary  organizational  principle.  It  also  would  appear  that  affective  growth  is  not  the  primary 
reason  for  including  certain  types  of  interaction.  Motivational  and  social  benefits  as  they  relate  directly  to 
the  student,  independent  of  cognitive  learning,  seem  to  be  the  underlying  results-not  growth  in  a direct 
sense. 

Previous  research  shows  that  affect  is  a significant  contributor  to  academic  achievement  (Khan,  1 969),  but 
the  area  still  has  much  that  is  unknown  (Brophy,  1999).  It  is  our  belief  that  affective  benefits  provide  value 
added  beyond  academic  achievement.  For  example,  it  would  seem  that  affective  benefits  can  be  reduced  or 
eliminated  (in  a purely  Spartan  approach  to  content  learning)  but  at  the  cost  of  lessened  student  satisfaction. 
Therefore,  we  believe  that  these  affective  benefits  need  to  be  examined  outside  of  their  role  in  academic 
achievement. 

Krathwohl,  Bloom,  and  Masia  (1964)  proposed  their  original  taxonomy  for  the  affective  domain  to  "clarify 
and  tighten  the  language"  of  instruction.  In  a similar  manner,  it  is  believed  that  the  CAFI^  framework  can 
help  to  link  a variety  of  research  interests  in  interactivity  in  instruction.  The  remainder  of  this  paper 
sketches  the  three  major  areas  of  the  framework. 


1.  Directive  (learner-instructor)  interaction. 

This  area  represents  the  face-to-face,  mediated  real-time,  and/or  asynchronous  contact  that  a student  has 
with  an  instructor.  In  some  cases,  the  instructor  may  be  mediated,  as  in  the  case  of  videotapes,  books,  or 
computer-based  training.  In  some  circumstances,  the  instructor  is  "live"  in  the  sense  of  being  physically 
proximate  to  the  student.  An  interesting  hybrid  is  the  "real-time  teleconference"  in  which  the  teacher  is  not 
in  physical  proximity  to  the  student,  but  is  indeed  in  mental  proximity— that  is,  the  teacher  and  student  are 
interacting  in  real  time  over  distance. 

The  Directive  interaction  produces  effects  in  two  areas  previously  described:  content  learning  and  affective 
benefits.  While  an  instructor  is  frequently  a primary  delivery  mechanism  for  content,  an  instructor  serves 
motivational  roles  as  well. 

The  instructor  as  dispenser  of  information  has  often  taken  the  form  of  teacher  as  lecturer  or  teacher  as 
Socratic  guide.  The  "lecture"  is  often  a misnomer,  as  (in  practice)  it  often  allows  for  questions,  answers, 
side  conversations,  and  spur-of-the-moment  inclusion  and  elimination  of  topics.  So,  while  a lecture  in  its 
most  extreme  form  is  clearly  one-way  preaching,  in  practice  the  lecture  tends  to  be  a more  interactive 
activity  between  teacher  and  student.  In  this  interactive  lecture  form,  the  instructor  provides  content, 
remediation  (in  the  form  of  alternative  explanations  and  review)  and  clarifies  relationships  with  other 
content. 

The  interactive  lecture  can  take  the  form  of  computer  based  training,  in  which  the  student  interacts  with  a 
computer-based  instructor.  In  this  tutorial  format,  the  software  often  takes  on  the  persona  of  a teacher,  and 
the  language  that  is  used  is  frequently  modeled  after  live  instructors. 

An  important  contributor  to  content  learning  is  the  instructor  interaction  that  provides  student  feedback  on 
performance.  Mager  (1988)  suggests  three  types  of  feedback  (adequacy,  diagnostic,  and  corrective)  that  can 
be  provided  systematically.  It  is  generally  acknowledged  that  knowledge  of  results  is  one  of  the  more 
critical  contributors  to  content  learning  (Schimmel,  1988). 

The  teacher  as  motivator  has  been  documented  in  many  educational  psychology  texts.  It  may  help  to 
substitute  other  terms  for  instructor,  such  as  facilitator  or  coach.  For  example,  few  would  argue  that  a 
primary  function  of  an  athletic  coach  is  to  provide  motivational  support  for  players.  It  is  from  this  direction 
that  we  ascribe  affective  benefits  to  an  instructor.  The  qualitative  evidence  that  a good  instructor  is 
motivational,  inspirational,  etc,  is  common.  It  is  not  likely  that  this  motivating  effect  is  related  to  actual 
physical  presence;  students  can  get  motivated  at  a distance  even  if  their  contact  with  instructors  is  limited  to 
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text  interactions.  Students  may  also  find  motivational  benefits  in  the  writing  style  of  computer-based 
tutorials  and  textbooks  that  are  written  so  as  to  foster  motivation. 


2.  Experiential  (learner-environment)  interaction 

The  Experiential  interaction  should  be  thought  of  as  a distinct  type  of  interaction  typified  by  a student's 
direct  interaction  with  materials  from  the  content  domain.  When  a student  works  directly  with  the  materials 
of  the  content  area  (or  with  simulations  of  these  materials)  we  can  say  that  the  student  is  interacting  directly 
with  the  content  environment.  In  this  type  of  interaction,  there  is  not  an  instructor  mediating  the  experience- 
-that  is,  the  feedback  that  the  student  receives  comes  directly  from  the  environments  (or  its  simulation). 

Kolb  (1984)  called  this  type  of  world/person  interaction  grasping  by  apprehension.  He  describes  this  type 
of  interactivity  thusly: 

"Apprehension  of  experience  is  a personal  subjective  process  that  cannot  be 
known  by  others  except  by  the  communication  to  them  of  the  comprehensions 
that  we  use  to  describe  our  immediate  experience."  (page  105). 

An  analogy  that  might  clarify  the  differences  between  Directive  and  Experiential  interactivity.  The 
Directive  (learner-instructor)  interactivity  is  like  a guided  walk  through  the  woods—the  path  has  been  well- 
paved  and  the  guide  protects  the  hikers  from  danger.  The  Experiential  (learner-content)  interactivity  is  like 
a visit  to  the  untamed  wildemess—there  is  no  path  provided,  and  the  hiker  faces  the  risks  that  an  explorer 
faces— there  is  no  certainty  of  success,  but  there  may  be  a bigger  personal  reward  when  complete. 

In  like  manner,  the  Experiential  interaction  provides  students  with  the  "raw  materials"  of  instruction  and 
allows  students  to  explore  and  construct  a path.  It  would  seem  that  discovery  learning  techniques  would  be 
natural  instructional  methods  to  be  used  to  foster  this  type  of  instructional  interactivity. 

Problem-based  learning  (Saveiy  & Dufly,  1996)  is  the  current  categorical  space  that  would  include 
Experiential  interactivity.  Among  other  interactions.  Experiential  interactivity  might  include: 

* checking  for  understanding  (working  with  materials  to  test  student  hypotheses)  and 

* organizing  raw  data  (creating  cognitive  structures  for  student-located  data). 

Kolb  claims  that  there  are  affective  benefits  associated  with  this  form  of  interactivity.  We  can  surmise  that 
certain  students  would  greatly  enjoy  this  type  of  interaction,  while  others  might  find  it  to  be  frustrating.  Our 
own  work  (Yacci,  1994)  shows  that  some  students  prefer  the  opportunity  to  explore  for  themselves,  while 
others  prefer  to  be  "guided."  Hence,  some  (but  certainly  not  all)  students  are  likely  to  find  that  there  is  an 
accompanying  affective  benefit  to  Experiential  interactivity. 

3.  Coordinate  (learner-learner)  Interactivity 

Coordinate  interactivity  occurs  when  students  communicate  with  each  other.  Typically,  students  might 
exchange  ideas,  collaborate  on  projects,  model  strategies  for  success,  or  share  "war"  stories.  These  are 
generally  used  to  provide  content  learning  to  students.  Coordinate  interactivity  can  also  be  a great  source  of 
affective  benefits  for  students  who  enjoy  working  with  other  students.  The  opportunity  to  work  with  others 
is  often  desirable  as  a means  of  motivating  students.  Coordinate  interactivity  also  allows  "unvoiced"  self- 
comparison between  students.  It  allows  them  to  get  closer  to  others  who  are  working  towards  the  same 
goals  and  to  "size  up"  work  habits  and  methods  of  others. 

Peer  tutoring  is  (simply  put)  a method  in  which  more  able  students  teach  less  able  students.  This  type  of 
interactivity  should  produce  content  learning  for  the  "learning  student"  and  might  provide  affective  benefits 
(friendship,  a sense  of  involvement,  greater  self  worth)  to  the  "teaching  student."  Less  formal  mechanisms 
for  student  cooperation,  such  as  team  projects  and  work  groups  often  provides  peer  modeling  of  strategies. 
As  students  work  together  to  complete  such  projects,  they  provide  insight  into  different  ways  of  problems 
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solving,  different  mechanisms  for  approaching  the  unknown.  Both  the  cognitive  and  affective  benefits  of 
work  groups  are  often  unstructured;  there  is  no  guarantee  that  peer  modeling  or  friendship  will  occur. 

When  students  communicate,  the  benefit  can  be  seen  in  changes  in  motivation,  as  some  students  use  the 
opportunity  to  compete  against  others  as  a motivator.  Others  students  might  use  the  communication 
channels  for  commiseration  and  support.  Still  others  have  subtle,  hidden  agendas  that  may  exceed  any 
given  course,  such  as  political  leadership. 
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Abstract:  The  education  arena  in  Singapore  has  seen  very  rapid  pervasion  of  Information  Technology 
(IT)  and  the  World  Wide  Web  (WWW).  With  the  proliferation  of  courseware  developed  for  the  WWW, 
management  of  courseware  development  becomes  an  important  issue  that  needs  to  be  addressed  for 
maintainability,  efficiency  and  effectiveness.  Even  as  the  volume  of  WWW-based  courseware 
development  under  Temasek  Polytechnic's  Online  Learning  Environment  (OLE™)  [3]  grows,  the  Center 
for  IT  in  Education  and  Learning  [4]  set  out  to  develop  a WWW-based  Resource  Management  System 
(RMS)  for  the  effective  management  of  courseware  development  in  three  aspects,  namely 
Documentation,  Management  and  Standards.  The  objective  of  this  paper  is  to  discuss  the  plans  and  effort 
of  Temasek  Polytechnic  in  the  development  of  RMS.  The  implementation  of  the  completed 
Documentation  module  has  improved  our  workflow  and  productivity  in  various  areas  including  the 
reduction  in  paperwork,  elimination  of  redundancy,  and  enhancement  in  the  presentation  of  Temasek 
Polytechnic’s  OLE”^^  courseware  offerings.  The  expected  benefits  from  the  future  development  of  the 
Management  module  include  more  effective  management  of  courseware  development  in  the  areas  of 
Project  Tracking,  Usage  Tracking,  and  Electronic  Commerce.  Compliance  to  the  Instructional 
Management  System  [9]  from  the  future  development  of  the  Standards  module  is  envisaged  to  make 
Temasek  Polytechnic's  courseware  offerings  more  accessible  worldwide. 


1 Introduction 

The  education  arena  in  Singapore  has  seen  very  rapid  pervasion  of  Information  Technology  (IT)  and  the  World 
Wide  Web  (WWW).  The  impetus  is  primarily  fueled  by  the  initiatives  of  the  Singapore  government,  which  has 
taken  an  aggressive  and  proactive  approach  in  this  area.  For  example,  the  Singapore  government  has  various  IT 
related  masterplans,  including  a Masterplan  for  IT  in  Education  [I]  as  a blueprint  for  the  integration  of  IT  in 
education  as  a strategy  to  meet  the  challenges  of  the  century. 

With  this  backdrop,  the  educational  institutions  in  Singapore,  and  in  particular,  the  tertiary  institutions  are  moving 
rapidly  to  adopt  IT  in  teaching  and  learning,  and  to  develop  courseware  for  the  WWW  to  harness  the  potential  it 
offers.  With  the  proliferation  of  courseware  development  for  the  WWW,  courseware  management  becomes  an 
important  issue  that  needs  to  be  addressed  for  maintainability,  efficiency  and  effectiveness. 

In  Temasek  Polytechnic,  Singapore  [2],  even  as  the  volume  of  WWW-based  courseware  development  under  the 
Online  Learning  Environment  (OLE™)  [3]  grows,  the  Center  for  IT  in  Education  and  Learning  (CITEL)  [4]  set  out 
to  develop  a WWW-based  Resource  Management  System  (RMS). 

(CITEL  is  a support  centre  that  facilitates  academic  staff  in  harnessing  information  technology  effectively  in  the 
teaching  and  learning  environment  so  as  to  make  the  teaching/leaming  experience  a creative,  innovative  and 
enjoyable  one.  CITEL  initiated  OLE™  in  1997  as  a service  to  offer  computer-based  learning  courses  for  students  of 
Temasek  Polytechnic  and  the  general  public  to  develop  work-place  skills  for  the  new  millennium.  It  also  offers 
useful  resources  for  staff  and  students.) 

The  objective  of  RMS  is  to  more  effectively  address  what  will  be  classified  here  as  three  main  aspects  pertaining  the 
effective  management  of  courseware  development  which  is  deemed  important  for  CITEL’s  requirements.  They  are 
namely  Documentation,  Management  and  Standards. 


The  objective  of  this  paper  is  to  discuss  the  plans  and  effort  of  CITEL  in  the  development  of  RMS  to  address  the 
three  aspects  of  courseware  development  highlighted.  The  RMS  project  is  divided  into  three  corresponding  phases. 
The  development  platform  used  is  PERL  programming  language  and  MS  SQL  database  under  the  Windows  NT 
operating  system. 


2 Documentation 

As  with  MIS  applications,  documentation  for  WWW-based  courseware  is  typically  few  and  far  between.  If 
documentation  does  exist,  it  is  usually  not  standardized  and  not  kept  up  to  date,  and  probably  only  the  creator  knows 
where  to  locate  it.  Furthermore,  bits  and  pieces  of  such  documentation  are  sometimes  duplicated  here  and  there,  on 
an  adhoc  basis.  This  is  not  particularly  effective,  and  it  poses  a challenge  for  attempts  to  update,  as  well  as  for 
maintaining  information  integrity. 

Phase  1,  the  Documentation  module  {Courseware  Information  System  [5])  designed  to  address  these  issues  was 
completed  in  1999,  and  is  operational.  Some  of  the  objectives  (features)  of  the  Documentation  module  are: 

1.  To  maintain  a centralised  database  for  online  creation,  updating,  viewing  and  reporting  of  all  courseware 
documentation. 

2.  To  provide  a standardised  and  consistent  look-and-feel  of  all  courseware  documentation  web  pages  generated 
on  the  fly. 

3.  To  dynamically  generate  Temasek  Polytechnic’s  OLE^^  User  Interface  [6]  links  for  a one-stop  access  to  all  the 
available  courseware  and  their  respective  documentation. 


3 Management 

Our  plans  for  the  future  development  of  the  Management  module  under  phase  2 of  the  CMS  project  consists  of  three 
areas,  namely  Project  Tracking,  Usage  Tracking  and  E-commerce. 

3.1  Project  Tracking 

Many  project-tracking  tools  exist  in  the  market.  However,  for  our  intent  and  purposes,  we  integrated  basic  project- 
tracking information  in  CMS  such  as  the  stage  of  development  of  the  project,  and  the  respective  time-line.  This  is 
useful  for  a number  of  areas  in  our  environment. 

For  example,  some  of  our  OLE*^^  projects  received  funding  from  the  Infocomm  Development  Authority  (IDA)  [7]  of 
Singapore.  As  such,  we  were  required  to  update  IDA  on  the  progress  of  these  projects  on  a regular  basis.  With 
basic  project  tracking  integrated  into  CMS,  we  were  able  to  issue  IDA  an  access  ID  to  CMS  for  real-time  viewing  of 
the  summary  of  the  projects’  progress  via  the  WWW,  as  and  when  needed,  rather  than  manually  preparing  them, 
which  can  be  both  tedious  and  time-consuming. 

Likewise,  authorized  users  login  to  CMS  to  view  reports  on  the  status  and  progress  of  the  courseware  projects,  again 
saving  the  effort  for  otherwise  paper-based  reports. 

3.2  Usage  Tracking 

Usage  tracking  is  typically  managed  by  the  courseware  development  tool  itself  such  as  Courselnfo,  or  programmed 
into  the  individual  courseware  as  necessary  if  the  tool  used  does  not  have  such  inherent  features  built-in. 

Until  there  is  a unified  standard  for  exchanging  such  data  among  different  software  tools  and  courseware 
management  systems,  our  plan  for  CMS  at  the  moment  is  to  just  keep  track  of  basic  information  like  ’’when  did  who 
access  or  update  the  courseware  and  its  documentation”.  This  will  provide  some  basic  courseware  access  statistics 
for  our  internal  use. 
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3.3  E-commerce 


Currently,  Temasek  Polytechnic  ties  up  with  a commercial  vendor  for  e-commerce  subscription  of  our  courseware 
offerings.  When  sufficient  volume  justifies,  Temasek  Polytechnic  will  examine  the  viability  of  managing  this  aspect 
by  herself,  with  e-commerce  functionality  incorporated  into  CMS.  The  design  and  implementation  will  be  decided 
upon  at  a later  stage,  depending  on  the  state  of  the  technology  and  available  tools  then. 


4 Standards 

The  development  and  sharing  of  instructional  courseware  have  been  hampered  by  a lack  of  standards  that  enables 
sharing  across  institutions  and  across  different  technical  platforms.  While  the  World  Wide  Web  has  facilitated 
access  to  instructional  contents,  its  effectiveness  in  this  aspect  is  at  best  limited.  Finding  relevant  and  useful 
information  on  the  web  is  likened  unto  finding  a needle  in  a haystack  because  of  a lack  of  existing  structure  or 
standards  for  documenting  available  content. 

To  address  this,  EDUCAUSE  [8]  launched  an  initiative  called  the  National  Learning  Infrastructure  Initiative  in 
November  1994.  The  Instructional  Management  System  (IMS)  [9]  was  formed  to  address  the  following  three 
obstacles  for  providing  effective  on-line  materials  and  learning  environments: 

1 . Lack  of  standards  for  locating  and  operating  interactive  platform-independent  materials 

2.  Lack  of  support  for  the  collaborative  and  dynamic  nature  of  learning 

3.  Lack  of  incentives  and  structure  for  developing  and  sharing  content 

(The  IMS  Consortium  is  a member-supported,  nonprofit,  global  organization  that  includes  more  than  200  campuses, 
university  systems,  tool  vendors,  system  consultants  and  service  providers,  content  developers,  and  U.S.  and 
international  governmental  organizations  committed  to  the  development  of  timely,  high-quality,  practical  open 
standards  for  interoperability  of  online  learning  and  training  tools,  content  and  services.) 

In  collaboration  with  various  institutions  and  organizations,  IMS  set  out  to  address  these  obstacles  via  a set  of 
higher-order  standards  and  tools.  These  standards  and  tools  will  enable  teachers,  learners,  software  developers, 
content  providers,  and  other  parties  involved  in  the  learning  process  to  create,  manage  and  access  the  use  of  on-line 
learning  materials  and  environments. 

Realizing  the  potential  merits  of  such  an  initiative,  IDA  of  Singapore  subscribed  to  IMS  (1999)  on  behalf  of  the 
institutions  of  higher  learning.  Temasek  Polytechnic,  together  with  the  various  tertiary  institutions  in  Singapore,  and 
co-ordinated  by  Kent  Ridge  Digital  Labs  [10]  actively  worked  with  IMS  to  contribute  towards  the  shaping  of  the  set 
of  standards  and  tools.  IMS  has  since  announced  the  release  of  various  sets  of  specifications,  namely  Meta-Data, 
Enterprise,  Content  Packaging,  and  Question  & Test  specifications. 

With  the  RMS  infrastructure  in  place,  CITEL  is  well  positioned  to  adopt  the  standards  when  it  takes  off  Making 
Temasek  Polytechnic's  courseware  IMS-compliant  will  be  primarily  what  phase  3 of  the  RMS  project  targets  to 
achieve.  This  will  make  our  online  courseware  offerings  more  readily  accessible  globally.  In  addition,  RMS  can 
also  be  integrated  with  our  organization’s  enterprise  systems  for  greater  synergy,  and  extended  for  uses  with  other 
resources  such  as  CD-ROMs. 


5 Conclusions 

The  development  of  RMS  (and  implementation  of  the  completed  modules  so  far)  has  significantly  improved  our 
workflow  and  productivity  in  the  various  areas  discussed.  Some  of  the  benefits  realized  from  the  completed 
Documentation  module  include  reduction  in  paperwork,  elimination  of  redundancy,  and  enhancement  in  the 
presentation  of  our  courseware  offerings. 
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The  expected  benefits  from  the  development  of  the  Management  module  include  more  effective  management  of 
courseware  development  in  the  areas  of  Project  Tracking,  Usage  Tracking,  and  E-commerce.  Compliance  to  IMS  for 
the  Standards  module  is  envisaged  to  make  Temasek  Polytechnic’s  courseware  offerings  more  accessible  worldwide. 
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Abstract:  Captions  do  not  always  facilitate  listening  comprehension;  the  degree  of  their 
effectiveness  varies  according  to  the  amount  of  information  that  the  individual  can  process  in  a 
limited  period  of  time.  In  order  to  reduce  the  information  that  a learner  processes  at  a time,  the 
effective  use  of  timing  is  considered  to  maximize  listening  comprehension.  The  results  show  that 
more  information  is  memorized  when  L2  captions  are  presented  before  the  corresponding  L2  audio. 
It  is  suggested  that  this  timing  facilitates  the  listening  comprehension  of  Japanese  EFL  students 
because  they  could  concentrate  on  reading  the  captions  and  could  prepare  for  the  following  audio. 
This  effect  was  not  seen  with  LI  captions  and  L2  audio;  student  performance  was  relatively 
independent  of  timing. 


Introduction 

Captions  are  used  in  second  language  (L2)  classrooms  as  an  effective  aid  for  listening  comprehension,  and 
their  effects  have  been  examined  in  previous  research  and  studies.  Yoshino  et  al.  (1997,  1998)  studied  Japanese 
university  and  college  students  with  English  as  the  foreign  language  (EFL),  and  found  that  both  LI  and  L2  captions 
facilitated  listening  comprehension.  The  effectiveness  of  the  LI  and  the  L2  captioning  varied  with  the  difficulty 
level  of  the  materials.  The  captions  were  more  effective  in  aiding  listening  comprehension  if  the  materials  were 
relatively  easy.  Yoshino  et  al.  argued  that  this  might  be  due  to  the  limitation  of  the  student’s  “processing  capacity” 
(Just  & Carpenter,  1992). 

LI  and  L2  captions  differ  in  terms  of  learning  support;  more  information  was  recalled  correctly  if  L2  captions 
were  used  than  LI  captions.  L2  captions  provide  exactly  the  spoken  words  as  while  LI  captions  provided  the  same 
information  in  a different  (subjects’  native)  language.  With  LI  captions,  Japanese  EFL  students  grasped  the 
meaning  by  reading  LI  captions,  and  simultaneously  translated  the  LI  caption  information  for  chunking  the  flow  of 
L2  audio.  This  process  of  translation  and  chunking,  if  successful,  allowed  them  to  understand  and  recall  what  have 
been  said.  Because  the  translation  process  is  rather  difficult  to  manage,  exceeding  the  learner’s  processing  capacity, 
say  by  increasing  the  presentation  speed  or  raising  the  difficulty  level  of  the  text,  may  lower  the  effectiveness  of  LI 
captioning. 

From  this  point  of  view,  the  effective  use  of  timing  was  considered  to  reduce  the  information  that  student  had 
to  handle  at  any  one  time  and  so  increase  caption  effectiveness.  By  showing  the  captions  in  a audio  dead  period 
before  the  corresponding  audio,  viewers  may  be  better  able  to  process  both  tasks:  understand  the  caption  and 
understand  the  audio. 

Yoshino,  Kano  & Akahori  (1999)  conducted  preliminary  research  to  examine  the  effects  of  caption 
presentation  timing.  In  their  experiment,  LI  and  L2  captions  were  presented  either  earlier  or  later  than  the 
corresponding  audio,  and  words  and  phrases  recalled  by  the  subjects  were  counted.  The  results  showed  that  recall 
was  highest  when  the  captions  were  provided  prior  to  the  audio.  These  results,  however,  were  gained  from  thirty- 
two  subjects  watching  five  different  sets  of  materials  with  five  different  caption  presentation  timing  arrangements. 
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It  is  presumed  that  the  difficulty  level  of  the  materials  influenced  the  results  to  some  extent.  We  decided  to  examine 
the  effect  of  caption  presentation  timing  by  designing  an  experiment  that  used  only  one  set  of  material.  Because  of 
the  limited  number  of  subjects,  the  effects  of  LI  and  L2  captioning  were  examined  in  two  discrete  experiments. 


Experiment  1 

Participants 

One  hundred  and  ten  Japanese  junior  college  students,  enrolled  in  the  course  “Language  Laboratory  I”, 
participated  in  this  experiment.  Based  on  the  results  of  a listening  comprehension  test  administered  at  the  beginning 
of  the  semester,  each  student  was  assigned  to  one  of  the  thirteen  classes;  the  level  of  the  students  in  each  class  was 
assumed  to  be  equal.  From  these  thirteen  classes,  five  classes  were  selected  and  each  class  was  randomly  assigned 
to  one  of  the  five  caption  timing  conditions. 

Material 

The  materials  used  to  measure  the  subjects’  listening  comprehension  were  developed  from  a TV  commercial 
presented  by  the  US  government.  The  commercial  presented  68  words  in  30  seconds;  that  is,  the  speech  speed  was 
136  words  per  minute  (wpm).  The  captions,  which  exactly  matched  with  the  corresponding  spoken  (audio) 
statements,  were  superimposed  using  the  video  edit  software  Media  100.  The  captions  were  presented  completely 
on  the  black  mask  which  appeared  under  part  of  the  screen. 

Five  versions  were  developed  from  the  same  TV  commercial  video:  1)  the  captions  appear  on  the  screen  two 
seconds  prior  to  the  corresponding  audio  (-2  Time  Lag  condition),  2)  the  captions  are  presented  one  second  before 
the  corresponding  audio  (-1  Time  Lag  condition),  3)  the  captions  and  the  corresponding  audio  are  simultaneously 
presented  (0  Time  Lag  condition),  4)  the  captions  appear  one  second  after  the  corresponding  audio  (+1  Time  Lag 
condition)  and  5)  the  captions  appear  two  seconds  after  the  corresponding  audio  statement  (+2  Time  Lag  condition). 

The  average  length  of  pauses  in  the  video  material  was  about  one  second;  that  is,  between  every  proposition 
(caption),  there  was  a second  audio  dead  period.  When  the  captions  appeared  and  disappeared  simultaneously  with 
the  audio  (0  Time  Lag  condition),  the  viewer  must  have  read  the  visual  verbal  information  (caption)  and  listen  to  the 
audio  verbal  information.  Moreover,  they  have  to  decode  the  visual  verbal  information  at  the  spoken  speed  of  audio. 
However,  when  the  caption  projected  in  audio  dead  period  before  the  corresponding  audio  (-1  Time  Lag  condition)  - 
the  caption  appeared  on  the  screen  just  after  the  forward  audio  was  finished,  the  subjects  might  be  able  to  read  and 
prepare  for  the  forthcoming  audio  statements.  Likewise  presenting  the  caption  in  no  audio  statements  time  after  the 
corresponding  audio  (+1  Time  Lag  condition),  the  subjects  were  compliant  what  have  been  said  with  the  caption 
remained  on  the  screen  one  second  after  the  corresponding  audio  was  finished.  Two  other  conditions  (-2  Time  Lag 
and  +2  Time  Lag  condition)  were  arranged  to  compare  with  the  -1  Time  Lag  and  +1  Time  Lag  conditions.  As  the 
average  length  of  audio  dead  period  was  one  second,  when  the  captions  presented  two  seconds  before  the 
corresponding  audio  (-2  time  Lag),  every  caption  were  provided  with  the  previous  audio  while  a second.  Contrary, 
in  +2  Time  Lag  condition,  the  captions  were  always  appeared  two  seconds  after  the  corresponding  audio  was  started. 
Then  the  captions  overlapped  with  just  after  spoken  statements  while  a second.  Therefore,  in  these  two  conditions, 
the  visual  information  and  the  verbal  information  were  discordant.  This  discordance  might  cause  the  viewers’ 
confusion  for  understanding  the  contents. 

Procedure 

The  subjects  were  instructed  to  watch  the  video  twice  with  a thirty-second  break,  which  is  a common  way  of 
viewing  English  videos  in  Japanese  EFL  classrooms.  They  were  asked  to  recall  the  video’s  narration  both  in 
English  and  in  Japanese  after  the  second  presentation.  They  were  informed  of  the  purpose  of  this  study;  that  is,  the 
experimenters  were  investigating  the  amount  of  information  the  subjects  could  remember  with  caption  use.  They 
knew  that  spelling  and  grammar  would  not  be  scored.  The  subjects  were  requested  to  write  down  every  word, 
phrase,  clause,  or  sentence  that  they  could  recall  after  each  viewing. 

Scoring  and  Data  Analysis 
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Two  measures  - the  number  of  recalled  words  and  accuracy  of  recalled  information  - were  used  to  examine  the 
effect  of  caption  presentation  timing.  The  first,  the  number  of  recalled  words,  was  number  of  words  recalled 
correctly  by  the  subjects.  The  second,  accuracy  of  recalled  information  was  needed  since  the  number  of  recalled 
words  does  not  confirm  that  the  subjects  understood  the  statement.  The  accuracy  of  recalled  information  was 
assessed  by  checking  whether  English  (L2)  recall  matched  Japanese  (LI)  recall  in  meaning.  First,  the  English  and 
Japanese  statements  made  by  the  subjects  were  broken  down  into  propositions;  these  were  categorized  as  to  either 
Correct  (C)  or  Incorrect  (I).  Correct  (C)  propositions  included  only  correct  statements  recalled  by  the  subjects. 
Incorrect  (I)  propositions  included  both  incomplete  and  false  statements,  however,  the  lacks  of  articles  were 
disregarded.  If  there  was  no  statement  for  a certain  proposition  in  the  statements,  it  was  categorized  as  Incorrect  (I). 
All  the  propositions  recalled  by  the  subjects  were  then  categorized  into  four  accuracy  groups:  1)  English-Correct  and 
Japanese-Correct  (EC-JC),  2)  English-Correct  and  Japanese-Incorrect  (EC-JI),  3)  English-Incorrect  and  Japanese- 
Correct  (EI-JC),  and  4)  English-Incorrect  and  Japanese-Incorrect  (EI-JI).  As  if  the  subjects  could  listen  the  audio 
statements  and  understand  the  meanings,  they  could  be  able  to  recall  both  in  English  and  in  Japanese.  In  the  four 
categories  of  accuracy,  the  number  of  propositions  which  were  categorized  as  EC-JC  was  calculated  for  every 
caption  presentation  timing  condition. 

Separate  one-way  analyses  of  variance  (ANOVA,  109  X 5 = subject  by  caption  presentation  timing)  were 
utilized  for  the  two  measures.  The  main  effect  of  caption  timing  was  checked  using  ANOVA  to  test  for  differences 
among  the  five  caption  timing  conditions. 

Results 


ANOVA  results  for  the  two  measures  are  given  below. 


Number  of  Recalled  Words 

The  ANOVA  results  showed  that  the  caption  presentation  timing  yielded  significant  effects  (F(4,l05)=15.3l, 
p<.001).  As  shown  in  Figure  1,  the  number  of  recalled  words  decreased  in  the  order  -1  Time  Lag  condition,  0 Time 
Lag  condition,  +1  condition,  -2  condition,  and  the  +2  condition. 

Since  significant  differences  were  found  among  the  five  conditions  at  the  .001  level,  the  LSD  multiple 
comparison  method  was  employed  to  determine  which  conditions  were  statistically  different.  The  LSD  multiple 
comparison  showed  that  the  -1  Time  Lag  condition  was  significantly  higher  than  the  other  conditions  at  least  at  the 
.05  level.  There  was  a significant  difference  in  the  number  of  recalled  words  between  the  -1  Time  Lag  condition 
and  the  0 Time  Lag  condition  at  the  .05  level.  In  the  second  highest  condition,  the  0 Time  Lag  condition,  the 
number  of  recalled  words  was  significantly  higher  than  those  were  in  the  +1,  -2,  and  +2  conditions  at  the  .01  level. 
No  significant  differences  were  found  in  the  number  of  recalled  words  among  the  lowest  three  conditions  - the  +l 
Time  Lag,  the  -2  Time  Lag,  and  the  +2  Time  Lag. 


-2  -1  +-0  +1  +2 
Time  Lag 


Figure  1 : Number  of  Recalled  Words  and  Accuracy  of  Recalled  Information  by  L2  Caption  Presentation  Timings 


Accuracy  of  Recalled  Information 


The  accuracy  of  recalled  information  showed  the  same  tendency  as  the  number  of  recalled  words  (Fig.  1);  that 
is,  it  decreased  in  the  order  -1  Time  Lag  condition,  0 Time  Lag  condition,  and  +1  Time  Lag  condition.  The  results 
of  ANOVA  showed  significant  difference  at  the  .05  level  among  the  caption  presentation  timing  conditions 
(F(4,105)=2.42,  p<.05).  Since  significant  differences  were  found  among  the  five  caption  presentation  timing  at  the 
.05  level,  the  LSD  multiple  comparison  method  was  employed  to  determine  which  conditions  were  statistically 
different.  The  LSD  multiple  comparison  showed  that  the  -1  Time  Lag  condition  was  significantly  higher  than  -2 
Time  Lag  condition  and  +2  Time  Lag  condition  at  the  .05  level. 

Summary  and  Discussion 

The  experiment  showed  that  the  number  of  the  recalled  words  and  the  accuracy  of  the  recalled  information 
were  high  when  the  captions  were  presented  one  second  prior  to  the  audio.  For  more  words  were  recalled  in  -1 
Time  Lag  condition  than  the  0 Time  Lag  condition,  it  seemed  that  the  subjects  could  manage  with  two  types  of 
information  - visual  and  verbal  - when  these  information  provided  with  time  lag  than  presented  simultaneously. 
When  the  captions  appear  on  the  screen  a second  before  the  audio,  the  students  could  concentrate  on  the  displayed 
text  which  led  to  better  recall.  The  information  gained  from  the  captions  may  have  functioned  as  prior  knowledge 
about  the  information  provided  through  audio.  It  might  have  helped  the  subjects  to  recognize  the  spoken  statements, 
with  which  most  Japanese  students  have  more  difficulties  than  with  understanding  written  statements.  The 
understanding  gained  from  the  written  statements  reinforced  the  recognition  of  the  spoken  statements.  In  the  +1 
Time  Lag  condition,  however,  failure  to  understand  the  spoken  statements  (which  is  highly  possible  for  Japanese 
students)  made  it  harder  for  the  students  to  focus  on  and  be  more  successful  at  reading  the  captions. 

In  the  0 Time  Lag  condition,  the  captions  and  audio  were  presented  simultaneously.  As  it  was  expected,  the 
viewer  had  to  read  the  visual  verbal  information  (caption)  and  listen  to  the  audio  verbal  information  at  the  same  time 
which  might  cause  overflow  of  learners’  processing  capacity.  Two  other  conditions  (-2  Time  Lag  and  +2  Time  Lag 
condition),  each  caption  was  overlapped  with  the  previous  or  the  forthcoming  audio  for  a second.  That  is,  they  have 
to  handle  two  different  verbal  contents  - one  from  audio  and  another  from  captions,  which  might  cause  the  confusion 
of  the  subjects  understanding  the  contents. 


Experiment  2 

The  effects  of  LI  caption  presentation  timing  was  investigated.  The  experiment  used  the  same  procedure  as 
Experiment  1. 

Participants 

Forty-three  Japanese  university  students  participated  in  Experiment  2.  They  were  freshmen  or  sophomores, 
their  average  of  TOEFL  score  was  444,  and  all  were  non-English  major  students.  They  were  randomly  assigned  to 
one  of  the  five  caption  timing  conditions. 

Instruments 

The  same  material  was  used.  Five  versions,  for  the  five  caption  timing  conditions  were  developed. 

Scoring  and  Data  Analysis 

The  TOEFL  scores  of  the  five  subject’  groups  were  statistically  different  (F(4,38)=2.77,  p<.05),  separate  one- 
way analyses  of  variance  (ANCOVA,  43  X 5 = subject  by  caption  presentation  timing)  were  utilized  for  the  two 
measures  of  the  number  of  recalled  words  and  the  accuracy  of  recalled  information.  The  main  effect  of  caption 
timing  was  checked  using  ANCOVA  to  examin  for  differences  among  five  caption  timing  conditions  in  the  recall  of 
EFL  university  students. 

Results 

Number  of  Recalled  Words 
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The  number  of  the  recalled  words  and  the  accuracy  of  recalled  information  by  the  five  caption  presentation 
timings  were  shown  in  Figure  2.  The  results  of  ANCOVA  presented  no  significant  effects  of  the  caption 
presentation  timing  (F(4,38)=2.18,  p=.09). 


Time  Lag  Time  Lag 

Figure  2 : Number  of  Recalled  Words  and  Accuracy  of  Recalled  Information  by  LI  Caption  Presentation  Timings 
Accuracy  of  Recalled  Information 

The  results  of  ANCOVA  showed  no  significant  difference  (F(4,43)=.91,  p=.47)  among  the  caption  presentation 
timing  conditions. 


Summary  and  Discussion 

In  this  experiment,  the  result  showed  the  LI  caption  presentation  timing  provided  no  effect  on  both  the  number 
of  recalled  words  and  the  accuracy  of  recalled  information  of  the  Japanese  university  students.  This  result  suggested 
the  Japanese  captions  information  and  the  English  audio  information  would  be  managed  without  overflowing 
individual’s  processing  capacity. 


Conclusions 

The  purpose  of  this  study  was  to  investigate  the  best  presentation  timing  of  LI  and  L2  captions.  When  the 
information  provided  in  a certain  period  of  time  exceeded  one’s  capacity,  there  would  have  a possibility  that  the 
captions  no  longer  work  as  an  effective  device  for  listening  comprehension.  In  this  study,  therefore,  the  effective 
use  of  timing  was  considered  to  diminish  the  information  presented  at  a time  and  to  maximize  the  caption  effects.  It 
was  expected  that  the  Japanese  EFL  students  had  a difficulty  to  read  the  L2  captions  presented  on  the  screen 
simultaneously  with  the  audio  at  the  spoken  speed.  However,  when  the  caption  presented  prior  to  the  corresponding 
audio  among  the  audio  dead  period,  they  could  read  the  English  captions  and  grasp  the  spoken  words  effectively. 
The  result  of  the  experiments  followed  this  expectation;  the  L2  captions  supplied  one  second  prior  to  the  audio 
contributed  better  listening  comprehension  of  the  Japanese  EFL  learners. 

It  was  also  predicted  when  the  LI  caption  which  provided  the  same  meaning  in  the  different  language 
presented  before  the  audio  would  be  more  helpful  because  the  viewers  could  translate  the  LI  caption  information 
and  prepare  for  the  forthcoming  L2  audio.  The  contradictory  result  was  found  from  the  experiment;  the  subjects’ 
listening  comprehension  was  independent  of  the  LI  caption  presentation  timing.  It  was  suggested  that  in  this 
experiment,  the  LI  captions  and  the  L2  audio  were  managed  without  exceeding  the  capacity  of  the  Japanese 
university  subjects  even  if  these  two  different  information  presented  simultaneously. 

The  result  of  LI  caption  presentation  timing  is  differed  from  the  preliminary  research  by  Yoshino  et  al.  (1999). 
It  was  unclear  what  causes  this  contradict  findings-  difficulty  levels  of  the  materials  or  difference  of  the  subjects 
including  the  processing  capacity  or  translation  ability.  If  the  Japanese  captions  were  always  handle  with  the 
English  audio  without  any  difficulty  by  Japanese  EFL  students,  the  reason  that  explains  the  lower  effects  of  LI 


1214 


caption  would  not  the  exceeding  of  viewers’  processing  capacity  by  an  additional  process  of  translation.  More 
research  to  conduct  the  LI  caption  presentation  timing  may  be  needed. 
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Abstract:  In  elementary  schools  there  is  increasing  use  of  computer-based  environments 
as  information  sources.  Information  is  hot-linked  creating  a hypertext  environment. 
CDROM  programs  are  closed  hypertext  environments  with  limited  links  between  nodes 
and  they  present  a unique  way  of  locating  information  with  no  print-based  equal.  As 
students  use  CDROM  programs  they  need  to  employ  search  strategies  relevant  to 
hypertext  and  the  program.  This  study  investigated  the  search  patterns  of  10  - 12  years 
old  students  using  a commercially  produced  encyclopedia-based  CDROM.  Each  group  of 
students  was  given  the  same  task,  which  was  set  in  context,  by  the  classroom  teacher. 
Students  using  the  encyclopedia  CDROM  set  up  mental  maps  and  predictions  based  on  a 
print  equivalent.  The  success  and  search  patterns  across  groups  varied  according  to  the 
search  strategy  employed  and  the  user  orientation  to  the  program  and  the  task.  The 
effectiveness  also  depended  on  the  group's  knowledge  of  the  topic,  knowledge  of  the 
program  and  their  mental  map  of  the  program’s  construction. 


Introduction 

Within  the  education  system  knowledge  is  gained  from  a variety  of  sources.  In  elementary  schools 
computer-based  technology  in  the  form  of  CDROMs  (Compact  Disk-  Read  Only  Memory)  and  the  Internet  are 
being  used  more  often  as  information  sources.  CDROMs  are  closed  hypertext  environments;  the  links  between 
nodes  of  information  is  limited.  While  the  Internet  can  be  considered  an  open  hypertext  environment,  as  there  are 
innumerable  links  to  be  made  to  a large  number  of  sites.  The  very  nature  of  these  resources  means  students  have  to 
access  information  from  a more  complex  multimedia  source. 

CDROM  programs  are  not  as  easily  accessible  as  other  resources  used  in  classrooms.  They  store  a large 
volume  of  information  and  can  also  be  a form  of  entertainment  - 'edutainment'  or  'infotainment'.  CDROM  programs 
are  non-linear,  unlike  books  (Slatin  1990;  Lawless  & Kulikowich  1998).  The  user  cannot  'see'  the  information 
contained  in  the  program  by  flicking  through  the  pages  or  turning  to  the  index  at  the  back.  Users  may  not  be  aware 
of  the  extent  of  the  information  or  that  they  are  taking  a wrong  path  (Edwards  & Hardman  1989;  Slatin  1990). 
Navigation  becomes  a critical  tool  in  a student's  learning.  As  a result,  CDROM  resources  create  a learning 
environment  dependent  on  students'  effective  information  retrieval  (search)  strategies. 

Learning  in  a hypertext  environment  also  involves  a high  degree  of  learner  control.  This  can  be  seen  as 
either  beneficial  to  students  or  a constraint.  Students  using  CD-ROMs  tend  to  wander  around,  interacting  with  the 
most  entertaining  parts  from  the  offerings  available  (Russell  1994).  This  wandering  can  be  construed  as  developing 
children's  metacognitive  skills  (McAleese  1989)  or  simply  as  an  inefficient  method  of  finding  information  with  the 
learner  expending  a lot  of  time  trying  to  find  their  way  around  (Russell  1994;  Marchioninni  1989). 

Studies  in  the  use  and  retrieval  of  information  from  multimedia  sources  have  found  that  students  have  little 
difficulty  using  and  finding  information  though  students'  search  strategies  and  their  methods  of  retrieving 
information  were  restricted  (Marchioninni  1989;  Oliver  & Perzylo  1994).  The  degree  of  success  or  effective  use  of 
the  CDROM  was  influenced  by  the  students'  understandings  of  the  system,  the  system  itself  and  students'  use  of 
search  strategies  and  retrieval  techniques.  However  the  issue  of  how  effective  students’  information  seeking 
methods  were  when  using  multimedia  needs  further  investigation  (Oliver  & Oliver  1996). 
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Users  of  CDROM  programs  employ  a variety  of  strategies  to  find  information.  McAleese  (1989)  presents  a 
number  of  studies  that  classify  these  strategies,  termed  ” bro\vsing”.  Canter  et  al.  (1985)  based  their  categories  on  the 
way  users  navigated  through  a database,  obtaining  a graphic  representation  of  their  pathways.  They  found  5 distinct 
search  strategies  - 

Scanning:  covering  a large  area  without  depth 
Browsing:  following  a path  until  a goal  is  achieved 
Searching:  striving  to  find  an  explicit  goal 
Exploring:  finding  out  the  extent  of  the  information  given 
Wandering:  purposeless  and  unstructured  globetrotting,  (p.  100) 

Lawless  & Kulikowich  (1996;  1998)  have  proposed  a different  classification.  Their  categories  incorporate 
the  users’  orientation  and  control  over  the  environment,  as  well  as  addressing  the  range  of  special  features  within  a 
program.  They  identified  3 types  of  common  navigational  profiles: 

Knowledge  seekers  (also  known  as  book  lovers):  those  readers  who  pursue  information  related 
to  the  content  of  the  hypertext;  navigate  to  screens  that  contain  information  needed  to  enhance 
comprehension;  strategic  in  selection  of  screens;  acquire  information  in  a systematic  manner. 

Feature  explorers  (also  known  as  resource  junkies):  those  readers  who  spend  a lot  off  time 
interacting  with  the  movies,  graphics  sound  & visual  effects;  invest  more  time  in  understanding 
how  the  hypertext  works  and  what  kind  of  screens  it  contains  than  gathering  important 
information. 

Apathetic  hypertext  users:  those  readers  do  not  appear  to  care  about  using  the  hypertext  either  to 
gather  information  or  to  explore  its  features;  characterized  by  short  intervals  interacting  with  the 
text;  navigation  paths  reveal  no  logical  order.  (Lawless  & Kulikowich,  1998,  p 52) 

The  provision  of  multimedia  resources  does  not  by  itself  increase  the  learning  outcomes  of  students  (Saga 
1992).  One  has  to  look  at  the  strategies  employed  by  students,  consider  the  construction  of  the  multimedia  programs 
and  the  teaching  and  learning  or  "training"  of  skills  required  in  the  classroom.  Knowledge  of  these  skills  and  the 
ability  to  work  in  a hypertext  CDROM  environment  may  also  have  implications  for  students'  effective  use  of  the 
Internet.  As  electronic  and  digital  texts  are  becoming  the  ‘new’  sources  of  information  in  education  and  the 
workplace,  the  development  of  students'  information  retrieval  skills  using  information  technologies  is  critical.  They 
will  assist  in  providing  the  background  for  the  essential  literacy  skills  for  the  information  age  - critical  literacy, 
visual  literacy  and  digital  literacy  skills  (Burbules  1997;  Lankshear  et  al.  1997). 

This  paper  will  present  preliminary  findings  about  the  search  strategies  used  by  10-12  year  old  students 
working  in  cooperative  groups  using  an  encyclopedia  CDROM  program  on  a set  task  based  on  the  class's  current 
theme. 

The  Investigation  of  Students^  Search  Strategies 

The  Program 

The  CDROM  Program  was  a commercially  produced  program  entitled  ENCARTA  96  (Microsoft  1993- 
1996).  It  was  part  of  an  Apple  Macintosh  hardware/software  package  purchased  by  the  school  in  1997.  The  program 
uses  an  encyclopedia  metaphor  to  frame  its  information  and  the  layout  of  the  screens.  However  unlike  a print-based 
encyclopedia  it  does  not  contain  a range  of  information  on  a huge  number  of  topics.  The  program  is  produced  in  the 
United  States  and  contains  limited  information  on  Australia,  in  particular  Australian  history. 

The  Students 

Students  were  aged  between  10  and  12  years.  They  all  had  used  the  program  before  in  a previous  unit  of 
work  on  "Rainforests".  The  majority  of  the  students  had  access  to  a computer  outside  the  school.  The  class  teacher 
allocated  the  thirteen  students,  who  took  part,  to  a group.  Each  member  was  assigned  according  to  their  different 
role:  recorder,  reader,  reporter  and  gopher.  Groups  ranged  in  size  from  3 to  5 students.  Four  groups  were  observed 
using  the  program. 

The  Task 

Students  were  asked  to  obtain  information  on  the  lifestyle  of  Australian  Aborigines  prior  to  European 
Settlement.  The  task  was  introduced  to  students  by  the  class  teacher  as  the  initial  session  of  a historical  theme  on 
Early  Australian  history.  The  teacher  scaffolded  students'  investigation  by  having  a discussion  with  the  class  about 
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what  areas  to  investigate,  writing  these  as  headings  and  sub-headings  on  the  blackboard.  This  provided  students  with 
possible  key  words  to  use  as  students  searched  any  of  the  resources. 

Methodology 

A naturalistic  form  of  data  gathering  was  employed  (Oliver  & Perzylo  1994).  Field  notes  were  kept  as  well 
as  a video  recording  of  each  session  to  document  the  pathways  taken  by  each  group.  The  video-recordings  provided 
detailed  information  about  the  interactions  and  pathways  of  each  group.  This  has  become  a useful  data  collection 
technique  for  computer-based  information-seeking  studies  (Oliver  & Perzylo  1994;  Edwards  & Hardman  1989).  The 
class  teacher  and  students  were  interviewed  to  obtain  information  about  students’  sociocultural  and  computing 
background.  During  the  session,  students  were  asked  to  articulate  the  reasons  they  chose  specific  paths  or  navigation 
tools  - a procedure  adapted  from  the  Think-aloud’  technique  (Afflerbach  & Johnson  1984).  There  was  no  time 
constraint  and  each  group  pursued  their  searches  until  satisfied  they  had  found  all  they  could  from  the  program. 


Results 

The  breadth  of  each  search,  was  obtained  by  tabulating  the  number  of  different  nodes  visited  (NV),  the 
number  of  nodes  visited  and  the  most  frequently  visited  nodes  (Tab.  1).  In  addition,  the  researcher  assessed  the 
quality  of  the  searches  in  relation  to  location  of  relevant  information  and  the  types  of  sources  used  to  gather 
information.  Relevant  information  was  defined  as  any  source  (section  of  written  text,  image,  caption  for  image, 
sound)  containing  either  specific  information  on  Aboriginal  people  before  the  Europeans,  as  well  as  general 
information  about  Aboriginal  Art  or  Culture,  for  example  Didjeridu  (Tab.  2). 


No.  of  different  nodes 
visited  (NV) 

No.  of  total  nodes 
visited  (NS) 

NV/NS: 

% 

Most  frequently  visited  nodes 

Group  1 

5 

8 

62.5 

Find:  3 
Aborigines;  2 

Group  2 

11 

17 

64.7 

Main  Menu:  4 
Find:  2 
Timeline:  2 

Group  3 

12 

23 

52.1 

Find:  3 
Print:  3 

Find  a place:  4 
Tribe:  2 

Map  of  Northern  Territory:  2 
Map  of  Azerbai  jan:  2 

Group  4 

30 

63 

47.6 

Find:  18 
Moari:  5 
Aborigines:  5 
Weapons:  5 
Australia:  3 
Flint:  2 

Table  1:  Nodes  (Screens)  Visited 


Number 

located 

% located  * 

No.  of  sources  : 
Information 
discussed 

No.  of  sources: 
notes  taken 

Type  of  source 

Group  1 

3 

21.4 

2 

sound;  caption 

1 

caption 

sound 

caption 

Group  2 

2 

14.2 

1 

sound 

0 

sound 

Group  3 

3 

21.4 

3 

image;  sound; 
caption 

2 

sound;  caption 

image 

sound 

caption 

Group  4 

5 

35.7 

0 

0 

NA 

* Total  number  of  relevant  information  sources  =14 

Table  2:  Location  of  Relevant  Information 
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Each  group's  session  was  graphically  represented  or  "notated".  The  search  patterns  were  compared  to  the 
visual  and  descriptive  categories  of  Canter  et  al  (1985)  and  the  categories  of  Lawless  & Kulikowich  (1996;  1998) 
(see  Tab,  3), 


Group  1 

Group  2 

Group  3 

Group  4 

Canter  et  al 

♦Wandering 

Browsing 

Scanning 

Browsing 

Lawless  & 

Kulikowich 

Apathetic  Hypertext 
users 

Feature  Explorers 

Knowledge  seekers 

Feature  Explorers 

♦This  session  was  extremely  short  with  only  8 screens  being  visited.  Hence,  comparison  with  the  written  descriptive  instead  of  the  graphic 
representation  was  used. 

Table  3:  Search  Strategies  and  User  Orientation 


Group  1 

This  group  visited  the  least  number  of  nodes  within  the  program  (Tab,  1)  and  spent  the  least  amount  of  time 
searching  for  information.  They  found  relevant  information  during  their  initial  search  using  the  FIND  tool.  The 
information  was  obtained  from  the  caption  under  the  'Aboriginal  Music  of  Australia’  image  and  sound  file  (Tab,  2) 
on  the  'Aborigines'  screen.  When  unsuccessful  in  further  searches  using  the  same  technique  they  chose  to  finish  their 
search  session.  The  diagrammatic  representation  of  their  search  could  be  equated  with  Wandering  but  the  group 
visited  so  few  screens  (nodes)  and  this  categorization  is  inconclusive.  They  did  exhibit  characteristics  of  apathetic 
hypertext  users. 

Group  2 

Group  2 chose  to  investigate  each  section  of  the  program  as  listed  on  the  Main  Menu  screen.  They  were  the 
only  group  that  did  not  begin  their  search  using  the  FIND  tool,  choosing  the  LOOK  & LISTEN  section  for  their 
initial  search.  They  found  useful  information  but  did  not  take  notes  on  any  of  this  information  for  the  task,  preferring 
to  talk  about  the  sound  file  accompanying  the  'Aboriginal  Music  of  Australia'  image  in  the  Media  Gallery,  accessed 
via  the  LOOK  & LISTEN  section.  They  visited  a high  number  of  different  nodes,  with  a medium  NV/NS  score.  The 
diagrammatic  of  their  search  and  the  medium  NV/NS  score  are  features  of  Browsing.  Their  user  orientation  can  be 
classified  as  Feature  Explorers. 

Group  3 

Group  3 visited  a variety  of  nodes,  revisiting  six  (half  of  the  number  of  different  nodes  visited)  more  than 
once  (Tab.  1).  They  located  useful  information  on  their  initial  search  using  the  FIND  tool  and  were  unsuccessful 
with  subsequent  searches  using  the  same  technique.  The  information  extracted  was  from  the  image,  caption  and 
sound  file  (Tab.  2)  entitles  'Aboriginal  Music  of  Australia’  on  the  'Aborigines’  screen.  They  also  located  information 
about  Aboriginal  populations,  which  they  deemed  to  be  important  but  did  not  infer  that  this  information  was  recent 
not  historical  data.  The  diagrammatic  of  their  search  was  similar  to  Scanning.  This  group  was  the  only  one  that 
exhibited  behavior  similar  to  Knowledge  Seekers.  Towards  the  end  they  moved  into  Feature  Explorer  behaviors. 

Group  4 

Group  4 visited  the  largest  number  of  nodes  of  all  the  groups.  They  revisited  5 nodes  frequently,  these 
being  the  core  or  base  nodes  (Tab.  1),  The  FIND  tool  was  revisited  the  most  (18  times)  as  they  used  this  node  to 
frame  their  search.  Hence  the  lower  NV/NS  ratio  in  comparison  to  the  other  groups.  Although  the  group  visited 
relevant  nodes  they  did  not  write  any  useful  information  from  any  of  these  texts,  nor  did  they  talk  about  this 
information.  They  believed  the  information  about  the  population  of  Aborigines  was  important  and  recorded  notes 
from  this.  Unique  was  their  investigation  of  each  section  of  a node  (Weapons)  which  contained  only  a list  of 
hotlinked  words.  This  group  visited  2 screens  not  found  by  any  of  the  other  groups:  Didgeridu’  and  Boomerang. 
However,  they  did  not  explore  the  "Related  Articles"  sections  nor  consider  any  of  the  information  relevant.  This 
may  be  due  to  locating  them  at  the  end  of  the  session.  The  group  lost  focus  and  interest  due  to  lack  of  success  with 
their  searches.  They  used  hotlinked  words  more  often  than  the  other  groups.  The  diagrammatic  of  their  search 
demonstrated  similarities  to  Browsing,  Their  orientation  was  equivalent  to  Feature  Explorers. 

No  group  located  the  written  text  with  specific  information  about  Aboriginal  people  prior  to  European 
settlement,  which  was  on  the  'Australia'  node.  Students  did  not  use  the  'View:  Outline'  strategy  to  locate  this 


'This  is  the  spelling  used  in  ENCARTA96  (Microsoft,  1993-1996).  Students  used  'didgeridoo'. 


information,  preferring  to  scroll  down  the  text  box  when  in  the  'Australia'  node.  As  this  node  is  extremely  long,  this 
was  not  an  effective  method  for  locating  this  section  of  text.  Students  did  not  understand  how  to  use  this  section  of 
the  program.  This  is  evidenced  by  3 of  the  four  groups  locating  this  section  of  the  screen,  'reading'  the  options  but 
not  clicking  on  any  of  the  alternative  views. 


Discussion 

The  information  the  groups  found  was  obtained  from  the  sound  and  visual  texts  of  the  program.  But  this 
was  not  consistent  for  all  groups  and  all  relevant  information  was  not  extracted.  This  reinforces  the  need  for  students 
to  be  given  experiences  on  how  to  extract  information  from  non-verbal  texts  (Oliver  & Perzylo  1994).  The  written 
text  was  not  helpful,  and  for  some  proved  to  be  a distraction  as  they  lost  sight  of  the  objective  of  the  task  and  were 
consumed  by  the  special  features  of  the  program  (Russell  1 994). 

Effective  search  strategies  rely  heavily  on  the  background  knowledge  of  the  topic  by  the  searcher  whether 
they  use  print-based  or  computer-based  texts.  In  a computer-based  environment,  this  knowledge  becomes  a crucial 
aspect  if  the  user  is  to  locate  relevant  information  using  the  search  tools  of  the  program  (Lawless  & Kulikowich 
1998).  Group  4 located  two  information  nodes  (boomerang  and  didjeridu)  at  the  end  of  their  search  but  did  not 
explore  any  of  the  related  articles  (key  words),  take  notes  or  consider  them  important.  The  information  did  not 
specifically  mention  early  Aboriginals  or  pre-European  settlement. 

The  most  relevant  information  was  to  be  found  in  three  sections  of  written  text  on  the  ‘Australia’  node. 
However,  this  screen  contained  a very  long  piece  of  text  and  students  did  not  use  'View  Outline'  as  a strategy  for 
locating  information  within  the  screen.  Students  obviously  did  not  know  about  this  organisational  feature  of  the 
program.  An  orientation  for  the  user  to  the  program,  its  sections,  ideology,  authorship,  limits  of  information  need  to 
be  covered  prior  to  use.  During  the  use  of  the  programs,  the  imperative  is  to  provide  scaffolding  for  students,  for 
example  guided  question  proforma  or  key  words  guide  (Zammit  1 999). 

One  of  the  reasons  for  the  use  of  the  'FIND'  tool  and  lack  of  use  of  the  'View  Outline'  tool  can  be  explained 
by  the  students'  orientation  to  program.  The  first  strategy  is  one  used  primarily  in  print-based  encyclopedias;  the 
latter  is  an  example  from  a hypertext  environment.  Students'  mental  model  of  how  to  access  an  encyclopedia,  what 
an  encyclopedia  should  and  would  contain  was  based  on  print  equivalents.  As  a result  they  did  not  locate  all  the 
information  in  the  program.  The  construction  of  the  program  influences  the  strategies  chosen  by  the  user. 

Even  though  graphical  representation  of  navigation  patterns  do  not  seem  to  'identify  individual  differences 
with  the  same  precision  as  cluster  analysis'  (Lawless  & Kulikowich  1998:  397),  they  do  provide  a visual  overview  of 
a distinguishable  search  pattern.  When  Canter  et  al.’s  (1985)  categories  of  browsing  are  used  in  conjunction  with 
Lawless  & Kulikowich's  (1998;  1996)  types  of  users,  one  can  gain  an  indication  as  to  why  some  students  are  more 
successful  than  others  at  locating  information  using  closed  hypertext  CDROM  programs. 

Those  students,  who  were  Browsing,  were  also  Feature  Explorers  and  they  were  absorbed  by  the  program. 
These  students  did  not  take  notice  of  relevant  information  nor  appear  to  know  what  was  relevant.  It  is  also  possible 
they  did  not  have  the  skills  to  extract  the  information  when  it  was  located  or  how  to  take  notes  As  a result  they  'lost' 
track  of  the  task,  being  more  interested  in  using  the  program’s  tools.  Those  students  who  were  Knowledge  Seekers 
used  Scanning  and  covered  a wide  area  of  the  program.  They  discussed  information  located,  assessed  the  value  of 
the  information  and  took  notes  from  a range  of  sources.  This  group  gained  some  information  for  the  task  they  were 
set.  For  Group  1 (the  Apathetic  Hypertext  Users)  the  data  is  not  as  conclusive,  given  the  short  amount  of  time  they 
spent  interacting  with  the  program.  They  went  in,  got  some  information  and  left.  Their  success  was  related  to  the 
quality  of  the  moves  made  not  to  the  quantity.  (Marchioninni  1 989) 

From  this  information,  two  searcher  orientations  are  proposed: 

Voyeurs-  they  are  like  tourists  in  a new  country.  They  have  difficulty  reading  the  visual  and 
written  signs;  locate  sources  but  have  difficulty  understanding  what  is  relevant;  have  limited 
discussion  of  content;  and  are  more  interested  in  clicking  and  acting  on  the  hypertext 
environment  than  negotiating  meaning.  They  are  not  very  successful  with  their  search;  they  may 
locate  the  information  but  not  recognize  its  value  or  use  it  for  notes. 

Investigators-  they  are  like  detectives  looking  for  clues.  They  are  focused  in  their  exploration; 
cover  a wide  area  within  the  program;  discuss  the  value  of  the  information  located;  negotiate 
meaning  with  the  program;  and  use  a range  of  sources  for  their  note  taking.  They  are  successful 
with  their  search,  locating  information  relevant  to  their  search. 

Just  as  two  people  negotiate  meaning  when  they  speak  to  each  other,  the  user  negotiates  meanings  with  the 
computer  as  the  other  interactant.  These  types  of  negotiation  or  engagement  are  a more  refined  ways  to  consider  the 


search  strategies  of  users  and  may  be  more  useful  than  the  broad  categories  and  types  of  users.  Further  research  into 
these  orientations  with  consideration  of  the  types  of  engagement  and  the  influence  on  learning  in  a hypertext 
environment  is  required.  Perhaps  certain  types  of  users  have  a preference  for  certain  types  of  interaction  and  avoid 
others.  Some  students  may  be  better  negotiators  in  a hypertext  ’conversation’  similar  to  those  who  are  better  talkers. 
In  addition  to  this  area,  there  is  a need  for  more  work  on  information  retrieval  using  a range  of  commercial  products 
in  classrooms. 


Conclusion 

The  engagement  of  young  students  during  information  seeking  tasks  is  influenced  by  their  orientation  to  the 
program  and  their  knowledge  of  the  topic.  These  in  turn  influences  the  search  strategies  students  select  when  given  a 
task  and  hence  the  effectiveness  of  their  search.  Students  are  drawn  to  the  image  and  sound  texts  of  a program  but 
are  not  necessarily  skilled  in  extracting  information  from  these.  In  addition  it  is  the  opaqueness  of  the  extent  of 
programs  and  their  construction  that  constrains  students’  effective  searching.  How  to  open  up  these  resources  for 
young  children  as  effective  learning  environments  needs  conscientious  efforts  by  educators. 
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Abstract:  On-line  learning  for  undergraduate  students  is  being  embraced  by  higher  education 
institutions  as  they  adapt  and  react  to  the  changing  circumstances  of  student  expectations,  societal 
demands  and  available  funding.  Students'  perceptions  of  learning  on-line  must  be  considered  if 
their  learning  needs  are  to  be  met  and  best  practice  is  to  be  achieved.  Over  the  1998-1999  period, 
an  undergraduate  teacher  education  subject,  convened  by  the  first  author  has  included  an  on-line 
component  in  the  delivery  and  assessment.  Rather  than  replacing  existing  practices  with  on-line 
teaching  and  learning,  lecturers  working  in  this  subject  used  mixed  mode  delivery  to  develop 
students’  understandings  and  skills.  They  encouraged  students*  independent  learning  through  on- 
line modules,  computer  based  assessment  tasks  and  presentation  of  subject  content  via  the  Internet, 
lectures  and  tutorials.  From  the  students'  evaluations  of  the  subject,  the  mixed  mode  of  delivery 
including  independent  learning,  the  use  of  technology,  and  on-line  tasks  were  fevorably  received 
and  had  positive  affects  on  the  students'  learning  in  the  subject. 


Changing  Practices  and  Relationships  in  Universities 

The  advent  of  Information  Technologies  (IT)  in  affluent  countries  has  changed,  fundamentally,  the  nature 
of  communication  and  learning  within  those  societies.  As  a result,  society  is  moving  away  from  a print  based  to  an 
information  technology  based  society  (Spender  1995;  Negroponte  1995).  The  medium  of  exchange  in  society  has 
moved  from  physical  things  represented  as  atoms  to  network  based  information  represented  as  electronic  bits. 
Higher  education  is  especially  affected  by  these  changes  and  those  involved  in  this  enterprise  must  adapt  to  take 
advantage  of  the  opportunities  offered  by  the  emerging  technologies. 

Universities  will  have  “to  change  their  purposes  and  their  practices.  Scholarship,  knowledge,  research  and 
teaching  are  significantly  different  when  done  electronically”  (Spender  1995,  p.  xxiii^  Universities  also  operate 
within  a political  and  economic  climate.  There  is  an  increasing  need  for  university  qualifications  in  order  to  gain 
paid  employment  and  a need  for  life  long  learning  in  order  to  maintain  credentials  in  a rapidly  changing  employment 
environment.  As  a result,  the  numbers  of  students  attending  university  has  increased  dramatically.  These  students 
come  increasingly  from  a diverse  population  and  have  changing  aspirations  (Le  Grew  and  Calvert  1998). 

Within  Australia,  relatively  few  universities  are  “as  yet  taking  a whole-of-institution  approach  to  . . . flexible 
delivery”  (Latchem  and  Moran  1998,  p.67).  Lewis  and  Merton  quoted  in  Lewis  (1998)  suggest  that  the  application 
of  technology  to  flexible  delivery  of  instruction  in  universities  has  been  “patchy”  in  that  it  has  met  with  mixed 
results.  This  is  especially  true  in  the  field  of  Initial  Teacher  Education  (Hacker  and  Sova  1998). 

One  important  dimension  for  university  based  flexible  delivery  is  a move  “from  a teacher-dominated, 
transmission-based  process  to  a learner  centred,  constructivist  process”  (Latchem  and  Moran  1998,  p.67)  with  the 
use  technology  to  maintain  contact  between  students  and  tutors.  Technology  in  flexible  delivery  can  support 
students  by  providing; 

“Information  on  the  curriculum, 

Recognition  for  existing  achievements  and  advice  on  appropriate  learning  routes, 

Flexible  access  to  resources,  facilities,  and  program  content, 

Opportunities  to  practice  and  apply  learning.”  (Lewis  1998,  p.27). 
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Innovation  and  change  in  practice  comes  about  from  what  has  gone  before:  it  is  not  entirely  ‘new’.  Branch 
and  Tonkin’s  (1997)  case  study  of  the  trial  of  multi-campus  video  on  demand  illustrates  this  principle.  New  learning 
environments  grow  out  of  old  learning  infrastructure  when  innovation  has  an  opportunity  to  flourish. 


Methodological  Principles 

Evaluation  is  a continuous  process  that  is  an  interaction  between  the  internal  and  external  learning 
processes  of  students  and  teachers.  The  purpose  of  evaluation  is  to  determine  the  value  of  a particular  learning 
experience  for  the  students  who  are  engaged  in  it.  There  are  two  aspects  to  developing  such  understanding.  The  first 
is  to  identify  what  is  not  working  for  a particular  student  or  group  of  students.  These  elements  of  the  experience 
need  to  be  changed  to  create  the  conditions  for  positive  outcome  for  learners.  The  second  aspect  is  to  find  and  retain 
what  is  helping  students  to  learn  what  it  is  they  need  to  know. 

Learning  is  an  internal  process  for  which  learners  are  individually  responsible.  Teaching  is  an  external 
process  in  which  teachers  are  responsible  to  their  students.  However  teachers  must  collect  information  from  their 
students  (external  process)  and  reflect  on  the  implications  of  this  information  (internal  process)  for  the  learning 
environments  that  they  have  provided  for  those  students.  Kirkpatrick,  quoted  in  Alexander,  McKenzie  and 
Geissinger  (1998),  provides  a four  level  framework  for  organising  the  collection  of  evaluation  information.  These 
principles  have  guided  the  examination  of  the  subject  that  is  the  focus  of  this  paper.  The  levels  parallel  the  integrated 
approach  suggested  by  Alexander  and  Hedberg  (1994).  The  levels  are: 

“reaction  to  the  innovation; 

achievement  of  learning  objectives; 

transfer  of  new  skills  to  the  job  or  task; 

impact  on  the  organisation.”  (Alexander  et.  al.  1998  p.254). 


The  Subject  - Curriculum  Studies  6:  English  and  Science  & Technology 

A site  for  the  subject  Curriculum  Studies  6:  English  and  Science  & Technology  (CS6)  was  developed 
containing  on-line  Internet  resources  organised  around  nodes,  which  included  information  and  activities  (Zammit, 
Nanlohy  & Corrigan  1998;  1999).  The  nodes  were  Your  Concept;  Factual  Genres;  Learning  and  Teaching; 
Programming  and  Planning;  Assignments  and  Tasks.  The  website  demonstrated  the  integration  of  the  two  subject 
areas  via  hotlinks  between  content  nodes.  Each  node  was  broken  into  sections;  each  of  these  varied  in  length 
depending  on  the  content.  Where  a section  was  a long  page,  the  parts  were  identified  at  the  top  and  internal 
navigation  used.  It  was  attempted  to  keep  the  amount  of  information  to  a screen-size,  but  this  was  not  always 
successful  and  different  computers  varied  the  display  of  the  information. 

The  subject  presentation  consisted  of  three  lectures;  eight  x two-hour  tutorials  in  both  the  English  and  the 
Science  & Technology  strands  interspersed  with  five  independent  study  weeks.  On-line  tasks  were  developed  for 
students  to  complete  during  independent  weeks.  In  1998  the  Science  tasks  were  compulsory,  the  English  tasks  were 
not  and  no  grade  was  attached  to  the  completion  of  tasks.  In  response  to  students'  evaluations  in  1 998  and  further 
investigation  of  students'  completion  of  on-line  tasks  (see  Alexander,  McKenzie  & Gessinger  1998,  p.234)  the  1999 
tasks  and  the  web  page  construction  assessment  item  were  refined. 

The  tasks  were  intended  to  assist  students  in  the  completion  of  their  final  assignment  - an  integrated 
program  for  a grade  of  their  choice,  and  covered  topics  not  addressed  during  lectures  or  tutorials.  The  Science  tasks 
extended  students  conceptual  development  within  a selected  concept  area  and  the  use  of  outcomes  in  Science.  The 
English  tasks  were  designed  to  extend  students  knowledge  about  language,  factual  genres,  information  skills  and 
critical  literacy.  In  1 999,  one  of  the  tasks  was  the  annotation  of  a text  written  in  a self-selected  genre  relevant  to  the 
concept  being  investigated  by  students.  Students  had  to  write  the  text,  annotate  the  text  with  the  structure  and  2 key 
language  features  (grammatical).  In  1998,  this  task  was  part  of  the  web  page  assignment.  The  number  of  overall  on- 
line tasks  decreased  from  eight  to  six.  All  six  on-line  tasks  were  compulsory  and  a pass/fail  grade  assigned  to  their 
completion.  Students  were  asked  to  submit  the  tasks  via  email. 


The  Students 


All  the  students  (1998:  153  students;  1999:  126  students)  were  in  the  second  year  of  their  three-year 
Bachelor  of  Teaching  (Primary)  degree.  Students  voluntarily  participated  in  the  evaluation  and  hence  the  numbers  of 
participants  varied  across  evaluation  forms.  Most  students’  previous  computing  experience  was  limited  to  word 
processing.  They  had  limited  knowledge  and  experience  of  the  Internet.  The  1999  cohort  reported  an  increased 
familiarity  with  the  Internet.  In  a preliminary  survey,  the  students’  initially  rated  using  information  and  receiving 
instructions  from  computers  as  one  of  their  least  preferred  mediums. 


Students  Opinion  of  Learning  in  a Mixed  Mode  of  Delivery  and  On-line 

In  both  years,  students  mid  semester  completed  an  open-ended  evaluation.  The  end  of  semester  evaluation 
instrument  combined  Lickert  scale  responses  and  open-ended  written  replies.  Students  also  completed  a pre-subject 
questionnaire  to  ascertain  their  use,  comfort  and  knowledge  of  computers  and  the  Internet. 

The  subject’s  web  site 

The  subject's  site  was  well  received  by  the  students  in  both  years  (Tab.  1).  In  1999,  however  it  was  rated 
more  highly  than  in  1998.  This  may  be  because  students  have  become  more  comfortable  with  the  Internet  and 
learning  with  computers  than  they  perceived  themselves  to  be  in  the  preliminary  survey.  It  may  also  be  because 
students  had  to  access  the  site  to  locate  tasks,  to  complete  tasks  and  find  information  to  assist  with  tasks  and 
assignments.  A few  students  completed  the  subject  on-line  and  did  not  attend  classes.  These  were  mainly  students 
who  were  repeating  the  subject. 


1998 

1999 

Extremely  helpful 

ii> 

21 

Quite  helpful 

38 

51 

Adequate 

32 

23 

A little 

17 

3 

Not  at  all 

2 

2 

Table  1:  Rating  of  subject  website:  end  of  semester 


1998 

1999 

A lot 

13 

15 

Often 

38 

38 

Sometimes 

29 

26 

A little 

14 

18 

Never 

0 

3 

No  comment 

6 

0 

Table  2:  Frequency  of  use  of  Website:  end  of  semester 


The  frequency  of  use  by  students  did  not  vary  over  the  two  years  (Tab.  2).  This  is  an  interesting  result  given 
that  the  1999  English  and  Science  on-line  tasks  were  compulsory  and  the  content  of  these  was  not  covered  in 
tutorials  or  lectures.  This  required  more  independent  learning  by  students  but  the  results  suggest  that  they  used  the 
site  and  the  information  more  effectively  than  the  1998  cohort  did.  It  is  also  important  to  note  that  3%  did  not  use 
the  site  at  all.  The  completion  of  on-line  tasks  could  be  done  with  a partner,  in  a small  group  or  individually.  These 
people  may  have  opted  to  work  in  with  other  students,  who  did  the  actual  'looking  up'  and  using  of  the  site,  while 
they  assisted  in  the  completion  of  the  tasks.  They  may  also  have  been  able  to  obtain  enough  information  about  tasks 
and  content  from  students,  books,  journals  or  other  sources. 

In  both  years,  the  site  was  mostly  used  for  accessing  information.  It  was  also  used  to  get  help  with 
assignments,  to  access  the  lecturers  and  obtain  clarification  about  the  content. 


On-line  Tasks 

Between  1998  and  1999  there  has  been  a significant  change  in  students  opinion  of  the  on-line  tasks  (Tab.  3).  The  on- 
line tasks  in  1999  were  more  favorably  rated  than  in  1998,  in  relation  to  increasing  students'  learning  in  the  subject. 
The  most  significant  change  occurred  for  the  English  on-line  tasks.  This  can  be  'related'  to  the  compulsory 
completion  of  these  tasks  in  1999.  However  it  may  also  provide  a better  reflection  on  the  quality  of  the  on-line  tasks 
as  all  students  completed  them  and  could  evaluate  them  more  effectively  than  the  1998  cohort. 


* In  all  tables  the  numbers  are  in  %,  rounded  to  nearest  whole. 
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RPcrrOPYAVAll  ARI  F 


1998  - 

1998  - 

1999- 

1999- 

voluntary: 

compulsory: 

compulsory  + 

compulsory  + 

English 

Science 

graded:  English 

graded:  Science 

Extremely  helpful 

0 

0 

8 

5 

Quite  helpful 

19 

31 

35 

30 

Adequate 

21 

22 

41 

38 

A little 

25 

24 

13 

21 

Not  at  all 

35 

22 

3 

7 

Table  3:  Rating  of  on-line  tasks:  end  of  semester 


In  1998,  students  who  did  not  attempt  the  English  on-line  tasks  may  have  rated  them  of  no  or  little  help  in 
their  learning  because  they  didn't  have  to  complete  them.  Students  in  1999  also  rated  the  Science  on-line  tasks 
higher  than  in  1998.  This  may  be  due  to  the  decreased  number  of  on-line  tasks  in  Science.  It  may  also  reflect  on  the 
different  learning  styles,  perceptions  of  the  1999  cohort  of  students,  and  their  increasing  familiarity  and  comfort  in 
learning  independently  and  in  learning  with  this  medium  (Tab.  4). 

Presentation  Mode 


1998 

1999 

Excellent 

4 

2 

Good 

25 

56 

Some  value 

21 

8 

Interesting 

2 

3 

Boring/unhelpful 

2 

3 

Other 

- 

7 

No  comment 

41 

20 

1998 

1999 

Extremely  helpful 

32 

33 

Quite  helpful 

28 

46 

Adequate 

19 

13 

A little 

9 

3 

Not  at  all 

11 

5 

Table  4:  Influence  of  Independent  study  weeks  on  Table  5:  Presentation  mode:  end  of  semester 

learning  in  subject:  end  of  semester 

Students  found  the  presentation  mode  of  the  subject  useful  to  their  learning  (Tab.  4 & Tab.  5).  In  1998  this 
subject  was  the  only  undergraduate  subject  with  a different  pattern  of  attendance  and  on-line  tasks.  For  1998  cohort 
it  was  a unique  and  challenging  experience.  In  1999,  the  subject  is  only  one  of  a number  of  undergraduate  subjects 
with  unusual  attendance  patterns,  combining  a range  of  delivery  modes.  Hence  1999  students  may  have  had 
previous  experience  or  other  subjects  where  they  had  to  keep  track  of  their  attendance  for  other  subjects  as  well  as 
Curriculum  Studies  6.  When  read  in  conjunction  with  Table  4,  the  results  also  show  that  students  can  learn  and 
appreciate  learning  from  a range  of  mediums  and  modes,  including  an  on-line  medium  and  an  independent  mode. 


Using  Computers 

Students  developed  a lot  of  confidence  in  using  computers  as  a result  of  this  subject  (Tab.  6).  Their 
experiences  working  with  computers  were  commented  on  favorably  mid-semester  and  at  the  end  of  semester  (Tab. 
7).  Again  the  difference  between  1998  and  1999  cohorts'  negative  opinion  may  be  due  to  the  changing  nature  of  the 
student  population  in  regards  to  use  and  exposure  to  computers.  The  more  negative  comments  in  1999  about  ‘using 
computers"  were  related  to  lack  access. 


1998 

1999 

Positive  experience/  good  or  great 

51 

53 

Some  value/  getting  better/more  confident 

15 

33 

Negative  experience/  difficult  or  hard 

28 

8 

Other 

0 

5 

No  comment 

6 

2 

Table  6:  Working  with  computers:  end  of  semester 


Technical  difficulties  and  access  to  computers  were  the  most  frequently  reported  problems  students 
encountered.  The  authoring  software,  although  different  in  each  year,  was  a problem  and  the  problems  with  email 


preferences  in  the  computing  laboratories  meant  students  in  1 999  couldn’t  email  directly  from  the  Internet  browser. 
This  required  students  to  print  out  or  send  tasks  as  attachments  using  their  personal  email  accounts. 

Students’  opinion  of  the  construction  of  a web  page  assignment  changed  over  the  2 years  (see  Tab.  7).  The 
1999  cohort  commented  most  favorably  about  this  assignment  in  the  formative  mid-semester  evaluation.  There  is 
also  a decrease  in  the  negative  comments  about  this  assignment  in  1999.  Students’  comfort  at  using  computers  may 
account  for  this.  Also  students  can  see  the  benefit  in  learning  about  web  page  construction  as  future  educators  of 
children  living  in  a digital  world. 


Positive  Comments 

Negative  Comments 

1998 

1999 

1998 

1999 

Web  page  construction 

10 

24 

15 

10 

Using  computers 

23 

22 

0 

11 

Course  structure 

20 

13 

10 

9 

Face-to-face  Content 

23 

13 

34 

37 

Other 

0 

10 

0 

7 

On-line  tasks 

2 

3 

13 

7 

Technical  problems 

0 

0 

11 

14 

CS6  website 

4 

0 

0 

0 

Assignments 

10 

0 

6 

0 

Table  7:  Mid-semester  evaluation 


Face-to-Face  Content 

The  largest  number  of  mid-semester  negative  comments  related  to  the  face-to-face  content  (Tab.  7).  What 
was  remarkable  was  one  of  the  two  content  areas  received  the  majority  of  unfavorable  comments  over  both  years. 
Even  with  changes  made  to  the  Science  subject  content,  it  was  still  not  well  received. 


Discussion 

Two  trends  are  clear  from  the  CS6  initiative.  The  first  is  that  students  are  increasingly  comfortable  with  on- 
line delivery  of  aspects  of  undergraduate  subjects.  The  second  is  that,  the  on-line  component  of  a subject  can 
become  increasingly  supportive  of  students’  learning  given  appropriate  evaluation  and  subject  design  changes. 

Simply  participating  in  a university  course  increases  students’  experiences  with  Information  Technologies 
(IT).  They  need  to  use  IT  for  research,  information  collection  and  production  of  assignments.  In  this  subject,  the  use 
of  IT  was  required  for  participation  and  for  the  production  of  assessment  items.  A third  dimension  was  added  in  that 
this  subject  also  studied  the  place  of  IT  in  children’s  learning.  Thus  the  use  of  the  on-line  resources  were  linked  to 
the  students’  future  professional  roles.  Significant  assistance  was  given  to  students  by  their  peers,  by  the  lecturers 
and  by  the  technical  staff  of  the  computing  center. 

It  is  important  to  note  that  comfort  increased  in  the  second  year  of  the  subject  when  there  was  a grade 
attached  to  the  completion  of  on-line  tasks.  This  is  a form  of  compulsion  and  as  such  needs  to  be  backed  by  support 
for  those  students  who  find  the  use  of  IT  a challenge.  From  the  student  evaluations  obtained  it  seems  that  the  central 
importance  of  the  tasks  and  the  support  provided  mitigated  the  compulsory  nature  of  the  on-line  tasks. 

The  second  trend  that  these  evaluations  highlight  is  the  need  for  continuous  monitoring  and  improvement  in 
the  on-line  learning  experiences  offered  to  the  students.  The  lecturers  responsible  for  the  delivery  of  this  subject 
responded  to  students’  evaluations  by  redesigning  the  number  and  nature  of  the  on-line  tasks.  They  made  changes 
that  were  designed  to  assist  students  to  be  more  independent  in  their  learning. 

In  addition,  the  responses  to  the  face-to-face  delivery  indicate  that  lecturers  also  need  to  evaluate  the 
content  and  presentation  of  the  face-to-face  lessons  if  using  a mixed-mode  presentation  for  a subject.  It  is  essential 
to  meet  the  needs  of  students,  consider  what  they  have  learnt  before  the  subject  and  extend  this  knowledge. 


Conclusion 


The  design  of  the  subject  and  the  improvements  made  were  part  of  an  attempt  to  develop  a more  "learner 
centred  constructivist"  learning  environment  (Latchem  & Moran  1998,  p.67).  The  students  approved  this 
philosophical  approach. 

The  success  if  this  subject  is  a significant  component  of  organisational  change  within  the  Faculty’s  pre- 
service teacher  education  program.  Indicators  of  organisational  change  since  the  inception  of  this  initiative  has  been 
the  interest  of  other  lecturers  in  doing  on-line  work;  involvement  in  Flexible  delivery  projects  funded  by  the 
university;  knowledge  sharing  and  collaborative  learning  of  lecturers;  the  use  of  an  integration  approach  to  on-line 
learning  in  subjects;  and  the  modification  of  subject  presentation  formats. 
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Abstract  This  paper  reports  our  work  towards  the  development  of  an  authoring  system  for  automating  the 
instructional  design  for  teaching  procedural  knowledge.  The  core  of  our  work  is  the  design  of  a set  of 
instructional  events  for  teaching  procedural  knowledge.  Introduction,  Presentation.  Demonstration, 
Exploration.  Practice,  and  Test,  and  the  development  of  corresponding  instructional  event  shells  and  an 
instructional  simulation  tool.  An  instructional  event  shell  is  a piece  of  program  code  that  executes  an 
instructional  event.  Demonstration,  Exploration,  Practice,  and  Test  event  shells  are  built  upon  the 
instructional  simulation  tool.  The  instructional  simulation  tool  provides  an  environment  for  creating 
simulations  of  procedural  knowledge.  These  simulations  may  be  used  for  Demonstration,  Exploration, 
Practice,  or  Test  The  focus  of  this  paper  is  a description  of  these  instructional  events  and  the  instructional 
simulation  tool. 


Introduction 

With  the  rapid  advances  of  computer  network  and  multimedia  technologies,  the  use  of  interactive 
instructional  technologies  has  been  experiencing  a dramatic  increase  in  recent  years.  However,  development  of 
high  quality  computer-based  courseware  is  both  time  consuming  and  labor  intensive.  Therefore,  only  the  best 
funded  training  programs  can  afford  to  be  computer-based.  Many  organizations  that  are  attracted  to  the  concept 
of  computer-based  training  have  to  give  up  when  they  discover  the  cost  and  the  time  to  implement  computer- 
based  courseware. 

A traditional  computer-based  courseware  development  team  involves  subject  matter  experts,  instructional 
designers,  and  programmers.  The  subject  matter  expert  determines  what  to  teach  and  provides  subject  matter 
knowledge.  The  instructional  designer  together  with  the  subject  matter  expert  determines  how  to  teach  and 
designs  the  instructional  treatments  for  subject  matter  knowledge  to  be  taught.  The  programmer  generates 
computer  code  to  implement  the  instructional  treatments  for  the  delivery  of  the  instructional  material.  The 
development  processes  of  both  instructional  design  and  programming  are  very  long  and  labor  intensive  even  for 
some  simple  training  programs. 

To  reduce  the  time  and  cost  spent  in  programming,  authoring  systems  have  been  developed  and  used 
widely.  The  goal  of  existing  authoring  systems  is  to  eliminate  programming  in  the  courseware  development 
process  and  lift  the  expensive  burden  from  the  course  developer’s  shoulders.  Solving  the  programming  problem 
for  course  developers  with  authoring  systems  has  greatly  influenced  the  productivity  of  development  of 
computer-based  instruction,  and  the  number  of  interactive  instructional  software  products  has  increased 
geometrically. 

However,  the  current  conception  of  authoring  systems  has  not  solved  the  cost  and  time  problem  to  a 
sufficient  degree.  The  quality  of  many  current  instructional  products  still  appears  to  be  far  below  the  potential 
promised  by  the  current  computer  technology.  Even  though  the  expensive  programming  burden  has  been  lifted 
from  the  course  developer’s  shoulders,  there  remains  a equally  cost  and  unnecessarily  repetitive  instructional 
design  process  burden  which  still  rests  on  the  developer  (Merrill,  1985). 


Merrill  and  his  colleagues  (Merrill,  Li,  & Jones,  1991)  introduced  the  instructional  transaction  theory  that 
maintains  that  the  basic  building  block  of  instruction  should  be  the  instructional  transaction.  This  principle 
applies  particularly  to  the  highly  interactive  medium  of  computer-based  instruction,  in  which  the  most  powerful 
vehicle  for  learning  is  the  pattern  of  interactions  between  the  student  and  the  computer.  Based  on  the 
instructional  transaction  theory,  a simple  prototype  named  ID  Expert  (Merrill  & Li,  1989)  was  developed.  ID 
Expert  was  intended  for  use  by  both  novice  subject  matter  experts  and  experienced  instructional  designers  and 
provides  instructional  design  and  templates  for  courseware  developers.  The  instructional  transaction  theory 
provides  a theoretical  foundation  for  the  development  of  authoring  systems  that  automate  the  instructional 
design  of  computer-based  instruction  (Zhang,  et  al.,  1997).  Recently,  a few  commercial  authoring  systems  such 
as  ToolBook  and  Authorware  included  some  templates  or  widgets  that  speed  up  the  courseware  development 
process. 

Instructional  simulation  is  an  important  instructional  design  technology,  especially  for  teaching 
procedural  knowledge.  A simulation  is  any  attempt  to  mimic  a real  or  imaginary  environment  or  system  (Alessi, 
& Trollip,  1991;  Reigeluth  & Schwartz,  1989;  Thurman,  1993).  A simulation  is  often  used  for  one  of  two 
purposes:  scientific  or  instructional.  The  reasons  for  using  simulations  instead  of  real  systems  are  cost,  danger, 
inaccessibility,  or  time.  Instructional  simulations  are  designed  to  teach  the  student  about  the  system  by 
observing  the  effects  of  actions  or  decisions.  Instructional  simulations  are  different  from  scientific  simulations 
in  that  it  should  provide  guidance,  explanations,  hints,  and  feedback  for  actions,  decisions,  or  results. 

Computer-based  simulation  has  been  recognized  as  having  cost  benefits,  safety  benefits  and  security 
advantages  in  teaching  procedural  knowledge.  Three  barriers  to  the  effective  use  of  simulations  in  education 
and  training  have  been  identified  (Leston  Drake,  1997).  First,  developing  simulations  is  extremely  expensive. 
Second,  once  a simulation  is  completed,  maintenance  and  modifications  to  the  simulation  are  difficult  and 
costly.  Finally,  not  all  simulations  are  instructional. 

This  paper  reports  our  work  towards  the  development  of  an  authoring  system  for  automating  the 
instructional  design  for  teaching  procedural  knowledge.  The  core  of  our  work  is  the  design  of  a set  of 
instructional  events  for  procedural  knowledge  teaching.  Introduction,  Presentation,  Demonstration, 
Exploration,  Practice,  and  Test,  (Merrill,  et  al.,  1992)  and  the  development  of  corresponding  instructional  event 
shells  and  an  instructional  simulation  tool.  An  instructional  event  shell  is  a piece  of  program  code  that  executes 
an  instructional  event.  The  ideas  introduced  in  this  paper  have  been  implemented  in  a prototype  named 
PKAuthor  using  Visual  C++  and  the  Access  database  system. 


Teaching  Procedural  Knowledge 

Course  Structure 

Authoring  a course  using  KPAuthor  is  to  create  a course  structure,  provide  content  material,  and  create 
instructional  simulations.  The  course  structure,  the  content  materials,  and  simulations  are  stored  in  a database. 

A course  structure  is  a tree  of  course  units.  The  root  of  the  tree  is  the  course  and  the  leaves  are 
instructional  events.  A course  may  consist  of  a set  of  events  and  a set  of  lessons.  Instructional  events  of  a course 
may  be  a pre-test,  a post-test,  a course  introduction,  a course  summary,  and  a comprehensive  practice. 

A lesson  may  be  composed  of  a lesson  introduction,  a lesson  summary,  a comprehensive  practice,  a test, 
and  one  or  more  instructional  sessions.  An  instructional  session  may  consist  of  a set  of  instructional  events  and 
zero  or  more  other  instructional  sessions.  A session  may  teach  one  or  more  procedures. 


Instructional  Events 

Procedural  knowledge  may  be  taught  using  six  instructional  events:  Introduction,  Presentation, 
Demonstration,  exploration.  Practice,  and  Test  (Merill,  et  al.,  1992).  These  six  instructional  events  may  be  used 
repeatedly.  In  PKAuthor,  six  corresponding  instructional  event  shells  are  implemented  to  execute  these  six 
events. 

Introduction.  It  informs  the  student  of  the  learning  objective:  what  the  student  can  do  after  he/she 
completes  the  learning  of  the  procedural  knowledge.  In  addition  to  the  learning  objective,  the  student  is  also 
informed  of  the  prerequisites:  what  the  student  must  know  before  he/she  can  start  learning  the  procedural 
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knowledge.  It  also  briefly  describes  the  tasks  to  be  completed  and  the  goal  to  be  achieved.  All  materials  may  be 
presented  using  different  media  such  as  text  (rich  text  format),  pictures,  still  images,  animations,  audio,  and 
videos.  All  materials  may  be  presented  in  one  or  more  pages.  Hyperlinks  are  allowed. 

Presentation.  It  displays  all  the  material  for  mastery  of  the  procedural  knowledge  to  be  taught. 
Presentation  presents  the  task  to  be  accomplished  with  a text  or  audio  description  of  all  steps  to  be  performed  in 
order  to  complete  the  task. 

Demonstration.  It  uses  examples  to  illustrate  the  material  presented  in  presentation  in  a simulated 
environment.  Each  step  is  demonstrated  by  showing  the  student  how  to  perform  the  step.  The  consequence  of 
each  step  is  also  demonstrated  by  showing  the  student  the  state  of  the  device  or  system  following  the  execution 
of  each  step. 

Exploration.  It  provides  the  student  with  an  opportunity  to  "play  with"  in  a simulated  environment.  This 
"play"  allows  the  student  to  explore  "what  happens  if  ..."  type  questions  by  actually  trying  out  a variety  of 
actions  and  observing  the  consequences  of  these  actions.  Exploration  is  the  mode  of  learning  by  discovery. 

Practice.  It  assigns  some  problems  to  the  student  and  then  the  student  solves  these  problems  in  a 
simulated  environment.  Problems  can  be  solved  with  or  without  guidance.  With  guidance,  the  system  shows  the 
student  what  to  do  at  each  step.  Without  guidance,  the  student  solves  problems  on  his/her  own.  Hints  for  solving 
a problem  are  always  available  upon  the  request  of  the  student.  Both  intrinsic  feedback,  observing  the 
consequences  of  a given  action  or  set  of  actions,  and  extrinsic  feedback,  informing  the  student  about  the 
appropriateness  of  a given  action  or  set  of  actions,  are  available. 

Test.  It  checks  the  level  of  mastery  the  student  has  attained  and  determines  if  the  student  passes  a 
session/a  lesson/a  course.  Test  questions  may  be  performance  questions  or  concept  questions.  A performance 
question  is  similar  to  the  problem  in  practice.  Neither  hints  nor  feedback  is  provided  in  test.  Only  the  score  is 
given  to  the  student.  Concept  questions  are  multiple  questions,  true/false  questions,  matching  questions, 
sequencing  questions,  and  fill-in-blank  questions.  The  student  is  informed  if  he/she  passes  or  not. 


Instructional  Simulation  Tool 

A simulation  provides  a controlled  learning  environment  that  attempts  to  replicate  key  elements  of  the 
real  world.  Simulations  allow  the  learner  to  act  and  then  provide  appropriate  outcomes  or  consequences. 
Simulations  can  best  serve  procedural  knowledge  teaching.  Four  of  the  six  instructional  events  for  procedural 
knowledge  teaching  need  to  use  simulations. 


Simulation  Components 

Components  of  an  instructional  simulation  are  objects,  states,  actions,  tasks,  and  rules.  Creating  a 
simulation  is  to  create  these  components.  A simulation  of  a system  consists  of  a space  of  states,  a set  of  actions, 
and  a set  of  tasks. 

Objects.  Simulation  objects  represent  entities  in  the  real  world.  They  can  be  physical  objects  or 
conceptual  objects.  They  are  the  basic  components  of  a simulation.  A simulation  object  is  composed  of  two 
parts:  internal  and  external  representations.  The  internal  representation  is  a set  of  attribute  value  pairs,  which 
represents  the  internal  status  of  an  object.  The  external  representation  is  a set  of  graphic  images. 

States.  State  is  one  of  the  most  important  components  used  in  the  instructional  simulation.  A state 
represents  the  status  of  the  current  world.  A state  consists  of  a set  of  objects  and  relationships  between  these 
objects.  A change  of  an  attribute  value  of  a state  or  a change  of  a relationship  between  two  objects  changes  the 
current  state  to  a new  state. 

Actions.  An  action  changes  the  state  of  current  world  whenever  it  is  executed.  Teaching  procedural 
knowledge  is  to  teach  how  to  choose  a sequence  of  actions  under  various  conditions  in  order  to  achieve  a goal 
state  from  an  initial  state.  An  action  consists  of  three  parts:  precondition,  consequence,  and  an  external  action. 
The  precondition  of  an  action  specifies  the  conditions  under  which  the  action  can  be  executed,  the  consequence 
specifies  the  effect  of  the  action.  The  external  actions  specify  how  to  execute  the  actions  and  include  mouse  and 
keyboard  actions.  An  action  moves  one  state  to  another  by  changing  some  attribute  values  of  some  objects  or 
some  relationships  between  objects.  Each  action  is  also  associated  with  some  instructional  materials.  The 
instructional  materials  teach  the  student  what  is  the  action,  when  to  apply  the  action,  how  to  perform  the  action. 
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and  what  is  the  effect  of  the  action.  An  action  may  be  taught  explicitly  or  taught  implicitly  using  the  exploration 
instructional  event. 

Tasks-  A task  defines  a problem.  A task  may  be  used  as  a demonstration  example,  a practice  problem,  or 
a test  problem.  A task  contains  one  initial  state,  one  goal  state,  and  sequences  of  actions.  The  problem  is  how  to 
apply  actions  to  achieve  the  goal  state  from  the  initial  state.  The  sequences  of  actions  are  the  solutions  of  the 
problem. 

A complex  task  may  be  divided  into  a few  subtasks,  each  of  which  has  its  own  initial  and  goal  states.  It 
makes  the  complex  task  easier  to  learn. 

Each  task  is  associated  with  some  instructional  materials.  The  instructional  materials  include  instructions 
for  the  student  how  to  solve  the  problem,  hints  at  each  step,  and  feedback  at  each  step.  The  feedback  can  be 
simply  right  or  wrong  for  an  action  or  detailed  explanations  of  why  the  action  is  correct  or  incorrect.  Feedback 
may  be  immediate  feedback  or  delayed  feedback.  Immediate  feedback  is  given  immediately  after  each  action. 
Delayed  feedback  is  given  after  a sequence  of  actions  is  performed. 

Rules.  Rules  are  represented  as  IF-THEN  rules.  The  IF  part  is  a conjunction  of  conditions  and  the  THEN 
part  is  a conjunction  of  consequences.  A condition  is  either  an  attribute  value  pair  or  a relationship  and  a 
consequence  is  either  an  attribute  value  pair  or  a relationship.  Whenever  all  conditions  in  the  IF  part  are 
satisfied,  all  consequences  in  the  THEN  part  become  true. 

An  action  changes  the  state  of  the  world.  These  changes  may  cause  a sequence  of  changes  of  states. 
Rules  are  used  to  determine  this  kind  of  changes  of  states. 


Creating  A Simulation 

Creating  a simulation  is  to  construct  objects,  states,  actions,  tasks,  and  rules.  The  simulation  authoring 
tool  is  composed  of  a set  of  editors:  Property  Editor,  Object  Editor,  Attribute  Editor,  Action  Editor,  State  Editor, 
Task  Editor,  and  Rule  Editor.  Fig.  1 show  the  screen  of  the  dialog  box  of  Action  Editor,  respectively. 


Fig.  1 . Action  Editor  Dialog  Box 

Property  Editor.  A property  is  an  attribute.  The  Property  Editor  is  used  to  define  properties  of  objects 
and  relationships  among  objects.  A property  may  have  four  types  of  values:  enumerated,  numeric,  set,  or  string. 
To  define  a property,  one  needs  to  specify  the  name  of  the  property,  the  value  type,  and  the  legal  values.  Fig.  2 
shows  the  screen  of  the  dialog  box  for  editing  value  type  and  values  of  a property. 

Object  Editor.  The  Object  Editor  is  used  to  collect  the  necessary  information  for  creating  an  object.  The 
information  includes  the  name  of  the  object,  the  type  of  the  object,  the  list  of  properties,  and  the  media 
representation. 
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Action  Editor.  The  author  defines  actions  through  Action  Editor.  Action  Editor  allows  the  author  to 
specify  the  name  of  an  action,  the  preconditions,  the  consequence,  and  the  mouse  and  keyboard  action. 
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Fig.  2 Dialog  box  for  editing  value  type  and  values  of  a property 

State  Editor.  The  author  creates,  renames,  and  removes  states  using  the  State  Editor.  To  create  a state, 
the  author  only  needs  to  give  a name  to  the  new  state,  specifies  the  property  values  of  objects,  the  relationships 
among  objects. 

Task  Editor.  The  author  makes  use  of  the  Task  Editor  to 

create  and  edit  tasks.  A task  is  one  of  the  most  complicated  simulation  components.  The  author  defines 
initial  and  goal  states  using  the  state  editor.  A graphic  editor  is  provided  for  the  author  to  specify  the  solutions 
(sequences  of  actions).  Detailed  feedback  may  be  constructed  using  the  graphic  editor. 

Rule  Editor  (Process  Editor).  Rule  Editor  allows  the  author  to  define  the  conditions  and  consequences 
of  a rule.  It  consists  of  a condition  editor  and  a consequence  editor.  Fig.  3 is  the  screen  of  the  dialog  box  of  the 
condition  editor. 
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Fig.3.  Dialog  box  of  condition  editor. 

Authoring  A Course  Using  Instructional  Event  Shells 


Authoring  a course  using  PKAuthor  consists  of  the  following  three  phases. 
• Content  Knowledge  Acquisition 
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• Simulation  Development 

• Course  structure  construction 

• Instructional  event  configuration 

Content  knowledge  acquisition  is  to  create  instructional  materials  for  courses,  lessons,  sessions,  and 
instructional  events. 

Simulation  development  is  to  construct  one  or  more  simulations.  A simulation  consists  of  a set  of  tasks 
for  demonstration,  practice,  and  test  and/or  an  exploration  environment  for  exploration. 

Course  structure  construction  is  to  create  the  tree  structure  of  a course  and  define  all  course  units 
including  instructional  events. 

Instructional  event  configuration  is  to  associate  instructional  event  with  the  content  knowledge, 
simulation  tasks,  and  simulated  exploration  environments. 


Conclusion 

In  this  paper,  we  reported  the  work  we  conducted  in  development  of  a procedural  knowledge  teaching 
authoring  system,  PKAuthor.  This  work  is  a part  of  a larger  project.  One  of  the  major  objectives  of  development 
of  PKAuthor  is  to  explore  the  methods  for  developing  authoring  systems  to  support  automated  instructional 
design.  PKAuthor  is  implemented  on  Windows  95  in  Visual  C-H-  and  the  Access  database  system. 

In  the  current  version  of  PKAuthor,  only  a subset  of  the  features  described  above  has  been  implemented. 
In  the  future,  we  will  add  more  features  into  the  system.  PKAuthor  has  only  been  tested  on  computer  literacy 
courseware.  PKAuthor  needs  to  be  tested  on  more  applications.  A more  user  friendly  GUI  interface  needs  to  be 
designed  and  implemented  in  PKAuthor.  In  the  future,  we  plan  to  implement  a web-based  version  of  PKAuthor. 
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Abstract:  This  article  describes  and  presents  the  results  of  an  18  month  project 
by  faculty  members  and  students  of  the  Computer  Science  Department  of  the 
Azad  University  in  Mashhad,  Iran,  at  an  approximate  cost  of  $10,000.  The 
result  was  the  creation  of  a simple  Intranet  which  employs  eight  phone  lines  and 
makes  it  possible  for  faculty  members,  students,  and  university  employees  to 
access  information  by  only  dialing  up.  Although  not  complex,  this  Intranet  does 
provide  information  services  through  the  use  of  Microsoft  Information  Internet 
Services  (Internet  Information  Server  - IIS)  and  NT  Information  Systems  support 
software  packages  (such  as  Back  Office),  which  offer  users  ready  pages  in  static 
or  dynamic  form  via  ASP  (Active  Server  Page).  This  Intranet  system  also 
features  E-mail  service  to  facilitate  the  communication  between  and  among 
students  and  faculty.  In  addition,  users  may  conduct  direct  conversations  on 
Web  pages  (Net  chatting)  and  participate  in  network  conferences  (Net 
meetings)  while  conveniently  using  the  local  language,  Farsi. 


Introduction 

Today  there  is  no  doubt  that  information  systems  play  an  important  role  in  the  development 
of  the  remote  education  culture.  From  remote  sites,  via  telephone  lines  and  satellites,  computer  users 
can  now  access  different  kinds  of  hypertext  information  data  bases  that  specialize  in  scientific, 
commercial,  artistic  and  other  areas  with  multimedia  user  friendly  formats.  Actually,  this  scenario 
is  the  basic  working  principle  of  remote  education.  However,  the  problem  remains  that  information 
is  too  widely  scattered,  with  no  dedicated  organization,  fixed  protocol  or  rules,  for  guiding  the  user 
in  efficiently  locating  and  utilizing  these  valuable  educational  resources  in  an  optimum  way  and 
format.  This  is  indeed  a universal  problem,  not  just  isolated  to  the  specialized  academic 
environment.  One  possible  solution  for  resolving  this  dilemma  is  the  creation  of  local  Intranets  in 
educational  institutions  and  their  departments.  With  proper  planning  and  organization,  direct  and 
indirect  communication  between  users  and  access  to  the  data  they  need  can  be  realized.  This  would 
also  make  possible  scientific  conferences  and  other  direct  and  indirect  means  of  conversation 
between  educators  and  students,  information  providers  and  searchers.  However,  because  of  the  high 
expense  and  technical  expertise  entailed  in  establishing  a comprehensive  information  system  with 
state-of-the-art  information  technology.  Intranet  systems  such  as  these  are  not  yet  common.  This 
simple  Intranet  provides  the  following  services: 

1 . Access  to  data  about  different  university  departments,  including  information  on: 

1.1.  University  departments:  Campus  locations  and  addresses,  the 
geographical  and  historical  development  of  the  departments,  their  academic 
goals,  along  with  related  pictures. 

1 .2  Class  schedules:  Current  semester  schedule  along  with  instructors’  names,  class 

times,  classroom  locations  and  other  information. 

1 .3  Faculty  members:  Campus  addresses,  phone  numbers.  E-mail  addresses, 
academic  experience,  qualifications,  classes  taught  and  now  teaching  ,research 
work,  scientific  articles  and  books  published. 
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1 .4  Students:  Majors,  E-mail  addresses,  projects  interested  in. 

1 .5  Academic  courses:  Curriculum,  university  departments,  number  of  units,  pre- 
requisites, course  resources  and  references,  short  descriptions  of  how  the  course 

will  be  conducted  including  laboratory  requirements,  projects,  exam 
materials,  dates  and  locations  of  exams.  If  possible,  a description  of 

exercises  related  to  each  course  is  provided. 

1 .6  Laboratories,  workshops,  practical  courses,  hardware  and  software  workshops: 
Along  with  a calendar  showing  schedules  and  work  to  be  accomplished, 
suggestions  are  included  on  how  these  special  classes  can  be  taken  advantage  of 

1 .7  Publications:  Articles  published  by  faculty  and  students  and  where  the  abstracts 

are  kept. 

1 .8  Books:  List  of  existing  books  within  the  different  university  departments. 

2.  E-mail,  on-line  conversations  and  Net  Meetings: 

Provided  are  E-mail  access,  on-line  conversations  through  the  Web  (Web  chatting), 
and  network  conferences  (Net  Meetings).  All  of  these  facilities  are  available  for 
those  having  User  LD.’s  and  E-mail  accounts.  In  on-line  conversations  , it  is 
possible  for  users  to  communicate  in  either  English  or  the  local  language,  Farsi.  In 
the  case  where  users  have  a CCD  camera  and  sound  card  connected  to  their 
computers,  they  will  also  be  able  to  see  and  hear  each  other. 

3.  Software  bank  at  the  university  computer  center  and  the  possibility  of  transferring 
files  to  and  from  remote  sites  (File  Transfer  Protocol,  FTP). 


Study  and  Investigations  Performed  in  this  Project 

This  project’s  main  functions  covered  six  areas: 

1 . Design  and  implementation  of  hypertext  pages  along  with  the  scripts  needed  in 
the  database.  This  was  accomplished  using  software  packages  such  as  Home 
Site,  Front  Page,  VB  and  Java. 

2.  Design  and  implementation  of  Access  software  for  university  departments 
which  store  information  obtained  from  the  University  Information  and 

Statistics  Center  and  which  is  updated  by  the  Computer  Service 
Department  of  the  university. 

3.  Creation  of  dynamic  Web  pages  (ASP)  for  accessing  related  information  on 
departments,  students  and  the  software  information  bank.  With  the  help  of 
these  pages,  users  can  access  databases  of  the  server  environment. 

4.  Creation  and  use  of  E-mail  by  users  from  within  the  internal  pages 
of  the  database  with  the  help  of  the  software  called  Exchange  Server. 

5.  Creation  of  Net  chatting  with  the  use  of  Web  pages. 

6.  Creation  of  Net  conferences  through  the  server  with  the  use  of  Net 
Meeting  software. 


1236 


Conclusion 


The  development  of  a simple  Intranet  can  provide  essential  information  for  faculty  and 
students,  plus  the  advantages  of  electronic  communication  in  the  forms  of  correspondence, 
conversation  and  netv^ork  conferencing.  The  use  of  the  Intranet  offers  endless  possibilities  in  the 
development  and  progress  of  the  remote  education  culture. 

This  article  outlines  the  conditions,  requirements  and  actions  that  were  taken  to  establish  a 
simple  Intranet.  One  must  consider  that  the  work  was  performed  under  limited  conditions,  by 
faculty  members  and  students  from  the  Computer  Science  Department  at  an  overall  low  cost.  This 
should  be  compared  to  the  unlimited  current  and  future  possibilities  in  the  development  of  remote 
education  that  the  Intranet  and  the  use  of  computers  provide  to  the  University.  It  naturally  follows 
that,  as  a result  of  this  project,  university  departments  will  take  more  advantage  of  this  and  related 
projects. 

The  following  outlines  other  areas  that  can  be  explored  to  make  this  Intranet  more 
profitable  and  successful: 

1 . Separation  of  E-mail  servers  and  the  transferring  of  files  from  other  system 
information  servers  so  as  to  increase  the  number  of  users  and  the  speed  of 
services. 

2.  Preparation  of  electronic  news  magazines  by  different  departments  which  students 
can  access  from  remote  sites. 

3.  Creation  and  development  of  Telecourses  in  CD  form  which  can  be  accessed  by 
students  on-line  from  remote  sources  or  off-line. 
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Abstract:  In  this  paper  we  are  mainly  interested  to  present  briefly  a teleleaming 
system  where  the  learner  is  able  to  attend  courses  to  his  own  rhythm  and  accesses 
only  the  elements  of  the  course  which  relates  to  him.  The  power  of  our  system  is  the 
use  of  widely  available  tools.  This  availability  will  guaranty  a large  opening  and  a 
larger  audience.  Our  objective  is  to  produce  one  generic  course,  which  will  be  used  to 
generate  several  specific  courses  according  to  the  learner  profiles. 


Several  teleleaming  applications  were  carried  out  to  provide  courses  on  the  Internet  (Harasim,  1999) 
(Padhye  &al.,  1999).  Generally,  the  courses  are  hypertext  documents.  In  this  approach,  a learner  can 
access  any  sequence  of  a course  without  order  constraints  on  these  sequences  and  without  having  the 
necessary  knowledge.  Consequently,  the  teaching  objectives  (specialty,  type  of  training...)  aimed  by  the 
access  to  the  course  will  not  be  reached.  In  our  view  and  to  ensure  a pedagogical  learning,  the  course 
must  take  account  of  the  learner  profile  (capacities  of  training,  previous  knowledge,  language...)  and  the 
learner  evolution  (prerequisite,  speed...)  (Fleury  M.)  (Bloom  &al.). 

In  this  paper,  we  are  mainly  interested  to  present  techniques  of  adaptability  of  the  courses  in  the 
SMART-Leaming  (System  for  Multimedia  Adaptative  and  coopeRative  Teleleaming)  project.  The  goal 
of  our  system  is  to  allow  a reliable  adaptability  of  the  courses  to  learners  based  on  tools  that  are  of  a great 
availability  and  easy  to  use. 

Functional  model 

The  fundamental  goal  of  SMART-Leaming  is  to  provide  the  learner,  with  the  sequences  from  the 
course  which  are  adapted  to  his  profile  and  according  to  the  progress  in  the  course.  The  profile  changes 
are  dynamically  processed  by  the  system.  The  author  produces  only  one  generic  course  and  at  the 
learning  time,  our  system  generates  an  individual  course  for  each  learner  [Ajhoun  & al.  2000]. 

To  adapt  a course  to  a given  profile  will  recommend  the  author  to  take  into  account  all  various  types  of 
profiles  as  well  as  the  pedagogic  path  to  follow  for  each  profile.  To  simplify  the  task  of  the  course 
production,  our  system  is  conceived  to  guide  the  author  by  proposing  some  courses  production  models. 

To  make  the  courses  evolutionary,  the  author  uses  the  interactions  of  learners  (questions,  remarks, 
badly  understood  sequences...)  with  the  system  to  improve  his  course.  Using  this  feedback  effect,  the 
author  will  have  the  possibility  of  modifying  the  contents  and  improve  its  teaching  techniques  if 
necessary.  This  significant  aspect  is  translated  in  the  architecture  of  our  system  by  a tight  dependence 
between  the  phase  of  production  and  that  of  learning 

At  the  learning  time,  the  profile  changes  with  the  advancement  of  the  course.  Every  pedagogical 
sequence  of  the  course  should  be  generated  in  real  time  to  take  account  of  the  new  calculated  profile.  For 
this  reason,  our  system  proposes  techniques  of  generation  of  the  courses,  based  on  the  generic  course 
produced  by  the  author  and  the  calculated  profile  of  the  learner. 

Methodology  of  adaptability 

In  order  to  respect  the  adaptability  objective,  we  have  designed  a model  for  the  production  of  generic 
courses.  This  model  provides  the  general  stmcture  of  the  course  in  accordance  with  the  profiles  of 
learners  and  the  objectives  aimed  by  this  course.  Our  system  integrates  in  the  generated  course  the 
mechanisms  to  take  account  of  the  dynamic  evolution  of  the  profile  during  the  learning  (Benkiran  & al. 
2000). 
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At  the  end  of  each  pedagogical  sequence  and  if  necessary,  the  learner  make  a self-evaluation  test  to 
control  the  knowledge  obtained  during  this  sequence.  According  to  the  test  result,  as  well  as  other 
parameters  based  on  the  interaction  with  the  system  (e.g.:  speed,  spent  time)  and  based  on  the  educational 
graph,  the  system  chooses  the  next  pedagogical  sequence  to  present  to  the  learner.  The  educational  graph 
represents  the  whole  of  the  learning  rules  which  control  the  succession  of  the  different  course  sequences 
in  order  to  achieve  a teaching  goal. 

We  applied  this  model  using  XML  language,  reputed  by  its  facility  to  exchange  complex  document  on 
the  Internet,  for  courses  production.  This  brings  a double  advantage: 

1.  The  simplicity  of  production  and  updating  of  courses; 

2.  The  ability  to  force  the  course  structure  and  the  links  between  the  elements  of  the  course. 

To  solve  the  problem  of  adaptability  in  its  practical  phase,  we  have  defined  an  approach  based  on  some 

XML  specific  tools  (XSL,  DOM...). 

Why  the  Web 

Make  our  system  accessible  to  a large  users  require  the  choice  of  means  and  tools,  which  are  available 
and  known  by  great  number  of  users.  In  our  view,  to  provide  courses  to  learners  anywhere  in  the  world, 
the  use  of  Internet,  is  impossible  to  circumvent.  Indeed,  the  number  of  users  of  Internet  increases 
continuously  (Bosak,  1997). 

Designing  software  for  the  World  Wide  Web  poses  particular  challenges  to  the  design  of  user 
interfaces  due  in  part  to  the  limitation  of  bandwidth,  and  to  the  very  nature  of  hypertext  transfer  protocol 
(HTTP)  browsers  as  document  retrieval  systems. 

The  main  disadvantages  of  the  Web  are  bad  interactivity,  non-guaranteed  quality  of  service,  the  difficult 
synchronization  between  the  media  and  especially  the  structure  of  the  HTML  are  opposed  to  some 
educational  principles.  This  led  us  to  find  solutions  at  the  design  in  order  to  avoid  these  problems. 

The  flow  rate  and  the  response  time  of  the  access  to  the  Internet  network  depend  mainly  on  the  time 
and  the  place  of  access  and  can  lead  to  bad  performances  to  transfer  the  elements  of  the  course  which  are 
multimedia  documents,  in  particular  in  southern  countries.  To  avoid  this  disadvantage,  we  propose 
several  access  modes  to  the  courses.  One  of  them,  the  off  line  mode  consists  in  storing  all  the  media 
elements  of  the  course  locally  (e.g.:  CD-ROM  or  disk).  Ail  the  learning  process  is  done  remotely.  To 
show  a media  element  of  the  course  to  the  learner  the  access  is  done  locally  instead  of  transferring  the 
information  from  the  server.  This  makes  it  possible  to  improve  the  quickness  of  the  presentation  and  the 
interactivity  with  the  system. 


Conclusion 

A prototype  of  our  system  was  carried  out  and  tested  at  our  research  unit  with  a great  success  and 
satisfaction.  Furthermore,  we  underline  the  simplicity  of  the  use  of  XML  to  provide  adaptable  courses  on 
the  Internet.  Our  approach  has  several  advantages.  We  will  mention  its  simplicity,  the  small  data  transfer 
between  the  learner  and  the  course  server  — because  only  profile  related  data  are  transferred  - as  well  as 
the  negligible  processing  to  adapt  a course  to  each  learner. 
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Abstract:  The  Cosmic  Explorer  is  a simulation-based  interactive  exploratory  system  aimed  to  provide 
an  interactive  means  of  simulating,  exploring  and  studying  cosmological  models  of  the  universe.  This 
paper  presents  the  web-based  explanatory  library  of  the  system  that  takes  into  account  the  leamef  s 
difficulties  in  envisaging  the  cosmological  concepts  and  adaptively  presents  the  domain  knowledge  to 
match  the  learner’s  demands. 


The  Cosmic  Explorer 

Modem  cosmology  is  an  example  of  a complex  dynamic  conceptional  domain  and  is  best  described  by  the  General 
Theory  of  Einstein.  The  Cosmological  Principle  (CP)  implies  a homogenous  and  isotropic  distribution  of  the  matter  in 
the  Universe  without  giving  an  explicit  definition  nor  sufficient  distinctive  description  of  what  this  mean  in  practice. 
Therefore,  students  face  difficulties  in  grasping  these  concepts  and  their  representations  because  their  imaginary  picture 
of  the  “big  bang”  event  is  often  based  on  firework  metaphor  (Odenwald  & Fienberg,  1 993). 

To  tackle  this  situation  we  are  developing  the  Cosmic  Explorer  that  uses  an  N-body  hydrodynamic 
cosmological  simulator  (namely  Hydra  by  Couchman,  et  al.  1995)  and  a research-specific  visualization  utility  (namely 
Tipsy  by  Katz  and  Quinn)  to  produce  a realistic  simulation  of  the  real  Universe  and  to  provide  scientific-class  analysis 
tools.  The  system  is  aimed  to  promote  conceptual  understanding  of  the  cosmological  principle  and  confront  several 
misconceptions  associated  with  the  expansion  of  the  universe. 

The  practical  goal  of  the  CosWbEL  is  to  support  self-driven/distance  learning,  and  to  complement  traditional 
learning  settings.  Here  we  adopt  the  view  that  Science  teaching  should  reduce  the  emphasis  on  problem  solving  in  favour 
of  a clear  and  deeper  understanding.  One  way  to  achieve  this  is  by  a heavy  usage  of  visual  illustrations  of  the  concepts 
as  well  as  comprehending  its  implication  before  concentrating  on  relevant  problem  solving  skills.  Consolidating  the 
proper  understanding  can  be  helped  by  extracting  several  visual  representations  from  real  or  simulated  world(s)  so  the 
de-abstraction  process  will  be  easier.  For  example,  the  students  can  be  shown  visually  the  data  form  which  the 
descriptive  properties  were  extracted.  Consequently,  reversing  the  process  will  be  easier  for  them,  since  they  have  visual 
images  that  go  side-be-side  with  the  description  rather  than  purely  theoretical  imaginative  thoughts. 

During  the  process  of  refining  the  design,  a preliminary  study  was  devised  to  probe  students  understanding  of 
a few  key  concepts.  The  study  indicated  a lack  of  proper  understanding  and  incomplete  background.  The  lack  varied  from 
one  subject  to  another  and  seemed  to  depend  on  the  speciality  of  the  subject  and  on  the  relevance  of  his  research  project 
to  the  expansion  of  the  universe  and  the  large  scale  structure.  Although  this  domain  is  addressed  nicely  in  most  of  the 
recent  astronomy  introductory  textbooks,  a deeper  understanding  was  not  achieved  probably  because  the  students  have 
taken  few  courses  that  cover  the  topic. 

This  situation  set  the  rational  for  developing  a web-based  explanatory  library  tuned  towards  filling  learners’ 
understanding  gaps.  Providing  the  necessary  coherent  background  is  crucial  since  it  is  believed  that  this  process  is  the 
most  effective  and  efficient  way  to  rectify  several  misconcepts  specially  naive  ones.  Another  valuable  gain  is 
comprehension  of  the  current  concepts  as  the  next  pivotal  step  towards  providing  a strong  unffagmented  foundation  on 
which  new  concepts  can  be  built.  The  ultimate  goal  of  the  research  is  “How  the  system  can  model  the  learner's  reflections 
and  provide  dynamically  generated  meaningful  choices.”.  The  next  section  outlines  the  proposed  library  resources  and 
the  categorisation  as  initial  attempt  to  grasp  the  nature  of  the  problem  and  possible  effective  solution. 

The  Library  Resources 
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The  CosWbEL  will  consist  of  a comprehensive  and  structured  database  of  the  domain  knowledge  and  a 
mapping  of  the  relevant  local  and  global  resources.  The  knowledge  database  will  consist  of  the  following: 

Explanatory  Text  Illustrative  Figures  Interactive  Diagrams 

Conceptual  Maps  Comparative  Tables  Associated  Misconceptions 

Formula  Sheets  Units  Listing  Sheets  Glossary  Of  Terminology. 

As  the  above  listing  suggests,  this  section  of  the  library  will  consist  of  online  documents  that  present  the 
cosmological  concepts  and  their  implications.  The  library  will  make  use  of  a lot  of  visual  illustrations  that  qualitatively 
show  how  these  concepts  apply  in  given  situations  as  well  as  how  they  fit  together  as  part  of  the  whole  picture. 
Furthermore,  it  will  present  examples  of  possible  homogenous  distributions  and  check  for  understanding  and  go  deeper 
if  necessary  as  a way  of  adaptive  to  the  learner.  Conceptual  representations  of  the  influencing  factors  have  been  prepared 
for  five  key  concepts  that  are  best  supported  by  the  simulation  outputs. 

Learners  will  be  encouraged  to  adopt  an  exploratory  learning  methodology  (Cox,  1992;  al-Shidhani,  1999) 
since  the  dynamically  constructed  knowledge  will  be  presented  in  a structured  network.  This  mode  focuses  on 
synthesizing  the  applications  of  a certain  concept  from  previously  introduced  cases.  In  addition,  it  emphasizes  the 
generality  by  showing  how  linking  between  the  concept  abstractions  and  the  specific  cases  has  the  same  theme.  The 
assumption  here  is  that  exploring  such  a conceptual  network  increases  the  understanding  of  the  dynamics  and  the  factors' 
interdependency.  This  approach  adds  elements  of  clarity  to  any  explanatory  text  as  well,  it  shows  some  implications  of 
the  concepts  that  are  taught.  In  this  way,  the  less  complicated  misconceptions  can  be  pointed  out  and  addressed  easily 
at  the  appropriate  place  within  the  text,  so  building  of  new  concepts  goes  step-by-step  with  comprehending  and  clarifying 
the  current  ones. 

The  mapping  of  relevant  local  and  global  resources  will  be  aimed  to  associate  the  related  resources  to  the  constructed 
knowledge  representation.  The  resources  are  classified  as  web  resources,  searchable  databases  and  online  discursive 
resources.  This  component  will  be  an  attempt  to  enrich  the  library  resources  by  providing  classified  map  to  the  diverse 
and  vast  resources  available  via  the  WWW  and  put  the  learners  in  control  of  their  exploration  and  acquisition.  The  main 
distinction  here  is  that  learners  are  unlikely  to  get  confused  or  lost  in  the  huge  web  resources  because  most  irrelevant 
pages  are  filtered  out.  The  detail  of  the  searching  and  filtering  codes  will  be  presented  at  the  conference. 


Conclusion 

The  CosWbEL  is  expected  to  facilitate  the  learning  of  complex  and  dynamic  domains  such  as  modem 
cosmology.  The  complementary  approach  of  the  knowledge  representation  has  the  potential  to  aid  learners  to  build  a 
coherent  well  structured  conceptual  understanding  of  the  domain  and  enrich  the  subject  teaching/leaming  experience 
and  take  advantage  of  the  valuable  www  resources.  Evaluation  of  its  effectiveness  helps  to  clarify  the  optimum  utilization 
of  the  web  in  designing  2 1 st  century  education. 
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Introduction 

Interactive  fiction  (IF)  appears  to  include  both  the  adventure  and  role-playing  games  genres.  However,  Desilets 
(1999)  defined  interactive  fiction  as  a ‘‘computer-based  form  of  literature  in  which  the  reader  plays  the  part  of  an 
important  character,  deciding,  within  limits,  what  action  that  character  will  take’’.  That  author  argues  that  IF 
promotes  student  motivation,  develops  insights  into  elements  of  conventional  literature,  problem-solving  skills, 
critical  thinking  and  problem  identification,  and  can  be  inexpensive  to  develop.  McKee  (1992)  and  Rieber  (1996) 
agree  that  games  can  affect  cognitive  functions  and  motivation,  and  inherently  stimulate  curiosity  (Thomas  and 
Macredia,  1994)  by  including  challenges  and  elements  of  fantasy,  novelty  and  complexity  (Malone,  1984;  Malone 
and  Lepper,  1987;  Rivers,  1990).  Educational  researchers  have  investigated  mainly  two  game  types,  simulation  and 
adventure,  as  viable  educational  tools  (Quinn,  1994,  1997;  Roberts,  1976).  Quinn  (1994)  proposed  a model  for  the 
development  of  educationally  sound  games  that  attempted  to  integrate  cognitive  science,  interface  design  and 
models  of  instruction.  However,  that  theoretical  model  offers  little  practical  help  to  guide  game  developers.  In  an 
attempt  to  formalize  the  relationship  between  educational  theory,  game  design,  play  and  development,  we  developed 
the  Game  Object  Model  (GOM)  (Amory  et  al,  1999)  (Fig.  1).  While  this  model  includes  interfaces  that  create  the 
ambiance  of  the  game,  Harrigan  (1999)  argues  that  dramatic  theories  (Laban,  1975;  Stanislavski,  1976)  could  be 
incorporated  into  the  design  of  educational  software  to  heighten  game  play. 
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Visualization  Space 
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Figure  1.  Game  object  model  [•-  abstract  interfaces, 
O - concrete  interfaces].  (From  Amory  et  al..  1999) 
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Figure  2.  Design  of  interactive  book  to  show  relationships  between  the 
frame  windows,  child  frames  and  objects  within  frames. 


The  primary  objective  of  this  study  was  to  develop  an  interactive  fiction  prototype  to  evaluate  the  use  of  GOM  and 
to  extend  this  model  to  include  an  interactive  actor.  Secondary  objectives  were  to  teach  young  people  basic 
astronomy  and  to  evaluate  the  use  of  Dynamic  Hypertext  Markup  Language  (DHTML)  as  a viable  authoring  tool. 


Materials  and  Methods 

The  basis  of  the  interactive  book  is  a story  by  Jenny  Marais,  a South  African  author  of  the  book  series  for  young 
people.  Learning  with  Granny.  Her  stories  attempt  to  integrate  scientific  knowledge  into  stories  that  stimulate  and 
encourage  scientific  discovery  in  the  reader.  Two  Microsoft®  technologies  were  used  to  create  the  learning 
resource:  Internet  Explorer  DHTML  version  5 and  Agent  version  2 (http://msdn.microsoft.com/workshop/c- 
frame.htm#/workshop/imedia/agent/default.asp).  The  character  selected  for  this  project  was  Peedy,  an  animated 
parrot.  While  the  development  of  the  user- interface  relied  on  VBScript,  the  puzzles  were  coded  in  JavaScript. 
Graphics  were  designed  using  Bryce  (MetaCreations)  and  Photoshop  (Adobe).  The  interactive  book  was  created 
using  DHTML  embedded  in  a number  of  HTML  frames  (see  Fig.  2).  The  pilot  software  was  tested  on  two  young 
people  (a  10  year  old  girl  and  an  1 1 year  old  boy).  An  informal  interview  was  held  before  and  after  the  game  was 
played. 
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Results  and  Discussion 

Both  participants  were  computer  literate  and  had  access  to  a computer  at  home  that  they  used  for  recreation  (playing 
games,  drawing)  and  for  school  projects.  They  reported  that  they  liked  playing  ‘shoot-em-up’  games  and  were 
familiar  with  Microsoft’s  Encarta,  WordPad,  Paint  and  CorelDraw.  Both  reported  that  they  used  the  Internet  to  find 
information,  downloading  computer  games  and  looking  at  wildlife  pictures  and  cartoons.  Analyses  of  video 
recordings  showed  that  participants  used  the  interactive  book  in  different  ways:  the  girl  spent  more  time  carefully 
exploring  and  reading  the  information  than  did  the  boy,  who  spent  more  time  solving  puzzles.  Klawe  and  Phillips 
(1995)  reported  similar  results.  Both  players  described  Peedy  as  “cute”  and  commented  on  his  friendliness  and 
bright  colours.  However,  they  felt  that  younger  children  (8  year  old)  might  find  Peedy  more  appealing.  When  asked 
what  type  of  a character  they  would  have  preferred,  both  agreed  that  an  alien  character  or  a parrot  with  a surfboard 
would  have  been  better.  The  players  enjoyed  the  interactions,  collecting  items  and  puzzle  solving  but  did  not  enjoy 
reading  all  the  text.  They  found  the  puzzle  that  required  “matching  the  zodiac  sign  with  its  constellation”  difficult 
and  suggested  that  Peedy  should  help  the  user  in  such  puzzle  by  providing  clues  about  what  to  do.  The  participants 
thought  that  the  story  was  a bit  too  simple,  lacked  a climax  and  did  not  support  the  complex  ideas  presented  in  the 
puzzles.  At  the  end  of  the  interview  they  commented  that  the  interactive  story  was  both  fun  and  educational  and 
would  prefer  other  literature  presented  in  this  medium.  Using  DHTML  to  create  an  interactive  story  was  fairly 
difficult  to  code  but  cheap  to  implement.  The  Common  Object  Model  of  Internet  Explorer  provided  a framework  for 
the  development  of  this  resource.  The  Agent  and  Speech  components  enriched  the  environment  in  that  the  inclusion 
of  an  interactive  actor  provided  a focus  for  the  player  and  made  the  interactions  with  the  computer  more  meaningful. 
The  inclusion  of  a virtual  actor  also  adds  an  additional  element  of  drama  to  the  story  and  creates  a more  meaningful 
interaction  with  the  player.  Gestures  made  by  the  actor  support  the  story  line  and  provided  additional  information  to 
the  player.  The  Game  Object  Model  and  inclusion  of  virtual  actors  proved  useful  in  the  creation  of  the  interactive 
story.  However,  DHTML  is  too  difficult  to  use,  as  there  are  no  software  tools  available  to  help  in  adding  complex 
scripts  to  dynamic  Web  pages.  The  young  players  enjoyed  interacting  with  the  software  and  puzzles  but  thought  that 
the  story  line  should  be  more  complex  and  felt  that  the  interactive  character  was  a bit  too  childish. 
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Abstract:  This  paper  reports  on  the  latest  design  updates  and  implementations  of  the  task-based 
information  and  performance  support  system  AIMS.  It  emphasizes  the  visualization  issues  related 
to  the  overall  system  construction,  implemented  in  the  latest  version  of  the  AIMS  prototype. 


Background 

AIMS  stands  for  Agent-based  Information  Management  System  that  aims  at  providing  combined  adaptive 
information  support  for  students  and  instructors  within  the  context  of  on-line  course  environments.  AIMS  helps 
students  to  get  an  overall  conceptual  view  over  the  subject  domain  in  which  they  work  and  to  make  a direct  link  with 
the  course  goals  and  assignments.  They  are  also  able  to  search  for  reference  information  and  materials  needed  to 
perform  a task  as  well  as  to  explore  the  subject  domain  for  better  comprehension  [Aroyo,  Dicheva  99].  AIMS  is 
built  as  an  agent-oriented  architecture,  where  agents’  behavior  is  modeled  in  correspondence  to  the  main  activities 
involved  in  the  process  of  task-based  information  retrieval.  As  a result  of  agents  collaborative  intelligent  behavior, 
the  system  provides  an  intelligent  user-oriented  information  support  for  learners  and  instructors. 


What  Do  We  Visualize  and  How  Do  We  Realize  it? 

Main  visualization  target  in  AIMS  is  the  presentation  of  the  subject  domain  structure  and  the  overlaying 
structures  of  the  course  and  of  the  user  model.  As  part  of  the  supporting  information  search  functionality  AIMS 
proposes  combined  visualization  of  the  search  results.  As  the  basic  underlying  mechanism  for  information 
structuring  and  presentation  we  use  the  concept  mapping  approach.  Concept  mapping  allows  the  learners  to  connect 
new  ideas  to  knowledge  that  they  already  have  and  to  organize  them  in  a logical  structure.  This  way  it  allows  them 
to  see  more  complex  relationships  between  ideas  than  just  sequence  and  hierarchy.  It  is  simple  and  intuitive  visual 
form  of  knowledge  representation  well  applicable  for  subject  domain  structuring  and  course  tasks  presentation.  In 
this  specific  educational  context,  concept  mapping  appears  to  be  a very  useful  technique  for  visual  presentation  of 
search  results,  domain  and  course  structures.  It  is  also  used  for  course  task  and  subtasks.  The  user  model  is  build  as 
an  overlay  model  of  the  domain  concept  map.  Search  results  are  also  partly  presented  with  this  graphical  approach. 
The  result  documents  are  related  to  a result  concept  map  of  domain  terms  used  in  those  documents.  Documents  are 
still  organized  in  the  traditional  way  of  textual  list-based  presentation.  Some  metadata  is  also  applied  in  order  to 
support  a better  and  quicker  overall  search  result  comprehension.  Metadata  in  the  form  of  short  descriptive 
information  and  structured  tags  is  also  used  in  order  to  describe  the  context  of  the  domain  terms  and  links. 

The  visualization  and  domain  navigation  facilities  help  students  to  get  orientated  .within  the  subject  domain 
and  to  build  up  their  own  understanding  and  conceptual  associations.  It  supports  their  visual  thinking  and 
imagination. 


1244 


N Main  Fiamo 


Domeiin  Virw  Go  Mrtp 


Curront  Task:  Design  of  Skeleton  Structure 

Current  Subtask:  Design  me  Skeleton  Structuro  for  your  courseware  product 


6 results 
prototyping 

Oveiview  oftne  Courseware  .. 
CE  model;  Level  3;  Prototypin.. 
CE  model;  Level  3:  Prototypin.. 
CE  model;  Level  2:  Prototypin.. 
HOME:  an  Environment  tor  Hy.. 
CE  model;  Design  and  Protot.. 
Prototyping  for  tiny  fingers 
CE.  Supporting  Examples:  Int.. 


Prototyping  for  tiny  Angers 

Authors:  M.  Retting,  year  1 994;  This  article  proposes  that  Prototyping  would  enhance  the 
quality  of  both  the  computer>based  training  courseware  development  process  and  the  final 


Ilia*  (Untignad  Java  Applet  Window 


Figure  1:  AIMS  Information  Search  and  Browsing  Environment 

The  GUI  Issues  in  AIMS 

Student  Support.  The  student  environment  consists  of  a search  facilitated  with  domain  concept  browser  aimed  at 
providing  support  to  the  student  in  the  context  of  information  search,  domain  exploration,  and  task  comprehension. 
When  the  student  logs  in  he  or  she  can  choose  between  working  on  a course  assignment  or  exploring  the  subject 
domain  and  performing  task  independent  search  queries.  The  browser  component  visualizes  a Concept  Map  of  a set 
of  domain  terms  and  links  in  the  context  of  the  overall  domain  structure  presentation,  search  results  and  course  task 
presentation.  Its  functionality  includes  navigating  (by  shortcuts,  history),  bookmarking  (terms,  tasks,  search  results) 
and  screen  editing.  The  main  AIMS  functionality  - searching  for  course  related  material  is  combined  together  with 
several  options  for  refining  search  query  and  search  results. 

Instructor  Support.  Instructor  support  tools  consist  of  domain,  course,  and  libreiry  editors.  The  domain  editor  is 
aimed  at  providing  support  for  the  instructors  to  build  and  maintain  the  AIMS  domain  knowledge.  Its  functions 
cover  building  a new  domain  structure,  that  is,  the  domain  concept  map  and  the  documents  related  to  it,  as  well  as 
editing  and  updating  the  existing  domain  structure,  including  terms,  links,  and  documents.  The  concept  map 
representing  the  subject  domain  knowledge  is  presented  to  the  students  to  explore  and  is  used  to  support  their 
information  search  activities.  The  course  editor  is  aimed  at  facilitating  the  instructor  to  build  the  structure  of  a 
specific  course  and  organize  along  going  tasks,  course  materials,  and  reference  documents.  Since  the  main  goal  of 
AIMS  is  to  provide  a task-based  information  support,  a series  of  tasks  are  associated  with  each  course.  The  main 
idea  is  that  the  system  uses  a task  to  constrain  the  information  provided  to  the  learner  in  support  of  his  or  her  efforts 
to  complete  this  task.  Each  course  task  is  associated  with  a list  of  domain  terms,  which  the  student  must  know  to 
perform  successfully  the  task.  Tasks  are  related  also  to  course  topics,  which  are  automatically  associated  with  the 
whole  collection  of  domain  terms  included  in  each  related  task. 
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Abstract:  The  purpose  of  this  study  is  to  understand  the  metacognitive  awareness  of 
variables  that  influence  information  seeking  skills  via  internet  among  a group  of 
university  staff  at  Middle  East  Technical  University,  Turkey.  Nine  professors  were 
answered  the  questions  related  to  personal,  task,  and  strategy  categories.  Most  of  the 
interviewers  perceive  the  basic  computer  usage  competence  as  a component  of 
information  seeking  via  Internet.  Most  of  them  complained  about  the  bulk  of  search 
results,  most  of  them  are  irrelevant  which  takes  too  much  time.  They  indicated  that 
goal  and  task  definition,  is  more  important  than  to  navigate  through  Internet.  All  of  the 
interviewers  indicated  the  importance  of  planning  before  starting  to  information 
seeking. 


Metacognitive  Knowledge 

The  general  knowledge  that  guides  effective  selection  and  of  task  relevant  skills  has  been  referred  as  metacognition 
(Flavell,  1977,  in  Paris  . & Myres  II,  1978).  Metacognition  has  been  used  to  describe  our  knowledge  about  how  we 
perceive,  remember,  think,  and  act  - that  is,  what  we  know  about  what  we  know.  (Melcalfe  & Shimamura,  1994) 

Metacognitive  knowledge  consists  primarily  of  knowledge  or  beliefs  about  what  factors  or  variables  act  and  interact 
in  what  ways  to  affect  the  course  and  outcome  of  cognitive  enterprises  (TMelson,  1992).  There  are  three  major 
categories  of  these  factors  or  variables  -person,  task,  and  strategy.  The  personal  category  encompasses  everything 
that  you  could  come  to  believe  about  the  nature  of  yourself  and  other  people  as  cognitive  processors.  The  task 
category  concerns  the  information  available  to  you  during  a cognitive  enterprise.  It  could  be  abundant  or  meager, 
familiar  or  unfamiliar,  redundant  or  densely  packed,  well  or  poorly  organized,  delivered  in  that  manner  or  at  that 
pace,  interesting  or  dull,  trustworthy  or  untrustworthy,  and  so  on.  The  metacognitive  knowledge  in  this  category  is 
an  understanding  of  what  such  variations  imply  for  how  the  cognitive  enterprise  should  best  be  managed  and  how 
successful  you  are  likely  to  be  in  achieving  its  goal.  As  for  the  strategy  category,  there  is  a great  deal  of  knowledge 
that  could  be  acquired  concerning  what  strategies  are  likely  to  be  effective  in  achieving  what  subclass  and  goals  in 
what  sorts  of  cognitive  undertakings. 

Information  seeking  is  a kind  of  problem  solving,  which  requires  goal  identification,  task  definition,  search 
strategies,  use  of  information,  synthesis  and  evaluation.  Internet  and  World  Wide  Web  are  changing  our 
conceptions  of  information  systems,  which  requires  new  search  skills  compared  to  the  traditional  environments, 
such  as  books  and  references.  Open  Ended  Information  Systems-  OEISs  differ  from  traditional  systems  in  several 
aspects  (Hill,  J.  R,  1999)  which  can  be  categorized  as  i.  User  orientation  ii.  System  form  and  iii.  Type  of 
information  retrieved.  The  user  actively  engages  navigating  and  searching  the  system  playing  a primary  role  in 
processing  the  information  retrieved.  On  the  other  hand,  there  are  knowledge  and  skills,  which  are  necessary  for 
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problem  solving  and  research  independent  of  information  system  environment.  In  this  study  the  metacognitive 
awareness  about  information  seeking  skills  for  Internet  were  investigated  with  respect  to  person,  task  and  strategy 
variables. 

Personal  Variables 

One  relevant  aspect  of  information  seeking  skills  via  Internet  is  the  individuals'  s perception  of  the 
characteristics  of  good  information  searcher.  All  of  the  interviewers  described  the  person  who  is  good  at 
information  seeking  via  Internet  as  the  person  having  information  seeking  skills  that  were  gained  at  classic 
environments.  Most  of  the  interviewers  perceive  the  basic  computer  usage  competence  as  a component  of 
information  seeking  via  Internet.  They  indicated  that  the  person  who  is  good  at  using  computers  is  the  person 
who  possesses  the  necessary  skills  for  actualizing  his  purpose,  is  patient  and  knows  the  limitations  of  computers. 
More  than  half  of  them  stressed  the  importance  of  using  package  programs.  Interestingly,  one  interviewer 
responded  that  the  person  who's  good  at  computers  knows  when  to  stop.  On  the  other  hand,  two  of  them  reported 
the  importance  of  knowing  English;  since  most  of  the  information  available  on  the  Internet  is  English.  Most  of 
the  interviewers  reported  that  there  is  no  difference  on  using  computers  regarding  the  gender.  Persons  stating  the 
no  differences,  also  proposed  that  success  is  related  to  the  metacognitive  skills.  Two  of  the  interviewers  indicated 
that  males  are  more  successful  while  other  two  proposed  that  females  are  more  successful.  One  of  the 
interviewers  stated  that  females  are  more  successful  since  they  are  more  patient. 


Task  Variables 

Most  of  the  interviewers  complained  about  the  bulk  of  search  results,  most  of  them  are  irrelevant  which  takes  too 
much  time.  They  indicated  that  goal  and  task  definition  is  more  important  than  to  navigate  through  Internet. 
Therefore  a person  who  will  start  to  search  on  something  need  the  necessary  background  and  knowledge  on  the 
topic.  In  addition  most  of  them  indicated  that  open  information  systems  don't  provide  them  the  articles  that  they 
need  for  their  research  study.  However  they  responded  that  because  of  the  interaction  between  the  colleagues  on 
the  WEB,  it  is  easy  to  access  recent  studies  on  one-to-one  basis.  One  of  the  interviewers  stressed  the  importance 
of  how  to  perceive  information  on  the  Internet.  He  said,  “ If  you  perceive  and  regard  the  information  as  raw  data 
which  should  be  treated  with  suspicion  and  to  be  elaborated  and  validated  then  it  will  cause  no  problem.  “ One  of 
the  difficulties  with  the  task  environment  shared  by  most  of  the  interviewers  is  the  difficulty  to  follow  and  read 
the  information  on  the  screen.  One  of  the  problem  that  most  of  the  interviewers  stated  that  in  open  information 
environments  it  is  not  possible  to  see  the  whole  and  also  it  is  not  possible  to  see  how  much  portion  that  you  reach. 
This  yields  a very  uncomfortable  situation. 

Strategy  Variables 

All  of  the  interviewers  indicated  the  importance  of  planning  before  starting  to  information  seeking.  More  than  half 
of  them  stressed  the  organization  for  efficient  search.  It  is  interested  that  interviewers  from  social  sciences  prefer  to 
access  reliable  and  well-known  institutions,  such  as  government  bodies,  universities  and  research  institutions.  On 
the  other  hand  interviewers  form  science  departments  prefer  keyword  and  author  search  using  search  engines. 
Most  of  the  interviewers  stated  they  prefer  to  read  and  analyze  the  material  on  paper  rather  than  on  screen.  In 
addition  more  than  half  of  the  interviewers  indicated  that  the  people  who  are  completing  the  search  through 
shorter  links  are  more  successful  than  the  others.  This  response  is  related  to  the  planning  and  organization,  which 
is  stressed  by  most  of  them. 
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Abstract:  Scientific  training  requires  much  more  than  simply  communicating  content  matter.  In 
adult  scientists,  research  activities  involve  complex  collaborative  work  through  which  scientific 
rigor  is  achieved.  This  suggests  that  kids  should  best  be  trained  into  critical  thinking  by  being 
immersed  into  real  scientific  communities  wherein  they  could  produce  relevant  knowledge  and 
pursue  significant  exchanges  with  «real»  experts.  Hypermedia  technologies  now  permit  creating 
learning  activities  that  meet  such  requirements. 


Introduction 

Within  the  typical  school  curriculum,  scientific  training  is  usually  achieved  through  presenting  the  kids 
with  the  normative  content  of  the  discipline.  More  active  and  involving  formulas  also  include  laboratory  work,  field 
observations  and  personal  research  projects,  yet  most  often  rigorous  and  critical  thinking  still  rely  upon  conformity 
with  traditional  material  and  examples,  and  upon  applying  logical  or  methodological  rules.  Moreover,  even  among 
scientists  of  the  cognitivist  tradition,  there  is  still  a clear  emphasis  on  learning  as  basically  an  individual  process, 
resting  upon  the  learner’s  past  knowledge  and  acquired  cognitive  skills.  But  should  this  epistemological  view  be 
applied  within  the  scientific  community,  not  much  progress  could  be  made.  Among  «real  researchers»,  collaborative 
work  and  mutual  dependence  are  rather  the  rule:  it  is  not  just  a matter  of  sharing  knowledge  and  data,  it  is  probably 
the  essential  way  through  which  scientific  rigor  is  achieved.  Theories  and  data  are  thus  compared  and  confronted, 
imaginative  and  provocative  new  ideas  are  generated  and  edited,  and  consensus  in  spite  of  the  diversity  of  data 
sources  and  viewpoints  provides  the  basis  for  critical  thinking  and  solid  theorizing.  The  question  is  then  to  engineer 
scientific  curricula  for  the  kids  that  would  put  them  into  the  role  of  «real»  researchers,  interacting  with  «real» 
members  of  the  scientific  community,  through  activities  that  make  sense  for  them.  Moreover,  these  projects  should 
constitute  «real»  knowledge  contributions,  and  should  by  the  same  token  attract  expert  scientists  to  interact 
imaginatively  with  the  kids. 


A Social  Constructivist  Perspective 

«Darwin’s  World»  (Aube,  de  la  Chevrotiere  & David,  1999  - http://darwin.cyberscol.qc.ca)  is  an 
educational  project  using  hypermedia  technology  so  as  to  meet  these  goals  with  children  from  elementary  and 
secondary  schools  all  over  the  world.  It  is  part  of  a larger  project,  «CyberScol»  (http://cyberscol.qc.ca),  aimed  at 
providing  school  teachers  with  powerful  tools,  developed  on  the  Internet,  to  help  them  meet  the  objectives  stated 
within  the  governmental  programs.  Many  disciplines  are  covered  in  CyberScol,  from  language  skills  to  humanities, 
to  physical  sciences.  Although  more  specifically  concerned  with  activities  from  Biology  and  the  natural  sciences, 
«Darwin’s  World»  has  been  developed  so  as  to  foster  and  enhance  rigorous  thinking,  along  with  the  skills  required 
to  apply  the  scientific  method.  Hypermedia  technology  is  used  to  enable  and  sustain  articulate  production  and 
communication,  and  for  inserting  the  youngsters  into  real  scientific  communities,  as  responsible  peer  researchers, 
interacting  through  electronic  mail  with  a community  of  Biologists  who  act  as  consulting  experts. 
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One  typical  example  of  a learning  activity  found  in  «Darwin’s  World»  consists  in  the  adoption,  by  a whole 
classroom,  of  an  animal  species  from  the  local  surroundings.  This  choice  has  to  be  approved  by  the  project  team, 
and  it  involves  a commitment  to  produce,  on  the  project’s  web  site,  the  best  card-file  describing  the  identity  and  the 
ecology  of  the  species.  Only  one  class  in  the  whole  world  could  adopt  a given  animal,  so  as  to  promote  responsible 
expertise  for  the  research  done.  The  scientific  content  has  to  be  validated  by  at  least  one  expert  Biologist,  most 
preferably  a specialist  on  the  adopted  species.  Moreover,  all  the  texts  written  have  to  be  validated  as  to  their 
linguistic  quality  (composition  as  well  as  spelling).  The  kids’  commitment  also  includes  constantly  updating  the 
card-file,  and  behaving  as  experts  in  case  of  public  (or  even  expert)  questions  coming  through  the  electronic  mail. 
Computer  forms  are  available  on  the  web  site,  to  guide  the  children  into  researching  and  selecting  the  relevant  data, 
and  the  results  are  saved  within  a growing  animal  data  base.  The  web  site  posting  the  card-file  is  automatically 
generated  from  these  forms,  and  various  multimedia  elements  (sounds,  pictures,  animations,  hyperlinks...)  could  be 
inserted  to  enrich  the  ongoing  description. 

Two  main  objectives  are  characteristic  of  «Darwin’s  World».  First,  kids  have  to  build  up  their  own 
knowledge,  but  this  process  has  to  be  achieved  through  collaborative  work,  among  the  children  themselves,  but  also 
with  a large  diversity  of  resources  (books,  web  sites,  CD,  TV  programs...),  and  with  a few  knowledgeable  experts. 
They  thus  have  to  compare  data,  elucidate  contradictions,  raise  new  questions,  and  by  doing  so,  to  develop  articulate 
thinking.  Second,  they  have  to  become  responsible  for  whatever  knowledge  they  produce.  Their  work  is  to  be  posted 
on  the  web  site;  with  the  help  of  experts,  they  have  to  guarantee  the  scientific  content  of  their  research;  they  have  to 
raise  new,  relevant  and  stimulating  questions;  their  names  and  class  photograph  clearly  associate  them  with  their 
contribution.  The  approach  advocated  here  stems  from  a socio-cognitive  approach,  and  is  strongly  based  upon 
empirical  results  about  collaborative  learning,  the  transfer  of  knowledge,  science  teaching  and  computer  applications 
in  education  (Campione,  Shapiro,  & Brown,  1995;  Palincsar,  1998).  One  central  idea  is  that  scientific  attitudes 
might  be  acquired  in  a way  similar  to  language  skills,  provided  that  the  kids  could  be  involved,  as  adult  researchers 
most  frequently  are,  in  significant  communication  and  exchange  with  other  researchers.  They  could  thus  participate 
into  building  knew  knowledge,  for  the  value  and  utility  of  which  they  could  truly  be  held  responsible.  Hypermedia 
technologies  now  permit  such  feats  by  combining  many  resources  at  once:  accessing  complex  information 
distributed  over  a variety  of  web  sites,  producing  and  publishing  complex  information  of  high  quality  content,  and 
communicating  quickly  with  a community  of  experts,  for  discussion  and  validation. 


Conclusion 

Many  objectives  are  reached  by  the  same  token.  The  kids  are  deeply  motivated  into  serious  work  and 
rigorous  verification  of  all  kind  of  information  to  be  posted.  By  requiring  constant  interaction  with  computers,  the 
activity  fosters  computer  literacy,  yet  it  also  incites  to  look  for  various  sources  of  information,  including  books, 
journals,  films,  and  other  people.  The  complexity  of  the  task  also  naturally  leads  to  cooperation:  individuals  are  not 
in  competition  and  they  need  each  other,  which  creates  a positive  climate  in  the  classroom.  The  quality  of  the  site 
also  incites  collaboration  from  the  experts,  and  their  comments  motivate  further  the  kids’  work.  Finally,  the 
approach  is  clearly  project-based,  creating  learning  opportunities  through  complex  problem  solving. 
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Abstract:  This  paper  describes  the  development  and  evaluation  of  a training  methodology  to  teach 
students  how  to  create  web  pages.  The  training  methodology  evolved  over  two  years  and  involved 
three  stages  of  testing.  The  data  from  1999  (current  model)  reveals  two  contradictory  results. 
Approximately  one-third  of  the  student  teams  did  worse  than  students  from  1998  but  half  of  the 
class  received  a better  score  than  the  previous  year.  Due  to  the  small  sample  sizes  and  the  results,  it 
is  impossible  to  state  whether  the  new  training  model  is  superior  to  the  previous  two  models. 


Introduction 

This  research  project  built  upon  previous  work  on  the  nature  of  student  use  of  computers  for  instruction  (Backer  97). 
Backer  (1997)  analyzed  student  learning  in  constructing  an  electronic  portfolio  using  computer-based  multimedia 
development  tools.  The  development  of  portfolios  by  students  was  not  a cursory  process.  In  order  to  be  effective, 
portfolios  require  students  to  become  self-reflective  and  view  their  work  as  a whole  rather  than  as  unrelated  pieces. 
Electronic  resumes,  as  natural  technological  outgrowths  of  portfolios,  can  expand  this  active  process  further  by 
encouraging  students  to  express  themselves  visually  as  well  as  in  written  form.  In  addition,  this  study  showed  that 
the  use  of  electronic  portfolios  brought  learning  that  is  interactive  and  socially  based. 

The  question  then  evolved  for  the  investigation  in  this  current  project.  Since  students  approach  the  design  of 
multimedia  in  diverse  ways  and  with  varying  results,  how  do  they  approach  and  learn  how  to  create  web  pages?  This 
question  formed  the  basis  of  this  qualitative  study  where  the  focus  was  on  attempting  to  determine  how  students 
learn  to  use  new  tools  rather  than  on  the  final  projects. 


Evolution  of  the  WWW  Training  Methodology 

The  goal  of  the  training  was  to  teach  students  how  to  design  web  pages.  The  actual  training  methodology  has 
evolved  over  the  past  two  years.  In  Stage  One  (1998),  the  training  model  for  web  authoring  was  simplistic.  The 
instructor  made  the  assumption  that  the  students,  since  they  had  experience  in  developing  multimedia,  would  have 
little  trouble  developing  their  first  web  page.  The  training  consisted  of  a presentation  on  HTML,  a class  exercise  on 
HTML,  and  a demonstration  of  HTML  editors. 

Based  upon  the  results  of  Stage  One  (see  Backer  98),  the  training  methodology  was  expanded  in  Stage  Two  and  was 
more  elaborate.  Stage  Two  was  done  as  a part  of  summer  1998  workshops  on  incorporating  multimedia  in  education 
for  K-12  teachers  (see  Backer  & Saltmarch  99).  These  training  workshops  were  organized  by  the  Mathematics  And 
Science  Teacher  Education  Program  (MASTEP),  a five  year  NSF  sponsered  consortium  to  improve  mathematics 
and  science  in  the  greater  San  Francisco  Bay  Area.  The  first  two  steps  were  the  same  as  in  Stage  One,  however, 
after  these  steps  there  were  additions  to  the  training  methodology.  The  students  were  given  a six-lesson,  self-paced 
tutorial  on  how  to  complete  a web  page  that  they  were  required  to  complete.  After  the  tutorial  on  HTML,  there  was  a 
demonstration  of  Adobe  Pagemill  followed  by  a self-paced  tutorial  on  Pagemill.  Pagemill  was  used  as  the  web 
editor  as  it  had  been  donated  to  MASTEP  by  Adobe  Systems  as  part  of  a software  package  for  the  students. 

The  quality  of  the  web  sites  produced  by  students  in  Stage  Two  was  significantly  better  than  the  quality  of  those 
produced  in  spring  1998  (Stage  One).  The  simplistic  error  problems  found  in  the  first  investigation  were  virtually 
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absent  this  time.  In  addition,  the  students  appear  to  have  the  better  grasp  of  the  nature  of  web  authoring  than  did  the 
students  in  spring  1998. 


The  Stage  Three  model  built  upon  the  results  of  the  previous  two  stages.  For  the  Spring  1999  class,  the  training 
methodology  differed  slightly  from  stage  two  and  was  enhanced.  Instead  of  starting  the  instruction  by  discussing 
HTML,  a section  on  web  design  and  web  structure  was  added.  In  addition,  the  first  day’s  lecture  focused  on  design 
criteria  for  effective  web  pages.  The  next  three  steps  were  the  same  as  in  stage  one  and  two.  After  this  section, 
additional  days  were  added  to  specifically  address  graphics,  JAVA,  and  complicated  HTML  tags.  The  training 
model  for  Stage  Three  was  as  follows: 

1 . The  instructor  gave  a web-based  presentation  on  web  structure  and  design  criteria  for  effective  web  pages. 
This  was  done  on  day  one. 

2.  The  instructor  gave  a computer-based  presentation  on  the  structure  of  a web  document  and  the  basics  of 
HTML.  Then,  the  students  were  given  handouts  on  basic  HTML  tags.  This  presentation  was  followed  by  a 
class  exercise  on  coding  a web  page.  For  this  exercise,  students  were  given  a screen  shot  of  a web  page 
along  with  the  text  present  on  the  page.  They  were  asked  to  add  the  correct  HTML  tags  to  make  the  basic 
text  have  the  same  format  as  the  screen  shot.  After  they  finished,  they  were  given  a handout  with  the 
correct  HTML  code  so  they  could  compare  it  to  their  own  coding.  These  steps  were  done  on  day  two. 

3.  Next,  the  students  were  given  a self-paced  tutorial  on  how  to  complete  a web  page.  The  students  were 
required  to  complete  the  six-part  tutorial.  The  students  were  given  the  diskettes  and  asked  to  return  them  on 
day  three. 

4.  Because  of  the  problems  with  graphics  evidenced  in  the  summer  workshops,  one  day  was  added  to  focus  on 
using  graphics  on  the  web. 

5.  At  this  point,  the  instructor  gave  a computer-based  presentation  on  Microsoft  FrontPage  to  construct  Web 
pages.  After  the  presentation,  the  students  were  given  self-paced  tutorials  on  Microsoft  FrontPage.  They 
were  required  to  complete  this  tutorial  before  beginning  their  own  web  design. 

6.  Three  additional  days  were  used  to  address  specific  web  issues.  The  topics  covered  in  these  presentations 
included  links,  lists,  tables,  JAVA  applets,  and  frames. 


Results 

The  analysis  of  the  outcomes  (i.e.,  the  web  pages  provided  by  the  students)  was  done  by  reviewing  the  errors  in  the 
final  web  pages  submitted  by  the  student  teams.  The  average  grade  for  the  final  web  project  in  1 999  was  75%;  this 
compares  to  a class  average  of  77%  from  1998.  A t-test  of  the  data  shows  there  were  no  differences  in  the  results 
from  the  two  years  although  the  variance  for  the  second  year  was  much  higher  than  in  1998  (105.17  for  1999  versus 
39.53  for  1998).  More  student  teams  received  higher  scores  (50  and  above)  in  1999  as  compared  to  1998.  However, 
in  both  years,  there  were  some  students  who  received  very  low  scores:  two  students  in  1998  received  scores  of  36/50 
and  three  student  teams  received  scores  of  30/50  in  1999. 

The  data  from  1999  reveals  two  contradictory  results.  Approximately  one-third  of  the  student  teams  did  worse  than 
students  from  1998  but  half  of  the  class  received  a better  score  than  the  previous  year.  Due  to  the  small  sample  sizes 
and  the  results,  it  is  impossible  to  state  whether  the  new  training  model  is  superior  to  the  previous  two  models.  The 
researcher  is  currently  re-testing  the  third  training  model  during  the  Spring  2000  semester  in  order  to  determine 
whether  it  provides  a more  effective  method  of  training  students  to  develop  WWW  materials. 
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Abstract:  The  leaders  of  24  computer  facilitated  learning  projects  in  higher  education  were  interviewed 
about  their  educational  beliefs  and  practices,  and  the  transcripts  and  associated  documents  were  coded  on 
16  belief  and  16  practice  dimensions.  Shared  patterns  of  beliefs  and  practices  were  sought  by  clustering  the 
projects  with  the  aid  of  hierarchical  methods.  Three  example  profiles  are  provide  to  illustrate  the 
belief/practice  connections  which,  although  apparent,  are  not  simply  defined.  Subsequent  analyses  will  be 
based  upon  project  narratives. 


The  impetus  for  the  present  work  came  from  recent  interest  in  the  beliefs  and  understandings  that 
academics  bring  to  their  teaching  (Kember,  1997;  Samuelowicz  & Bain,  1992)  including  their  use  of 
educational  technology  (Bain  & McNaught,  1996).  There  is  some  research  suggesting  that  academics’ 
educational  beliefs  and  practices  are  closely  coupled  (Bain  1 998;  Quinlan,  1 999),  but  more  work  is  needed 
to  uncover  the  details  of  the  relationships  involved,  particularly  in  relation  to  educational  technology 
(Reeves  1992). 

For  this  study  we  selected  24  university  projects  incorporating  computer  facilitated  learning  (CFL),  based 
on  documentary  material  provided  by  an  Australian  competitive  granting  agency.  We  conducted  two 
detailed  interviews  with  each  project  leader,  each  of  about  two  hours  duration.  The  first  interview  sought 
information  about  the  academics’  development  and  implementation  of  CFL,  and  the  second  probed  their 
epistemological  and  educational  beliefs.  The  semi-structured  interviews  were  audio  recorded  and 
transcribed.  Two  interrelated  analyses  are  being  undertaken,  one  to  result  in  detailed  narratives  of  the  cases, 
the  other  to  provide  an  analysis  of  the  patterns  of  belief  and  practice.  A preliminary  version  of  the  second 
analysis  is  reported  here. 

The  transcripts  and  documentary  material  were  used  to  code  academics/projects  on  16  belief  and  16 
practice  dimensions,  with  coding  based  on  the  full  weight  of  evidence  rather  than  localised  interview 
comments  or  archival  details.  Some  of  the  dimensions  were  drawn  from  published  sources  (Bain  et  al., 
1998;  Samuelowicz  & Bain,  1 992;  Reeves  1 992)  and  others  from  preliminary  examination  of  the  interview 
transcripts  and  project  documentation.  Most  dimensions  were  five  point  bipolar  rating  scales  (c.f..  Reeves, 

1 992),  but  others  involved  qualitative  differences  that  could  be  ordered  from  less  to  more  sophisticated. 
Examples  are  presented  in  (Fig.  1). 

A preliminary  analysis  of  the  data  was  undertaken  by  applying  hierarchical  clustering  (using  several 
methods  and  distance  measures)  to  the  full  set  of  dimensions.  The  aim  was  to  find  similarities  in  the 
belief/practice  profiles  of  the  participants.  Six  clusters  were  tentatively  identified,  three  of  which  are 
illustrated  with  individual  cases  in  (Fig.  1),  using  15  of  the  32  dimensions.  What  is  evident  from  (Fig.  1)  is 
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that,  although  relationships  between  belief  and  practice  profiles  can  be  discerned,  they  are  not  simply 
defined.  Some  beliefs  and  practices  seem  to  be  interconnected  in  ways  anticipated  by  Reeves  (1992)  and 
Bain  et  al.  (1998)  (e.g.,  beliefs  concerning  the  origin  of  knowledge,  the  type  of  understanding  and  the  role 
of  discussion,  and  practices  such  as  learning  control,  learning  process  and  learning  framework),  but  others 
are  not. 


Although  this  profiling  approach  is  useful  as  a preliminary  way  to  compare  and  contrast  projects,  we  are 
convinced  that  a full  understanding  of  the  connections  between  beliefs  and  practices  awaits  the  writing  of 
the  academics’  narratives,  based  on  their  detailed  commentaries  and  our  rich  descriptions  of  the  CFL  and 
its  educational  context  (c.f , Bain,  1998;  Quinlan,  1999).  We  anticipate  that,  once  the  narratives  are  to  hand, 
we  will  be  able  to  group  like  projects  more  convincingly  than  the  profiling  and  clustering  methods  allow. 
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Figure  1 : Profiles  of  three  CFL  projects  on  seven  belief  and  eight  practice  dimensions 
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Abstract:  The  Supporting  Collaborative  Community  (SCC)  model  of  instruction  restructures 
learning  environments  to  access  the  strengths  of  both  individuals  and  the  class  as  a whole  by 
overtly  emphasizing  those  moral  relationships  that  define  true  community  interaction.  The 
SCC  model  creates  this  enriched  environment  through  emphasis  of  four  main  features:  support 
of  clearly-articulated  purposes  or  tasks,  regard  for  each  individual’s  unique  worth  and 
diversity,  acknowledgment  of  one’s  responsibility  to  others,  and  a sense  of  obligation  to  share. 
Allowing  students  to  “own”  their  experience  and  enabling  the  group  to  benefit  from  each 
individual’s  strengths  and  assets  fosters  a supportive  community  where  members  feel  safe  to 
take  risks  and  collaboratively  explore  new  learning  experiences.  The  SCC  model  lends  itself 
well  to  the  dynamic  nature  of  online/distance  learning  when  diverse  members  must  establish 
their  learning  environment  without  the  usual  interaction  that  occurs  in  traditional  face-to-face 
settings.  Because  online/distance  learning  so  often  occurs  asynchronously  amid  other 
priorities  and  commitments,  a community  providing  dynamic  social  and  collaborative  support 
can  motivate  learners  while  providing  the  reinforcement  necessary  to  take  ftill  advantage  of 
the  learning  experience. 


Introduction 

The  Supporting  Collaborative  Community  (SCC)  model  of  instruction  restructures  learning 
environments  to  access  the  strengths  of  both  individuals  and  the  class  as  a whole  by  overtly  emphasizing  those 
moral  relationships  that  define  true  community  interaction.  Where  traditional  classroom  models  focus  mainly 
on  the  relationship  between  the  learner  and  instructor  and  test  only  individual  accomplishment,  the  SCC  model 
overtly  emphasizes  the  positive  dynamics  created  through  community  interaction.  This  model  actively  seeks  to 
establish  an  environment  with  three  elements:  one,  encouragement  of  exploration  without  ridicule  for  mistakes 
or  shortcomings  (support);  two,  emphasis  on  sharing  previously  and  newly  acquired  knowledge  (collaboration); 
three,  a sense  of  belonging  through  participation  and  interaction  with  others  (community).  Within  this 
environment  learners  are  encouraged  to  take  responsibility  for  creating  and  "owning”  their  educational 
experiences.  In  doing  so  they  become  co-instructors  interested  in  making  the  experience  valuable  for 
themselves  as  well  as  others.  Attainment  of  the  group  goals  then  becomes  worthwhile  to  each  individual. 

The  SCC  model  enhances  learning  in  situations  where  there  are  multiple  learners,  where  there  is 
enough  opportunity  and  time  to  establish  a connected  community,  where  interaction  between  participants  is 
encouraged,  and  where  learning  is  more  than  memorization  and  recall  of  facts.  Although  the  specific  structure 
of  the  model  will  be  different  for  every  instantiation,  the  appropriateness  of  using  this  model  is  not  restricted  by 
course  content,  type  of  knowledge,  learners’  ages,  or  proximity  to  other  group  members. 


The  see  Model 


The  see  model  enriches  learning  by  emphasizing  four  main  features.  The  first  is  support  of  clearly- 
articulated  purposes  or  tasks  which  help  orient  the  learners  and  define  the  community.  These  must  be 
considered  important  both  to  the  community  and  its  individuals  so  each  group  member  will  support  them.  The 
goals  or  tasks  should  address  something  the  community  sees  as  valuable  rather  than  unimportant  exercises. 
Learners  must  understand  the  purpose  for  any  learning  in  which  they  engage.  The  second  feature  is  regard  for 
each  individual  s unique  worth  and  diversity.  Each  member  needs  to  feel  that  they  are  esteemed  by  others  as  a 
valuable  person  with  individual  skills  and  perspectives  to  contribute  to  the  whole.  Likewise,  they  must 
sincerely  learn  to  perceive  others  as  such.  Establishing  a pattern  of  valuing  diverse  opinions  and  perspectives 
encourages  the  community  to  seek  participation  from  all  members,  including  those  who  are  infrequent 
contributors  in  more  traditional  settings.  Though  diversity  often  requires  compromise  and  a willingness  to  think 
differently,  articulating  this  as  a group  goal  and  explicitly  emphasizing  its  importance  helps  alleviate  some  of 
the  frictions  that  naturally  occur  when  diverse  opinions  are  expressed. 

Third,  community  members  need  to  acknowledge  a responsibility  to  others^  which  is,  fundamentally,  a 
commitment  of  accountability  to  and  for  each  other.  This  responsibility  creates  a relationship  of  support.  When 
members  agree  to  learn  with  and  from  each  other  they  strengthen  the  power  of  their  interaction.  Fourth,  when 
learners  feel  the  obligation  to  share,  they  are  prompted  to  enter  into  interchange  and  interaction  with  others. 
Sharing  is  an  essential  component  that  makes  collaboration  so  effective.  Without  it,  a community  cannot 
determine  its  goals  or  come  to  know  each  other  through  anything  but  superficial  stereotypes.  Sharing,  even 
when  doing  so  may  put  one  in  a vulnerable  position,  provides  both  opportunities  to  define  oneself  and  allows 
bonding  relationships  to  form.  Sharing  also  benefits  the  community  by  giving  it  access  to  everyone’s  resources. 


Online  Learning  Implications 

The  see  model  lends  itself  well  to  the  dynamic  nature  of  online/distance  learning  where  diverse 
members  must  create  an  environment  with  online’s  own  unique  methods  of  interactions.  Lacking  a well-known 
schema  often  supplied  in  traditional  settings  (classroom  structure,  nature  of  the  instructor,  etc.),  purposes,  tasks 
and  even  roles  are  often  negotiated  and  must  be  clearly  defined  online.  While  virtual  environments  naturally 
allow  participants  the  freedom  to  define  themselves  without  as  many  physical  or  other  identifying  features 
affecting  others’  perceptions,  care  and  sensitivity  must  be  exercised  so  that  exchange  between  learners  fosters 
true  relationships  of  trust  where  each  individual’s  worth  and  diversity  can  safely  be  shared.  Acknowledging  a 
responsibility  to  others  helps  to  ensure  honest  interaction  takes  place.  It  also  encourages  all  learners  to 
contribute  what  they  can  to  the  experience. 

Additionally,  a sense  of  community  commitment  helps  bond  its  members.  Online/distance  learning 
often  occurs  asynchronously  around  other  priorities,  commitments,  and  time  constraints.  The  SCC  model  of 
instruction  can  help  form  a community  providing  dynamic  social  and  collaborative  support  that  motivates 
learners  while  providing  the  reinforcement  necessary  to  take  full  advantage  of  a rich  learning  experience. 
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Abstract:  Rapid  development  in  the  field  of  Multimedia,  Hypermedia  and  Tele- 
communication (MH&T)  is  continually  providing  new  work,  study  and  research  areas.  The 
authors  are  of  the  opinion  that  these  trends  have  found  their  expression  within  the  ED- 
MEDIA  (EM),  which  has  meanwhile  become  the  most  important  conference  in  this  field,  and 
intend  to  prove  this  hypothesis  by  the  evaluation  and  classification  of  the  published  papers 
presented  at  the  conference.  The  study  is  based  on  the  assessment  and  analysis  of  theme 
structures,  technological  backgrounds  and  questions  of  contents,  beginning  with  the 
clarification  of  methodological  questions.  Due  to  this  rapid  development,  prefabricated 
categories  cannot  encompass  a change  in  trend,  rather,  the  categories  must  be  developed  on 
the  basis  of  the  information  at  hand.  The  authors  present  the  argument  that  the  "grounded 
theory",  a procedure  developed  in  social  science,  is  a suitable  methodical  tool  for  the 
building  of  these  categories. 

1.  Methodological  Approach 

In  1999,  600  submissions  from  almost  50  different  countries  were  presented  in  Seattle.  For  the  purpose  of  our 
trend  analysis,  a methodological  approach  should  include  all  papers  submitted,  since  our  analysis  will  only 
include  the  papers  accepted  for  publication  in  Proceedings  of  EM  99,  the  discrepancy  may  be  considered  a 
significant  intervening  variable.  We  feel  that  exploring  the  trends  in  MH&T  in  detail  could  lead  to  a more 
active  and  therefore  influential  implementation.  Previous  studies  of  this  subject  were  mainly  cursory,  the 
evaluation  being  based  solely  on  the  subject  of  the  paper.  In  1996,  Jim  Devines'  (NDEC)  evaluation  of  the  EM 
papers  from  1 993-95  into  didactic  sub-categories  showed  an  increase  in  the  instructionist  approach  (cf  Devine 
(1996)).  We  intend  a more  rigorous  evaluation  of  the  texts  contributed  during  1999.  Pre-definition  of 
categories,  which  fails  to  recognize  the  continual  fluctuation  within  the  media,  is  unsuitable  for  our 
undertaking.  We  believe  that  the  papers’  contents  are  equally  relevant  to  this  study,  not  just  the  subject  of  the 
paper.  It  is  of  greater  importance  to  explore  the  relationship  between  the  two  main  reference  points:  Education 
and  Media,  is  this  instrumental  or  are  there  other  connections  to  be  considered  - social,  organizational  or 
political? 

In  Montreal,  we  intent  to  exemplify  our  proposed  Hypothesis  and  operationalize  the  resulting  category 
construction.  Using  the  material  of  the  EM  99  in  Seattle  as  a temporal  slice  we  shall  demonstrate  and  discuss 
the  proportional  distribution  of  the  individual  categories  (representation  in  percentages). 

After  this  presentation,  we  intend  to  implement  our  theory  with  data  from  the  complete  volumes  of  the  EM,  for 
the  EM  2001  in  Amsterdam.  The  extensive  data  mining  and  evaluation  will  be  carried  out  by  students  of 
Business  Education,  Innsbruck  University,  Austria.  In  spite  of  the  relatively  extensive  effort  required  in  this 
research  we  believe  that  the  results  are  relevant  to  the  scientific  community  - understanding  the  direction  of 
the  research  in  this  comparatively  new  field  is  almost  as  important  as  the  research  itself 
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2.  Tools:  The  Grounded  Theory 


From  the  viewpoint  of  Constructivism  it  is  appropriate  to  place  emphasis  on  the  fact  that  truth  and  reality  does 
not  have  an  existence  independent  of  the  subject.  This  should  not  be  misunderstood  as  solipsism.  If  we  arrange 
the  available  data  according  to  pre-developed  categories  our  preconceptions  appear  to  be  confirmed.  Our 
failure  to  reflect  the  process  of  assignation,  and  our  methods,  preclude  our  awareness  of  the  blocking  and 
changing  done  to  the  material.  These  considerations  caused  us  to  choose  an  alternative:  The  Grounded  Theory 
(cf  Glaser  & Strauss  (1967),  Strauss  (1987)),  which  aims  at  overcoming  this  methodological  circular 
reasoning.  In  a iterative  research  process  the  universal  should  emerge  from  the  individual  through  induction. 
Also  in  the  Grounded  Theory,  inductive  conclusions  are  not  used  for  logical  argumentation  and,  currently, 
have  no  empirical  validity.  They  are  merely  heuristic  instruments  for  the  development  of  a theory.  None  the 
less,  the  validity  of  the  inductive  assumptions  constructed  still  have  to  be  checked. 

The  theoretical  assumptions  still  cannot  be  proved  (verified)  but  only  disproved  (falsified).  This  falsification, 
as  well  as  questions  of  actual  data  distribution,  does  not  lie  in  the  scope  of  the  Grounded  Theory,  whose  aim  is 
the  generation  of  new  ideas,  categories  etc.,  but  of  other  >-  more  traditional  >-  empirical  research  methods.  To 
discover  a theory  from  data  systematically  obtained  from  research  means  to  ’’ground”  the  theory  into  empirical 
research,  because  the  construction  of  the  hypothesis  is  not  based  on  the  researchers  preconceived  ideas  but  on 
the  data  and  material  itself,  the  supporting,  objective  structures  can  be  made  visible.  Examining  the  actual 
data,  we  designate  codes  as  nominators  for  the  incidencies  of  every  interesting  case  to  be  investigated  further 
(indicies),  reducing  the  endless  complexity  of  the  reality  (lateral  coding).  Next,  we  compare  the  different 
nominators,  the  defining  rule  for  this  method  is:  ’’While  coding  an  incident  for  a category,  compare  it  with  the 
previous  incidents  in  the  same  and  different  groups  coded  in  the  same  category  (Glaser  & Strauss  1967, 
p.l06)”.  Slowly,  the  idea  of  a new  concept  emerges,  which  is  able  to  catch  the  different  features  of  some  of  the 
indices:  A new  category  has  appeared  and  is  developed  by  the  researcher. 

Whether  the  constructed  category  is  actually  relevant  (theoretically  ’’good”)  can  only  be  confirmed  by 
searching  the  data  for  further  indications  of  the  new  category  (axial  coding).  Closer  consideration  of  the 
characteristics  (dimensions)  of  the  category  indicate  the  suitability  of  further  subdivision,  or  of  generalization 
(construction  of  meta-categories).  With  this  step  we  have  arrived  at  the  level  of  core  categories.  The 
description  of  the  relation  of  these  meta-categories  will  generate  a special  kind  of  theory  called  ’’Substantive 
Theory”.  In  contrast  to  "Formal  Theory"  (Glaser/Strauss),  this  has  a very  close  connection  to  the  subject  field. 
It  is  a kind  of  ’’thick  description”  as  Clifford  Geertz  calls  the  construction  of  meaning  in  an  anthropological 
description  (Geertz  1973).  Insert  the  Picture  here:  http://www- 

ang.kfunigraz.ac.at/-holzinge/edmedia/proposal.html 

3.  Theoretical  Sampling: 

Every  category  which  has  emerged  is  just  a tentative,  and  temporary,  hypotheses.  It  is  therefore  possible  that 
for  a specific  category  no  appropriate  passage  can  be  found.  In  order  to  find  the  reason,  we  must  investigate 
that  section  of  our  original  data  input  that  caused  the  construction  of  the  new  category  and  to  look  for  other 
possibilities  of  building  categories;  the  sample  is  extracted  according  to  factors  dictated  by  theory.  This 
"Theoretical  Sampling"  (Glaser  & Strauss)  is  not  concerned  either  with  chance  or  with  statistically 
representative  samples,  since  a single  case  is  sufficient  for  both  the  construction  and  the  interconnection. 
A research  completed  in  the  Grounded  Theory  style  is  therefore  not  concerned  with  the  quantitative 
distribution  of  particular  proportions  but  rather  the  discovery  of  categories  where  we  can  put  the  data  in.  We 
have  to  repeat  this  process  from  data  to  category  and  back,  following  a cyclic  procedure  until  all  our  categories 
fit  to  the  data  and  no  further  categories  emerge.  Glaser  & Strauss  call  this  the  saturation  of  the  sample. 
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Abstract:  This  paper  presents  a system  of  help  for  the  exploitation  of  hj^ermedia  documents. 
It  was  derived  from  a common  type  of  educational  CD-ROM  in  which  we  added  search  and 
note-taking  tools.  The  evolving  note-taking  system  interacts  continuously  with  the  document 
being  explored,  and  allows  the  user  to  keep  the  explored  sections  in  mind.  A study  on  usability 
of  the  search  tools  showed  that  the  students  prefer  the  index  or  the  table  of  contents  and 
neglected  the  cross-reference  links.  The  activity  of  note-taking  at  every  stage  of  their  search 
favored  their  planning  of  the  search. 


Introduction 

The  consultation  of  hypermedia  documents  can  be  easy  and  satisfying  when  it  is  simply  a question  of 
discovering  an  individual  item  of  content,  it  can  prove  to  be  more  complex  when  it  is  a question  of  in-depth 
research  in  the  context  of  one  particular  project.  In  a previous  study  (Beaufils,  1998),  we  sought  to  identify 
certain  difficulties  encountered  by  secondary  school  students  using  a typical  hypermedia  (the  CD-Rom 
encyclopedia  which  deals  with  ancient  Greece)  in  order  to  respond  to  a series  of  questions  on  larger  themes. 

In  this  paper,  we  describe  a prototype  which  includes  specific  tools  for  information  seeking  in  a 
hypermedia;  then  we  discuss  the  results  of  several  investigations  on  the  usability  and  the  usage  of  these  tools. 


A prototype  for  information  seeking 

Although  it  conserved  the  content,  the  structure  and  the  index  of  the  original  CD-ROM,  the  prototype 
includes  several  new  tools  for  searching,  selecting  and  note-taking. 

The  search  tools,  such  as  the  index  or  semantic  cross-references,  function  by  interacting  with  the  table 
of  contents.  Colored  highlighting  allows  the  student  to  use  these  tools  in  order  to  situate  the  pages  that  have  been 
found  within  the  hierarchical  plan.  Users  also  have  the  option  of  placing  the  headings  of  all  the  pages  which 
interest  them  in  their  own  personal  area.  The  personal  area  can  be  divided  into  several  compartments,  allowing 
the  students  to  organize  the  material  they  had  selected  into  such  categories  as:  pages  to  visit,  pages  already 
visited,  pages  corresponding  to  themes  existing  in  the  document,  or  pages  corresponding  to  themes  resulting 
from  their  personal  research  plans. 

This  workspace  is  intended  to  function  by  interacting  with  the  selection  system.  Several  different  types 
of  notes  can  feature  on  it:  preparatory  notes  (clues,  bookmarks,  personal  knowledge),  personal  comments  and 
pieces  of  information  gathered  during  the  consultation.  The  notepad  is  structured  by  topics  defined  by  the  user. 
She  can  put  together  the  heading  of  a page,  some  excerpts  of  a page  and  also  personal  comments,  arrange  them 
under  a named  topic  or  move  them  to  another  topic. 


The  usage  of  the  search  and  note-taking  tools 

The  study  involved  six  students,  all  volunteers  aged  16  and  17  coming  from  a high  school  in  the  suburb 
of  Paris.  Students  spent  three  sessions  working  on  three  questions:  (1)  the  image  and  place  of  women  in  Greece, 
(2)  the  importance  of  the  body  in  Greek  civilization,  (3)  the  good  and  bad  aspects  of  Athenian  democracy. 
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In  numerous  cases,  the  students  only  used  one  search  tool  during  any  one  session.  They  were  tempted 
to  repeat,  in  a somewhat  systematic  fashion,  the  strategies  that  had  already  succeeded,  as  Weyer  has  previously 
noticed  about  dynamic  books  (Weyer  1 982).  The  index  and  the  table  of  contents  were  the  most  commonly  used 
tools.  The  index  was  often  selected  as  the  first  tool  and  generally  enabled  a highly  exhaustive  search.  The 
readability  of  the  table  of  contents  ensures  the  fruitftilness  of  searches  conducted  using  this  tool  (Dee-Lucas, 
1996).  Few  attempts  to  extend  the  search  using  semantic  cross-references  were  noted,  mostly  during  the  last 
session.  Linear  exploration  was  only  used  by  students  as  a complement  of  other  consultation  methods.  The 
number  of  pertinent  pages  found  during  the  three  sessions  was  high  (85%  or  higher)  which  shows  the 
effectiveness  of  the  strategies  followed,  whatever  tools  were  used. 

All  the  students  used  the  selection  tool  but  they  usually  inserted  the  page  headings  in  a single 
compartment,  limiting  this  tool  to  its  "storage"  function.  The  putting  aside  of  page  headings  helped  in  consulting 
these  pages  and  reduced  the  risk  of  forgetting  about  them.  The  loss  of  pertinent  pages  was  moderate  (less  than 
30%  for  the  two  first  questions),  which  attests  to  a satisfactory  command  of  the  operations  of  consultation  and 
exploitation.  In  few  cases,  the  possibility  of  creating  categories  within  the  selection  gave  rise  to  diverse 
strategies:  planning  which  sections  to  consult  or  structuring  those  categories  that  had  already  been  consulted. 

The  number  of  notes  produced  by  the  students  at  the  beginning  of  the  session  ranged  between  three  and 
around  ten.  These  notes  essentially  designated  sections  to  explore.  We  noticed  the  strong  influence  the 
preliminary  planning  had  on  the  search  for  information  and  on  the  organization  of  the  final  response.  The 
production  of  an  initial  structured  plan  led  to  a precise  but  inflexible  representation  of  the  goal  to  be  attained. 
During  their  consultations,  students  may  discover  pieces  of  information  that  do  not  figure  in  their  preliminary 
evaluation  of  the  question.  Such  discoveries  can  lead  to  modifications  in  the  final  plan  with  respect  to  the  initial 
plan.  Over  the  course  of  75  operations  of  note-taking  recorded,  17  corresponded  to  an  event  leading  to  a 
noticeable  modification  of  the  final  plan.  Our  observations  show  that  in  the  majority  of  cases,  the  representation 
of  the  response  did  not  develop  in  any  notable  fashion.  One  might  suppose  that  the  more  elaborate  the 
representation,  the  harder  the  subject  will  try  to  preserve  it  intact  (Rouet  & Tricot,  1998). 


Closing  remarks 

We  observed  that  the  activity  of  note-taking  performed  by  the  students  at  every  stage  of  their  search  for 
information  (evaluation,  selection  and  treatment)  unquestionably  favored  how  they  managed  the  search  and  the 
planning:  (1)  the  initial  notes  stimulate  the  process  of  evaluating  the  task,  giving  the  students  points  of  reference 
and  freeing  their  memories  from  certain  work;  (2)  the  workspace  the  students  are  able  to  use  must  be  unified  but 
with  the  constant  possibility  of  being  reorganized;  its  function  is  to  put  three  types  of  notes  in  relation  with  each 
other:  the  preliminary  search  plan,  a selection  of  pertinent  pages,  and  a selection  of  information  gathered  from 
these  pages;  (3)  some  aspects  of  the  process  of  note-taking  are  neither  spontaneous  nor  common,  thus  it  is 
desirable  to  give  the  students  specific  advice  at  the  beginning  of  the  session  or  to  expose  them  to  some  prior 
training. 

By  using  search  mechanisms  that  are  relatively  independent  of  content,  associated  with  an  open, 
flexible  and  evolving  note-taking  system,  students  can  have  the  possibility  of  constructing  a personalized 
hypermedia  system  within  the  document  they  are  exploring.  We  are  now  studying  how  such  tools  can  be 
adapted  to  facilitate  similar  activities  with  web  resources. 
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Assessing  the  Use  of  Technology  to  Enhance  Learning  in  Higher  Education. 
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Abstract:  The  present  paper  reports  on  the  development  of  an  instrument  called  "The 
Use  of  Technology  in  the  Classroom",  that  assesses  the  use,  application  and  quality 
of  technology  to  enhance  education.  A brief  summary  of  statistical  analysis  of  the 
reliability  and  validity  of  the  instrument  is  presented. 

Introduction 

The  process  of  integrating  technology  to  enhance  learning  implies  a change  in  the  curriculum,  which  means 
a shift  on  the  philosophy  of  teaching,  every  day  classroom  practices  and  reassessing  the  way  we  learn. 

One  of  the  responsibilities  of  the  Academic  Computing  department  in  Mexico's  Anahuac  University  is  to 
work  together  with  the  faculty  in  determining  needs  and  opportunities  to  introduce  technology  to  enhance  the 
curriculum  or  course  content,  and  to  facilitate  the  research,  development  and  training  to  support  educational 
technology. 

A proposal  was  advanced  to  formally  assess  the  projects  that  where  generated  and  supported  by  the 
department. 

The  use  of  technology  in  higher  education 

With  the  arrival  of  the  new  Millennium  the  use  of  technology  in  higher  education  has  become  a "must 
have"  rather  than  a "nice  to  have".  And  "...  any  institution  that  is  not  able  to  integrate  elements  of  technology  into 
their  current  offerings  will  find  themselves  at  a competitive  disadvantage  with  the  virtual  universities"  (Rao  & Rao, 
1999). 

The  rapid  changes  in  technology,  the  constant  demand  from  the  working  environment  to  prepare 
professionals  capable  of  using  the  new  technology,  the  institutional  to  make  decisions  related  to  the  acquisition  of 
hardware  and  software,  are  some  of  the  reasons  that  make  the  introduction  of  the  use  of  technology  so  important  in 
higher  education. 

Bennet  (1987)  described  three  different  roles  computers  have  in  education:  education  with  the  computer, 
education  about  the  computer  and  management  of  instruction. 

The  use  of  technology  in  education  has  advantages  (Russell,  1999)  and  the  diversity  of  ways  it  can  be 
integrated  and  applied  is  immense. 

Evaluation  of  the  use  and  quality  of  educational  technology  has  always  been  problematic  because  of  the 
inherent  diversity  and  complexity  of  the  possible  applications  e.g.  (Biermann,  1994;  Doll  1987)  created  check  lists 
that  help  teachers  and  administrators,  assess  different  options  for  educational  systems.  Those  lists  are  practical  to 
evaluate  independently  of  the  classroom  learning  context.  They  are  simple  to  use  to  assess  if  the  system  is  user 
friendly,  well  designed,  has  no  bugs  and  is  well  documented. 

However  more  complete  require  an  empirical  evaluation,  that  may  be  time  consuming  and  costly. 
Comparing  a group  of  students  with  control  groups  is  one  way  of  doing  this. 

Both  methods  of  evaluation  reduce  the  learning  process  specific  factors:  list  of  criteria  to  assess  a systems 
like  if  was  never  going  to  be  used  by  students,  and  students  that  are  consider  like  only  receptors  of  knowledge. 

The  use  of  technology  in  education  can  be  regarded  as  a social  process  in  at  least  two  ways  (Baumgartner, 
Payr  (1995): 

a)  It  takes  place  in  a social  situation  (classroom,  work,  home). 

b)  The  goal  of  a learning  process  is  to  help  us  adapt  with  prevailing  social  conditions. 

This  report  purposes  the  evaluation  of  the  use  of  technology  to  enhance  learning  through  the  three  social 
dimensions  that  the  student  perceives: 

a)  The  Teaching  Quality 

b)  The  Classroom  Climate 

c)  The  Integration  of  Technology  in  the  classroom. 

Process: 
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The  idea  was  to  create  an  instrument  that  experiment  the  level  of  satisfaction  that  the  students  perceive 
from  the  integration  of  the  technology  (basically  software)  with  the  course  content. 

Based  on  the  idea  that  student's  satisfaction  would  derive  from  having  a good  instructor,  being  in  a 
comfortable  learning  environment  (climate)  and  good  application  of  the  technology. 

Scales  of  the  Instrument 


Scales 

Description 

Sample  Item 

Teaching  Quality  (Climate)(TQ-C) 

The  instructor  of  the  course 
knowledge  of  the  content,  clarity  of 
explanation,  fair  assessment,  and  is 
interested  in  students  needs  and 
opinions. 

The  teacher's  explanations  keep  me 
focused. 

System  as  a Medium  to  Enhance 
Learning  (SM) 

Extent  to  which  the  technology  used 
on  the  course  is  effective. 

The  use  of  the  system  enhances  my 
learning 

Equipment  (E) 

The  technology  provided  at  the 
laboratories  is  of  good  quality,  under 
good  working  conditions. 

The  systems  work  properly  on  the 
lab  equipment 

Satisfaction  (S) 

To  what  extent  the  students  feel  that 
what  they  are  learning  is  worth 
while. 

Students  enjoy  coming  to  this  class 

System  Characteristics  (SC) 

Extent  to  which  the  technology  used 
on  the  course  is  task  focused  and 
user  friendly. 

This  system  is  easy  to  use 

Use  of  the  System  in  the 
Professional  Field  (PU) 

Extent  to  which  the  technology  used 
on  the  course  is  updated  based  on 
the  needs  of  the  field. 

This  systems  and  similar  ones  are 
used  on  the  professional  field 

Method 

An  exploratory  study  of  the  psychometric  properties  of  the  instrument  was  carried  out  with  students  of 

undergraduate  courses  where  using  technology  to  enhance  learning. 

There  were  490  participants  from  34  courses,  224  male,  266  female;  from  the  schools  of  Architecture,  Social 

Communications,  Design,  Business  & Economy,  Engineering,  Pedagogy  and  Psychology. 

38%  were  in  year  one,  14%  were  in  year  two,  29%  were  in  year  three  and  19%  were  in  year  four. 

Procedure: 

The  project  was  an  internal  evaluation,  created  and  applied  by  staff  of  the  university. 

One  person,  the  author  carried  out  the  administration  of  the  instrument  giving  any  additional  instructions  needed 

beyond  those  printed  on  the  questionnaire. 

Results: 

The  following  statistical  treatments  were  applied: 

A)  To  validate  the  internal  structure  of  the  instrument  a factorial  analysis:  with  a varimax  rotation. 

B)  To  validate  the  internal  consistency  of  each  scale  a Cronbachs'  alpha  coefficient  was  calculated  for  each 
item  and  scale.  Alpha  of  the  instrument  = 0.9604 

C)  Discriminant  validity  of  the  scales  was  indicated  by  correlation  analysis  between  scales:  (using  the  mean 
correlation  of  a scale  with  the  other  five  scales  as  a convenient  index).  All  values  are  small  enough  to 
demonstrate  that  each  scale  has  adequate  discriminant  validity,  but  the  conceptual  distinctions  among 
scales  are  important  enough  to  retain  the  six  dimensions  within  the  instrument. 

d)  W conducted  an  Analysis  of  Variance  for  items  for  each  scale  to  demonstrate  that  there  is  significant 
difference  between  items  within  each  scale.  All  values  have  p<0.001. 


e)  Differences  between  classes  were  identified  using  an  analysis  of  variance  between  different  measures  of 
groups,  using  the  groups  as  the  unit  of  analysis.  The  results  showed  that  the  instrument  did  significantly 
differentiate  between  the  perceptions  of  the  students  of  the  classes. 

f)  A final  ANOVA  was  calculated  to  demonstrate  the  independence  of  the  scale.  With  an  F=  64.55  p<0.001. 

g)  Multiple  regression  analysis  was  conducted  to  determine  the  relationships  between  variables. 

h)  Structural  Analysis. 

Conclusions 

The  instrument  has  proven  to  be  reliable  and  valid. 

The  structural  analysis  showed  the  Teaching  Quality  (FI)  as  the  center  of  the  use  of  technology  to  enhance 

learning. 

This  means  that  the  instructor  has  an  influence  in  the  way  the  student  perceives  the  Equipment  (F2),  Use  of 
the  System  in  the  Working  Environment  (F3),  System  as  a Medium  to  Enhance  Learning  (F4),  System 
Characteristics  (F6)  and  all  of  this  factors  affect  the  students  Satisfaction  (F5).  As  a teacher  or  as  a facilitator  he 
has  a great  influence  in  the  classroom  climate,  students  satisfaction  and  the  way  they  perceive  the  use  and 
application  of  the  technology. 

The  technology  has  advanced  so  much  in  the  last  two  years  since  the  data  was  gathered  that  some 
adjustments  may  be  necessary  especially  integrating  items  related  to  the  more  advanced  ways  of  using  technology  in 
education  and  considering  some  adaptation  for  online  learning. 

This  instrument  was  created  in  Spanish  and  validation  was  based  on  the  Spanish  version.  Future  research 
has  been  done  with  translation  and  validation  of  the  instrument  in  English. 
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Abstract:  The  Technical  University  of  British  Columbia  is  committed  to  de  development  and 
delivery  learner  centered  courses  that  have  an  online  component  an  emphasis  on  collaborative 
learning.  Assessing  the  quality  of  these  courses  requires  a new  approach , The  TQM  theory 
suggests  that  providing  the  client  with  a high  quality  product  at  all  times  is  a very  important 
constituent  of  quality.  The  social  environment  ’’climate”  has  a positive  effect  on  students  satisfaction. 

This  paper  describes  the  development  and  adaptation  of  an  instrument  that  measures  climate  for 
online  courses  and  the  importance  of  using  the  results  of  the  applications  as  an  important  stage  of  the 
of  the  overall  TQM  process  that  includes  an  ongoing  monitoring  and  modification  of  the  learning 
environment  based  on  climate  assessment. 

Introduction 

The  Technical  University  of  British  Columbia  (TechBC),  the  newest  university  in  Canada,  has  been  created 
as  a unique  learning  environment  to  educate  the  work  force  needed  for  high-growth  advanced  technology  industries. 

Recognizing  the  rapid  growth  in  information  and  knowledge,  TechBC’s  programs  will  emphasize  not  only 
the  acquisition  of  knowledge,  but  also  the  ability  to  utilize  and  apply  knowledge  appropriately,  critically  and 
creatively. 

The  university  is  implementing  a learner-centered  education  (Bonk  & Cummings,  1998),  where  students’ 
cognitive,  metacognitive,  motivational,  affective,  developmental,  and  social  needs  are  met.  Although  perhaps  not 
unique,  the  creation  of  a new  institution  affords  opportunity  to  more  fully  realize  this  approach  in  the  learning 
process.  All  courses  have  an  online  component,  and  most  courses  (with  the  exception  of  the  flexible  learning)  have 
an  emphasis  on  collaborative  learning. 

Assessing  the  quality  of  courses  delivered  in  a team-based,  online  learning  environment 

Total  Quality  Management  for  Education:  Climate 

In  the  last  two  decades  there  has  been  an  emphasis  on  the  importance  of  quality  in  higher  education,  with 
attention  focusing  first  on  the  definition  of  quality  and,  more  recently,  on  how  to  evaluate  and  measure  it  (Harvey, 
Green,  Burrows,  1993;  Brindley,  1995,  Municio,  1995,  Williams,  Watson,  1995). 

The  increasing  number,  and  demand  for,  distance  education  programs  is  prompting  researchers  to  ask  what 
constitutes  a good  distance  learning  course  from  the  perspectives  of  both  the  institutions  and  client.  Some,  like 
Scriven  (1995),  have  even  provided  indicators  such  as  policy  development  and  management,  staff  development, 
service  provision,  and  processes  of  open  learning.  Certain  levels  on  these  indicators  have  been  suggested  as 
benchmarks  for  measuring  quality  in  distance  education. 

Municio  (1995),  William  & Watson  (1995),  Scriven  (1995),  and  Brindley  (1995)  all  mention  that  providing 
the  client  with  a high  quality  product  at  all  times  is  a very  important  constituent  of  quality.  From  the  assertion  by 
authors  like  Fernandez  (1995)  and  Lampikoski  (1995)  that  client  satisfaction  is  a good  indicator  of  quality,  it  follows 
that  student  satisfaction  is  a key  criterion  for  institutions  to  determine  quality  in  distance  education. 

Many  researchers  have  talked  about  students’  satisfaction,  and  the  effect  it  has  on  academic  achievement 
(Fraser  & Deer,  1983;  Fraser,  Seddon  & Eagleson,  1982;  Munoz,  1996).  Peterson,  et  al.  (1986)  established  that  a 
good  social  environment  or  ’’climate”  has  a positive  effect  on  students’  satisfaction. 

In  Canada,  the  Ontario  Ministry  of  Colleges  and  Universities  prepared  Vision  2000:  Quality  and 
Opportunity  (1990)  that  emphasized  the  challenge  of  developing  well-educated  citizens  for  a culturally  diverse 
society.  As  one  possible  answer  to  that  challenge,  Evans-Harvey  (1995)  proposed  that  administrators  and  teachers 
promote  and  develop  of  positive  learning  environments  in  which  culturally  diverse  students  work  in  harmony  while 
meeting  their  personal  aspirations. 

Climate  has  an  influence  on  behavior,  and  can  determine  educational  outcomes  such  as  cognitive 
achievement,  satisfaction,  motivation,  and  personal  development  (Fernandez,  Asensio,  1989,  p.  2). 
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The  definition  of  climate  is  abstract  and  its  measurement  is  typically  based  on  subjective  self-reports. 
Mainly  what  we  see  is  that  the  definitions  of  climate  are  limited  to  the  instrument  used  to  measure  it  (Anderson, 
1982). 

There  have  been  many  instruments  created  to  measure  classroom  climate  in  higher  education  for  a face-to- 
face  environment.  These  include  the  Classroom  Climate  Scale  (CES)  (Moos  & Trickett,  1974),  the  College  and 
University  Classroom  Environmental  Inventory  (CUCEI)  (Fraser  & Treagust,  1986),  the  Science  Laboratory 
Environmental  Inventory  (SLEI)  (Fraser  & Wilkinson,  1993),  The  College  Classroom  Environment  Scales  (CCES) 
(Winston,  Vahala,  Nichols,  & Gillis,  1989).  However,  only  the  Distance  and  Open  Learning  Environment  Scale 
(DOLES)  has  been  created  for  the  specific  purpose  of  assessing  climate  in  distance  education. 

Procedure 

After  reviewing  the  DOLES  literature  where  the  authors  describe  the  choice  of  assessment  method,  criteria 
for  method  development  and  validation  used,  it  was  judged  to  be  effective  in  assessing  most  aspects  of  climate  for 
online  learning. 

For  this  instrument  to  work  in  our  university  the  measure  of  climate  has  to  integrate  both  online  and  face  to 
face  components,  so  we  developed  an  adapted  and  augmented  version  of  DOLES  that  consists  of  8 scales.  The 
descriptions  of  each  scale  are  presented  in  Table  1. 


Table  1.  Scale  description 


Scales 

Description 

Sample  Item 

Students  Cohesiveness 

Extend  to  which  students  know, 
help,  work  together  and  are  friendly 
towards  each  other. 

It  is  easy  to  organize  a group  for  a 
project. 

Teacher  support 

Extent  to  which  the  instructor 
befriends,  trusts  and  is  interested  in 
students. 

The  instructor  sends  me 
comprehensive  feedback  on  my 
assignment 

Personal  Involvement  / Flexibility 

Extend  to  which  the  students 
participate  and  independently  decide 
individual  ways  to  approach  course 
discussions  and  activities 

I get  a chance  to  discuss  my  relevant 
personal  experiences 

Task  Orientation  / Material 
Environment 

Extent  to  which  activities  are  clear 
and  well  organized 

Expectations  of  assignments  are 
clear  in  this  course. 

Interaction 

Extent  to  which  the  students 
interacts  with  the  instructor,  other 
students  and  the  content. 

The  interaction  between  all 
participants  is  high 

Learner  centered 

Extent  to  which  the  cognitive, 
motivational,  developmental  and 
individual  needs  of  the  students  are 
satisfied 

The  course  offers  a variety  of 
activities 

Technology  Resources 

Extent  to  which  the  technology  is 
used  to  enhance  learning 

The  technology  resources  in  this 
course  allow  interaction  between 
instructor  and  student. 

Conclusion 

Revisiting  connection  between  TQM  and  climate,  the  five  areas  of  focus  for  the  theory  of  TQM,  suggested 
by  Sherr  and  Lozier  (1999),  were  mission  and  customer  focus,  systematic  approach  to  operations,  vigorous 
development  of  human  resources,  long-term  thinking,  commitment.  These  five  points  form  a complete  theoretical 
system,  and  one  cannot  fully  understand  any  one  point  without  the  other  four.  Creating  the  instrument  that  measures 
climate  in  online  learning  was  just  the  first  stage  of  a larger  process  that  includes  an  ongoing  monitoring  and 
modification  of  the  learning  environment  based  on  climate  assessment. 
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Abstract: 

Internet  communication  tools  like  Web  sites,  listservs,  e-mail,  newsgroups,  bulletin  boards,  and 
real-time  chat  programs  have  been  embraced  by  many  educators  for  their  potential  to  enhance 
traditional  classroom  instruction.  Specifically,  the  use  of  these  tools  is  believed  to  enhance  the  three 
major  activities  of  all  teachers:  to  counsel  students  individually,  to  deliver  information,  and  to 
encourage  classroom  discussion.  This  paper  reviews  the  findings  from  a study  conducted  by  the 
author  that  measures  the  effectiveness  of  using  electronic  discussion  groups  to  enhance  teaching 
and  learning. 


Survey 

Convinced  of  the  value  of  electronic  discussion  groups  in  enhancing  traditional  classroom  instruction,  the  author  has 
been  requiring  each  of  the  students  in  an  upper-level,  undergraduate  marketing  course  to  subscribe  to  a listserv  set 
up  specifically  for  the  class.  Students  used  the  listserv  to  argue  pro  and  con  positions  on  statements  posted  to  the 
discussion  list  by  the  instructor.  Half  of  the  students  on  the  list  argue  the  pro  position  and  the  other  half  argue  the 
con  position.  Their  responses  are  then  posted  to  the  list  and  critiqued  by  other  students.  A different  statement  is  sent 
to  the  list  every  two  weeks  and  is  often  related  to  a current  news  event  related  to  the  concepts  being  studied.  When 
possible,  students  are  sent  links  to  Web  sites  that  provide  additional  information  about  the  topic  being  discussed. 

To  assess  whether  the  electronic  discussion  groups  were  producing  the  desired  student  outcomes,  the  author 
conducted  a survey  of  all  students  that  had  participated  in  the  electronic  discussion  groups  over  a two  semester  time 
period.  A total  of  52  students  were  included  in  the  survey  and  all  responded. 

Data  were  gathered  from  the  students  by  means  of  a questionnaire  that  was  distributed  both  by  e-mail  and  in  class. 
The  evaluation  instrument  measured  1)  the  extent  to  which  students  had  used  or  participated  in  e-mail  discussions; 
2)  their  opinions  on  the  effect  that  the  listserv  assignments  had  on  their  learning  of  course  content;  3)  a self- 
assessment  of  their  level  of  computer  experience  prior  to  the  course;  4)  a self-assessment  of  their  level  computer 
experience  after  the  course;  5)  their  assessment  of  the  negative  features  of  electronic  discussion  groups,  and  6)  their 
assessment  of  the  positive  features  of  electronic  discussion  groups. 

The  results  indicated  that  approximately  67%  of  the  students  surveyed  participated  in  e-mail  discussions  on  a 
weekly  basis,  13%  on  a daily  basis,  and  7%  on  a monthly  basis.  13%  of  the  respondents  reported  that  they  did  not 
participate  in  the  e-mail  discussions  at  all.  Among  those  that  participated  in  the  electronic  discussion  groups  on  a 
weekly,  daily  or  monthly  basis,  the  majority,  70%,  reported  that  e-mail  discussions  had  a moderate  influence  on 
learning  the  course  content.  15%  reported  that  the  e-mail  discussions  had  very  little  influence  on  learning  the  course 
content  with  the  remaining  1 5%  indicating  that  e-mail  discussions  had  a substantial  influence  on  learning. 

Among  those  students  who  participated  in  the  e-mail  discussions  on  a weekly  basis,  the  majority,  70%,  reported  that 
the  discussions  had  a moderate  influence  on  learning.  20%  reported  that  the  discussions  had  a substantial  influence 
on  learning  with  the  remaining  10%  indicating  that  the  discussions  had  very  little  influence  on  learning. 

Among  those  students  who  participated  in  e-mail  discussions  on  a daily  basis,  50%  indicated  that  the  discussions 
had  a substantial  influence  on  learning  the  course  content.  The  remaining  50%  indicated  that  the  discussions  had  a 
moderate  influence  on  learning.  In  other  words,  all  of  the  students  in  this  group  indicated  that  the  discussions  had  at 
least  a moderate  degree  of  influence  on  learning. 

All  of  the  students  who  participated  on  a monthly  basis  indicated  that  the  discussions  had  a moderate  influence  on 
learning  the  course  content. 
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The  students  were  asked  some  open-ended  questions  to  assess  their  perceptions  of  the  advantages  and  disadvantages 
of  electronic  discussion  groups  in  the  classroom.  Inconvenient  access  to  computers  with  Internet  connections  was 
the  least  liked  aspect  of  using  e-mail.  Students  also  indicated  that  they  felt  overwhelmed  by  the  number  of  e-mail 
messages  they  were  receiving  each  week.  This  was  particularly  true  among  those  students  who  were  on  several  class 
lists.  Some  students  noted  that  there  was  too  little  discussion  of  the  pedagogical  benefits  of  electronic  discussion 
groups.  Consequently,  at  the  beginning  of  the  semester  they  failed  to  understand  why  in-class  discussions  were 
being  supplemented  with  e-mail  discussions.  It  was  only  after  participating  in  the  electronic  discussions  for  a few 
weeks  that  the  students  began  to  understand  and  appreciate  their  value.  Finally,  some  students  lacked  experience 
using  a listserv  and  expressed  a fear  of  this  technology.  The  students  noted  that  their  participation  in  e-mail 
discussions  would  have  been  greater  if  they  had  received  some  remedial  assistance  with  the  technology  being  used. 

One  benefit  mentioned  by  several  of  the  students  was  that  electronic  discussions  provided  a convenient  means  of 
interacting  with  peers  as  well  as  the  instructor.  They  appreciated  being  able  to  send  and  reply  to  an  e-mail  message 
at  a time  of  their  choosing.  Students  also  noted  that  it  was  easier  to  collaborate  with  their  peers  on  group  projects. 
The  frustration  caused  by  trying  to  arrange  convenient  meeting  times  for  a group  of  students  to  physically  meet  and 
work  on  a project  are  largely  eliminated  with  electronic  discussion  groups. 

A few  of  the  students  commented  that  they  were  more  comfortable  with  written  rather  than  oral  discourse.  They 
stated  that  they  felt  more  willing  to  make  provocative  assertions  during  e-mail  discussions  and  to  challenge  the 
views  of  their  peers  and  the  instructor.  In  short,  they  felt  liberated  by  this  new  medium.  However,  a couple  of 
students  did  indicate  that  had  they  been  able  to  maintain  their  anonymity  they  would  have  been  more  willing  to 
challenge  the  viewpoints  of  their  peers.  This  is  consistent  with  research  showing  that  people  are  more  daring  and 
confrontational  regarding  the  expression  of  ideas  when  allowed  to  remain  anonymous  (Lauzon,  1 992). 

Conclusion 

Based  on  the  survey  results,  electronic  discussion  groups  do  appear  to  enhance  learning  of  course  content.  This  is 
especially  true  among  those  students  who  reported  that  they  participated  in  e-mail  discussions  on  a daily  basis. 
Therefore,  instructors  should  consider  incorporating  electronic  discussion  groups  in  their  classes  and  using  strategies 
for  encouraging  students  to  participate  in  the  discussions  on  a regular  basis.  To  avoid  the  problem  of  students  feeling 
overwhelmed  by  the  number  of  postings  made  to  the  list,  a limit  should  be  placed  on  the  size  of  the  discussion 
group.  In  large  classes,  several  discussion  groups  could  be  used. 

Critical  to  getting  students  to  be  active  participants  in  electronic  discussions  is  to  provide  them  with  an 
understanding  of  the  pedagogical  benefits  of  this  learning  tool.  A few  of  the  students  surveyed  indicated  that  they 
did  not  understand  the  need  for  the  listserv,  which  may  have  resulted  in  a lower  level  of  participation. 

Consideration  should  be  given  to  the  students’  level  of  expertise  with  communication  technologies.  The  survey 
found  that  several  students  were  unfamiliar  with  listservs  and  would  have  appreciated  some  training  in  their  use. 
Providing  this  training  at  the  beginning  of  the  course  would  help  ease  these  students’  fear  of  these  technologies  and 
increase  their  level  of  participation  in  the  electronic  discussions. 

Those  considering  incorporating  communication  technologies  in  their  courses  should  also  be  mindful  of  the 
difficulty  students  may  have  getting  access  to  computers  connected  to  the  Internet.  Ideally,  all  students  would  have 
their  own  computers  that  would  provide  access  to  the  Internet.  However,  it  is  more  realistic  to  assume  that  at  least  a 
few  students  will  be  wholly  dependent  on  the  institution’s  computer  resources.  Should  access  to  computers  on 
campus  be  a problem  for  students,  it  may  not  be  possible  to  make  participation  in  electronic  discussions  a 
requirement  in  the  course. 

Finally,  in  deciding  whether  to  use  electronic  discussion  groups  it  is  important  to:  1)  identify  the  best  teaching  and 
learning  strategies  for  the  class,  especially  those  that  would  not  be  feasible  without  electronic  discussion  groups,  and 
2)  identify  which  technologies  are  best  for  supporting  those  strategies.  It  may  be  found  that  electronic  discussion 
groups  would  be  poorly  suited  to  support  the  teaching  strategies  being  used.  There  may  be  other  technologies,  even 
conventional  ones,  that  would  are  better  suited  for  carrying  out  a particular  teaching  task. 
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Introduction 

“Emerging  approaches  to  developing  rich  learning  environments  combine  multimedia,  computer  mediated 
communication,  and  a host  of  interactive  strategies  to  connect  people  in  varied  and  robust  ways.” 

(Schwier,  1999) 

The  virtual  learning  community  can  be  the  key  to  opening  the  door  to  the  world  for  the  learner. 
Instructional  technology  is  a powerful  tool  for  students  to  communicate  within  a global  community  and 
develop  knowledge,  skills  and  attitudes  about  the  world.  Students  can  electronic  communication  tools, 
Internet  resources,  and  multimedia  documents  to  research  information,  develop  inquiry  skills,  or  discuss 
diverse  viewpoints.  Communication  tools  can  link  professionals,  educators,  and  students  interested  in 
social  studies,  art  or  music.  Projects  can  be  developed  for  students  in  different  settings  to  work 
collaboratively  and  internationalize  the  curriculum.  Designing  global  multimedia  projects  can  enrich  the 
art,  music  and  social  studies  knowledge  base  and  broaden  their  global  perspective.  Through  global 
methods  such  as  these  and  regular  integration  of  technology  a community  of  learners  can  be  developed 
(White,  1996). 


The  Model 

This  web-based  project  was  initiated  in  the  fall  of  1 997  to  meet  the  Principles  and  practices  of  the 
Undergraduate  Teacher  Development  Center  in  the  College  of  Education  at  the  University  of  Missouri- 
Columbia  framework  for  diversity  and  technology  (1995).  The  technology  benchmark  is;  “Effective 
teachers  use  technology  to  participate  in  a community  for  professional  development,  to  engage  in 
personally  meaningful  inquiry,  and  to  support  teaching,  learning,  and  assessment.”  The  diversity 
benchmark  is:  “Effective  teachers  encourage  students  to  value  commonalties  and  differences  of  our 
pluralistic  world.” 

The  project  emphases  promoting  and  facilitating  internationalizing  the  curriculum  and  uses 
technology  as  an  integral  component  in  the  learning  process.  This  project  is  an  effort  to  implement  the 
college’s  framework  by  designing  undergraduate  curriculum  to  increase  global  perspectives  and 
technology  as  they  relate  to  art,  music  and  social  studies.  Each  term  approximately  sixty  prospective 
elementary  teachers  are  enrolled  in  the  elementary  art,  music,  and  social  studies  methods  block  in  which 
this  model  is  implemented. 
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The  framework  for  the  model  includes: 


• Using  technology  is  an  integrated  component  in  internationalizing  of  the  curriculum  of  art,  music  and 
social  studies 

• Developing  Internet  resources  for  internationalizing  the  in  art,  music  and  social  studies  curriculum 

• Accessing  electronic  communication  tools  to  discuss  internationalizing  the  curriculum 

• Infusing  technology  into  internationalizing  the  assignments,  class  experiences  and  electronic  portfolio 
in  art,  music  and  social  studies 

The  performance  expectations  for  students  in  the  project  are: 

• Use  their  e-mail,  a class  discussion  group,  a newsgroup,  and  online  forms.  Each  of  these  forums  for 
communication,  inquiry,  and  reflection  support  the  articulation  of  ideas,  diverse  viewpoints,  and 
creative  problem  solving. 

• Design  interdisciplinary  units  of  study,  lessons  and  children’s  museums,  which  infuses  technology 
into  the  internationalized  curriculum.  Within  these  assignments  the  students  are  designing 
multimedia  projects  and  using  interactive  strategies  for  learning. 

• Building  a model  for  discovering  quality  Internet  resources  for  internationalizing  the  art,  music  and 
social  studies  curriculum 

• Disseminating  the  projects  in  local  elementary  schools 

This  report  is  on  the  development  of  the  program  and  the  delivery  systems  for  the  infusion  of 
technology  into  the  internationalizing  of  the  curriculum.  As  an  ongoing  project,  the  curriculum  is 
being  refined  based  on  feedback  each  semester  from  instructors  and  students. 

Conclusions 

This  model  will  stimulate  an  awareness  of  possible  means  for  delivering  an  integrated 
internationalized  curriculum  through  technology.  The  world  is  at  the  fingertips  of  any  learner  when 
quality  Internet  sites  are  woven  into  the  curriculum.  In  a virtual  learning  environment  the  learner  can 
build  a model  of  discovery  to  find  exemplary  resources.  Students  can  develop  international  attitudes  and 
views  through  online  communication  tools,  collaborate  with  students  around  the  world  on  projects, 
research  new  bodies  of  knowledge,  and  design  global  multimedia  projects. 

One  student’s  response  represents  an  example  of  how  students  in  the  project  make  the  transfer 
from  the  integration  of  technology  and  meeting  the  needs  of  diversity  in  an  elementary  school. 

“I  think  as  future  teachers  we  need  to  know  how  to  use  the  technology  because  many  of  our  future  students 
will  be  more  knowledgeable  about  technology  than  me.  Technology  communication  allows  students  to 
correspond  with  other  children  around  the  world.  With  the  ever  increasing  diversity,  children  need  to  be 
able  to  talk  with  other  students  that  are  not  exactly  like  themselves.” 

As  instructors  and  prospective  teachers  make  use  of  technology  as  an  integrated  component  of 
their  daily  life,  then  we  can  discover  new  methods  for  combining  the  art,  music  and  social  studies  and 
infusing  diversity  and  technology.  The  shift  in  the  pedagogical  infusion  of  technology  and  diversity  are 
key  to  the  success  of  teacher  education. 
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Abstract:  There  are  a great  many  books  introducing  computer  science,  yet  few  use  the  computer 
itself  as  a dynamic  instructional  tool.  Interactive  multimedia  opens  a door  for  students  whose 
learning  styles  lean  more  to  the  concrete,  participative  approach,  rather  than  the  typically  more 
abstract,  reflexive  approach  of  most  computer  science  textbooks.  Our  experience  confirms  our 
approach,  by  way  of  significantly  improving  final  examination  scores.  We  also  introduce  our 
plans  for  a more  constructive,  inquiry-based  approach  to  multimedia  education. 


We  have  written  yet  another  textbook  for  the  first  semester  course  on  computer  science  (CSO  or  CSl),  which 
WCB/McGraw-Hill  published  in  1998.  It  is  accompanied  by  a multimedia  CD-ROM.  Media-rich  material  makes 
computer  science  accessible  to  a broader  audience  and  stretches  what  an  introductory  course  can  cover.  There  are 
still  relatively  few  CSl  books  that  emphasize  the  breadth  of  computing  as  an  intellectual  discipline^  rather  than  as 
just  programming.  Nevertheless,  CSl  also  requires  programming  in  languages  such  as  C++.  Our  multimedia 
presents  the  dynamic  aspects  of  program  structure  and  provides  online  help  driven  by  a student’s  errors.  Though 
C++  is  now  the  most  popular  language  for  introductory  courses,  its  complexity  poses  certain  disadvantages  as  well. 
We  have  therefore  focused  on  a subset  of  C++  (which  we  call  C++--),  together  with  a integrated  programming 
environment  catering  to  beginners.  Finally,  we  believe  it  is  essential  to  teach  problem-solving  strategies  before 
taking  up  programming— especially  before  the  student  dives  into  a programming  language  as  complex  as  C++! 
The  multimedia  helps  students  visualize  and  experiment  with  analytical  and  analogical  problem  solving  strategies. 
The  Universal  Machine  therefore  has  three  major  goals,  followed  in  the  structure  of  both  book  and  multimedia: 

1.  To  emphasize  the  breadth  of  computing  as  a discipline.  A narrow  concentration  on  programming  can  lead  to 
a misconception  that  computer  science  is  little  more  than  programming.  True,  computer  scientists  use 
programming  as  an  experimental  tool,  but  like  other  scientists,  they  emphasize  abstraction.  They  also  use 
mathematical  techniques  to  study  computing  theory  as  well  as  engineering  techniques  to  improve  the  design  of 
effective  systems.  (Denning  1988  emphasized  these  three  approaches  to  the  study  of  computing:  abstraction, 
theory,  and  design.)  We  follow  the  recommendations  of  Tucker  1991  by  introducing  a broad  treatment  of 
computing  as  an  intellectual  discipline,  with  topics  ranging  from  computer  architecture  to  artificial 
intelligence. 

2.  To  encourage  students  to  think  about  and  practice  programming  as  systematic  problem  solving.  An  early 
chapter  examines  three  alternative  approaches  to  problem  solving:  hacking  (illustrating  the  difference  between 
blind  and  informed  hacking  with  the  well-known  fox-goose-com  problem),  analytical  (which  includes  top- 
down  decompositon),  and  analogical  (ranging  from  copy  and  paste  to  object-oriented  reuse  and  inheritance). 
Early  on,  students  practice  these  approaches  with  a simple  programming  environment  called  Knobby  *s  World. 
“Knobby  the  Knowbof  ’ is  a simple  agent  that  moves  and  picks  up  objects  in  a micro  world.  Using  a graphical 
simulation  program,  students  first  learn  to  program  Knobby  by  issuing  simple  commands,  then  by  progressing 
to  defining  and  calling  macro-instructions,  and  finally  by  writing  programs.  Knobby  is  a relative  of  Karel  the 
Robot  (Pattis  1981).  The  purpose  of  Knobby  is  in  part  much  the  same  as  that  of  Karel— to  serve  as  a “gentle 
introduction  to  the  art  of  programming,”  though  Knobby  is  more  oriented  to  teaching  C++  than  LISP.  Later 
chapters  studying  the  software  life  cycle  emphasize  the  systematic  use  and  reuse  of  functions  and  classes. 

3.  To  provide  an  introduction  to  computer  programming  that  gives  an  adequate  background  for  a continuation  to 
a second-semester  course  suitable  for  computer  science  majors,  without  disenfranchising  those  students 
without  previous  computer  experience.  Because  C++  (unlike  Pascal,  BASIC  or  LOGO)  was  not  intended  as 
instructional  language,  it  can  be  rather  daunting  for  beginners.  We  therefore  concentrate  on  a subset  of  this 
very  rich  language,  which  we  dub  C+  + --.  The  multimedia  CD-ROM  includes  a tool  called  Lookout  which 
will  “look  ouf’  for  errors  in  C++--  programs.  Beginners  typically  have  difficulty  understanding  the  error 
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messages  that  modem  compilers  emit.  LOOKOUT  provides  detailed  explanations  of  error  messages.  Moreover, 
the  multimedia,  after  presenting  expository  and  interactive  material  on  C++,  invites  the  student  to  invoke 
Lookout  to  study  sample  programs  and  develop  simple  programs.  Thus  the  student  learns  abstract 
presentation  (in  both  text  and  audio  mode)  interactive  exercises  in  the  multimedia  itself,  then  by  example  and 
problem  solving  in  the  integrated  LOOKOUT  environment.  LOOKOUT  also  supports  a small  library  of  software 
“classes”  which  lets  students  solve  a variety  of  problems  (especially  non-numerical  ones)  using  simple 
graphical  shapes,  musical  notes,  external  files,  and  strings  and  arrays  that  catch  “out  of  bounds”  errors.  For 
those  who  want  to  go  beyond  the  C++  - - subset,  there  are  optional  “Beyond  C++-  -”  sections  at  the  end  of 
C++  chapters  in  the  book. 

Creating  The  Universal  Machine,  was,  to  put  it  mildly,  a challenge.  Early  on,  while  writing  proposals  to 
publishers,  talking  to  people  with  more  experience  than  we  then  had  in  developing  multimedia,  we  began  to  talk 
about  our  “Greek  chorus.”  They  warned  us  gloomily  about  the  inevitable  pitfalls  and  problems  we  would 
encounter  and  advised  us  to  increase  our  budget.  Sure  enough,  we  did  encounter  many  snafus,  personnel  changes, 
miscommunications,  conflicting  priorities,  and  learning  curve  issues.  It  took  two  and  a half  years  to  bring  out  the 
first  edition  of  the  book  with  multimedia.  A year  and  a half  later  we  released  a second  version  of  the  CD-ROM, 
cleaning  up  many  bugs,  adding  multimedia  material  for  two  chapters  which  we  were  not  able  to  include  in  the  first 
edition,  and  giving  the  user  more  control  over  the  interface. 

The  initial  user  interface  put  the  user  at  the  console  of  a space  ship  and  navigated  into  one  of  three  galaxies  (the 
three  major  themes)  and  thence  into  star  systems  or  planets  (chapters).  The  analogy  of  this  interface  is  that 
exploring  outer  space  is  like  exploring  abstract  problem  space.  As  we  expected,  many  beginning  students  enjoy 
this  interface.  However,  many  sophisticated  students,  instructors  (who  make  textbook  decisions)  and  our 
publisher’s  marketing  staff  (alas!)  thought  it  was  “juvenile,”  unfortunately  misleading  them  about  the  level  of  most 
of  the  content.  So  we  have  developed  an  alternative  interface,  presenting  themes  and  chapters  in  text  rather  than 
galactic  graphics,  with  rollovers  bringing  up  summaries  of  each  chapters.  The  user  can  choose  between  either 
interface  at  installation  and  switch  between  them.  The  user  can  also  choose  to  disable  sound,  have  the  presentation 
stop  after  each  screen  (by  default  it  advances  automatically,  which  most  of  our  students  seem  to  prefer),  skip  the 
opening  and  credit  sequences,  etc. 

While  developing  The  Universal  Machine,  the  lead  author  taught  CSl  (to  a mixed  audience  of  majors  and  non- 
majors) at  Lehigh  University,  about  70-80  students  each  year.  From  the  first  year,  for  which  only  the  manuscript 
of  the  book  was  available,  to  the  second,  in  which  an  incomplete  version  of  the  multimedia  was  introduced,  mean 
final  examination  scores  improved  about  six  points.  Improvements  were  especially  notable  in  coverage  of  breadth 
topics.  In  the  third  year,  once  the  complete  first  edition  was  available,  the  mean  final  examination  scores  improved 
another  seven  points.  This  time  there  was  significant  improvement  in  the  programming  problems,  suggesting  that 
integration  of  LOOKOUT  with  the  multimedia  was  indeed  crucial.  We  expect  further  improvement  this  year,  with 
the  introduction  of  two  new  chapters  and  many  bug  fixes. 

We  now  envision  an  avatar-based  model  of  learning,  in  which  students  would  interact  with  multimedia  personae 
modeling  the  experience  of  learning  at  different  levels.  The  Universal  Machine  features  a narrator,  who  speaks 
text  in  a professional  voice,  but  otherwise  has  no  identifiable  persona,  and  also  a cartoon-like  character  named 
"Knobby"  who  pops  up  from  a pedestal  to  ask  the  user  to  do  exercises  or  to  dialog  with  the  narrator  in  the  role  of  a 
learner.  A lesson  learned  from  this  first  attempt  is  that  while  less  sophisticated  users  enjoy  interacting  with 
Knobby,  more  sophisticated  students  find  him  juvenile.  We  envision  designing  several  different  characters, 
representing  learners  with  whom  a wider  range  of  students  can  identify,  as  well  as  different  teachers,  such  as  a 
professor,  graduate  teaching  assistant  and  undergraduate  apprentice  teachers.  These  avatars  would  model  students 
and  teachers  studying  expository  multimedia  material  together,  as  well  as  guiding  students  though  interactive 
exercises,  ranging  from  point-and-click  quizzes  to  exploratory  research  using  online  information,  such  as 
Compendex  and  INSPEC. 
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Abstract:  In  the  last  few  years,  many  different  research  areas  have  raised  from  CSCW.  However, 
we  can  notice  a general  goal  shared  by  all  of  those  domains:  create  more  user  centered  groupware, 
i.e.  design  system  better  responding  to  user’s  needs.  Our  experience  in  CSCL  system  design 
teaches  us  that  those  needs  cannot  be  totally  pre-defined  but  seem  to  emerge  from  user’s  activity. 
This  involves  reformulating  the  problem:  we  have  to  design  CSCL  systems  supporting  non  pre- 
defined but  emerging  user  needs.  Work  based  on  Activity  Theory  helped  us  in  creating  a new 
CSCL  environment  called  DARE  (Distributed  Activities  in  a Reflective  Environment)  designed  for 
this  dare. 


Introduction 

The  observation  of  the  evolution  of  real  uses  by  advanced  educational  actors  (pedagogical  designers,  tutors)  shows 
that  they  have  a light  planning  of  the  activities:  they  start  with  a pattern  of  proposed  activities  but,  during  the 
process,  they  have  an  opportunistic  behavior,  revising  and  evolving  their  instructional  strategies  and  learning 
activities  according  to  the  learner  population  reactions  and  progresses.  It  appears  clearly  that  they  need  more 
evolving  CSCL  systems.  Below,  we  present  the  design  properties  of  DARE  (Distributed  Activities  in  a Reflective 
Environment),  an  environment  for  groupware  support.  DARE  emphasizes  on  co-construction  and  expansiveness 
(Ruutti  1991)  (Bardram  1998)  properties  of  activity.  DARE  is  a meta  groupware  that  takes  elements  coming  from  both 
social  science  and  computer  science  for  its  design.  Roots  of  this  model  are  coming  from  Activity  Theory. 


Key  Concepts  for  design 

Activity  Theory  (AT)  has  been  identified  as  a hot  issue  for  HCI  and  CSCW  over  the  10  last  years,  due  to 
contributions  from  (Engestrom  1987),  (Kuutti  1991)  or  (Bodkers  1991).  We  will  not  expose  here  all  AT.  However,  we 
would  like  to  present  some  elements  we  found  essential  to  understand  DARE’s  philosophy.  First  and  according  to 
Kuutti  (Kuutti  1991),  we  define  activity  as  our  basic  unit  of  analysis.  Secondly,  activity  is  mediated  by  artifacts  and 
we  use  Engestrom’s  basic  structure  of  activity  (Engestrom  1987)  model  to  create  our  conceptual  model.  Finally,  we 
accept  that  elements  appearing  in  this  model  are  creating  a situation  that  influences  activity  and  that  they  are 
subjects  to  modification  because  activity  always  modifies  its  situation  in  a reflective  fashion. 

The  concepts  we  use  for  our  conceptual  model  are  the  object  (the  motive  of  activity),  the  subject  (or  actor),  the 
community  (a  set  of  subjects  sharing  the  same  object),  the  tool  (mediating  the  subject  and  object  relationship  ; 
anything  used  in  the  transformation  process),  the  rules  (mediating  the  subject-community  relationship  ; explicit  and 
implicit  laws,  accepted  practices...)  and  the  division  of  labor  (mediating  the  community-object  relationship  ; explicit 
and  implicit  organization  of  a community).  We  have  decided  to  synthesize  the  two  last  concepts  in  only  one  named 
role.  Rules  it  represents  strongly  influence  actions  that  subject  may  perform  during  activity  by  defining  an  evolving 
set  of  rights.  Role  mediates  the  relationship  between  community  and  object  by  determining  what  each  role  existing 
in  community  has  to  do  to  reach  the  object  of  activity,  this  corresponding  to  the  division  of  labor. 

Bedny  (Bedny  & Meister  1997)  mentions  that  task  is  the  basic  component  of  activity.  According  to  (Leont’ev  1977), 
task  is  “a  situation  requiring  realization  of  a goal  in  specific  conditions*'.  AT  argues  that  situation  is  never  given  in 
advance  and  is  the  result  of  interactions.  Activity  starts  with  an  initial  situation  given  in  the  task.  This  situation  is 
shared  by  all  subjects  involved  in  activity;  it  is  transformed  by  task  performance  and  concludes  with  a final  situation. 
For  our  design,  we  accept  that  activity  is  created  according  to  a given  task  defining  the  object  of  activity,  the  tools  to 
be  used  and  the  roles.  However,  activity  tends  to  modify  its  task  until  the  end. 


1277 


Open  Implementation 


Usually,  CSCW  systems  implement  a specific  task  that  has  been  constructed  thanks  to  a task  analysis. 
Unfortunately,  breakdowns  often  appear  generally  due  to  a shift  between  system’s  embedded  task  and  user’s  one  (or 
the  representation  he  has  constructed).  If  user  is  unable  to  understand  the  task  the  system  really  implements,  or  to 
adapt  it  to  its  one,  then  appropriation  does  not  happen.  Our  meaning  is  then  to  allow  users  to  access  to  the  task  in 
order  to  help  in  understanding  the  real  task  behind  the  system  and,  if  needed,  to  modify  it.  This  corresponds  to  the 
Open  Implementation  approach  described  by  (Kiczales  1996).  An  existing  technique  for  Open  Implementation  is 
Meta  Object  Protocol  (MOP)  (Kiczales  et  al.  1991)  that  helps  in  providing  system  interfaces  for  examination  and 
modification,  in  other  words,  providing  a meta  interface.  We  have  used  MOP  technique  to  implement  DARE.  MOP 
defines  a meta  level  description  of  the  system  that  is  accessible  to  users.  It  allows,  through  a meta  application,  the 
examination  {introspection)  and  manipulation  {intercession)  of  the  system  during  its  own  execution  while 
maintaining  a causal  connection  between  this  execution  and  its  meta  level.  Our  meaning  is  to  apply  those  concepts 
in  order  to  create  a reflective  system  based  on  task  and  activity. 

In  languages  implementing  MOP,  the  meta  level  corresponds  to  the  classes  and  the  execution  to  their  instances.  In 
DARE,  that  is  realized  in  Distributed  Smalltalk  & Java,  everything  concerning  task  is  traduced  in  term  of  class,  and 
activity  in  term  of  instance.  This  correspondence  takes  advantages  of  reusing  the  reflective  properties  of  the 
languages  that  are  used.  This  allows  to  modify  the  task  during  the  corresponding  activity’s  execution.  Task  itself  is 
an  artifact  manipulated  by  subjects  according  to  their  role.  Task  is  part  of  and  is  modified  by  activity  it  defines  in  a 
causally  connection.  Accessing  to  task  is  a meta  activity  in  which  subjects  may  modify  their  working  situation  in  a 
cooperative  way.  This  offers  a support  for  activity  implementing  concepts  and  mechanisms  as  described  in  AT. 


Conclusion 

This  paper  has  briefly  presented  how  we  have  used  a simple  model  and  mechanisms  coming  from  Activity  Theory 
to  design  a meta  groupware  that  helps  users  to  define  their  own  activities.  This  meta  groupware  is  named  DARE. 
However,  Activity  Theory  teaches  us  that  activity  is  reflective  in  the  sense  that  it  is  co-constructed  and  expansive. 
This  is  why  DARE  is  more  than  a meta  groupware  but  a reflective  groupware  allowing  users  to  access  the  meta  level 
of  their  activity  during  its  execution. 

During  the  following  months,  DARE  is  going  to  be  used  for  experimentation  in  CSCL  domain.  The  main  idea  is  to 
provide  to  users  (students  and  teachers)  a bootstrap  CSCL  activity  and  to  study  how  they  use  DARE  reflective 
properties  to  adapt,  evolve  and  co-construct  this  CSCL  environment.  This  possibility  to  allow  self-organization 
inside  a learner  group  and  co-construction  of  the  collaborative  learning  activities  is  seen  for  us  as  an  important 
contribution  to  self  reflection  activities. 
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Abstract:  This  paper  reports  on  an  initial  and  on-going  study  into  the  use  of  haptic  (Schiff  & 
Foulke,  1982)  technology  in  veterinary  education.  The  University  of  Glasgow  has  a commitment  to 
the  implementation  of  non-invasive  procedures  for  diagnostic  and  education  purposes  in  Veterinary 
Medicine,  wherever  possible.  As  part  of  the  continued  development  in  the  use  of  technology  in 
veterinary  education,  Virtual  Reality  is  now  being  explored  as  a possible  aid  to  teaching  and  in 
particular  to  replace  invasive  examination  procedures. 


THE  PROBLEMS 

One  major  problem  in  the  education  of  veterinary  students  is  the  danger  faced  by  animals  when  being  examined  by 
inexperienced  students.  The  students  need  to  gain  experience  in  internal  examinations  (a  key  method  for  diagnosing 
problems  and  diseases)  but  it  can  be  dangerous.  The  animals  may  become  stressed,  be  injured  or  may  even  die 
because  of  unskilled  internal  examinations.  Large  classes  of  students  mean  that  each  person  may  only  get  a very 
limited  amount  to  time  learn  the  practical  examination  skills  required.  Another  problem  is  that  the  students  must 
learn  about  a whole  range  of  diseases  and  problems  as  part  of  their  education.  At  the  time  during  their  training  when 
they  are  learning  about  a particular  disease  there  may  be  no  animal  available  with  the  particular  disease  in  question. 
This  means  that  students  may  not  be  able  to  consolidate  their  learning  with  practical  experience. 


THE  SOLUTIONS 

We  are  investigating  ways  to  solve  these  problems  by  using  haptic  devices  that  allow  users  to  feel  virtual  objects. 
There  are  three  main  benefits  from  our  solution: 

Safety:  The  use  of  haptic  models  to  simulate  problems  allows  the  students  to  learn  in  a safe  environment.  They  can 
learn  on  the  models  without  endangering  any  animals.  Once  they  have  gained  sufficient  skills  on  the  models  they  can 
then  move  on  to  the  live  animals  with  much  less  chance  of  doing  harm. 

Cost:  The  cost  of  education  is  also  reduced.  A large  number  of  students  can  interact  with  the  haptic  models  very 
quickly  and  cheaply.  They  can  also  do  this  much  more  frequently  than  would  be  possible  with  live  animals. 
Flexibility:  The  flexibility  of  the  models  is  great.  It  is  possible  to  simulate  a range  of  diseases  that  the  students  would 
not  normally  experience.  Different  stages  in  the  progress  of  a disease  or  condition  can  also  be  simulated.  This  can  be 
done  at  the  time  the  students  are  learning,  allowing  them  to  try  out  their  theoretical  knowledge  immediately  in  a 
practical  setting. 


THE  TECHNOLOGY 
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The  technology  to  feel  virtual  objects  is  just  now  becoming  available  (Massie  & Salisbury,  1994,  Bryson,  1995).  It 
was  first  developed  so  that  users  could  feel  objects  in  virtual  environments.  Minsky  (in  Blattner  & Dannenberg, 
1992)  describes  the  technology  thus:  ’’Force  display  technology  works  by  using  mechanical  actuators  to  apply  forces 
to  the  user.  By  simulating  the  physics  of  the  user’s  virtual  world,  we  can  compute  these  forces  in  real-time,  and  then 
send  them  to  the  actuators  so  that  the  user  feels  them”.  The  device  used  for  the  work  described  is  a PHANToM.  This 
is  a very  high  resolution,  six  degrees-of-freedom  device  in  which  the  user  puts  his/her  finger  in  a thimble  or  holds  a 
pen  at  the  end  of  a motor-controlled,  jointed  arm.  It  provides  a programmable  sense  of  touch  that  allows  users  to  feel 
textures  and  shapes  of  virtual  objects,  modulate  and  deform  objects. 


THE  HAPTIC  MODELS 

As  mentioned,  we  are  developing  haptic  software  models  that  will  simulate  the  feel  of  parts  of  a horse  under 
examination.  This  shows  two  horse  ovaries.  The  ovaries  feel  correct  and  they  move  in  appropriate  ways  (if  pressure 
is  applied  they  can  be  moved  in  three  dimensions  in  a realistic  way).  Both  ovaries  have  very  smooth,  taut  surfaces. 
The  left  ovary  is  softer  and  slightly  smaller  than  the  right  one.  A follicle  can  be  seen  on  the  left  ovary  (indicating 
ovulation  is  about  to  take  place).  This  sticks  out  from  and  is  much  softer  than  the  rest  of  the  surface.  The  visual 
representation  of  the  ovaries  is  built  in  OpenGL.  The  haptic  representation  is  built  using  the  GHoST  software  toolkit 
from  SensAble.  When  vets  are  doing  internal  examinations  they  cannot  see  the  ovaries,  so  the  visual  images  are 
mainly  for  illustrative  purposes.  However,  a visual  representation  is  beneficial  when  learning  to  use  the  system.  Our 
software  allows  the  visual  representation  to  be  removed  as  required.  The  PHANToM  device  only  allows  the  user  to 
feel  with  one  finger,  not  the  whole  hand.  This,  however,  is  not  a limitation  as  vets  will  only  use  their  thumb  when 
feeling  the  ovary  in  a real  examination. 


EVALUATION 

The  development  of  the  models  has  been  performed  using  an  iterative,  participatoiy  approach.  Computing  scientists 
have  been  working  closely  with  vets  to  ensure  that  the  models  are  correct.  This  has  involved  the  building  of  initial 
prototype  models  and  their  continual  refinement  via  expert  user  evaluation.  The  next  stage  in  our  work  is  to  cany  out 
evaluations  with  vet  students.  We  will  discover  if  the  models  are  usable:  can  the  vet  students  interact  with  them,  use 
the  device,  etc.  We  will  also  be  able  to  find  out  how  effective  they  are:  can  students  correctly  identity  the  particular 
diseases  and  conditions  we  have  simulated  in  our  models. 


CONCLUSIONS 

One  problem  in  veterinary  education  is  that  students  must  learn  how  to  do  internal  examinations  of  animals  but  in 
doing  so  can  be  dangerous  for  the  animals.  Another  difficulty  is  that  there  may  not  be  an  animal  available  with  a 
particular  disease  when  the  students  are  studying  it.  The  availability  of  haptic  devices  means  that  these  problems  can 
now  be  overcome.  We  have  developed  a set  of  haptic  models  of  horse  ovaries  that  will  allow  students  to  learn  in  a 
safe,  cheap  and  flexible  environment.  Once  they  have  learned  the  necessary  skills  they  can  move  to  working  with 
live  animals  with  much  less  risk  of  danger. 
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Online  Education  in  Schools:  Expectations,  Facts,  and  Fiction 
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Guntram  Geser,  Techno-Z  FH  Research  & Development,  Austria 
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As  institutions  of  knowledge  transfer  and  know-how  generation  schools  are  particularly  effected  by  the 
rapid  spread  and  extended  functionality  of  the  new  information  and  communication  technologies.  The 
expectations  concerning  the  benefits  of  the  integration  of  ICTs  in  schools  are  high.  Yet,  there  are  many 
obstacles  for  a sensible  use  of  these  technologies  in  schools  which  are  not  easy  to  overcome.  The  financial 
capacity  of  schools  to  make  the  necessary  investments  into  hard-  and  software  are  restricted  by  severe 
budgetary  constraints.  Local  networks  and  internet  access  have  to  be  adjusted,  which  is  often  a delicate 
exercise  with  flexible  costs.  Much  of  the  multimedia  content  available  on  the  market  fails  to  serve  the 
pedagogical  needs;  and  - maybe  the  biggest  problem  - the  new  learning  tools  cannot  be  integrated  into 
teaching  practices,  unless  the  teachers  themselves  have  been  trained  adequately  and  are  ready  to  transform 
their  established  working  habits. 

The  paper  will  address  the  bundle  of  expectations  around  the  efforts  schools  are  making  in  order  to  be  part 
of,  if  not  to  say  spearheading,  the  Information  Society.  These  expectations  are  set  in  relation  to  available 
hard  facts  on  the  effects  of  ICT  implementation  in  schools.  What  happens,  if  school  classes  are  equipped 
with  computers  and  get  connected  to  the  Internet?  What  is  the  impact  on  classes,  how  does  this  influence 
the  roles  of  teachers  and  pupils,  and  what  is  the  impact  on  teaching  and  learning  processes?  The  paper  will 
contrast  expectations  with  empirical  facts,  whilst  ’’fictions"  will  be  singled  out  and  realistic  and  manageable 
routes  to  effective  online  education  in  schools  will  be  suggested. 
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Abstract;  This  article  describes  a case-based  undergraduate  business  ethics  team  project 
that  uses  CD  and  web-based  computer  technologies  to  advance  the  moral  decision- 
making skills  of  students. 


Today,  computer  technologies  are  fairly  well  integrated  in  business  school  programs.  They  are  not 
so  firmly  established,  however,  in  the  humanities  and  combination  academic  disciplines  such  as  business 
ethics.  The  following  describes  one  successful  case-based  team  project  that  uses  CD  and  web-based 
computer  technologies  in  an  undergraduate  business  ethics  course.  As  does  business  ethics,  this  project  co- 
joins academic  disciplines.  Unlike  typical  business  ethics  projects,  however,  it  links  computer  technology 
with  business  management  and  ethics. 

An  Elementary  Business  Ethics  Model 

Although  similar  to  other  team  projects  in  its  attempt  to  teach  students  to  describe,  analyze,  and 
resolve  business  ethics  problems,  this  project  is  unusual  for  its  concentration  on  a relatively  young 
undergraduate  population  (first  year/freshman  and  second  year/sophomore  students)  having  little  or  no 
previous  formal  ethics,  business,  or  web  design  training.  The  project  itself  is  simple.  It  merely  has  the 
students  apply  theory,  principles,  and  select  cases  to  a business  ethics  problem.  Although  this  approach  to 
moral  problem  solving  is  not  as  useful  as  a rich  business-specific  casuistry,  it  is  helpful  in  educational 
settings  to  convey  the  underlying  norms  and  issues  at  play  in  practical  circumstances.’ 

In  essence,  the  project  is  a summary  exercise  introduced  at  the  beginning  of  the  course.  Students 
are  assigned  to  four  or  five  member  teams,  encouraged  to  meet  with  the  professor  to  discuss  acceptable 
cases,  and  then  sent  off  to  work  independently.  Students  design  or  select  a case  embodying  a particular 
moral  business  problem.  They  then  analyze  the  case  using  the  theories  and  cases  learned  in  class,  produce  a 
CD-ROM  containing  their  analyses,  and  present  their  findings  to  the  class  at  the  end  of  the  academic  term. 

Technology 

Not  surprisingly,  the  project  has  evolved  over  the  course  of  its  implementation.  Early  on,  we  used 
a series  of  linked  web  pages  to  describe  it.  These  pages  were  to  serve  a dual  purpose:  they  were  to  describe 


' For  more  on  this  distinction,  see  Martin  Calkins,  S.J.  “Casuistry  and  the  Case  Method.”  Business  Ethics 
Quarterly.  Forthcoming. 


the  project  to  students  and  to  be  used  as  a template.  Unfortunately,  the  template  notion  was  lost  on  most  of 
the  students.  To  rectify  this  problem,  we  introduced  a simpler  template  containing  a single  sample  image 
and  basic  navigation  and  e-mail  links.  This  seemed  to  work  better.  We  also  revised  the  description  pages, 
reducing  multiple  pages  to  a single  page.  This  made  the  pages  easier  to  read  and  revealed  how  web  pages 
can  be  simple,  yet  effective  and  user-friendly.  For  early  and  later  versions  of  our  project  description  pages, 
see: 

Early  version:  http://www-acc.scu.edu/^mcalkins/BEI0777Pgl  .html 
Later  version:  http://www-acc.scu.edu/^mcalkins/B El 45 73 Pgl.html 
Template:  http://www-acc.scu.edu/^mcalkins/index.htm 
From  the  beginning,  we  promoted  the  use  of  HTML  because  we  believed  that  HTML  allowed 
students  to  create  and  distribute  rich  and  easy-to-use  multimedia  presentations  over  a variety  of  computer 
platforms  (MAC  or  PC).  We  also  chose  Macromedia’s  Dreamweaver  as  our  web-authoring  tool  because 
Dreamweaver,  unlike  other  web  authoring  tools,  relies  on  traditional  word  processing  functions  already 
familiar  to  students.  Dreamweaver  also  lets  more  advanced  users  include  visual  page  design,  dynamic 
HTML,  and  Java  Script  in  pages  relatively  easily.  In  addition,  we  required  that  students  place  their  work  on 
a CD-ROM.  This  was  to  avoid  university  server  space  problems,  potential  server  inaccessibility,  hassles 
involving  the  promulgation  of  ftp  software,  possible  copyright  violations,  the  necessity  to  establish  student 
web  accounts,  and  so  forth.  At  presentation  time  then,  students  simply  bring  their  CDs  to  class  and  flip 
them  into  a laptop  that  is  hooked  up  to  a large  screen  projector. 

Assessment 

At  present,  we  have  conducted  this  project  over  three  academic  terms.  Our  students’  work  can  be 
viewed  at: 

BE  Team  Projects  Winter  1999:  httD://mcdialab.scu.cdu/mcalkins/winter99/indcx.htm 
BE  Team  Projects  Spring  1999:  http ://medi alab.scu.edu/mcalki ns/spri ng99/i ndex.htm 
BE  Team  Projects  Fall  1999:  http://mcdialab.scu.edu/mcalkins/fall99/indcx.htm 
It  is  our  view  that  the  project  has  many  positive  attributes.  It  is  simple  and  forthright.  It  promotes 
clear  thinking  and  linear,  deductive  reasoning.  It  encourages  students  to  reflect  deeply  on  cases,  to 
concentrate  on  the  gist  of  arguments,  and  to  think  creatively. 

At  the  same  time,  it  may  be  no  more  effective  than  traditional  paper-based  projects  in  teaching 
business  ethics  and  team  participation.  To  date,  it  has  always  been  implemented  alongside  a traditional, 
paper-based  project.  Each  time,  paper  and  CD/web-based  projects  were  of  similar  quality  in  terms  of  the 
students*  ability  to  apply  theory  and  principles  to  situations.  Interestingly,  student  evaluations  reveal  that 
two  out  of  three  times  students  expend  the  same  or  less  effort  on  CD/web-based  projects  than  they  do  on 
paper-based  projects  (see  Table  1).  This  finding  initially  surprised  us,  but  makes  sense  given  some  of 
today’s  students’  familiarity  with  computer  technology. 


Table  1 : Student  Evaluations:  Rated  1 (definitely  false)  to  5 (definitely  true) 


Question 

PaperAVeb_Wlnter-99 

Paper/Web_Sprine-99  1 

1 Paper/Web  Fall-99 

1 worked  harder  on  this  course  than 
on  most  courses  1 have  taken. 

3.55/3.5 

4.06/3.79 

.V61/3.76 

1 found  this  course  more  difticult 
than  most  courses  1 have  taken. 

3.58/3.4 

4.1.1/4.12 

3.52/3.79 

Conclusion 

In  the  end,  we  recommend  this  CD/web-based  project  because  we  believe  that  it  successfully 
integrates  three  factors  essential  to  the  success  of  technology-based  learning  projects.  It  (I)  facilitates 
cooperation  between  the  course  instructor  and  the  technology  specialist,  (2)  keeps  course  goals  before  those 
associated  with  technology,  and  (3)  fosters  a supportive  environment  for  student  creativity.  We  recommend 
it,  too,  for  its  ability  to  help  students  see  how  ethics,  management,  and  computer  technology  are  more 
compatible  than  unrelated. 
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At  our  institution  we  have  been  designing  and  developing  interactive  courses  and  course  modules  for  some 
time.  For  example,  recently  we  converted  twelve  one-semester  Distance  Education  courses  from  traditional 
paper,  audio  and  video  to  online  and  we  are  scheduled  to  convert  many  more  in  the  next  few  years.  These 
online  courses  are  each  a one  semester  course  (36  hours  of  instruction)  containing  various  modules  some  of 
which  are  highly  interactive  and  some  of  which  are  not  highly  interactive. 

The  highly  interactive  components,  also  called  Multimedia  Interactive  Learning  Environments  (MILEs) 
(Plowman  et  al.,  1999),  that  we  have  been  building  for  these  Distance  Education  courses  and  other  courses 
typically  cover  1-3  hours  of  the  course  instructional  material.  The  components  are  addressing  instructional 
bottlenecks  - parts  of  the  course  which  students  have  been  struggling  with  year  after  year 

Our  institution  has  been  creating  a development  process  for  high  quality  MILES,  which  optimizes  the  use 
of  faculty  time,  student  engagement  and  instructional  staff  support.  By  structuring  development  around  a 
set  of  incremental  prototypes  and  instructional  design  tools,  the  7 C’s  Model  insures  that  support 
resources  are  applied  at  the  right  time  and  in  the  right  measure  to  sustain  learnware  development.  We  are 
currently  scaling  up  this  process  for  use  across  the  university,  and  developing  partnerships  for  sharing  the 
products  and  the  process  with  other  institutions. 

The  7 C’s  MODEL  has  the  following  components:  (a)  Content  bottleneck  (b)  Course  prototype  (c)  Co-op 
project  (d)  Class  pilot  (e)  Courseware  Exchange  (f)  Commercial  product  (g)  resource  Centre.  The  process 
begins  with  (a)  a content  bottleneck  identified  by  a faculty  member.  This  is  a part  of  the  course  that  has 
students  have  had  difficulty  understanding  year  after  year.  The  faculty  member  feels  that  technology  may 
be  useful  in  overcoming  this  bottleneck.  The  faculty  member  sponsors  a team  of  four  students  who  build 
and  test  a (b)  prototype  learnware  solution  for  course  credit  in  a third  year  project-based  Independent 
Studies  course  called  Designing  Learning  Activities  with  Interactive  Multimedia.  Successful  prototypes 
are  developed  into  (c)  pilot  systems  by  students  on  co-op  work  terms  (normally  with  support  from  the 
university  and  government).  The  resulting  product  from  the  co-op  work  term  is  then  (d)  tested  with  pilot 
classes.  The  best  of  these  learnware  solutions  are  shared  with  other  institutions,  either  as  (f)  commercial 
products  or  through  a (e)  courseware  exchange  where  the  market  is  limited.  All  of  these  activities  are 
supported  by  a (g)  central  resource  group:  in  our  case  it  is  the  Centre  for  Learning  and  Teaching  Through 
Technology. 


Status 

We  have  developed  the  7 C’s  Model  through  pilot  studies  with  14  teams  of  faculty,  students  and  staff 
Five  systems  have  reached  the  class  pilot  stage,  one  is  now  ready  for  commercial  productization,  and  others 
are  working  their  way  through  this  pipeline.  The  process  has  now  been  institutionalized  in  the 
undergraduate  course.  Designing  Learning  Activities  with  Interactive  Multimedia,  which  is  supported  by  a 
Web-based  support  tool,  the  Learner-Centred  Design  Idea  Kit  (Carey,  Harrigan,  Palmer  & Swallow,  1999). 
Two  of  the  main  features  of  the  kit  are  a detailed  learner  profile  section  and  a visualization  of  the  learning 
activities  based  on  Laurillard's  Conversational  Model  (Laurillard,  1993).  The  design  of  the  kit  itself  was 
informed  by  constructivist  educational  literature  and  thus  the  kit  guides  our  novice  designers  through  a 
design  process  which  is  based  on  the  latest  research. 
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An  Example 


Perhaps  this  process  is  best  illustrated  with  an  actual  example.  This  representative  7 C’S  Model  project 
originated  in  Fall’ 98  when  an  Environmental  Studies  professor  identified  an  Instructional  Bottleneck  in  that 
year  after  year  his  students  had  difficulties  appreciating  and  understanding  Life  Tables.  There  seemed  to  be 
two  main  concerns.  First,  the  students  didn’t  seem  to  appreciate  that  life  tables  are  important  in  many 
fields.  Life  tables  are  used  in  many  fields  to  determine  the  life  expectancy  - in  environmental  studies  they 
are  often  used  for  plants  and  animals,  in  accounting  they  are  used  by  actuaries  in  evaluations,  and  in  the 
insurance  business  they  are  used  in  determining  policy  premiums.  Second,  about  half  the  students  did  not 
have  the  math  background  to  easily  understand  the  math  used  in  calculating  the  columns  in  a life  table. 

Students  in  our  course  created  a design  and  produced  a very  primitive  prototype  in  Winter’ 99.  The  design 
was  broken  into  three  main  parts  the  first  two  of  which  addressed  the  lack  of  appreciation  of  life  tables  and 
the  last  of  which  addressed  the  math  issue.  The  first  section  was  a relevance  section  where  video  was  used 
to  go  and  interview  people  who  work  with  life  tables  to  show  students  that  people  in  industry  do  use  life 
tables.  The  second  part  was  field  trips  that  used  video  to  show  people  collecting  data  for  life  tables  in  the 
field  - for  example  one  was  tagging  birds  and  another  was  counting  the  survival  rate  of  trees.  The  third 
section  was  really  the  core  of  the  system  that  allowed  students  to  actually  build  a life  table.  This  section 
had  a lot  of  feedback,  examples,  and  assistance  to  help  learners  who  were  having  difficulties  understanding 
the  math. 

In  Summer’ 99  one  of  the  students  from  the  class  group  and  one  other  were  hired  to  finish  the  project.  The 
resulting  application  is  being  piloted  in  the  Fall’99  semester  with  approximately  100  students  who  are 
currently  taking  the  course.  In  is  anticipated  that  because  Life  Tables  are  used  in  many  disciplines  this 
product  will  be  a good  candidate  for  the  courseware  exchange  or  commercialization.  Even  within  our  own 
institution  we  anticipate  that  other  departments  many  adopt,  or  modify  and  adopt,  this  application  for  their 
use. 

All  throughout  this  process  the  Centre  for  Teaching  and  Learning  Through  Technology  has  provided 
assistance  with  things  such  as:  selection  of  tools,  project  management,  feedback  at  various  stages  in  the 
design/implementation,  and  overall  advise  on  issues  such  as  how  to  perform  the  evaluation. 


Conclusion 

In  presenting  our  model  we  are  showing  the  model  that  seems  to  work  at  our  institution.  We  are  not 
suggesting  at  all  that  this  exact  model  is  the  best  one  for  any  other  institution.  It  is  a model  that  works  and 
thus  others  may  gain  by  exploring  the  model  and  adapting  all  or  parts  of  the  model  for  their  institution. 
Perhaps  the  main  point  is  that  institutions  need  some  such  model  - good  instructional  material  does  not 
come  out  of  thin  air  - it  has  to  be  well  thought  out,  plan,  and  implemented  using  a defined  process.  Also,  it 
should  be  grounded  in  the  research  and  practice  that  is  documented  in  the  literature  (and  which  our  Kit 
supports). 


References 

Carey,  T.T.,  Harrigan,  K.  A.,  Palmer,  A.,  & Swallow,  J.  (1999).  Scaling  up  a learning  technology  strategy:  Supporting 
student/faculty  teams  in  learner-centered  design.  ALT-J,  the  Journal  of  the  Association  of  Learning  Technologies, 
7(2),  15-26. 

Laurillard,  D.  (1993).  Rethinking  university  teaching:  A framework  for  the  effective  use  of  educational  technology. 
London:  Routledge. 

Plowman,  L.,  Luckin,  R.,  Laurillard,  D.,  Stratford,  M.  & Taylor,  J.  (1999).  Designing  multimedia  for  learning: 
Narrative  guidance  and  narrative  construction.  In  Proceedings  of  CHP99  Human  Factors  in  Computing  systems,  pp. 
310-317.  Addison  Wesley. 


WWW-based  Tools  to  Manage  Teaching  Units  in  the  PLAN-G  Distance 

Learning  Platform 


Clara  Ines  Pena  de  Carrillo 
Ramon  Fabregat  Gesa 
Jose  Luis  Marzo  L^aro 
BCDS-DEIA 
University  of  Girona 

Building  PII,  Avinguda  de  Luis  Santalo,  S/N,  17071 
Gerona,  Spain 

{ clarenes, ramon, marzo  }@eia.udg.es 


Abstract:  This  paper  describes  a set  of  tools  to  assist  teachers  to  support  open  and  distance 
learning  by  means  of  the  PLAN-G^ hypermedia  system,  developed  at  the  University  of  Girona 
with  the  aid  of  computer  networks  and,  in  particular,  WWW  technology.  The  web-based  tools 
permit  the  creation  and  edition  of  didactic  material,  the  trmsfer,  oiganization  and  management  of 
document  contents  files,  the  generation  and  management  of  different  types  of  interactive  exercises 
associated  with  a teaching  unit,  and  the  creation  and  management  of  a teaching  unit.  The  current 
PLAN-G  prototype  has  been  successfully  used  on  courses  in  different  educational  fields. 


Introduction 

At  present,  the  PLAN-G  is  a hypermedia  system  that  allows  access  to  learning  material  in  a static  way  (Marzo, 
1998a,  1998b).  We  are  currently  working  to  include  adaptivity  in  it  for  a specific  student  based  on  the  knowledge  of 
a student  model  (Pena,  1999).  The  teacher  builds  a linear  navigational  path  by  defining  a directed  graph  (structure), 
using  the  contents  stored  in  HTML  page  files  (teaching  contents),  and  linking  them  according  to  curriculum 
organization  criteria.  To  make  creation  easy  and  to  manage  the  teaching  domain,  we  developed  some  useful  web- 
based  teaching  contents  authoring  and  managing  tools  (Fabregat,  1999)  such  as  the  contents  generator  JVS,  the 
exercise  generator,  the  contents  organizer,  publisher  and  searcher,  and  the  teaching  units  editor.  This  set  of 
applications  is  included  in  the  PLAN-G  platform  environment.  The  PLAN-G  system  was  designed  as  a learning 
platform,  in  which  the  teacher  supplies  the  course  contents  and  the  students  can  navigate  through  them.  Each  course 
is  called  a “teaching  unif  ’ and  is  made  up  of  a group  of  HTML  page  files  structured  by  a directed  graph.  By  means 
of  the  administration  options  of  the  graph,  it  is  possible  to  create,  modify  and  delete  “teaching  units”,  so  giving  the 
possibility  of  changing  navigation  destinations  or  defining  linear  paths.  The  navigational  structure  is  managed 
independently  of  the  contents  and  is  stored  in  a database.  Only  the  user  type  “teacher”  has  the  power  to  adjust  the 
structure  and  unit  contents,  as  required.  Basically,  the  PLAN-G  works  around  three  concepts:  users,  teaching  units 
and  a performance  level  defined  by  certain  languages  (the  platform  is  multilingual).  The  system  gives  a 
configuration  and  a group  of  sessions  and  visits  (static  and  dynamic  data)  to  each  user.  Each  node  of  the  teaching 
unit  graph  has  an  associated  group  of  basic  exercises  or  bibliography  and  a selected  language  to  fix  the  contents. 


PLAN-G  Teaching  Units  Management 

The  PLAN-G  architecture,  as  a hypermedia  system,  has  general  components  of  generic  intelligent  tutoring  systems. 
It  has  a hypermedia  component  represented  by  an  interface,  a navigation  control  area  (buttons)  and  a navigation 
space.  The  tutor’s  component  adds  intelligent  behaviour  to  the  system  based  on  the  pedagogic  organization  of  the 
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domain  of  learning  and  the  student’s  preferences  and  activities.  The  JVS  Contents  Generator  Editor ^ enables  the 
teacher  to  create  didactic  material  without  having  to  write  a line  of  code  and  with  use  of  only  the  mouse  buttons  or 
input  of  some  data  through  the  keyboard.  With  JVS,  each  didactic  content  is  made  by  a photogram  sequence  or  some 
screen  displays  related  to  each  other,  which  should  include  texts,  lines,  arrows,  squares,  circles  or  images  to  create 
an  interactive  teaching  content  with  graphic  effects  and  animations.  By  creating  interactive  graphics,  the  teacher  or 
the  student  can  perform  multiple  activities  from  the  interactive  exposure  of  an  image.  To  generate  the  exercise 
domain  associated  with  teaching  concepts,  the  Exercises  Generator  was  developed  as  a user  friendly  tool  that 
enables  interactive  closed-answer  exercises  such  as  multiple  choice,  gap-filling,  associations  drag  & drop,  selecting 
the  correct  answer,  etc.  to  be  created  and  managed.  The  exercise  domain  can  be  managed  either  by  the  teacher  or  by 
the  platform  administrator  who  can  create,  import,  export  or  modify  any  exercise,  create  an  HTML  page  with  a 
particular  execution  exercise  independently  of  the  PLAN-G,  move  exercise  parts  from  one  exercise  to  another,  ask 
for  statistics  of  results,  pre-watch  the  exercise  and  include  multimedia  files  to  complete  the  exercise.  The  exercise 
generator  also  acts  in  the  navigation  module  of  the  PLAN-G  structure  by  collecting,  analyzing  and  evaluating  the 
student’s  activities  over  the  proposed  training  or  evaluation  material,  and  storing  this  knowledge  in  the  database. 
Once  the  teacher  has  defined  and  created  the  didactic  material,  it  is  ready  to  be  managed  (organized,  classified  and 
published)  in  the  server.  The  teacher  must  use  the  Documentation  Manager  tool  to  fix  the  files  in  the  server  if  he/she 
has  used  any  standard  HTML  editor  to  make  the  didactic  material,  because,  the  files  made  using  the  JVS  tool  are 
organized  directly  in  the  server  by  the  tool  itself  All  the  essential  information  for  the  documentation  manager  like, 
file  owners,  folders  and  filenames,  folder  locations,  HTML  and  reference  document  words,  is  stored  in  the  PLAN-G 
relational  database;  in  consequence,  any  action  over  any  document  file  that  will  be  used  to  build  a teaching  unit  will 
be  updated  in  the  database  in  order  to  ensure  data  integrity.  The  documentation  manager  also  allows  users  to  publish 
HTML  documents  on  the  network  without  specialist  knowledge  and  to  define  easy  search  spaces  over  them,  without 
any  maintenance  tasks  or  software  installation.  By  means  of  a web-browser,  the  user  may  utilize  the  documentation 
manager  tool  to  interchange  files,  to  classify  them  and  to  design  query  forms  for  different  document  ambits;  this  tool 
is  useful  when  the  user  decides  to  publish  a large  volume  of  documents  and  a rapid  search  through  their  contents  is 
needed.  Once  the  didactic  material  exists  in  the  server  in  its  corresponding  directory  path,  the  teacher  should 
proceed  to  create  the  teaching  unit  by  means  of  the  Teaching  Units  Editor  that  enables  the  structure  of  the  learning 
domain  (hyperspace  or  traversing  structure)  to  be  created,  defined  and  managed.  Currently,  PLAN-G  just  allows 
navigation  on  static  way,  and  so  the  teacher  can  only  create  a directed  graph  over  the  learning  domain.  However 
there  are  some  projects  under  way  on  using  a multi-agent  system  to  improve  PLAN-G  adaptivity  by  means  of  the 
information  stored  in  user  models. 
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Abstract:  A web-based  distributed  collaborative  learning  community  based  on  knowledge  building  for 
academic  growth  of  college  students  was  implemented  and  evaluated.  The  evaluation  results  indicate  that 
it  provides  college  students  with  effective  telecommunication  and  collaboration  tools,  and  students’ 
responses  toward  the  WBDLC  have  been  very  positive  and  encouraging. 

Learning  Environment  of  WBDLC 

Students  may  actively  learn  through  participating  in  knowledge  building  and  collaborative-based  activities 
as  a member  of  an  online  group  (Brandon  & Hollingshead,  1999).  An  evaluation  by  Scardamalia  & Bereiter 
(1994)  of  the  CSILE  (Computer- Supported  Intentional  Learning  Environment)  project  clamed  that  collaborative 
learning  community  formed  by  students  facilitated  knowledge  building  and  generated  positive  learning  outcomes 
with  the  use  of  telecommunication  technologies. 

Through  the  use  of  telecommunication  technologies  of  the  Web-Based  Distributed  Learning  Community  of 
Academic  Growth  (WBDLC,  http://etwblc.et.tku.edu.tw),  it  is  intended  to  transform  traditional  classroom 
learning  to  better  enable  college  students  to  engage  the  virtual  and  social  practice  of  learning.  The  WBDLC 
consists  of  various  functional  elements  and  interactive  types  of  telecommunication  tools,  thus  users  may  learn 
the  content  relating  to  the  course  and  social  interaction  with  peer  at  the  same  time.  These  include:  asynchronous 
discussion  board,  online  synchronous  chat  room,  real-time  call  on  line,  desktop  video  teleconferencing,  access  to 
the  learning  resources  on  the  Web,  links  to  the  relative  Web  sites,  online  multimedia  materials,  files  for  upload 
and  download,  e-mail  listserv,  online  learning  portfolio,  and  asynchronous  advice  through  field  experts. 

Evaluation  Results  of  WBDLC 

User  Evaluation 

Resulting  from  the  survey  of  students’  satisfaction  showed  that  in  general,  they  rated  their  learning 
experiences  in  the  web-based  learning  environment  consistently  higher  than  those  in  the  instruction  of  traditional 
classrooms.  Students  indicated  that  they  learned  more,  the  class  was  more  interesting,  they  were  more  motivated, 
and  that  they  were  more  activate  to  learn.  They  also  reported  that  they  had  a greater  opportunity  to  be  heard  by 
the  teachers,  they  heard  more  from  their  peers,  and  that  they  gained  more  help  from  and  interaction  with  their 
classmates.  As  far  as  the  students’  feedback  on  the  satisfactions  for  the  content,  interface,  and  functions  of 
WBDLC  system  reported  that  the  system  interface  is  easily  accessible  and  usable  to  users.  Comments  from 
students  showed  that  they  liked  having  hands-on  experiences  with  the  web,  they  felt  comfortable  with  the 
telecommunication  capabilities  provided  by  the  WBDLC  system,  and  that  they  would  be  willing  to  participate 
the  learning  activities  of  similar  system  to  the  WBDLC.  The  in-deep  interviews  of  small-group  were  also 
conducted  by  group  sampling  based  on  students’  performances  on  WBDLC.  The  student  feedback  from  the 
small-group  interviews  formed  by  several  deeper  questions  provided  some  insight  into  what  needs  to  be  taken 
into  consideration  when  using  the  WBDLC.  Interviewing  comments  from  students  also  provided  more  deep 
insight  into  the  advantages  and  disadvantages  in  the  functions,  uses  and  effectiveness  of  WBDLC  system. 

Although  the  majority  of  students  offered  a very  positive  response  in  the  survey  and  small-group 
interviews  on  the  functions  and  uses  of  WBDLC,  the  majority  also  offered  anticipated  disadvantages  including 
not  enough  time  to  participate  the  web-based  activities  and  insufficient  feedback  of  field  experts.  The  majority  of 
the  students  indicated  that  knowledge  building  through  the  sharing  of  student  work  and  idea  was  a great  activity 
and  was  helpful  to  their  learning  outcomes.  On-line  activities  like  hands-on  learning,  resources  hyperlink, 
synchronous  chat,  and  asynchronous  group  discussion  were  common  preference  for  the  majority  of  the  students. 
Moreover,  students  liked  getting  access  to  resources  such  as  websites  relevant  to  course  and  learning  materials, 
to  information  such  as  course  information  and  teacher  notes,  and  to  knowledge  such  as  posting  and  replying 
ideas  from  peer  in  the  Discussion  Board. 

Expert  Evaluation 

The  teachers  reported  that  additional  time  is  required  to  prepare  learning  activity  and  make  teaching 


materials  on  the  web,  however  it  is  well  worth  the  effort  doing  that  in  terms  of  greater  learning  efficiency  and 
make  themselves  growth  on  teaching  experiences.  Moreover  teachers  also  encountered  obstacles  in  developing 
web-based  collaborative  learning  activities  for  online  groups  due  to  the  lack  of  support  of  providing  teaching 
resources,  experience  of  using  telecommunication  technologies,  and  familiarity  with  preparing  and  managing 
online  activities  on  a web  environment.  It  reveals  that  online  teaching  assistants  are  needed  for  web-based 
learning  to  help  teachers  doing  relevant  teaching  activity  and  making  decision  on  web.  Our  key  insight  to  the 
evaluation  results  of  the  WBDLC  system  is  the  verification  of  its  design,  creation  and  use  experiences. 

Conclusions  and  Future  Work 

We  work  continuously  with  teachers  and  students  at  the  Tamkang  University  to  develop  new  curricular  as 
well  as  more  proper  teaching  approaches  and  strategies  that  use  project-based  and  problem-oriented  learning 
activities.  The  critical  challenges  the  WBDLC  will  face  in  the  future  are  not  simply  technological  uses,  but  also 
the  challenges  of  social  interaction,  knowledge  share,  and  creating  online  communities  of  learners.  The  WBDLC 
can  be  a base  to  inform  educators  and  policy  makers  on  the  effective  and  sustainable  application  of  social, 
distributed,  and  collaborative  virtual  community  on  Internet  for  college  education. 
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ABSTRACT 


Abstract:  A VR-based  interactive  and  simulated  environment  on  web  for  constructive  learning  slide 
projector  operation  has  been  evaluated.  The  evaluation  results  report  that  this  learning  material  has 
positive  effects  on  students'  learning  outcomes.  Finally  some  indication  of  further  researches  to  be 
done  will  be  given  based  on  using  the  material  developed  in  this  study. 

Learning  Environment  of  WebVR 

With  the  multimedia  capacity  such  as  3D  graphics  and  animation.  Virtual  Reality  technology  could 
dynamically  simulate  the  manipulating  processes  of  psychomotor  skills  for  the  goals  of  immersive  learning 
and  situated  learning  (Youngblut,  1997).  Students  could  then  manipulate  the  VR-based  system  dynamically 
and  learn  by  their  own  hands-on  experiences  how  slides  are  produced  and  a slide  projector  is  operated  that  is 
impossible  to  observe  in  real  situation  (Moore,  1995).  The  types  of  activities  supported  by  VR  capability 
facilitate  current  educational  application  that  students  are  better  able  to  acquire,  master,  retain,  transfer,  and 
generalized  new  knowledge  whenever  they  are  actively  engaged  constructing  their  knowledge  in  a leaming- 
by-doing  situation  (Youngblut,  1997). 

In  the  VR-based  learning  world  of  operating  slide  projector  on  the  Web  (WebVR),  students  can  easily 
move  around  within  and  freely  navigate  through  the  virtual  worlds  and  do  all  kinds  of  actions,  such  as  freely 
manipulating  virtual  objects  in  any  degrees  of  freedom.  Additional,  students  immersively  existing  in  the  3D 
environment  of  WebVR  may  experience  several  degrees  of  freedom,  including  full  freedom  in  navigation, 
the  access  to  information  through  multiple  interactions,  and  the  capability  to  figure  their  own  learning 
process. 

Evaluation  Methodologies  and  Results  of  WebVR 

User-Based  Evaluation 

The  survey  was  involved  35  undergraduate  students  in  Teacher  Preparation  Program  and  undertaken 
by  sending  the  post-course  questionnaires  to  them.  We  evaluated  the  quality  of  the  WebVR  through  doing 
qualitative  study  for  mainly  examining  the  functions,  interface,  content,  and  learning  experiences  by  means 
of  questionnaires  that  embrace  specific  questions.  The  results  from  the  survey  of  students'  feedback  on  the 
satisfactions  for  the  WebVR  reported  that  the  system  interface  is  easily  accessible  and  usable  to  users.  They 
also  indicated  that  there  are  many  learning  materials  that  can  be  delivered  more  effectively  by  video  and 
audio  media  that  are  not  provided  in  the  WebVR  materials.  Students  reported  that  they  liked  having  hands- 
on  experiences  with  the  virtual  worlds,  and  they  felt  comfortable  with  the  simulated  capabilities  provided  by 
the  WebVR  material.  The  results  from  the  survey  of  students'  perceptions  on  their  learning  experiences 
showed  that  in  general,  they  rated  their  learning  experiences  in  the  simulated  VR-based  learning 
environment  consistently  higher  than  those  in  the  instructions  of  traditional  classrooms.  Students  indicated 
that  they  learned  more,  the  materials  was  more  interesting,  they  were  more  motivated,  and  that  they  were 
more  activate  to  learn.  They  also  reported  that  they  had  a greater  opportunity  to  be  heard  by  the  teachers, 
they  heard  more  from  their  peers,  and  that  they  gained  more  help  from  and  interaction  with  their  peers. 

Although  the  majority  of  students  offered  a very  positive  response  in  the  survey  on  the  ftinctionality 
and  usability  of  WebVR  material,  the  majority  also  offered  anticipated  disadvantages.  These  disadvantages 
contained:  not  enough  time  to  participate  the  web-based  VR  activities,  low  speed  of  system,  insufficient 
realism  of  scene  in  the  virtual  worlds,  and  insufficient  feedback  of  the  materials,  teacher  and  peers. 

Expert-Based  Evaluation 

Three  experts  involved  in  the  expert-based  evaluation.  The  three  experts  consisted  of  a teacher  as  a 
subject  matter  expert  in  slide  production,  a field  expert  in  web-based  VR,  and  a field  expert  in  computer- 
aided  instruction  and  web-based  learning.  A teacher  experienced  slides  production  instructor  who  has  been 
teaching  "instruction  media"  for  3 years  was  invited  to  examine  the  correctness,  appropriateness,  and 
consistence  of  the  learning  content  and  objects.  A field  expert  in  VR  experienced  the  design  and 
development  of  web-based  VR  system  was  invited  to  check  the  functionality,  usability,  and  interface  of  the 


WebVR.  A field  expert  in  computer-aided  instruction  and  web-based  learning  experienced  the  design  and 
development  of  CAI  and  WBL  was  invited  to  provide  opinions  on  the  VR-based  instructional  design  and 
learning  strategies  used  in  the  WebVR. 

Conclusions  and  Summary 

Based  upon  the  evaluation  results  of  WebVR,  it  is  a helpful  tool  both  for  students  and  teachers  in  the 
learning  process.  This  is  concerned  about  the  experiences  students  gained  from  the  free  navigation  and  high 
interaction  afforded  to  them,  and  from  the  exploration  of  basic  skills  of  operating  slide  projector.  As  we  have 
emphasized,  WebVR  is  a learning  tool  that  allows  students  to  manipulate  objects,  navigate  through  the 
virtual  worlds,  and  can  stimulate  them  to  construct  their  own  knowledge. 
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Abstract:  This  article  discussed  the  application  of  multimedia  system  to  motor  skills 
learning.  First  the  authors  reviewed  motor  learning  theories.  Then  the  development  of 
application  of  information  technology  in  physical  education  and  sport  learning  in  Taiwan  was 
introduced.  A specific  study  regarding  the  effect  of  multimedia  computer-assisted  learning  in 
short  service  of  badminton  with  forehand  was  described.  According  to  the  result,  it  is 
necessary  that  more  efforts  on  design  and  development  of  the  contents  of  multimedia  that 
sufficiently  represent  the  connection  between  cognition  and  skill  performance  on  motor 
learning. 


Introduction 

Traditionally,  motor  skills  learning  proceeds  the  steps  that  include  the  learner  observes,  imitates  the  coach’s 
modeling,  refines  his  motor  skills,  and  repeatedly  practices.  Diffusion  of  sports  was  limited  by  the  lack  of 
enough  coaches,  relative  knowledge,  and  experiences.  Information  technology  breaks  through  the  obstacles  of 
time  and  space,  expands  the  approaches  of  learning,  contribute  to  almost  every  part  in  contemporary  human  life 
and  civilization.  Therein,  multimedia  computer  system  condensing  the  necessary,  versatile  information  on  CD- 
ROM  makes  the  diffusion  of  sports  rapid,  cheap,  and  efficient  possibly.  Motor  skills  learning,  as  a part  of  human 
life,  was  overlooked  for  a long  period  in  my  country,  Taiwan.  Therefore,  trying  to  apply  multimedia  system  to 
motor  skills  learning  is  being  invoked  because  of  the  popularization  of  information  technology. 


Motor  Learning  Theories 

Fitts  and  Posner  (1967)  have  provided  a model  of  motor  learning  that  includes  verbal-cognitive  stage,  associative 
stage,  and  autonomous  stage.  Jack  Adams  (1971)  offered  the  verbal-motor  and  autonomous  two-stage  model. 
Schmidt  (1975,  1982)  proposed  the  schema  theory  of  motor  skill  learning  and  emphasized  learners  to  get  sensory 
information  before  executing  movement.  Moreover,  borrowed  from  Bandura’s  observation  learning  theory 
(1977),  a notion  was  provided  that  the  best  situation  for  motor  learning  is  to  offer  an  appropriate  model 
simulation  (luppa  1984).  In  sum,  observation  of  accurate  motion  demonstration  and  understanding  cognitive 
activities  behind  motor  skill  performance  will  be  helpful  on  learning  motor  skills.  Many  studies  have  showed 
that  the  cognitive  effects  of  multimedia  computer-assisted  instruction  are  significant.  Therefore,  it  is  desirable 
that  application  of  the  features  of  multifunction  and  consistency  of  multimedia  provide  accurate  modeling  to  help 
the  sport  learner  acquire  cognition  and  build  up  correct  actions. 


Research  Tryout 

The  institute,  which  the  authors  belong  to,  has  set  up  a particular  section  at  graduate  level  to  implement  the 
information  technology  application  in  physical  education  and  sport  learning.  Some  projects  involving  in  fitness, 
nutrition,  health,  and  teaching  in  P.  E.  are  in  process.  The  following  will  introduce  a study  which  researchers 
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developed  a multimedia  prototype  on  short  serve  of  badminton  and  conducted  an  experimental  research  with  this 
prototype  to  explore  whether  the  multimedia  material  is  a helpful  tool. 

Subjects  participating  in  this  study  were  1 1 1 sixth-graders  from  three  classes  in  an  elementary  school.  Each  one 
of  three  classes  received  a specific  one  of  three  teaching  methods:  multimedia  computer-assisted  learning 
(MCAL),  multimedia  computer-assisted  instruction  (MCAI),  and  traditional  instruction  (TI.) 

The  whole  teaching  procedure  included  three  sessions.  In  each  session  divided  into  three  segments.  During  the 
second  and  the  third  segments,  subjects  received  the  same  kind  of  activities  in  separate  class  at  gymnasium.  The 
first  segment  taking  about  five  minutes  was  the  three  different  treatments  for  respective  groups.  According  to  the 
results  showed  on  the  cognitive  test,  providing  accurate  video  demonstration  plus  teacher’s  explanation  and 
demonstration  had  significantly  better  effect  than  those  of  traditional  instruction.  The  results  indicated  that 
presentation  of  image,  video,  sound,  and  related  text  explanation  (MCAL  group)  indeed  generated  more  efficient 
cognitive  learning.  Adding  a human  teacher’s  interpretation  (MCAI  group)  could  yield  better  learning. 
Performance  test  evaluated  the  students’  posture,  standard,  pace,  coordination,  and  stability  while  they 
performed.  The  results  of  analysis  of  variance  on  the  performance  test  revealed  the  MCAI  group  had 
significantly  better  performance  than  that  of  the  TI  group.  Result  of  analysis  of  variance  on  skill  test  failed  to 
yield  main  effect.  The  result  had  the  same  finding  as  Ross  (1994)  did.  But,  it  is  different  from  Stiffen  and 
Hansen’s  (1987)  research  finding  that  showed  the  CAI  group  had  better  skill  test  scores  than  those  of  the 
traditional  instruction  group.  Because  motor  ability  and  practice  affect  sport  skill  learning,  the  reason  of  no 
difference  might  be  that  short  serve  of  badminton  is  not  so  complicated  to  need  more  strength  and  endurance. 
Therefore,  we  suggest  the  continued  research  on  other  type  of  sports,  time  arrangement  of  practice,  or  different 
age  learners  to  explore  the  appropriate  explanation. 


Conclusion 

Motor  skill  learning  has  its  own  features  that  emphasize  physical  execution  and  practice.  Furthermore,  motor 
skill  learning  involves  different  response  types  that  are  either  open  or  closed  and  either  rapid  or  slow  (Poulton, 
1957).  In  other  words,  different  sport  type  probably  needs  very  different  motor  skills  to  perform.  To  a specific 
sport  type,  from  the  sensory  stimuli,  perception,  memory,  to  movement,  it  could  have  its  own  particular  cognitive 
process.  From  the  study  of  short  serve  of  badminton  we  can  only  assure  the  effect  of  multimedia  application  on 
cognitive  learning  of  motor  skill.  However,  mental  training  and  mental  rehearsal  have  showed  effects  in  sport 
psychological  research  recently.  Apparently,  we  need  to  make  more  efforts  on  design  and  development  of  the 
contents  of  multimedia  that  sufficiently  represent  the  connection  between  cognition  and  skill  performance  on 
motor  learning. 
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Abstract:  This  short  paper  is  a report  on  efforts  at  the  University  of  South  Carolina  to 
encourage  and  train  foreign  language  faculty  to  be  literate  and  enthusiastic  in  the  area  of 
instructional  technology.  The  creation  of  a new  position  for  the  Academic  Director  of  the 
Foreign  Language  Learning  Center  illustrates  the  schooTs  belief  in  the  importance  of 
instructional  technology  and  its  dedication  to  ensuring  that  faculty  in  all  of  its  departments  be 
given  the  opportunity  to  incorporate  it  into  their  curricula. 


Introduction 

An  increasingly  “wired”  learning  environment  compels  faculty  in  foreign  languages  to  learn  and  use  a 
variety  of  new  technologies.  To  that  end,  the  University  of  South  Carolina  has  created  a new  role  for  the 
Academic  Director  of  its  Foreign  Language  Learning  Center.  The  position  bundles  administrative  duties 
(including  the  daily  operations  of  the  Center)  with  teaching  foreign  language  (in  this  case,  German),  and 
directing  faculty  development  in  the  area  of  instructional  technology. 


Faculty  Development  Opportunities 

According  to  the  Department  of  Education’s  1993  paper,  “Using  Technology  to  Support  Education 
Reform”,  the  challenges  faced  by  today’s  faculty  include: 

• Learning  to  use  a variety  of  technology  applications; 

• Using,  adapting,  and  designing  technology-enhanced  curricula  to  meet  students  (sic)  needs; 

• Expanding  content  knowledge; 

• Taking  on  new  roles;  and 

• Responding  to  individual  students. 

(Means,  1993,  http://www.ed.gov/mibs/EdReformStudies/TechReforms/chap4c.html) 

Furthermore,  Kassen  and  Higgins  (1997)  include  the  following  necessities  in  their  list  of  requirements  for 
faculty  development  programs: 

• Establishing  a comfort  level  with  technology; 

• Integrating  technology  into  the  curriculum; 

• Developing  the  critical  skills  to  use  technology  effectively,  (p.264) 

One  key  aspect  of  the  new  directorship  entails  creating  and  overseeing  a Faculty  Development  Center 
for  foreign-language  faculty  interested  in  increasing  their  employment  of  instructional  technology.  The  Center 
provides  an  area  where  faculty  can  develop  multimedia  courseware,  design  and  maintain  Web  pages,  work  with 
audio  and  video  materials,  and  experiment  with  instructional  technology  in  a non-threatening  environment. 
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The  Workshop  Series 


Included  in  the  faculty  development  package  is  a series  of  instructional  technology  workshops 
designed  specifically  with  foreign-language  faculty  in  mind.  Some  of  the  topics  covered  in  the  workshop  series 
include:  using  Microsoft  PowerPoint  in  the  language  classroom  (especially  as  it  applies  to  distance  learning), 
hypermedia  in  foreign  language  instruction,  the  hyper-syllabus  and  online  course  delivery,  creation  of  CD- 
ROMs,  courseware  authoring  and  authoring  software,  and  orientations  to  commercially  available  language- 
learning  software  packages. 

Workshop  participants  are  asked  to  arrive  with  a didactic  project  in  mind  that  they  can  complete  during 
the  course  of  the  meeting.  Most  workshops  are  two  hours  in  length,  with  the  first  half  consisting  primarily  of 
hands-on  instruction,  and  the  second  half  being  set  aside  for  participants  to  work  on  their  own,  ask  questions, 
and  get  better  acquainted  with  the  application  being  covered. 

At  present,  the  workshop  schedule  is  such  that  one  workshop  is  offered  each  month.  I anticipate, 
however,  an  increase  in  the  frequency  of  workshops  as  demand  has  been  steadily  increasing. 


Evaluation 

Any  new  undertaking  must  be  evaluated  in  order  to  ascertain  whether  it  represents  an  improvement 
over  past  conditions.  Thus,  at  the  close  of  each  workshop,  participants  are  asked  to  fill  out  a short 
questionnaire.  The  results  are  compiled  and  will  be  presented  as  part  of  the  Language  Center’s  annual  report,  as 
well  as  becoming  part  of  the  Director’s  own  research  on  faculty  development.  An  additional  aspect  of  this 
training  is  determining  if  the  material  covered  is  actually  learned,  and,  in  the  case  of  faculty  development, 
subsequently  utilized  by  instructors  in  their  classrooms.  In  order  to  measure  these  criteria,  a second  set  of 
questionnaires  will  be  distributed  to  workshop  participants  at  the  end  of  the  semester  following  their  workshop 
attendance,  with  the  ultimate  goal  of  establishing  whether  the  workshops  have  encouraged  increased  use  of 
technology  in  the  foreign-language  classroom. 

As  the  workshop  series  has  only  been  ongoing  for  one  semester,  questionnaire  B has  not  yet  been 
distributed  to  foreign  language  faculty.  Initial  results  of  questionnaire  A are  promising,  however,  and  workshop 
attendance  has  progressively  increased  throughout  the  course  of  the  semester. 


Conclusion 

Foreign-language  faculty  must  be  given  the  opportunity  to  develop  their  teaching  skills  in  a non- 
threatening environment  where  support  is  available  if  it  is  necessary,  and  where  creativity  is  encouraged  and 
appreciated.  A program  like  that  currently  in  place  at  the  University  of  South  Carolina  removes  some  of  the 
unknowns  that  prevent  many  instructors  from  introducing  technology  into  their  courses  by  letting  them  take  the 
technology  for  a “test  drive”  ahead  of  time.  Ultimately,  I hope  that  all  foreign  language  faculty  at  the 
University  of  South  Carolina  will  feel  comfortable  enough  with  instructional  technology  to  make  it  a part  of 
their  courses.  In  the  meantime,  it  is  encouraging  to  see  language  professors  attending  the  workshops,  working 
in  the  faculty  development  center,  and  taking  an  interest  in  what  they  can  do  with  technology  in  the  classroom, 
as  well  as  what  technology  in  the  classroom  can  do  for  them. 
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Abstract:  This  paper  introduces  a major  European  collaborative  project,  PEARL,  that  is 
developing  a system  to  enable  "real  world"  teaching  experiments  to  be  undertaken  by  students 
working  collaboratively  over  the  Internet.  The  paper  outlines  issues  in  university  level  education 
that  led  to  the  project’s  definition,  summarizes  its  pedagogic  foundations,  and  highlights  the 
emphasis  on  maldng  the  science  and  engineering  curricular  accessible  for  students  with  disabilities. 


Introduction  to  the  PEARL  project 

The  PEARL  (Practical  Experimentation  by  Accessible  Remote  Learning)  is  researching  and  developing  a 
system  to  enable  students  to  conduct  real-world  experiments  as  an  extension  of  computer  based  learning  (CBL)  and 
distance  learning  systems.  The  objectives  being  to  give  high  quality  learning  experiences  in  science  and  engineering 
education  by  bringing  the  teaching  lab  to  the  students;  offering  flexibility  in  terms  of  time,  location  and  special 
needs.  This  will  extend  Internet  course  delivery  to  accommodate  collaborative  working  in  practical 
experimentation.  The  project  is  developing  a modular  system  for  flexibly  creating  diverse  remotely  controlled 
experiments,  integrating  this  with  a collaborative  working  environment  and  accessible  user  interfaces.  The  project 
will  research  the  pedagogic  impact  of  this  approach,  validating  its  developments  in  different  educational  contexts 
and  subject  areas.  These  include:  foundation  level  physical  sciences  (as  part  of  an  open  & distance  learning 
introductory  course);  cell  biology  (as  part  of  a final  year  undergraduate  course);  manufacturing  engineering  (post- 
graduate training)  and  digital  electronics  (as  part  of  undergraduate  courses  in  design  and  testing). 


Educational  Issues 

Experimental  work  is  a vital  part  of  science  and  engineering  teaching  at  all  levels.  There  is  an  increasing 
trend  to  use  multimedia  science  education  packages  and  "virtual  science"  in  both  school  and  university  level 
education.  These  approaches  have  great  value  and  if  done  well,  however  they  generally  focus  on  the  teaching  of 
science  facts  and  principles  and  not  the  teaching  of  the  process  of  scientific  enquiry  or  engineering  practice.  A key 
objective  in  PEARL  is  to  create  a (CBL)  facility  for  learning  science  and  engineering  as  a process. 

Access  to  experimental  work  in  distance  education  has  traditionally  been  achieved  by  including  simple 
home  experiment  kits  or  intensive  residential  schools  as  part  of  the  course.  The  project  is  seeking  to  facilitate  a 
wider  participation  in  experimental  work  in  distance  learning  courses  and  to  make  this  available  throughout  the 
course.  For  campus  based  universities,  the  expansion  of  student  numbers  has  put  increasing  strain  on  facilities  and 
has  made  it  difficult  to  provide  adequate  access  to  teaching  laboratories.  By  making  these  facilities  remotely 
accessible  they,  can  be  offered  to  students  at  different  times  and  with  less  space  demands  on  the  teaching  laboratory. 

Increasingly  there  are  demands  on  universities  to  teach  their  students  the  use  of  complex  technical 
equipment.  This  pressure  comes  both  from  the  need  to  introduce  the  students  to  state  of  the  art  practices  in  their 
subject  and  to  fulfil  the  expectations  of  future  employers  or  industrial  purchasers  of  university  education.  By 
providing  remote  access  to  experimental  work  involving  expensive  and  sometimes  safety  critical  equipment,  the 
widespread  use  of  such  equipment  by  undergraduates  and  industrial  clients  becomes  feasible.  Further  it  becomes 
possible  to  share  the  use  of  key  teaching  facilities  with  other  universities  with  inherent  economies  of  scale. 


The  distinction  between  distance  learning  and  traditional  university  learning  is  blurring  with  the  rapid 
increase  in  Internet  course  delivery.  However  there  is  widespread  concern  about  the  quality  of  the  educational 
experience  offered  in  some  current  Internet  based  courses.  The  PEARL  system  supports  high  quality  science  and 
engineering  courses  by  enabling  practical  work  to  be  integrated  with  other  Web/multimedia  based  teaching  material. 


Pedagogic  Foundations 

Much  educational  research  has  shown  that  a key  factor  in  the  pedagogic  advantage  of  practical  work  is  that 
it  is  normally  undertaken  by  pairs  or  groups  of  students.  This  facilitates  the  development  of  the  students’  ideas  in 
conversation  with  their  peers  (Scanlon,  et.  al.  1993).  The  practical  work  provides  the  focus  for  peer-to-peer 
interchange  normally  supported  by  the  tutor.  Important  motivational  gains  for  collaborative  working  have  also  been 
frequently  demonstrated  (e.g.  Issroff  1993).  Hence  it  is  vital  that  the  PEARL  system  facilitates  such  collaborative 
working  at  a distance. 

Contemporary  accounts  of  student  learning  accept  that  it  is  an  active  process  and  depends  on  interaction. 
Laurillard  (see  Laurillard  1993)  offers  a model  of  student  / tutor  / courseware  interaction.  She  considers  the  learning 
process  as  a kind  of  conversation,  and  asserts  that  this  process  ‘must  be  constituted  as  a dialogue  between  teacher 
and  student  (or  student  and  student),  operating  at  the  level  of  description  of  actions  in  the  world’.  She  classifies  the 
types  of  interaction  between  instructor  and  student  as  being  discursive,  adaptive,  interactive  or  reflective.  In  the 
PEARL  system  the  teacher  can  implement  a wide  range  of  leaning  modes.  It  facilitates  the  exchange  and  discussion 
of  knowledge;  reflection  (by  student  and  tutor);  interaction  at  the  level  of  the  real  world  experiments;  construction  of 
and  subsequent  adaptation  the  experiments,  by  either  the  student  or  the  tutor.  There  is  thus  a multiplicity  of 
"conversations"  envisaged  in  a PEARL  practical  session  through  which  the  learning  objectives  are  achieved. 

Science  learning  should  also  be  an  introduction  to  a community  of  practice  (Lave  and  Wegner  1991),  and 
this  means  that  science  learners  need  to  be  involved  in  the  type  of  activities  that  real  scientists  perform.  Therefore, 
all  students  need  to  experience  practical  work  and  all  students  need  to  experience  collaborative  working  mediated  by 
information  and  communications  technologies,  as  these  are  the  contemporary  experiences  of  working  scientists. 


Access  for  Students  with  Disabilities 

The  PEARL  project  also  seeks  to  make  experimental  work  in  science  and  engineering  accessible  to  students  with  a 
wide  range  of  disabilities.  Indeed  the  project  was  originally  conceived  in  to  achieve  this.  Disabled  students  are 
grossly  underrepresented  in  the  science  and  engineering  subjects  at  higher  education.  The  reasons  for  this  are 
complex,  however  access  to  experimental  work  is  often  cited  as  a key  barrier.  Much  work  has  been  done  over  the 
past  20  years  to  make  computers  accessible  to  people  with  all  kinds  of  disabilities  and  a high  degree  of  success  has 
been  achieved.  Hence  the  tactic  for  the  PEARL  project  is  to  firstly  to  make  practical  work  computer  controlled  and 
then  to  ensure  that  the  software  and  interface  design  follow  well-established  design  for  accessibility  principles  and 
that  they  are  compatible  with  the  available  access  technologies. 
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ABSTRACT:  Traditionally,  data  relating  to  the  conduct  of  subjects  at  the  University  of 
Wollongong  has  been  collected  for  teachers  with  one  main  purpose  in  mind:  to  provide  the 
teacher  with  supporting  information  as  to  their  teaching  ability  for  the  purposes  of  promotion. 

SOS  is  a web  based  system  which  teachers  can  use  to  author  customised  surveys  to  collect 
information  about  the  subject  they  teach.  These  surveys  are  completed  anonymously  by  the 
students  via  the  web  (using  randomly  generated,  survey  specific  numeric  tokens)  and  the  data  is 
automatically  collated  and  returned  to  the  teacher.  The  teacher  may  also  produce  the  surveys  in 
hard  copy,  for  manual  distribution  and  collation. 

The  system  provides  a ‘non-threatening’  and  ‘informal’  mechanism  by  which  teachers  can 
obtain  useful  information  about  the  subjects  they  develop  and  teach  in  terms  of  subject  based 
criteria  rather  than  the  'teacher  based  criteria'  of  formal  Teaching  Surveys.  In  this  time  of 
declining  University  budgets,  SOS  provides  a very  cost  efficient  solution  to  the  collection  of 
this  data. 

This  paper  provides  a background  to  the  development  of  SOS  in  the  context  of  the  needs  of  the 
University  of  Wollongong  and  its  teachers.  It  briefly  outlines  the  main  features  of  the  system, 
new  features  being  added  to  Version  2 and  reports  on  the  outcomes  for  the  early  adopters. 


BACKGROUND 

In  1998,  the  Director  of  CEDIR,  Associate  Professor  Sandra  Wills,  secured  funding  to  conduct  research  on  the 
development  of  an  ‘informar  online  survey  system.  The  goal  of  this  system  was  to  provide  teachers  with 
personal  formative  data  on  the  success  of  their  subjects  from  a student’s  perception,  allowing  them  to  improve 
teaching  and  learning  outcomes  for  students  without  the  need  to  resort  to  numerous  and  expensive  formal 
Teaching  Surveys.  The  research  working  party  examined  the  online  methods  used  to  conduct  teaching  surveys  by 
15  Australian  universities  including  the  Subject  Evaluation  kit  developed  by  the  Griffith  Institute  for  Higher 
Education  (GIHE)  at  Griffith  University  [1].  In  summary  the  findings  included: 

• most  existing  web-sites  were  highly  University  specific 

• none  were  using  the  web  exclusively  to  distribute  and  collate  the  data 

• most  surveys  seemed  to  consist  of  set  questions  plus  some  optional  ones  selected  from  item  banks. 

From  this  research  the  working  party  made  several  recommendations  which  later  formed  the  basis  of  the 
development  specifications  for  Subject  OnLine  Survey  (SOS).  These  included: 

• the  tool  should  be  totally  web  based  and  should  require  minimum  web  skills  to  operate  it; 

• there  should  be  an  item  bank  of ‘standard  questions’  and  the  ability  to  author  individual  questions; 

• there  should  be  a variety  of  questions  types  available  such  as  Likert  scale.  True  and  False,  Yes  and  No 

• there  would  need  to  be  a concurrent  staff  development  program 

During  the  research  phase,  a generic  evaluation  tool  LEO  [2]  or  Learning  Evaluation  Online,  which  is  a template 
based  in  the  as  OXYGEN  (Object  extensible  analYsis  and  Generation  of  Education  coNtent)  software  engine 
developed  by  Albert  Ip  [3]  was  identified  as  a possible  engine  and  later  implemented. 


SUBJECT  ONLINE  SURVEY  (SOS) 

SOS  is  a web  based  system  which  provides  a simple  and  intuitive  interface  through  which  teachers  can  construct 


and  author  customised  surveys  designed  to  collect  information  about  the  success  or  otherwise  of  the  subjects 
and/or  subject  components  that  they  provide  for  their  students.  These  surveys  and  the  returned  data  are  password 
secure  and  student  access  is  anonymous,  survey  specific  and  controlled  via  'tokens'.  The  data  is  auto-collated  and 
is  made  available  through  a data  collection  site  and/or  via  automatic  email  posting. 


DEVELOPING  A QUESTION  DATABASE 

The  Teaching  Survey  database  previously  in  use  within  the  university  contained  many  hundreds  of  questions 
which  were  either  directly  suited  or  with  modification  could  be  used  successfully.  Many  additional  questions 
have  been  added  to  take  account  of  a wider  range  of  delivery  options  than  the  traditional  lectures,  tutorials,  and 
laboratories  including  flexible  and  online  delivery. 


EVALUATION 

Version  1 of  the  system  was  made  available  to  three  faculties  within  the  university:  Engineering,  Commerce  and 
Public  Health.  Over  a period  of  approximately  6 months  a total  of  almost  1000  students  were  surveyed  using  SOS 
In  2000,  the  University  has  launched  two  new  remote  centres  at  which  two  full  degree  programs  are  being  offered 
flexibly  including  online.  SOS  will  be  used  as  part  of  the  evaluation  strategy  for  this  initiative. 

Much  of  the  evaluative  data  collected  so  far  is  in  the  form  of  comments  and  suggestions  from  users.  The  most 
common  responses  include 

• the  simplicity  of  the  authoring  interface  and  low  level  of  computer  skills  needed  to  operate  it; 

• the  ease  and  speed  of  survey  and  the  quick  turn  around  compared  to  paper  based  surveys; 

• the  ease  of  customisation  for  a variety  of  different  student  groups  and  needs. 

Teachers  using  the  system  to  date  have  identified  the  following  possible  weaknesses  of  the  current  version: 

• the  'set'  7 point  Likert  scale,  possibility  of  others  being  available; 

• no  mechanism  for  students  to  see  aggregated  results,  and; 

• the  need  to  manually  modify  the  data  collected  before  exporting  to  a spreadsheet. 


FUTURE  DIRECTIONS 

User  feedback  over  the  trial  6 moths  period  has  given  us  direction  in  further  development  into  version  two, 
greatly  expanding  the  applicability  of  the  system  to  all  campus  teaching  and  learning.  Some  of  these  include: 

• The  existing  databases  will  be  greatly  expanded  to  incorporate  all  questions  currently  in  use  at  UOW. 

• Question  categories  will  be  expanded  and  current  question  categorisation  will  be  re-assessed. 

• Sets  of  faculty  based  "Standard  Surveys"  will  be  built  up  and  provided  at  the  top  level  of  authoring. 

• In  the  longer  term,  normative  data  on  each  question  in  the  database  will  be  collected,  collated  and 
analysed. 

• Preliminary  data  analysis  by  teachers  will  be  supported  by  a down  loadable  Excel  macro  which  will 
have  built-in  frequency  counts,  bar  chart  generation  and  cross-tabulation. 

• Finally,  the  program  of  staff  training  needs  to  be  extended  to  include  developing  an  understanding  of  the 
application  of  the  survey  results  to  teaching. 
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Abstract:  Learning  Objects  have  the  potential  to  revolutionize  the  delivery  of  technology  enhanced 
courses.  To  realize  this  potential,  standards  for  the  production  and  transmission  of  learning  objects 
must  be  developed  and  adopted.  The  IEEE  and  IMS  have  made  a large  effort  in  producing 
standards  for  describing  meta-data,  but  there  has  been  little  work  in  producing  standards  for 
reliably  connecting  the  learning  object  and  its  related  meta-data.  In  this  paper,  we  will  describe  an 
enhancement  of  an  existing  standard  which  allows  effective  encapsulation  of  both  the  learning 
object  and  meta-data  in  a single  container. 


The  IEEE  Learning  Object  Meta-data  Working  Group*  defines  a learning  object  as  “any  entity,  digital  or  non- 
digital, which  can  be  used,  re-used,  or  referenced  during  technology  supported  learning.”  For  the  purposes  of  this 
paper,  we  will  limit  our  scope  to  defining  a learning  object  as  any  digital  entity  that  can  be  used  or  referenced  during 
technology  supported  learning.  Currently,  there  is  much  existing  work,  by  groups  such  as  IEEE  and  IMS,  on 
developing  standards  for  the  meta-data  used  to  describe  learning  objects.  However,  less  attention  has  been  paid  to 
the  binding  between  the  meta-data  describing  the  learning  object  and  the  object  itself 

Without  some  means  of  reliably  storing  and  transferring  the  data  and  meta-data  as  a single  entity,  two  problems 
easily  occur.  First,  if  the  learning  object  is  not  packaged  or  appropriately  connected  with  its  descriptive  meta-data,  it 
retains  little  information  about  the  appropriate  use  of  the  learning  object,  reducing  its  usefulness  to  the  educator. 
The  second  problem  is  that  of  possessing  meta-data  which  describes  and  links  to  a learning  object  which  either  no 
longer  exists  or  has  moved  elsewhere,  making  the  learning  object  useless  to  the  educator.  Unfortunately,  both 
problems  currently  exist. 

Binding  the  two  (or  ^^Ensuring  the  twain  shall  meet’’) 

Currently,  the  IMS  Meta-data  specification^,  based  on  the  IEEE  specification^,  describes  a large  amount  of 
standardized  meta-data  to  be  used  in  describing  a learning  object.  As  well,  they  provide  a proposed  XML  binding 
for  this  meta-data"*. 

With  little  effort,  this  XML  binding  for  learning  object  meta-data  may  be  extended  to  include  the  associated  data. 
All  that  is  required  is  a few  additional  XML  tags,  and  a data  encoding  scheme.  The  first  tag  to  introduce  would  be  a 
pair  of  tags  to  delimit  the  block(s)  which  contains  the  data  files  (“sub-objects”).  Keeping  it  simple,  we  could  use 
<DATA>  and  </DATA>  tags  for  this  purpose.  This  data  block  could  adequately  be  placed  within  the  record  block, 
as  described  in  the  IMS  XML  bindings  specification.  Within  this  data  block,  specific  files  must  be  enclosed.  For 
this  purpose,  additional  tags  may  be  used:  <FILE  name=”filename”>  and  </FILE>. 

Now,  the  problem  of  conflicts  between  the  XML  allowed  character  limitations,  and  the  contents  of  the  data  arises. 
This  is  easily  remedied  by  using  a base64  encoding  scheme^  for  files  contained  within  the  XML  document.  This 
format  uses  only  characters  which  are  valid  for  inclusion  in  XML  fields. 

As  an  illustrative  example: 


<?xml  version=’T.O”  encoding="UTF-8”?> 

<!DOCTYPE  RECORD  SYSTEM  "http://www.imsproject.org/XML/IMS-MD01.dtd"> 
<RECORD  xmlns=" http:// WWW. imsproject.org/metadata/"> 


13G1 


<! — ^the  standard  IMS  specified  data  would  be  here, 

<! — now  for  the  added  data  component  -> 

<DATA> 

<FILE  name=”instruction,ppt”>sdkjKJSDiwnxc/sdksKDKED  ...  sdkKFWlkdf+dQ==</FILE> 

<FILE  name=”exercise,swf’>dsk7dsFlS9dsaKu7  ...  adskQ8xsdk39asdFH==</FILE> 

</DATA> 

</RECORD> 

Linking  the  meta-data  and  data  (or  “Letting  the  left  hand  know  what  the  right  hand  is  doing”) 

Allowing  more  powerful  connection  between  the  meta-data  and  data  is  the  XLink  specification^,  currently  under 
development  by  the  W3C,  The  XLink  specification  allows  for  linking  between  XML  documents,  and  linking  within 
XML  documents.  This  allows  for  a more  natural  connection  between  the  meta-data  and  data,  and  even  allows  for 
more  powerful  learning  objects. 

For  example,  in  the  meta-data,  you  may  wish  to  identify  which  concepts  are  required  in  order  to  gain  most  benefit 
from  the  learning  object.  With  the  use  of  these  XLinks,  you  may  explicitly  link  to  learning  objects  which  instruct 
the  required  concepts.  Also,  the  sub-object  you  use  to  demonstrate  the  concept  to  be  learned,  may  be  a different 
sub-object  from  that  which  you  would  use  to  test  or  exercise  the  student’s  learning  of  the  concept.  This  linking 
specification  provides  a means  of  using  other  applications  to  describe  the  particular  metadata  element,  and  through 
the  use  of  multiple  sub-objects,  provide  a more  complete  educational  experience  for  the  learner. 

Developing  a solution 

Currently,  an  application  for  using  learning  objects  of  this  style  is  under  development.  It  uses  a small  Java  based 
web  server  as  an  intermediary  between  a web  browser  client  and  the  XML  files  containing  the  learning  objects. 
This  is  intended  to  be  able  to  run  on  personal  systems  as  well,  using  any  web  browser  as  the  primary  user  interface 
for  accessing  the  learning  objects.  More  information  is  available  at  http://aitt.acadiau.Cci/researcli/lo,  and  an  example 
implementation  should  be  available  in  late  April  for  download. 
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Introduction 

Much  of  the  focus  in  the  distance  education  (DE)  literature  centers  around  the  learner  and  learning,  almost  to 
the  exclusion  of  the  teacher  and  teaching.  Consequently,  instructors  of  DE  courses  have  been  left  suspended 
in  virtual  space.  This  paper  focuses  on  the  teachers'  position  in  the  DE  course  context,  especially  with  respect 
to  the  type  of  training  they  receive.  An  example  of  an  alternate  mode  of  teacher  preparation  is  presented  and 
suggestions  for  further  improvement  in  DE  teacher  training  are  discussed. 


The  Challenge  of  Providing  Adequate  DE  Preparation 

The  majority  of  faculty  have  little  knowledge  about  what  DE  is,  what  it  entails,  and  how  it  is  successfully 
taught  (Moore  & Kearsley,  1996).  Yet  instructors  receive  little  or  no  training  prior  to  being  assigned  a DE 
course  (Dooley,  1995;  Gehlauf,  Shatz  & Frye,  1991)  which  leads  to  teachers  being  under-prepared,  frustrated, 
isolated,  and  disillusioned.  Teaching  at  a distance  requires  methods  of  instructional  delivery  that  are  different 
from  the  strategies  and  approaches  commonly  used  in  a traditional  classroom  setting  (Northrup  1998,  Moore 
& Kearsley,  1 996)  because  the  structure  of  the  teaching  is  changed.  Rather  than  being  the  typical  source  of 
information,  the  instructor  assumes  the  role  of  facilitator,  monitor  and  peer  collaborator,  engaged  in  creating 
discourse  with  students  (Shedletsky,  1997).  DE  teachers  must  be  willing  and  able  to  develop  new 
methodologies  of  teaching  that  re-distribute  power,  roles,  and  responsibility  within  the  learning  community. 
Therefore,  to  ensure  the  success  of  learning  activities  carried  out  at  a distance,  DE  faculty  must  not  only 
develop  the  necessary  competencies  to  do  their  work  but  must  also  develop  a clear  understanding  of  their  re- 
defined roles  and  responsibilities. 


The  Training  Strategy 

While  the  importance  of  technical  training  for  distance  educators  is  well  documented  (see  Hiltz,  1986; 
Schuttloffel,  1 998),  it  is  often  the  only  type  of  training  offered  to  faculty,  in  insufficient  amounts  and  using 
inappropriate  techniques  (Schuttloffel,  1998).  Northrup  (1998)  argues  that  additional  training  on  new 
instructional  techniques  and  strategies  for  promoting  interactivity  and  for  providing  adequate  and  timely 
feedback  may  promote  a DE  environment  more  conducive  to  learning.  Teachers  need  to  extend  their  learning 
to  beyond  how  they  will  use  the  technology.  They  also  need  to  understand  how  the  technology  enables  and 
affects  their  pedagogy  and  need  to  be  given  the  time  to  digest  and  accommodate  this  new  knowledge. 

In  our  situation,  the  teacher  training  differed  from  the  traditional  DE  training  described  above  in  the 
following  ways:  a)  it  was  designed  to  take  place  over  the  summer  months  prior  to  the  course  implementation; 
b)  the  teacher  (second  author)  worked  in  collaboration  with  a Ph.D.  student  in  Educational  Technology  (first 
author)  who  assumed  the  roles  of  pedagogical  technology  consultant  and  teaching  assistant;  c)  in  contrast  to  a 
"just  in  time"  training  approach,  the  instruction  was  structured  over  a period  of  several  weeks,  building  in 
time  for  practice  and  reflection;  d)  the  training  program  was  experiential  in  that  the  teacher  was  able  to 
acquire  technological  skills  within  the  context  that  she  would  be  later  teaching;  e)  training  was  carried  out  in 
close  proximity  to  course  implementation  so  that  skills  and  knowledge  were  retained,  practiced,  discussed  and 
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contemplated  upon,  in  relation  to  the  DE  environment;  and,  f)  the  training  focused  on  helping  the  teacher  to 
understand  and  become  familiar  with  her  new  role  as  a DE  teacher.  More  specifically,  the  teacher  created  a 
metaphor  which  helped  the  instructor  to  deal  with  her  own  feelings  of  isolation,  especially  from  her  students 
and  from  colleagues  who  had  no  experience  with  the  DE  environment. 


Future  Implications  for  DE  Teacher  Training 

Presently,  adequate  training  and  support  for  distance  teachers  remains  piece-meal  and  strongly  associated 
with  the  efforts  of  key  individuals,  not  with  the  university  community.  Bringing  about  large  institutional 
reform,  however,  is  no  easy  feat  and  certainly  not  accomplished  in  a short  period  of  time.  As  DE  increases  in 
popularity,  teachers  will  need  to  be  prepared  for  new  distance  teaching  assignments.  Below  are  some 
strategies  that  can  help  DE  teachers  develop  the  skills  and  knowledge  required  to  comfortably  grow  into  their 
new  roles. 

With  respect  to  faculty  development,  it  is  important  to  institute  a training  program  for  initial  preparation  and 
ongoing  support  that  is  sensitive  to  the  needs  of  the  teacher  and  the  learning  goals.  Such  a program  should 
recognize  that  teachers  need  time  to  become  familiar  with  the  technology,  their  pedagogy  in  the  DE 
environment  and  their  role  as  DE  teachers.  If  DE  faculty  training  continues  to  focus  only  on  technological 
skills,  at  best  what  is  addressed  is  something  akin  to  what  Abhram  Maslow  (1987)  refers  to  as  teachers’  basic 
needs.  Therefore,  it  is  critical  that  training  of  DE  teachers  evolve  and  expand  to  incorporate  issues  such  as 
defining  one’s  role  as  teacher  in  a virtual  learning  environment,  building  interactive  learning  communities  of 
responsible  learners,  and  creating  a supportive  collegial  infrastructure  so  as  to  minimize  not  only  isolation 
but  increase  feelings  of  belonging  to  a community  of  scholars,  learners,  and  compassionate  individuals.  An 
effective  training  program  should  foster  a community  of  resources  (i.e.,  people,  materials,  technology)  so  that 
faculty  can  share  knowledge  and  expertise  as  well  as  stories  of  successes  and  failures.  This  will  not  only  help 
distance  teachers  overcome  feelings  of  detachment  from  their  peers  but  also  build  a strong  collective 
knowledge  of  what  teaching  methodologies  are  most  effective  in  DE  environments. 
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Abstract:  In  this  paper,  we  present  a multimedia-based  system  facilitating  the  training  of  an 
organization's  employees  in  Intemet/lntranet  technologies  over  a LAN  connected  to  Internet. 
The  system  uses  several  types  of  multimedia  educational  patterns,  such  as  hyperlinked  text  and 
images,  special  sounds,  animations,  interactive  simulations  demonstrating  the  use  of  various 
Internet  tools,  e.t.c.  It  contains  several  training  courses,  depending  on  each  trainee's 
background  in  using  Intemet/lntranet  technologies.  All  the  multimedia  material  is  stored  in  a 
multimedia  database,  which  can  be  accessed  and  updated  by  the  trainer  through  a user-friendly 
interface  consisting  of  forms. 


1.  Introduction 

The  use  of  multimedia  technology  has  given  a great  impetus  to  the  training  evolution.  Multimedia 
systems  provide  an  integrated  environment  for  the  creation,  storage  and  presentation  of  a variety  of  media  types 
enabling  the  development  of  a great  number  of  successful  prototype  educational  applications  (Faconti  1994), 
This  paper  presents  the  architecture  of  an  interactive  multimedia-based  environment  as  an  assistant  for  training 
an  organization's  employees  in  Intemet/lntranet  technologies  over  a LAN  connected  to  Internet,  The 
environment  incorporates  a multimedia  interface,  and  a multimedia  database  for  storing  and  organizing  all  the 
multimedia  assets.  The  information  presented  to  the  trainees  via  the  multimedia  interface  during  a course  is 
dynamically  accessed  from  the  multimedia  database.  This  feature  facilitates  the  modification  of  the  multimedia 
database's  contents  by  the  trainer  through  a user-friendly  interface. 


2.  The  Front-End  Component 

The  main  issue  was  to  develop  an  interactional ly  rich  system  that  would  support  user  functionality 
efficiently  and  effectively,  taking  advantage  of  the  new  infrastructures  currently  available.  The  functionality 
concerns  both  training  activities  and  navigational  ones,  like  moving  through  hypermedia  objects  and  browsing 
large  multimedia  structures  (Schnaze  1993). 

The  information  provided  by  the  system  is  presented  in  a variety  of  ways,  such  as  interactive 
simulations  demonstrating  the  use  of  various  Internet  tools  (e.g.  Internet  browser),  hypertext,  special  sounds, 
appropriate  images  and  animations,  hypermedia  object,  educational  games.  Certain  hyperlinks  correspond  to 
URLs  of  WWW  servers.  Therefore,  from  within  the  multimedia  system  the  trainee  can  start  his/her  computer’s 
default  Internet  browser  and  thus,  put  everything  he  learns  in  the  specific  section  into  practice.  The  content  is 
distributed  in  sections,  subsections  and  topics.  Each  topic  comprises  a series  of  educational  screens.  The  user 
determines  the  presentation  flow,  by  using  hot  area  selections  as  well  as  buttons.  On-line  help  is  available, 
usually  via  messages  at  the  bottom  of  the  active  screen,  where  necessary.  Furthermore,  the  user  can  consult  a 
glossary  of  primary  Internet  terms  in  order  to  enrich  his/her  knowledge. 

There  are  different  components  of  the  system  interface  (front-end),  related  to  the  user  views,  described 
in  section  4.  One  of  the  distinctive  features  of  our  approach  is  dynamic  linking  to  the  various  multimedia  assets 
(e.g.  images,  audio  and  video  tracks,  text  files,  animation  files)  by  submitting  queries  to  the  multimedia 
database.  This  means  that  the  contents  of  the  multimedia  interface  are  established  dynamically,  at  run-time. 


since  the  information  is  structured  appropriately  in  the  database.  This  minimizes  the  cost  of  the  system's 
maintenance,  results  in  consistent  presentations  and  ease  of  tailoring.  In  addition,  the  data  modification  effort  is 
reduced. 


3.  The  Back-End  Component 

The  back-end  component  consists  of  the  SQL  components,  ODBC  software  and  a multimedia 
database.  The  multimedia  database  is  used  to  store  and  organize  all  the  multimedia  assets  of  the  system.  The 
SQL  components  contain  the  SQL  statements  used  to  access  the  multimedia  database  through  the  ODBC 
software.  The  two  SQL  components  related  to  the  user  views  (section  4)  are  as  follows: 

1)  SQL  component  for  the  trainer:  This  component  enables  the  trainer  to  access  and  update  the 
multimedia  database  keeping  thus,  all  the  multimedia  material  up-to-date  with  advances  in  the  Intemet/Intranet 
technologies.  Furthermore,  it  enables  the  trainer  to  establish  login  names  and  passwords  for  the  trainers  which 
are  stored  in  the  database. 

2)  SQL  component  for  the  trainees:  This  component  is  responsible  for  validating  the  trainee's  login 
name  and  password  and  accessing  the  system's  multimedia  assets. 


4.  User  Views 

The  user  interface  is  unified  offering  different  view  to  different  users.  We  can  distinguish  three  basic 
views  as  far  as  the  users  are  concerned  corresponding  to  the  components  of  the  system's  user  interface; 

})  General  view:  In  the  general  view,  the  user  can  access  all  available  information  regarding 
Intemet/Intranet  technologies  by  navigating  through  the  sections,  subsections  and  the  topics  of  the  multimedia 
application. 

2)  Trainer  view:  In  the  trainer  view,  the  user  is  able  to  modify  the  contents  of  the  multimedia  database 
with  a user-friendly  interface  consisting  mainly  of  forms  and  establish  login  names  and  passwords  for  the 
trainees. 

3)  Trainee  view:  In  the  trainee  view,  the  user  identifies  himself/herself  to  the  system. 

The  trainees  can  use  the  general  and  the  trainee  view  whereas  the  trainer  all  the  views. 


5.  Implementation  Aspects 

The  system,  in  the  form  of  an  advanced  prototype,  has  been  developed  under  the  Windows  98  and 
Windows  NT  operating  systems.  The  multimedia  interface  (front-end)  was  developed  using  the  Macromedia 
Authorware  5.0  and  Macromedia  Afterburner  authoring  tools.  Additional  code  was  written  using  Visual  Basic. 
The  multimedia  database  application  was  developed  using  the  Microsoft  SQL  Server  on  Windows  NT  4.0 
Server  operating  system.  TTie  application  handling  the  communication  between  the  user  interface  and  the 
database  is  comprised  of  a number  of  SQL  components  using  Authorware's  ODBC  functions.  These  functions 
are  called  via  calculated  icons.  Dynamic  linking  is  implemented  by  connecting  Authorware  parameters  to 
database  items. 
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Abstract:  Research  libraries  have  always  built  and  organized  authoritative  discipline-based 
collections.  This  short  paper  provides  a preliminary  look  at  the  state  of  discipline  or  subject-based 
web  gateways  to  so-called  authoritative  scholarly  works  in  large  research  libraries.  The  study  is 
based  on  an  in-depth  analysis  of  the  30  largest  academic  research  libraries  (based  in  universities) 
as  defined  by  the  Association  for  Research  Libraries’  index  ranking.  The  study  also  draws  upon 
the  results  of  a survey  to  each  library  revealing  comparative  data  on  gateway  content  decision- 
making and  application  of  learning  technologies  to  promote  gateway  usage  within  the  universities’ 
respective  curricula.  Gateway  content  is  examined  in  terms  of  subjects  covered,  methods  of 
content  selection,  formats  pointed  to,  and  integrative  features  with  “traditional”  online  catalogs  of 
the  libraries’  owned  physical  and  electronic  holdings.  Goals  for  the  study  are  two-fold:  a) 
obtaining  a profile  of  subject-based  gateway  development,  structure,  and  maintenance  practices, 
and  b)  seeking  models  for  effective  subject  gateway  develq^ment  in  light  of  efforts  to  improve 
aggregation  of  web  resources  with  online  library  catalogs. 


Introduction 

The  explosion  of  information  resources  available  on  the  web  has  driven  many  research  libraries  to  develop  a subject- 
based  gateway  to  selected  sites  deemed  particularly  useful  for  the  scholarly  community.  The  labor  intensive  nature 
of  developing,  maintaining,  and  updating  these  gateways  provides  motivation  to  identify  best  practices  which  can  be 
applied  successfully  to  other  institutions.  This  analysis  is  also  important  given  the  growing  number  of  web-wide 
search  engines  available  to  the  user  (Yahoo,  Alta  Vista,  etc.)  as  well  as  software  to  apply  to  a more  selective  group 
of  sites  chosen  by  research  library  subject  specialists.  Boundary  crossing  between  institutionally  created  subject 
gateways,  library  catalogs,  commercial  indexes,  and  other  sites  on  the  libraries’  web  presence  is  becoming  more 
essential  for  comprehensive  navigation  of  scholarship.  Another  factor  is  the  emerging  ease  by  which  library  users 
can  create  and  customized  their  own  gateways. 

Methodology 

The  target  population  for  this  study  are  the  30  largest  Association  of  Research  Libraries  (ARL)  academic  research 
libraries  based  on  the  ARL  membership  criteria  index.*  The  large  academic  research  libraries^  were  examined  in  two 
ways:  1)  an  examination  by  the  author  of  the  libraries’  Web  presence  in  terms  of  gateway  existence,  and  2) 
administration  of  a questionnaire  to  each  library  to  self-identify  their  gateways  and  answer  specific  questions 
regarding  the  creation,  content,  and  maintenance  of  the  site.  These  include  existence  of  a locally  maintained  subject 
gateway,  its  name,  description  of  search  engine,  selection  information,  comprehensiveness  and  depth,  type  of 
resources  included,  maintenance  procedures,  and  prominence  factors  within  the  library  website. 

An  Example  — Iowa’s  Gateway  to  Online  Resources 

As  with  most  research  libraries  the  University  of  Iowa’s  efforts  in  developing  an  internal  subject-based  gateway  are 
continually  evolving.  The  goal  of  the  Gateway  to  Online  Resources  is  to  allow  users  to  find  high  quality  sources 
quickly  and  efficiently.  The  resources  in  the  Gateway  have  been  selected  by  the  Libraries’  bibliographers  and 
reference  librarians,  who  use  their  knowledge  of  faculty  research  interests  and  class  assignments  to  select  the  most 
reliable  and  well-designed  electronic  resources.  These  resources  include  authoritative  Web  sites  maintained  by 
professional  societies  or  leading  faculty  as  well  as  databases  licensed  by  the  Libraries  from  reputable  publishers.  It 


also  includes  databases  and  reference  to  materials  in  the  online  catalog.  The  Gateway 

(http://www.lib.uiowa.edu/gw/)  consists  of  two  sections:  1)  reference  categories  including  links  to  dictionaries,  style 
manuals,  encyclopedias,  etc.  and  2)  subject  headings  including  links  to  57  subject  areas  at  this  wnting.  The  Gateway 
holds  a prominent  place  within  the  Libraries’  web  presence  and  uses  File  MakerPro  as  its  search  engine.  At  this 
point  in  time  Iowa’s  Gateway  does  not  feature  automated  web  checking  and  links  are  corrected  as  errors  are 
reported. 

Preliminary  Survey  Results 

Preliminary  results  of  the  survey  are  trickling  in  at  this  writing  but  of  10  respondents  eight  report  having  gateways. 
Of  these  eight,  two  are  unnamed.  One  of  the  two  responding  in  the  negative  did  indicate  they  had  separate  pages 
with  subject  access  to  areas  such  as  electronic  journals,  indexes,  reference  links,  and  bibliographer  pages.  Seven 
report  having  search  engines  for  their  gateways  including  WWW  wais,  htdig,  InfoSeek,  Index  Servicer,  Filemaker 
Pro,  Glimpse,  and  MySQL.  Most  of  the  respondents  indicate  their  gateways  are  somewhat  selective.  Respondents 
with  gateways  indicate  that  they  include  links  to  resources  on  the  web,  in  the  catalog,  databases,  electronic  journals, 
and  references  to  print  materials.  None  indicated  specializing  in  a particular  subject  area.  The  majority  of 
respondents  indicated  that  their  gateway  holds  a prominent  place  within  the  library  website.  Three  institutions  used  a 
software  program  to  check  links  while  the  rest  used  staff.  Software  included  Linkchecker,  MySQL,  and  Linklint. 

Conclusion 

Best  practices  among  the  research  libraries’  gateways  to  scholarship  will  inform  next  steps  in  individualized  and 
collaborative  development  to  improve  web-based  navigation  of  scholarship  (regardless  of  format),  gateway 
placement,  maintenance  and  long-term  viability.  New  developments  and  trends  in  commercial  browser  development 
will  impact  in  a variety  of  ways  on  the  structure  and  proliferation  of  individually  crafted  gateways.  At  the  same  time 
more  and  more  individual  scholars  are  building  their  own  customized  gateways.  These  factors,  along  with  the 
implementation  of  new  web-based  automated  catalog  systems  in  research  libraries,  will  affect  future  subject  gateway 
development  and  architecture. 


[1]  Based  on  the  1997-98  ARL  membership  criteria  index  and  index  formula.  The  index  is  a summary  measure  of  relative  size 
among  the  university  members  of  the  association.  The  index  is  calculated  using  five  data  elements  including  number  of  volumes 
held,  number  of  volumes  added,  number  of  current  serials  received,  total  operating  expenses,  and  number  of  professional  and 
support  staff. 

[2]  Harvard,  Yale,  UCLA,  Toronto,  UC-Berkeley,  Stanford,  Illinois-Urbana,  Columbia,  Michigan,  Cornell,  Texas,  Washington, 
Penn  State,  Indiana,  Minnesota,  Wisconsin,  North  Carolina,  Princeton,  Chicago,  Pennsylvania,  Duke,  Virginia,  Ohio  State,  New 
York  University,  Iowa,  Rutgers,  Arizona  State,  Georgia,  Alberta,  Northwestern 
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Abstract:  The  University  of  Iowa  Libraries  recently  received  a two-year  $270,000  grant  from  the 
Roy  J.  Carver  Charitable  Trust  to  develop  a new  program  called  the  Science  Information  Literacy 
Initiative  (http://sciencelit.lib.uiowa.eduO,  which  is  designed  to  help  University  of  Iowa 
undergraduates  taking  courses  or  majoring  in  the  sciences  to  develop  skills  in  locating,  critically 
analyzing,  organizing,  and  applying  scientific  information.  Participating  academic  departments 
include  Biological  Sciences,  Chemistry,  Geology,  Mathematical  Sciences,  Psychology,  Science 
Education,  and  Engineering.  This  paper  will  describe  the  steps  taken  with  the  College  of  Liberal 
Arts  prior  to  receipt  of  the  grant  to  launch  the  more  general,  UIII  (University  of  Iowa  Information 
Literacy  Initiative)  (http://www.lib.uiowa.edu/user-ed/uiii/index.html).  Concepts  behind  planning 
and  implementing  the  Science  project  will  then  be  examined.  Project  staff,  science  librarians,  and 
faculty  design  discipline-specific  components  and  assignments  to  meet  a set  of  learning  objectives. 
Faculty  and  librarians  test  these  components  in  actual  courses  and  present  them  in  a variety  of 
formats  depending  on  the  course  requirements,  needs,  and  structure.  Information  literacy 
components  deemed  successful  will  be  further  developed  into  what  is  hoped  to  be  an  easily 
adaptable  program  for  widespread  use  throughout  the  sciences.  Issues  and  challenges  faced  in 
launching  the  Science  Information  Literacy  Initiative  will  also  be  featured. 


Why  Science  Information  Literacy? 

The  ability  to  locate,  critically  analyze,  organize  and  apply  scientific  information  is  an  important  skill  for  University 
of  Iowa  science  and  technology  graduates.  On  a daily  basis  students  preparing  for  professional  and  academic  careers 
in  science  and  technology  will  need,  to  correctly  locate,  evaluate,  and  apply  science  information  from  the  constantly 
growing  and  rapidly  changing  universe  of  print,  electronic,  multimedia,  and  web-based  sources. 

Advancing  information  literacy  in  the  sciences  and  technology  is  of  major  importance  at  the  highest  levels  of 
national  policy.  Albert  Henderson  writes  that  “an  ideal  vision  of  science  requires  goals  and  strategies  to  deal  directly 
with  the  growth  of  new  information.  “Information  Age”  science  policy  fails  to  do  this.  By  ignoring  the  study  of 
science  communications,  it  fosters  a policy  vacuum  on  information.”'  Although  the  more  general  concept  of 
“science  literacy”  has  been  recognized  as  a concern  and  challenge  in  the  U.S.,  particularly  since  the  post-Sputnik 
era,  proponents  have  focused  primarily  on  recruiting  students  into  the  sciences  and  heightening  the  public’s 
understanding  of  science  issues  rather  than  on  addressing  effective  scientific  information  seeking."  Academic 
libraries  have  not  developed  comprehensive  science-based  literacy  programs  although  many  science  librarians  have 
been  successful  in  promoting  certain  information  seeking  skills  such  as  specific  database  manipulation  and  web 
searching.  For  example,  a widely  cited  College  & Research  Libraries  article  reviewing  the  advances  of  information 
literacy  efforts  by  librarians  made  no  mention  of  specific  applications  in  science  or  technology.'"  The  University  of 
Iowa  Science  Information  Literacy  Initiative  seeks  to  address  this  void  for  the  campus  and  as  a potential  model  for 
other  institutions. 

The  Roy  J.  Carver  Charitable  Trust 

A major  factor  in  the  development  of  the  Science  Information  Literacy  initiative  was  the  interest  and  subsequent 
funding  of  the  project  by  the  Roy  J.  Carver  Charitable  Trust.  The  Carver  Trust  had  funded  the  successful 
Information  Arcade,  the  Information  Commons,  TWIST,  and  a new  Biological  Sciences  Library,  a total  of 
$2.13million  in  awards.  Senior  library  staff  developed  a prospectus  and  presentation  for  Carver  and  were  then 
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encouraged  to  submit  a formal  grant  proposal.  Funding  of  $270,000  was  recently  received  for  the  two-year  project. 
Funding  supports  a coordinator,  an  instructional  designer,  and  two  graduate  assistants. 

University  of  Iowa  Information  Literacy  Initiative  (UIII) 

Both  the  UIII  and  the  Science  Information  Literacy  Initiative  are  based  on  the  University  of  Iowa  Libraries’ 
philosophy  that  integrated  curriculum-based  user  education  in  the  most  effective  and  scaleable  approach  for 
developing  students’  portfolio  of  information  literacy  skills.  An  important  outgrowth  of  the  information  literacy 
programs  will  be  development  of  alternative  methods  for  faculty  to  incorporate  into  their  courses  for  each 
component  including  web-based  methods,  in  class  presentations,  use  of  existing  library  web  tutorials,  and  other 
means  since  librarians  are  not  numerous  enough  to  appear  in  every  introductory  class  at  a university  of  Iowa’s  size. 

Student  Learning  Objectives 

Development  of  relevant  student  learning  objectives  in  partnership  with  faculty  is  an  essential  part  of  the  Science 
Information  Literacy  Initiative.  Below  are  examples  of  objectives  likely  to  be  included  in  the  program: 

• Identify  problems  that  require  solutions  based  on  scientific  infomiation 

• Identify  appropriate  scientific  information  sources  and  execute  effective  search  strategies 

• Interpret  and  analyze  search  results 

• Critically  evaluate  scientific  information  retrieved 

• Organize,  synthesize,  and  apply  scientific  information 

• Understand  the  structure  of  the  information  environment  and  the  process  by  which  scholarly  and  other 
information  in  the  science  and  technology  fields  are  produced  and  disseminated 

• Understand  the  ethical  issues  related  to  access  and  use  of  scientific  information. 

Methodology  and  Project  Evaluation 

Science  Information  Literacy  project  staff  and  science  librarians  are  working  directly  with  select  faculty  in  each 
science  discipline  to  customize  curriculum  components  which  meet  the  student  learning  objectives.  Faculty  and 
librarians  will  test  these  curriculum  components  in  actual  courses  and  may  present  them  as  web-based  instruction, 
in-class  instruction  or  some  other  format  depending  on  course  requirements,  needs,  and  structure.  Evaluation  of  the 
project  will  be  accomplished  using  appropriate  tools  to  measure  students’  acquisition  of  the  learning  objectives. 
Information  literacy  components  deemed  successful  on  the  basis  of  student  outcomes  will  be  developed  into  an 
easily  adaptable  program  for  widespread  use  throughout  the  science  and  technology  curriculum. 

Initial  Projects.  Besides  enhancing  course  assignments,  the  University  of  Iowa  plans  to  develop  an  easier  interface 
for  beginning  level  students  in  the  sciences  who  need  to  access  various  science  information  databases.  This  part  of 
the  project  is  currently  in  the  initial  development  stage,  but  will  include  examination  of  various  basic  science 
databases  to  determine  how  beginners  might  best  choose  which  database  to  use  in  response  to  particular  needs. 

Conclusion 

Gradually  integrating  new  initiatives  into  the  existing  framework  is  sometimes  a slower  process  than  we  might  like, 
but  patience  developing  the  collaborative  process  often  pays  off  in  the  final  outcome.  We  have  at  this  point  laid  the 
groundwork  for  successful  implementation  of  our  project  goals  and  hope  to  report  on  our  further  progress  at  a later 
date. 
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Technology  Librarians  hip,  Fall  1998:1.  http://www.ucsb.edu/istl/98-fall/articlel.html 

“ Paisley,  William  J.  “Scientific  Literacy  and  the  Competition  for  Public  Attention  and  Understanding,”  Science  Communication 
(September  1998)  20(l):70-80. 

Behrens,  Shirley  J.  “A  Conceptual  Analysis  and  Historical  Overview  of  Information  Literacy,”  College  & Research  Libraries 
(July  1994)  55(4):  309-322. 
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Abstract.  Visualization  of  computational  models  is  at  the  heart  of  educational  software  for  com- 
puter science  and  related  fields.  In  this  paper  we  look  at  how  generation  of  such  visualizations  and 
the  visualization  of  the  generation  process  itself  increase  explorativity.  Four  approaches  of  increased 
explorativity  are  introduced. 


1 Introduction 

Educational  software  mostly  aims  at  width  of  knowledge  (facts)  and  is  not  able  to  teach  complex  processes. 
The  majority  of  systems  are  intrinsic  electronic  books,  encyclopaedias  and  dictionaries.  Process-oriented 
systems  exist  mostly  for  physics.  In  the  area  of  computer  science  tutorials  of  programming  languages 
prevail.  Particularly  in  computer  science  and  compiler  design  the  theory  and  algorithms  are  very  abstract 
and  usually  complex.  Therefore  visualizations  are  appropriate  for  computer  science  instruction.  Although 
compiler  design  is  often  considered  a practical  field  within  computer  science,  most  of  its  techniques 
are  based  on  work  in  theoretical  computer  science,  e.g.  formal  languages,  automata  theory  and  formal 
semantics.  In  recent  years  we  have  developed  several  educational  software  systems  for  topics  in  compiler 
design  and  theoretical  computer  science.  These  systems  have  in  common  that  they  teach  computational 
models  by  animating  computations  of  instances  of  these  models  with  example  inputs.  But  they  differ  in 
the  level  of  explorativity. 


Approach 

Input 

Computational  Model 

Generator 

Static 

fixed 

fixed 

none 

Interactive 

user 

fixed 

none 

First-order  generative 

user 

user 

yes 

Second-order  generative 

user 

user 

yes/visualized 

Table  1.  Levels  of  explorativity 

Table  1 not  only  reflects  the  increased  flexibility  of  the  software  developed,  but  also  the  chronological 
development  of  software  in  our  group.  Higher  levels  of  explorativity  demand  more  prerequisites  and  self- 
control  by  the  learner.  Thus,  in  the  educational  software  the  learner  should  start  with  static  examples 
and  as  the  user  advances  the  level  of  explorativity  should  be  increased.  Exercises  and  textual  hints  in  the 
educational  software  should  guide  the  learner,  to  make  sure  he/she  doesn’t  miss  the  important  issues. 

2 Approaches 

Static  Approach  In  the  static  approach  the  execution  of  an  instance  of  a computational  model  is  animated 
for  a given,  fixed  input.  The  educational  software  ’’Animation  of  Lexical  Analysis”  [Braune  et  al.  99]  is 
an  example  for  this  approach  (http://www.cs.uni-sb.de/RW/anim/animcomp_e.html). 

Interactive  Approach  In  the  interactive  approach  an  instance  of  a computational  model  is  animated  for 
an  example  entered  by  the  user /learner.  An  example  for  this  approach  is  our  application  ’’Animation  of 
Semantical  Analysis”  [Kerren  99]  (http://www.cs.uni-sb.de/RW/anim/animcomp_e.html). 

First-Order  Generative  Approach  In  the  first-order  generative  approach  the  user  enters  the  specification  of 
an  instance  of  a given  computational  model.  Then  an  interactive  visualization  of  this  instance  is  generated 
and  the  user  can  enter  an  example  input  as  in  the  interactive  approach.  As  an  example  of  the  first-order 
generative  approach  we  present  GANIMAM  [Diehl  et  al.  99],  our  web-based  generator  for  interactive 
animations  of  abstract  machines  (Fig.  1 and  http://www.cs.uni-sb.de/RW/users/ganimal/GANIMAM/). 
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' Fig.  1.  Screenshot  of  an  animated  abstract  machine 

Second- Order  Generative  Approach  As  in  the  first-order  generative  approach  the  user  enters  a specifi- 
-cation  of  an  instance  of  a given  computational  model.  But  in  the  second-order  generative  approach  in 
addition  to  visualizing  the  computation  also  the  generation  process  is  shown  as  an  interactive  visualiza- 
tion. An  example  of  this  approach  is  our  application  GANIFA,  a web-based  generator  for  animated  finite 
automata  (http://www.cs.uni-sb.de/RW/users/ganimal/GANIFA/). 


3 Explorativity  and  Learner  Control 

In  conventional  educational  software  answers  of  an  exercise  are  checked  for  correctness.  Unfortunately 
such  correctness  checking  is  not  possible  in  most  interesting  cases.  In  computer  science,  many  properties 
of  computational  models  can  not  be  checked  because  of  the  halting  problem.  As  a consequence  we  need 
alternative  ways  to  provide  feedback  for  the  learner. 

In  the  generative  approaches  an  interactive  animation  is  produced  from  the  response  (specification) 
of  the  learner.  Then  the  learner  can  test  it  on  the  basis  of  own  examples.  In  this  way  he/she  can  detect 
errors.  The  generative  approach  offers  a way  for  explorative,  self-controlled  learning.  The  learner  can 
focus  on  certain  aspects  in  the  generated,  interactive  animation  and  see  what  effects  small  modifications 
in  the  .specification  have.  With  the  help  of  such  observations  he/slie  formulates  hypothe.ses  and  checks 
these  empirically.  In  the  interactive  approach  such  checkable  hypotheses  are  restricted  to  the  instance  of 
the  computational  model.  In  the  first-order  generative  approach  also  hypotheses  about  the  computational 
model  and  in  the  second-order  generative  approach  about  the  generation  process  itself  can  be  checked. 
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Abstract:  New  technology  - Microcomputer  Based  Lab  (MBL)  in  particular  - was  beneficially 
used  to  support  Mechanics  teaching  at  Junior  High  School  students.  Two  series  of  seven 
experiments  each  were  developed  and  implemented  at  five  schools,  including  control  and 
experimental  groups  of  approximately  two  hundred  students  in  total.  The  experiments  focused  on 
linear  motion,  Newton  laws  and  conservation  laws.  The  students  of  the  experimental  group 
demonstrated  a significantly  more  successful  ability  of  distinguishing  among  position,  velocity  and 
acceleration  versus  time  graphs.  Furthermore,  they  appeared  to  better  understand  the  proportional 
relation  between  force  and  acceleration,  the  equality  between  action  and  reaction,  as  well  as 
momentum  conservation  and  mechanical  energy  conservation  laws.  In  conclusion,  new  technology 
enables  students  to  build  scientific  concepts  taking  profit  of  their  own  experience  of  real  world. 


Introduction. 

As  we  approach  the  new  millenium,  a general  attitude  is  developed  towards  the  improvement  of  quality  and 
effectiveness  of  the  provided  education,  regarding  Natural  Sciences,  which  has  two  objects.  The  scientific  and 
technological  "literacy"  of  the  average  citizen,  on  one  hand,  and  on  the  other,  the  preparation  of  future  scientists  and 
instructors  so  as  to  utilize  the  outcomes  of  the  rapid  scientific  and  technological  developments  of  our  times.  Since 
the  beginning  of  our  decade,  an  intense  concern  has  been  created  among  the  natural  sciences  education  researchers 
about  the  effectiveness  of  the  traditional  instructive  system  as  far  as  the  understanding  of  the  natural  world  on  behalf 
of  the  students  is  concerned.  Almost  everybody  seems  to  agree  that  listening  to  someone  talking  about  the  natural 
phenomena  does  not  constitute  a productive  way  of  understanding  the  laws  and  concepts  of  nature.  Students  of  all 
ages  approach  science  better  by  actively  participating  in  the  examination  and  the  interpretation  of  the  natural 

phenomena . 

Subject/problem. 

Many  researchers  hold  the  view  that  the  use  of  new  technologies  will  supply  students  with  the  ability  to  successfully 
take  in  concepts  of  physics  by  putting  to  use  their  own  experiences  from  the  natural  world.  One  of  the  interactive 
educational  technologies  is  the  real  - time,  computer-based  data-logging  tools  (often  called  microcomputer-based 
laboratory  or  MBL).  These  new  methods  using  the  L were  implemented  for  teaching  the  Newtonian  Mechanics 
to  first-year  students  and  schoolchildren  of  the  last  classes  of  senior  high  school  in  the  USA,  leading  to  particularly 

positive  results  . Up  until  now  and  to  our  knowledge,  no  attempt  has  been  made  towards  the  utilization  of  new 
technologies  regarding  the  teaching  of  Physics  to  students  of  younger  age.  The  Greek  syllabus  does  not  give  such  an 
opportunity,  since  it  provides  the  teaching  of  the  elementary  concepts  of  the  Newtonian  Mechanics  in  Grammar 
school  (age  13-15). 

Design  and  procedure. 

In  order  to  investigate  the  effectiveness  of  the  new  teaching  methods  of  Physics  in  the  low  secondary  education  we 
put  to  practice  a project  with  the  general  title  «Physics  and  New  Technologies)).  The  chapters  of  Heat  and 
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Mechanics  were  chosen  based  on  the  criterion  of  their  position  in  the  Greek  curriculum  and  their  suitability  for  the 
development  of  laboratory  experiments  in  an  MBL  environment.  The  part  of  the  suggestion,  which  is  related  to 
Mechanics,  is  presented  in  this  paper.  Two  series  of  7 experiments  were  designed;  one  for  the  groups  of  control 
classes  and  one  for  the  classes  of  experimental  application.  The  duration  of  every  laboratory  experiment  was  the 
same  for  both  classes.  The  experiments  included  all  the  basic  topics  of  Mechanics  at  this  level:  linear  motions, 
Newton's  laws  and  conservation  of  momentum.  The  design  of  the  experiment  for  the  control  classes  was  based  on 
the  electrical  ticker-timer  and  the  cart  that  comes  along  with  it.  The  experimental  application  classes  used  various 
simple  means  (their  hands,  little  wagons,  toys  etc.)  as  mobiles,  and  worked  in  MBL  environment,  which  disposed  a 
computer,  proper  software,  a printer,  a projection  screen,  a video  projector,  sensors  of  position  and  force,  as  well  as 
an  interface  for  the  sensors  and  the  computers.  Five  grammar  schools  in  Athens  were  selected  to  be  school  units  of 
application,  so  that  the  sample  would  represent  the  average  Greek  student.  The  experiment  and  control  classes 
operated  equally  in  every  school  unit,  on  every  aspect.  The  teachers  were  the  same  for  both  classes.  Students  in  both 
classes  worked  under  the  supervision  of  two  teachers  and  they  filled  in  worksheets.  The  efficiency  of  each  teaching 
intervention  was  evaluated  through  a questionnaire,  common  for  both  classes.  (The  filling  in  of  the  questionnaires 
was  conducted  a week  after  every  laboratory  experiment). 


Results. 

The  results  presented  here  concern  concepts  of  kinematics  motion  and  immobility,  velocity,  acceleration,  gravity 
acceleration  while  research  related  to  dynamics  is  in  progress.  As  a sample  of  the  experiment  we  mention  the 
following:  One  out  of  two  students  of  the  experimental  group  E G (48,5%)  and  one  out  of  five  of  the  control  group 
C G (20,3%)  recognized  motion  with  uniform  velocity  in  a positioni/time  diagram.  The  difference  is  more  striking  in 
the  identification  of  the  immobile  condition:  E G (68,7%),  C G (25,2%);  as  well  as  the  backward  motion:  E G 
(67,7%)  and  C G (13,1%).  At  this  point  we  should  mention  that  the  electrical  ticker-timer  or  every  other  «typical» 

means  cannot  register  the  motion  backwards  . At  a linear  motion  with  constant  acceleration,  80%  of  the  E G and 
about  55%  of  the  C G understood  the  concept  of  velocity,  while  for  the  acceleration  the  results  were:  E G (65%)  and 
C G (50%).  Finally,  for  the  case  of  the  free  fall  of  a rubber  ball  and  a metal  sphere,  60%  of  the  E G and  40%  of  the 
C G understood  that  the  two  bodies  have  the  same  acceleration. 

Conclusions. 

Based  on  these  results,  it  becomes  obvious  that  the  use  of  new  technologies  in  the  Physics  laboratory  offers  the 
possibility  of  a simultaneous  supervision  of  motion  and  of  the  model  of  its  representation  (diagram).  This  activity  - 
based,  computer  - supported,  interactive  learning  environment  helps  students  of  this  age: 

1 . To  identify  the  trajectory  of  a motion  from  the  depicting  motion  diagram.  Not  to  confuse  the  position,  velocity 

and  acceleration  diagrams  in  relation  to  time^^^  . To  discern  the  analogy  of  force  - acceleration.  To  comprehend 
the  third  law  of  Newton  and  the  conservation  of  momentum,  which  derives  from  it. 

2.  To  acquire  the  ability  to  describe  the  behavior  of  a natural  system  based  on  the  model  it  depicts  . 

References: 

1.  NATO  Advanced  Study  Workshop:  Student  development  of  physics  concepts:  The  role  of  educational 
technology,  (Pavia,  Italy,  1989) 

2.  Redish,  E.F.  at  all  On  the  effectiveness  of  active-engagement  MBL,  American  j.  of  Physics  65  (1)  45-54,(1997) 

3.  Thornton,  R.K.(1999)  Using  the  results  of  research  in  science  education  to  improve  science  learning,  1th 
International  Conference  on  Research  in  Science  education.,  Cyprus  3-5  January  1999. 

4.  Panagiotis  Dimitriadis  at  all:  Linear  motion  study  through  graphs  -A  new  Technology-based  learning 
environment, 2"*^  International  Conference  of  the  European  Science  Education  Research  Association  Proceedings 
,Kiel,  Germany,  August  31  - September  4,  1999. 

5.  Me  Dermott,  L.,  Rosenquistm,  M.,  van  Zee,  E.  Student  difficulties  in  connecting  graphs  and  physics:  Examples 
for  kinematics,  American  J.  of  Physics55(6)503-513,  (1987). 

6.  Hestenes  D.  :Modeling  games  in  the  Newtonian  Word,  American  j.  of  Physics,  60  (8),  732-748,  (1992) 


Supporting  Developers  of  Inquiry  Based  Instruction: 
Performance  Support  on  the  Web 


Jackie  Dobrovolny 
University  of  Colorado  at  Denver 
12281  E.  Villanova  Drive 
Aurora,  CO.  80014 
jdoffice@home.com 

Scott  Grabinger 

University  of  Colorado  at  Denver 
Campus  Box  106 
P.O.  Box  173364 
Denver,  CO  80217-3364 
scottgrabinger@ceo.cudenver.edu 

Jenn  Light 

University  of  Colorado  at  Denver 
Campus  Box  106 
P.O.  Box  173364 
Denver,  CO  80217-3364 
Jenn_Light@ceo.cudenver.edu 


Abstract;  The  University  of  Colorado  at  Denver,  Indiana  University,  Western  Governors 
University,  Educause/IMS  and  Phi  Delta  Kappa  are  working  on  a five  year  FIPSE  (Fund  for  the 
Improvement  of  Post-secondary  Education)  grant  to  develop  the  Learning  to  Teach  with 
Technology  Studio  (LTTS).  The  LTTS  is  an  Internet-based,  anywhere,  anytime  learning 
environment  designed  to  help  KI2  teachers  integrate  technology  into  their  instruction.  The 
instruction  in  the  LTTS  is  designed  from  an  inquiry  based  learning  model.  A key  objective  of  this 
project  is  to  ensure  that  it  is  replicable  and  self-sustaining.  The  strategy  for  achieving  this  goal  is  to 
develop  an  initial  set  of  modules  and  then  invite  other  professionals  in  the  field  (teachers,  university 
faculty,  technology  coordinators)  to  contribute  modules.  Thus,  it  is  critical  to  provide  strong 
support  for  the  multitude  of  authors  that  may  contribute  to  this  project.  The  Design  Strategy 
Environment  (DSE)  provides  this  support;  it  is  a resource-rich  environment  that  helps  developers 
create  high  quality,  self-contained,  problem-centered  learning  modules. 


Background 

The  National  Center  for  Education  Statistics  (1999)  found  that  80%  of  K12  teachers  do  not  believe  they  are  prepared 
to  integrate  technology  into  their  curricula  and  the  Milken  Exchange  (Solmon,  1998)  found  that  teachers  do  not 
model  the  integration  of  IT  skills  in  their  teaching.  One  of  the  reasons  for  this  ill -preparedness  is  a lack  of  relevant 
instruction  for  both  pre-service  and  in-service  teachers. 

To  meet  this  need,  the  University  of  Colorado  at  Denver  (UCD),  Indiana  University  (lU),  Western  Governors 
University  (WGU),  Educause/IMS  and  Phi  Delta  Kappa  began  work  in  September,  1999  on  the  Learning  to  Teach 
with  Technology  Studio  (LTTS),  an  Internet-based,  anywhere,  anytime  learning  environment.  The  LTTS  will 
provide  self-paced  instruction  in  the  following  six  content  areas:  (1)  design  of  learning  environments,  (2) 
measurement  and  evaluation,  (3)  technology  and  production  proficiency,  (4)  project  management,  (5)  leadership, 
and  (6)  applied  research.  This  work  is  funded  by  a five-year,  FIPSE  grant. 
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The  instruction  in  the  LTTS  uses  an  inquiry  based  learning  (IBL)  model;  each  instructional  unit  is  structured  around 
a problem-based  case.  While  many  potential  developers  probably  have  prepared  materials  for  information 
transmission-based  online  courses,  few  have  experience  in  preparing  inquiry-based  online  activities.  Shifting  to  an 
inquiry-based  mode  of  instruction  is  a difficult  process  for  many  educators,  a fact  confirmed  by  the  professional 
development  literature  (e.g.,  Richardson,  1998).  In  our  extensive  experience  offering  problem-based  learning 
workshops,  we  have  found  that  nearly  all  educators  willingly  accept  the  idea  that  students  must  learn  to  solve 
problems,  work  with  others,  and  create  meaningful  products.  However,  we  have  also  found  that  they  are  reluctant  to 
change  their  teaching  methods,  believing  that  their  students  need  to  know  “stuff”  Unfortunately,  the  information 
transmission  model  of  learning  conflicts  with  the  inquiry  approach  to  instruction, 

The  change  to  an  inquiry  based  learning  model  is  even  more  complicated  on  the  LTTS  project  because  the 
instruction  is  self-paced  and  web-based,  Most  of  the  literature  and  guidelines  for  developing  inquiry  based  learning 
are  designed  for  group-paced,  classroom-based  instruction.  Further,  in  addition  to  developers  from  Indiana 
University  and  the  University  of  Colorado  at  Denver,  we  will  invite  other  professionals  in  the  field,  e.g.,  teachers, 
university  faculty  and  technology  coordinators,  to  contribute  modules  to  the  LTTS.  Thus,  we  anticipate  the 
possibility  of  having  hundreds  of  developers  contribute  instructional  material  to  the  LTTS. 


Design  Strategy  Environment 

To  assist  and  support  those  who  are  developing  instruction  for  the  LTTS,  we  are  creating  the  Design  Strategy 
Environment  (DSE),  The  DSE  is  a resource-rich,  performance  support  site,  which  provides  instructional  guidance, 
templates,  and  examples  for  developers.  Our  goal  for  the  DSE  is  to  make  the  work  of  our  developers  as  efficient  as 
possible,  and  to  insure  a consistent  look  and  feel  of  the  interface.  In  order  to  achieve  this  goal,  the  DSE  uses  forms, 
templates  and  other  automated  techniques. 

We  provide  a Developer  Notes  Worksheet  which  developers  can  use  as  they  review  the  DSE  site.  The  purpose  of 
the  worksheet  is  for  developers  to  record  their  ideas,  notes,  reflections  and  concerns  as  they  research  and  review  the 
information  in  the  DSE.  TTiis  worksheet  is  divided  into  four  sections  that  correspond  to  the  four-step  development 
process:  (1)  Describe  the  problem,  (2)  Describe  and  define  the  resources,  (3)  Describe  the  learning  activities,  and 
(4)  Describe  the  assessment  criteria. 

For  each  of  these  four  steps,  the  DSE  provides  the  following  information: 

• A brief  description  (50-100  words)  of  that  component  of  the  inquiry  based  instruction.  For  example,  the 
brief  description  of  the  first  step  (Define  the  problem),  indicates  that  a good  IBL  problem  is  ill-structured, 
situated,  authentic  and  complex. 

• Links  to  our  examples.  These  examples  are  other  LTTS  modules  that  contain  comments  and  notes.  That 
is,  for  the  first  step  (Define  the  problem),  an  example  problem  is  displayed  in  the  first  column  of  a two- 
column  table.  The  second  column  contains  comments,  notes  and  explanations  about  how  this  problem 
exemplifies  the  critical  characteristics  of  a good  problem. 

• Link  to  detailed  explanation.  This  information  is  essentially  an  elaboration  on  the  brief  description  (a. 
above).  It  contains  not  only  a full  description  of  the  topic,  but  also  provides  a reference  list  and  links  to 
LTTS  examples. 

• Link  to  necessary  features.  This  is  a checklist  developers  can  use  to  evaluate  their  work.  It  is  a list  of  the 
necessary  features  each  of  the  four  components  of  their  module  must  contain. 
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Abstract:  This  is  a research  project  conducting  a comparison  of  traditional  classroom  instruction 
and  self-paced  multimedia  training  in  "advocacy"  for  people  with  disabilities.  The  three  main 
populations  are  Deaf  sign  language  users,  blind  users  who  do  not  read  print,  and  people  using 
alternative  input,  such  as  Dragon  Dictate  or  puff  and  sip  switches.  Technical  specifications  for 
access  to  html  using  WWW3  universal  design  standards  will  be  discussed.  Advantages  and 
disadvantages  to  both  classroom  and  virtual  training  will  be  compared  using  evaluation  data. 


Comparing  Methods  for  Delivering  Advocacy  Training 

Advocacy  is  a skill  required  for  navigating  life  as  a person  with  a disability.  The  Canadian  Association  of 
Independent  Living  Centres  was  funded  by  the  Office  of  Learning  Technologies  to  conduct  a research  and 
demonstration  project  to  understand  the  differential  impacts  of  learning  technologies  on  people  with  disabilities. 

Traditional  Classroom  Delivery 

The  advantage  of  classroom  delivery  is  the  interactivity  and  direct  feedback  available  in  real  time.  Students  may  ask 
questions,  have  them  answered  and  participate  in  group  activities.  For  Deaf  students,  another  advantage  is  that  the 
course  content  is  delivered  directly  in  sign  language.  Although  handouts  and  exercises  are  in  written  English,  there  is 
an  opportunity  to  ask  directly  for  clarification  and  access  new  concepts  in  sign  language.  Students  who  do  not  read 
print,  and  who  process  information  aurally,  are  able  to  benefit  also  from  spoken  discussions,  role  plays  and 
interactivity.  No  reading  or  print  processing  is  required  in  a traditional  classroom  situation.  The  provision  of 
handouts  in  alternate  formats-  such  as  in  Braille,  diskette  or  on  audiotape,  also  enables  learners  to  access  material 
after  the  class  is  completed.  Because  the  classroom  course  delivery  involves  other  peers,  similarly  situated  blind 
people,  the  participants  directly  benefit  from  experiences  of  others.  This  is  a key  in  the  process  of  adult  learning.  For 
students  who  cannot  use  text  input  due  to  physical  limitations  or  learning  disabilities  classroom  participation  can  be 
very  enabling.  The  verbal  participation  and  support  given  by  human  staff  in  a classroom  situation  may  enable 
learners  to  avoid  some  of  the  limitations  that  their  disabilities  create  when  interacting  with  text  or  technology.  For 
example,  even  turning  a page  of  a workbook  may  require  a technical  aid  for  a quadripelegic  and  a person  with  a 
learning  disability  may  need  a screen  reader  to  process  assignments  in  writing.  Having  a human  instructor  also 
provides  the  flexibility  of  gearing  the  course  to  the  topical  and  learning  needs  of  participants. 

Virtual  Training  Delivery 

This  project  was  designed  to  test  virtually  learning  in  comparison  to  classroom  learning.  Realizing  that  private 
citizens  have  a wide  range  of  access  to  computers  and  the  internet,  the  "lowest"  common  denominator  was  selected 
for  delivery.  (Cantor  1998)  CD  technology  using  html  allowed  any  one  with  basic  internet  software  and  NO 
connection  to  the  internet  to  take  the  course.  Students  would  be  able  to  email,  phone,  fax  or  even  write  to  instructors. 
(Burgstahler  1995)  By  providing  OFF  line  access,  students  without  fast  modems,  students  with  time  limited  access 
and  people  who  have  CDROM  drives  but  no  internet  access  can  still  benefit  from  the  course  material.  This  is 
particularly  important  for  disabled  learners  who  use  augmentative  technology  that  slows  down  participation  and 
interaction.  (Coombs  1999)  Participation  in  the  on-line  version  of  the  course  (same  content)  will  only  be  different 
because  it  allows  direct  emailing  to  the  instructor  and  potentially  a response  within  a very  short  time.  There  will  be 
on  interaction  with  other  learners  in  this  project,  but  students  will  be  asked  at  the  end  of  the  project  if  that  was 
something  they  would  be  interested  in.  A bulletin  board  style  noteboard  will  be  used  for  students  to  post  their 
comments  about  the  course  and  any  suggestions-  either  anonymously  or  through  their  own  email  address.  (Brooks 
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1 997)  The  text  only  version  of  the  course  will  enable  blind  users  to  access  the  text  and  exercises  through  both  screen 
readers  and  audio  files  without  navigating  through  frames.  The  deaf  users  and  the  people  using  alternative  input  will 
benefit  from  the  enhanced  visual  navigation  and  easy  to  use  self-pacing.  Interactive  exercises,  sign  language  video 
and  audio  files  will  be  available  on  demand.  The  advantages  of  these  processes  include  a slower  or  more 
individualized  pace  of  learning,  opportunity  to  repeat  and  review  more  frequently,  a chance  to  complete  exercises 
before  and  after  course  work,  and  the  ability  to  stop  the  process  and  return  to  it  at  any  time.  Classroom  teaching 
doesn't  allow  this  flexibility  and  the  presence  of  other  students  often  reduces  the  opportunity  to  individual  the  steps 
to  learning  achievement. 

Training  Design  Standards 

While  WWW  standards  for  accessibility  do  exist,  and  the  WWW  consortium  has  an  ongoing  list  of  requirements, 
(Web  Accessibility  Initiative  1 999)  html  is  not  always  used  to  deliver  training.  Other  sources  of  accessibility  design 
standards  will  also  be  used  to  inform  the  training  design  standards.  (Waddell  1998;  Vanderheim,  Chisholm  & Ewers 
1996)  This  project  seeks  to  combine  accessibility  standards  with  training  design  standards  to  ensure  that  web/on  line 
training  is  as  flexible  and  accessible  as  possible  for  diverse  learners.  This  project  is  still  in  progress  but  the  following 
technical  specifications  have  been  used  to  implement  the  project.  The  starting  point  for  the  content  was  the  pre-post 
test  which  would  be  used  to  measure  achievement  of  skills  and  knowledge  about  advocacy.  The  contents  of  the  in 
class  instruction  and  the  multi-media  instruction  had  to  be  compatible  so  we  began  with  the  text  and  graphical 
content.  Images  were  imported  from  Clip  Art  and  custom  drawn  by  an  artist.  All  images  have  alt- tags  but  the  html 
text-only  site  does  not  rely  on  graphics  for  navigation.  (Khan  1 997)  In  addition,  we  provided  sign  language  versions 
of  instructions  and  difficult  vocabulaiy  which  was  accessible  by  clicking  a particular  location  identified  by  signing 
hands.  In  addition,  the  "hot  spots"  or  links  in  the  software  were  made  to  work  without  timing  limits  and  through 
keystrokes  so  that  people  using  Dragon  Dictate  and  alternative  input  switches  did  not  always  need  to  move  the 
"mouse"  but  could  directly  control  the  navigation.  By  building  in  accessibility  at  the  design  stage,  more  flexibility  for 
consumers  is  achieved.  (Waddell  1 999)  Evaluation  to  date  has  shown  that  disabled  participants  benefit  from  both  in 
class  and  on-line  delivery  methods.  Different  individual  preferences,  access  to  technology  and  learning  needs  help 
determine  which  suits  each  student.  Design  standards  will  be  developed  as  a result  of  this  project  to  increase  access 
to  future  training  initiatives. 
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Abstract:  In  this  text,  we  explore  the  similarities  between  engineering  and  instructional  design,  in 
terms  of  methodology,  tools  used  and  emerging  trends  (growing  consideration  for  the  client, 
research  of  quality,  prototyping,  concurrent  approach).  We  conclude  that  the  expression 
“instructional  engineering”  is  amply  justified. 


Introduction 

In  recent  years,  instructional  design  (ID)  is  more  and  more  often  equated  with  instructional  engineering  (Paquette, 
Crevier  and  Aubin,  1999).  Engineering  is  a process  covering  the  whole  life-cycle  of  a product.  The  engineering 
process  is  generic  as  is  applied  in  all  fields  where  one  has  to  image  a solution  that  will  satisfy  the  needs  of  potential 
customers  whether  those  needs  be  educational,  financial,  medical,  etc.  When  this  process  is  put  to  use  to  solve  a 
problem  industrial  in  nature,  it  is  said  to  be  applied  to  the  field  of  engineering.  Similarly,  the  ID  process  has  been 
traditionally  defined  as  this  whole  cycle  of  analysing,  designing,  producing  (or  developing),  evaluating, 
implementing  and  revising  a learning  system.  In  this  sense,  the  expression  “instructional  engineering”  is  well 
justified.  As  we  will  see  throughout  this  presentation,  it  is  not  the  only  reason. 

Tools 

In  both  fields,  engineering  and  education,  introduction  of  computers  has  greatly  increased  the  productivity  of  the 
designers  by  easing  the  creative  process,  by  facilitating  modifications,  by  performing  simulations  more  rapidly.  But 
what  about  supporting  decision  making  within  the  engineering  process? 

In  1 992,  Spector  and  his  colleagues  predicted  that  the  automation  of  the  instructional  engineering  process  (through 
the  use  of  expert  systems)  would  evolve  in  much  the  same  way  as  word  processors  have.  Unfortunately,  the  passage 
of  time  has  not  proven  them  right.  The  use  of  expert  and  other  knowledge-based  systems  is  much  more  frequent  in 
engineering.  This  may  be  explained  by  the  fact  that  a large  portion  of  the  body  of  knowledge  is  rooted  in  “hard” 
science  as  opposed  to  “soft”  science,  much  more  open  to  debate,  in  the  field  of  education.  Furthermore,  the  rules 
included  in  educational  expert  systems  are  much  more  difficult  to  formulate. 

Quality 

One  usually  associates  quality  with  a lack  of  defects,  with  reliability.  In  engineering,  nowadays,  a quality  product  is 
one  that  responds  to  the  needs  of  the  customer.  One  can  therefore  never  overemphasize  the  importance  of  the  first 
phase  of  the  engineering  process:  problem  definition.  This  is  true  irrespective  of  the  field. 
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Traditionally,  problem  definition  has  been  performed  through  analysis.  A growing  number  of  companies  developing 
new  engineering  products  rely  on  Quality  Functional  Deployment  (QFD),  a method  devised  to  clearly  identify 
customer  needs.  The  method  rests  on  a graphical  tool  called  the  House  of  Quality.  While  developing  a technology- 
enriched  course,  we  used  this  tool  and  found  it  to  be  a highly  effective  communication  and  design  tool. 

Another  mean  of  achieving  quality  in  engineering  is  through  a prototyping  approach  as  proposed  by  Smith  (1991), 
which  is  also  an  emerging  trend  in  education  (Moonen,  1996;  Tripp  and  Bichelmeyer,  1990).  Needs  and 
requirements  are  no  longer  viewed  as  input  but  as  output  of  the  design  phase.  The  goals  of  the  prototypes  are  to 
solicit,  to  elucidate,  to  define  customer  needs  and  requirements.  The  method  is  useful  if  a prototype  can  be  easily 
and  quickly  produced,  thus  the  expression  “rapid  prototyping”.  In  general,  the  more  specific  the  tool  used  (from 
general  programming  languages  to  learning  systems  shells),  the  more  quickly  the  prototype  is  produced.  Merrill 
(1996)  does  warn  however  that  education  authoring  tools  do  require  some  type  of  programming,  that  they  are  not 
neutral  with  regard  to  instructional  theory  and  that  the  user  must  have  some  knowledge  of  ID.  Ignoring  these 
warnings  may  result  in  an  inadequate  product,  that  is  one  that  lacks  quality! 

Concurrent  engineering 

The  classical  work  organization  is  sequential.  This  type  of  organization  is  often  referred  to  as  “over-the-wall”  or 
“waterfall”  since  each  team  works  independently  of  one  another,  with  minimal  communication.  This  lack  of  true 
communication  results  in  multiple  errors,  increase  in  development  time  and  cost.  Concurrent  engineering  was 
devised  to  overcome  these  shortcomings.  The  proposed  solution  is  simple  : create  a single  multidisciplinary  team, 
whose  members  work  concurrently  on  different  aspects  of  the  product.  A mounting  body  of  literature  is  available  to 
document  the  effectiveness  of  the  concurrent  engineering  process  when  applied  to  the  field  of  engineering. 
Meanwhile  timid  incursions  are  observed  in  the  field  of  education.  Most  classical  ID  models  have  a bias  for 
sequential  engineering  with  numerous  feedback  loops  providing  for  iterations,  very  few  incorporate  task  parallelism. 
However,  there  exists  a few  attempts  to  embed  concurrent  engineering  in  ID  (Paquette,  Crevier  and  Aubin,  1 999). 


Conclusion 

Our  analysis  shows  that  the  expression  “instructional  engineering”  in  amply  justified.  The  similarities  between 
engineering  and  ID,  in  terms  of  methodology,  tools  used  and  emerging  trends  (growing  consideration  for  the  client, 
research  of  quality,  prototyping,  concurrent  approach)  are  quite  amazing.  Up  to  now,  there  have  been  very  few 
occurrences  of  borrowing  engineering  principles  to  shed  new  light  in  the  field  of  ID  (Teslow,  1997;  Yang,  Moore 
and  Burton,  1995).  Our  own  experience  of  combining  ID  and  engineering  expertise  to  the  task  of  developing  a 
computerized  learning  environment  and  to  reflect  together  on  the  theoretical  underpinnings  of  both  fields  has  been 
an  experience  which  gains  to  be  repeated  by  others  and  become  a practice  more  widely  spread. 
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Abstract:  The  College  of  Arts  and  Sciences  (CAS)  at  George  Mason  University  has 
launched  a program  - Technology  Across  the  Curriculum  (TAC)  - to  promote  student  use 
of  information  technology.  The  TAC  initiative  supports  faculty  to  develop  technology- 
enhanced  course  assignments  with  funding  provided  by  the  Vice-President  for 
Information  Technology.  Through  their  work  to  promote  these  innovations,  faculty  have 
opportunities  to  take  a deeper  look  at  their  teaching  and  scholarship.  Students  who  enroll 
in  TAC  courses  are  not  only  empowered  with  advanced  tools  for  problem  solving,  they 
also  acquire  a conceptual  understanding  of  “applied”  technology  so  that  the  utility  of 
these  tools  can  be  transferred  to  different  domains  over  time. 


The  College  of  Arts  and  Sciences  (CAS)  at  George  Mason  University  has  launched  a program  - Technology  Across 
the  Curriculum  (TAC)  - to  promote  student  use  of  information  technology.  The  TAC  initiative  supports  faculty  to 
develop  technology-enhanced  course  assignments  with  funding  provided  by  the  Vice-President  for  Information 
Technology.  Through  their  work  to  promote  these  innovations,  faculty  have  opportunities  to  take  a deeper  look  at 
their  teaching  and  scholarship.  Students  who  enroll  in  TAC  courses  are  not  only  empowered  with  advanced  tools 
for  problem  solving,  they  also  acquire  a conceptual  understanding  of  “applied”  technology  so  that  the  utility  of  these 
tools  can  be  transferred  to  different  domains  over  time.  With  this  extensive  exposure  to  a wide  range  of  information 
technologies,  CAS  students  graduating  from  George  Mason  will  have  an  education  that  is  both  grounded  in  the 
critical  perspective  of  the  liberal  arts  and  bolstered  by  new  ways  of  thinking  for  the  digital  age.  The  broader 
community  served  by  the  University  is  the  ultimate  beneficiary  of  the  TAC  program.  Present  and  future  generations 
of  students  will  enter  the  region’s  workforce  confidant  in  their  ability  to  master  new  technologies  and  therefore 
better  prepared  to  face  the  challenges  of  a rapidly  changing  global  economy. 

The  TAC  program  has  identified  three  basic  types  of  technology  competencies  that  would  have  practical  value  in  the 
classroom  as  well  as  the  workplace:  information  management,  electronic  collaboration,  digital  authoring.  Each  is 
supported  by  a subset  of  specific  tools  and  activities  that  will  be  an  integral  part  of  course  assignments  and  the 
learning  outcomes  intended  for  TAC  courses. 

1 . Information  Management 

Students  will  collect,  organize,  evaluate,  and  analyze  different  types  of  data.  They  will  conduct  research  using 
web-based  directories  and  search  engines  or  on-line  library  catalogs.  They  will  evaluate  digital  archives  using 
sources  and  citations.  They  will  use  databases  to  create  relational  data  structures  that  are  the  basis  for  queries 
and  reports.  They  will  use  spreadsheets  for  data  entry,  plotting  and  graphing  charts,  and  for  descriptive 
statistics.  They  will  use  geographical  information  systems(GIS)  for  spatial  analysis  for  descriptive  statistics, 
economic  geography,  and  environmental  data. 

2.  Electronic  Collaboration 

Students  will  engage  in  online  collaboration  for  learning  and  to  produce  knowledge.  They  will  communicate 
using,  e-mail,  listserves,  and  asynchronous  conferences.  The  will  exchange  documents  and  other  materials 
using  e-mail  attachments,  FTP,  and  netware.  They  will  collaborate  on  group  projects  and  organize  workflow 
using  online  tools  that  permit  conferencing  and  the  posting  of  documents. 

3.  Digital  Authoring 

Students  will  be  able  to  present  information  in  an  engaging,  effective  manner  using  digital  authoring  tools.  They 
will  design,  develop  and  maintain  websites  using  an  HTML  editor.  They  will  learn  visual  principles  of 


A 


communication(effective  color  and  graphics,  text  treatment,  and  audience  targeting)  using  PowerPoint 
a technology-enhanced  presentations. 
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Overview:  In  this  paper  we  describe  a layered  approach,  which  allows  the  adaptation  of  the 
visualization  content  according  to  the  user's  needs  using  meta-data.  In  contrast  to  existing 
approaches,  the  educator  can  adjust  both  the  level  of  explanation  and  the  level  of  interactivity  of  an 
animation,  thus  influencing  the  presentation  and  the  results  of  the  algorithms  being  illustrated 
according  to  a desired  level  of  functionality  and  appearance,  suitable  for  the  specific  needs  of  the 
learner.  Our  approach  is  similar  to  a middleware  approach.  In  this  three-tier  model,  the  lowest  layer 
of  this  paradigm  suits  the  programmer.  On  the  middle  layer,  which  is  composed  of  various  services 
common  to  all  educational  animations,  such  as  manageme;nt  of  data,  control,  interaction  level  and 
presentation,  acts  the  educator.  The  user  interface  of  the  current  educational  visualization  is  the  top- 
layer  medium  with  which  the  end-user  (learner)  interacts. 


Meta-Data  within  Algorithmic  Animations 

One  way  in  which  software  evolves  is  to  push  a configuration  out  onto  the  user  and  to  make  the  program  as 
interactive  as  possible.  An  alternative  way  is  to  defer  such  decisions  until  runtime,  or  to  push  configuration  decisions 
out  into  the  data.  Data  itself  become  more  universal  and  reusable  when  it  is  accompanied  by  meta-data.  As  meta- 
data we  define  data  that  describe  other  data,  rather  than  aspects  of  the  application  domain  itself  These  data  may 
become  even  more  general  and  independent  when  they  are  integrated  into  the  code,  which  may  be  transferred  over 
the  Internet.  Using  the  object-oriented  language  Java  to  implement  data  to  be  described,  the  use  of  meta-data  or  code 
descriptions  as  objects  has  great  advantages.  Instead  of  using  meta-data  in  its  general  term  and  traditional  role,  to 
create  universal  descriptions  of  objects  (for  example  in  digital  library  systems)  these  meta-data  are  used  instead  to 
describe  properties.  Runtime  mechanisms  for  accessing,  altering,  adding  and  removing  meta-data  or  properties  at 
runtime  should  be  provided  by  the  programmer  in  order  to  integrate  data  with  meta-data  seemlessly. 

Several  works  have  dealt  with  classifications  of  meta-data,  for  example  [Bohm  et  al.  94]  or  [InfoQuilt].  InfoQuilt 
classified  meta-data  into  two  categories:  content-dependent  and  content-independent  meta-data.  All  of  the  methods 
used  to  classify  meta-data  make  use  of  meta-data  in  its  traditional  sense  of  describing  data  to  facilitate  search 
activities  of  the  data  (in  general  the  documents)  they  describe.  That  is,  the  meta-data  descriptors  are  associated  in  a 
fixed  way  with  the  data  sets  and  as  granular  as  defined  initially.  However,  meta-data  can  also  be  seen  as  parameters 
that  are  passed  to  some  object  of  a system  in  order  to  change  its  behaviour  as  well  as  providing  information,  which 
will  help  search  engines.  In  order  to  change  the  behaviour  (dynamic  property  of  a resource  allowing  it  to  be 
integrated  into  a desirable  context)  of  an  object,  it  should  be  possible  to  change  parameters  on  the  fly.  For  this 
purpose  dynamic  meta-data  are  needed,  extending  the  static  nature  of  meta-data  in  the  traditional  sense. 

• Static  Meta-data 

Static  meta-data  are  meta-data  that  are  used  to  create  universal  and  widely  applicable  descriptions  of  objects, 
which  may  not  change  the  behavior  of  the  described  object.  Examples  are  [lEEE-LOM]  and  [DC]. 

• Dynamic  meta-data 

We  suggest  the  use  of  the  new  term  “dynamic  meta-data”,  describing  the  possibility  to  adapt  the  content  of  an 
object  according  to  the  value  given  in  the  description  and  the  ability  to  change  the  value  of  a parameter  of  a 
given  object  in  order  to  change  its  behaviour.  As  an  example  of  dynamic  meta-data  we  examined  the  Ethernet 
protocol.  In  the  visualization  of  this  protocol,  changing  the  value  of  the  meta-data  field  “PROBLEM”  (being 
represented  in  the  program  as  a property)  from  “Collision”  to  “Shortframe”,  may  change  the  whole  behavior  of 
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the  algorithm  to  be  visualized.  Dynamic  meta-data  can  in  its  turn  be  divided  into  the  following  two  categories. 
We  further  subdivide  into  horizontal  and  vertical  dynamic  meta-data.  Horizontal  meta-data  contain  information 
that  does  not  depend  on  the  content  of  the  algorithm  to  be  visualized.  The  meta-data  fields  defined  within  this 
category  are  common  for  all  visualization  units,  and  thus  can  be  applied  to  all  animations.  Vertical  meta-data 
are  used  when  the  decision  of  changing  the  value  of  a field  is  up  to  the  user.  Parameter  panel  is  an  adequate 
representation  of  these  meta-data. 

itBeanKit 

We  use  the  itBeanKit  (interactive  teaching  Bean  Kit)  to  develop  animation  chains  consisting  of  modular  animations 
visualizing  the  steps  of  an  algorithm.  The  itBeanKit  comprises  the  following  levels: 

• Programmer 

First,  the  programmer  identifies  animation  objects  that  visualize  the  steps  of  an  algorithm.  He  defines  the 
smallest  entities  of  the  algorithm  and  develops  them  as  black-box  software  components,  which  may  be  re-used 
for  the  development  of  other  algorithms.  The  programmer  specifies  animation  actions  at  particular  points  in  the 
algorithm  chain  according  to  a predefined  set  of  meta-data.  The  programmer’s  view  has  been  described  in 
detail  in  [Elsaddik  et  al.  99]. 

• Educator 

As  a middleware  actor,  according  to  the  needs  of  the  end-user,  the  educator  may  convert  an  algorithm 
developed  by  the  programmer  to  a series  of  animation  sequences  by  mapping  algorithm  variables,  specifying 
animation  actions,  and  associating  execution  points  in  the  algorithmic  chain  to  perform  the  desired  animation. 
He  uses  dynamic  meta-data,  which  have  been  pre-defined  by  the  programmer. 

• Learner 

For  the  learner,  a visualization  window  is  divided  into  three  areas:  an  animation  pane,  an  explanation  area,  and 
a parameter  pane.  The  animation  pane  displays  the  resulting  animation  envisioned  by  the  educator.  The 
explanation  area  displays  some  hints  and  information  concerning  the  visualized  algorithm.  The  parameter  pane 
is  divided  into  two  parts:  interactive  utilities  and  a VCR,  allowing  the  learner  to  control  the  progress  of  the 
animation.  The  interactive  utilities  pane  depends  on  the  algorithm  and  on  the  topic,  defined  by  the  programmer 
and  customized  by  the  educator.  The  animation  may  request  intermediate  input  from  the  learner,  allowing  him 
to  control  the  path  of  the  algorithm. 

CONCLUSION  AND  OUTLOOK 

The  itBeanKit  provides  an  environment  in  which  an  educator,  without  conventional  programming  skills,  can  build 
an  interactive  visualization  of  an  algorithm.  We  currently  extend  the  itBeanKit,  particularly  by  increasing  the 
available  choice  of  components.  As  a prototype,  we  implemented  animations,  for  example  “Ethernet”  or  “JPEG”. 
Details  can  be  found  under  the  URL 

http  :/Avww. kom.e-technik.tu-darmstadt.de/proiects/iteach/itbeankit/applets/paradelektion/index.html. 
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Introduction 

The  Lilienthal  project  is  a collaboration  of  5 European  flight  schools  (Lufthansa,  KLM,  EPAG,  lAJM  and 
HORIZON),  and  the  universities  of  Berlin,  Stuttgart  and  Amsterdam.  Within  this  project  a Distance 
Learning  Platform  (DLP)  for  pilot  training  is  developed.  Students  can  study  250+  modules  and  they  can 
communicate  with  their  tutor  and  peer-students  via  e-mail,  chat,  discussion  groups  and  bulletin  boards. 

In  a distance  learning  environment,  the  role  of  the  tutor  is  different  from  the  role  of  a classroom  teacher. 
Traditional  teachers  from  the  flight  schools  now  had  to  learn  to  become  a successful  distance  tutor.  A Tele- 
Tutor  Training  has  been  set-up  to  provide  for  this  need.  This  paper  describes  the  systematic  set-up  and  the 
evaluation  of  this  Tele-Tutor  Training. 

Systematic  set-up  of  Training  Tele-Tutors 

From  the  early  beginnings  of  the  project  it  was  considered  necessary  that  the  classroom  teachers  would  be 
trained  for  their  role  as  distance  tutors.  Since  communication  between  students  and  teacher  is  of  vital 
importance  for  the  learning  process  (see  for  example  Laurillard,  1 993),  the  DLP  was  designed  with  much 
possibilities  for  interaction  (e-mail,  chat,  bulletin  board  and  discussion  groups).  Now  the  teachers  had  to  be 
trained  to  work  with  these  communication  tools.  They  need  both  practical  knowledge  of  how  to  use  the 
tools  and  strategic  knowledge  about  when  to  use  the  communication  tools. 

For  the  training  program  a user-driven  approach  was  chosen.  The  (future)  tutors  were  asked  to  fill  in  (part) 
of  the  training  objectives.  They  were  provided  with  an  electronic  template,  which  they  could  use  to 
describe  the  requirements  and  skills  that  they  would  need  while  teaching  with  the  DLP.  The  basis  of  this 
approach  was  formed  by  the  activity  template  provided  in  Barnard  and  Sandberg  (1994)  which  links 
activity  descriptions  to  technical  support  that  is  required  for  facilitating  or  enabling  the  performance  of 
activities  (Sandberg,  Christoph  and  Emans,  1999).  This  inventory  formed  the  basis  of  a list  of  six  main 
categories  of  requirements  for  distance  tutors  on  the  DLP  (see  table  1). 


Requirement  categories 

1 . Background  in  distance  learning  and  the  DLP 

2.  Communication  requirements 

3.  Information  storage  and  retrieval  requirements 

4.  Adaptation  requirements 

5.  Administrative  and  management  requirements 

6.  Coping  with  difficult  situations 

Table  1:  Six  main  categories  of  requirements  for  distance  tutors. 

Categories  1 and  2 contain  mainly  basic  requirements  and  skills  in  the  field  of  distance  learning,  the  DLP 
itself  and  the  communication  tools.  Categories  3 to  6 contain  the  more  advanced  requirements  and  skills  for 
being  a distance  tutor. 

Description  of  the  training 

The  different  requirements  and  skills  were  trained  in  three  phases.  First  a two-day  seminar  was  organised  to 
teach  the  tutors  the  basic  skills  of  the  communication  tools  and  the  DLP.  Then  a simulation  program  was 
set  up.  During  two  weeks,  the  tutors  had  the  opportunity  to  practice  these  skills  in  a real  distance  education 
simulation.  Each  tutor  was  assigned  a class  of  students.  The  tutor  never  lived  in  the  same  city  as  his/her 
students,  so  no  other  than  distance  communication  was  possible.  Before  this  simulation  the  teachers  were 
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asked  to  formulate  their  own  learning  goals,  besides  getting  practical  experience  with  the  DLP.  (For 
example  “To  be  able  to  interest  students”.) 

The  Tele-Tutor  Training  concluded  with  another  two-day  seminar.  During  this  seminar  the  experiences  of 
the  simulation  program  were  discussed.  Furthermore  the  advanced  skills  and  requirements  like  time- 
management  and  the  strategic  use  of  communication  tools  were  trained. 

Evaluation  Results 

The  three  parts  of  the  Tele-Tutor  Training  were  evaluated  separately,  and  one  over-all  evaluation  was 
performed.  The  evaluations  focused  on  the  actual  skills  the  tutors  gained  during  the  different  parts  of  the 
training,  and  on  the  confidence  and  motivation  of  the  teachers  to  become  distance  tutors. 

After  the  first  Tele-Tutor  Training  seminar,  all  tutors  mastered  the  basic  skills  for  using  the  DLP.  This  was 
a good  result,  since  some  teachers  that  had  never  used  e-mail  before  now  were  able  to  use  the  different 
communication  tools  on  the  DLP  swiftly.  And  more  important,  the  motivation  and  confidence  of  most 
teachers  to  become  distance  tutors  grew  during  the  seminar.  The  simulation  program  resulted  in  82%  of  the 
teachers  achieving  (partly)  their  learning  goal.  Furthermore,  experience  with  the  ‘real  situation*  was  of 
great  value. 

All  subjects  in  the  last  Tele-Tutor  Training  was  reported  to  be  useful..  The  tutors  were  asked  for  their 
opinion  on  the  usefulness  of  the  different  parts  of  the  training  and  of  the  complete  training  as  a whole. 
These  overall  figures  can  be  found  in  figure  1. 


Figure  I:  Mean  answers  to  the  question  on  the  usefulness  of  different  training  parts.  (I  = strongly  disagree,  6 = strongly  agree) 


Conclusion 

We  tried  to  set  up  a Tele-Tutor  Training  that  trained  all  skills  and  requirements  needed  to  become  good 
distance  tutors.  The  requirements  where  gathered  by  the  (future)  tutors  themselves,  giving  them  influence 
in  their  own  learning  process.  The  first  skills  and  requirements  that  were  trained  were  the  basic  skills  for 
handling  distance  communication  and  the  practical  and  technical  knowledge  for  working  with  the  DLP. 

The  training  seminar  for  this  purpose  was  successful,  since  the  confidence  of  the  tutors  grew,  and  they  were 
all  able  to  handle  the  communication  tools  and  DLP  on  a basic  level. 

The  follow  up  training  would  involve  more  advanced  skills.  It  was  felt  that  these  could  only  be  trained  in  a 
real  distance  course  simulation.  It  turns  out  that  this  feeling  was  correct.  Real  problems  only  come  up  in  a 
simulation  as  we  performed  it.  And  in  a simulation  as  we  set  up,  the  tutors  finally  got  an  idea  of  what  it 
would  be  like  being  a distance  tutor.  The  set-up  was  such  that  every  tutor  could  individually  train  on  his 
own  personal  learning  goals,  while  all  the  tutors  were  forced  to  train  the  essential  skills,  and  build  up 
experience  with  the  DLP. 

We  strongly  advise  everyone  to  incorporate  a simulation  in  a distance  tutor  course.  Distance  tutoring  only 
can  be  taught  over  a distance.  During  the  presentation  I would  like  to  illustrate  this  with  some  more 
examples  and  evaluation  results. 
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A simple  definition  of  a portfolio  is  "a  container  of  collected  evidence  with 
a purpose"  (Collins,  1992).  Many  schools  of  education  across  the  coxmtry  ask 
students  who  are  entering  the  teaching  profession  to  create  these  "containers" 
and  provide  evidence  of  their  successful  completion  of  their  preservice 
experience.  Some  of  these  schools  have  offered  students  the  opportumty  to 
create  their  portfolios  through  the  use  of  technology.  Rather  than  creating  a 
binder  with  a collection  of  artifacts,  students  who  use  a technological  approach 
to  create  a multimedia  presentation  of  their  preservice  experience.  It  seems  that 
the  purpose  of  most  portfolios  completed  by  preservice  students  is  to  exhibit 
their  performance  during  course  work  and  student  teaching.  Electronic 
portfolios  seem  to  have  a secondary  purpose;  they  commvmicate  an  expertise 
and  enthusiasm  for  the  use  of  technology  on  the  part  of  the  student.  Our  study 
considers  this  secondary  purpose  from  the  students'  viewpoint. 

The  National  Council  for  Accreditation  of  Teacher  Education  (NCATE) 
issued  a report  called  "Technology  and  the  New  Professional  Teacher:  Preparing 
for  the  21st  Century  Classroom"  in  1997.  The  NCATE  technology  task  force 
suggests  that  perhaps  the  best  way  teacher  education  faculty  can  inspire  future 
teachers  to  use  technology  is  "to  cast  themselves  as  learners  and  to  experiment 
fearlessly  in  the  applications  of  technology,"  making  themselves  "role  models  of 
lifelong  learning"  (p.  9).  The  U.S.  Department  of  Education  states  that 
technology  offers  "numerous  possibilities  for  integrating  assessment  into  the 
daily  life  of  the  classroom"  (1997,  April,  p.  6).  This  study  asked  students  to  reflect 
upon  their  experiences  and  perceptions  as  "fearless  experimenters"  with 
electronic  portfolios. 

This  study  asked  students  from  a small  California  vmiversity  to  reflect 
upon  the  process  of  creating  multimedia  portfolios  during  their  teacher 
education  program.  This  study  considers  student  perceptions  of  electronic 
portfolios  during  four  stages  of  their  teacher  preparation  program;  1)  during  the 
very  first  course  in  education,  2)  at  the  end  of  their  teacher  education  course 
work,  3)  during  student  teaching,  and  4)  after  completing  the  education  program 
and  obtaining  a teaching  position.  Students  in  each  of  these  categories  were 
interviewed  to  gain  insight  into  the  experience  of  collecting  and  preserving 
artifacts  electronically.  From  these  interviews  with  students,  their  perceptions 
evolved  into  themes  that  focused  on  issues  such  as  the  experiences  of  creating 


1328 


the  portfolio,  the  students'  personal  reflections  during  the  portfolio  process,  the 
manner  in  which  students  selected  particular  artifacts,  and  the  usefulness  of  the 
electronic  portfolio  once  it  was  completed.  The  students'  insight  was  significant 
and  may  have  an  impact  upon  other  schools  of  education  as  they  consider 
embarking  upon  the  electronic  portfolio  process. 

Some  of  the  emerging  themes  mentioned  by  the  students  included  1)  time 
constraints  and  issues,  2)  familiarity  with  software  used  to  create  electronic 
portfolios,  3)  the  need  for  guidance  and  support  from  those  with  expertise  in 
creating  these  portfolios,  and  4)  the  opportunity  to  manipulate,  experiment,  and 
create  by  themselves,  rather  than  having  the  "expert"  provide  short  cuts  by 
pushing  the  student  through  the  process  more  quickly  than  their  current 
knowledge  would  allow. 

Students  expressed  that  the  most  pressing  need  for  creating  an  electronic 
portfolio  is  the  issue  of  time.  Within  a hectic  schedule,  students  found  that  when 
asked  to  create  an  electronic  portfolio,  they  were  overwhelmed  by  the  process. 
Some  indicated  that  they  had  no  imderstanding  of  the  time  commitment  needed 
to  create  this  type  of  document.  They  also  expressed  that  the  concern  about  time 
was  very  individualized.  Some  students  needed  time  to  become  familiar  with 
the  software  program  while  others,  who  already  had  experience  with  the 
program,  needed  time  to  push  themselves  beyond  their  current  level  of 
understanding. 

Familiarity  with  the  software  was  a concern  for  some  students  who 
wished  to  create  a portfolio.  They  believed  that  it  was  a great  burden  to  become 
familiar  with  the  software  at  the  same  time  as  they  tried  to  create  their  portfolio. 
Prior  working  knowledge  of  the  software  was  seen  as  crucial  by  these  students. 

A third  concern  expressed  by  the  experimenting  students  was  that  they 
needed  access  to  individuals  who  had  some  expertise  either  with  the  software, 
with  creating  portfolios,  or  with  creating  actual  electronic  portfolios  in  the  past. 
The  students  did  not  care  whether  the  "expert"  was  a faculty  member  or  simply 
another  student.  They  seemed  to  be  very  comfortable  with  learning  from  their 
peers.  Some  students  expressed  that  other  techonology-using  students  were 
often  more  imderstandable  than  technology  faculty  because  their  level  of 
expertise  was  more  comparable  to  that  of  the  novice  student. 

Another  concern  expressed  by  students  focused  on  the  nature  of  the  help 
they  received  from  "experts".  Students  mentioned  that  often,  the  expert 
"grabbed  the  mouse  and  sped  through  the  process"  rather  than  guiding  the 
students  and  allowing  them  to  experience  the  process  for  themselves.  The 
students  felt  that  once  the  "expert"  left  them  to  work  on  their  own,  their 
understanding  was  not  enhanced  by  the  assistance.  The  students  expressed 
frustration  that  they  were  often  left  to  work  their  way  through  the  process  alone. 
Because  the  "expert"  had  by-passed  the  student's  involvement  rather  than 
serving  as  a guide  in  the  process,  their  assistance  was  not  seen  as  helpful. 
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In  conclusion,  if  faculty  members  in  schools  of  education  continue  to 
encourage  students  to  create  an  electronic  portfolio  of  their  college  and/ or 
student  teaching  experience,  they  must  remember  that  the  students  have 
concerns  about  the  process  of  creating  such  a document.  As  stated  previously, 
the  issues  found  in  the  study  focused  on  time,  software  knowledge,  and  the 
helpfulness  of  those  with  expertise  in  the  area  of  technology.  As  students  work 
through  the  process  of  creating  electronic  documents,  faculty  members  need  to 
listen  to  and  compile  their  concerns  in  order  to  better  meet  the  needs  of  future 
students.  Through  a deliberate  conversation  with  student  experimenters,  those 
interested  in  promoting  technology  among  preservice  teachers  may  obtain 
valuable  insights  that  will  assist  students  in  the  creation  of  creative,  organized 
and  frustration-free  projects. 
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Abstract:  This  p^er  describes  the  design,  development,  evaluation  and  revision  of  an  online 
course  over  the  six  semesters  that  it  has  been  offered.  The  course  is  designed  for  K-12  teachers 
and  librarians  interested  in  integrating  Internet  technology  into  their  teaching.  In  this  course, 
students  research  educational  resources  on  the  Internet,  design  their  own  educational  websites, 
and  develop  educational  activities  that  utilize  the  Internet.  Recommendations  for  others  teaching 
similar  courses  are  offered. 


Introduction 

According  to  the  National  Center  for  Education  Statistics  (1999),  in  the  Fall  of  1998,  89%  of  public 
schools  were  connected  to  the  Internet.  But  do  our  K-12  teachers  know  how  to  use  this  new  technology?  The  same 
study  states  that  “teachers  commonly  report  that  they  have  not  received  adequate  preparation  in  the  effective  use  of 
computers  within  the  classroom.*’  Throughout  the  nation,  K-12  teachers  are  feeling  pressure  to  integrate  Internet 
technology  into  their  teaching,  but  even  if  the  hardware  and  technical  support  is  available,  pedagogical  support  is 
often  lacking  or  totally  unavailable.  Now  that  the  classroom  is  wired,  how  can  the  elementary  and  secondary 
educational  process  benefit  from  the  availability  of  the  Internet?  This  online  course  was  developed  for  K-12 
teachers  who  are  not  finding  much  support  locally  in  answering  this  crucial  question. 


Description  of  Course 

My  decision  to  offer  this  course  came  out  of  experiences  I had  in  the  real  world  of  K-12  education.  While 
teaching  a summer  computer  course  at  a local  elementary  school,  I introduced  my  students,  fourth  through  eighth 
graders,  to  the  wealth  of  educational  resources  available  on  the  Internet,  and  I gave  them  the  opportunity  to 
practice  their  research  skills  and  explore  their  creativity  through  creating  their  own  websites.  The  other  teachers  at 
the  school  were  intrigued  by  our  use  of  the  Internet  and  wanted  to  learn  more  about  how  they  could  integrate  this 
new  technology  into  their  teaching  practice.  Because  of  this  experience,  I decided  to  develop  an  online  course  on 
the  topic. 

The  audience  for  this  course  is,  as  I have  stated,  K-12  teachers;  however,  I was  pleasantly  surprised  to 
discover  that  K-12  librarians  were  also  interested  in  the  course.  I have  had  students  from  all  over  the  US  in  the 
course,  as  well  as  from  as  far  away  as  the  Philippines  and  Colombia.  The  course  is  a graduate-level,  one-credit- 
hour  course  offered  for  credit  by  the  School  of  Education  at  a leading  Midwestern  state  university.  The  course 
delivery  is  100%  online  (the  students  and  instructor  never  meet  face-to-face),  though  the  students  still  are  assigned 
a print  textbook  (though  this  may  change  in  the  future).  The  course  was  first  offered  in  the  summer  of  1998,  and 
has  been  offered  every  semester  since  then. 

My  initial  design  for  the  course  was  based  on  my  own  experience  as  a teacher,  both  at  the  K-12  and  the 
postsecondary  level,  and  an  instructional  designer,  having  spent  four  years  working  with  instructors  to  develop 
other  online  courses.  After  reviewing  other  courses  and  textbooks  on  this  integrating  technology  into  the  K-12 
classroom,  I decided  on  the  following  objectives  for  the  course. 

In  this  course,  the  students  will  gain  experience  in: 

• Using  email  for  contacting  outside  experts  and  other  students 
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• Utilizing  an  online,  asynchronous  bulletin  board  system  for  class  discussion 

• Communicating  with  other  students  and  the  instructor  via  a live,  synchronous  chat  program 

• Evaluating  existing  online  resources  for  educational  value 

• Using  HTML  to  create  web  pages 

But  above  and  beyond  the  above  technology-related  goals  stated  above,  the  students  will  also  gain 
experience  in: 

• Designing  lesson  plans  that  integrate  Internet  usage  into  the  curriculum 

• Sharing  their  experiences  with,  and  concerns  about,  using  the  Internet  in  their  classroom  with  fellow 
teachers 

• Dealing  with  issues  of  Internet  safety  and  student  use  of  Internet  resources 

• Reviewing  current  research  on  teaching  with  Internet  technology 

Changes  in  the  Design  of  the  Course 

Over  the  six  semesters  that  the  course  has  been  offered,  I have  made  many  changes  to  the  course  design, 
both  for  technological  and  pedagogical  reasons.  In  terms  of  the  technology-related  changes  that  were  made,  it  is 
obvious  to  even  a casual  observer  that  computer  hardware  and  Internet  software  have  changed  enormously  since 
Summer  1998.  Faster  computer  processors,  increases  in  modem  speed,  more  advanced  browsers,  and  improved 
email  and  synchronous  and  asynchronous  conferencing  programs  have  all  influenced  the  design  of  the  course,  and 
they  will  continue  to  do  so  as  Internet  technology  continues  to  develop. 

Pedagogical  changes  that  occurred  over  the  six  semesters  were  primarily  related  to  my  interest  in 
improving  the  quantity  and  quality  of  our  online  class  discussion.  Feenberg  (1993)  described  the  role  of  the 
instructor  teaching  through  computer-mediated  communication  technology  as  having  three  roles:  that  of  providing 
context,  monitoring,  and  meta-commenting.  Feenberg’ s three  roles  provided  me  with  a useful  framework  for 
thinking  about  my  activities  within  our  online  conferencing  forum.  In  terms  of  my  contextualizing  function,  I 
would  open  discussion,  generally  by  posting  a question  to  respond  to,  or  by  giving  them  a task  to  perform.  Early  on 
in  the  semester,  it  is  important  to  set  the  norms  of  the  conversation  by  making  a few  postings  which  will  serve  as 
models  for  the  students,  in  terms  of  the  language  and  tone  and  level  of  formality  that  our  conversation  should  have. 
I also  provided  context  by  providing  the  schedule  and  an  overall  agenda  for  our  discussions  over  the  semester.  My 
monitoring  functions  included  welcoming  students,  assigning  them  to  teams,  and  nagging  them  when  they  weren’t 
posting  enough.  Finally,  my  metacommenting  included  providing  students  with  summaries  of  what  we  had 
accomplished  in  the  discussion  up  to  that  point,  and  working  with  students  to  resolve  any  problems  in 
communication  or  misunderstandings  of  the  context  of  our  communication  that  occurred. 


Conclusion 

If  our  nation’s  classrooms  are  to  truly  benefit  from  the  sudden  infusion  of  Internet  technology  that  the 
latter  half  of  this  decade  has  witnessed,  teachers  must  be  given  both  technological  and  pedagogical  support  as  they 
begin  to  integrate  this  technology  into  their  teaching.  It  is  my  belief  that  online  courses  such  as  the  one  described 
in  this  paper  are  an  important  way  to  address  this  need,  especially  for  teachers  who  find  it  difficult  to  attend  on- 
campus  courses. 
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Abstract:  The  education  of  preservice  K-12  teachers  in  using  communications  technologies  has  not  kept  pace  with  the 
technologies  themselves.  Consequently,  when  new  teachers  enter  the  classroom,  they  routinely  limit  their  use  of 
technologies  to  add-ons  and  skill/drill  exercises  rather  than  integrating  those  technologies  into  their  lessons  to  create 
optimal  learning  environments  for  students.  A consortium  of  Central  New  York  educators,  led  by  the  State  University 
of  New  York  College  at  Oswego,  along  with  both  rural  and  urban  K-12  school  districts,  will  remedy  this  situation  by 
implementing  a program  to  provide  preservice  teachers,  teacher  educators,  and  K-12  teachers  with  improved  access  to 
and  support  with  technology.  The  plan  also  provides  preservice  teachers  the  opportunity  to  model  teachers  who  are 
experienced  in  infusing  technology  into  the  New  York  State  Standards-based  curriculum 

NEED  FOR  PROJECT 

Educators  have  seen  a recent  proliferation  of  initiatives  aimed  at  infusing  technology  into  Teacher 
Education  Programs.  Current  conditions  contributing  to  this  effort  are: 

1 . Recent  standards  developed  by  the  International  Society  for  Technology  in  Education  (ISTE) 
and  adopted  by  the  National  Council  for  the  Accreditation  of  Teacher  Education  (NCATE). 

2.  The  NCATE  Task  Force  on  Technology  in  Teacher  Education’s  recommendation  that  schools 
of  education  have  a technology  infusion  plan  (NCATE,  1997), 

3.  The  subsequent  National  Education  Technology  Standards  Project  (Thomas  & Knezek,  1998), 

4.  The  availability  of  federal  funds  from  several  sources,  as  well  as, 

5.  The  demand  of  the  K-12  schools  for  technologically  sophisticated  teachers. 

With  all  of  these  initiatives,  why  haven’t  teacher  education  programs  kept  up  with  these  crucial 
technology  requirements?  We  all  know  that  generally  change  is  a slow  process.  It  is  time  that  helps 
change  take  place.  Already  over-scheduled  educators  with  full  class  loads  are  reluctant  to  spend  time  on 
initiatives  that  take  time  away  from  tried  and  true  programs  and  curricula.  Of  course,  times  change  and 
educators  must  change  from  what  worked  well  or  adequately  in  the  50’s  and  60’ s,  and  maybe  up  until 
recent  years,  to  what  our  children  need  to  function  optimally  in  the  global  marketplace  of  today.  The  major 
effect  of  the  multimedia  emplacement  may  be  in  adapting  students  for  the  technological  world  they  are 
soon  to  enter  as  productive  society  members.  Additionally,  the  traditional  methodology  is  no  longer 
suitable  for  preparing  preservice  teachers  for  dealing  with  the  complex  demographics  of  the  schools  (Allen, 
1997);  they  can  easily  communicate  with  people  around  the  world,  and  the  schools  no  longer  have  to  rely 
on  outdated  texts  to  deliver  this  information  to  them. 

Clearly  our  K-16  schools  have  not  kept  up-to-date  with  what  is  happening  with  regard  to 
technological  advances.  The  millennium  is  upon  us,  and  our  teachers  are  not  prepared  to  teach  the  children 
in  a manner  that  will  help  to  insure  their  future  success.  Although  e-mail  is  a word  that  is  on  the  lips  of 
most  people  today,  whether  or  not  they  use  it  themselves,  some  educators  do  not  use  it,  even  when  they 
have  access  to  it.  Access  to  new  technologies  will  not  by  itself  solve  this  problem  because  it  is  two-fold: 
educators  need  to  have  access,  but  they  also  need  confidence  and  knowledge  about  the  pedagogical 
potential  of  the  new  technologies.  Preservice  teachers  need  to  experience  ways  in  which  new  technologies 
can  be  integrated  seamlessly  into  lessons  in  order  to  make  active  and  pedagogical ly  sound  use  of 
technology. 

A key  element  in  promoting  a natural  integration  of  technology  into  lessons  is  an  opportunity  to 
model  excellent  classroom  use  of  technology.  One  can  lecture  about  how  a unit  plan  is  created,  BUT  it  is 
not  until  a preservice  teacher  actually  creates  a unit  plan  that  real  learning  and  understanding  about  unit 
planning  takes  place.  The  same  is  true  for  teaching  with  technology.  Preservice  teachers  can  know  about 
technology,  word  processing,  e-mail,  and  the  World  Wide  Web;  however,  one  cannot  conclude  that  they 
will  be  able  to  utilize  technology  appropriately  in  the  K-12  classroom  to  enhance  the  teaching/leaming 
process.  In  fact,  the  opposite  is  the  case;  preservice  teachers  who  are  technologically  literate  have  indicated 
that  they  had  false  notions  about  what  you  would  do  with  technology  in  a K-12  classroom  before  they  were 
exposed  to  methods  courses  where  their  instructors  served  as  models  for  technology  infusion;  had  field 
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experiences  in  classrooms  with  master  teachers  who  integrated  technology  into  their  curriculum  and  used  it 
as  a tool,  instead  of  an  add-on  or  skill/drill;  and  watched  excellent  (model)  teachers  via  videoconferencing 
(Vannatta,  1999).  Preservice  teachers  need  technology  to  get  a glimpse  of  real-world  contexts  and  to 
connect  to  practitioners  in  the  real  world  (Jonassen,  1995). 

Current  practice,  however,  provides  too  few  opportunities  for  preservice  students  to  model  after 
others.  A consortium  of  schools  including  the  State  University  of  New  York  at  Oswego  and  partner  K-12 
schools  will  continue  its  efforts  to  remedy  this  problem  by  building  on  lessons  learned  in  an  earlier  effort  to 
infuse  new  technologies  into  teacher  education.  Specifically,  through  the  establishment  of  a plan  that  is 
inclusive  of  more  teacher  educators  than  are  presently  included  through  our  Goals  2000  Grant,  we  will 
provide  technology,  technological  support,  and  mobile  teacher  stations  that  include  the  ability  to  do 
desktop-to-desktop  videoconferences  with  partner  schools.  Overall,  we  will  improve  preservice  teachers 
access  to  observing  and  modeling  ’’real  classroom"  uses  of  technology. 

Gaps  and  Weaknesses  in  Current  Practice— Project  2000 

The  consortium’s  participation  in  a project  funded  by  a Goals  2000  Grant  has  allowed  us  to  see 
first  hand  the  weaknesses  in  current  practice  and  the  ways  they  can  be  remedied.  The  Goals  2000 
technology  infusion  project  began  in  1997  with  the  purpose  of  increasing  preservice  teachers’  proficiency  in 
classrooms  through  videoconferencing  and  technology  infusion.  The  effort  was  supported  theoretically  by 
a constructivist  view  of  learning,  which  identifies  three  essential  elements  as  contributors  to  preservice 
teachers  learning  how  to  teach:  reflection,  active  student  involvement,  and  development  of  a community  of 
learners  (White,  1 996).  All  of  these  elements  can  be  enhanced  through  the  use  of  computer-mediated 
instruction  (Ford,  et  al,  1999). 

The  Goals  2000  project  involved  partnering  college  faculty  from  the  School  of  Arts  and  Sciences 
and  the  School  of  Education  with  K-12  teachers  to  infuse  technology  in  preservice  education.  The  project 
supported  a continued  revision  of  education  courses,  implementation  of  an  integrated  MST  course  at  one  of 
the  partner  high  schools,  ongoing  training  and  mentoring  of  faculty  and  inservice  teachers,  and  infusion  of 
technologies  into  science  foundations,  general  methods,  special  education  methods,  and  reading  methods 
courses.  A key  component  of  these  activities  was  the  targeting  of  methods  courses  offered  each  semester 
for  enhanced  infusion  of  technology.  Enhancements  were  chosen  or  developed  in  partnership  with  Arts 
and  Sciences  faculty  and  K-12  partners  during  a summer  workshop  and  subsequent  team  meetings.  Among 
other  activities  in  these  methods  courses,  there  was  also  a concerted  effort  at  modeling  integrated  classroom 
uses  of  technology.  Methods  classes  used  the  School  of  Education’s  computer  lab  and  the  Goals  2000 
Project’s  "technology"  classroom.  There,  instead  of  using  computer-  aided  instruction  as  add-ons  to 
lessons,  or  sending  preservice  students  to  the  computer  for  additional  research,  a methods  faculty  member 
would  link  writing  assignments  to  her  web  page,  provide  a link  to  a group  unit  assignment  and  then  to 
hyperlinks  on  related  topics.  Research  assignments  seamlessly  lead  to  the  use  of  web  technologies  to 
enhance  research  and  encourage  group  interaction.  In  addition,  students  in  methods  courses  were 
periodically  linked  via  videoconferencing  to  K-12  project  partners  in  order  to  observe  the  technology  at 
work  in  real  classrooms. 

These  activities  provided  students  with  experiences  that  would  increase  their  confidence  with  the 
technology,  permit  them  to  model  imaginative  uses,  and  forestall  the  tendency  to  use  the  technologies 
merely  as  add-ons.  Moreover,  and  importantly,  the  videoconferencing  provided  them  with  an  early  and 
real-time  experience  in  the  potential  use  of  technologies  in  teaching  specific  subject  matters. 

Formal  evaluation  after  the  second  year  of  the  Goals  2000  Project,  as  well  as  formative  focus 
group  feedback,  testified  to  the  success  of  the  project.  Pre-  and  post-tests  indicated  that  the  technology 
infusions  did,  in  fact,  significantly  increase  preservice  teachers'  proficiency  in  integrating  technologies. 
After  preservice  students  completed  their  practicum  experiences,  supervising  teachers  indicated  their 
significant  approval  of  the  students;  they  were  seen  as  “heads  and  shoulders”  above  earlier  practicum 
students  in  their  ability  to  infuse  technology  into  the  New  York  State  Standards-based  curriculum. 

So  what  is  the  problem?  If  we  know  what  our  preservice  teachers  need  to  develop  into  the 
teachers  that  we  need  for  tomorrow,  why  haven’t  we  restructured  our  teacher  preparation  programs  to 
reflect  those  necessary  changes?  The  answer  to  that  question  is  not  a simple  one;  it  is  complex  and  multi- 
faceted. It  has  to  do  with  change,  human  nature,  knowledge,  money,  and  much  more.  It  is  indeed  a 
dilemma.  How  does  one  accomplish  teaching  a new  group  of  teachers  to  be  technologically  prepared  for  a 
technological  environment  if  they  are  not  prepared  themselves?  One  can  compare  it  to  trying  to  teach 
someone  to  fly  an  airplane,  when  they  are  not  truly  comfortable  with  flying  an  airplane  themselves!  A 
whole  generation  of  teachers  needs  to  learn  new  tools,  new  approaches,  and  new  skills.  This  will  be  a 


challenge,  not  just  because  of  the  slow  moving  wheels  of  the  change  process,  but  because  of  financial 
cutbacks,  low  teacher  morale,  increased  workloads,  and  reduced  retraining  budgets  (Tapscott,  1999). 


References 


Allen,  K.W.  (1997).  Enhancing  multicultural  education  with  technology.  Technology  & Teacher 
Education  Annual,  1997,  Charlottesville,  VA:  Association  for  the  Advancement  of  Computing  in 
Education. 

Ford,  M.J.,  Dobyns,  S.M.,  & Poe,  V.  (1997).  Using  the  Internet  and  children’s  literature  to 
understand  cultures.  Technology  & Teacher  Education  Annual,  1997.  Charlottesville,  VA:  Association  for 
the  Advancement  of  Computing  in  Education. 

Jonassen,  D.H.  (1995).  Supporting  Communities  of  learners  with  technology:  A vision  for 
integrating  technology  with  learning  in  schools.  Education  Technology/Julv-August  1995,  60-63. 

National  Council  for  the  Accreditation  of  Teacher  Education  (1997).  Technology  and  the  new 
professional  teacher.  Washington.  DC:  National  Council  for  Accreditation  of  Teacher  Education. 

Roberts,  G.  (1999).  Lessons  from  Sisyphus  in  a technological  Age.  Educational 
Leadership/Februarv  1999.  75-77. 

Tapscott,  D.  (1999).  Educating  the  net  generation.  Educational  Leadership/February  1999,  7-11. 

Thomas,  L.G.  & Knezek,  D.G.  (1998).  National  educational  technology  standards.  Educational 
Leadership,  56  (5),  27. 

Vannatta,  R.  (1999)  Goals  2000  technology  infusion  project:  Evaluation  of  year  2.  Evaluation 
report  available  from  Vannatta,  Bowling  Green,  OH:  Bowling  Green  State  University. 

Wetzel,  K.  (1993).  Teacher  educators’  uses  of  computers  in  teaching.  Journal  of  Technology  and 
Teacher  Education,  1 (4),  22-27. 

White,  C.  (1996).  Relevant  social  studies  education:  Integrating  technology  and  constructivism. 
Journal  of  Technology  and  Teacher  Education,  4(1),  69-76. 


User  Studies:  Evaluating  the  Use  of  EPSS  Tools  for  Self-Management  by  Children 
Overview  of  the  Presentation 

The  presenters  will  report  the  results  of  a research  project  designed  to  evaluate  the 
effectiveness  of  new  EPSS  software  created  for  children  to  provide  them  tools  for 
managing  their  personal  behaviors  in  classroom  settings.  The  presenters  will  demonstrate 
the  EPSS  software  and  an  online  resource  infobase  that  supports  teachers'  use  of  the 
software  tools.  Methodology  for  the  user  studies  will  be  explained.  These  include  the 
use  of  a behavior  rating  scale  by  teachers,  perceptions  of  locus  of  control  by  children, 
software  audit  trail  logs,  an  examination  of  artifacts  created  by  children,  and  results  of 
interviews  with  teachers.  This  research  study  is  underway  during  the  1999-2000  school 
year  and  results  will  be  available  for  presentation  at  Ed-Media. 

Theoretical  Base  for  the  Use  of  EPSS  with  Children 

Computer-based  training  and  support  mechanisms  are  an  innovative  approach  for  helping 
children  gain  control  over  personal  behaviors.  Although  there  are  limited  data  on  the  use 
of  computer-based  instruction  to  support  behavior  change  in  children  to  date,  research 
results  are  promising.  Fitzgerald  and  Werner  (1996)  reported  success  with  a 
computerized  verbal  mediation  essay  as  a cognitive  retraining  procedure  to  assist  a 
student  with  significant  behavioral  disorders  in  changing  his  behavior;  the  computerized 
essay  provided  consistent  practice  and  focused  the  child’s  attention  and  thoughts  on 
behavioral  choices  and  consequences.  In  another  case  study,  the  same  researchers 
reported  a procedure  in  which  software  templates  were  developed  for  a student  to  create 
self-monitoring  materials.  This  study  was  the  pre-cursor  to  this  work  to  develop  and 
investigate  the  use  of  EPSS  tools  with  children  and  their  teachers  as  part  of  a federally- 
funded  initiative  focusing  on  children  with  classroom  behavior  problems  (Fitzgerald  & 
Semrau,  1998-2001). 

The  Two  Components  of  the  EPSS  System 
1.  KidTools 

KidTools  is  a software  program  designed  as  easy-to-use  templates  that  can  be 
individualized  by  children  and/or  their  teachers.  The  software  is  kid-friendly  with 
colorful  graphics,  text-with-audio  directions,  and  simple  formats.  KidTools  has 
undergone  usability  testing  with  children  ages  7-10  to  ensure  literacy  is  appropriate  for 
young  children  (Fitzgerald  & Lynch,  1999).  The  text  and  audio  in  KidTools  utilize 
natural  language  of  children,  graphic  characters  serve  as  “guides”  to  the  different  tools, 
and  audio  directions  are  provided  to  supplement  the  simplified  text  instructions. 

KidTools  is  designed  for  children  to  use  independently  on  classroom  computers.  To  use 
the  templates,  the  child  simply  clicks  on  “hot  words”  on  the  template  form  to  enter 
personalized  content  and  then  prints  the  completed  form  for  use  in  the  classroom.  The 
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program  automatically  enters  the  child’s  name,  date,  and  establishes  an  audit  trail  for 
recordkeeping  purposes  and  user  studies. 

2.  Teacher  Resource  Infobase 

The  online  resource  infobase  provides  adults  with  information  to  assist  children  in  using 
KidTools.  The  tools  in  the  program  are  grouped  into  six  types  of  self-management 
procedures:  point  cards,  countoons,  self-monitoring  cards,  STAR  (Stop-Think-Act- 
Results)  cards,  make-a-plan  procedures,  and  contracting.  The  infobase  includes 
descriptive  information  for  each  type  of  procedure,  steps  for  implementation,  a variety  of 
examples  for  students,  troubleshooting  tips,  and  recommended  resources.  It  is  expected 
that  teachers  introduce  children  to  effective  use  of  KidTools  and  troubleshoot  the 
procedures  as  they  are  implemented  in  classrooms.  This  information  resource  component 
has  been  developed  in  conjunction  with  KidTools  for  teachers  and  is  available  on 
CDROM  as  well  as  online  for  easy  access. 

Participants 

The  research  project  is  underway  in  three  elementary  schools  in  two  states.  The 
participants  are  children,  ages  7-10,  whose  teachers  nominate  them  for  inclusion. 

Children  selected  to  participate  are  those  either  (a)  showing  behavioral  difficulties  in 
classroom  settings,  or  (b)  having  personal  habits  that  can  be  improved  through  a self- 
management process.  Participants  can  be  attending  general  and/or  special  education 
programs  and  there  are  no  disability  requirements.  To  be  eligible,  each  child  and  his/her 
parent  must  voluntarily  agree  to  participate  and  the  teacher  must  attend  an  orientation 
session. 

Methodology 

Each  child  utilizes  KidTools  under  the  guidance  of  the  classroom  teacher  for  a four- 
month  period.  The  process  of  identifying  behaviors  and  selecting  the  tools  for  self- 
management is  negotiated  between  the  child  and  the  teacher.  Implementation  decisions 
also  include  length  of  time  for  use  of  each  tool  and  positive  outcomes  for  success  with  the 
tools.  Parents  may  be  involved  in  daily  communication  regarding  use  of  the  tools  at 
school.  It  is  anticipated  that  tools  will  be  used  for  a wide  variety  of  behaviors  and  that 
tool  usage  will  change  as  the  child  demonstrates  progress.  These  kinds  of  unique 
applications  that  will  be  examined  through  the  audit  trail  data  and  tool  artifacts. 

Data  Collection 

1 . Gruen,  Korte,  Stephens  Internal-External  Scale  for  Children:  Each  child  completes  a 
simple  questioimaire  about  his/her  perceptions  of  internal  versus  external  behavior 
control  in  school. 

2.  Comprehensive  Behavior  Rating  Scale  for  Children:  The  primary  teacher  who 
nominates  the  child  completes  a normed,  objective  rating  scale  on  45  behavior  items. 


3.  The  audit  trails  stored  on  the  computer  disks  will  be  analyzed  for  tool  selection, 
content  entered  into  the  tools,  and  frequency  of  use  by  each  participant. 

4.  Artifacts  created  by  the  participants  and  saved  by  the  teachers  will  be  collected  and 
analyzed  for  indicators  of  success  and  actual  use  by  the  children. 

5.  Teachers  will  be  interviewed  at  the  close  of  the  project  regarding  use  of  the  tools, 
perceived  impact  on  children’s  behavior,  usability  of  the  resource  infobase, 
satisfaction  with  the  software  tools,  and  recommendations  for  improvement. 

Data  Analysis  and  Interpretation 

Results  from  the  locus  of  control  questionnaire,  behavior  rating  scale,  and  usage  data  will 
be  entered  into  a computer  file  for  statistical  analysis.  These  data  will  be  linked  to 
qualitative  information  derived  from  audit  trail  data,  artifacts,  and  teacher  interviews. 

The  interpretation  will  include:  (a)  an  examination  of  the  behavior  characteristics  of  the 
participants  (Comprehensive  Behavior  Rating  Scale  for  Children),  (b)  an  examination  of 
the  beliefs  children  have  about  behavior  control  in  the  classroom  (Gruen,  Korte,  Stephens 
Internal-External  Scale  for  Children),  (c)  descriptive  data  on  frequency  of  use  and  content 
entered  into  the  tools  by  children  grouped  by  age  and  locus  of  control,  and  (d)  qualitative 
analysis  of  artifacts  and  teacher  perceptions. 

The  results  from  these  user  studies  will  allow  developers  to  improve  future  design  and 
development  of  EPSS  software  for  children.  Interview  responses  will  help  define  the 
need  and  use  of  supports  for  teachers  who  implement  EPSS  procedures  with  young 
children.  Finally,  the  approaches  utilized  in  these  user  studies  may  provide  guidance  to 
other  researchers  who  conduct  usability  tests  with  young  children. 
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Abstract:  Thomas  Jefferson  University,  an  urban  health  sciences  university  located  in  downtown 
Philadelphia,  Pennsylvania,  has  produced  and  evaluated  two  very  different  distance  learning  format 
courses.  Each  course  was  a partnership  between  TJU’s  Academic  Information  Services  and 
Research  (AISR)  division  and  the  College  of  Health  Professions’  Nursing  Department.  The 
Nursing  Department  was  concerned  about  local  competition  for  students  and  wanted  to  attract 
potential  program  candidates  for  its  graduate  degree  program  unable,  or  unwilling,  to  avail 
themselves  of  the  traditional  campus-based  course  schedule.  This  short  paper  describes  the 
development  team  approach  used  to  produce  each  course  and  presents  descriptive  and  anecdotal 
feedback  from  students. 


Development  Team 

The  literature  in  software  development  and  distance  learning  indicates  considerable  benefits  in  using  a team 
approach.  As  Ramesh  Rao,  associate  dean  of  the  Graduate  Business  School  at  the  University  of  Texas- Austin  states, 
"From  the  standpoint  of  development,  creating  courses  in  a new  medium  such  as  the  Internet  can  be  very  exciting. 
Very  often,  however,  the  faculty  member,  well-versed  in  his  or  her  own  subject  area,  does  not  possess  sufficient 
knowledge  in  the  areas  of  technology  or  instructional  design.”  (Jeff  Cobb,  1999)  AISR  has  effectively  used 
development  teams  in  designing  and  producing  high  quality  instructional  material.  The  team  contributes  qualities 
no  one  person  can  master.  Development  team  roles  are  outlined  below. 

Subject  Matter  Expert  (SME)  is  the  primary  content  person  and  responsible  for  writing,  or  identifying  an  existing 
source  of  course  content,  determining  appropriate  exercises,  and  reviewing  all  material  for  accuracy. 

Course  Editor  is  responsible  for  keeping  the  group  organized  and  on  schedule,  as  well  as  reviewing  material  for 
consistency.  The  role  of  editor  is  similar  to  that  in  the  print  industry  and  a project  manager. 

Instructional  Designer  works  closely  with  the  programmer  and  artist  to  develop  the  user  interface  for  the  course, 
identify  appropriate  learning  technology  tools  for  the  course,  and  converts  the  content  supplied  by  course  faculty 
into  storyboards.  The  instructional  designer  analyzes  the  educational  goals  and  determines  the  best  activities  to 
allow  learners  to  reach  those  goals. 

Programmer  uses  the  supplied  storyboards,  text,  and  graphics  to  code  the  online  course  using  a combination  of 
HTML,  Flash  Shockwave  files,  JavaScript,  and  PL/SQL  commands.  At  Jefferson,  we  use  a number  of  programming 
tools  to  facilitate  production  and  improve  the  level  of  interactivity  with  the  content. 

Graphic  Artist  works  closely  with  the  SME  and  instructional  designer  to  produce  graphics  and  animations  that 
engage  the  learner  with  the  content. 

Database  Administrator  is  responsible  for  developing  the  database  tables  and  access  procedures,  and  for  creating 
routines  to  record  the  students’  progress  through  the  material  and  responses  to  questions.  At  this  time  Jefferson  is 
using  Oracle  as  its  database  allowing  considerable  customization  to  meet  both  instructional  and  faculty  needs. 
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Course  Design  and  Production 


Epidemiology^  a graduate  course  for  advance  practice  nurses,  was  selected  as  the  first  course  to  produce  in  this 
format.  A web-based  version  of  the  course  was  developed  over  a nine-month  schedule.  The  distance  learning 
course  covered  the  identical  content  as  the  campus-based  course,  but  substituted  alternative  activities  for  those  in  the 
classroom.  Online  course  materials  included  graphics,  animation,  over  1200  information  "pages,"  a search  utility, 
glossary,  self-assessment  practice  quizzes,  exercises,  discussion  boards,  and  case  studies. 

The  second  course.  Health  Policy,  Ethical  and  Legal  Dimensions  of  Care,  was  developed  soon  afterwards  using  a 
very  different,  much  less  labor  intensive,  approach  to  distance  learning.  Whereas  the  Epidemiology  course 
produced  a completely  self-contained  learning  environment  and  used  standard  examination  techniques.  Health 
Policy  used  an  existing,  print-based  textbook  and  online  group  work  for  evaluation. 

These  two  courses  (in  either  campus-based  or  distance-learning  format)  are  required  of  all  matriculating  graduate 
nursing  students.  However,  non-matriculating  students  may  take  these  courses  as  well,  allowing  interested 
candidates  the  option  to  experience  the  curriculum  before  committing  to  a degree  program.  Current  plans  include 
developing  online  versions  of  all  core  courses  in  the  graduate  nursing  program,  with  the  goal  of  reaching  a wider 
pool  of  students  from  which  to  recruit. 


Course  Results  and  Comparison  to  Campus  Group 

Quantitatively  there  was  no  significant  difference  between  the  distance  learning  and  campus-based  groups.  The  no- 
significant  difference  result  is  not  unusual  in  alternative  education  programs.  Years  of  research  studies  designed  to 
compare  traditional  instruction  and  different  computer-based  learning  models  have  suggested  that  the  actual  method 
of  delivery  may  matter  very  little  in  the  final  outcome  of  student  learning.  Perhaps  the  most  reliable  variable  for 
predicting  student  results  is  their  involvement  with  the  content.  Time  involved  with  the  content  or  time  interacting 
with  the  content  seems  most  related  to  student  success.  On  a subjective  measure,  students  in  the  online 
Epidemiology  course  reported  greater  satisfaction  with  the  course  overall,  and  rated  the  course  as  more  relevant  to 
nursing  practice  than  did  the  campus-based  group. 


Summary 

Faculty  and  most  students  expressed  their  satisfaction  with  the  distance  learning  format.  It  should  be  noted  that 
distance  learning  is  not  suitable  for  every  student.  Distance  learning  requires  a largely  self-directed  learner,  with  a 
fair  amount  of  self-discipline  to  regularly  interact  with  the  content.  Some  students  enrolled  in  the  distance  learning 
format  course  expecting  it  to  be  easier  or  require  less  work  than  the  traditional  course.  They  were  surprised  by  the 
amount  of  effort  required.  As  universities  learn  more  about  the  needs  of  distance  education  students,  new  tools  will 
be  developed  to  meet  these  needs.  A larger  instructional  design  toolbox  will  allow  developers  to  include  an 
increasing  variety  of  interactive  exercises  and  make  learning  more  meaningful  for  all  students. 

Our  foray  into  distance  learning  was  very  successful.  We  are  pleased  to  report  that  not  only  did  the  majority  of 
students  say  they  would  take  additional  distance  learning  courses,  several  have.  Since  the  first  course  was  offered  in 
1998  two  additional  courses  have  been  released  and  course  enrollment  has  increased  with  each  semester. 
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Abstract;  There  is  an  increasing  role  for  3D  imaging  and  animation  in  clinical  medicine 
and  in  scientific  and  educational  visualization  in  recent  years.  Indeed,  it  is  evident  that  computer 
graphics  are  poised  to  replace  hand-painted  textbook  illustrations  to  become  the  main  vehicle 
through  which  complex  scientific  phenomena  are  visually  expressed.  This  paper  describe  steps 
taken  to  help  visualize  and  popularize  the  research  done  by  the  Cancer  Biology  Research  Group  of 
the  University  of  Calgary  on  the  reovirus  (Dr.  Patrick  Lee,  Dr.  P.  Forsyth,  Dr.  M.  Coffey  and  Dr.  P. 

Strong). 

Introduction 

Generally  speaking  our  survey  has  found  that  medical  animation  are  used  for;  1)  demonstrating  and  popularizing 
surgical  procedures  and  biological  processes;  2)  simulating  surgical  procedures;  3)  educating,  students,  patients  and 
healthcare  professionals;  4)  visualizing  macroscopic  and  microscopic  anatomical  structures;  5)  introducing  new 
products  to  physicians  or  consumers.  In  the  field  of  medical  animation,  popularization  can  be  described  as  the 
breakdown  of  the  seemingly  impenetrable  wall  that  sometimes  stands  between  hard  medicine,  student  and  the 
public.  The  focus  of  this  work  is  to  render  any  discovery  in  a visually  arresting  manner  without  sacrificing  the 
clarity  of  message.  The  goal  is  to  represent  complex  phenomena  as  understandable  and  plausible  animations. 

The  reovirus  (reo  is  an  acronym  for  respiratory  enteric  orphan)  is  a naturally  occurring  virus  that  is  believed  to  cause 
mild  infections  of  the  upper  respiratory  and  gastrointestinal  tract  of  humans  (2).  Preliminary  research  shows  that  the 
reovirus  is  a potent  killer  of  cancer  cells  and  can  selectively  kills  cells  with  an  hyperactivated  Ras  biochemical 
pathway  (2).  The  Ras  pathway  normally  regulates  cell  growth  but  becomes  highly  active  in  about  75%  of  tumors 
(including  breast,  pancreatic  and  brain  cancers).  Infection  of  a single  cancer  cell  by  a single  reovirus  particle  can 
generate  a thousand  progeny  reovirus  particles,  which  in  turn  infect  and  destroy  neighboring  cancer  cells.  This  cycle 
continues  until  all  cancer  cells  are  eliminated.  Throughout  this  process,  normal  cells  are  spared.  Tested  on  mice 
only  at  this  point,  this  approach  could  have  a tremendous  implication  for  the  treatment  of  cancer  in  humans. 

The  Method 

An  animation  was  developed  to  illustrate  the  research  work  and  educate  the  student  about  how  reovirus  kills  cancer 
cells.  Our  animation  team  chose  SOFTIMAGE  | 3D.  Softimage  is  well  designed  for  scientific  visualization  where 
the  goal  is  not  solely  to  represent  scientific  data,  but  to  also  tell  a story.  To  represent  these  complex  events  (the 
processes  of  initiation,  infection  and  replication),  we  used  the  following  steps  to  achieve  our  goals: 

a)  Storyboading 

Several  meetings  were  held  with  the  research  team  to  discuss  the  purpose  of  the  animation,  the  intended  audience 
and  the  level  of  detail  required.  Several  two-dimensional  illustrations  were  provided  for  the  animation  team  to 
work  from  The  sketches  and  eventual  storyboards  for  the  animation  were  planned  using  these  reference  materials 
and  related  literature  in  molecular  biology.  One  of  the  first  facts  that  became  evident  in  the  planning  period  was 
the  varied  levels  of  scale  that  would  have  to  be  represented.  A way  had  to  be  found  that  would  enable  the  story  to 
move  from  a macro  scale  (the  mouse),  to  a cellular  scale,  and  finally  to  a biochemical  scale.  These  scale  jumps  had 
to  be  seamless  enough  to  prevent  the  viewer  from  becoming  disoriented  about  what  was  being  shown.  One  way 
this  was  achieved,  for  example,  was  zooming  into  the  cancerous  tissue  on  the  mouse  and  then  fading  into  a scene 
showing  the  cellular  surface.  In  addition,  aesthetic  choices  about  how  the  environments  would  look  had  to  be 
made.  It  was  decided  that  monochromatic  tones  would  be  used  to  de-emphasize  often  complex  background  scenes. 
Elements  that  needed  to  be  highlighted  against  this  background  would  be  given  different  colours  to  bring  them  to 
the  viewer’s  attention. 

b)  Model  Building 
1)  The  Mouse 

To  design  the  mouse  we  used  photographs  of  mice  taken  from  a top,  side  and  front  views.  The  pictures  were 
scanned  and  converted  into  tiff  format.  The  scans  were  then  imported  into  SOFTIMAGE  | 3D.  Using  both  the 
autotrace  and  the  rotoscope  option,  the  mouse  was  designed  with  nurbs  (Non-Uniform  Rational  B-Splines).  The 
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tumor  location  on  the  hind  flank  of  the  mouse  was  based  on  actual  experimental  protocols  used  by  the  research 
team.  The  final  stage  involved  placing  and  positioning  fur  as  a two-dimensional  (2D)  texture  map. 

2)  Cell  Surfaces  and  Proteins 

The  plasma  membrane  of  each  cell  was  modeled  from  a primitive  two-dimensional  grid.  An  earth  tone  was  given  to 
the  outer  surface  of  the  cell,  while  the  inside  surface  was  given  a water  tone.  Various  surface  proteins  and  receptors 
were  created  from  other  primitives  and  revolved  two-dimensional  profiles,  and  attached  to  the  plasma  membrane 
model  using  constraints.  To  conserve  disk  space,  instances  of  these  proteins  and  receptors  were  duplicated  over  the 
surface  of  the  plasma  membrane  model.  This  made  the  editting  of  hundreds  of  these  shapes  simple  and  screen 
refresh  rates  much  faster.  A cyclic  wave  deformation  was  assigned  to  the  plasma  membrane  to  help  create  a 
dynamic,  “living”  environment  . Special  attention  was  given  to  the  proteins  and  receptors  of  the  Ras  biochemical 
pathway,  in  lieu  of  its  importance  to  the  story  being  told.  These  models  were  built  in  consideration  of  their  affinity 
to  the  plasma  membrane  and  to  each  other.  Plausible  structural  details  were  created  on  each  model  to  illustrate  its 
binding  specificity  to  the  next  and  previous  proteins  in  the  chain.  To  emphasize  their  importance,  the  proteins  of  the 
Ras  pathway  were  given  contrasting  textures  to  the  blue  and  purple-shaded  elements  comprising  the  background  . 

3)  The  Reovirus 

A detailed  model  of  the  reovirus  was  created  based  on  the  structure  of  the  actual  virus  as  determined  from  electron 
and  cryoelectron  micrographs,  and  image  reconstructions  (4,6).  The  ultrastructure  of  the  reovirus  consists  of  a non- 
enveloped  double  shell  of  inner  and  outer  capsids  which  enclose  several  proteins  which  aid  in  its  replication,  as  well 
as  a genome  of  double-stranded  RNA  (3).  The  components  of  the  capsids  were  modeled  with  extruded  polygons 
and  circles,  the  proteins  with  metaballs,  and  the  RNA  stands  with  simple  paired  spheres.  Where  possible,  instances 
of  these  models  were  used  in  place  of  actual  geometry  to  reduce  the  model’s  polygon  count.  This  detailed  model 
was  used  in  those  scenes  where  the  virus  was  disassembling,  replicating  and  assembling  itself.  A simple 
dodecahedron  was  substituted  in  those  animation  sequences  which  called  for  multiple  instances  of  the  virus.  In  both 
cases,  a green  texture  map  was  applied  to  the  models  to  accentuate  its  foreign  nature  within  the  cell. 

C)  The  Animation 

The  main  purpose  of  the  animation  was  to  explain  how  the  reovirus  kills  cancer  cells.  Accomplishing  this  meant 
showing  what  normal  cells  look  like,  comparing  normal  cell  growth  to  that  in  cancerous  cells,  and  finally 
demonstrating  how  the  reovirus  interacts  with  cancer  cells  to  ultimately  destroy  them.  This  task  involved  the 
visualization  of  a complex  series  of  biochemical  interactions  to  simulate  both  an  active  and  a hyperactive  Ras 
pathway,  some  of  which  were  simplified  to  preserve  the  clarity  of  the  story.  The  storyboards  for  the  animation 
composed  five  sequences:  1)  an  introduction,  2)  a depiction  of  normal  cell  growth  via  the  Ras  pathway,  3) 
uncontrolled  cellular  growth  in  a cancer  cell,  4)  the  introduction  and  replication  of  the  reovirus  in  a cancer  cell,  and 
5)  cell  lysis  (disintegration)  and  tumor  regression.  The  narrative  was  added  to  simplify  and  reinforce  concepts  that 
were  visually  being  presented. 

D)  Rendering 

Approximately  1000  hours  were  used  to  construct  the  models  and  render  the  resulting  five  minute  animation  within 
SOFTIMAGE  | 3D  version  3.8SP1.  The  animation  sequences  were  rendered  using  the  Softimage  rendering 
algorithm  on  10  Intergraph  workstations  and  one  SGI  Octane  workstation  at  a resolution  of  640  by  480  pixels.  The 
five  sequences  were  then  composited  and  rendered  with  the  narration  using  Adobe  Premiere  5.1. 

Conclusion 

In  the  end.  Dr.  Lee  and  his  research  teams  were  exuberant  in  their  praise  of  this  project.  The  animation  was  selected 
for  the  prestigious  SIGGRAPH  1 999  computer  animation  festival  and  a frame  from  one  sequence  has  appeared  in 
Computer  Graphic’s  World  (August  99). 
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Abstract:  This  paper  is  a report  on  the  findings  of  a study  conducted  with  a hypertext  learning 
system  used  in  a sophomore  history  class.  The  participants  were  given  an  orienting  task 
(browsing,  searching,  connecting)  to  guide  their  use  of  the  system.  The  connecters  performed 
the  best  on  conceptual  understanding  of  the  material,  followed  by  the  searchers  with  the 
browsers  performing  the  worst.  A number  of  en-route  indicators  also  showed  similiar  patterns 
of  access  by  all  three  groups,  including  access  to  articles  directly  related  to  items  of  the 
posttest  measures.  Connecters  also  reported  the  highest  levels  of  database  utility  for 
completing  their  task  and  the  posttest.  These  findings  demonstrate  positive  effects  for  both 
learning  and  attitudes  when  students  are  given  a conceptual  orienting  task  in  the  use  of  a 
hypertext  system. 


Introduction 

Hypertext  is  a method  for  organizing  information  that  allows  meaningful,  non-linear  access  to  text- 
oriented  resources.  Unlike  traditional  computer-based  instruction  and  databases,  hypertext  systems  allow  the 
user  to  access  information  by  “jumping  around”  via  a series  of  electronic  links,  whether  in  encyclopedias, 
textbooks,  magazines,  journals,  databases,  knowledge  bases  or  other  resources.  Although  the  concept  of 
hypertext  can  be  traced  back  at  least  50  years  (Bush,  1945),  it’s  “practical”  history  has  unfolded  over  the  past 
ten  years.  Personal  computer  software,  such  as  HyperCard  and  Toolbook,  have  allowed  the  most  novice 
computer  user  to  create  hypertext.  The  emergence  of  the  the  World  Wide  Web  in  the  past  few  years  has  pushed 
the  concept  of  hypertext  into  popular  usage. 

The  ease  in  both  producing  hypertext  environments  and  providing  access  to  students  has  created 
considerable  research  opportunties.  Gall  and  Hannafin  (1994)  outlined  a framework  for  the  study  of  hypertext. 
Within  this  framework,  they  suggested  examing  hypertext  from  the  perspective  of  a learning  environment, 
consisting  of  the  hypertext  learning  system,  learner  attributes,  the  learning  task,  and  educational  setting.  More 
recently,  Ayersman  (1996)  has  summarized  a number  of  research  studies  on  hypermedia-based  learning.  He  has 
organized  this  review  into  categories  based  on  the  main  focus  of  each  study  (perceptual/attitudinal,  individual 
differences,  systems  analyses,  and  performance).  In  short,  the  careful  analysis  of  hypertext  from  a number  of 
perspectives  is  a timely  and  important  topic  for  educators  and  researchers  in  the  field  of  educational  technology. 

At  a surface  level  the  concept  of  hypertext  is  appealing  for  education  due  to  its  flexibility  and  degree  of 
learner  control.  However,  past  research  on  learner  control  with  other  media  has  led  educators  to  be  wary  of  the 
effectiveness  of  giving  learners  too  much  responsibility  for  managing  their  learning.  Unless  they  are  highly 
experienced  or  dealing  with  familiar  content,  this  additional  managment  chore  can  easily  detract  from  the 
learning. 


This  presentation  reports  the  results  of  a study  conducted  on  the  use  of  external  orienting  tasks  to 
facilitate  learning.  Within  the  context  of  a learning  environment,  it  was  proposed  that  providing  learners  with 
goals  in  the  form  of  conceptual  questions  would  allow  them  to  better  manage  their  use  of  a hypertext  system. 
Many  of  the  advantages  of  hypertext,  such  as  flexibility  and  non-linear  access,  would  still  be  available  to  the 
student.  However,  the  conceptual  questions  would  foster  better  understanding  of  the  material  with 
corresponding  positive  attitudes  towards  the  use  of  hypertext.  Whether  differences  in  these  measures  existed  or 
not,  it  was  also  important  to  record  the  en-route  behaviors  of  learners  to  better  understand  the  way  in  which  this 
flexible  medium  was  used.  This  insight  came  from  critiques  of  earlier  learner  control  research. 
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The  Study 


The  purpose  of  this  study  was  to  determine  the  effects  of  various  external  orienting  tasks  on  learning, 
learner  attitudes,  and  en-route  behaviors  in  the  use  of  an  educational  hypertext  system.  Thirty-one  sophomore 
students  enrolled  in  three  honors  sections  of  an  introductory  military  history  course  at  a major  service  academy 
served  as  participants  in  this  study.  During  three  50-minute  class  periods,  the  participants  used  a hypertext 
database  containing  approximately  200  articles  on  World  War  I.  To  examine  the  efficacy  of  the  hypertext,  it 
was  important  to  conduct  the  study  within  the  context  of  an  authentic  learning  environment.  The  content  was 
both  relevant,  and  with  regard  to  the  structure  of  the  course,  timely  for  the  participating  students.  The  prior 
knowledge  and  experience  of  the  students  suggested  that  they  would  be  likely  to  be  able  to  successfully  manage 
their  learning  even  when  presented  with  a novel  delivery  method. 

Each  section  was  given  one  of  three  orienting  tasks~one  in  which  they  were  asked  to  browse  the 
database  to  learn  about  airpower  in  World  War  I (browsing  condition),  a second  in  which  they  were  required  to 
answer  factual  questions  about  the  material  in  the  database  (searching  condition),  or  a third  in  which  they  were 
required  to  answer  conceptual  questions  (connecting  condition).  These  tasks  were  chosen  as  representative  of 
the  types  of  activities  (except  for  the  creation  of  links)  with  which  hypertext  users  are  typically  involved  (Gall 
& Hannafin,  1994). 


Findings 

The  results  indicated  that  the  connecters  performed  the  best  on  conceptual  understanding  of  the 
material,  followed  by  the  searchers  with  the  browsers  performing  the  worst.  It  would  follow  that  those  pursuing 
the  answers  to  conceptual  questions  would  perform  well  on  that  type  of  question.  However,  it  is  more 
interesting  that  high-achieving  students  in  the  other  conditions  did  not  develop  that  same  understanding.  The 
factual  questions  answered  by  the  searchers  were  specifically  keyed  to  lead  them  to  the  same  information 
accessed  by  the  connecters.  A number  of  en-route  indicators  also  showed  similiar  patterns  of  access  by  all  three 
groups,  including  access  to  articles  directly  related  to  items  of  the  posttest  measures.  Connecters  also  reported 
the  highest  levels  of  database  utility  for  completing  their  task  and  the  posttest. 


Conclusions 

These  findings  demonstrate  positive  effects  for  both  learning  and  attitudes  when  students  are  given  a 
conceptual  orienting  task  in  the  use  of  a hypertext  system.  These  findings  do  not  discount  the  value  of  searching 
or  browsing.  However,  they  do  suggest  that  these  activities  may  limit  understanding  when  they  serve  as  the 
primary  mode  of  interaction.  Further  research  could  focus  on  having  students  develop  their  own  goals  or 
questions  for  exploration  in  addition  to  other  types  of  self-monitoring  activity. 
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Abstract:  This  paper  describes  the  prototype  design  of  a split-brain  user  interface  which 
delivers  two  independent  audio-visual  streams  simultaneously  to  each  hemisphere  of  the  brain. 
Users  view  digitized  C-SPAN  Archival  Video  of  the  emotionally  charged  testimony  of 
Clarence  Thomas  and  Anita  Hill  from  the  1991  United  States  Senate  Supreme  Court 
Nomination  Hearings.  This  interface  presents  these  two  diametrically  opposed  versions  of 
events  to  each  hemisphere  of  the  brain  creating  a kind  of  artificial  cognitive  dissonance. 


Introduction 

This  paper  describes  the  prototype  design  of  a split-brain  human  computer  user  interface.  Like 
research  in  augmented  virtual  reality,  augmented  interaction  and  wearable  computers  [Mann  1998]  this 
approach  seeks  to  extend,  enhance  and  augment  natural  intelligence  by  matching  the  innate  capacity  to  process 
and  understand  parallel  streams  of  dynamic  audio-visual  data  with  a new  interface.  Inspired  by  the  split  brain 
research  this  interface  delivers  dynamic  audio-visual  data  independently  to  each  hemisphere  of  the  normal 
brain. 

The  initial  prototype  was  tested  with  the  Virtual  Research  Systems  V-6  Head  Mounted  Display  which 
supports  two  640X480  60Hz  VGA  Active  Matrix  Liquid  Crystal  Displays.  A split-brain  human  computer  user 
interface  permits  the  user  to  comfortably  watch  one  digital  video  delivered  to  the  left  eye  while  watching  a 
different  digital  video  delivered  to  the  right  eye.  Simultaneously  the  user  hears  different  speakers  in  the  right 
and  left  ears.  By  restricting  the  resolution  and  position  to  one  half  of  the  raster  graphic  liquid  crystal  display  the 
left  or  right  hemisphere  can  be  targeted.  The  interface  permits  the  user  to  fluently  control  playback  by  pushing 
buttons  or  turning  control  handles. 


Background 

This  "split-brain"  approach  to  user  interface  design  was  inspired  the  "split-brain"  research  conducted 
by  Roger  Sperry  and  Ronald  Meyers,  J.E.Bogen  and  Michael  Gazzaniga  (Sperry  et  al.  1969).  Gazzaniga  along 
with  others  tested  ‘split-brain’  patients  who  have  undergone  the  surgical  procedure  of  commissurotomy  in  order 
to  limit  the  spread  of  an  epilectic  seizure  from  one  hemisphere  to  the  other.  Gazzaniga's  research  in  particular 
suggested  the  feasibility  of  a split-brain  interface. 

In  a study  of  a split-brain  patient  who  developed  a capacity  for  speech  in  the  right  hemisphere, 
Gazzaniga  presented  two  stories  separately  to  the  left  and  right  hemisphere.  When  read  by  a normal  subject 
these  two  stories  would  be  read  as  one  complete  story.  The  split-brain  patient  first  reports  only  what  the  left  or 
normally  verbal  hemisphere  sees.  Then  the  right  hemisphere  reports  what  it  read  and  when  the  patient  is  asked  a 
third  time  to  explain  what  was  read,  a third  story  is  told  made  up  of  what  was  read  by  both  hemispheres  and 
embellished  by  items  not  contained  in  either  source  story  (Gazzaniga,  1985).  Gazzaniga  attributes  to  the  left 
hemisphere  a “creative  narrative  talent"  which  he  calls  "the  interpreter  mechanism”  which  “is  constantly 
looking  for  order  and  reason,  even  when  there  is  none-which  leads  it  continually  to  make  mistakes.  It  tends  to 
overgeneralize,  frequently  constructing  a potential  past  as  opposed  to  a true  one.”  (Gazzaniga  1998) 
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Splitting  the  Brain 


The  split-brain  effect  requires  matching  the  visible  image  to  the  nasal  (closest  to  the  nose)  or  temporal 
half  of  the  retina.  As  the  optic  nerve  leaves  the  eye,  fibers  from  the  nasal  half  of  the  retinas  cross  at  the  optic 
chiasma  and  continue  along  the  optic  track  to  opposite  side  brain  hemisphere.  The  fibers  from  the  temporal  half 
of  the  retinas  do  not  cross  but  continue  to  the  same  side  hemisphere  (Swanston  & Wade  1991).  The  view  of  the 
world  seen  by  each  eye  can  be  divided  in  half  by  a line  at  the  nose.  The  left  and  right  field  of  view  are  referred 
to  as  the  left  and  right  hemifields.  The  light  coming  from  each  hemifield  is  refracted  and  inverted  through  the 
lens  of  the  eye  to  the  opposite  side  retina.  The  temporal  retina  of  the  left  eye  together  with  the  nasal  retina  of  the 
right  eye  see  the  right  hemifield  of  the  visual  world  while  the  temporal  retina  of  the  right  eye  together  with  the 
nasal  retina  of  the  left  eye  see  the  left  hemifield.  The  hemifields  overlap  which  is  essential  to  providing  the 
stereo  depth  perception  of  binocular  vision. 

Because  the  resolution  of  the  digital  video  was  selected  to  be  less  than  half  the  display  resolution  the 
digital  video  image  could  be  registered  along  the  inside  (nasal  edge)  or  the  outside  (temporal  edge)  of  each 
LCD.  Two  placements  of  the  digital  video  afforded  two  viewing  modes  which  I term  Convergence  and 
Divergence  Mode.  The  user  can  choose  either  Viewing  Mode  and  switch  them  at  will. 

Divergence  Mode  delivers  the  digital  video  to  each  nasal  retina  and  therefore  to  the  opposite  side 
hemisphere  of  the  brain.  With  Divergence  Mode  the  majority  of  first  time  users-viewers  can  quickly  and 
comfortably  view  both  digital  videos  as  separate  images.  With  Convergence  Mode  the  digital  video  is  seen  only 
by  the  temporal  half  of  the  retinas  of  each  eye  and  is  routed  to  the  same  side  hemisphere.  With  Convergence 
Mode  the  two  image  appear  to  be  superimposed  and  overlap.  At  least  a third  of  the  users  experienced  difficulty 
in  fusing  the  images. 


Conclusion 

This  ‘split-brain’  interface  does  seek  to  induce  in  the  ‘normal’  person  the  strange  behavior  and 
experience  of  split-brain  patients.  Rather  it  is  intended  to  create  a new  and  different  kind  of  viewing  experience. 
Additional  research  is  required  to  unravel  how  the  normally  connected  brain  filters  and  apportions  attentional 
focus  when  two  parallel  audio-visual  data  streams  are  presented  independently  to  each  hemisphere.  The  results 
of  such  experiments  may  yield  additional  insights  that  can  be  applied  to  the  design  of  a split-brain  interface. 
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Abstract:  Currently,  in  the  area  of  computer  aided  learning,  the  field  of  Computer  Supported 
Collaborative  Learning  (CSCL)  presents  research  opportunities.  Of  special  interest  of  our  group 
has  been  the  development  of  a CSCL  environment  based  on  a Digital  Library.  In  this  context,  we 
have  been  working  on  the  modeling  of  software  agents  that  assist  the  participants  in  the 
establishment  of  virtual  communities  of  practice  and  locate  easily  the  knowledge  considered  of 
their  interest,  making  a digital  library  a source  for  a more  adaptive  and  collaborative  learning 
environment. 


Introduction 

Digital  libraries  are  repositories  developed  in  order  to  collect  and  organize  knowledge  and  information,  making 
them  easily  accessible  to  users  [Fox,  98].  At  the  Centro  de  Investigacion  en  Tecnol 

Automatizacion  (CENTIA)  of  the  Universidad  de  las  Americas,  we  have  been  developing  BIDACI  (Biblioteca 
Digital  para  el  Aprendizaje  Colaborativo  en  Informatica),  a collaborative  learning  environment  based  on  a digital 
library  in  computer  science. 

Digital  Libraries  in  Education 

Digital  libraries  in  education  offer  opportunities  to  provide  access  to  resources  beyond  the  school  and  establish 
distributed  learning  communities,  increasing  the  integration  of  formal  learning  and  informal  learning  at  the 
workspace  or  home  [Marchionini,  97].  In  this  perspective,  we  consider  digital  libraries  as  the  fundamental 
repositories  for  CSCL  environments  and  lifelong  learning  environments. 

The  objective  of  this  project  has  been  the  research  and  development  of  software  components  that  allow  the 
distributed,  continuous  and  organized  construction  of  educational  material  in  computer  science  by  academics  from 
diverse  institutions  in  Mexico.  The  idea  was  to  develop  a digital  library  that  allow  the  exploration,  construction 
and  presentation  of  the  material.  The  project  has  been  supported  by  the  Informatics  Research  & Development 
Network  (REDII)  of  the  CONACyT,  the  Mexican  National  Council  of  Science  and  Technology.  The  central 
objective  is  to  increase  the  quality  of  education  in  computer  science  and  information  technologies  in  Mexican 
universities  For  the  development  of  BIDACI  we  considered  three  primary  roles  for  digital  libraries  in  education 
[Maly,  98]: 

1)  Course-related  information  and  course  materials  for  asynchronous  distance  learning. 

2)  Tools  for  constructing  and  organizing  materials.  The  processes  for  construction  must  be  easy  for  authors 
(technical  skills  and  motivation). 

3)  Tools  for  an  active  virtual  session.  It  must  support  active  learning,  where  learners  construct  their  own 
knowledge  through  resource  discovery. 

Software  Agents  in  BIDACI 

The  first  version  of  BIDACI  allow  the  users  to  organize  and  locate  easily  the  material,  as  well  as  communicate, 
cooperate  and  collaborate  using  groupware  tools.  For  the  next  version  we  proposed  the  development  of  three 
software  agents  in  order  to  increase  the  adaptiveness  with  the  users  and  facilitate  the  maintenance  and  location  of 
material  in  BIDACI.  Based  on  previous  work  on  agents,  digital  libraries  and  learning  environments  [Sanchez,  98] 
we  modeled  and  developed: 

1)  A user  agent. 

2)  An  information  agent. 

3)  A management  agent. 


BIDACrs  User  Agent 


The  user  agent  in  BIDACI  allows  it  to  be  an  adaptive  environment.  The  agent  holds  a user  model  [Ayala,  96] 
which  represents  the  user's  preferences,  being  a representation  of  her/his  interests  about  the  material.  Based  on  this 
information  and  data  from  other  user  agents,  it  autonomously  proposes  the  configuration  of  discussion  groups, 
helping  in  the  commitment  of  a chat  session  among  possible  participants  in  the  virtual  community.  The  user  agent 
proposes  a discussion  group  when  there  are  potential  participants  in  BIDACI's  community.  A user  is  considered  a 
potential  participant  when  s/he  has  shown  interest  in  a given  topic  or  area,  which  is  supported  by  the  data  of  the 
user  model  provided  by  her/his  user  agent.  Once  a group  has  been  proposed  the  potential  participants  are  notified. 
At  any  time  a learner  can  manually  make  a subscription  to  a given  discussion  group,  or  remove  her/his  name  from 
the  list.  The  user  agent  notifies  the  elimination  of  a discussion  group  where  only  one  member  remains. 

BlDACrs  Information  Agent 

The  user  must  be  aware  of  the  changes  of  the  dynamic  repository  of  distributed  educational  material.  S/he  must  be 
aware  of  the  material  that  is  considered  relevant  and  popular  by  the  community  and  the  new  members  who  show 
interest  in  the  same  topics  or  areas.  The  information  agent  has  been  developed  in  order  to  keep  the  user  informed 


a)  new  educational  material  considered  of  her/his  interest. 

b)  updated  educational  material  considered  of  her/his  interest. 

c)  educational  material  considered  popular  (and  therefore  relevant)  by  the  community. 

d)  chat  sessions  established  related  to  the  discussion  of  a topic  considered  of  her/his  interest. 

BIDACFs  Management  Agent 

The  management  agent  is  designed  in  order  to  provide  information  to  the  manager  of  BIDACI  concerning  the 
status  of  the  educational  material.  Periodically  it  autonomously  checks  the  material  and  obtains  information  about: 

a)  the  web  sites  that  have  not  become  accessible. 

b)  the  material  that  have  not  been  updated  in  a given  period  of  time. 

The  management  agent  sends  warning  e-mail  messages  to  the  authors,  with  a copy  to  the  manager.  It  constructs 
also  a report  for  a given  period.  Here  is  convenient  to  mention  the  commitment  of  the  authors  of  provide  to 
BIDACI  educational  material  every  year,  and  keep  it  accessible  and  updated. 

Conclusions 

We  have  been  working  on  the  modeling  and  development  of  software  agents  that  assist  the  participants  in  the 
establishment  of  virtual  communities  of  practice,  and  locate  easily  the  knowledge  considered  of  their  interest, 
making  a digital  library  a more  adaptive  and  collaborative  learning  environment.  We  believe  that  software  agent 
modeling,  from  the  perspective  of  learning  environments,  will  make  digital  libraries  the  fundamental  repositories 
for  CSCL  environments  and  lifelong  learning  environments 
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Abstract:  The  University  of  Western  Ontario  (UWO)  is  a large,  research-intensive  university 
(equivalent  to  a Class  I institution  in  the  US)  with  a traditional  undergraduate  curriculum  as  well  as 
a number  of  professional  schools.  As  a regional  institution,  UWO  is  situated  in  the  centre  of  the 
southwestern  Ontario  industrial  and  agricultural  heartland.  This  presentation  will  describe  the 
ongoing  evolutionary  process  that  has  given  rise  to  a centralized  facility,  the  Instructional 
Technology  Resource  Centre  (ITRC),  that  supports  faculty  in  the  development  of  on-line 
curriculum  and  course  materials.  Issues  that  are  relevant  to  this  process  touch  on  the  very  core  of 
the  educational  enterprise  in  an  institution  of  higher  learning. 


The  Instructional  Technology  Resource  Centre  (ITRC)  at  the  University  of  Western 
Ontario 

The  Present 

The  ITRC  was  bom  in  November  of  1 998  when  the  Vice-President,  Academic  allocated  targeted  funds  for  the 
creation  of  a central  support  area  dedicated  to  supporting  faculty  in  the  development  and  use  of  computer-based 
instmctional  materials.  The  ITRC  is  administratively  located  within  the  central  unit  that  supports  all  computational 
activity  on  campus,  the  Information  Technology  Services  (ITS)  unit.  The  ITRC  mandate  states,  ‘The  Instmctional 
Technology  Resource  Centre  is  a multi-media  support  facility  for  faculty  who  wish  to  integrate  technology  into  their 
courses.” 

ITRC  is  managed  by  an  Advisory  Board  composed  of  the  Directors  of  the  Ancillary  Academic  units  (Libraries,  the 
Educational  Development  Office,  the  Western  Centre  for  Continuing  Studies,  ITS,  the  Coordinator  of  Summer  and 
Distance  Studies)  and  a faculty  member  from  each  of  the  12  Faculties  selected  by  the  Dean.  Logistical  support  is 
provided  by  ITS. 

ITRC  is  fundamentally  an  instmctor  driven  enterprise  in  that  it  functions  as  a resource  centre  for  those  who  have  the 
interest  in  the  development  and  use  of  computer-based  teaching  and  learning  materials.  ITRC  is  managed  on  a day- 
to-day  basis  by  a full-time  Coordinator  who  has  a strong  background  in  both  information  technology  and 
educational  practice.  As  well,  two  part-time  ITS  staff  provide  technical  support  and  assistance  in  high-end  content 
development  for  specific  projects.  Student  developers  are  hired  from  the  undergraduate  population,  and  all  have 
skills  in  the  production  of  on-line  curriculum  materials. 


BEST  COPY  AVAILABLE 


The  ITRC  operates  in  two  main  modes:  first,  as  a drop-in  centre  where  faculty  can  make  use  of  the  human,  hardware 
and  software  resources.  Secondly,  ITRC  issues  requests  for  proposals  to  the  campus  community  twice  yearly. 
Faculty  and  professional  librarians  submit  proposals  to  the  Advisory  Board  where  they  are  reviewed  according  to 
established  criteria  and  voted  upon  for  support.  Supported  projects  are  assigned  a student  developer  who  then  works 
with  the  faculty  member  to  develop  the  project  from  concept  to  product  under  the  supervision  and  advice  of  the 
Coordinator.  Although  faculty  are  encouraged  to  acquire  themselves  the  skills  needed  to  develop  their  project,  this 
is  not  an  absolute  requirement.  The  ITRC  student  developer  is  considered  to  be  the  technical  “hands-on”  individual. 

The  Past 

The  ITRC  represents  the  most  recent  iteration  of  central  support  for  instructional  technology  at  the  University  of 
Western  Ontario.  In  1 992,  the  so-called  “multi-media”  studio  was  established  and  operated  for  four  years  on  grant 
funds  that  were  used  to  “buy”  faculty  from  a half-course  teaching  load  allowing  them  to  spend  the  time  in  the  studio 
acquiring  the  necessary  technical  skills  and  developing  their  projects.  The  studio  was  located  within  one  of  the  12 
Faculties  and  did  not  have  a high  profile  on  campus. 

This  model  proved  to  be  relatively  unsuccessful  in  that  faculty  were  unable  in  most  instances  to  develop  the  depth  of 
necessary  skills  and  create  their  products  in  the  time  available.  Only  a very  few  of  the  projects  that  were  completed 
ever  did  see  use  in  the  classroom  or  computer  lab.  However,  the  studio  did  serve  very  effectively  as  a pilot  project 
and  contributed  greatly  to  the  design  and  concept  of  the  ITRC. 

The  Future 

The  University  of  Western  Ontario  is  currently  developing  an  institutional  strategic  plan  for  instructional  technology 
(IT).  IT  has  become  a very  visible  issue  with  the  Faculty  Association,  with  academic  departments,  with  the  Distance 
Studies  operation  and  with  the  central  administration.  The  planning  process  is  affected  by  both  internal  and  external 
factors:  educational  quality,  demand,  opportunity  and  alternative  sources  of  instruction.  The  universal  input  from 
faculty  members  to  this  process  has  been  to  support  an  expanded  version  of  the  ITRC  with  additional  professional 
staff  and  financial  resources. 

The  goals  of  the  planning  process  will  be  to  maintain  the  current  philosophy  of  the  ITRC  as  an  instructor-led  as 
opposed  to  top-down  operation  and  to  respond  to  the  needs  of  both  instructors  and  students.  Consideration  of  how 
to  arrive  at  these  goals  has  touched  on  the  very  core  of  the  university’s  mandate;  issues  such  as  the  role  and  place  of 
teaching  at  a research-intensive  university,  the  promotion  and  tenure  process  as  it  relates  to  teaching  as  a scholarly 
activity,  the  role  of  part-time  faculty,  the  perceived  versus  actual  demands  from  students  for  flexible  learning 
opportunities  and  the  financial  implications  of  technology-based  teaching  and  learning. 
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Abstract:  Educational  Institutions  have  been  using  the  World  Wide  Web  for  a long  time  in  the 
provision  of  education.  Currently  we  can  observe  a growing  knowledge  complexity  and  a widening 
variety  of  backgrounds  and  qualifications.  As  a consequence,  we  need  modularly  structured 
contents,  which  are  linkable  across  subject  domains  and  support  different  kinds  of  usages. 
However,  the  demanding  requirements  for  comprehensive  and  flexible  educational  services  clash 
with  the  complexity  of  its  development.  To  alleviate  this  tension,  the  IMS  Project  proposes  a 
generic  architecture  for  online  learning  based  on  extensibility  and  composability  of  its  components. 


WWW  and  University  Education 

People  from  all  comers  of  the  university  community  now  have  the  ability  to  create  Web  pages  and  make  them 
available  to  the  world.  While  the  benefits  of  HTML-based  documents  are  clear,  it  is  also  evident  that  this  use  has 
some  deficiencies:  there  is  no  interactivity,  the  concept  of  role  is  not  included,  it  does  not  take  into  account  the  state 
of  the  learning  process,  it  is  restricted  to  a few  information  formats  and  there  is  no  quantitative  measure  of  student 
performance  or  the  effectiveness  of  teaching.  There  are  several  technologies  that  overcome  the  HTML  limitations 
(Allen  1998):  interactivity  can  be  achieved  with  Java  applets  embebed  in  HTML  pages,  or  JavaScript  combined  with 
HTML  forms;  with  VRML,  teachers  can  create  virtual  environments  where  students  are  invited  to  deal  with  realistic 
situations;  XML  - a system  for  defining,  validating,  and  sharing  document  formats  on  the  Web,  appropriate  for 
separating  user  interface  behaviour  and  the  semantic  content  of  information;  SMIL  - an  application  of  XML  to 
coordinate  a streaming  multimedia  presentation  with  more  than  one  type  of  media  file. 

However,  we  can  not  talk  only  about  technologies,  there  is  also  a new  learning  style  and  it  is  necessary  to  design 
tools  that  support  it  (Schlichter  1998).  In  this  way  it  is  common  that  students  work  in  groups  to  write  and  submit 
reports,  or  debate  alternative  perspectives  and  solutions  on  a common  problem.  Tools  to  engage  this  collaborative 
work  in  a distributed  way  are  needed,  like  discussion  forums,  chats,  whiteboards  and  collaborative  authoring  tools. 
This  leads  to  the  creation  of  new  powerful  learning  environments,  which  are  not  adaptations  from  classical  teaching 
and  training,  able  to  produce  high  quality  multimedia  education  material,  learner-centered  and  network-based.  To 
show  our  work  in  this  line,  in  the  next  section  we  present  one  of  the  main  standardisation  effort  in  the  educational 
environment  the  IMS  project. 


The  IMS  System  of  the  University  of  Murcia 

The  Instructional  Management  Systems  (IMS)  project  (http://www.imsproject.com)  is  a partnership  of  academic, 
commercial  and  government  organisations  to  cooperate  in  the  definition  of  an  Internet  architecture  for  instructional 
systems  and  learning  content,  with  a primary  objective:  enable  an  open  architecture  for  online  learning  in  all  sectors 
of  lifelong  learning.  Learning  materials  are  collected  in  IMS  containers,  which  include  a unique  identifier  and 
metadata  that  describes  the  container  and  its  contents  to  facilitate  search,  storage,  retrieval  and  use  rights.  Learning 
materials  can  be  web  pages,  interactive  content  or  collaboration  tools.  Containers  can  be  aggregated  or  nested,  and 
the  contents  of  a container  may  be  sequenced  according  to  different  rules.  IMS  environments  are  software 
developments  that  implement  the  IMS  Specification.  The  IMS  environment  extracts  the  contents  from  IMS 
containers  and  delivers  them  to  users,  normally  with  a Web  server. 


The  next  figure  shows  the  components  of  the  IMS  System  and  the  relations  between  them.  The  central  element  is  the 
Management  System,  the  component  that  provides  the  interface  to  access  or  administer  learning  materials.  It 
identifies  users,  creates  sessions  to  facilitate  a uniform  mechanism  of  access  control  and  supervises  the  actions  that 
users  can  perform,  according  to  their  roles.  Through  the  Management  System,  an  administrator  can  install  new 
content  packages,  and  perhaps  create  new  users  or  groups  and  edit  access  rights.  A student  also  uses  the 
Management  System  to  operate  with  content. 
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The  Content  Server  (there  would  be  more  than  one)  stores  the  learning  materials  and  their  metadata,  and  provides 
the  IMS  CORBA  interfaces  to  access  those  materials.  A Search  Engine  can  use  those  interfaces  to  perform 
intelligent  searches.  As  a first  prototype,  our  IMS  System  will  use  already  created  html -based  documents  as 
learning-material,  with  associated  XML  files  to  describe  them.  The  Management  System  also  uses  Content  Servers 
to  get  the  materials  requested  by  users.  User  profiles  are  stored  on  the  Profile  Server,  which  also  provides  a special 
interface  to  access  this  information. 


The  University  of  Murcia,  like  other  Spanish  universities  - UNED,  UOC  - has  a clear  interest  in  the  use  of  new 
technologies  applied  to  higher  education  as  a way  to  improve  the  quality  of  the  learning  process.  This  interest  took 
shape  last  year  in  a project  called  SUMA  - Servicios  Universidad  de  Murcia  Abierta.  SUMA  affects  two  main 
fields:  network  infrastructure,  that  will  be  scaled  to  support  10.000  users  and  1.000  concurrent  connections,  and 
telematic  services.  We  are  designing  the  new  architecture  of  the  SUMA  framework  taking  into  account  the  ideas 
defined  in  IMS,  and  considering  CORBA  and  XML  as  fundamental  elements  of  the  learning  support.  We  are  also 
working  in  the  introduction  of  videoconference  tools  and  security  elements  in  this  system.  There  is  a PKI  based  on 
the  use  of  X509  certificates  that  has  been  completed  successfully  with  the  development  of  a PKCSll  module  that 
lets  students  use  web-based  services  securely  with  HTTPS.  Each  student  and  teacher  has  his/her  private  key  stored 
on  a smartcard  protected  by  a PIN.  When  the  user  connects  to  a secure  web  server,  the  Netscape  browser  activates 
the  PK.CS1 1 module,  which  performs  cryptographic  operations  with  the  smartcard.  SUMA  project  of  the  University 
of  Murcia  represents  one  step  in  the  development  of  an  Open  University,  based  on  the  used  of  the  WWW 
technologies.  In  order  to  be  successfully  implemented,  it  is  necessary  not  only  to  create  innovative  collaborative 
tools  that  support  the  active  learning,  but  also  to  design  the  whole  system  under  the  light  of  open  specifications  like 
the  one  proposed  by  IMS. 
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Art  and  Technology  in  the  Classroom:  The  Activities  and  Impact  of  a National 
Project . 

"The  Community  Discovered:  The  Search  for  Meaning  Through  the  Integration  of  Art 
and  Technology  in  K-12  Education"is  now  in  its  final  year  of  its  five  years  of 
funding,  and  is  being  conducted  by  Wests ide  Community  Schools  in  Omaha, 

Nebraska.  The  major  focus  of  The  Community  Discovered  project  is  represented  by 
its  five  goals,  which  are  carefully  integrated  into  the  objectives  undertaken  in 
the  project.  These  five  goals,  and  brief  examples  of  activities  within  the 
project,  consist  of  the  following: 

Goal  1:  To  enable  students  to  achieve  high  academic  standards  by  integration  of 

the  arts  and  technology  into  core  subject  areas 

Goal  1 Example  Activities:  The  project  has  developed  a web  based  curriculum  unit 
structure  by  which  teachers  document  developing  curriculum  units  which  target 
the  effective  integration  and  technology  into  core  subject  areas.  In  addition, 
teachers  complete  electronic  growth  plans  which  describe  this  ongoing  effort  and 
focus  within  the  project  to  actively  engage  students  in  discipline  based  art 
activities  using  technology,  and  resources  available  over  the  Internet. 

Goal  2 : To  provide  students  and  educators  equity  in  access  to  information  and 

museum  resources  at  the  state  and  national  levels 

Goal  2 Example  Activities:  The  project  has  developed  a model  web  site  for 

convenient  access  of  the  information  and  museum  resources  developed  by  all  of 
the  project  partners.  This  web  site  can  be  accessed  at: 


Goal  3:  To  enable  educators  to  effectively  use  appropriate  technologies  for 

constructivist  teaching  and  learning  across  the  curriculum 

Goal  3 Example  Activities:  Educators  within  the  project  have  participated  in 

extensive  inservice  activities  targeted  at  constructivist  learning  and 
interdisciplinary  teaching  across  the  curriculum.  These  have  entailed  summer 
institutes,  periodic  inseirvices  during  the  regular  year,  and  a variety  of  local 
efforts,  such  as  "curriculum  street  fairs"  within  the  schools. 

Goal  4:  To  enable  educators  to  implement  effective  curricula  incorporating  the 

arts  and  technology  in  core  subject  areas 

Goal  4 Example  Activities:  Participating  educators  have  also  engaged  in 

considerable  inservice  efforts  related  to  the  use  of  technology  in  teaching  and 
learning,  with  a focus  on  the  integration  of  the  arts  into  core  disciplines.  A 
particular  innovative  example  during  this  last  year  was  a national  ConferNet 
effort,  which  was  a model  on-line  student  conference  offered  for  participating 
classes  of  K-12  students  and  teachers.  A second  academic  on-line  conference  for 
students  is  being  planned  for  the  Spring  of  2000. 

Goal  5 : To  create  a national  network  of  educators  to  support  the  development 
and  implementation  of  appropriate  learning  strategies  integrating  technology, 
and  the  arts  with  other  core  subject  areas. 

Goal  5 Example  Activities:  The  project  has  been  highly  successful  in  creating  a 

network  of  individuals  assisting  with  the  project.  In  particular,  an  extensive 


http : //communitydisc . wst . esu3 . kl2 . ne . us . 


listserv  process,  special  interest  groups,  and  a three  level  national  advisory 
board  is  in  operation  and  is  being  utilized  throughout  the  project.  The  members 
of  the  advisory  boards  are  often  asked  to  interact  with  project  staff  for  the 
purpose  of  guiding  the  evolution  of  the  project  and  providing  feedback  regarding 
the  outcomes. 

THE  IMPACT: 

The  Community  Discovered  Project  has  had  considerable  success  with  both  students 
and  teachers  in  the  project,  and  is  evolving  into  a national  model  for  the 
effective  integration  of  technology  and  art  into  education.  The  ongoing 
evaluation  process  of  the  project  uses  multiple  sources  of  information,  and 
includes  a comprehensive  approach  to  data  collection  that  is  targeting 
information  related  to  each  project  goal  and  objective.  These  data  types 
include:  1)  teacher  survey  data,  2)  electronic  data,  such  as  listserv 
participation  and  electronic  logs,  3)  classroom  observations,  4)  teacher  and 
student  interviews,  5)  student  projects  and  portfolios,  6)  teacher  growth  plans, 
7)  focus  groups,  8)  stakeholder  surveys,  and  9)  teacher  based  classroom  research 
initiatives . 

All  data  is  summarized  and  placed  within  a World  Wide  Web  page  format  that  is 
available  for  review  by  anyone  interested,  and  especially  project  staff, 
participants,  and  stakeholders.  The  web  site  representing  the  evaluation 
portfolio  is: 

http : //ois . unomaha . edu/cdeval/ 

The  evaluation  web  site  is  fairly  comprehensive  and  is  organized  using  a variety 
of  themes  and  graphics  in  order  to  present  the  evaluation  information  clearly. 
The  evaluation  team,  composed  of  three  education  professors  from  the  University 
of  Nebraska  at  Omaha,  and  two  assessment  specialists  from  WestEd  Laboratories, 
has  documented  the  impact  of  the  CD  project  within  this  web  site.  Some  of  this 
impact  has  included: 

1)  In-depth  Participation:  The  project  has  already  successfully  involved  over 

300  teachers,  300  teacher  partners,  and  24,000  students,  with  in-depth 
technology  and  art  integration  experiences  within  the  project. 

2)  An  Impact  on  Student  Achievement:  The  project  has  resulted  in  enhanced 

student  achievement  as  documented  by  carefully  controlled  classroom  studies 
involving  over  600  students.  These  studies  have  demonstrated  significant 
project  impact  through  improved  reading  scores,  mathematical  patterning  skills, 
writing  fundamentals,  art  interpretation  skills,  and  general  attendance  of  the 
students . 

3)  An  Excellence  in  Student  Products:  The  project  has  collected  a wide  range 

of  innovative  student  products  which  show  the  power  of  students  working  within 
technology  based  environments,  including  projects  involving  hypermedia,  the 
World  Wide  Web,  and  virtual  reality. 

4)  An  Impact  on  Teaching  Knowledge  and  Skills:  Teacher  knowledge  has  greatly 

improved  within  the  project.  Through  surveys  and  interviews,  96%  of  the 
teachers  report  improvement  in  technnology,  99%  report  improvement  in  art 
integration,  85%  report  improvement  in  interdisciplinary  teaching,  and  90% 
report  improvement  in  constructivism.  In  addition,  a careful  analysis  of  the 
teaching  process  through  formal  classroom  observations,  has  documented  that 
teachers  are  indeed  involved  in  effective  instructional  techniques  related  to 
these  four  areas. 


5)  An  Extensive  Contribution  to  the  Professional  Literature:  The  Community 

Discovered  Project  has  been  aggressive  in  its  contribution  to  the  professional 
literature,  and  has  already  resulted  in  6 professional  refereed  articles/papers, 
2 graduate  theses,  and  more  than  20  refereed  presentations  at  national 
conferences  including  well  respected  conferences  such  as  the  National 
Educational  Computing  Conference  (NECC) , the  Society  for  Information  Technology 
in  Education  (SITE)  annual  conference,  the  American  Education  Research 
Association  conference  (AERA) . 

6)  A Model  Web  Based  Environment:  The  project  has  developed  an  impressive 

presence  and  use  of  the  World  Wide  Web.  As  mentioned,  this  presence  is 
accessible  at  http://communitydisc.wst.esu3.kl2.ne.us,  and  includes  an  extensive 
project  web  site,  an  on-line  evaluation  portfolio,  databased  curriculum  units, 
and  substantial  listserv  and  conferencing.  The  project  has  also  conducted  the 
well  respected  ConferNet  99,  the  first  ever  national  student  conference 
conducted  on  the  Internet.  Museum  partners  also  maintain  impressive  project 
related  web  sites,  which  include  a variety  of  digitized  works,  virtual  museum 
tours,  on-line  magazines,  and  a wide  range  of  Internet  based  curriculum  support 
materials . 

A FINAL  THOUGHT: 

It  is  said  that  a good  project  will  always  continue  to  grow  and  evolve,  and  "The 
Community  Discovered:  The  Search  for  Meaning  Through  the  Integration  of  Art  and 
Technology  in  K-12  Education"  is  now  well  established,  growing  systematically, 
and  evolving  fully  in  its  final  year  of  formal  activities.  The  project  has 
embraced  advanced  information-based  technologies  and  has  refined  its  plans  as 
technology-based  applications  continue  to  evolve  with  emerging  capabilities, 
such  as  new  capabilities  on  the  World  Wide  Web.  The  project  is  building  upon 
its  earlier  successes,  upon  a strong  organizational  foundation,  and  with  an 
aggressive  commitment  by  its  many  partners. 

As  The  Community  Discovered  project  moves  forward,  perhaps  most  importantly, 
there  continues  to  be  a real  team  effort  underway  by  all  stakeholders  in  the 
project  to  positively  effect  the  learning  environment  for  all  students.  This 
teamwork  is  growing  and  becomes  more  solid  during  each  successive  year  of  the 
project.  As  any  project  desiring  to  stay  on  the  "cutting  edge",  it  is  expected 
that  The  Community  Discovered  project  will  continue  to  evolve  as  new  educational 
technologies  and  understandings  become  available.  The  project  is  a 
comprehensive  one,  and  its  use  of  a systematic  implementation  process  and  the 
incorporation  of  a careful  evaluation  plan  are  no  doubt  critical  components  in 
its  effectiveness  to  date  and  promise  of  effectiveness  in  the  future.  The 
strong  and  full  commitment  by  all  partners  to  contribute  to  the  overall  success 
of  the  project  will  no  doubt  continue  to  provide  a natural  catalyst  for  success 
as  the  project  strives  to  implement  its  very  aggressive  set  of  goals  and 
objectives . 
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Abstract:  This  paper  presents  an  overview  over  an  experiment,  which  investigated  the  effect 
of  dynamic  (e.g.  animations)  and  static  (e.g.  pictures)  media  types  for  process  oriented 
learning  content.  Teachers  who  want  to  ^ply  new  technology  in  education  need  guidelines  in 
order  to  develop  the  presentations  themselves,  supervise  programmers  or  to  know  what  to  buy. 
Many  fragments  for  such  guidelines  do  exist,  our  experiment  is  a contribution  to  build  a 
consistent  empirically  based  knowledge  base  in  this  area.  A small  advantage  of  animations  for 
pictures  was  found,  more  attention  should  be  given  to  the  design  of  such  presentations. 


Introduction 

Multimedia  is  a challenge  for  the  educational  field,  insofar  as  consistent  guidelines  for  how  to  apply  the  different 
media  types  for  the  presentation  of  learning  materials  are  lacking.  The  tools  for  developing  multimedia  presenta- 
tions are  still  not  easily  applicable  for  teachers.  Hence,  teachers  may  have  the  choice  between  not  applying  new 
technology,  learning  how  to  implement  presentations  themselves  or  organise  the  necessary  productions  with 
external  resources.  Whenever  the  two  latter  options  are  chosen  guidelines  about  how  to  make  high  quality 
presentations  are  needed,  either  for  supporting  own  development,  to  supervise  a programmer  or  to  know  what  to 
buy.  This  paper  presents  an  experiment  in  which  features  relevant  for  the  presentation  of  dynamic  learning  con- 
tents (processes)  were  compared.  A former  experiment  investigated  how  to  present  static  learning  contents  (prin- 
ciples) (Guttormsen  Schar,  Kaiser,  & Krueger,  1999).  We  have  taken  a task  dependent  approach  to  media  alloca- 
tion, based  on  a general  categorisation  of  the  learning  content  as  concepts,  principles,  facts  and  processes 
(Merrill,  1983).  In  the  multimedia  context,  dynamic  features  could  be  animation,  film,  voice  or  simulation.  Em- 
pirical guidelines  for  the  implementation  of  multimedia  for  dynamic  learning  material  are  only  slowly 
approaching.  The  reason  is,  on  the  one  hand,  that  the  tools  and  technical  resources  for  the  production  of  dynamic 
presentations  are  only  recently  easy  to  apply.  On  the  other  hand,  the  design  of  experiments  in  this  area  requires 
much  effort,  because  the  production  of  unique  testmaterial  (e.g.  animations)  is  both  time  and  resource  demand- 
ing. Nevertheless,  to  realise  the  pedagogical  potential  of  multimedia,  there  is  a great  need  for  such  research. 


Experiment 

The  general  hypothesis  was  that  process  oriented  learning  material  is  better  learned  with  dynamic  presentations, 
in  this  case  operational  defined  as  animations.  The  experimental  conditions  are  shown  in  Table  1.  Unique  anima- 
tions of  5 different  processes  were  produced;  the  static  presentations  were  derived  from  the  animations.  The 
experimental  situation  was  fully  computerised  and  all  the  data  was  registered  in  a log-file.  Operational  definitions 
of  learning  performance  are  given  below: 

Visual.  The  recognition  of  logically  correct  pictures,  under  a selection  of  similar  pictures  with  small  errors. 
Active:  Complete  incomplete  sentences  in  which  essential  concepts  or  expressions  were  missing. 

Passive:  Choosing  correct  statements  from  a collection  of  both  wrong  and  correct  examples  from  the  tasks. 
Preferences:  “Which  of  the  presentation  forms  (/  tasks)  did  you  like  most?” 


Experiment 

Presentation  form 

Task 

5x5  within  group 
Randomisation:  form  and  task 
Subjects:  27  adult,  both  sexes 
Learning  performance: 
visual,  passive  & active 

Animation 
Animation  + voice 
Animation  + QT  control 
Picture  + voice 
Picture  + text 

Heart  functions  / ECG 
The  protein  synthesis 
The  Malaria  cycle 
Off-side  rules  in  soccer 
Solar  eclipse 

ERIC 
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Table  1:  presentation  forms  and  tasks  in  the  experiment 


Results 

Table  2 shows  the  effect  of  presentation  form  on  the  subjects  learning  performance.  No  effects  of  the  control 
variables  (sex,  task  order,  computer  knowledge,  age)  were  found. 


Performance 

Effects  of  presentation  form  (*  sign,  below  p < 0.05) 

Visual  knowledge 

No  effect 

Active  knowledge 

No  effect 

Passive  knowledge* 

Animation  + Voice  & Picture  + Voice  > 
Animation  & Animation  +Control 

Preference* 

Animation  + Voice  > 

Animation  & Animation  + Control  & Picture  + Text 

Table  2:  Effects  of  Presentation  Forms 


Discussion  and  Conclusion 

The  benefit  of  dynamic  presentation  was  smaller  than  expected.  However,  the  fact  that  we  found  no  difference 
between  presentations  with  animation  (without  voice)  and  picture  + text  shows  that  movement  possesses  basic 
informational  quality.  The  hypothesis  is  further  supported  in  that  the  best  learning  effect  was  achieved  with  pres- 
entation forms  including  a dynamic  factor  (animation  and  voice).  The  pictures  were  careflilly  produced  to 
represent  the  same  information  as  the  animations  (e.g.  movements  were  indicated  with  arrows).  Hence,  the 
relative  good  effect  of  pictorial  information  may  reflect  good  picture  quality  more  than  low  importance  of 
animation.  In  conclusion,  animations  alone  are  no  guarantee  of  good  learning  performance  when  pictures  and 
voice  can  be  produced  with  high  information  quality.  Still,  animated  presentations  were  preferred  by  the  subjects. 
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Research  has  been  conducted  on  building  tools  that  allow  users  to  control  the  speed  of  audio  when 
playing  audio  on  a computer  and  some  commercial  products  are  beginning  to  appear  on  the  market  (Eloguent, 
1999).  The  research  to  date  has  focused  exclusively  on  time-compressed  (i.e.  speeded-up)  audio  with  an 
emphasis  on  improving  the  user  interface  (Arons,  1997)  and  improving  learning  efficiency  (i.e.  learning  per 
unit  of  time)  (Harrigan,  1995a;  Harrigan,  1999).  The  research  has  shown  that  audio  can  be  time-compressed 
by  approximately  50%  without  any  loss  of  comprehension  or  retention  and  that  learners  do  use  the  time- 
compressed  audio  if  given  the  controls  in  the  user  interface.  The  research  has  also  shown  that  using  a known 
technique,  called  SOLA  ( Roucos  & Wilgus,  1985),  speech  can  be  time-compressd  without  any  significant 
change  of  pitch  which  is  much  preferred  by  learners  over  simply  playing  the  audio  faster  which  producing  a 
change  in  pitch  and  what  is  often  referred  to  as  the  “Mickey  Mouse”  effect. 

This  paper  reports  on  the  first  experiments  that  have  been  conducted  using  SOLA  to  time-expand  (i.e. 
slow  down)  audio  without  any  significant  pitch  change.  These  preliminary  experiments  used  subjects  who  had 
no  special  needs  but  the  end  use  of  this  technique  is  envisioned  to  be  for  people  with  special  needs  such  as 
those  with  acquired  brain  injuries. 

Research  Questions 

The  two  questions  we  addressed  were: 

1)  When  time-expanded  audio  is  played  to  learners  at  what  point  will  they  notice  that  the  audio  has  been 
time-expanded?  In  other  words,  what  is  the  Just  Noticeable  Difference  (JND)  of  time-expanded  audio. 

A)  When  the  audio  has  been  time-expanded  simply  by  playing  the  audio  . 

B)  When  the  audio  has  been  time-expanded  using  the  SOLA  technique. 

2)  What  is  the  time-expansion  percentage  at  which  learners  will  not  be  able  to  understand  the  audio.  In 
other  words,  the  intelligibility  level.  This  has  typically  been  measured  by  playing  speech  segments  to 
subjects  at  an  accelerated  rate  and  determining  the  point  at  which  the  subject  cannot  understand  50%  of 
the  speech  segments  (Harrigan,  1995b). 

The  Experiments 

As  stated,  we  intend  to  use  time-expanded  audio  with  people  with  special  needs.  However,  in  these 
initial  experiments  we  used  subjects  who  had  no  special  needs  as  we  felt  we  could  gain  two  things  (a)  we  could 
ensure  that  our  software  worked  properly  and  (b)  we  could  address  some  fundamental  questions  that  may  shed 
some  light  on  issues  related  to  time-expanded  audio  and  may  help  us  calibrate  our  future  experiment.  We 
were  successful  on  both  counts. 

Experiments  One:  We  had  a speech  file  on  our  computer  that  was  a help  file  from  an  application. 
We  used  the  SOLA  technique  to  create  versions  time-expanded  by  5%,  10%,  15%,  20%,  and  25%  for  a total  of 
six  files  including  the  original.  We  then  created  a screen  with  thirty  buttons  labelled  1-30.  As  each  button 
was  selected  one  of  the  six  speech  files  was  played  with  each  speech  file  being  played  five  times  in  total.  After 
selecting  each  button,  the  learner  had  to  circle  on  a piece  of  paper  whether  the  speech  sounded  normal  or  time- 
expanded.  The  results  from  the  seven  learners  (N=7)  showed  that  the  average  JND  was  5.4  % with  a SD  of 
3.9. 
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Experiment  Two:  Experiment  two  was  identical  to  experiment  one  except  that  the  audio  files  were 
created  by  simply  playing  the  audio  slower  with  a resulting  change  in  pitch.  The  six  learners  (N=6)  had  an 
average  JND  of  9.2%  with  a SD  of  2.6. 

Experiment  Three:  In  this  experiment  we  had  five  files  time-expanded  using  the  SOLA  technique  by 
38%,  58%,  84%,  100%,  and  121%.  Some  calibration  was  done  prior  to  the  experiment  in  order  to  choose  the 
appropriate  compression  levels.  We  had  25  continuous  speech  segments  from  a file  in  an  educational 
multimedia  CD  that  we  had  created.  Segments  were  about  six  words  each.  As  an  example,  the  first  speech 
segment  was  *‘it  was  important  to  open  my  mouth”.  We  created  a screen  with  25  buttons.  The  six  learner’s 
(N=6)  task  was  to  click  on  each  button  in  sequence,  listen  to  the  speech  segment,  and  then  write  down  the 
phrase  that  had  been  spoken.  After  the  experiment  the  two  investigators  read  each  response  and  made  a 
judgement  as  to  whether  they  understood  the  phrase.  The  results  showed  an  intelligibility  level  of  92%  with  a 
SD  of  zero.  This  means  that  all  subjects  understood  the  audio  at  84%  but  not  at  100%,  The  intelligbility  is 
calculated  as  the  middle  point  between  these  two  levels  (i.e.  ((84+100)/2)=92).  An  intelligibility  level  of  92% 
means  that  a one  minute  file  plays  in  1.92  minutes  (1  minute  55.2  seconds)  which  is  almost  twice  the  length  of 
the  original  file. 


Conclusion  and  Future  Work 

The  results  of  the  first  two  experiments  clearly  indicate  that  the  SOLA  technique  is  superior  to  simply 
playing  the  audio  slower  because  when  using  the  SOLA  technique  learners  don’t  realize  the  audio  is  time- 
expanded  until  a higher  time-compression  percentage  is  used. 

The  results  from  the  third  experiment  are  close  to  what  we  expected.  When  audio  is  speeded-up  it 
can  be  speeded-up  by  double  (i.e.  played  twice  as  fast)  before  intelligibility  begins  to  drop  (Foulke  & Sticht, 
1969)  and  with  this  experiment  we  show  that  it  can  be  slowed  by  almost  double  before  intelligibility  begins  to 
drop.  Since  people  normally  speak  at  150  words  per  minute  (wpm)  this  means  people  can  listen  at  rates  up  to 
300  wpm  and  as  low  as  almost  75  wpm.  The  results  of  this  experiment  confirm  that  using  time-expanded 
audio  has  the  potential  to  be  used  to  half  the  speaking  rate  and  thus  time-compressed  audio  may  be  useful  for 
people  with  certain  special  needs. 

We  now  plan  to  continue  our  work  with  the  following  initiatives:  (a)  produce  a system  that  can  be 
used  by  the  learners  to  control  the  speed  of  audio  and  test  it  with  learners,  (b)  test  our  system  with  learners 
with  special  needs  who  might  benefit  from  this  system  that  supports  time-expanded  audio  and  (c)  build  a 
system  that  includes  audio  as  well  as  video  and  test  it  with  learners. 
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Abstract:  Project  DEED  is  a four-year  internet  based  special  education  personnel  preparation 
program  funded  by  the  U.S.  Dept,  of  Education,  Office  of  Special  Education  Programs  (OSEP)  as 
located  at  UTB/TSC.  The  project  involves  the  preparation  of  practicing,  bilingual  teachers  located 
in  8 rural  sites  to  become  educational  diagnosticians,  assessing  students  with  possible  disabilities. 
Instruction  is  accomplished  through  a combination  of  available  resources,  i.e.,  multimedia  modules 
in  synchronous  format  with  videoconferencing,  CD-ROMs,  email,  chat  rooms  and  local  mentors. 
Extensive  time  was  required  for  the  initial  videoconference  setup  between  all  nine  distal  sites. 
Although  there  remain  a variety  of  ongoing,  technology  related  problems,  evaluative  data  available 
on  completion  of  the  second  year  of  the  project  supports  the  use  of  multimedia  modules  in 
conjunction  with  videoconferencing  in  the  preparation  of  rural,  special  education  diagnosticians. 


Overview 

Use  of  the  internet  for  one-way  instruction  is  not  particularly  new,  neither  has  it  been  actively  employed  in  an 
interactive  manner  to  deliver  multiple-course  programs.  However,  it  would  seem  to  hold  promise  for  content 
delivery  in  a manner  acceptable  to,  if  not  preferred  by,  consumers  in  rural,  geographically  isolated  areas.  With  the 
advent  of  T-1  fiber-optic  lines  connecting  the  local  school  systems  throughout  South  and  South  West  Texas  in  the 
past  few  years,  the  internet  has  been  made  available  as  a vehicle  for  distal  instruction.  The  project  to  be  described 
herein  is  considered  innovative  in  that  it  is  new  to  the  geographical  area  and  combines  a variety  of  available, 
relatively  inexpensive  technological  resources  in  a manner  directly  relating  to  the  learning  styles  of  the  targeted 
population.  The  intent  of  the  project  is  the  production  of  a cohort  of  bilingual  educational  diagnosticians  in  an  area 
in  which  there  is  a serious  dearth  (Hausman,  1997). 

Project  DEED,  Distance  Education  for  Educational  Diagnosticians 

Federal  legislation  (IDEA,  1990,  1997)  mandates  the  comprehensive  individual  assessment  of  school  age  students, 
i.e,,  from  three  through  21  years  of  age,  to  establish  eligibility  for  special  services.  A variety  of  school-based 
personnel  have  been  developed  in  response  to  these  mandates,  among  them  educational  diagnostician.  The 
diagnostician’s  certificate  involves  specialty  training  on  a graduate  level  in  a wide  range  of  areas  related  to 
assessment.  IDEA  also  mandates  nondiscriminatory  evaluation  in  the  students’  native  language.  This  poses  a 
problem  in  that  of  the  3,748,167  students  enrolled  in  Texas  schools,  46.4%  were  White  (not  of  Hispanic  origin), 
36.7%  were  Hispanic,  16.9%  were  other  minorities.  Thus,  minorities  comprised  53.6%  of  the  student  population  in 
Texas  public  schools  with  Hispanics  as  the  largest  minority  group.  Yet,  only  10%  of  the  active  diagnosticians  had 
Hispanic  surnames  (and  not  all  were  bilingual).  Along  the  TexasMexico  Border  Corridor,  i.e.,  an  area  extending  50 
to  70  miles  North  of  the  border  along  the  nearly  900  miles  of  its  length,  the  proportion  of  Hispanic  youth  soars  to  a 
significant  majority  of  students,  particularly  in  ^e  Rio  Grande  Valley  (Texas  Education  Agency,  1996). 

Project  DEED  was  designed  to  connect  the  UTB/TSC  site,  via  the  internet,  with  8 distal-based  school  systems 
located  in  the  Border  Corridor  up  to  400  miles  distant.  The  cohort  consists  of  20  certified  bilingual  individuals  (2 
males  and  18  females)  currently  employed  with  the  public  school  chosen  as  a site.  Program  delivery  involves 


1362 


specially  designed  interactive  multimedia  modules  (based  on  the  QUEST  utility  package).  The  overall  program  was 
structured  with  reported  preferred  learning  styles  characteristics  of  Hispanic  students  in  mind  (e.g.,  Griggs  & Dunn, 
1995).  For  example,  videoconferencing  among  all  9 sites  during  scheduled  classes  each  week  was  selected  to  appeal 
to  the  traditional  conceptualization  of  formal  instruction,  respect  for  authority,  and  need  for  cooperative  or  peer- 
based  learning  activities.  The  QUEST-based  multimedia,  interactive  modules  were  also  included  to  fit  reported 
preferences  for  structure,  vivid  stimulus  displays,  variety  and  kinesthetic  instructional  approaches. 

The  QUEST  utilities  package  (Allen  Communication,  1998)  was  selected  as  the  basis  of  module  development  in  that 
it  permits  the  combination  of  video  clips,  various  illustration  types,  graphics  animation,  audio  overlays,  and  text,  all 
with  preprogrammed  instructions,  into  interactive  modules  designed  for  use  over  the  internet.  The  modules  consists 
of  a compilation  of  frames  containing  color-coded  text  with  specific  concepts  illustrated,  and  popup  insets  triggered 
to  key  underlined  words  or  to  provide  in-text  references.  Extensive  use  of ‘side  bars’  and  animations  has  also  found 
favor  with  the  cohort  members.  Thus  far,  the  module  format  that  has  been  received  most  positively  has  included  a 
brief  overview  of  the  focal  topic,  followed  by  simulation  activities  to  allow  learners  to  experience  a variety  of 
difficulties  that,  hypothetically,  are  experienced  by  students  with  disabilities.  In  addition,  the  content  includes 
definitions  (federal  and  state),  terminology,  student  eligibility  requirements,  categories,  history,  legislation,  and 
services  associated  with  the  specific  topical  area.  Interspersed  throughout  the  modules  will  be  case  studies  or 
scenarios  with  questions  similar  to  those  included  on  the  state  mandated,  certification  examination.  CD-ROMs 
mailed  to  the  sites  remain  the  preferred  delivery  system  for  high-security  items  such  as  test  kit  training  units. 

A variety  of  potentially  useful  videoconferencing  systems  were  reviewed.  The  CU-SeeMe  (1995)  based  ClassPoint 
and  MeetingPoint  software  packages  were  selected  due  to  the  need  for  multipoint  connections  and  as  they  were  the 
most  economical  at  the  time.  The  primary  problems  we  have  experienced  this  first  year  of  online  classes  have 
included  (a)  broaching  a regional  firewall  (surrounding  5 of  our  8 sites),  (b)  fine  tuning  the  instructor  (oncampus) 
setup  (e.g.,  software  as  well  as  hardware),  (c)  arranging  for  site-based,  site-supported  technical  staff  to  work  together 
with  the  other  schools  systems  technical  staff  members  and  (d)  interconnecting  differing  computers  and  computer 
systems  to  function  as  an  extended  network.  The  forces  of  nature,  including  the  effects  of  a hurricane  also 
contributed  challenges.  Then,  fine-tuning  intersite  connectivity  is  required  to  adjust  to  each  of  our  university 
system’s  internet  access  upgrades,  requiring  extensive,  readily  available  support  from  our  Academic  Computing 
department.  Yet,  the  student’s  grades  remain  high,  as  does  their  reported  sense  of  accomplishment. 

Throughout  the  Fall,  1 999,  semester  offering  of  the  first  online  course  (an  introduction  to  special  education),  the 
sites  used  a conference  telephone  call  and  oncampus  audio  bridge  to  supplement  the  videoconferencing.  Since 
March,  however,  our  transmission  delay  time  has  significantly  reduced  as  has  the  number  of  ‘hops’  reported 
between  connections,  and  our  audio  quality  has  improved  to  the  point  that  the  conference  telephone  line  serves  only 
as  a backup  system. 
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Abstract:  DHTML  (Dynamic  HTML)  has  the  potential  to  deliver  interactive  hypermedia  cheaply 
from  a CD,  without  the  courseware  developer  having  to  cook  up  solutions  for  all  the  cross- 
platform,  rendering  or  navigation  problems  that  the  browser  resolves.  Visual,  extensible  HTML 
editors,  and  the  talented  group  of  people  developing  extensions  for  them,  are  building  a 
development  environment  that  might  just  make  the  HTML  editor/browser  combo  a workable 
substitute  for  older  Hypermedia  authoring  environments. 


Educator’s  Dilemmas 

The  limited  budget  and  political  surroundings  of  educational  researchers  and  developers  often  require  us  to  do  our 
own  software  engineering,  or  at  least  to  know  what  we’re  asking  a contractor  to  provide.  This  paper  outlines  some  of 
the  technical  and  pedagogical  issues  involved  in  this  endeavor. 

Design  Methodology 

We  have  found  that  a sound  theoretical  background  and  curricular  analysis  are  essential  to  a worthwhile  educational 
effort.  We  started  with  a modified  implementation  of  Rand  Spiro’s  Cognitive  Flexibility  Theory  and  soon 
incorporated  elements  from  other  Constructivist  and  Situativist  learning  theories.  This  paper  will  not  attempt  to 
summarize  those  theories,  but  merely  to  point  out  that  we  have  not  simply  let  the  available  technology  shape  our 
designs. 

To  CD  or  not  to  CD 

Custom  Browsers 

Netscape  and  Microsoft  both  provide  free  customizing  kits:  NS-CCK  and  MSIE-IEAK  (modifying  a browser  may 
be  too  involved  to  be  practical,  but  often  3^  party  vendors  will  do  it  for  you).  Information  is  available  at: 
http://home.netscape.com/download/cck.html  and  http://www2.microsoft.com/windows/ieak/en/default.asp 

CD  Production 

Major  browsers  cannot  run  off  a CD;  browsers  that  are  able  to  do  so  are  usually  not  very  good  at  Javascript  or 
DHTML  support.  Therefore,  your  CD  should  include  an  option  to  install  the  browser  of  choice  on  the  learner’s 
computer.  The  many  issues  of  providing  multimedia  content  on  CD-ROM  are  nicely  summarized  at  this  Web 
address: 

http://www.phdcc.com/helpindex/cdroms.html 
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Persistent  Storage 

Cookies-  Most  browsers  make  something  like  this  available,  and  it’s  the  most  obvious  way  you  can  put  semi- 
permanent information  on  a computer  (test  scores,  student  info,  learner-selections),  but  users  also  have  the  option  of 
turning  cookies  off,  and  the  learner  might  change  computers. 

Parent  page  globals—  If  you  declare  a variable  on  a page  that  remains  open  but  opens  other  pages  in  front  of  it,  that 
variable  remains  useable  by  any  of  the  other  pages.  This  is  a valid  technique  is  some  situations,  and  we  use  it  at: 
http://www.bonfils.org/courseware/ 

Hidden  form  fields  in  frames-  If  a standard  frame  appears  beside  your  other  pages,  that  frame  can  contain  hidden 
fields  with  information  generated  by  the  learner’s  interactions.  See  the  article  at  this  Web  address: 
http://developer.netscape.com/viewsource/goodman_nocookies/goodman_nocookies.html 

Strings  appended  to  link  addresses—  Another  possible  technique,  described  in  detail  in  the  article  at: 
http://developer.netscape.com/viewsource/goodman_url_pass/goodman_url_pass.html 

CD  as  Server 
CD  speed  equivalents 

(this  information  was  gleaned  from  various  sources  and  is  not  authoritative;  the  point  is  that  even  the  slowest  CD  is 
faster  than  the  fastest  modem.) 

lx:  150  Kbytes  per  second  (about  three  times  faster  than  the  fastest  modem). 

2x:  300  KBPS 
8x:  1,200  KBPS 

1 6x:  2,200  KBPS  (comparable  with  a slow  hard  drive) 

24x:  3,600  KBPS 

Check  this  Web  address  for  details:  http://webopedia.intemet.eom/TERM/C/CD_ROM.html 

Compare  the  T1  rate:  http://www.ora.eom/reference/dictionary/terms/D/Digital_Transmission_Rate_l.htm 

Browser  Wars 
DOMinance 

The  Document  Object  Model  is  a way  to  categorize  everything  on  a web  page  as  a part  of  one  big  hierarchy- of 
objects  (like  tables  or  lines  of  text).  These  program  items  can  be  given  specific  names  and  attributes.  By  referring  to 
these  items  by  name,  and  by  responding  to  events  (like  ‘mouseover’  or  ‘dragdrop’),  a program  can  make  a web  page 
interactive  or  “dynamic.” 

For  more  about  the  DOM,  go  to:  http://webopedia.intemet.eom/TERM/D/DOM.html 
Standards?  We  don’t  need  no  stinkin’  standards! 

The  group  that  first  designed  and  enabled  the  Web  now  calls  itself  the  W3C,  and  it  noticed  that  scriptable  browsers 
used  DOMs  that  bore  only  a passing  resemblance  to  one  another.  So,  in  association  with  the  ISO,  an  official, 
detailed,  standard  DOM  has  been  prepared.  MSIE  5.0  and  Netscape  5 (as  yet  unreleased,  but  available  in 
development  form  as  Mozilla),  both  claim  to  use  “most”  of  the  W3C  DOM. 

Platform  Issues 
Rendering,  type  size 

A difference  in  the  way  Windows  and  Macintosh  machines  draw  text  on  the  screen  often  results  in  Macs  displaying 
type  smaller  than  Windows  machines. 

Debugging  the  browser 

Once  a company  feels  it’s  Windows  version  is  ready  for  release,  it  seems  that,  whatever  shape  the  current  Mac 
software  is  in,  the  Mac  version  goes  out  the  door  too.  We  have  found  that  some  functions  that  are  fully  implemented 
in  the  Windows  version  of  a browser  are  not  quite  as  well  handled  in  the  Mac  version  of  that  same  browser. 

Browser  Plug-ins:  Authorware,  Shockwave,  VRML,  etc. 

At  the  student’s  end,  most  major  multimedia  producers  provide  extensions  to  the  browser  that  allow  the  student  to 
view  your  web  page’s  content  from  within  their  browser.  However,  you  must  be  sure  your  server  (not  just  their 
browser)  can  handle  the  files  that  contain  the  multimedia  content. 
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Abstract:  Learning  objects  are  rapidly  finding  a place  among  the  standard  teaching  tools 
upon  which  educators  depend.  Although  they  have  the  potential  to  enhance  student  learning, 
this  potential  may  not  be  realized  unless  standards  are  developed  to  communicate 
information  about  the  learning  object  and  its  educational  role.  In  this  paper,  we  will  discuss  a 
way  that  the  educational  role  of  learning  objects  could  be  communicated  to  educators  who 
are  considering  its  use  in  instruction.  We  propose  a format  that  describes  four  key  features  of 
the  learning  objects  relationship  to  the  instructional  context.  The  four  categories  of 
information  are  entitled  concept,  context,  instruction  and  activity.  Including  this  instructional 
information  will  facilitate  the  incorporation  of  learning  objects  into  instructional  contexts. 


The  IEEE  Learning  Object  Metadata  Working  Group  (1999)  defines  a learning  object  as  any  entity,  digital  or 
non-digital,  that  can  be  used,  re-used  or  referenced  during  technology  supported  learning.  This  definition  is 
broad,  demonstrating  that  learning  objects  still  tend  to  be  ill  defined  and  lack  standards  (Cox,  1999).  For  the 
purposes  of  this  paper  we  will  be  defining  them  as  digital  media,  designed  to  be  used  by  educators  and  learners 
to  achieve  particular  learning  outcomes.  Part  of  the  reason  for  the  increased  interest  in  learning  objects  is  that 
greater  numbers  of  people  are  able  to  create  them,  as  applications  that  were  once  the  sole  domain  of 
multimedia  experts  become  more  accessible.  While,  on  the  one  hand,  the  proliferation  of  learning  objects  is  a 
positive  development  for  education,  on  the  other,  the  existence  of  large  numbers  of  learning  objects,  created 
many  different  people  using  various  development  tools  may  result  in  mass  confusion.  Finding,  assessing,  and 
using  a learning  object  may  become  so  difficult  and  time  consuming  that  few  educators  would  see  the  value  in 
doing  so.  The  development  of  learning  objects  needs  to  be  accompanied  by  a standard  way  of  communicating 
information  about  them  (metadata)  so  that  they  can  be  easily  found,  assessed  and  used.  Part  of  what  needs  to 
be  communicated  needs  to  refer  specifically  to  the  ways  in  which  the  learning  object  can  be  incorporated  into 
instructional  contexts. 

An  example  of  a learning  object  could  be  "Strike  Lines",  a fundamental  concept  in  the  discipline  of  geology 
required  to  fully  understand  geology  maps,  mining  and  oil  geology.  The  learning  object  would  include  a 
description  of  the  significance  of  strike  lines,  put  them  in  context  relative  to  rocks,  geology  maps,  contours  and 
other  related  concepts  or  learning  objects. 

The  instructional  context 

All  educational  media,  digital  or  not,  is  best  judged  upon  its  ability  to  support  student  learning.  For  that 
reason,  it  is  necessary  to  embed  any  discussion  of  learning  objects  within  the  language  and  conceptual 
framework  of  current  learning  theory.  Constructivism  (Cobb,  1994;  von  Glasersfeld,  1989)  has  been  the  most 
influential  theory  of  learning  in  recent  years.  In  addition  to  having  been  prominent  in  educational  literature,  it 


has  become  the  standard  perspective  on  learning  put  forth  in  many  curriculum  documents.  Constructivist 
views  on  learning  recognize  that  students  are  active  constructors  of  their  own  knowledge.  Students  enter  a 
learning  situation  with  prior  knowledge.  It  is  upon  this  prior  knowledge  that  new  constructs  are  acquired  by 
the  learner.  Sometimes  the  construction  process  is  one  of  simply  relating  new  constructs  to  existing  ones, 
while  at  other  times  acquiring  new  constructs  requires  that  exisiting  ones  be  altered  or  replaced.  Recognizing 
and  facilitating  the  construction  process  is  the  work  of  the  teacher. 

When  a learning  object  is  used  in  the  classroom  it  is  important  that  the  idea  or  concept  that  the  learning  object 
is  intended  to  support  be  seen  in  relation  to  other  related  concepts.  This  provides  a teacher  with  the  conceptual 
landscape  in  which  the  learning  object  is  located.  Knowing  the  conceptual  landscape  helps  the  teacher 
appreciate  the  context  in  which  the  instruction  takes  place  and  the  sort  of  linkages  students  need  to  make. 
Planning  instruction  requires  this  appreciation  be  used  to  in  the  development  of  instructional  strategies  aimed 
help  students  achieve  the  desired  cognitive  goals.  Even  when  a good  instructional  plan  is  created  and  enacted, 
students  frequently  have  difficululty  constructing  new  understandings,  especially  when  they  conflict  with  their 
prior  understandings.  For  this  reason,  learning  activities  for  practice  and  follow-up  must  be  carefully 
developed  and  administered. 


Communicating  the  Potential 

We  suggest  that  measures  be  taken  at  the  time  when  learning  objects  are  produced  to  ensure  that  they  are 
easily  used  in  instruction.  Part  of  the  information  given  should  refer  to  the  instructional  role  of  the  learning 
object,  taking  into  account  many  of  the  issues  highlighted  by  a constructivist  perspective  on  learning. 
Specifically,  we  propose  that  four  categories  of  information  accompany  learning  objects: 

1 . concept  - a description  of  what  the  student  will  be  learning  from  the  learning  object. 

2.  context  - information  on  expected  prior  knowledge  and  links  to  other  concepts  and/or  learning 
objects.  The  context  may  also  describe  applications  of  the  concept. 

3.  instruction  - details  the  way(s)  that  the  learning  object  could  be  used  in  instruction. 

4.  activity  - describes  or  includes  activities  in  which  a student  could  use  the  learning  object  to 
practice  and  perfect  the  concept  as  well  as  to  assess  their  progress 

On  the  basis  of  this  information,  we  believe  an  educator  would  be  able  to  make  a decision  about  the  ways  in 
which  any  particular  learning  object  could  be  incorporated  into  instruction.  This  information  would  support 
the  usage  of  the  the  learning  object  for  concept  introduction,  practice,  and/or  review.  In  addition  to  being 
informed  of  the  educational  role  of  a particular  learning  object,  educators  will  be  able  to  consider  the 
relationships  between  various  learning  objects  available  for  instruction.  Conceivably,  a web  of  learning  objects 
could  be  designed  to  support  students  as  they  learn  complex,  interrelated  concepts. 
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My  presentation  is  about  supervising  students  on  their  terms  of  probation  abroad. 
For  this  aim  I have  developed  several  years  ago  an  internet  site  in  HTML. 

We  have  tested  this  site  in  co-operating  with  our  students  abroad. 

To  improve  this  application  I have  developed  a new  tool  in  Lotus  Notes/Leaming 
Space. 

During  the  presentation  I want  to  show  and  discuss  the  results  and  our  experiences. 
With  this  presentation  I hope  to  achieve  the  following  aims: 

1 . To  show  how  useful  the  Internet  can  be  in  education  particularly  to  supervise 
the  students  on  their  terms  of  probation. 

2.  To  get  feedback  and  critics  on  our  product. 

3.  To  establish  contacts  with  other  colleagues  in  case  of  this  subject. 


Today’s  Situation 

- All  our  students  abroad  have  to  send  their  reports  to  the  supervisors. 

- Nowadays  they  use  the  postal  service  or  send  a fax.  Sometimes  the  documents 
Don’t  arrive  at  the  workplacementoffice. 

- When  they  are  in  trouble,  they  use  the  telephone  for  the  communication  with 
their  supervisor.  This  method  is  very  expensive. 

Vision  Statement 

Students  and  supervisors  need  a good  tool  for  their  communication. 

Using  the  Internet  is  the  solution. 

Goal 

To  create  a possibility  for  the  communication  between  students  abroad  and  their 
supervisors  at  the  university. 

The  solution 

- First  we  have  created  a website  with  all  the  possibilities  available  for  the  students 
and  their  supervisors. 

- Nowadays  we  are  working  with  Lotus  Notes/Leaming  Space 
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Perspective 

In  a time  when  modem  culture  is  linked  by  telecommunications  technologies  in  ways  never  before 
possible,  people  transmit  and  process  information,  conduct  business,  collaborate  in  research,  and  engage 
in 

social  and  political  discourse  in  dramatic  new  ways  (Kapor,  1998,  Wriston,  1996).  Amidst  this  context, 
many  educators  search  for  a cost-effective  way  to  develop  a technological  infrastmcture  that  can 
effectively  serve  the  needs  of  their  students.  When  their  searches  yield  promising  results,  educators  who 
attempt  to  initiate  such  infrastmctures  can  become  stifled  by  bureaucratic  norms  and  historical  values,  or 
thwarted  by  political  and  economic  crosscurrents  within  their  institutions.  Furthermore,  the 
implementation  of  new  technologies  within  higher  education  institutions— environments  almost 
perpetually  preoccupied  with  finances  and  funding— must  be  done  with  utmost  economic  efficiency.  The 
development  of  new  technologies  must  also  support  the  diverse  needs  of  student  populations  whose 
characteristics  are  continually  transforming  beyond  those  of  the  traditional  student  (Dalziel,  1997).  For 
example,  many  contemporary  students  need  flexible  scheduling  of  classes  and  alternative  ways  to  access 
and  distribute  information  because  they  work  and  attend  school.  For  these  students,  distance  education  is 
often  heralded  as  an  excellent  way  to  help  them  obtain  the  benefits  of  higher  education  while  balancing 
their  non- traditional  roles  and  responsibilities.  Yet,  ironically,  the  potential  benefits  offered  by  something 
as  non-traditional  and  futuristic  as  distance  education  classes  may  be  undermined  by  their  incorporation  of 
pedagogical  techniques  and  traditions  that  are  not  consistent  with  the  changing  educational  needs  of 
society.  If  distance  education  classes  merely  entail  the  uploading  and  distribution  of  didactic  lectures, 
quizzes  and  paper  assignments,  then  they  may  be  falling  far  short  of  their  potential  to  serve  contemporary 
students  who  want — and  need — to  become  successful  members  of  modem  society.  Today's  highly 
interactive  society  demands  that  its  citizens  think  critically,  problem-sol ve,  and  see  outside  traditional 
paradigmatic  structures  (Cox,  1997,  Frazier,  1997,  and  Boetcher,  1997),  and  educators  should  be 
preparing  students  in  a manner  consistent  with  the  needs  and  demands  of  their  society.  Distance  education 
in  particular  needs  to  provide  the  types  and  extent  of  interactivity  consistent  with  the  new  global  society. 
But  the  question  is:  Can  distance  education  courses  actually  be  aligned  to  pedagogical  practices  which 
serve  the  needs  of  a rapidly-evolving,  technology-assisted,  information-driven  society? 


Purpose  of  the  Study 

The  purpose  of  this  case  study  was  to  analyze  a distance  learning  program  designed  to  align  technological 
methods  and  pedagogical  practices  to  better  meet  the  needs  of  its  students.  This  study  focuses  on  a block 
of  secondary  teacher  education  methods  courses  that  employed  web-based  distance  education.  The 
methods  courses  were  taught  at  a mid-size  state  university  to  sixteen  post-baccalaureate  students  who  were 
teaching  in  grades  7-12  on  emergency  certification  during  the  1998-99  school  year.  Strategies 
implemented  to  support  the  program’s  goal  were: 

Professors  created  problem-posing  curriculum  that  required  students  to  assimilate  information 
presented  in  their  methods  courses,  synthesize  it,  and  apply  it  within  the  context  of 
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their  own  school  or  classroom. 

Professors  created  public  cyber  spaces  for  students  to  publish  their  work  and  to  react  to  the 
work  of  their  peers. 

Professors  created  public  cyber  spaces  for  students  to  debate,  solve  problems,  and  dialogue 
about  current  issues  and  concepts  associated  with  the  courses  and  their  work. 

Professors’  modeling  of  meaningfully  integrated  technology  in  order  to  support  a problem-posing 
curriculum. 

Students’  learning  of  requisite  technological  skills  for  engaging  in  class  activities. 

Students’  reflection  on  their  learning  through  journals,  conversations,  and  electronic 
portfolios. 

Methodology 

Informants  for  this  study  included  sixteen  post-baccalaureate  university  students  teaching  on 
emergency  certification,  two  professors,  and  one  graduate  student/instructor.  The  students'  electronically 
posted  papers,  projects,  field  notes,  response  logs,  and  forum  responses  were  analyzed  qualitatively 
looking  for  evidence  of  theory-to-practice  connections  reflective  of  the  changing  pedagogy.  Course  syllabi 
and  instructors'  field  notes  were  analyzed  qualitatively  for  evidence  of  professors’  reflection  about  their 
own  theory-to-practice  . Sixteen  end-of-course  evaluations  were  examined  for  student 
satisfaction  in  meeting  their  learning  needs  according  to  the  traditional  university  evaluation  rubric. 

Findings 

Formative  evaluations  made  at  the  close  of  Fall  1998  indicated  student,  instructor,  and  supervisor 
satisfaction  with  the  methods'  block.  Students  reported  a high  correlation  between  material  studied  and  its 
relevance  to  their  classroom  practice.  Students,  professors  and  supervisors  also  addressed  areas  of  concern 
and  recommendations  for  improvement.  The  professors  for  the  methods-on-line  class  published  web  pages 
to  deliver  information  about  the  structure  and  content  of  the  course,  but  the  threaded  message  boards 
became  the  forum  for  students  to  interact  with  the  class  community,  professors,  and  other  students  in  the 
class.  Threaded  message  boards  served  multiple  instructional  purposes  in  the  Spring  1 999  component  of 
the  block.  They  provided  a mechanism  for  the  professors  and  students  to: 

Deliver  information  about  the  structure  and  content  of  lessons, 

Post  expository  assignments  discussing  the  application  of  theory  to  classroom  practice, 

Report  primary  data  collected, 

Post  lesson  plans  and  construct  instructional  strategies, 

Summarize,  evaluate,  and  reflect  upon  assigned  readings  from  texts  and  journals, 

Present  persuasive  arguments  on  educational  issues, 

Analyze  educational  issues  and  current  events,  and 
Simulate  classroom  practices  through  role  play. 

In  a typical  lesson,  a professor  posted: 

Information  about  the  dates  and  times  the  assignments  were  due  for  any  particular  class 
meeting. 

Questions  which  guided  student  inquiry  into  the  lesson, 

Introductory  information  about  the  topic  and  supporting  links  to  sites  on  the  web,  which 
provided  a basis  for  linking  student  prior  knowledge  to  the  topic  to  be  studied,  and 
Threaded  message  boards  for  students  to  respond  to  the  readings,  discuss  their  responses  with 
each  other,  or  post  other  instructional  activities  products. 

Recommendations 

Educators  must  place  learning  within  a problem-solving  framework  that  compels  students  to 
synthesize  the  information  acquired  in  their  courses  and  apply  it  to  real  world  scenarios. 

Educators  must  facilitate  virtual  communities  of  learners  who  work  in  small,  collaborative 
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groups  to  achieve  a common  goal. 

Educators  must  create  public  cyber-spaces  for  students  to  debate,  solve  problems,  and 
dialogue  about  current  issues  and  concepts  associated  with  their  courses  and  their  work 
environment. 

Educators  must  model  how  to  meaningfully  integrate  technology  to  support  a problem-posing 
curriculum. 

Students  should  learn  technological  skills  as  they  are  engaging  in  class  activities. 

Student  should  have  opportunities  to  reflect  about  their  learning  through  authentic  means  such 
as  journals,  conversations,  and  electronic  portfolios. 
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Abstract:  We  are  developing  an  online  interactive  Web-Based  instructional  module,  which 
provides  our  user  community  with  basic  tools  to  understand  microarray  technology. 
Additionally,  providing  basic  theories  associated  with  microarray  technology  may  allow  the 
user  community  the  necessary  means  to  appropriately  and  innovatively  apply  the 
technology.  We  describe  a longitudinal  learning  environment  designed  to  educate  the 
novice  undergraduate  and  practiced  researcher.  Preliminary  studies  of  senior  level 
undergraduate  students  suggests  that  such  a system  will  enhance  the  learning  experience  of 
current  technologies  as  they  move  from  development  in  the  lab  to  useful  research  tools.  This 
case  study,  exemplifies  a novel  use  of  the  web  to  expeditiously  disseminate  information  and 
training  in  emerging  technologies. 
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Graphical  Representations  of  Convergence  in  Web-based 
Educational  Computer  Conferencing:  A Prototype 


L.  Hopperton,  R.  Rupp,  M.  Campos,  B.  Fisher,  L.  Harasim,  M.  Luk,  C.  Ostler,  & C.  Xin 
Virtual-U,  Simon  Fraser  University 

Introduction 

Graphical  representations  of  the  structure  of  information  are  not  uncommon  on  large. 
World  Wide  Web  sites.  But  because  of  the  continued  addition  of  notes  in  evolving 
computer  conferencing,  coupled  with  the  labour  intensive  activity  of  continually  creating 
and  updating  these  navigational,  conceptual  maps,  these  site  maps  have  not  been  adapted 
to  evolving,  computer  supported  discussions.  These  systems  actively  support  idea 
divergence  but  are  limited  in  providing  hypertextual  tools  that  support  idea  convergence 
and  structuring  (Harasim,  1 990).  This  limits  their  ability  to  scaffold  the  development  of 
mental  models  of  a domain  of  inquiry  based  upon  progressive  reconsideration  and 
synthesis  of  content. 

The  research  and  development  team  of  the  Virtual-U  project  has  developed  a prototype  of 
a graphical  representation  of  convergence,  VU-ROC,  for  online  learning  environments. 

The  prototype  shows  promising  results  for  the  construction  of  an  active,  message  mapping 
system  for  online  learning.  It  is  designed  to  be  an  external  support  tool  that  will  encourage 
reflective  thinking  by  illustrating  the  structures  of  existing  knowledge  and  the  dominant 
hubs  and  conceptual  triggers  over  time  within  the  corpus  of  information.  While 
computationally  based  and  dependent  upon  user-specified  categories,  it  is  relatively  user 
passive,  requiring  only  reflection  upon  the  nature  of  a note  after  it  is  written.  It  is  intended 
to  scaffold  the  developing  mental  models  of  students  by  mapping  the  relationships 
between  notes  as  they  are  entered  thereby  illustrating  and  stimulating  convergent  thinking. 

The  VU-ROC  model  used  27  actual  notes  from  a college  seminar  on  two  poems  by  Blake, 
“The  Lamb”  and  “The  Tyger”,  which  present  omplementary  visions  of  the  nature  of  God 
and  creation.  Figure  1 presents  a lull  message  map  of  the  Blake  seminar.  By  clicking  on 
any  node  within  the  message  map,  the  lull  note  text  is  brought  onto  the  screen  along  with 
a small  representation  of  the  notes  to  which  it  directly  refers.  This  allows  students  to 
browse  through  hubs  of  information  in  a simple,  intuitive  maimer. 

The  legend  for  notes  appears  on  the  left-hand  side  of  the  screen.  This  acts  as  a reminder  to 
students  about  the  nature  of  any  given  entry.  In  the  visual  window,  a selection  box 
indicates  the  note  that  a student  has  selected  by  pointing  and  clicking.  The  visual 
relationship  of  this  note  to  its  referents  is  highlighted  using  darker  lines.  Unread  notes  are 
solid  colours  while  read  notes  are  in  outlined  colours.  The  text  of  the  selected  note 
appears  at  the  bottom  of  the  screen.  Tabs  allow  users  to  manipulate  the  representation  in 
the  graphical  display  window. 
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Using  Groc 


Following  the  writing  of  a note,  when  the  student  is  ready  to  post  their  entry,  they  are 
required  to  specify  the  note  categories  using  a simple  drop-down  box  in  which  students 
point  and  click  their  choices.  Drop-down  boxes  contain  optional  reminders  of  literary 
periods  and  note  types  that  are  pre-set  by  the  course  instructor. 


iSTopic:  Songs  of  Innocence  ond  Expenei^ 


0123456730  tOll  12  IS  10  ^3  2^  23  20  27  1:3  ^ | I'OpOOtW  | Sl8n^"^ 


m m 
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j Intro 
I •Conti  ibulifin 
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Dace:  10/07/1999  05.1)0:00 

, S«ii|ect:  Reconciino  "The  Lamb"  and  "The  Tyoer" 
Keywords:  The  Lama,  The  Tiger 
Author  Ame  Student 


Sequence  (t.  14 
cateoory:  4S^hesis 


Repyiothte 


bi  Reply  UK  i Click  to  see  the  Ust  of  messages 


Responses:  i CflcK  to  see  the  list  of  messages 


_n  these  two  seemingly  simple  poems,  we  are  given  opposing  viewpoints  on  the  same  creator.  In  The  Lamb*  he  le  the  dMne  protector 
t MiereaelnTheTiger  he  lethe  creator  of  fear  and  danger.  Each  poem  le  complete  and  euccesefljl  but  the  real  Image  of  the  nature  ofOod 
lues  In  the  amalgamation  of  the  two.  Since  Blake  Intentlonallywrotetheee  as  companion  pieces  from  THE  SONGS  OF  INNOCENCE  and  THE 
? BONOS  OF  EXPERIENCE,  the  whole  nature  ofOod,  as  Blake  presents  It,  Is  In  neither  "The  Lamb*  or  "The  Tiger,  but  In  the  combination  of  the 
> Iwo, 


Figure  1 : Full  message  map  of  the  Blake  seminar 


Discussion 

Current  development  is  focused  on  the  full  semester  conference,  and  this  will  be  followed 
with  full  field  tests.  It  is  anticipated  that  a beta  test  version  of  the  prototype  will  be  ready 
for  these  field  tests  by  200 1 . 

The  application  of  computer  generated,  graphical  information  management  and  structuring 
tools  are  positive  and  useful  additions  to  educational  computer  conferencing  systems. 
These  tools  facilitate  the  process  of  progressive  reconsideration  of  course-based  ideas, 
and,  potentially,  support  the  inclusion  of  external  resources  such  as  web  pages. 

Knowledge  synthesis,  as  demonstrated  through  the  linking  of  conference  entries,  can  be 
brought  to  bear  upon  multiple  domains  of  inquiry,  and  so  requires  successive  re- 
juxtaposition, re-examination,  and  re-interpretation  of  ideas  by  students. 
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The  full  text  of  this  paper  can  be  found  at 

http://virtual-u.cs.sfu.ca/vuweb/Vuenglish/papers/papers.html 
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Abstract:  The  paper  discusses  the  STYLE  (Situated  Technology  Yielding  Learning  Enhancement) 
program  where  unique  on-the-job  faculty  development  in  technology  skills,  integrating  technology 
into  post-secondary  classes,  and  promoting  learning-centered  classrooms  in  one  logical,  easy  to 
implement  and  assess  innovation.  Currently,  STYLE  is  a FIPSE-sponsored  program  at  Columbia 
College  Chicago.  We  discuss  qualitative  and  quantitative  evidence  of  program  success.  The  model 
can  be  adapted  across  a continuum  of  existing  technology  configurations  that  exist  in  post- 
secondary  education  settings  today. 


The  Teaching  with  STYLE  Model 

At  the  heart  of  STYLE  is  a faculty  developed  interactive,  non-linear  collaborative  project  to  be  created  by  two 
different  classes  according  to  a flexible  plan  created  by  the  faculty  involved  in  the  paired  collaboration  aimed  at  the 
transformation  of  the  learning  environment  (Tinto,  1998.)  Instead  of  following  the  familiar  form  of  learning 
communities  wherein  the  same  students  take  two  concurrent  classes  taught  by  different  teachers  who  communicate 
about  their  learning  objectives  and  teaching  in  general,  STYLE  is  a loosely  coupled  simulation  of  the  modem 
workplace  (Iverson  & Torriera,  1999.)  The  learning  community  formed  in  STYLE  collaborations  is  made  up  of  a 
pair  of  classes  with  different  students  and  teachers.  One  is  a traditional  class,  such  as  a liberal  arts  class,  where 
lecture  is  a prevalent  teaching  method,  and  there  is  little,  if  any  integration  of  technology  into  the  classes.  The  other 
is  a technology  -oriented  class,  such  as  New  Media  projects,  or  Multimedia  Producer  that  are  "technology  heavy" 
and  "content  light."  The  technology  teacher  serves  as  a leader  in  using  tech  to  collaborate,  and  as  a research  tool  in 
the  classrooms.  Often  the  content  teacher  has  many  ideas  about  what  they  want  to  do  with  tech  in  the  classroom, 
they  simply  lack  experience  and  expertise.  Experience  has  shown  that  the  tech  teacher  comes  to  the  collaboration 
believing  they  are  using  tech  fully  in  their  teaching,  but  they  may  discover  from  their  teaching  collaborator  new 
ways  tech  facilitates  learning  and  communication  beyond  its  uses  as  a production  tool  (Iverson,  1999.) 

STYLE  synthesizes  several  simple  but  important  ideas  about  learning,  pedagogy,  and  technology  into  an  easy  to 
implement  model  that  introduces  project-based  collaboration  across  departments  and  disciplines.  A benevolent 
"Trojan  Horse"  approach  is  used  to  get  faculty  to  use  various  digital  tools  and  technologies  by  focusing  on  teaching 
tasks,  not  skills  learning  per  se.  The  model  improves  learning,  increases  faculty  digital  skills,  and  helps  integrate 
technology  into  the  fabric  of  teaching,  avoiding  the  less  effective  "bolt  on"  approach  to  technological  integration  in 
post-secondary  settings. 

STYLE  is  based  on  an  emergent  understanding  of  human  learning  (Schank,  1992;  1994)  and  of  what  works 
(Jonnessen,  et.  al,  1999),  that  is  "praxis"  (theory  in  practice)  in  higher  education.  Its  simple  premises  are: 

1 . Learning  is  more  meaningful  if  learners  are  engaged.  Active  learning  is  more  engaging  than  passive  learning. 
Time  on  task  is  a potent  predictor  of  achievement. 
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2.  Learning  is  a social  activity  for  students  and  faculty  alike.  Communication,  collaboration  and  teamwork  provide 
an  effective  setting  for  teaching  and  learning. 

3.  Situated  learning  about  any  subject,  including  learning  to  use  technology,  is  more  effective  for  faculty  as  well  as 
students. 

4.  Technology  enables  us  to  provide  necessary  passive  learning  (formerly  transmitted  via  lecture)  outside  the 
classroom  (via  on-line  technologies)  and  to  use  in-class,  face-to-face  for  communication  and  active  learning 
activities.  Witnessing  knowledge  in  action,  and  working  as  knowledge  organizers,  rather  than  being  passive 
knowledge  consumers  makes  students  better  learners. 

5.  Encouraging  students  in  sustained  perspective  taking  (as  in  creating  interactive,  non-linear  versions  of  their  work) 
promotes  higher  order  thinking  and  learning,  as  well  as  fostering  the  appreciation  of  diversity  of  opinion. 

Implementation 

Teaching  with  STYLE  collaborations  retain  a set  of  objective  unique  to  the  content  class,  a set  unique  to 
the  technology  class  but  include  a shared  set  of  learning  objectives  that  involve  teamwork  and  communication 
centered  around  the  collaborative  project.  The  shared  objectives,  scope,  and  form  of  the  project  are  designed  by  the 
participating  faculty  with  input  from  participating  students.  They  vary  from  semester  to  semester.  Each  class  also 
engages  in  a variety  of  other  assignments  related  to  and  separate  from  the  shared  projects. 

In  some  institutions  it  is  difficult  to  add  new  classes  to  the  curriculum.  The  STYLE  approach  allows  updating  of 
existing  courses  and  integrating  facilitating  technologies  into  these  classes.  It  avoids  proliferation  of  new  courses 
and  the  "bolt-on”  technology  syndrome  that  results  from  mandates  to  use  tech  in  teaching. 

The  ease  with  which  web  sites  can  be  created  using  modem  WYSIWG  HTML  editors  makes  projects  like  this 
possible  in  a range  of  institutions  that  may  not  have  high-end  technology  classes.  Through  FTP,  use  of  a project  site, 
and  other  facilitating  technologies  that  encourage  asynchronous  communication,  new  forms  of  information  and 
virtual  collaboration  are  possible.  The  inter-class  communication  requirements  of  STYLE  can  be  met  with  the  use  of 
a range  of  tools,  from  simple  and  ubiquitous  shareware  to  high-end  proprietary  packages,  as  fits  a given  setting. 
Faculty  seeking  to  implement  STYLE  can  begin  with  a shared  set  of  PowerPoint-like  presentations  and  e-mail  and 
evolve  to  the  use  of  GroupWare,  DHTML,  and  more  sophisticated  tools  as  suits  their  needs.  The  point  of  STYLE 
intervention  is  not  the  product,  but  the  process. 

The  faculty  in  the  collaborations  differ  in  their  skills  with  digital  technology.  Generally  one  faculty  is  more 
technologically  savvy  than  the  other  is.  This  is  where  the  "Trojan  Horse”  approach  comes  in.  It  is  in  the  interest  of 
both  faculty  that  they  can  communicate  easily  and  conveniently.  They  help  each  other  in  using  specific  software  and 
hardware  in  specific  settings  to  accomplish  their  shared  tasks  and  goals.  Results  of  a digital  skills  survey  given 
before  and  after  project  participation  shows  that  faculty  develop  their  skills  as  a result  of  the  project.  This  is  more 
effective  in  encouraging  the  transfer  and  retention  of  learning  than  the  pullout  workshop  model. 

There  is  flexibility  built  into  STYLE  collaborations  because  institutional  and  bureaucratic  impediments  to 
collaborations  like  this  include  scheduling  issues,  length  of  classes,  and  resource  issues,  etc.  The  ideal  collaboration 
involves  classes  that  have  overlapping  meeting  times  so  as  to  provide  for  face-to-face  meetings.  This  is  preferred  in 
every  evaluation  we  have  conducted.  However,  STYLE  can  work  even  if  the  classes  meet  on  different  days  because 
of  its  reliance  on  simple  electronic  collaboration  tools  (e-mail,  FTP  directories,  project  web  sites,  and  threaded 
discussions.) 

Results 

It  is  often  problematic,  to  compare  achievement  between  the  STYLE  program  and  traditional  classes.  In  order  to 
look  at  the  effectiveness  of  the  model  as  a method  of  improving  teaching,  we  use  the  whole  class  as  a unit  of 
analysis,  and  focus  on  changes  in  the  psycho-social  learning  environment  of  the  classroom  that  result  from 
collaborative  learning  strategies.  We  look  for  changes  in  the  class'  sense  of  cohesiveness,  cooperativeness, 
democracy,  diversity,  and  satisfaction.  These  factors  are  stable  and  robust  in  large-scale  research  studies  of 
cooperative  and  collaborative  learning  in  pre-computer  times,  and  are  correlated  with  academic  achievement  (Fraser 
& Walberg,  1984.)  Faculty  significantly  enhance  these  aspects  of  the  classroom  learning  environment  through 
instructional  grouping,  project-based  assignments,  and  an  increase  in  student  participation,  communication  and 
collaboration,  in  the  projects,  and  our  quantitative  and  qualitative  results  show  this.  This  signals  success  for  the 
teaching  with  STYLE  model. 
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The  exciting  thing  about  STYLE  collaborations  is  that  we  document  program  effects  from  anecdotal  and  open-ended 
questionnaire  data  but  we  triangulate  from  data  based  on  assessing  the  classroom  learning  climate,  and  by  assessing 
the  project  products  as  well.  Summary  data  is  viewable  at  http ://ac web. colum.edu/users/biverson/ed  media2000/. 
The  projects  themselves  are  viewable  at  http://digdesignxolum.edu/fipse. 

In  conclusion,  we  discuss  how  to  match  up  the  current  hardware  and  software  and  expertise  available  with  the  kind 
of  projects  that  can  be  best  accomplished  in  any  setting.  The  key  point  is  that  the  collaboration  learning  objectives 
and  the  five  premises  of  STYLE  listed  in  the  beginning  of  the  paper  are  most  important.  Fit  the  project-based 
collaboration  to  the  tools  at  hand.  Using  e-mail  and  threaded  discussion,  and  having  a website  or  even  a simple 
networked  and  thus  accessible  site  where  you  post  work  for  clients  to  observe  will  put  you  on  the  road  to  success. 
Faculty  using  the  "each  one,  teach  one"  method  to  work  together,  and  modeling  this  kind  of  on-going  learning  for 
students  set  the  process  in  motion.  Process  is  what  should  be  emphasized  in  adopting  STYLE  in  your  teaching,  not 
products,  either  the  technical  ones  used  for  making  work,  nor  the  outcome  project  of  the  collaboration.  Our 
evaluation  results,  pulling  from  what  participants  say,  and  what  they  do  show  the  program  is  effective  and  holds 
promise  for  promoting  active  learning,  providing  effective  on-the-job  faculty  development,  and  in  integrating  a 
range  of  technologies  as  tools  to  facilitate  learning  and  communication  into  classroom  practice. 
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Abstract:  This  paper  describes  the  reasons  to  consider  developing  a customized  media 
management  system,  along  with  the  disadvantages  of  these  systems.  A step-by-step  system  for 
designing,  developing  and  evaluating  a customized  media  management  system  is  described. 


Overview 

Most  web  based  educational  systems  develop  begin  with  a single  project  in  mind.  As  new  projects  evolve,  bits 
and  pieces  are  borrowed  from  previous  projects,  and  over  time  the  overall  system  can  become  quite  complex, 
and  difficult  to  maintain.  At  a certain  point  a critical  mass  is  reached:  either  more  resources  have  to  be  allocated 
to  maintain  the  current  way  of  doing  things,  or  a more  efficient  system  must  be  developed.  The  goal  of  this 
paper  is  to  describe  a way  of  systematically  choosing  and  developing  an  efficient  database-web  resource 
management  system  to  meet  your  specific  needs. 

A web-database  resource  management  system  offers  several  advantages  of  ‘‘hard  coded’'  HTML  pages. 
Resources  such  as  pictures,  sounds  , movies  and  teaching  cases  can  be  indexed  with  metadata  so  that  they  may 
be  easily  searched  and  reused  in  multiple  projects.  Web  based  forms  can  be  developed  so  that  textual  content 
can  be  easily  edited,  separating  the  creation  of  content  from  design  and  delivery,  allowing  authors  to  focus  on 
content,  not  technology.  Sites  can  easily  have  a consistent  look  and  feel  through  the  use  of  templates,  with 
content  being  served  up  dynamically  via  a database.  Updating  a single  template  can  then  update  an  entire  site. 
E-commerce  and  other  transactional  system  can  be  implemented.  Faculty  can  track  student  progress  and 
performance  through  a series  of  exercises.  Correspondence  between  faculty  and  students  can  be  facilitated  and 
archived  for  later  analysis.  Some  off-the-shelf  systems  offer  these  features  as  pre-designed  widgets  that  can  be 
customized  with  a minimum  of  programming,  or  a custom  system  can  be  built  from  the  ground  up.  Either 
approach  maintains  content  in  a database,  which  makes  migrating  to  new  delivery  technologies  like  DHTML  or 
XML  fairly  painless. 

On  the  other  hand,  these  database-web  systems  typically  require  a higher  level  of  technical  expertise  to  plan  and 
develop  than  HTML  only  sites.  Adding  new  features  to  these  systems  usually  requires  a programmer,  instead  of 
a web  designer.  And  when  a bug  appears  in  a templated  page,  it  typically  is  repeated  again  and  again  throughout 
a whole  site.  The  overall  system  becomes  more  complex,  with  more  opportunities  for  technical  problems.  We’ll 
look  at  how  each  of  these  factors  plays  out  in  choosing  or  creating  your  own  system. 


Developing  a Media  Management  System 

The  process  of  choosing  and  designing  your  own  resource  management  system  involves  these  steps: 

1.  Needs  analysis  - Clearly  define  the  current  and  projected  production  and  delivery  issues. 

2.  Review  available  tools  - Given  your  needs  analysis,  review  options.  Narrow  the  field  to  three  or 
four  that  best  match  your  needs. 

3.  Test  drive  - Get  trial  copies  of  the  “final  four”  products  identified  in  step  two  and  build  a typical 
application.  Develop  a formal  evaluation  checklist  to  compare  the  outcomes  of  your  test  drive. 

4.  Review  outcomes  - Your  evaluation  may  have  unexpected  results.  Does  adopting  a new 
technology  change  your  operations  plan? 
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5.  Develop  Budget  and  Implementation  Plan  - Project  your  costs  and  get  approval,  if  necessary. 
Develop  a stepped  implementation  plan  with  target  dates  for  each  part  of  the  transition. 

6.  Evaluate  the  outcomes  - Did  you  accomplish  what  you  expected? 

Let’s  next  look  at  each  one  of  these  steps  in  detail  and  examine  what  can  be  involved  in  each. 

Needs  Analysis 

Each  institution  will  have  different  needs,  but  a common  one  is  the  ability  to  reuse  resources  such  as  images, 
sound  clips,  chunks  of  text,  teaching  cases,  and  movies,  in  different  ways  on  different  projects.  We  found 
ourselves  with  thousands  of  images  that  had  the  potential  of  being  used  across  many  disciplines — if  anybody 
could  find  what  they  needed.  As  a result,  our  top  priority  became  a sharable  image/sound/movie  database.  We 
also  had  thousands  of  practice  quiz  questions,  which  were  difficult  to  update  in  HTML.  So  our  second  priority 
was  a question  database  with  easy  access  by  faculty  for  editing.  You  may  have  similar  needs,  or  ones  that  are 
completely  different. 

Review  Available  Tools 

There  are  hundreds  of  tools  currently  on  the  market,  from  simple  image  databases  to  turnkey  management 
systems  from  the  likes  of  Oracle  and  IBM,  to  specialized  course  management  systems.  Some  of  these  systems 
can  be  learned  in  a day,  while  others  require  programmers  earning  $80K  a year.  With  your  needs  analysis  in 
hand,  choose  the  8-10  more  promising  tools  for  your  project.  Read  everything  you  can  find  in  print  and  on  the 
web  and  talk  to  people  using  those  products.  Monitor  the  support  newsgroups  for  the  tools.  Create  a checklist  of 
your  desired  features  and  narrow  the  field  to  the  best  matched  three  or  four. 

Test  Drive 

With  trial  copies,  build  the  same  application  in  your  “final  four”,  one  that  you  would  actually  use.  We  built  an 
image  database  and  an  evaluation  system  using  multiple  choice  questions.  Can  it  handle  the  required  security? 
Can  it  handle  the  number  of  users  you  project?  Is  setting  up  a new  project  a breeze  or  a nightmare?  Are  there 
good  support  options  available?  Is  there  a clear  migration  path  to  more  powerful  system  as  your  needs  increase? 
Use  these  and  your  own  questions  to  formally  compare  each  of  the  products.  Identify  the  best  choice(s). 

Review  Outcomes 

So  you  think  you  have  chosen  the  best  tool  for  your  situation.  Now  you  should  take  some  time  to  mull  over  the 
possible  outcomes.  Will  existing  staff  be  able  to  meet  the  new  challenges?  What  training  will  be  necessary? 
How  will  your  clients  react?  Is  it  worth  the  investment  given  the  projected  outcomes?  Take  time  to  talk  again  to 
people  using  the  product  and  find  out  their  likes  and  gripes.  In  short,  perform  a reality  check  before  moving 
ahead. 

Budget  and  Implementation 

Draw  up  a formal  budget  that  includes  not  only  software  tool  cost,  but  also  staff,  training,  support,  and 
hardware.  Develop  a-step-by  step  plan  on  how  you  will  migrate  to  the  new  system.  What  components  will  be 
required  first,  second  and  third?  Include  adequate  time  for  testing  of  the  new  system  by  staff  and  clients  before 
rollout  of  each  phase.  Take  you  initial  time  estimates  and  double  them. 

Evaluate  Outcomes 

Take  time  to  review  your  accomplishments  after  six  months  and  a year.  Has  the  process  evolved  as  you 
expected?  Have  you  met  the  goals  identified  in  your  initial  needs  analysis?  Is  your  team  happy?  Are  your  clients 
happy? 

After  this  stage  the  next  step  is. ...  do  it  all  over  again!  With  the  pace  of  technological  change  you  may  actually 
be  faced  with  a whole  new  set  of  problems  and  opportunities  that  require  going  through  this  process  again.  But 
using  the  systematic  approach  described  above,  good  planning,  and  a little  bit  of  luck,  you  will  have  chosen  a 
system  that  will  grow  with  you  into  the  future. 
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Abstract  This  paper  describes  and  critiques  current  user-designer  feedback  channels,  and 
suggests  why  and  how  these  could  be  augmented,  particularly  in  educational  settings.  It  is 
argued  that  current  feedback  mechanisms  exclude  the  majority  of  users.  Although  academics 
remain  sceptical,  increasing  numbers  of  software  designers  in  New  Zealand  are  recognising 
lost  opportunities  for  client  support  as  well  as  customization  and  improvement  of  services, 
and  are  pursuing  new  paradigms  for  designer-user  communication. 


Introduction 

Mechanistic* sender-receiver’  models  of  the  communication  process  provide  a superficially  convincing 
representation  of  the  interaction  between  computer  users  and  their  PCs,  with  the  computer  application  seeming 
to  show  communicative  behaviour  similar  to  that  of  a human  partner. 

While  acknowledging  this  seemingly  ‘social’  interaction  with  the  technology,  however,  it  is  easy  to 
undervalue  the  real  (if  asynchronous)  communication  relationship  that  exists  between  software  designers  and 
their  users.  The  design  process  involves,  in  effect,  the  construction  of  a delayed  ‘conversation’  between  designer 
and  user,  where  the  users’  needs  are  anticipated  and  predicted,  and  appropriate  responses  are  provided  to  meet 
them.  However,  in  this  relationship,  only  the  designer/sender  is  actively  aware  that  communication  (albeit 
indirect)  is  taking  place,  while  the  user/reciever  participates  in  the  process  without  having  to  recognize  it  as 
communication  at  all. 

The  result  is  communication  that  is  one-sided.  Despite  ‘Know  thy  user’  being  accepted  as  axiomatic  by 
modem  developers,  occasions  for  actually  getting  feedback  from  users  are  limited.  For  many  developers,  the 
only  such  occasions  would  occur  during  usability  testing,  ending  shortly  after  system  implementation  (Pausch, 
Conway  & DeLine  1992,  Petersen  C,  1996;  Saltzman  & Rosenthal  1994). 


Adequacy  of  Existing  Feedback  Channels 

One  needs  to  distinguish,  at  this  point,  between  Product  Support  Desks  that  exist  for  users  who  simply  need  help 
operating  a piece  of  software,  and  channels  that  allow  for  queries  about,  commentary  on  and  criticism  of  various 
features  of  a software  product.  One  is  tempted  to  assume  that,  for  users  who  want  to  engage  in  the  latter, 
emailing  or  telephoning  the  developers  would  be  the  obvious  option.  Consequently,  a survey  to  assess  the 
usefulness  of  existing  user-designer  feedback  channels  was  conducted  at  UNITEC  Institute  of  Technology,  four 
Auckland  secondary  schools,  and  at  the  Chalksoft  educational  website.  A summary  of  the  results  follows 
(Detailed  results  can  be  found  at  www.chalksoft.com/questionnaireresults). 

Overall,  88%  of  respondents  had  never  contacted  the  developers  of  the  software  they  used.  Reasons  for 
not  contacting  developers  were  given  as  follows  (respondents  were  allowed  to  choose  more  than  one  reason):  It 
never  occurred  to  me  (72%);  It  would  be  too  much  trouble  to  try  to  get  hold  of  the  developers  (52%);  I have 
nothing  I would  like  to  tell  the  developers  of  the  software  I use  (51%);  Software  developers  don’t  have  time  for 


1381 


users  like  me  (26%);  It  would  be  too  much  hassle  to  try  to  explain  myself  (9%).  In  addition,  fifty-eight  percent 
of  the  respondents  who  had  never  contacted  software  developers  did  not  know  how  to  go  about  doing  so. 

These  results  suggest  that,  at  least  in  the  limited  educational  context  of  this  survey,  the  majority  of  users 
are  unlikely  to  take  advantage  of  (or  in  some  cases  even  know  about)  the  channels  currently  available  to  them  for 
communicating  with  the  designers  of  the  software  they  use,  the  main  reasons  being  that  the  possibility  of,  (and  to 
a lesser  extent  the  need  for),  such  a ’conversation*  does  not  occur  to  them,  and  that  finding  appropriate  channels 
would  be  simply  too  much  trouble. 


Current  Channels  for  User  Feedback 

Recent  demonstrations  of  interest  in  ongoing  user  feedback  by  major  companies  include  dialogue  boxes  which 
appear  when  the  application  detects  an  error  condition  (eg  Netscape,  Muzak)  and  the  provision,  within  the 
application,  of  a link  to  a company  web  page  or  a trouble-shooting  discussion  forum  (eg  Blackboard).  However, 
the  weakness  of  the  former  lies  in  that  the  user  has  no  control  over  when  feedback  is  possible.  In  addition,  there 
is  the  nigh-impossibility  of  users  being  able  to  identify  and  then  describe  accurately  in  retrospect  their  exact 
sequence  of  acts  preceding  an  error  message. 

A weakness  of  the  discussion  forum  option  lies  in  the  assumption  that  users  have  the  confidence  and  the 
vocabulary  to  make  their  difficulties  or  suggestions  public.  This  is  particularly  untrue  of  many  teachers,  whose 
use  of  computers  is  intermittent  (but  not  unimportant)  and  who  do  not  regard  themselves  as  part  of  the 
computing  inner  circle  (Dawes,  1999,  Pate,  1999).  It  also  does  not  address  the  problem  of  non-expert  (but  not 
necessarily  unintelligent)  users  having  to  explain  in  writing  what  they  mean  or  want,  while  not  knowing  what  is 
relevant  to  achieving  their  goals  (Sellen  and  Nichol  1 990,  Pausch,  Conway  & Define  1 992). 


Design  Solution 

The  authors  of  this  paper  have  designed  a feedback  feature  that  addresses  these  obstacles.  Although  useful  in  any 
organisational  context,  it  has  been  designed  for  inclusion  in  the  authors’  educational  software,  and  is  as  a result 
aimed  at  educators  in  particular. 

The  feature  provides  the  user  with  the  opportunity  at  any  time  when  using  an  application  (not  only 
when  the  application  encounters  an  error  condition)  to  click  on  an  ‘AUTHOR’  button  permanently  on  the  screen. 
This  calls  up  a feedback  form,  addressed  to  the  software  developers,  with  space  for  the  user  to  write  down  any 
comments  about  the  programme’s  performance.  What  makes  this  feedback  form  unique,  however,  is  that  it 
automatically  adds  to  the  user’s  message  a complete  history  of  the  user’s  choices  since  the  session  was  started. 
This  does  away  with  the  need  for  the  user  to  explain  what  he/she  was  doing  at  the  time  a given  problem  arose. 

The  advantage  to  the  user  is  access  to  software  developers  in  the  midst  of  the  routines,  breakthroughs 
and  breakdowns  that  define  typical  use.  With  increased  demands  being  made  on  faculty  to  manage  their  own 
instructional  technology,  this  not  only  provides  increased  support,  it  also  recognises  the  user  as  a ‘dialogue 
partner’  (Anderson  1 999)  rather  than  merely  a consumer.  The  advantage  to  the  developer,  on  the  other  hand,  is 
"backtalk":  that  is,  feedback  not  only  from  what  users  say,  but  from  what  they  actually  do  (Brown  & Duguid, 
1992).  Developers  can  also  easily  check  and  reproduce  phenomena  reported  by  their  users.  When  the 
application  is  installed,  moreover,  the  user  has  the  opportunity  to  add  individuals  or  groups  (eg  IT  support)  to  the 
mailing  list,  so  that  they  too  can  keep  a tally  of  the  sort  of  problems  or  issues  users  are  encountering. 


Consequences  of  Implementation 

Since  the  software  feature  described  in  this  paper  is  itself  still  in  the  beta-testing  phase,  it  remains  to  be  seen 
whether  users  find  the  possibility  of  immediate  feedback  practical,  and  whether  developers  find  the  ensuing 
response  worth  the  effort.  It  would  further  be  interesting  to  observe  whether  such  a feature  adds  to  or  detracts 
from  users’  ‘social/dialogue’  mental  model  of  their  interaction  with  their  technology. 

Meanwhile,  what  this  paper  advocates  is  an  exploration  of  the  wider  communication  opportunities,  the 
new  ’conversations’  (Winograd,  1986)  that  are  becoming  possible  during  routine  software  use.  To  this  end,  this 
paper  argues  that  software  applications  can,  in  addition  to  their  primary  functions,  become  feedback  channels 
that  allow  shared  responsibility  for  the  success  of  user-developer  communication. 
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Abstract:  In  1997  a partnership  was  created  between  the  Monterrey  Institute  of  Technology 
(ITESM)’s  Virtual  University  and  the  University  of  British  Columbia  (UBC).  This  partnership 
resulted  in  the  development  of  a Certificate  in  Technology-based  Distributed  Learning,  a series  of 
five  graduate  courses  launched  internationally  in  the  fall  of  1997.  This  paper  and  presentation  will 
explore  the  international  project  work  and  online  collaborative  experiences  of  a group  of 
international  students  registered  in  a web-based,  graduate-level  distance  education  course  offered 
by  UBC  and  its  partner,  ITESM. 


Background 

On  June  21,  1997  the  Virtual  University  of  ITESM  and  Distance  Education  and  Technology  signed  an  agreement  for 
the  development  and  delivery  of  five  courses  in  technology-based  distributed  learning  (Bates,  1 999).  This  became 
the  post-graduate  Certificate  in  Technology-based  Distributed  Learning  (http://itesm.cstudies.ubc.ca/info/). 

Consisting  of  five  graduate-level  courses,  the  Certificate  was  launched  in  September  with  the  first  course. 
Designing,  Developing  and  Delivering  Technology-based  Distributed  Learning,  opening  in  September  1997.  This 
was  followed  by  Selecting  and  Using  Technology  for  Distributed  Learning  (winter,  1998),  Planning  and  Managing 
Technology-based  Distributed  Learning  (fall,  1998),  Social  Issues  in  Technology-based  Distributed  Learning 
(winter,  1999),  and  Research  and  Evaluation  Issues  in  Technology-based  Distributed  Learning  (September  1999). 

Designed  using  international  collaboration,  team  meetings  were  held  and  decisions  were  made,  via 
videoconferencing  and  email.  The  course  technologies  included:  print  — textbook  and  selected  readings  delivered  to 
students  in  their  package;  a Web  Site  - comprised  of  study  guide,  original  teaching  materials,  online  discussion 
groups  (in  both  English  and  Spanish),  online  resources  (online  articles,  journals,  access  to  the  extension  library),  a 
bookmark  database  (dynamic  due  to  student  input),  student  and  instructor  biographies;  e-mail;  and  on  one  course 
(Selecting  & Using  TBDL)  a specially-designed  CD-ROM  (Bates,  1 999) 

In  addition  to  discussions  and  an  international  cafe,  the  program  also  invited  international  guest  visitors,  often  the 
authors  of  the  texts  or  readings,  to  conduct  weeklong  discussions  with  the  students. 


Social  Issues  in  Technology-based  Distributed  Learning 

Social  Issues  in  Technology-based  Distributed  Learning  was  offered  to  students  for  the  first  time  in  1999.  Team- 
taught,  the  course  had  an  enrolment  of  38  students  from  lO’  countries  while  its  partner  University  (with  its  own 
Spanish-speaking  instructors)  had  a concurrent  enrollment  of  100+  students  from  Mexico,  Chile  and  Venezuela. 
Over  the  13  weeks  of  the  course,  the  students  engaged  in  a number  of  online  and  interactive  activities  designed  to 
increase  their  contact  with  each  other.  Three  short  papers  discussed  the  social  issues  and  technology-based  learning 
issues  raised  by  the  course  readings  and  analyzed  using  several  frameworks.  The  online  ‘Brainstorm’,  which 


* The  10  countries  represented  were  Australia,  Botswana,  Canada,  China  (Hong  Kong),  Finland,  France,  Germany,  Mexico,  the 
United  States,  Yugoslavia. 
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followed  the  short  papers,  had  the  students  identifying  the  issues  they  felt  needed  to  be  discussed  in  the  course. 
Immediately  after  the  brainstorming  period,  students  were  divided  randomly  into  groups.  Each  group  (consisting  of 
3 international  students)  was  assigned  the  task  of  deciding  on  what  issue,  or  part  of  an  issue,  raised  by  the 
brainstorm  and  set  out  by  the  framework,  they  wanted  to  research  and  present  to  the  larger,  full  class.  The  final 
assignment  was  a formal  paper,  which  analyzed  and  reflected  on  an  issue  raised  by  the  course. 


Collaboration  and  Leadership  in  International  Groups 

The  observation  of  this  course  and  its  students  with  respect  to  collaboration  and  leadership  within  international 
groups  was  quite  valuable  for  the  instructors  and  the  student/participants.  Interestingly,  many  of  the  issues  that 
seemed  to  be  prevalent  could  be  considered  issues  in  Of  collaboration  (Janes,  1999).  Areas  that  required 
consideration  included 

• ‘disappearance’  and  ‘silence’  of  team  members  for  long  periods  of  time  when  decisions  were  necessary  within 
the  group; 

• the  encouragement  of  some  of  the  more  ‘quiet’  group  members  by  some  of  the  more  ‘vocal’; 

• the  recognition  that  ‘lurking’  does  not  mean  lack  of  engagement  with  the  ideas  or  the  learning 

• the  need  to  respect  the  ‘time’  it  takes  to  collaborate  online,  to  achieve  consensus  without  having  met  the  person 
or  persons  you  are  working  with;  and 

• to  respect,  at  all  times,  the  points  of  view  of  the  group  and  to  create  a ‘safe’  environment  for  discussion, 
disagreement  and  compromise 

/ learned  a lot  about  the  collaborative  learning  process... This  certainly  has  been  important  for  me  and  dramatically 
extended  my  understanding  of  the  complexity  of  TBDL  [technology-based  distributed  learning]  in  the  real  world. 
Article  posted  by  Student  T 

...Groups.  Again,  a completely  different  experience  than  other  ones!  It's  amazing  the  juggling  that  goes  on.  In  a way,  I 
rather  liked  that  we  were  assigned  to  groups  because  it ’s  more  true  to  life.  It 's  very  time-consuming  to  organize  groups 
virtually.  Personalities  came  through  which  humanized  the  process  for  me.  Two  things  were  difficult  for  me.  (i)  some 
members  didn ’t  make  contact  for  ages,  so  I even  wondered  if  they  were  still  there;  (ii)  Sometimes  feedback  was  needed 
asap  and  some  group  members  seemed  to  vanish  just  then.  tVe  resorted  to  a lot  of  communication  via  email  as  some  of 
us  had  difficulty  with  web  and  interaction  was  critical...  Article  posted  by  Student  K 


Conclusion 

This  course  was  the  first  offering  of  an  online  course  that  was  complex  in  design.  While  the  course  designers  and 
instructors  anticipated  many  of  the  outcomes  (Bullen  & Janes,  1998),  this  course  was  a testament  of  the  dedication 
and  interest  of  its  participants.  From  all  comers  of  the  world  and  from  many  and  varied  backgrounds  came  a group 
willing  to  tackle  some  of  the  most  current  and  interesting  topics  with  respect  to  teaching,  learning,  technology  and 
society.  Many  of  the  techniques  and  strategies  for  international  group  work  come  from  those  who  have  experienced 
this  phenomena  and  who  come  to  the  challenge  from  a common  sense  perspective,  a healthy  respect  for  those 
involved  and  a willingness  to  try  a new  way  of  interacting. 
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Abstract:  This  presents  the  findings  of  a study  of  the  use  of  a hypermedia  program.  Perspectives  on 
Emotional  and  Behavioral  Disorders,  with  undergraduate  students  in  introductory  courses  in  special 
education  (SE).  Upon  completion  of  the  chapter  on  BD  we  examined  1)  patterns  of  usage  created  by 
audit  paths,  2)  perceptions  of  program  quality,  and  3)  perceptions  of  feasibility  of  use  between  non- 
special education  (NSE)  majors  and  SE  majors. 

Instructional  programs  using  interactive  hypermedia  case  studies  have  been  created  and  demonstrated  to  be  an  effective 
tool  in  preservice  training  of  individuals  preparing  to  work  with  children  and  adolescents  with  emotional  and  behavioral 
disorders  (Fitzgerald  & Semrau,  1996).  The  instructional  approach  used  in  this  interactive  hypermedia  material  strives 
to  develop  cognitive  flexibility  for  problem  solving  within  an  ill-structured  knowledge  domain.  There  was  some  concern 
that  learners  who  may  have  little  background  information  or  interest  in  the  area  of  special  education  are  at  a disadvantage. 
Therefore,  we  were  concerned  that  the  use  of  a hypermedia  program  on  a topic  in  special  education  by  NSE  majors  may 
place  students  at  a great  disadvantage  and  result  in  negative  opinions  of  the  program  and  consequently  reduced  usage 
of  the  program. 

Research  Questions 

Do  major  and  nonmajor  preservice  teachers:  1)  utilize  the  hypermedia  training  program  differently  with  regard  to 
engagement  time;  2)  have  different  perceptions  of  the  quality  of  the  hypermedia  training  program,  and  3)  have  different 
perceptions  with  regards  to  the  feasibility  of  using  the  hypermedia  training  program? 

Participants 

Non-special  education  major  students  (n=34)  enrolled  in  the  undergraduate  class,  The  Exceptional  Student  in  the  Regular 
Education  Classroom,  and  special  education  majors  (n=19)  enrolled  in  their  initial  special  education  program  course, 
Introduction  to  Exceptional  Children,  participated  in  this  study.  All  participants  were  trained  in  using  the  interactive 
program  on  CD.  None  of  the  students  had  prior  special  education  course  experience.  The  hypermedia  program  used  in 
this  study  was  Perspectives  on  Emotional  and  Behavioral  Disorders  - Zach  (Fitzgerald  & Semrau  1 999). 

Measures  and  Results 

Total  Elapsed  Program  Engagement  Time 

Using  audit  trail  data  provided  by  the  AStudent@  disk,  the  researchers  computed  a total  number  of  minutes  in  which  each 
student  used  the  training  program.  The  mean  elapsed  time  for  NSE  majors  was  1 10  minutes,  while  the  total  elapsed  time 
for  SE  majors  was  134  minutes.  The  difference  between  the  two  groups  was  not  significant.  The  total  engagement  time 
was  substantially  less  than  the  mean  of 420  minutes  reported  by  undergraduate  students  who  had  completed  foundational 
courses  in  special  education,  behavior  management,  assessment  procedures,  and  instructional  strategies  as  reported  by 
Fitzgerald  and  Semrau  (1996). 
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Perceptions  of  the  Value  of  Using  the  Multimedia  Training  Program 


Five  items  from  the  Multimedia  Training  Program  Survey  HVITPS).  created  by  the  authors  of  the  program,  were  used 
to  investigate  the  students’  perceptions  of  the  usefulness  of  the  multimedia  program  for  teacher  training  in  emotional  and 
behavior  disorders:  (e.g.  I feel  this  program  was  a productive  use  of  my  time  because  it  added  to  my  knowledge).  For 
each  item  the  student  responded  on  a scale  of  1-5  with  l=strongly  disagree  to  3=don=t  know  to  5=strongly  agree. 
Responses  for  each  of  the  above  were  summed  and  divided  by  the  number  of  items  to  form  a score  for  Perceptions  of 
the  Value  of  Use  which  was  used  in  the  data  analysis.  Special  education  majors  had  significantly  more  positive  (p  = .02) 
perceptions  of  the  training  program  (mean  = 4.06)  than  the  NSE  major  students  (mean  =3.72),  Both  groups  thought  the 
program  offered  a positive  learning  opportunity  and  was  effective  in  accomplishing  its  instructional  objective.  In 
exploring  the  data  for  relationships  beyond  those  stated  in  the  purpose,  we  found  a weak  positive  relationship  between 
a student's  program  engagement  time  and  perception  of  program  value  (r  = .234). 

Perceptions  of  the  Feasibility  of  Using  Multimedia  Programs  for  Teacher  Training 

Three  items  from  the  Multimedia  Training  Program  Survey  (MTPS)  were  used  to  investigate  the  students  = perceptions 
of  the  feasibility  of  using  the  multimedia  program  for  teacher  training  with  SE  and  non-SE  majors  in  emotional  and 
behavior  disorders:  (e.g.  14. 1 became  confused  while  using  the  multimedia  training  program).  Using  the  same  scale  as 
described  above  and  correcting  for  the  negative  wording  of  the  items,  a mean  was  determined  and  subtracted  from  5 to 
form  a score  for  Perceptions  of  the  Feasibility  of  Use  which  was  used  in  the  data  analysis.  There  was  no  significant 
difference  between  the  SE  majors’  (mean  = 2.70)  perceptions  of  the  feasibility  of  using  the  program  and  NSE  majors’ 
(mean  = 2.71)  (p  = .970).  Both  groups  viewed  the  program  as  somewhat  difficult  to  use  and  mildly  frustrating  for  use 
in  teacher  training.  A weak  negative  relationship  was  found  between  a student's  program  engagement  time  and 
perceptions  of  feasibility  (r  = -.082).  A moderately  positive  relationship  between  a student's  perception  of  program  value 
and  perception  of  feasibility  (r  = .371). 

Discussion 

With  multimedia  programs  becoming  more  readily  available  for  use  in  teacher  education,  instructors  must  carefully  select 
programs  which  meet  the  content  and  skill  needs  of  students,  present  information  in  a manner  which  is  perceived  as 
useful,  and  be  feasible  to  use  by  the  student  outside  of  the  formal  structure  of  the  classroom.  We  know  that  case-based 
hypermedia  programs  are  well  received  by  students  with  background  knowledge  of  the  content  being  taught  by  the 
program  Fitzgerald  & Semrau,  (1996);  therefore  this  study  was  designed  to  explore  how  learners  responded  to  a 
hypermedia  learning  environment  in  content  in  which  they  were  novices.  While  there  was  a significant  difference 
between  the  ratings,  both  groups  of  students  perceived  the  training  program  to  be  a valuable  learning  experience.  Also, 
both  groups  of  novice  learners  rated  the  program  as  Adon=t  know@  when  asked  to  rate  the  program  on  the  feasibility 
of  using  the  CD-based  program  for  instruction.  Based  on  the  students’  perceptions  of  feasibility,  it  is  imperative  to 
support  student  use  of  hypermedia  programs  with  in-class  instructor  or  peer-led  guided  activities. 
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Abstract:  Developing  and  using  active  learning  exercises  in  classroom  settings  works  well  to 
engage  the  learners,  but  the  instructor's  challenge  is  to  take  those  same  exercises  and  make  them 
meaningful  in  a distance  education  model.  This  paper  describes  several  techniques  used  by  the 
author  in  a creative  problem  solving  course  redesign. 


Creative  Problem  Solving  Course  Redesign 

Teaching  creative  problem  solving  requires  some  creativity  on  the  part  of  the  instructor.  Rote  lectures  have  never 
been  optimal  learning  experiences,  nor  are  they  conducive  to  teaching  creativity.  In  the  design  of  the  course,  the  first 
challenge  was  to  get  the  students  to  loosen  up  and  understand  paradigm  boundaries  that  are  often  self  imposed. 
During  the  semester,  the  exercises  were  chosen  to  demonstrate  specific  uses  and  points  of  creativity  techniques  for 
problem  definition,  alternative  identification,  evaluation  of  alternatives,  planning/implementation,  change 
management,  and  monitoring. 


Problem  Definition 

The  exercise  to  demonstrate  self  imposed  paradigms  requires  25  inflated  balloons  and  at  least  8-10  people.  In  the 
class  room  the  students  are  formed  into  two  circles,  and  not  allowed  to  say  anything  to  each  other  or  ask  any 
questions.  Two  students  are  designated  as  observers  and  balloon  helpers.  The  students  are  given  the  instructions  to 
not  let  the  balloons  hit  the  floor.  Then  five  or  six  balloons  are  tossed  inside  the  circle.  After  they  toss  them  back  and 
forth  for  a minute,  the  rest  of  the  balloons  are  tossed  into  the  circle.  Some  of  the  students  figure  out  they  can  hold  the 
balloons,  pop  them,  push  them  into  the  other  circle,  or  rub  them  on  their  heads  to  generate  enough  static  electricity 
to  stick  the  balloons  onto  the  walls.  At  this  point,  some  students  start  declaring  that  these  tactics  aren’t  fair.  The 
instructor  stops  the  exercise,  and  asks  them  to  observe  what  happened  and  why.  The  observers  describe  the  tactics 
used,  and  the  means  of  non-verbal  communication  employed.  They  eventually  come  to  the  realization  that  the  ’rules' 
were  self  imposed  and  the  instructor  had  set  no  limits  on  actions  to  keep  the  balloons  off  the  floor.  The  instructor 
then  asks  them  to  relate  the  strategies  for  balloon  management  to  the  behaviors  at  work.  Tossing  the  balloons  to  the 
next  circle  is  similar  to  passing  off  bad  work  or  problems  to  another  department. 

When  this  exercise  was  given  to  the  distance  ed  learners,  they  were  asked  to  gather  a group  of  family  or  friends  to 
do  the  exercise,  and  act  as  the  instructor/observer.  Then  each  student  posted  his/her  observations  to  the  interactive 
area  of  the  WWW  based  class  management  software.  The  groups  included  family  members  from  2 to  80,  a cub  scout 
pack,  a coffee  group  from  work,  and  a group  of  friends  in  a bar.  The  results  were  consistent,  and  the  comments 
among  the  students  lively. 


Problem  Solving  Strategy 

This  assignment  was  about  designing  a new  monetary  system  for  a small,  island  country  in  the  South  Pacific.  First 
the  students  had  to  identify  all  the  paradigms  associated  with  currency  as  used  in  the  US  (shape,  color, 
denomination,  media,  uses,  ownership  proof,  durability,  portability,  and  drawbacks).  This  provided  students  with 
experience  with  other  monetary  systems  to  bring  in  samples  and  provide  descriptions.  Many  students  did  WWW 
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research  on  money/currency  prior  to  the  class  and  brought  in  the  printouts.  The  consistent  response  was  that  no-one 
had  ever  considered  that  all  currency  wasn’t  just  like  the  US  system  worldwide.  Then  the  students  were  divided  into 
groups  of  4 or  5,  given  paper  and  crayons,  and  sent  off  to  design  a new  monetary  system  that  could  be  used  by  the 
islanders  and  the  tourists  visiting  the  island.  Although  the  exercise  write  up  named  the  island  Freedonia,  only  one 
student  out  of  250  over  5 years  was  a Marx  Brothers  fan  and  recognized  the  name.  The  usual  solutions  are  for  chip 
embedded  plastic  cards  that  draw  down  on  an  account  established  with  an  island  bank,  with  a hole  in  the  card  for  a 
plastic  chain  to  wear  around  tourists  necks  or  pin  to  bathing  suits  (and  a solar  powered  energy  source  on  the  island 
so  the  card  readers  would  keep  working). 

Translating  this  exercise  for  the  distance  ed  format  was  relatively  easy.  Each  student  did  his/her  own  WWW 
research  and  personal  research  for  other  currencies,  then  submitted  a list  of  paradigms  for  US  currency.  The  class 
was  then  divided  into  groups,  and  they  collaborated  via  e-mail  and  fax  for  designing  a solution.  The  group  posted 
their  solution  on  the  interactive  area  of  the  WWW  class  management  tool.  Some  students  even  drew  solutions  and 
scanned  them  into  the  site  for  other  students  to  view. 


Process  Mapping 

This  exercise  is  based  on  a common  student  experience  of  getting  advisor  approval,  registering  and  paying  tuition 
for  a class,  and  getting  books.  The  students  first  list  all  the  steps  they  go  through,  and  all  the  departments  of  the 
university  that  are  part  of  the  process.  The  students  are  given  pads  of  sticky-paper  notes  on  which  they  list  each 
process  step  individually.  They  post  the  steps  on  a white  board  or  large  piece  of  paper  by  department,  and  draw  in 
lag  times  to  determine  elapsed  cycle  times  for  each  step.  Then  the  students  are  asked  to  redesign  the  process  using 
on-line  WWW  registration,  fee  payment  and  book  ordering  and  recalculate  cycle  times  and  the  effects  on  each 
department  involved.  The  costs  and  benefits  are  estimated  for  the  current  and  proposed  processes.  The  students  also 
have  to  identify  opportunities  for  fraud  and  where  security  was  needed.  This  exercise  usually  stimulates  questioning 
as  to  why  the  university  doesn't  do  all  this  right  now,  which  leads  to  a good  discussion  on  impediments  to  achieving 
perfection  in  the  real  world. 

Converting  this  exercise  to  distance  ed  teams  was  more  challenging  because  the  WWW  tool  did  not  have  a site 
amenable  to  using  sticky  paper  notes  for  posting.  The  teams  used  faxes  with  each  other,  and  assigned  process 
streams  to  each  member.  The  results  were  posted  on  the  interactive  site  in  words,  unless  a team  member  had  a 
scanner  available.  The  results  were  similar  though,  because  the  students  had  experience  with  both  traditional  in- 
person  classes  and  distance  ed  classes.  The  distance  ed  students  were  more  ingenious  in  devising  ways  to  break  into 
the  system  to  subvert  it,  as  well  as  identifying  security  risks. 
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Abstract:  Distance  education  programs  throughout  the  world  continue  to  grow  in  size  and  scope. 
As  online  courses  escalate  in  size  and  numbers,  educational  leaders  say  content  and  customization 
is  the  key  to  successful  learning  and  not  the  numbers  of  students  served.  Understanding  best 
practices  in  distance  education  via  tips  and  tricks  for  development,  delivery  and  management  of 
online  courses  will  be  central  to  the  future  of  this  industry. 


Innovative  Online  Course  Ideas-  Tips  and  Tricks 

Background 

USU  currently  offers  over  70  fully  online  courses  and  two  masters  degree  programs.  In  the  past  year  we’ve 
developed  nearly  100  courses  with  web  supplements.  We’ve  delivered  over  100  courses  via  digital  satellite  to  over 
90  sites  across  the  intermountain  west.  We  have  had  over  3000  students  registered  for  courses  this  past  semester  and 
expect  nearly  6000  this  coming  semester.  Multimedia  and  Distance  Learning  Services  and  the  Faculty  Assistance 
Center  for  Teaching  provide  workshops,  instructional  design,  video  production,  web  development,  engineering, 
electronic  classroom  support,  etc. 

Online  Course  Questions 

What  are  some  of  the  issues,  strategies  and  best  practices  in  developing  online  courses? 

What  are  some  of  the  innovative  uses  of  emerging  technologies? 

Problem  Identification: 

Is  your  organization  fragmented?  Are  faculty  moving  in  multiple  directions?  Are  you  struggling  with  rapid 
technological  changes?  Is  there  a lack  of  focus?  How  do  you  cope  with  the  myriad  of  software  applications? 

How  We  Began... 

We  started  by  doing  our  own  programming,  looking  at  and  using  many  software  applications  such  as  Web  Course  in 
a Box,  Learning  Space,  WBT,  Blackboard,  Syllabase,  etc.  We  eventually  settled  on  WebCT  as  our  core  program. 

What  We  Did... 

We  pulled  together  the  various  organizations  involved  in  online  course  development  across  campus.  We  then  looked 
at  and  narrowed  the  hardware  and  software  technology  we  were  using.  Our  third  step  involved  offering  appropriate 
training  to  faculty  and  staff.  We  completed  our  initial  efforts  with  a focus  on  student  learning  assessments. 

Tip  1 Instructional  Design  Steps 

1.  Analyze,  2.  Design,  3.  Develop,  4.  Implement,  5.  Evaluate 
Tip  2 Conversion  Process 

Design  appropriate  templates/architecture.  Specify  content/media  needs.  Author/customize  interactions.  Develop  a 
content  outline.  Convert  multimedia  assets.  Test  and  maintain  applications.  Create  a pilot  project. 


Tip  3 E-Learning  (Elliot  Masie-  Techleam) 

Incorporate  the  following  in  your  online  courses;  Engagement,  Curiosity,  Simulation  and  Practice,  Remediation, 
Coaching,  Peer  Learning,  Social  Dimensions,  Action  Learning,  Performance  Support,  Intensity,  Assessment  and 
feedback,  Teaching  Culture. 

What  Should  You  Do? 

Take  baby  steps,  engage  your  students  and  unify  your  efforts-  work  with  others-  don’t  give  up 
Tricks 

EMAIL  Overload-  Online  courses  can  generate  lots  of  emails.  Technical  issues,  assignment  clarification,  content 
issues,  turning  in  assignments,  etc. 

Use  bulletin  board  instead  of  Email.  Then  you  only  have  to  answer  the  question  once.  Tell  your  students  to  read 
them  first  before  asking  a question.  Compile  and  use  this  for  future  sections.  (FAQ) 

Getting  Started  Section-  Use  tutorials  (how  to  use  tools,  WebCT,  etc),  downloading  and  installing  plug-ins,  getting 
latest  browsers,  could  be  part  of  the  first  lesson. 

Frequently  Asked  Questions  - You  know  most  of  the  questions  you  get  asked,  keep  track  and  compile  them  after  the 
first  class,  put  them  online  and  consider  using  a discussion  group. 

Clear  Assignments  and  Objectives-  Clearly  state  objectives,  clearly  state  what  is  expected  to  complete  the  current 
lesson,  keep  pages  consistent  and  don’t  hide  them  deep  in  the  course  pages. 

Example  of  good  objectives- 

Read  chapter  2 in  your  textbook  (pages  19-32) 

Carefully  study  the  important  concepts 
Watch  the  videos  on  vibrating  systems 

Be  able  to  determine  the  frequency  of  vibration  for  simple  harmonic  motion  of  different  systems. 

Be  able  to  describe  the  motion  of  complex  vibrating  systems  as  harmonic  or  non-harmonic. 

Take  the  self-test  by  clicking  on  the  “self-test”  icon  at  the  top  of  the  page. 

Printable  Documents- 

Create  online  learning,  not  online  reading,  use  Adobe  Acrobat,  format  clean  printable  HTML  pages,  stay  away  from 
dark  backgrounds  with  light  text,  they  don’t  print  well. 

Student  Loneliness- 

Have  them  introduce  themselves  to  you  and  the  class  in  a discussion  group,  do  collaborative  projects  in  small 
groups,  provide  a chat  room  and  provide  a list  of  students  in  the  class.  (Email) 

Quizzing- 

Provide  a sample  quiz  online  so  students  can  gain  some  experience  using  the  testing  interface  and  technology. 

Summary 

“The  society  that  will  be  successful  in  the  future  will  not  be  the  nation-state  or  community  that  can  accommodate  a 
steady  stream  of  students,  but  that  nation  that  can  lift  the  level  of  learning  among  all  its  students  rapidly  and 
repeatedly,.,  .higher  education  ought  to  be  focused  on  content,  not  hardware.”  Gov.  Michael  O.  Leavitt 

The  following  book  citation  is  an  excellent  resource  for  understanding  key  elements  of  online  course  development- 

Building  Learning  Communities  in  Cyberspace 
■Authors:  Rena  M.  Palloff,  Keith  Pratt 
■Publisher:  Jossey-Bass,  San  Fransico 
■www.josseybass.com 
■ISBN:  0-7897-4460-2 
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Abstract 

We  are  developing  3D  visualizations  for  use  in  AI  classes.  This  paper  presents  the  process  of  developing 
visiaulization  for  symbolic  information  for  classroom  use.  The  basic  process  is  to:  determine  the  questions  the  user 
should  be  able  to  ask,  determine  what  information  will  help  in  answering  the  questions,  find  a representation  to 
present  the  information,  prototype  the  visualization  in  VRML  and  build  tools  to  allow  students  to  build  the 
visualization.  The  emphasis  is  on  how  to  visualize  symbolic  information  rather  than  numeric  data.  The 
visualizations  are  to  aid  in  presenting  and  understanding  complex  symbolic  based  material.  We  are  using  them  to 
present  the  internal  processes  of  the  AI  techniques.  The  visualizations  are  presented  as  interactive  3D  worlds.  The 
worlds  are  written  in  VRML  and  accessed  by  the  Web.  This  combination  allows  easy  distribution  of  models  for  the 
students  to  inspect  and  control.  The  students  use  the  tools  to  build  visualizations  of  their  own  programs.  The 
response  to  visualizations  and  the  visualization  building  tools  has  been  positive.  The  author  feels  they  have 
significantly  improved  the  student  understanding  of  concepts. 

Introduction 

Visualizations  can  help  in  the  process  of  understanding  classroom  material,  techniques  and  processes.  This  paper 
presents  suggestions  to  aid  in  developing  visualizations  for  a class.  The  focus  of  this  work  is  on  symbolic 
information  rather  than  quantitative  information.  Traditionally  visualizations  have  been  used  with  quantitative 
information  [Tufte  83].  The  Visual  Program  project  http://www.ndsu.nodak.edu/instruct/iuell/vD  fJuell  August 
1999]  [Juell  March  1999]  is  working  on  improving  the  tools  that  can  be  used  in  an  Artificial  Intelligence  class. 
This  project  has  been  developing  visualizations  to  aid  in  the  understanding  of  various  AI  techniques.  This  paper 
presents  the  analysis  and  recommendations  that  produced  our  best  products.  The  process  includes  the  following 
stages: 


• Determine  the  question(s)  you  want  the  viewer  to  be  able  to  answer. 

Each  visualization  allows  someone  to  see  some  patterns  and  hides  other.  For  the  visualization  to  be  useful 
and  valuable,  you  need  to  address  the  particular  questions  that  are  useful  in  the  situation  being  studied. 

• Determine  the  information  you  want  to  show. 

Limit  the  information  to  that  which  is  salient  to  the  questions  you  want  answer.  Ignore  bookkeeping  and 
internally  used  information.  This  will  result  in  simpler  and  clearer  visualization  and  make  the  rest  of  the 
process  simpler. 

• Design  and  prototype  the  visualization. 

Once  you  have  a goal  of  answering  a question  and  the  information  you  are  going  to  use,  build  a prototype 
of  the  visualization. 

• 2D  Diagrams 

We  start  our  projects  with  a 2D  design.  Symbolic  information  can  be  specified  by  character 
strings  positioned  in  2 dimension  space  on  paper.  We  then  consider  replacing  the  strings  with 
graphics.  We  are  interest  in  showing  the  operations  of  a process,  so  one  dimension  is  normally 
time.  For  several  problems,  we  have  then  used  a mapping  of  the  state  space  for  the  other 
dimension. 

• 3D  Diagrams 

We  then  consider  how  to  use  3D  to  better  show  data  and  information.  3D  diagrams  can  be  more 
attractive  than  the  2D  images.  In  addition,  we  found  the  extra  dimension  important  because  of 
the  high  number  of  dimensions  in  our  original  problems.  Visualizations  work  by  allow  the  eye  to 
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see  patterns.  For  symbolic  information,  using  a tall  box  for  each  symbol  can  allow  the  eye  to  see 
repeated  patterns  in  spatial  location.  If  the  data  is  presented  so  that  the  eye  identifies  important 
patterns,  this  presentation  can  allow  the  user  to  convert  the  data  into  information. 

• Interactive  Diagrams 

We  use  interactive  diagrams  to  aid  active  learning.  Most  of  our  work  uses  3D  worlds  created  in 
VRML  [Hartman  and  Wemecke  1996].  The  VRML  allows  the  presentation  of  movies  and  allows 
the  viewer  to  control  flying  through  the  images.  In  addition,  motion  can  be  supported  directly  in 
VRML  or  by  adding  JAVA  applets. 

• Layering  of  Information  (hot  links,  layered  displays) 

Traditional  visualization  techniques  can  add  additional  dimensions  of  information  to  the  picture. 
You  can  map  information  into  false  colors,  sizes  and  other  information  overlayed  on  the  icons  in 
the  images.  In  addition,  you  can  link  the  items  to  other  images  and  pages. 

• Instrument  the  program/process  at  the  symbolic  level. 

We  want  the  students  to  build  visualizations  of  their  own  programs.  To  do  this,  we  need  to  provide  tools 
which  will  create  the  visualizations  for  them.  They  run  their  program.  Its  output  is  then  processed  to  build 
the  visualization.  To  make  this  work,  the  program  is  instrumented  to  capture  data  to  be  converted  into  the 
visualization.  The  instrumentation  code  needs  to  capture  the  state  information  of  the  state  space  search.  It 
is  very  hard  to  reconstitute  this  state  information  later.  The  code  may  have  to  make  explicit  state 
information  which  is  implicit  in  the  program. 

Example  - State  Space  Navigation 

A small  example  may  clarify  some  of  the  points.  A typical  example  of  a state  space  search  is  the  river  crossing 
problem.  We  built  a visualization  for  this,  with  a goal  of  addressing  "where  does  the  time  do  in  a blind 
backtracking  search"?  For  one  dimension  we  used  spatial  information  about  the  boat  and  the  people.  We  used  three 
positions:  the  side  the  people  start  at,  a position  in  the  middle  of  the  river  and  the  river  side  the  people  want  to 
reach.  At  each  of  these  locations,  we  placed  objects  representing  the  people  and  the  boat.  The  father,  the  two  sons 
and  the  boat  were  each  represented  by  a different  sized  block.  We  tried  to  stay  with  very  simple  objects  because  we 
felt  a small  amount  of  detail,  would  allow  the  viewer  to  more  easily  see  patterns.  For  the  other  dimension  (for  the 
2D  example)  we  used  time.  One  image  then  showed  all  of  the  steps  of  the  search.  In  the  process  of  developing  our 
visualizations,  these  images  underwent  a number  of  changes  to  improve  the  overall  usability  of  the  image.  One 
problem  we  repeatedly  had,  was  that  improving  the  esthetics  of  the  images  often,  in  the  author's  opinion,  made  it 
less  likely  that  that  the  useful  patterns  would  be  visible  in  the  final  image.  Once  acceptable  prototype  of  the 
visualization  was  developed,  an  instrumented  program  was  prepared  and  a tool  was  developed  to  produce  the 
visualization.  This  combination  was  then  used  by  the  students  for  their  programming  assignment. 

Summary 

We  have  presented  the  process  we  are  using  to  develop  abstract  visualizations.  These  visualization  are  intended  to 
be  used  in  class  to  aid  in  understanding  complex  concepts.  These  guidelines  have  helped  us  to  produce  a variety  of 
visualizations.  We  have  used  these  in  the  classroom  and  as  parts  of  homework  projects.  The  students  were  very 
positive  in  their  response  to  the  visualizations.  The  author  has  noticed  that  questions  asked  by  students  changed 
following  the  use  of  the  visualizations.  The  questions  now  represent  a higher  level  of  thinking.  The  questions 
before  were  mainly  "what  does  this  line  of  code  do?"  and  now  they  are  frequently  "how  does  this  effect  the 
visualization  we  are  creating?"  We  feel  the  above  notes  give  a good  start  in  developing  visualizations  for  symbolic 
information.  We  also  believe  these  visualization  are  important  in  allowing  students  to  explore  and  understand  a 
wider  scope  of  information  about  the  material  presented  in  the  class. 
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Abstract:  Distance  education  programs  throughout  the  world  continue  to  grow  in  size  and  scope. 
As  online  courses  escalate  in  size  and  numbers,  educational  leaders  say  content  and  customization 
is  the  key  to  successful  learning  and  not  the  numbers  of  students  served.  Understanding  best 
practices  in  distance  education  via  tips  and  tricks  for  development,  delivery  and  management  of 
online  courses  will  be  central  to  the  future  of  this  industry. 


Innovative  Online  Course  Ideas-  Tips  and  Tricks 

Background 

USU  currently  offers  over  70  fully  online  courses  and  two  masters  degree  programs.  In  the  past  year  we’ve 
developed  nearly  100  courses  with  web  supplements.  We’ve  delivered  over  100  courses  via  digital  satellite  to  over 
90  sites  across  the  intermountain  west.  We  have  had  over  3000  students  registered  for  courses  this  past  semester  and 
expect  nearly  6000  this  coming  semester.  Multimedia  and  Distance  Learning  Services  and  the  Faculty  Assistance 
Center  for  Teaching  provide  workshops,  instructional  design,  video  production,  web  development,  engineering, 
electronic  classroom  support,  etc. 

Online  Course  Questions 

What  are  some  of  the  issues,  strategies  and  best  practices  in  developing  online  courses? 

What  are  some  of  the  innovative  uses  of  emerging  technologies? 

Problem  Identification: 

Is  your  organization  fragmented?  Are  faculty  moving  in  multiple  directions?  Are  you  struggling  with  rapid 
technological  changes?  Is  there  a lack  of  focus?  How  do  you  cope  with  the  myriad  of  software  applications? 

How  We  Began... 

We  started  by  doing  our  own  programming,  looking  at  and  using  many  software  applications  such  as  Web  Course  in 
a Box,  Learning  Space,  WBT,  Blackboard,  Syllabase,  etc.  We  eventually  settled  on  WebCT  as  our  core  program. 

What  We  Did... 

We  pulled  together  the  various  organizations  involved  in  online  course  development  across  campus.  We  then  looked 
at  and  narrowed  the  hardware  and  software  technology  we  were  using.  Our  third  step  involved  offering  appropriate 
training  to  faculty  and  staff.  We  completed  our  initial  efforts  with  a focus  on  student  learning  assessments. 

Tip  1 Instructional  Design  Steps 

1.  Analyze,  2.  Design,  3.  Develop,  4.  Implement,  5.  Evaluate 
Tip  2 Conversion  Process 

Design  appropriate  templates/architecture.  Specify  content/media  needs.  Author/customize  interactions.  Develop  a 
content  outline.  Convert  multimedia  assets.  Test  and  maintain  applications.  Create  a pilot  project. 


Tip  3 E-Learning  (Elliot  Masie-  Techleam) 

Incorporate  the  following  in  your  online  courses;  Engagement,  Curiosity,  Simulation  and  Practice,  Remediation, 
Coaching,  Peer  Learning,  Social  Dimensions,  Action  Learning,  Performance  Support,  Intensity,  Assessment  and 
feedback,  Teaching  Culture. 

What  Should  You  Do? 

Take  baby  steps,  engage  your  students  and  unify  your  efforts-  work  with  others-  don’t  give  up 
Tricks 

EMAIL  Overload-  Online  courses  can  generate  lots  of  emails.  Technical  issues,  assignment  clarification,  content 
issues,  turning  in  assignments,  etc. 

Use  bulletin  board  instead  of  Email.  Then  you  only  have  to  answer  the  question  once.  Tell  your  students  to  read 
them  first  before  asking  a question.  Compile  and  use  this  for  future  sections.  (FAQ) 

Getting  Started  Section-  Use  tutorials  (how  to  use  tools,  WebCT,  etc),  downloading  and  installing  plug-ins,  getting 
latest  browsers,  could  be  part  of  the  first  lesson. 

Frequently  Asked  Questions  - You  know  most  of  the  questions  you  get  asked,  keep  track  and  compile  them  after  the 
first  class,  put  them  online  and  consider  using  a discussion  group. 

Clear  Assignments  and  Objectives-  Clearly  state  objectives,  clearly  state  what  is  expected  to  complete  the  current 
lesson,  keep  pages  consistent  and  don’t  hide  them  deep  in  the  course  pages. 

Example  of  good  objectives- 

Read  chapter  2 in  your  textbook  (pages  19-32) 

Carefully  study  the  important  concepts 
Watch  the  videos  on  vibrating  systems 

Be  able  to  determine  the  frequency  of  vibration  for  simple  harmonic  motion  of  different  systems. 

Be  able  to  describe  the  motion  of  complex  vibrating  systems  as  harmonic  or  non-harmonic. 

Take  the  self-test  by  clicking  on  the  “self-test”  icon  at  the  top  of  the  page. 

Printable  Documents- 

Create  online  learning,  not  online  reading,  use  Adobe  Acrobat,  format  clean  printable  HTML  pages,  stay  away  from 
dark  backgrounds  with  light  text,  they  don’t  print  well. 

Student  Loneliness- 

Have  them  introduce  themselves  to  you  and  the  class  in  a discussion  group,  do  collaborative  projects  in  small 
groups,  provide  a chat  room  and  provide  a list  of  students  in  the  class.  (Email) 

Quizzing- 

Provide  a sample  quiz  online  so  students  can  gain  some  experience  using  the  testing  interface  and  technology. 
Summary 

“The  society  that  will  be  successful  in  the  future  will  not  be  the  nation-state  or  community  that  can  accommodate  a 
steady  stream  of  students,  but  that  nation  that  can  lift  the  level  of  learning  among  all  its  students  rapidly  and 
repeatedly,... higher  education  ought  to  be  focused  on  content,  not  hardware.”  Gov.  Michael  O.  Leavitt 

The  following  book  citation  is  an  excellent  resource  for  understanding  key  elements  of  online  course  development- 

Building  Learning  Communities  in  Cyberspace 
■Authors;  Rena  M.  Palloff,  Keith  Pratt 
■Publisher:  Jossey-Bass,  San  Fransico 
■www.josseybass.com 
■ISBN:  0-7897-4460-2 
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Abstract:  This  paper  will  review  the  progress  of  work  being  done  by  a consortium  of  higher 
education  institutions,  working  with  a public  broadcasting  television  station  and  commercial 
courseware  developer  to  create  and  deliver  courses  in  the  converging  environments  of  the  Internet 
and  broadcast  digital  television.  The  experience  of  moving  traditional  broadcast  telecourses  to  an 
online  courseware  package  will  be  presented,  along  with  the  opportunities  and  challenges  faced  as 
digital  television  broadcasting  technology  evolves. 


Introduction 

WHYY,  the  Public  Broadcasting  Service  affiliate  in  Philadelphia,  Pennsylvania,  established  and  hosts  the  Delaware 
Valley  Distance  Learning  Consortium  to  address  the  common  distance  learning  needs  of  the  region’s  higher 
education  institutions.  Under  the  leadership  of  Delaware  County  Community  College,  Camden  County  College  and 
Community  College  of  Philadelphia,  the  WHYY  Home  College  Service  broadcasts  over  five  hours  of  telecourse 
programming  daily,  during  overnight  hours.  Students  enrolled  at  DVDLC  institutions  videotape  the  programs  for 
convenient  viewing,  complete  assignments  and  receive  college  credit  for  their  work. 

With  the  growth  of  the  Internet,  and  recognizing  the  pedagogical  opportunities  online  courses  present,  WHYY  and 
the  DVDLC  began  a cooperative  venture  to  deliver  courses  on  the  World  Wide  Web.  In  addition,  all  United  States 
television  broadcasters  are  under  mandate  from  the  Federal  Communications  Commission  to  transition  to  digital 
television  broadcasting.  Along  with  the  larger  convergence  of  media  types  and  delivery  methods,  this  presents  an 
opportunity  to  develop  next-generation  courses  that  repurpose  current  content  using  converging  technologies. 
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WHYY  is  the  central  technology  provider,  licensing  the  courseware  and  integrating  the  courseware  servers  into  the 
WHYY  Technology  Center.  WebStudy,  Inc.,  a Philadelphia  region  company,  supplies  the  courseware,  WebStudy. 
WHYY  provides  technology  support  and  faculty  training.  WebStudy  supports  the  courseware  and  the  faculty  and 
student  help-desk.  Consortium  institutions  offer  their  courses,  enroll  students,  and  are  responsible  for  the  quality  of 
the  courses  and  establishing  content  ownership.  Beta  testing  of  the  courseware  began  with  fall  semester  in  1 998. 

Early  Experience 

In  fall  1999,  Delaware  County  Community  College  began  building  “hybrid  courses,”  using  the  curriculum  already 
developed  for  telecourse-based  courses.  Hybrid  courses  include  media  streams  of  broadcast  telecourses,  traditional 
reading  and  writing  assignments,  and  interactive  asynchronous  forums.  In  January  2000,  three  courses,  English 
Composition  I,  English  Composition  II  and  Geology,  were  offered  to  DCCC  students  in  the  Philadelphia  region, 
either  as  traditional  telecourses  or  as  hybrid  courses.  Students  in  the  hybrid  courses  have  the  choice  of  viewing 
lower  quality  streaming  media  or  higher  quality  broadcasts.  An  additional  test  group  of  students  from  outside  the 
Philadelphia  region  was  enrolled  in  the  courses,  but  not  able  to  receive  the  telecourse  broadcasts.  Other  DVDLC 
members  are  offering  courses  using  WebStudy  with  content  unrelated  to  the  broadcast  telecourses. 


Digital  television  is  a work  in  progress.  The  technology  promises  high  bandwidth  distribution  combining  the 
advantages  of  broadcasting  with  the  interactive  and  data  features  of  online  learning.  Digital  television  is  a broadcast 
signal  with  an  effective  datarate  of  approximately  19  megabits  per  second.  Like  today’s  analog  television,  the 
datastream  can  be  received  by  anyone  in  the  broadcast  area.  Many  types  of  data,  including  high  definition 
television,  multiple  channels  of  standard  definition  television,  packages  of  data  delivered  like  a CD-ROM, 
datastreams  that  accompany  broadcasts  or  stand  alone,  Web  pages,  pre-delivered  content  - essentially  anything  that 
can  be  delivered  in  digital  form,  can  be  in  the  datastream.  While  the  technology  is  still  in  its  early  stages  of 
development,  digital  television  receiver  cards  are  available  that  can  be  installed  in  home  PCs.  Manufacturers  are 
designing  "set  top  boxes"  that  include  an  operating  system  and  the  necessary  hardware  and  software  to  decode,  store 
and  display  digital  broadcast  data,  either  in  real  time  or  acting  as  home  servers.  A relatively  low  bandwidth  Internet 
connection  will  allow  data  to  be  returned  to  central  servers,  either  in  real  time  or  on  a scheduled  basis.  Each  device 
will  have  its  own  unique  identifier,  similar  to  an  IP  address,  allowing  content  to  be  directed  to  a subset  of  users. 

We  are  investigating  many  ways  to  use  the  datastream.  Hybrid  courses  now  streamed  on  the  Web  could  download 
to  the  student's  "digital  television  device,"  an  advanced  set  top  box  with  capabilities  of  a computer,  so  the  student 
could  work  off  line,  CD-ROM  style,  with  updated  content  delivered  when  necessary.  Content,  including  "television 
quality"  video,  could  be  pre-delivered  to  the  student  when  bandwidth  is  available,  using  the  device  as  a local  cache. 
Using  the  low  bandwidth  Internet  connection,  real  time  interaction  can  occur,  and  work  can  be  submitted  to  the 
instructor  in  the  background.  The  "digital  television  device"  will  be  standards  based,  with  a common  interface, 
significantly  reducing  support  issues.  The  experience  will  be  like  "turning  on  the  television,"  though  not  like 
television  as  we  know  it  today. 


Carvin,  A.  (1998)  Digital  Television:  A New  Tool  for  Education.  Online:  http://edweb.gsn.org/teled98/speech.html 

H2snsCor^ox2d:\on.{\99%)MediaGuideto  DigitalTelevision.  Online:  http://dtvexpress.org/facts/facts.htm 

Weber,  B.  (1999)  Philadelphia  PBS  Station  WHYY  Launches  New  $25M  Digital  Broadcast  Facility— Part  /. 
Online:  http ://www2. digitalbroadcasting. com/content/news/article. asp?DocID={EBA44A2D-5 998- 1 1 D3-9A5F- 
00A0C9C83AFB} 

Weber,  B.  (2000)  Philadelphia  PBS  Station  WHYY  Launches  New  $25M  Digital  Broadcast  Facility — Part  II. 
Online:  http://www.digitalbroadcasting.com/Content/News/article.asp?DocID={5A  177725-8937-1 1D3-9A68- 
00A0C9C83AFB} 
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Abstract:  This  paper  shows  how  concept  maps,  a tool  utilized  rather  widely  these  days, 
can  be  advanced  to  new  horizons  using  computer-based  approach.  The  author  discusses 
general  structure  of  concept  maps,  their  use  as  dynamic  instructional  tools,  and  then 
presents  a model  for  implementation  of  computer-based  concept  maps  in  college-level 
business  courses.  Most  advanced  concept  maps  are  generated  using  multimedia 
authoring  software  and  can  be  either  qualitative  or  quantitative  in  nature.  Such  computer- 
based  concept  maps,  located  either  on  a computer  in  a classroom  or  on  the  Internet,  are  a 
very  powerful  and  unique  tool  contributing  to  other  learning  techniques  and  assessment 
methods. 


Concept  Maps:  An  Introduction 

A concept  map  is  defined  as  a graphical  knowledge  representation  that  hierarchically  links  concepts.  Novak  (1984), 
a leading  authority  on  concept  mapping  in  education,  proposed  that  different  learners  structure  their  new  current  and 
new  knowledge  in  a variety  of  ways,  therefore  validating  use  of  concept  maps  in  teaching  and  learning.  If  the 
instructor's  role  is  to  guide  the  learner  in  creating  new  meanings  and  deeper  levels  of  conceptual  understanding,  then 
concept  mapping  can  be  an  effective  instructional  and  assessment  tool  to  use  in  undergraduate  lower  division 
business  courses.  Knowledge  is  accumulated  in  a continuous  fashion,  and  developing  connections  between  prior 
and  new  knowledge  is  critical  to  the  learning  process. 

In  a traditional  concept  map  design,  the  most  general  or  most  important  concept  is  placed  at  the  top  of  map,  and 
concepts  on  lower  levels  are  subsequently  more  detailed.  Key  concepts  are  connected  by  links  that  have  descriptive 
words  on  them  explaining  the  relationship  between  concepts.  Concepts  maps,  by  their  visual  presentation,  have 
traditionally  been  used  as  very  effective  vehicles  for  organization  of  concepts  within  the  domain.  Traditional 
concept  maps  were  of  "paper  and  pencil"  type,  and  were  used  mainly  to  organize  qualitative  or  conceptual 
knowledge.  Thus,  in  rethinking  course  curriculum  at  the  undergraduate  level,  the  number,  organization  and  plan  for 
the  systematic  development  of  concepts  are  worthy  considerations  in  terms  of  curriculum  restructuring  necessary  for 
improved  student  performance. 

As  concept  maps  evolve,  there  are  variations  in  their  development,  discussed  by  Shavelson  et  al  (1994),  including 
free-form  versus  hierarchical,  concepts’  position  on  the  map,  collaborative  effort  by  a group  of  students.  These 
variations  provide  for  greater  instructional  flexibility  as  well  as  for  more  diverse  learning  experience.  In  the  last 
decade,  a noted  increase  in  interest  in  and  use  of  concept  maps  has  resulted  in  the  identification  of  many  valuable 
instructional  purposes  for  mapping. 

A Model  for  Implementation  of  Computer-based  Concept  Maps  in  College  Business  Courses 

An  advent  of  computers,  word  processing  and  flow-chart  software,  as  well  as  later  developments  in  multimedia 
authoring  software  enables  instructors  to  push  the  limits  of  usefulness  of  traditional  concept  maps  far  beyond  "pencil 
and  paper"  representation.  It  is  worth  noting  that  in  this  respect  the  following  stages  of  software  development  can  be 
identified: 


1 ) Non-dedicated  word-processing  or  flow-chart  software  allowed  for  passive  creation  and  use  of  computer- 
based  concept  maps 

2)  Software  dedicated  to  generating  concept  maps  results  in  production  of  more  responsive  and  interactive 
computer  tool 

3)  Multimedia  authoring  software  for  fully  interactive  knowledge  building  and  assessment  using  computer- 
based/multimedia  concept  maps. 

This  article  presents  initial  approach  undertaken  in  implementation  of  concept  maps  in  college-level  business 
courses,  and  creates  a model  suitable  for  implementation  in  any  business  courses  for  computer-based  knowledge 
building  and  alternative  assessment  using  concept  maps.  Since  this  represents  significant  evolution  of  concept 
maps,  we  will  refer  to  them  as  (c)omputer-(b)ased  qualitative  and  quantitative  (k)nowledge  (m)aps  (CBKM). 

Software 

For  the  purpose  of  this  model,  software  from  above  mentioned  stages  2)  and  3)  is  used.  Software  dedicated  to 
generating  concept  maps  such  as  Inspiration  ® (or  comparable)  is  used  by  students  almost  exclusively.  Instructors 
use  Inspiration®  for  knowledge  maps  presented  or  developed  during  classes.  These  presentations  are  very  effective 
and  lead  to  interesting  interactions  when  presented  on  a large  projector  screen.  The  set  of  standard  knowledge  maps 
as  well  as  the  ones  developed  during  classes  is  uploaded  on  the  Internet  under  the  course’s  Web  page. 

Multimedia  authoring  software  such  as  Authorware®  by  Macromedia  (or  comparable)  is  used  by  instructors  to 
prepare  highly  interactive  software  used  by  students  in-  and/or  out-of-classroom  to  build  new  knowledge,  form 
linkages,  and  to  build  critical  thinking  and  analytical  skills.  Multimedia  authoring  software  enables  instructors  to 
provide  students  with  an  interactive  study  tool  that  provides  them  with  an  instant  feedback,  where  each  one  of 
superordinate  or  subordinate  concepts,  linkages  and  propositions  is  a hyperlink  providing  immediate  response  or 
guidance.  In  addition  to  having  such  developed  and  customized  software  place  on  computers  in  business  computer 
labs  or  individualized  study  labs,  this  software  can  be  made  available  on  the  Internet. 


Conclusions  and  implications. 

Computer-based,  qualitative  and  quantitative  knowledge  mapping  is  a tool  that  can  be  used  by  both  instructors  as 
well  as  students  to  meet  the  challenge  of  planning,  organizing,  teaching  and  learning.  Computer-based  knowledge 
mapping  assists  students  in  fulfilling  quality  outcomes  in  learning  business.  Used  traditionally,  concept  maps 
provide  concrete  visual  help  to  organize  information  to  be  learned.  Business  instructors  for  lower  division  courses 
at  colleges  who  have  been  exposed  to  this  tool  are  finding  that  they  provide  a logical  basis  for  deciding  which  main 
ideas  to  include  in  a course.  Computer-based  knowledge  maps  can  be  as  large  as  the  entire  course,  one  or  more  unit, 
or  even  a single  class  period. 

It  has  been  shown  before  that  traditional  concept  maps  are  valid  vehicles  for  documenting  and  exploring  conceptual 
change  (Wallace  & Mintzes,  1990).  This  proposed  model  of  computer-based  qualitative  and  quantitative  knowledge 
maps  that  is  currently  used  as  a pilot  in  a limited  number  of  chemistry  classes,  in  conjunction  with  authors 
experience  in  practical  using  concept  maps,  have  provided  convincing  evidence  that  many  instructors  would  be  able 
to  present  more  continuous  learning  experience  for  their  students,  who  in  turn  would  benefit  from  CBKMs  as 
described  above.  Computer-based  knowledge  maps  can  be  a powerful  tool,  that  contributes  to  the  other  learning 
techniques  and  assessment  methods,  and  addresses  both  knowledge  and  its  organization. 
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Research  on  field  dependence/independence  in  the  learner  reveals  a number  of  interesting  factors  about  preferred 
learning  modalities.  The  field  independent  learner  is  most  typically  intrinsically  motivated,  for  example,  while 
highly  field  dependent  people,  who  are  more  socially  oriented,  are  motivated  by  external  rewards.  Additionally, 
and  more  important  for  the  design  of  on-line  instruction,  field  dependent  learners  require  and  prefer  more 
structure,  direction,  and  support  from  authority  figures  and  colleagues  than  field  independent  learners. 

Field  independent  learners  are  able  to  discriminate  key  concepts  in  the  material  to  be  learned  or  analyzed  to 
construct  a "big  picture"  of  the  material  with  little  auxiliary  assistance.  Field  dependent  learners,  on  the  other 
hand,  prefer  to  proceed  step  by  step  and  are  most  likely  to  benefit  from  outlines  and  advance  organizers  to  have  the 
context  constructed  for  them. 

Fortunately,  measuring  field  independence/dependence  is  uncomplicated.  The  Group  Embedded  Figures  Test 
(GEFT)  measures  the  cognitive  style  dimension  of  field  independence  and  field  dependence.  The  GEFT  consists  of 
18  complex  figures.  Individuals  must  find  a simple  embedded  geometric  figure  that  is  hidden  in  a complex  one 
(Donlon,  1977;  Thompson  & Meloncon,  1987;  Willard,  1985  and  Witkin,  Oltman,  Raskin  & Karp,  1971).  The 
figures  are  easily  picked  out  by  field  independent  learners;  field  dependent  learners  have  more  trouble  isolating  the 
figure  from  its  background.  The  GEFT  can  be  administered  in  under  one-half  hour  and  can  help  instructors 
identify  students’  tendencies  towards  field  dependence  or  independence. 

In  the  summer  of  1999,  we  taught  a graduate  level  class  that  was  conducted  entirely  on  line.  That  is,  we  had  no 
face-to-face  meetings  with  students  and  conducted  all  communication  through  e-mail,  chat  rooms,  bulletin  boards, 
and  web  sites.  This  was  the  first  Web-based  course  conducted  in  the  department,  and  therefore  we  knew  this 
experience  would  be  the  first  of  its  kind  for  all  of  our  students.  We  found  that  1 2 of  the  1 6 students  enrolled  in  the 
class  were  field  independent. 

Our  approach  to  design  of  the  site,  course  requirements,  course  activities,  and  communication  came  from  a 
constructivist  perspective  (Duffy  & Cunningham,  1996;  McManus,  1996).  The  site  was  divided  into  twelve 
primary  sections;  Welcome,  Navigating  this  Site,  Course  Overview,  Calendar  of  Events,  Syllabus,  Grading, 
Outcomes  and  Criteria,  Class  Project  Topics,  Sample  Project,  Participant  menu,  Bulletin  Board-Chat-Surveys,  and 
Resources.  Being  aware  of  the  literature  concerning  field  dependence/independence,  we  purposefully  designed  the 
site  to  provide  for  the  needs  of  both  types  of  learners,  giving  detailed  instructions,  examples,  and  templates  in  more 
than  one  context  to  provide  maximum  support  for  all  field  dependent  individuals.  For  example,  we  provided  two 
locations  that  specifically  detailed  all  course  requirements  in  locations  that  were  identified  by  traditional  names. 
Syllabus  and  Outcomes  and  Criteria.  Likewise,  due  dates  for  class  assignments  were  identified  in  multiple 
locations;  Calendar  of  Events  and  under  each  Outcome  section.  We  included  a resources  area,  giving  detailed 
instructions  on  obtaining  email  accounts,  designing  a web  site,  and  accomplishing  a web  search. 

One  of  the  course  requirements  was  that  students  prepare  a web  page  that  summarized  findings  of  an  extensive 
search  for  information  on  a topic  they  selected  from  a provided  list.  Since  this  was  not  a class  in  designing  a Web 
site,  students  were  given  the  opportunity  to  attempt  development  of  the  site  by  themselves,  seek  out  the  assistance 
of  an  individual  they  knew  with  experience  in  Web  site  creation,  or  send  all  graphics  and  text  files  from  Word  files 
and  we  would  place  the  materials  on  the  course  Web  site  for  them. 

The  record  of  students’  interactions  with  the  instructors  reveals  interesting  interaction  patterns.  While  all  students 
were  a bit  apprehensive  at  the  start  of  class,  Field  independent  learners  tended  to  proceed  with  comfort  and 


confidence  very  quickly.  Field  dependent  learners’  questions  and  messages  increased  significantly  in  number  over 
those  from  field  independent  learners  and  tended  to  express  discomfort  or  fear  about  navigation,  competence,  and 
about  the  formal  structure  of  their  own  web  page,  even  though  the  template  was  available  at  all  times. 

Research  also  reveals  differences  between  field  dependent  and  field  independent  learners'  satisfaction  with 
collaborative  learning.  A study  conducted  by  Whyte,  Knirk,  Casey,  and  Willard  (1991)  investigated  satisfaction 
and  achievement  factors  among  students  paired  for  a learning  experience.  Pairs  included  two  field  independent 
learners,  two  field  dependent  learners,  and  mixed  pairs.  The  results  indicate  that  mixed  pairs  achieved  the  highest 
results,  followed  by  pairs  of  field  independent  learners  and  then  pairs  of  field  dependent  learners.  Field  dependent 
learners,  who  have  a higher  likelihood  of  preferring  social  interaction,  were  most  satisfied  with  collaboration. 

Field  independent  pairs  showed  dissatisfaction  with  pairing  for  collaboration. 

Inclusion  of  collaborative  projects  and  on-line  chats  in  WBI  is  often  incorporated  into  class  design.  During  the 
WBI  class  we  taught,  we  placed  four  to  five  students  in  working  groups  for  the  purpose  of  sharing  their  summaries, 
discussing  common  problems  in  project  completion  and  resources,  sharing  common  experiences  on  specific 
questions  revealed  in  the  summaries,  and  conducting  on-line  chats  to  discuss  the  issues.  This  group  work  was 
accomplished  through  both  synchronous  and  asynchronous  means.  The  learner  experiences  in  a group  setting, 
indicated  that  field  dependent  individuals  relied  on  feedback  from  their  peers  significantly  more  frequently  than 
did  field  independent  individuals.  The  statistics  from  these  observations  will  be  shared  in  the  presentation. 

We  propose  to  present  the  results  of  these  two  studies,  participant  experiences,  guidelines  for  improving  the  Web- 
Based  Instructional  learning  experience  from  a perspective  of  page  design,  determination  of  graded  events,  and 
communication  issues  for  both  types  of  learners. 
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Abstract  We  live  and  learn  in  a world  filled  with  information.  To  prepare  students  and 
employees  for  the  competition  and  globalization  of  tomorrow's  world,  universities  and 
enterprises  are  faced  with  an  increasing  need  of  up-to-date  information  and  learning  material  at 
reasonable  costs.  Because  of  the  fast  changing  requirements  of  professionals,  lifelong  learning 
is  required  in  practically  all  areas,  especially  those  related  to  information  technologies.  The 
objective  of  the  project  OR-World  is  to  respond  to  today’s  increased  education  need  and 
develop  a World  Wide  Web  based  framework  for  sharing  high-quality  learning  material 
between  universities,  educational  institutes,  self-learners,  and  interested  companies,  in  various 
countries. 


Project  Description 

Although  the  World  Wide  Web  has  experienced  an  unprecedented  growth  during  the  last  five  years,  its  potentials 
are  by  far  not  used  optimally  yet.  In  the  world-wide  digital  network,  teachers  and  learners  do  not  have  to  be 
physically  present  at  same  place  and  time  in  order  to  be  in  contact  and  interact.  The  availability  of  electronic 
communication  and  network-structured,  hypermedia  content  opens  up  completely  new  ways  of  higher  education 
in  universities  and  education  centers.  The  project  OR-World  has  been  proposed  in  the  European  5‘‘"  Framework 
Programme  and  will  be  funded  by  the  European  Community  (EC).  The  goal  of  the  project  OR-World  is  to 
achieve  innovations  in  areas  such  as  hierarchic  structuring  of  content  modules  and  flexible  thematic  combination 
(metastructure)  of  content  modules,  use  of  case  studies  within  a learning  network,  and  distributed  self  assessment 
and  grading.  The  project  consortium  consists  of  seven  partners,  four  universities  and  three  companies. 

A central  goal  of  the  project  is  the  development  of  a user-friendly.  World  Wide  Web  based  framework  to 
generate  hypermedia  content  for  distributed,  collaborative  learning.  The  framework  can  be  used  to  create 
learning  environments  that  enable  education  and  training  centers  to  share  reusable  learning  objects  and  maintain 
integrated  learning  services.  With  the  term  “learning  objects”  we  refer  to  any  digital  or  non-digital  entity,  which 
can  be  used,  re-used  or  referenced  during  any  learning  activity,  according  to  lEEE-Standards  (IEEE  1999).  An 
important  issue  when  dealing  with  learning  objects  is  granularity.  In  the  project,  we  will  work  on  several 
granularity  levels.  This  means  that  the  general  framework  will  support  the  following  building  blocks  to  be  used 
in  constructing  flexible,  reusable  learning  environments: 

• a media  element^  refers  to  text,  animation,  simulation,  video  or  audio  sequence, 

• a learning  element  comprises  one  or  more  media  elements, 

• a content  module  consists  of  one  or  more  learning  objects,  and  is  understood  as  a node  in  the  hypermedia 
network, 

• a thematic  metastructure  defines  guidelines  how  to  use  content  modules  to  build  thematic  structures  relevant 
for  a specific  study  goal;  such  a structure  can  be  put  together  in  individual  ways,  thus  adapting  to  different 
combinations  and  profiles. 


Thus,  a teacher  can  set  up  his  or  her  own  content  modules  and  offer  them  to  others  via  the  Internet.  We  suggest 
to  build  a consortium  of  trusted  members,  who  want  to  work  together,  thus  extending  their  own  learning 
materials  offering  in  a meaningful  way.  Above  these  content  modules,  various  thematic  networks  can  be  built, 
which  represent  larger  learning  units,  like  a course  or  guided  tour.  The  reusability  will  make  it  possible  to  create 
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a “web  of  networks”  using  the  same  building  blocks  but  setting  different  viewpoints  and  preferences.  The  objects 
at  each  hierarchy  level  are  tagged  with  metadata  to  facilitate  their  flexible  use  in  different  contexts.  The 
framework  will  be  usable  in  a multilingual  environment.  It  will  support  networked  communication  between 
tutors  and  learners  outside  the  classroom.  The  framework  will  provide  a general  structured  environment  to  input 
any  (structured  or  semi-structured)  contents.  A certain  level  of  structuring  of  the  learning  material  is  a 
prerequisite  for  applying  the  framework.  Teachers  will  be  able  to  use  the  framework  in  creating  their  own 
specific  learning  environments  where  they  can  combine  their  own  learning  objects  with  those  generated  by 
others.  Thus,  there  will  be  a metastructure  above  the  hyperspace  of  learning  objects  allowing  multiple  views  on  a 
certain  subject  area. 

Project  Objectives 

Within  the  project,  use  of  the  framework  will  be  demonstrated  with  contents  of  the  subject  area  Operations 
Research/Management  Science  (OR/MS).  However,  the  framework  itself  will  be  content-independent  and  usable 
to  teach  most  content  areas.  It  will  be  suited  for  using  by  a consortium  of  teachers  (or  learning  institutes)  who  are 
willing  to  share  material  and  are  convinced  about  the  quality  of  all  partners,  thus  allowing  a flexible  extension 
scheme  by  each  member  without  bureaucracy.  Furthermore,  if  several  universities  and  companies  adopt  the 
framework  and  methodology,  they  will  be  able  to  share  a learning  environment  and  strengthen  their  co- 
operation, thus  providing  high-quality  teaching  and  learning  materials.  The  project  OR-World  itself  gives  an 
example  of  sharing  a learning  environment,  it  involves  four  universities  from  three  different  countries,  all 
offering  courses  in  joint  central  subject  areas. 

Case  studies  are  a well-known  interactive  teaching  and  learning  method,  not  only  in  business  and  management, 
but  also  in  various  other  fields,  like  engineering,  law,  agriculture,  etc.  (Erskine  98).  OR-World  will  offer  a 
concept  and  toolset  to  facilitate  the  use  of  cases  in  a virtual,  distributed  learning  and  teaching  environment.  Over 
one  hundred  years,  the  case  method  has  proven  to  be  successful;  we  aim  to  bring  the  method  over  to  the 
environment  of  computer-supported  distance  learning.  However,  we  do  not  believe  that  cases  can  be  used  totally 
without  personal  communication  between  learners  and  instructors.  Thus,  a goal  of  the  project  is  to  develop  new 
guidelines  and  tools  in  order  to  achieve  an  optimal  organizational  form  for  the  use  of  case  study  teaching  within 
the  virtual  network. 

Another  objective  of  the  project  is  to  carry  out  a large-scale  experiment  to  promote  European  integration  and  to 
develop  a European  identity  for  the  subject.  Thus  the  participating  four  universities  and  three  companies  will 
work  closely  together,  each  providing  specific  individual  strengths  either  in  the  area  of  emerging  digital 
technologies  or  content  production  and  structuring.  The  learning  objects  will  allow  new  schemes  for  virtual 
collaborative  learning  to  be  tested  in  cooperative  joint  university-industry  education  frameworks. 

Because  the  initial  standard  Web  markup  language  HTML  does  not  support  efficient  structuring,  retrieval,  and 
linking  among  very  large  amounts  of  documents,  more  general  technologies,  such  as  the  extensible  markup 
language  XML  and  XSL,  are  emerging.  XML  responds  to  a strong  need  to  separate  content  of  a document  (text, 
graphics,  video  etc.)  from  its  logical  structure  (content  table)  and  representation  (formatting  on  the  screen  or 
printer).  The  project  OR-World  Nvill  use  XML  on  each  granularity  level  to  describe  the  elements  and  hierarchic 
structure  of  the  learning  hyperspace.  The  IEEE  Working  Group  PI 484. 12:  Learning  Objects  Metadata  Working 
Group,  is  currently  working  on  standards  for  describing  learning  objects  in  terms  of  metadata.  According  to  the 
working  group,  “This  standard  will  specify  the  syntax  and  semantics  of  Learning  Object  Metadata,  defined  as  the 
attributes  required  to  fully/adequately  describe  a Learning  Object.  Learning  Objects  are  defined  here  as  any 
entity,  digital  or  non-digital,  which  can  be  used,  re-used  or  referenced  during  technology  supported  learning. 
Examples  of  technology  supported  learning  include  computer-based  training  systems,  interactive  learning 
environments,  intelligent  computer-aided  instruction.  The  proposed  metadata  scheme  will  be  used  in  the  OR- 
World  project  to  facilitate  the  re-use  of  existing  material. 

To  demonstrate  and  test  the  power  of  the  OR-World  concept,  we  will  implement  some  contents  from  the 
interdisciplinary  area  Operations  Research/Management  Science.  The  exemplary  implementation  will  comprise 
the  whole  functional  spectrum  including  multilingual  content  components,  distributed  learning,  self  assessment, 
and  grading. 
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Abstract:  This  paper  describes  the  transformation  of  a traditional  medical  curricula  into  a 
problem-based  learning  technology-enhanced  educational  setting.  The  use  of  Information 
Technology  (IT)  is  an  important  feature  of  the  new  curriculum.  Case-based  medical  problems 
are  delivered  to  students  in  a tutorial  setting  via  TopClass  over  the  Web.  Additional  learning 
resources  such  as  Web  sites,  image  banks,  practical  experimental  simulations,  self- 
assessment  tests,  Shockwave  learning  modules  and  multimedia  modules  are  also  accessed  via 
the  computer.  This  paper  outlines  the  critical  attributes  of  this  educational  context,  and  then 
discusses  the  web-based  and  computer  facilitated  learning  (CFL)  based  aspects  of  the 
curriculum. 

Educational  Context 

Traditionally,  the  medical  course  at  The  University  of  Melbourne  has  been  taught  using  a discipline-based 
approach.  In  the  early  years  of  the  course,  students  undertook  discrete  subject  blocks  from  the  pre-clinical 
departments  of  Anatomy  and  Cell  Biology,  Biochemistry  and  Molecular  Biology,  Microbiology  and 
Immunology,  Pathology,  Pharmacology  and  Physiology.  Exposure  to  clinical  scenarios  in  situ  came  later  in 
the  course  (predominantly  in  years  4-6),  after  students  had  gained  an  understanding  of  the  basic  sciences 
relevant  to  medicine.  Internal  review  mechanisms  and  student  feedback  in  recent  years  highlighted  a number  of 
deficiencies  in  the  traditional  course.  These  included  insufficient  integration  between  basic  and  clinical 
science,  insufficient  attention  to  communication  skills,  problem  solving  skills  and  social  aspects  of  health, 
an  overload  of  biomedical  detail  and  unnecessary  duplication  of  content.  In  an  effort  to  incorporate  current 
theories  of  medical  education  a new  medical  curriculum  was  introduced  in  1999.  The  pedagogical  model  for 
the  new  medical  curriculum  incorporates  elements  of  Problem-Based  Learning  (PBL)  and  Self-Directed 
Learning  (SDL)  (Koschman,  Kelson,  Feltovich  & Barrows,  1996).  The  primary  focus  of  learning  in 
Semesters  2-5  will  be  medical  problems  (referred  to  as  ‘Problems  of  the  Week’)  which  are  presented  to 
students  in  small  group  tutorial  settings.  Key  features  of  the  new  curriculum  are  the  horizontal  integration 
across  disciplines  and  the  vertical  integration  of  clinical  situations  with  basic  scientific  material  (Keppell, 
Elliott,  & Harris,  1998). 

The  use  of  Information  Technology  (IT)  is  an  important  feature  of  the  new  curriculum.  IT  is  utilised  to 
deliver  medical  content  in  three  ways.  These  include  the  use  of  a Learning  Framework  (TopClass);  the  web- 
based  problem  of  the  week  embedded  in  TopClass  and  multimedia  stand-alone  modules.  The  TopClass 
Learning  Framework  provides  a central  access  point  for  students  to  enter  the  on-line  course  work,  complete 
self-assessment  tests,  view  class  announcements,  participate  in  discussion  groups  and  send  and  receive 
messages  from  teachers  or  peers.  The  problems  of  the  week  are  delivered  to  student  tutorials  over  the  web  via 
TopClass.  At  the  completion  of  the  first  year  of  the  medical  course  18  problems  of  the  week  have  been 
designed,  developed  and  implemented.  Approximately  160  problems  of  the  week  are  required  throughout  the 
entire  medical  curriculum.  Self-directed  learning  resources,  in  the  form  of  multimedia  teaching  and  learning 
modules,  are  also  required  to  support  the  core  content  of  the  problems.  Approximately  40  modules  are 
currently  under  development  and  it  is  envisaged  that  80  modules  will  be  required  to  support  the  curriculum 
(see  http://www.medfac.unimelb.edu.au/bmu/proiects/survey.asp).  This  paper  will  describe  and  demonstrate 
the  features  of  one  Problem  of  the  Week  and  one  computer  facilitated  learning  (CFL)  module.  Through  this 
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demonstration,  the  way  in  which  IT,  interactive  multimedia  and  web  based  resources  are  being  used  to  support 
the  new  integrated  curriculum  will  be  shown. 

Computer-based  Problems  of  the  Week 

The  problem  of  the  week  begins  with  the  “trigger”.  The  aim  of  this  photograph  or  video  sequence  is  to  set- 
the-stage  for  the  student  by  providing  a visual  of  the  hypothetical  patient  and  the  circumstances  surrounding 
the  case/problem.  In  development  we  attempt  to  match  the  appropriate  media  type  (video,  Shockwave  movie, 
photograph)  to  the  nature  of  the  medical  condition  or  context  which  needs  to  be  displayed.  For  example 
medical  triggers  which  need  to  portray  a medical  condition  such  as  myasthenia  gravis  need  to  show  the 
progressive  nature  of  the  fatigue  that  occurs  in  skeletal  muscle.  Consequently  video  is  the  most  appropriate 
media  type  for  illustrating  this  progressive  nature.  Other  medical  triggers  which  need  to  demonstrate 
distinctive  changes  in  a sequence  can  be  portrayed  through  a series  of  photographs  using  a Shockwave  movie. 
In  other  medical  triggers  a single  photograph  is  sufficient  for  conveying  the  context  and  the  necessary 
information  required  for  beginning  the  problem-based  learning  approach.  Following  the  trigger,  students  are 
instructed  to  list  information  about  the  patient,  identify  the  presenting  problems,  list  possible  causes  of  each 
problem  (hypotheses),  provide  a rationale  for  each  hypothesis,  prioritise  the  list  of  hypotheses  and  then 
determine  what  other  additional  information  (physical  examinations,  laboratory  tests,  etc)  is  required  to 
differentiate  between  the  hypotheses.  During  this  process  students  are  given  supporting  information  in  the 
form  of  a history,  past  medical  history,  physical  examination,  progress,  investigation  results.  An  example  of 
a problem  of  the  week  for  the  medical  curriculum  is  shown  at: 
http://www.medfac.unimelb.edu.au/Med/examplePQW/trigger.html 

Multimedia  Modules 

An  important  aspect  of  our  approach  is  the  emphasis  on  interactive  CFL  modules.  These  multimedia  teaching 
modules  are  used  extensively  in  the  new  course  as  learning  resources  in  areas  that  students  have  traditionally 
had  difficulty  understanding  or  in  areas  where  the  use  of  media,  such  as  video,  audio  or  animation  is 
particularly  appropriate  to  demonstrate  a concept  or  principle.  The  emphasis  of  these  teaching  modules  is  on 
complementing  the  learning  objectives  of  the  problem  of  the  week. 

For  example,  we  have  designed  and  developed  a multimedia  module  which  examines  the  communication 
process  between  doctors  and  patients  and  focusses  on  integrating  biological,  psychological  and  social  factors 
in  clinical  diagnosis.  When  investigating  a patient’s  complaint  (such  as  chronic  tiredness)  the  body  language 
and  emotional  response  of  a patient  is  often  as  important  as  his  or  her  verbal  response.  Video  is  an  ideal 
medium  to  capture  both  verbal  and  nonverbal  responses  and  to  portray  these  for  students.  The  module  we  have 
developed  allows  medical  students  to  listen  to  a doctor’s  questioning  method  and  to  see  how  question  tone  and 
wording  affect  a patient’s  reactions  and  demeanour.  Students  are  able  to  view  a patient’s  response  and  answer 
questions  about  their  perceptions  and  undertanding  of  the  clinical  interview.  The  introduction  of  this  module 
into  first  year  medicine  should  begin  to  sensitise  students  to  the  complexities  of  the  communication  process 
between  doctor  and  patient.  It  is  hoped  that  this  will  enhance  their  own  clinical  interactions  in  the  long  term. 
The  presentation  associated  with  this  paper  will  demonstrate  this  module  as  an  example  of  the  types  of 
interactive  multimedia  that  are  being  used  to  support  the  new  curriculum. 
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A Cenceptual  Framework  for  Web-Based  Authoring  System 


Badrul  Khan,  The  George  Washington  University,  USA 


After  reflecting  on  the  various  factors  critical  to  open  and  flexible  learning  environments,  Khan  (1997) 
developed  a Framework  for  Web-based  learning.  Various  WBL  factors  are  clustered  into  the  following 
eight  dimensions:  pedagogical,  technological,  interface  design,  evaluation,  management,  resource  support, 
ethical  and  institutional.  AuthorWeb  - a comprehensive  authoring  system  is  rooted  in  the  WBL  framework. 
The  inception  of  AuthorWeb’s  development  began  with  a vision  of  providing  a tool  that  would  assist 
educators,  trainers  and  course  developers  to  create  meaningful  online  courses  without  prior  skills  or 
knowledge  in  instructional  design,  computer  programming  and  issues  critical  to  distance  learning 
environments.  AuthorWeb’s  development  process  benefits  from  the  input  of  various  substantiated 
perspectives,  including  instructional  design,  cognitive  and  educational  psychology,  curriculum 
development,  distance  education,  innovation  and  change,  telecommunication,  database,  artificial 
intelligence,  software  engineering,  and  media  production. 
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Abstract:  These  days,  a variety  of  educational  methods  using  computer  games  have  been  studied 
to  increase  interests  regarding  the  study  using  adventure  game  for  educational  purpose.  If  a part  of 
learning  contents  of  elementary  school  students  is  developed  in  courseware  of  adventure  game  way 
rather  than  conventional  CAl  way  to  apply  it  to  the  class  of  the  students,  learning  effect  is  likely  to 
improve  greatly.  From  the  viewpoint  of  developmental  psychological  features,  elementary  school 
students  are  much  interested  in  courseware  of  game  way.  As  a result,  the  study  suggests  function 
design  of  adventure  game  for  educational  use  considering  developmental  psychological  features  of 
elementary  school  students. 

Adventure  game 

Definition  of  concepts 

Adventure  game  may  be  classified  with  contents  and  methods  in  respect  of  software.  The  one,  who  enjoys  the 
game,  takes  proper  action  of  accidents  or  problems  depending  upon  the  game's  story  to  resolve  them  and  reach  final 
destination.  The  game  shall  include  contents  of  adventure  or  rare  events.  Furthermore,  there  exists  only  one  kind  of 
order  and  solution,  which  should  be  followed  to  solve  the  event. 

General  features  of  adventure  game 

General  features  of  adventure  game  may  be  analyzed  as  follows: 

1)  There  exists  only  one  solution  based  on  preset  scenario. 

2)  Most  of  time  background  occupies  unknown  world. 

3)  A hero  shall  never  die. 

4)  There  exists  a sequence  of  scenario  to  solve  an  event. 

5)  Graphic,  music  and  other  multimedia  effects  are  used. 

Essence  of  computer  game 

Roger  Gaiyo,  a French  sociology  scholar,  has  classified  a play  as  four  kinds(See  Fig.l)  Horizontal  axis  means 

"Does  intention  of  a player?",  while  vertical  axis  does  "Is  there  a rule?". 

The  game  is  said  to  be  a summary  of  four  kinds  of  the  plays  that  Roger  Gaiyo 
has  defined.  In  other  words,  the  elementary  school  students  concentrate  on  the 
game  because  it  includes  a variety  of  competitions  to  allow  them  to  enjoy 
imitation,  luck  and  dizziness  at  the  same  time  while  playing  the  game.  Such  a fact 
can  be  understood  as  follows:  When  each  scene  of  the  game,  which  can  draw 
attention  of  elementary  school  students,  is  analyzed,  it  mostly  includes  four  kinds 
of  play  factors  evenly.  Nonetheless,  elaborate  arrangement  of  competition, 
imitation,  luck  and  dizziness  does  not  always  stimulate  interests  of  the  students  to 
become  an  educational  game. 
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They  are  continuously  so  mad  on  good  games  for  educational  puipose  without  boring  feeling  thanks  to  following 
reasons:  1)  continuous  supply  of  stimulus  enough  to  create  motivation,  2)  the  stimulus  considering  educational 
accomplishment  that  includes  qualitative  change.  In  other  words,  to  design  a good  game,  quantity  of  the  stimulus 
shall  be  effectively  supplied,  and  furthermore  quality  of  the  stimulus  shall  properly  change. 

Function  design  of  adventure  game  for  educational  purpose 

Factors  of  adventure  game  for  educational  purpose 

The  computer  game  may  be  used  in  various  ways  depending  upon  purposes  and  contents  of  learning,  while 
adventure  game  for  educational  purpose  can  be  used  more  effectively  in  the  learning  contents  requiring  empirical 
situation.  The  learning  contents  may  be  presented  to  students  naturally  through  empirical  situation  to  let  them  learn 
effectively  through  their  experience. 

Factors  of  adventure  game  for  educational  purpose  shall  be  understood  as  the  ways  of  not  only  developmental 
psychological  features  but  also  computer  game  to  improve  their  study  efficiency  and  make  use  of  the  game  as  an 
effective  media.  Major  factors  of  the  game  are: 

1)  The  game  shall  be  easy  to  make  use  of  it  through  easy  and  convenient  visual  design,  and  shall  be  equipped  with 
various  kinds  of  events  to  let  students  be  interested  in  it.  Furthermore,  it  shall  include  various  kinds  of  educational 
strategies  and  study  methods. 

2)  They  shall  be  allowed  to  learn  different  game  stage  depending  upon  individual  difference.  The  game’s  process 
shall  include  curriculum  and  training  procedure. 

3)  Outcome  of  education  shall  be  shown  in  accordance  with  development  of  the  game  by  stages. 

4)  Conditional  and  unconditional  divergence  function  shall  allow  each  user  to  play  game  depending  upon  his  or  her 
feature. 

5)  Flexible  extent  of  permission  of  exactness  is  to  be  given  against  a user’s  response. 

6)  Progress  of  the  game  shall  be  recorded,  managed  and  maintained. 

7)  If  possible,  cooperative  game  through  network  shall  be  done. 

8)  They  have  to  learn  ’’social  nature”  while  playing  a game  for  multiple  players. 

9)  The  game  shall  maintain  its  activities  continuously,  and  furthermore  shall  be  rewarded  or  punished  depending 
upon  outcome  of  their  actions. 

10)  Students  shall  be  allowed  to  repeat  to  play  a part  of  the  game  that  they  want  to  do. 

1 1)  Elementary  school  students  are  easy  to. give  up  or  express  strong  rejection  when  they  are  unable  to  play  game 
smoothly  or  do  not  understand  it  enough.  As  a result,  they  have  to  be  given  Help  or  Tip  menu  to  help  them  solve 
problems. 

Functions  for  adventure  game  for  educational  purpose 

1 ) Mouse 

□ Movement 

- Rotation  mode:  360°rotation,  adjacent  area,  up/down  and  left  hand/right  hand. 

- Movement  mode:  Walking  and  running  each  one  step  forward. 

- Jumping  mode:  Jump  over  obstacles  forward. 

□ Selection 

“ Move  mouse  to  change  its  shape  and  select  or  pick  up  an  object. 

“ Select  a variety  of  options. 

2)  Keyboard 

□ Enter  user’s  information.  □ Supply  an  abbreviated  command  key. 

3)  Help  mode 

They  may  select  ”Be  quiet”,  ’’Common”  and  ’’Talkative”  of  the  game  environment  to  let  Help  mode  to  give  a 
suitable  tip  while  playing  game. 

4)  Zoom-in  and  zoom-out  mode:  The  mode  enables  a player  to  watch  surrounding  area,  environment  or  objects 
carefully.  5)  Chatting  with  other  characters. 

6)  Interactive  mechanism  mode:  Tlie  mode  enables  a player  to  pick  up,  collect  or  make  use  of  an  object. 

7)  Time  and  space  movement:  A player  is  allowed  to  move  under  a variety  of  time  and  space  zones. 

8)  Bonus  and  penalty:  When  a character  faces  a certain  situation,  he  or  she  is  given  bonus  or  penalty  to  carry  out 
job  in  easy  or  difficult  way. 
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Abstract:  Lifelong  learning  denotes  learning  as  a process,  which  continues  after  leaving  school. 
This  perspective  has  a far-reaching  influence  on  conceptual  design  models  of  educational  software. 
It  regards  learning  as  one  aspect  of  work  and  opens  the  way  to  including  concepts  from  industrial 
psychology  into  software  design.  This  paper  will  discuss  factors  influencing  the  learning  process, 
such  as  burden,  demand  level  or  motivation,  and  the  resulting  consequences  for  the  development  of 
educational  software. 


Lifelong  Learning 

Lifelong  learning  denotes  that  learning  does  not  end  after  finishing  school  or  university.  More  and  more  people  have 
to  attend  courses  in  further  and  continuing  education  to  keep  their  knowledge  level  and  market  value  or  to  gain 
knowledge  for  another  kind  of  professional  work.  Situations  demanding  for  learning  will  be  pervasive  in  people’s 
life  and  in  consequence  the  division  between  education,  work  life  and  leisure  time  will  fade. 

Learning  is  one  aspect  of  work 

Understanding  lifelong  learning  as  a permanent  process  also  involves  the  perspective  on  learning  as  one  aspect  of 
work.  This  view  allows  us  to  integrate  concepts  from  industrial  psychology  into  conceptual  models  shaping  the 
design  of  educational  software.  Industrial  psychology  has  a holistic  view  on  human  nature.  Humans  are  supposed  to 
search  for  self-realization  and  autonomy,  which  should  be  supported  by  changes  in  work  places  and  organizational 
structures  in  order  to  enlarge  the  individual’s  area  of  responsibility.  It  involves  personality  development,  formed  in 
the  individual’s  tackling  with  challenges  provided  by  solving  the  tasks  at  work. 

Industrial  psychology  deals  with  aspects  of  qualification.  Learning  tasks  are  viewed  as  a sort  of  chain,  where  burden 
and  demands  alter  with  effects  on  the  change  of  mental  states.  In  order  to  initiate  successful  and  effective  learning 
processes  it  is  most  important  to  find  an  adequate  demand  level,  which  causes  positive  mental  states  like  happiness, 
motivation,  increase  in  performance  and,  in  the  long  distance,  even  qualification  and  positive  development  of 
personality.  On  the  other  side,  inadequate  demand  levels  cause  negative  and  undesirable  mental  states,  e.g.  a feeling 
of  tiredness,  drop  in  performance,  frustration,  and  even  psychosomatic  illness. 

Lifelong  Learners 

It  is  a challenge  to  develop  educational  software,  which  is  adaptable  to  burden  and  demand  level.  Software  designers 
have  to  find  concepts  and  techniques  to  make  software  as  flexible  and  interactive  (Fischer  1996)  as  to  adapt  to 
altering  demand  levels.  One  step  into  this  direction  is  to  orient  teaching  strategies  to  the  characteristics  of  groups  of 
learners,  which  may  be  deduced  by  taking  into  consideration  the  context  of  learning  and  the  purpose.  One  can 
distinguish  different  learner  groups.  For  example,  learner  group  1 wants  to  do  basic  studies  in  the  domain  in  order  to 
pass  exams  and  earn  diploma.  The  learners  want  to  study  all  modules  relevant  to  the  curriculum,  need  guided  tours 
to  differ  relevant  from  irrelevant  knowledge  with  respect  to  passing  exams.  Learner  group  2 does  not  want  to  go 
through  all  the  knowledge  modules  offered  in  the  database,  but  wants  to  study  only  some  modules  for  task-oriented 
knowledge  acquisition.  Often  a current  problem  or  a knowledge  gap  is  the  starting  point  for  learning.  Learner  group 
3 is  characterized  by  exploring  the  knowledge  domain  according  to  personal  interests.  The  behavior  of  learners  in 
this  group  is  comparable  to  browsing  libraries,  reading  books  and  magazines. 
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Conceptual  Modeling 


The  important  role  of  the  conceptual  model  for  the  development  of  educational  software  has  been  strengthened  by 
didactics  as  well  as  by  software-ergonomics  (Herczeg  1994).  With  this  background  in  mind,  we  want  to  ask  what 
kind  of  conceptual  models  do  developers  of  educational  software  have  in  mind. 

Theories  of  Learning  and  Teaching 

Theories  of  learning  and  teaching,  which  are  mainly  derived  from  psychology,  normally  shape  the  conceptual 
models  of  educational  software  developers.  Examples  are  behaviorism,  cognitive  information  processing,  theories 
on  constructive  and  situated  learning.  In  the  early  phase  of  computer-based  training,  dominated  by  behaviorism, 
reinforcement,  serial  ordering,  small  chunking  were  seen  as  the  basic  mechanism  enabling  learning  processes. 
Information  processing  approaches  focused  on  cognitive  aspects  of  knowledge  representation  like  conceptualization 
or  schemata.  Other  theories  strengthen  the  vital  role  of  experience  for  the  learner’s  perception  and  activities  in 
knowledge  construction,  which  change  cognitive  structures  (Bonner  1998),  stressing  the  importance  of 
communication  and  social  negotiation  as  inevitable  for  learning  processes  to  occur. 

Task  Analysis 

Learning  theories  focus  on  either  behavioral  or  on  cognitive  aspects  of  the  learning  process.  However,  software- 
ergonomics  claims  that  conceptual  modeling  should  also  take  into  consideration  the  domain,  the  user’s  skills  and 
experiences,  certain  cognitive  aspects  (like  memory  capacity,  laws  of  perception  etc.),  and  also  the  task  and  the 
context  of  completing  the  task.  Especially  task  and  task  contexts  have  been  neglected  by  current  psychological 
theories.  Task  analysis  intends  to  reveal  relevant  characteristics  of  the  learner  and  her  or  his  task  (Herczeg  1999). 
Some  aspects  of  the  learner’s  task  have  been  discussed  in  the  section  on  lifelong  learners.  The  three  groups 
mentioned  above  may  further  differ  according  to  aspects  like  previous  knowledge,  learning  style  and  motivation.  For 
example,  not  the  aspect  of  being  motivated  is  important,  but  the  reasons  that  cause  motivation.  That  is,  the 
distinction  between  intrinsic  and  extrinsic  motivation,  as  intrinsic  motivated  learners  will  hardly  ever  need  external 
stimuli,  whereas  others  do. 

Summary 

Regarding  lifelong  learning  as  one  aspect  of  work  enriches  developer’s  conceptual  models  with  concepts  from 
industrial  psychology.  An  important  aspect  is  task  analysis  in  order  to  shape  the  task  according  to  demand  levels. 
Task  orientation  means  that  interests  and  engagement  are  caused  by  characteristics  of  the  task  itself.  The  conditions 
are  to  give  a person  as  much  control  as  possible  over  the  subtask  and  over  the  devices  for  task  completion. 
Furthermore,  the  task  has  to  be  structured  in  a way  that  the  level  of  burden  and  demands  is  adequate.  Too  simple  or 
less  structured  tasks  will  result  in  monotony  or  disappointment.  More  research  on  this  subject  is  needed  in  order  to 
develop  educational  software  with  a more  holistic  model  of  the  learner,  and  by  the  way  improving  job  and  learning 
conditions,  and  last  but  not  least  preventing  illness. 
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Abstract:  This  presentation  addresses  the  debate  between  quality  of  instruction  issues 
of  adjunct  versus  lull  time  faculty.  With  an  almost  40%  increase  in  the  numbers  of 
contract  and  adjunct  faculty  in  all  institutions  in  the  past  decade  (Hancock,  1998),  very 
real  questions  need  to  be  raised  about  the.technology  development  of  this  cohort  of 
faculty  and  their  abilities  to  transmit  critical  technology  skills  and  information  to 
university/college  students. 


Introduction 

While  considerable  efforts  are  being  made  to  provide  professional  development  opportunities  with  the  full 
time  faculty  (Jacobsen,  1998;  Nance  & Strohmaler,  1998)  very  little  is  being  done  to  provide  information,  skill 
training,  and  experiences  with  technology  for  adjunct  faculty  development.  As  J.  Digranes  reports  (1995),  ” 
Adjuncts  frequently  receive  less  institutional  support  including  e-mail  accounts,  computers,  technology 
training... than  their  full  time  colleagues"(p.  162).  The  impact  of  this  lack  of  training  in  technology  is  felt  by  students 
who  "are  dumped  in  a lab  almost  as  though  it  were  a 'babysitter'  with  the  inexperienced  and 
ill  trained  adjunct  trying  to  maintain  order"  according  to  Steve  Leone  (1997,  p.  8). 

Purpose.  This  paper/  presentation  examines  the  documented  level  of  competencies  in  technology  of  a large  group  of 
adjunct  faculty  at  an  urban  college  and  describes  a professional  development  program  in  technology  training  that 
enhances  these  faculty's  ability  to  use  technology  in  their  classrooms  to  improve  student  learning. 

Review  of  the  Literature 

Limited  research  documenting  the  adjunct  faculty's  knowledge  and  competency  in  applying  technology  to 
the  classroom  exists  in  the  literature.  This  is  largely  because  institutions  as  well  as  researchers  have  refused  to 
acknowledge  the  growing  role  of  this  group  of  faculty  and  their  subsequent  influence  on  teaching  and  student 
learning.  Like  their  full-time  counterparts,  adjuncts  exhibit  vaiying  degrees  of  knowledge  about  technology.  To  test 
the  specific  knowledge  (cognitive  and  application)  of  adjunct  faculty  at  the  institution,  a survey  was  constructed  and 
delivered  to  a sample  of  the  adjunct  faculty  (10%),  full  time  temporaiy  faculty  (10%)  and  full  time  faculty  (10%). 
The  following  data  reflects  the  significant  differences  between  the  faculty 
responses  to  survey: 

• Despite  a significant  difference  in  the  degree  of  importance  placed  on  knowledge  of  the  Windows  95 
Operating  system  for  all  three  levels  of  faculty  only  6 (adjuncts)  claimed  to  have  no  knowledge  of  this 
basic  operating  system. 

The  adjuncts  were  more  inclined  to  not  see  the  importance  of  Excel  software. 

• A majority  of  all  three  types  of  faculty  said  that  MSWord  was  extremely  important,  a program  that 
they  used  constantly.  However,  of  the  8 people  who  said  is  wasn't  important,  7 were  adjuncts. 

• There  is  a general  proficiency  in  MSWord  for  adjunct  instructors.  MSPowerPoint  like  MSExcel  was 
split  between  those  who  liked  and  used  the  program  and  those  who  didn't.  Its  perceived  importance 
matched  the  existing  skill  level  among  adjuncts  surveyed. 

• Ability  to  use  Web  browsers  is  generally  viewed  as  important,  although  a nine  adjunct  said  they  never 
used  them. 

• Conducting  searches  for  research  was  considered  extremely  important  by  adjuncts.  50%  in  them 
claimed  to  have  excellent  to  mid-range  abilities.  Of  those  who  didn't  have  excellent  to  mid-range  skill, 
majority-adjuncts(38%)). 


• Half  of  the  adjuncts  felt  creating  Web  Pages  was  important.  50%  said  it  wasn't  important. 80%  of  the 
adjuncts  said  they  had  little  to  no  experience  with  this  competency. 

• Over  70%  of  the  adjuncts  had  no  experience  with  Web  applications  at  all. 

• The  use  of  currently  available  classroom  systems  was  viewed  as  important  by  the  majority  of  adjuncts. 
Those  who  said  it  was  unimportant  were  predominately  adjuncts  (80%)..  Eighty-eight  percent  of  all 
adjunct  faculty  claimed  no  experience  with  this  application. 

• Using  multimedia  applications  was  also  viewed  as  important  by  adjuncts.  Approximately  80%  of  the 
adjunct  faculty  had  not  experience  with  this  application. 

Of  the  three  groups  surveyed  in  this  study,  adjuncts  demonstrated  the  fewest  competencies  in  technology. 
In  many  cases,  they  had  little  opportunity  to  develop  those  competencies  through  their  attendance  at  professional 
development  workshops,  seminars,  classes  that  emphasized  these  technology  skills.  There  are  fewer  rewards  for 
these  faculty  to  make  extra  assignments  that  involve  technology  applications 
in  their  courses. 

The  Plan  for  Addressing  this  Need 

To  address  the  lack  of  technology  professional  development  of  adjunct  faculty,  our  institution  applied  for 
and  received  $508,  000  U.S.  Department  of  Education  Title  III  grant.  It  provides  technology  training  (among  four 
other  areas)  for  the  650  adjunct  faculty  at  the  institution. 

Population  and  Research  Model  is  a simple  pre-test,  intervention,  post  test  design  research  model  provides 
the  organizing  function  for  this  grant.  Each  year  approximately  10%  of  the  adjunct  faculty  enroll  in  the  program  of 
workshops,  technology  fellowship  program  and  other  grant  supported  activities.  For  the  first  year,  sixty  adjunct 
faculty  participated.  All  adjunct  faculty  provided  two  forms  of  baseline  data,  student  evaluations  for  two  semesters 
(from  1998  to  1999)  and  one  peer  observation  data  from  the  same  time  period. 

Technology  Competencies  is  used  to  determine  what  technology  education  for  adjunct  faculty  was  needed, 
the  grant  staff  developed  a survey  on  technology  skills.  The  survey  paralleled  that  instrument  used  by  the  college’s 
Technology  Task  Force  and  its  Academic  Technology  unit  for  full  time  faculty  at  the  institution.  Calibrated  by  its 
close  similarity  (questions)  to  those  asked  in  the  AAHE  "Flashlight"  project,  the  instrument  developed  for  adjunct 
faculty  requires  knowledge  of  types  of  software,  hardware  and  applications  of  technology  to  curricular,  classroom 
management  of  instruction  and  technology  infusion. 

Based  on  this  survey,  six  technology  workshops  were  built  for  the  second  year  of  the  grant.  In  September  a 
workshop  plan  and  training  schedule  was  developed.  Instructors  were  secured  from  among  the  full  time  faculty  and 
technology  staff.  Since  many  adjunct  faculty  find  it  difficult  to  attend  workshops  even  when  these  kinds  of 
professional  development  aid  their  knowledge  base  and  make  them  more  valuable  instructors  at  the  institution, 
migrant  adjunct  faculty  have  difficulty  attending  training  (Geppa,  1993).  Multiple  sections  of  each  workshop  had  to 
be  arranged  to  accommodate  the  adjunct  faculty's  schedules.  Since  some  of  this  faculty  cohort  are  already  experts 
(early  adopters)  in  technology,  effort  was  made  to  use  these  faculty  to  assist  other  adjunct  faculty  in  the  development 
of  advanced  training  in  technology  Seven  adjunct  faculty  were  selected  from  those  participating  who  had 
recommended  technology  expertise.  These  adjuncts  were  given  an  additional  contract  to  work  40  hours  during  the 
academic  year  in  small  group  training  sessions  with  adjunct  faculty  needing  technology  assistance. 

Each  adjunct  Technology  Fellow  had  to  develop  a Technology  Assistance  Plan  covering  goals,  goal 
management  and  measurable  outcomes  that  was  negotiated  with  the  dean  of  the  school  in  which  the  adjunct's 
discipline  was  housed.  Each  adjunct  produced  40  hours  of  documented  technology  work  for  which  the  adjunct  was 
paid  $1000.  Several  of  the  adjuncts  included  goals  addressing  the  finalizing  of  courses  on-line.  Other  adjuncts 
specified  training  for  their  adjunct  colleagues  on  a variety  of  topics  including  Web  Authoring,  Researching  on  the 
Internet  and  Multimedia  presentations.  Other  (2)  adjuncts  set  up  one-on-one  sessions  with  other  adjuncts  in 
their  academic  divisions  (sciences  and  humanities).  It  is  evident  that  adjunct  faculty  need  more  assistance  in  both 
learning  specific  technology  programs  and  in  applying  the  technology  into  their  coursework.  The  technology 
fellowship  program  using  the  "Train-the-Trainer"  model  is  yielding  some  remarkable  results  including  increased 
numbers  of  faculty  invested  in  using  the  technology  in  the  classroom  in  research,  assignments  on  the  web,  and  the 
development  of  five  additional  on-line  courses  including  ones  in  biology,  speech  communication  and  sociology. 

In  conclusion,  adjunct  faculty  who  increasingly  instruct  large  numbers  of  undergraduate  students  need  to 
invest  in  faculty  development  activities,  in  particular  technology  training.  Usually  interested  in  their  own 
development,  institutuions  need  to  make  such  training  readily  available  (multiple  sections)  to  adjuncts  and 
encourage  (reward)  them  for  building  competencies  in  it  to  improve  their  students’  learning. 
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Abstract:  The  paper  discusses  factors  that  affect  users’  learning  and  performance  when  they  interact  with  a computer 
interface.  Research  in  Instructional  Systems,  Cognitive  Science,  Human  Computer  Interaction  and  Design  and  Human 
Factors  show  an  overlap  of  findings.  All  disciplines  contribute  to  understanding  how  the  human  and  the  machine  relate  to 
one  another  and  offer  some  very  useful  insights  in  our  quest  to  producing  better  designs  for  multimedia  learning  systems. 


Cognition 

Perception 

Research  in  interface  development  has  shifted  from  perceptual  and  motor  issues  to  cognitive  issues  such  as  perception 
(Aspillaga,  1996;  Winn,  1993),  attention  (Taylor,  1992),  selective  encoding  (Taylor,  1992;  Paivio;  1991;  Hodes,  1994), 
contiguity  effect  (Mayer,  R.  & Sims,  1994),  and  concurrency  of  information  (Paivio,  1991 ) with  implications  to 
learning  and  performance  (Blattner,  1994).  These  factors  influence  a person's  ability  to  process  the  visual  and  verbal 
information  on  a computer  screen.  Research  has  shown  that  a well-designed  presentation  of  information  with 
consideration  to  a person's  cognitive  resource  positively  correlates  with  improved  learning,  retention  and  performance. 
During  ‘perception*  vision  is  not  yet  ruled  by  attention  because  the  viewer  is  confronted  with  processes  occurring 
rapidly  and  simultaneously 

The  eye  therefore,  merely  retains  what  it  visualizes.  For  designers  it  is  important  to  know  what  these  perceptual 
processes  are  because  any  change  in  the  instructional  message  can  lead  the  user  to  unconsciously  misinterpret  the 
message.  One  of  these  early  perceptual  process  is  stimulation  change,  i.e.  change  in  vision  from  light  to  dark  and  vice 
versa.  Another  perceptual  process  is  the  figure-ground  which  explains  why  users  tend  to  create  figures  against  a 
background.  During  the  design,  the  designer  should  make  clear  distinctions  between  the  figure  and  the  ground  by 
maing  the  boundaries  of  an  area  very  clear,  coloring  or  filling  the  bounded  area  and  judging  the  completeness  of  the 
figure. 

Objects  are  perceived  as  complete  shapes  on  the  basis  of  their  arrangement  and  the  way  their  geometric  boundaries 
follow  simple  alignment.  Similarly,  objects  that  line  up,  in  a straight  line  or  on  a curve  are  grouped  together  (Winn, 

1 993  A viewer  is  attracted  to  the  simplest  organization  of  the  parts  of  the  display  explained  by  the  concept  of  emergent 
property  . In  conformance  with  the  Gestalt  principle  of  close  proximity  or  spatial  proximity  i.e.,  parts  of  a message 
that  are  perceived  as  being  close  together  will  be  processed  as  one.  Various  views  have  proliferated  on  where  the  eyes 
tend  to  focus  in  a group  of  images.  Winn  (1993)  cited  the  work  of  Kinchla  and  Wolfe  (1979)  espoused  a middle  out 
approach.  Others  hold  that  people  choose  to  view  an  image  which  is  more  dense  and  detailed.  There  are  others  who 
believe  that  perceptual  organization  naturally  orients  towards  the  horizontal-vertical  overriding  the  perceptual  factors 
of  focusing  attention,  interpretation  and  behavior.  Visual  elements  such  as  shape,  space,  size,  color  and  location  are 
processed  according  to  their  properties.  Perceptual  Grouping  or  Similarity  Grouping,  categorizes  similar  and  different 
Aspillaga  cited  Kawabata's  work  to  emphasize  that  the  density  and  orientation  of  elements  affect  their  perceptual 
grouping.  This  research  proves  that  perceptual  grouping  and  layout  uniqueness  account  for  higher  recollection 
(Aspillaga,  1991 ; Bundensen,  1990  & Gropper,  1991  as  cited  in  Aspillaga  1991).  Defined  as  the  size  of  the  image 
relative  to  the  visual  angle,  a person's  unique  visual  perception  will  ultimately  determine  where  he  focuses  and  gives 
attention  to  group/s  of  images  (Winn,  1993). 

Attention 

A key  component  in  learning  and  communication,  attention  has  limited  capacity  for  performing  several  processes  at 
one  time  since  it  requires  the  use  of  cognitive  resources  of  apprehension,  temporary  storage,  elaboration  of  information 
and  the  use  of  short  term  memory.  Short-term  memory  and  processing  speed  are  limited  resource  since  it  is  serial, 
slower  and  draws  heavily  on  long-term  memory.  Being  selective,  both  resources  do  not  need  irrelevant  information. 
For  a multimedia  interface  designer  it  is  important  to  determine  which  parts  of  the  message  should  be  emphasized 
because  the  cognitive  mechanisms  guiding  attention  determines  where  the  user  focuses  attention.  Parts  of  the  message 
that  stand  out  due  to  contrasting  qualities  are  brightness  in  color,  size,  shape,  typestyle  and  motion.  Color  in  black  and 
white  is  effective.  Similarly,  red  will  stand  out  because  of  its  single  unique  feature  (Nakayama,  & Joseph,  1998)  in  a 
field  of  green  distractors.  Speed  of  presenting  information  affects  depth  of  processing.  Researchers  conflict  in  their 
advice  on  the  best  rate  of  presenting  information  so  that  a user  has  enough  time  to  decode,  interpret  and  acquire  the 
message.  Some  (Bellard,  Taylor,  Canelos,  Dwyer  & Baker  1985  as  cited  in  Winn,  1993)  recommend  seven  seconds  of 
processing  time.  But  researchers  such  as  Hannafin  recommend  learner  control  which  is  contradicted  by  others  who 
recommend  that  learners  must  be  adviced  on  how  to  proceed  with  the  lesson.  Providing  the  appropriate  sequence  of 
attending  to  information  through  the  use  of  graphic  devices  such  as  arrows  and  lines  will  assist  in  guiding  the  eyes 


towards  this  information  because  the  eyes  tend  to  follow  lines  and  contours.  Being  a medium  for  attention  getting, 
multimedia  can  be  used  creatively  by  designers  to  gain,  hold,  control,  and  direct  learner's  attention  (Taylor,  1992). 

After  attention  is  given  to  the  instructional  message,  the  learner  tends  to  focus  on  information  that  is  perceived  to  be 
important.  Through  a process  called  selective  encoding  (Modes,  1994),  this  information  processing  is  strengthened. 
Through  the  use  of  elaboration  strategies  e.g.,  image  generation,  the  user  is  able  to  meaningfully  process  and  learn  the 
instructional  material.  Modes  emphasized  the  significance  of  presenting  clear,  well -planned,  and  organized  visual 
information  to  facilitate  selective  encoding  and  elaborative  processing. 

Schemata  and  Mental  Models 

Schemata  is  a person's  organized  network  of  prior  knowledge.  It  is  often  said  that  the  more  sophisticated  a person's 
schemata,  the  better  one  is  able  to  assimilate,  arrange,  construct,  and  reconstruct  prior  information  with  new 
information.  In  the  same  light,  one  who  has  acquired  an  elaborate  and  diverse  schemata  possesses  better  metacognitive 
skills  i.e.,  one  is  able  to  regulate  his/her  cognitive  processing  and  monitor  information  needs.  Anderson,  R.  & Spiro,  R. 
(1978)  conducted  a study  on  schemata  as  scaffolding  for  representing  information  in  connected  discourse  and  found 
that  food  from  categories  determined  to  be  part  of  people's  restaurant  schemata  were  better  recalled  by  students  who 
read  the  relevant  narrative.  When  faced  with  new  information,  a person  matches  that  information  with  slots  in  his 
schema  and  if  it  matches,  it  is  learned  and  remembered.  If  not,  it  is  considered  insignificant.  Mental  models  are  based 
on  schemata  but  possess  both  the  figurative  representational  attributes  of  schemata  and  the  dynamic  properties  of 
representation  systems 
Concurrency  of  Information 

Dual  Code  Theory  DCT  holds  that  verbal  and  non-verbal  systems  are  processed  independently  yet  interconnected^ 
(Paivio,  1991 ).  An  extension  of  the  dual  code  theory,  contiguity  effect  holds  that  learners  build  referential  connections 
faster  when  verbal  and  visual  materials  are  presented  contiguously  than  when  they  are  presented  separately  (Mayer  & 
Sims,  1994).  In  a verbal/visual  presentation,  the  learner  constructs  a mental  representation  in  his/her  working  memory 
of  the  verbally  presented  material  and  a mental  representation  of  the  visually  presented  material.  The  process  of  going 
from  an  external  to  an  internal  representation  is  called  building  representational  connection  or,  verbal  encoding  and 
visual  encoding  as  the  case  may  be.  As  a result  of  all  three  viz.,  verbal  encoding,  visual  encoding  and  referential 
connections,  the  user  responds  favorably  to  tests  of  retention  and  transfer  which  results  in  better  problem-solving 
transfer  and  performance. 
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Abstract:  A learning-oriented  web  environment  is  a place  of  highly  demanding  learner  control. 
The  current  research  appraises  learners’  perception  of  and  learning  styles  and  strategies  in  a web- 
based  learning  environment.  A few  laggards'  pains  buried  under  the  majority  who  well  adjusts 
oneself  should  be  recognized  and  resolved.  There  is  a strong  need  for  instructional  efforts  to 
maintain  the  proper  levels  of  learning  and  instructional  demanding.  In  contrast,  deliberate  efforts 
are  demanded  in  order  to  induce  active  discussions  and  interactions  among  learners.  In  addition, 
this  research  shows  significant  gender  differences  in  the  categories  of  expression  strategy  and 
information  processing  strategy,  in  which  males  showed  stronger  abilities  and  positive  attitudes 
without  exception.  The  findings  are  not  entirely  surprising,  since  they  replicate  many  of  the 
existing  findings  from  the  areas  of  communication,  linguistics,  and  sociology  and  more. 


Introduction 

A learning-oriented  web  environment  is  a place  of  highly  demanding  learner  control.  In  this  environment, 
learner  issues  are  the  core  element  for  success  (Bonk  & Dennen,  1999).  The  web  environment  has  been  expanding 
to  educational  communities  worldwide;  however,  it  is  not  clear  whether  all  and  every  learner  will  benefit 
equivalently  from  this  new  learning  technology.  So  far,  insufficient  attention  could  be  found  in  the  literature 
concerning  learning  strategies  in  the  web.  In  addressing  this  question,  the  current  research  (a)  appraises  learners’ 
perception  of  and  learning  styles  in  a web-based  learning  environment,  and  (b)  determine  whether  online  learning 
strategies  varied  by  gender. 

The  learner-oriented  environment  requires  learners  to  make  decisions  by  themselves  during  this  process 
and,  therefore,  requires  a high  level  of  self-regulation  and  metacognitive  abilities  in  the  learning.  Substantial 
numbers  of  literature  inform  the  important  roles  of  learning  strategies  for  learning  performance  and  some 
researchers  worked  on  the  learning  strategies  in  the  cyber  context.  Burge  (1992)  and  Eastmond  (1992)  found  the 
fact  that  learners  transfer  many  of  leaming-to-leam  approaches  from  other  traditional  learning  context  to  cyber 
setting.  At  the  same  time,  however,  they  also  identified  cyber  environment  request  learners  idiosyncratic  ways  of 
learning  strategies  of  dealing  the  nature  of  cyber  environment.  Lyman  (1998)  stressed  that  cyber  learners  need  to 
develop  learning  strategies  called,  ‘information  literacy’  in  order  to  take  benefit  especially  in  a resource-based 
learning  environment  through  the  Internet. 


The  Research 

Subjects  were  156  undergraduate  students  - 35  males  and  121  females-  from  a medium  size  university  in 
Seoul,  Korea.  They  were  from  three  different  educational  technology  courses.  The  web  system  in  this  research  was 
utilized  mainly  to  facilitate  discussion  among  learners,  to  submit  assignments,  to  provide  feedback  from  instructors, 
and  to  provide  additional  learning  materials.  Especially,  the  web  used  for  the  purpose  of  discussion  was  operated 
with  asynchronous,  open,  text-based,  task-oriented,  and  gender-mixed  features. 

Two  different  surveys  with  42-items  questionnaire  were  implemented  and  the  survey  data  were  computed 
by  mean  scores,  chi-square  and  t tests.  At  the  first  week  of  the  semester,  14  items  survey  was  conducted  in  order  to 
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find  out  subjects’  information  literacy  level.  A 28  items  survey  questionnaire  was  conducted  during  the  last  week  of 
the  semester  to  determine  learning  styles  and  strategies. 


Findings  and  Conclusions 

The  findings  of  this  study,  taken  together,  suggest  various  ideas  regarding  instructional  strategies  for  better 
online  instruction.  A few  laggards'  pains  buried  under  the  majority  who  well  adjusts  oneself  should  be  recognized 
and  resolved.  It  could  be  a starting  point  to  find  out  some  answers  about  what  common  characteristics  those  learners 
share,  how  to  identity  them,  and  how  to  support  them,  and  so  on.  We  also  need  to  consider  feedback  systems  that 
will  satisfy  learners’  high  expectation  and  concurrently  avoid  instructors’  working  overload.  There  is  a strong  need 
for  instructional  efforts  to  maintain  the  proper  levels  of  learning  and  instructional  demanding.  Most  practitioners 
agree  that  an  online  environment  intrinsically  has  a high  potential  of  producing  learning  and  instructional  overload. 
Nonetheless,  there  is  a rare  perception  that  this  quality  may  exhaust  learners  and  instructors  and,  as  a result, 
seriously  hinder  learning  performance.  It  is  desirable  and  yet  easy  to  demand  learners  to  share  with  others 
information  and  learning  outcomes.  In  contrast,  deliberate  efforts  are  demanded  in  order  to  induce  active 
discussions  and  interactions  among  learners. 

It  seems  clear  that  males  tended  to  use  computers  more  at  least  in  their  daily  life  and  to  have  stronger 
experiences  with  the  Internet  than  females.  This  research  shows  significant  gender  differences  in  the  categories  of 
expression  strategy  and  information  processing  strategy,  in  which  males  showed  stronger  abilities  and  positive 
attitudes  without  exception.  The  findings  are  not  entirely  surprising,  since  they  replicate  many  of  the  existing 
findings  from  the  areas  of  communication,  linguistics,  and  sociology  and  more  (Ebben,  1994;  Hatton,  1995;  Herring, 
1993;  Herring,  1994;  Herring,  Johnson,  & Dibenedetto,  1992;  Kramarae  & Taylor,  1991;  McDowell  & Schuelke,  1998; 
Selfe  & Meyer,  1991;  Spender,  1995;  Tannen,  1991).  The  current  research  also  confirms  that  awareness  of  gender 
differences  in  learning  strategies  is  critical  to  know  its  potential  effects  on  learning  process  and  outcomes.  Currently 
dominating  modes  of  cyber  courses  demonstrate  structures  and  functions  in  favor  of  males.  However,  it  should  not 
be  taken  for  granted  but  we  have  to  move  toward  a gender-equal  cyber  learning  environment.  This  can  be  achieved 
by  including  instructional  design  and  implementation  which  might  more  respond  to  learning  strategies  in  favor  of 
females.  Of  greatest  potential  that  leads  to  gender  inequality  in  a cyber  learning  environment  is  its  text-based, 
public,  and  information  overload  natures. 
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Abstract:  This  paper  describes  PHYLOGENE,  a multimedia  open  environment  we  designed 
for  evolutionary  biology  teaching  and  learning.  The  progressive  approach  of  evolution  it 
implements,  relies  on  a set  of  biologic,  anatomic,  morphologic  and  molecular  data  and  allows 
different  articulated  activities  from  observation  to  evolutionary  trees  construction,  including 
comparison,  selection  and  classing  of  animal  species  and  characters.  All  these  activities  provide 
the  students  with  multiple  entries  at  different  teaching  level. 

The  main  feature  of  PHYLOGENE  is  that  it  combines  on  one  hand  an  hypermedia  environment 
enabling  navigation  among  documents  describing  species  and  their  characters,  and  on  the  other 
hand,  analysis  tools  enabling  to  select,  class  and  organize  the  hypermedia  explored  data.  It 
provides  thus  as  well  knowledge  exposition  as  learning  activities  that  encourage  an  active 
involvement  of  the  learner.  We  argue  for  the  interest  of  such  multimedia  open  environments  in 
science  learning. 


A lot  of  software  tools  can  be  of  great  interest  in  learning  about  life  evolution,  but  most  of  them  only  offer  a 
particular  approach.  We  describe  in  this  paper  PHYLOGENE  a multimedia  open  environment  that  integrates 
varied  cognitive  activities  to  allow  a global  approach  of  evolution. 

Learning  the  Concept  of  Evolution 

Evolution  of  life  is  a fundamental  concept  in  the  understanding  of  a great  number  of  biological  phenomenon. 
This  concept  relies  on  various  kind  of  knowledge  relative  to  Earth  history,  biochemical  process  at  the  origin  of 
life,  living  organisms  development  conditions  and  stages,  species  apparition  and  extinction  (biological  crisis  and 
crossing  off),  species  relationships.  Therefore  evolution  is  central  in  the  biology  curriculum  organization  at  all 
school  levels. 

At  these  different  levels,  varied  cognitive  abilities  are  required  from  the  students.  At  the  beginning,  they  are 
centered  on  a global  recognition  of  macroscopic  characters.  Then  they  have  to  analyze  (in  a qualitative  or 
quantitative  way)  details  or  data  able  to  constitute  arguments  to  reconstruct  evolution  and  to  integrate  selected 
information  in  a global  representation  such  as  an  evolutionary  tree.  Finally,  they  are  led  to  integrate  new 
knowledge  in  the  framework  of  evolution  theory,  taking  into  account  the  current  debates. 

Though  some  software  tools  can  be  useful  in  the  learning  of  evolution  related  concepts  (Jones  et  al.  1997),  none 
of  them  provides  an  integrated  approach.  That  led  us  to  design  a multimedia  open  environment  called 
PHYLOGENE,  allowing  such  an  integrated  approach  by  giving  access  to  the  different  kind  of  knowledge 
implicated  in  the  study  of  evolution  and  by  enabling  different  cognitive  activities  (Dupuis  et  al.  1997). 


Description  of  PHYLOGENE 

In  order  to  support  a global  approach  of  life  evolution,  PHYLOGENE  has  the  main  following  features  : 

- It  allows  to  combine  different  abstraction  levels  through  different  kind  of  activities, 

- Different  entry  points  are  possible  , taking  into  account  the  nature  of  the  question  to  study  and  the  students 
school  level  (middle  school,  high  school,  university), 

- Students  can  choose  freely  the  species  and  characters  they  want  to  study. 


One  of  the  major  goals  is  to  let  the  student  be  active  by  solving  problems,  while  following  a naturalist  approach 
(morphologic  and  biologic  data).  Unlike  other  tools,  the  tree  structure  is  not  pre-established  and  the  species 
characters  are  not  all  evolutionary  characters. 

PHYLOGENE  is  thus  organized  in  different  activities  exploiting  a database  of  species  : 


Figure  1:  Two  activities  of  PHYLOGENE 


- Observation  activity  allows  to  study  the  animal  morphology,  to  discover  some  of  its  biologic  characters 
(habitat,  diet,  fertilization,  social  structure)  and  anatomic  characters  (breathing  organ,  locomotor  organ,  skull, 
extra  embryonic  membrane)  characters. 

- Comparison  activity  (Fig.  1)  allows  to  display  simultaneously  up  to  four  species  images  or  characters  sketches. 
New  species  and  new  characters  can  be  easily  taken  into  account. 

- Grouping  and  classing  from  a range  of  species  gives  the  possibility  to  choose  a subset  of  species  and 
characters,  to  fill  a table  with  the  observed  characters  states  and  then  to  build  a hierarchical  classification  on  the 
basis  of  these  states. 

- Trees  constructing  (Fig.  1)  can  be  realized  on  the  basis  of  a table  or  a classification.  Trees  can  be  constructed 
step  by  step  in  a MacClade  like  way  (Madison),  by  choosing  the  positions  and  grouping  together  of  the  branches. 

The  main  feature  of  PHYLOGENE  is  that  it  combines  on  one  hand  an  hypermedia  environment  enabling  a 
navigation  among  species  and  their  characters  and  on  the  other  hand  analysis  tools  enabling  to  select,  class  and 
organize  the  data  explored  in  the  hypermedia.  It  provides  as  well  knowledge  exposition  as  learning  activities  that 
encourage  an  active  involvement  of  the  learner,  allowing  discovery  learning  (Van  Joolingen  1999).  We  think  that 
such  hybrid  tools  are  well  adapted  in  experimental  science  learning. 

PHYLOGENE  has  been  experimented  with  success  in  a few  middle  school  classrooms.  It  will  be  distributed  in 
the  course  of  the  year  2000.  A scientific  and  pedagogical  documentation  will  go  with  it  and  additional  data  will 
be  found  on  our  web  site  (http://www.inrp.fr/Acces/Biogeo/accueil.htm). 
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How  does  one  evaluate  the  effectiveness  of  electronic  learning  in  a South 

African  context? 


Significant  aspects  of  the  context  include  the  following.  The  University  of  the  Western 
(http://www.uwc.ac.za/)  has  adopted  the  mission  to  - “Assist  educationally  disadvantaged  students  gain 
access  to  higher  education  and  succeed  in  their  studies”.  Consequently,  the  university  has  provided  access 
to  students  from  previously  disadvantaged  communities.  As  a consequence  our  classes  are  ethnically  and 
culturally  diverse.  Moreover,  the  quality  of  the  educational  experiences  of  these  students  also  vary  widely. 
From  such  a heterogeneous  student  population,  one  infers  that  a rich  learning  environment,  that 
accommodates  individualisation,  flexibility  and  self-paced  learning,  would  be  ideal.  Over  the  last  few 
decades  computer  assisted  learning  has  been  suggested  as  a resource  to  meet  such  needs.  The  technological 
infrastructure  at  the  institution  is  still  underdeveloped,  but  it  is  improving  rapidly.  Despite  the  enormous 
inequalities  of  the  past,  the  recently  established  democracy  brings  the  promise  of  rapid  progress.  Such  a 
context  requires  rigorous  evaluation  of  the  impact  of  the  new  technologies  on  education  in  order  to  ensure 
that  our  society  reaps  the  maximum  benefits  and  avoids  possible  pitfalls  (Reeves  ,1998). 

Rationale  for  evaluation 

What  is  evaluation  and  how  is  it  related  to  research?  Evaluation  is  defined  here  as:  the  systematic 
acquisition  and  assessment  of  information  to  provide  useful  feedback  about  some  object  (Trochim,  1999). 
The  object  in  this  case  is  technologically  enhanced  learning.  The  generic  goal  of  evaluation  is  to  provide 
useful  feedback  to  various  stakeholders  that  influences  decision  making  at  various  levels.  Evaluation,  the 
same  as  research,  is  concerned  with  the  collection  and  analysis  of  valid  and  reliable  empirical  data.  The 
two  may,  therefore,  utilise  many  of  the  same  methodologies.  However,  because  evaluation  takes  place 
within  an  organisational  and  political  context,  it  requires  sensitivity  to  multiple  stakeholders  as  well  as 
managerial  and  political  factors  (Trochim,  1999).  In  addition,  in  practice  the  relationship  between 
evaluation  and  the  actual  decisions  taken  is  not  a simple  one.  Thus,  the  question  arises  is  evaluation 
necessary? 

Reeves  (1997)  argues  that  there  are  at  least  four  reasons  for  the  lack  of  useful  evaluation  of  CBE.  Firstly, 
consumers  of  technological  innovations  for  education  uncritically  accept  the  advertised  claims  of  the 
producers  of  the  technology.  Secondly,  evaluation  of  CBE  has  often  been  reduced  to  a numbers  game. 
Thirdly,  evaluations  that  have  been  previously  conducted  lacked  utility  and  impact.  Fourthly,  the 
inappropriate  reliance  on  traditional  empirical  evaluation  methods  that  compare  an  instructional  innovation 
with  another  approach,  frequently  led  to  disappointing  results.  He,  therefore,  proposes  fourteen 
pedagogical  criteria  for  evaluating  various  forms  of  CBE  in  order  to  obtain  more  valid  and  useful 
evaluations. 


Quality  learning 

The  key  outcome  of  this  project  should  be  an  improvement  in  the  quality  of  learning  by  our  students  and 
teaching  by  our  academics.  The  Multimedia  Computer  together  with  its  sophisticated  software  is  merely  a 
means  to  this  end.  Laurillard  (1993)  argues  that  a mix  of  teaching  and  learning  methods  will  always  be  the 
most  efficient  way  to  support  student  learning,  because  only  then  is  it  possible  to  embrace  all  the  activities 
of  discussion,  interaction,  adaptation,  and  reflection  that  are  essential  for  quality  learning.  Entwistle  (1996) 
stresses  the  importance  of  research  into  student  learning  for  improving  tertiary  teaching. 

Methodologies 

Nightingale  and  O’Neill  (1994)  recommend  Action  Research  as  an  approach  to  enhancing  the  quality  of 
learning  in  higher  education.  It  is  adopted  as  a broad  approach  because  of  its  focus  on  improving  practice 
rather  than  merely  proving  theories.  It  is  seen  as  a self-reflective  spiral  that  involves  planning,  acting. 


Context 


observing  and  reflecting  with  further  iterations  of  the  same  cycle.  Trochim  (1999)  discusses,  four  strategic 
approaches  to  evaluation:  scientific-experimental  models,  management-oriented  systems  models, 
qualitative/anthropological  models  and  participant-oriented  models.  He  also  points  out  that,  although 
vigorous  debates  may  rage  among  the  proponents  of  these  schools,  one  should  borrow  from  each  as  the 
need  arises. 

Action  research  defines  the  broad  approach  within  which  more  specific  research  techniques  will  be 
employed.  For  many  reasons  a pure  experimental  design  is  unlikely  to  be  fruitful.  Ethical,  practical  and 
technical  considerations,  all  render  controlled  experimentation  inappropriate  (Entwistle  1 996).  Both 
qualitative  and  quantitative  techniques  will  be  employed  as  deemed  appropriate.  Specific  techniques  used, 
include:  surveys  and  questionnaires,  interviews,  student  achievement  under  different  regimes.  In  brief, 
careful  observation  of  students  involved  in  CAL;  supported  by  surveys  and  questionnaires  as  well  as 
detailed  interviews  will  be  employed  to  provide  the  raw  data. 

Case  studies 

Case  studies  of  efforts  to  introduce  computer  supported  education  and  its  effects  on  student  learning  will  be 
presented.  Some  questions  to  be  explored  are  - 

How  do  students  interact  with  CD-ROM  delivered  material? 

What  factors  promote  or  hinder  their  learning  when  using  this  medium? 

How  should  CD-ROM  be  used  without  compromising  quality  of  learning  outcomes? 

Does  technologically  delivered  learning  materials  live  up  to  its  promise? 
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Abstract:  An  introductory  mathematics  course  in  linear  algebra  was  taught  using  laptops  and 
Scientific  Notebook.  Computer  technology  was  utilized  as  an  integral  part  of  presentation  as  well 
as  learning  and  testing  to  the  extent  that  the  instructor  did  not  write  one  thing  on  the  board  the 
entire  semester.  This  approach  has  implications  for  other  disciplines  besides  mathematics. 


Introduction 

Stevens  Institute  of  Technology  has  required  its  students  to  purchase  a PC  since  1984.  This  requirement  has  now 
been  replaced  with  a requirement  that  all  students  purchase  a laptop.  To  prepare  for  the  switch  from  PCs  to  laptops, 
last  year  Stevens  set  up  a laptop  classroom  so  that  faculty  could  experiment  with  teaching  in  a new  environment. 
The  author  used  this  classroom  to  teach  an  introductory  one-semester  course  in  linear  algebra. 

Scientific  Notebook 

The  author  has  used  Scientific  Notebook  (SNB)  for  several  years  in  the  second  year  mathematics  sequence  taught  at 
Stevens.  SNB  is  a word  processing  package  that  incorporates  a Maple  kernel.  This  Maple  kernel  allows  one  to 
perform  a wide  variety  of  mathematical  operations.  A particularly  nice  feature  of  SNB  is  that  one  can  use  the  Maple 
kernel  without  typing  any  code.  One  simply  types  the  mathematical  expression  and  then  clicks  on  the  appropriate 
operation  to  be  performed.  SNB  then  performs  the  operation. 

Structure  of  the  linear  algebra  course 

SNB  was  used  to  prepare  a complete  set  of  lecture  notes.  The  notes  incorporated  in-class  exercises  to  be  worked  on 
during  the  lectures.  Thus  each  class  meeting  was  a combination  of  lecture  and  hands-on  doing.  When  the  instructor 
was  lecturing,  laptops  were  required  to  be  closed,  whereas  when  the  students  were  working  on  the  exercises,  the 
laptops  were  open.  The  class  moved  seamlessly  between  the  two  environments  of  a “standard”  classroom  and  a 
computer  laboratory.  Homework  had  to  be  done  using  SNB.  No  written  work  was  accepted. 

Use  of  the  Web 

Notes  for  each  class  were  posted  on  the  Web.  A feature  of  SNB  is  that  one  can  put  a tex  file  on  the  Web,  and  then 
download  by  simply  clicking  on  it.  Thus  each  lecture  began  with  the  instructor  downloading  the  lecture  from  the 
Web  to  his  laptop  and  then  projecting  it  for  the  class  to  see. 

All  homework  assignments  were  also  posted  on  the  Web.  In  addition,  solutions  to  homework  problems  and  exams 
were  put  on  the  Web  site.  A good  portion  of  the  quizzes  and  exams  were  also  given  over  the  Web.  The  Web  served 
as  a crucial  resource  for  the  course.  To  see  what  was  actually  done  go  to  http ://atti la. stevens-tech/edu/Hle vine/ . 

Exams  and  Quizzes 

Many  of  the  short  quizzes  and  portions  of  all  exams  were  designed  so  that  the  students  had  to  use  SNB  to  solve  the 
problems.  These  had  to  be  downloaded  by  the  students,  worked  on,  and  then  uploaded  to  a site.  The  exams  and  the 
final  were  in  two  parts;  one  part  to  be  done  with  paper  and  pencil,  and  the  other  part  requiring  use  the  Maple  kernel 
in  SNB  to  solve  the  problems.  Having  a networked  laptop  classroom  made  this  approach  possible. 
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Engaging  the  student  in  the  teaching/learning  experience 

One  of  the  goals  of  a good  educational  experience  is  to  engage  students  in  the  teaching/leaming  experience.  Active 
learning  is  certainly  more  desirable  that  passive  learning  for  a wide  range  of  reasons.  However,  within  the  classical 
lecture  format  it  is  often  difficult  to  get  students  actively  involved  in  what  is  going  on  in  class.  Given  the  approach 
taken  in  this  linear  algebra  course  active  student  involvement  became  the  rule  rather  than  the  exception. 

Advantages  of  this  approach 

Integrating  the  use  of  computer  technology  into  this  one  semester  linear  algebra  course  in  the  manner  described 
above  had  the  following  advantages: 

• Students  learned  to  use  sophisticated  mathematical  software  as  a tool  in  mastering  mathematics. 

• Mastery  of  linear  algebra  entails  being  able  to  perform  many  arithmetic  operations.  Students  often  make 
mistakes  while  performing  these  operations.  Using  software  judiciously  avoids  this  problem  and  allows  the 
student  to  concentrate  on  the  underlying  concepts. 

• A laptop  classroom  is  actually  two  classrooms  in  one:  with  the  covers  of  the  laptops  up  it  is  a computer 
laboratory,  whereas  with  the  laptop  covers  down  one  has  a “standard”  classroom. 

• The  active  learning  described  above  is  certainly  a major  advantage  of  this  approach. 

Problems  encountered 

• The  time  investment  required  preparing  for  and  teaching  such  a course  is  considerable. 

• Preparation  of  exams  that  incorporated  the  meaningful  use  of  SNB  required  considerably  more  thought  and  time 
than  preparation  of  “standard”  exams. 

• Some  students  found  it  difficult  to  deal  with  a mathematics  course  in  which  writing  was  not  a key  component. 

• There  were  a number  of  technical  difficulties  encountered  vis  a vis  the  hardware. 

• Many  of  the  short  quizzes  and  all  of  the  exams  had  a computer  component.  This  component  was  downloaded 
from  a server,  worked  on  by  the  students  and  then  uploaded  to  the  server  upon  conclusion.  Thus  security  was 
very  much  an  issue. 

Applicability  to  other  courses 

The  approach  taken  with  this  linear  algebra  course  can  certainly  be  used  to  teach  other  mathematics  courses  as  well 
as  courses  in  other  disciplines.  Indeed  it  is  this  author’s  opinion  that  using  such  an  approach  would  enhance  the 
learning  experience  for  many  students. 

However,  this  approach  does  have  limitations.  The  linear  algebra  course  had  28  students  in  it.  There  was  also  a 
teaching  assistant  present  during  all  classes.  Thus  two  people  were  necessary  to  handle  a class  or  28.  The  class  size 
could  have  been  increased  to  perhaps  40  without  foreseeable  problems.  However,  this  is  probably  the  upper  limit  for 
such  a course.  The  point  is  that  this  approach  will  not  work  with  a lecture  class  of  say  eighty  or  a hundred. 

Conclusions 

Students  were  asked  to  complete  surveys  on  the  web  a couple  of  times  during  the  semester.  The  vast  majority  of 
students  liked  the  course,  but  there  were  a few  who  were  very  unhappy  with  the  approach.  Approximately  90%  of 
the  students  indicated  that  they  would  be  willing  to  take  another  course  that  was  given  in  this  format.  Perhaps  the 
overall  conclusion  to  be  drawn  is  that  while  some  students  will  always  be  unhappy  with  a course  that  employs  the 
extensive  use  of  technology,  the  vast  majority  will  benefit  greatly  from  courses  that  judiciously  employ  technology 
as  part  of  the  teaching/leaming  experience.  The  challenge  is  for  faculty  to  find  meaningful  ways  to  incorporate  this 
technology  into  their  courses.  This  is  far  from  a trivial  challenge. 
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Abstract:  An  authoring  shell  that  supports  the  reuse  of  teaching  material  should  not  only 
allow  the  author  to  search  for  the  existence  of  the  material  and  retrieve  it.  When  there  are  too 
many  “matched”  items,  an  authoring  tool  should  be  able  to  identify  those  that  are  relevant  and 
create  a suggestion  list  to  the  author.  When  there  are  “no  matched”  item,  an  authoring  shell 
should  be  able  to  suggest  a list  of  materials  that  are  closely  related.  This  article  describes  how 
an  Intelligent  Authoring  Shell  (IAS)  can  provide  suggestions  to  a teacher  at  authoring. 


Introduction 

To  enable  reuse,  IAS  has  employed  database  repository  method  (Sarti  and  Marcke  1 995)  to  store  useful 
information  of  each  piece  of  teaching  materials,  activities  and  lessons.  Along  with  database  repository  method, 
IAS  also  uses  learning  objective  as  the  key  to  reuse  teaching  materials  (Kam  Wing  and  Warren  1998).  Learning 
objective  has  two  main  components:  an  object  and  a keyword,  the  letter  drawn  from  a learning  objective.  An 
object  is  a concept,  or  skill,  or  entity  that  is  described  in  a learning  objective.  Keyword  is  the  verb  being  using  in 
the  learning  objective.  With  object,  we  can  search  for  the  existence  of  a related  teaching  mated al/activity/lesson. 
With  keyword,  we  can  know  the  purpose  and  usage  of  each  piece  teaching  material/activity/lesson.  Therefore, 
learning  objective  is  the  backbone  for  IAS  to  enable  reuse. 

IAS  has  three  main  modules:  keyword  module,  strategy  design  module  and  course  design  module. 
Keyword  module  and  strategy  design  module  are  bases  to  provide  intelligence  in  IAS.  Reuse  of  teaching 
materials  and  providing  intelligent  suggestions  to  a teacher  on  reusing  teaching  materials  take  place  in  course 
design  module. 


Keyword  module 

It  facilitates  the  input  of  learning  objective  taxonomy  {keyword)  into  IAS  database.  IAS  uses  the 
structure  of  Bloom’s  taxonomy  (Bloom,  Englehart  et  al.  1964)  as  reference.  Bloom  has  classified  learning 
objective  taxonomies  into  three  different  domains:  cognitive  domain,  affective  domain  and  psychomotor  skill 
domain.  Under  each  domain,  taxonomies  are  further  classified  into  different  levels  forming  a class  hierarchy.  In 
IAS,  we  have  used  this  module  to  enter  Bloom’s  taxonomy  on  cognitive  domain.  Users  can  use  this  module  to 
enter  other  taxonomies  or  even  create  their  own  set  of  taxonomy  and  define  the  use  for  each  keyword. 


Strategy  design  module 

It  allows  teachers  to  design  their  own  teaching  strategy.  In  IAS,  a teaching  strategy  is  composed  of  a set 
of  activities  with  different  activity  type.  Each  activity  type  may  consist  of  sub- activity  type.  For  example,  a 
lesson  using  concept  attainment  (Joyce  and  Weil  1986)  as  its  teaching  strategy  will  contain  introduction. 
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example(s),  counter-example(s),  analysis  and  conclusion.  Example  is  composed  of  sub-activities  such  as 
description,  question,  answer  and  explanation.  In  designing  a teaching  strategy,  the  author  will  construct  the 
teaching  strategy  structure  by  arranging  activity  types  in  order.  The  author  will  also  indicate  which  domain  and 
taxonomy  class  are  likely/unlikely  to  be  suitable  for  the  teaching  strategy.  With  this  information  available,  IAS 
can  use  the  keyword  from  learning  objective  to  provide  suggestion  list  of  appropriate  teaching  strategies  to  a 
teacher  at  authoring. 


Course  design  module 

It  helps  a teacher  on  designing  curriculum  of  a course,  lesson  and  learning  activities  within  a lesson  by 
providing  suggestions  on  reusable  materials  to  a teacher  at  authoring. 

In  designing  the  curriculum,  a teacher  will  construct  a hierarchy  of  learning  objectives  for  the  course. 
On  inputting  a learning  objective,  IAS  will  check  the  keyword  with  previously  entered  learning  objectives’ 
keyword  by  using  the  class  hierarchy.  This  is  useful  to  detect  any  illogical  sequence  of  learning  objective. 
According  to  constructivism  learning  theory,  any  illogical  order  of  learning  objective  hierarchy  will  increase 
student’s  difficulty  in  learning.  Therefore,  such  a checking  feature  can  help  to  improve  quality  of  a course.  At 
the  same  time  a teacher  is  building  learning  objective  hierarchy,  an  object  hierarchy  is  also  built  using  the 
objects  from  each  learning  objective.  With  this  object  hierarchy,  we  can  identify  the  relation  between  each 
object.  This  relation  will  be  useful  in  deciding  which  teaching  materials  may  be  appropriate  for  reuse  and  also  in 
searching  and  retrieving  closely  related  materials  for  the  object. 

In  creating  a lesson,  a teacher  will  select  a teaching  strategy  from  the  appropriate  teaching  strategy  list. 
This  list  is  suggested  according  to  the  class  of  domain  keywords  being  used  in  the  learning  objective  and  the 
infomiation  from  the  strategy  design  module. 

After  teaching  strategy  of  a lesson  is  selected,  the  author  will  follow  the  structure  of  the  teaching 
strategy  to  design  learning  activities  and  sub-activities.  URLs  of  multimedia  files  for  each  activity  and  sub- 
activity will  be  recorded  into  IAS  database.  In  designing  learning  activities,  IAS  will  suggest  reusable  activities 
and  teaching  materials  for  the  author  to  select.  The  suggestion  criteria  available  are  based  on  activity  type, 
learning  objective,  keyword  and  object.  Suggestion  criteria  based  on  activity  type  will  use  activity  type  as 
filtering  element.  For  example,  if  the  activity  at  authoring  is  an  example,  then  IAS  will  only  list  those  activities 
that  are  examples  only.  Suggestion  criteria  based  on  keyword  and  object  will  list  related  activities/material 
according  to  the  keyword  class  hierarchy  and  object  hierarchy.  This  will  reduce  author’s  time  in  searching  for 
reusable  teaching  materials,  particularly  when  the  database  has  hundreds  of  teaching  materials  available. 


Conclusion 

Using  database  repository  method,  normal  authoring  shell  is  able  to  provide  simple  reuse  of  teaching 
materials.  Using  learning  objective  with  database  repository  method,  a normal  authoring  shell  will  become  more 
intelligent.  It  can  provide  some  suggestions  to  teacher  and  thus  reducing  a teacher’s  effort  in  creating  on-line 
course.  Even  though  our  IAS  can  provide  some  suggestions  to  teacher  at  authoring,  we  are  looking  forward  to 
optimising  the  suggestion  criteria  to  improve  the  effectiveness  of  suggestions. 
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Abstract:  This  paper  explores  what  it  means  to  be  a visual  artist  in  the  age  of  information 
technology.,  and  it  explores  how  the  degree  in  Computer  Mediated  Art  at  Victoria  University 
attempts  to  answer  this  question.  A number  of  Informing  Principles  are  discussed  which  provide 
insight  into  the  development  and  implementation  of  the  program,  a program  which  draws  upon 
both  traditional  studio  practice  and  digital  studio  practice.  The  paper  concludes  by  examining  how 
the  technological  shaping  of  (visual)  culture  produces  an  art  of  technology  and  a technology  of  art. 


Art  @ Technology. 

Victoria  University  was  initially  configured  as  a university  of  technology  intended  to  service  the  needs  of  its  local 
industry  based  community  in  the  economically  disadvantaged  western  suburbs  of  Melbourne,  Australia.  In  1997  it 
was  decided  to  change  the  low  visual  arts  profile  at  the  university  into  a high  arts  profile  by  introducing  for  the  first 
time  in  the  western  suburbs,  a fine  arts  degree.  The  degree  however  had  two  briefs:  it  could  not  be  a conventional 
visual  art  degree  as  found  in  other  universities  around  Australia;  and  it  had  to  make  digital  technology  central  to  it 
program. 

Artists  Adopting  Technologies. 

Artists  have  adopted  technologies  to  their  studio  practice  as  soon  as  those  technologies  become  available.  These 
technologies  have  included  the  development  of  different  types  of  paint  such  as  oil,  tempera,  acrylics  and  gauche,  the 
development  and  implementation  of  perspective  and  a variety  of  painting  surfaces.  In  addition  to  these  technologies, 
artists  have  also  employed  more  mechanical  innovations  such  as  the  camera  obscura,  photography  and  various 
printmaking  methods  such  as  off-set,  silkscreen  and  etching.  Electronic  technologies  such  as  video,  photocopy  and 
fax  have  been  coopted  into  studio  practice  and  more  recently  again,  digital  technologies  through  a variety  of 
computer  hardware  and  software  applications. 

Informing  principle:  If  technology  has  always  been  a part  of  an  artist’s  studio  practice,  then  the  inclusion  of  digital 
technologies  would  simply  be  a continuation  of  that  process  and  in  this  way,  technology  would  be  defined  as  a range 
of  ideas  and  tools  which  are  available  to  the  artist  in  order  to  realize  a particular  set  of  intentions. 

Technology  and  Art  History. 

Computers  in  general,  and  art  software  packages  in  particular,  do  not  represent  a significant  discontinuity  from 
previous  technologies  used  in  the  artist’s  studio.  Software  effects  such  as  morphing  were  used  by  Escher  in  the  early 
part  of  this  century  as  were  effects  like  interactivity  found  in  Duchamp’s  Readymade  series.  Zooming  and 
magnifying  techniques  were  used  by  Lichtenstein  in  the  sixties  while  Oldenburg  played  with  changes  in  scale  at  the 
same  time.  As  for  hardware,  the  monitor,  which  physically  resembles  the  landscape  shape/format  of  a traditional 
canvas,  is  inherently  perspectival. 

Informing  principle :Corr\^\xitr  mediated  art  is  primarily  art  made  using  the  brain  rather  than  the  hand  and  it  is  this 
difference  that  turns  the  computer  into  more  than  an  ordinary  tool  studio  tool  because  it  disturbs  the  ‘taken  for 
grantedness’  of  art  praxis.  Is  the  computer  a tool  like  a paintbrush  or  pencil?  Of  course  the  answer  will  depend  upon 
how  ‘tool’  is  defined,  but  what  makes  the  computer  interesting  in  the  creative  process  is  its  potential  to  take  on  a 
role  that  traditional  tools  cannot  by  for  example,  generating  automated  creative  processes  through  the  use  of  scripts. 
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Used  this  way,  the  computer  becomes  a decision  making  tool  which  acts  to  amplify  intelligence  rather  than  replace 
it. 

Atoms  and  Bits. 

Central  to  this  new  artistic  environment  -the  artist  in  the  age  of  information  technology-  is  an  emphasis  upon  both 
atom  based  art  and  bit  based  art.  Digital  art  consists  of  a series  of  electronic  charges  on  a monitor  rather  than  the 
physical  presence  of  an  analogue  painting.  Inherent  in  bit  based  art  is  computer  memory  and  its  ability  to  store, 
retrieve  and  manipulate  image,  sound  and  text  in  ways  not  previously  possible  in  analogue  art.  In  this  way  the 
distinction  between  master  and  copy  typically  found  in  the  artmaking  process  is  no  longer  viable.  Each  subsequent 
copy  is  now  a perfect  master. 

Informing  principle:  Students  will  develop  computer  mediated  art  work  which  is  either  static  two  dimensional 
images  in  much  the  same  way  as  atom  based  art  work  on  the  wall  is  static  two  dimensional  as  well  as  time  based  art 
which  may  or  may  not  be  interactive.  As  well,  they  may  produce  work  which  is  three  dimensional  that  can  move 
fluidly  without  constraint,  conveying  us  through  various  modes  of  discourse.  Students  will  complete  the  degree  with 
both  a folio  of  analogue  work  and  a folio  of  digital  work  on  a CD-ROM/Web/DVD. 

Aesthetics  and  Technology. 

Computer  mediated  art  provides  no  panacea  for  aesthetic  problem  solving  nor  for  resolving  the  issue  that  while 
computers  are  being  used  to  generate  images,  meaningful  art  still  needs  to  be  created  irrespective  of  the  technology. 
Artists  need  to  consider  what  the  end  result  might  be  and  what  part  a computer  might  play  in  this  being  achieved. 
What  can  be  done  with  traditional  tools  does  not  necessarily  become  more  interesting  because  it  has  been  generated 
on  a computer:  what  is  interesting  is  what  was  not  conceivable  before  computers.  The  art  software  packages  not  only 
allow  artists  to  work  in  substantially  different  ways  to  analogue  art  by  allowing  the  combination  of  what  would 
normally  be  considered  mutually  exclusive  techniques  -for  example  oil  paint  and  watercolor-  they  also  undermine 
one  of  the  central  goals  of  analogue  art-  the  goal  of  fixed  perfection.  The  Undo  command  allows  artists  to  undo  at 
will  any  mark  or  fill  deemed  inappropriate  which  may  result  in  an  art  which  lacks  commitment  or  be  too  timid. 

Informing  principle:  Alongside  both  analogue  and  digital  art  students  will  participate  in  a program  which  examines 
aesthetic  issues  in  relation  to  technology  as  well  as  an  art  history  which  does  not  focus  only  on  dead,  European, 
white  males.  Aesthetic  issues  based  on  Asian  art  as  well  as  issues  of  gender,  particularly  in  relation  to  the  art 
software  packages  being  used,  will  be  examined. 

Virtual  Studios. 

A virtual  studio  will  be  established  between  students  enrolled  in  the  Bachelor  of  Arts-Computer  Mediated  Art  and 
students  enrolled  in  the  visual  arts  stream  of  a Bachelor  of  Education  at  Deakin  University  in  order  to  investigate  the 
use  of  that  virtual  studio  as  a collaborative  learning  forum  for  creating  computer  mediated  art.  It  will  also  examine 
the  way  artists  are  grappling  with  new  digital  technologies  in  their  studio  practice.  The  software  which  will  be  used 
for  this  virtual  studio  will  be  Painter  5.5  because  it  has  tools  which  are  familiar  and  recognizable  to  artists.  To 
achieve  this,  Painter  relies  on  analogies  with  traditional  artistic  techniques  such  as  virtual  pens,  virtual  pencils  and 
virtual  canvas.  Painter  is  also  being  used  because  it  has  a facility  which  allows  it  to  be  used  via  a TCP/IP 
connection.  Essentially,  there  are  two  frames  on  the  monitor-  one  for  the  artwork  which  is  asynchronous  and  one  for 
a chatline  which  is  synchronous.  Students  at  Deakin  will  be  able  to  watch  digital  art  being  made  at  Victoria 
University  and  when  that  work  in  finished  it  can  be  sent  down  to  the  other  student  as  a file,  and  vice  versa.  Painter 
also  has  the  ability  to  record  every  stroke  and  combination  of  strokes  in  the  creation  of  the  visual  image  which  can 
then  be  replayed  at  later  stages  to  investigate  issues  of  participation,  patterns  of  use  and  the  application  of  virtual 
technologies  to  the  art  making  process. 

Conclusion. 

The  Bachelor  of  Arts-Computer  Mediated  Art  is  not  a multi  media  program,  although  possibilities  exist  for  its  use. 
Rather,  it  is  a course  aimed  at  fine  artists  who  wish  to  use  traditional  paint  brushes  as  well  as  paintbrushes  which  are 
plugged  in  to  the  computer  in  their  studio  practice,  and  to  understand  the  implications  and  consequences  of  so  doing. 
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Abstract:  The  authors  have  conducted  a survey  to  identify  online  learning  resources  specifically 
designed  to  support  the  professional  development  of  higher  education  staff.  This  short  paper 
outlines  the  survey  and  principle  findings,  and  presents  an  overview  of  the  pdonline  website  with 
links  to  the  identified  'best*of  online  resources.  The  authors  offer  the  pdonline  website  for  use 
with  professional  development  activities  to  support  higher  education  staff. 


Introduction 

The  rhetoric,  rationale  and  emerging  practices  of  ‘online  learning’,  ‘flexible  delivery’,  ‘just-in-time’, 
‘anytime  anywhere’  education  for  learning  and  teaching  programs  is  just  as  relevant  for  staff  development.  Online 
communication,  activities  and  resources  can  enable  more  flexible  ways  of  supporting  diverse  individual  staff 
learning  needs.  Online  media  can  be  used  to  improve  the  support  for  mentored  and  facilitated  ‘just-in-time’ 
professional  development  services. 

When  considering  the  design  issues  of  online  individualised  staff  development  the  authors  considered  a 
necessary  first  step  was  to  survey  and  identify  relevant  online  resources.  This  survey  informed  the  content  of  a 
website  of  links  to  what  we  considered  were  the  best  of  the  identified  online  resources.  We  offer  the  pdonline 
website  for  use  by  higher  education  centres  of  staff  development  to  assist  their  activities.  The  website  is  located  at 
<www.cpd.mq.  edu.au/pdonli  ne>. 


Survey  Description 

The  survey  aimed  to  identify  the  quality  and  scope  of  accessible,  online  resources  specifically  designed  to 
support  the  development  of  English  language  university  staff.  The  research  was  conducted  from  September  1 998  to 
February  1999.  All  Australian,  New  Zealand,  and  United  Kingdom  universities,  all  Canadian  and  American 
universities  with  online  teaching  centres,  and  all  known  virtual  campuses  were  investigated  via  the  web.  A total  of 
385  university  websites  were  surveyed  in  detail. 

The  survey  was  designed  to  investigate  and  scan  the  web  for  accessible  online  university  staff  development 
programs  and  resources  in  learning  and  teaching,  leadership  and  management,  research,  administrative  and  specialist 
support,  and  community  service.  The  study’s  secondary  focus  incorporated  other  university  centres  such  as  flexible 
and  distance  learning  centres,  materials  developed  by  faculty  such  as  tips  on  teaching,  web  resource  links 
recommended  by  staff  development  centres,  and  general  web  and  database  searching  such  as  for  corporate  training 
websites. 

The  authors  acknowledge  that  not  all  relevant  websites  were  able  to  be  examined  within  the  funding  and 
time  constraints  of  this  survey.  Websites  are  constantly  changing  and  without  doubt  there  are  now  new  useful 
resources  online.  Nevertheless  we  believe  the  study  is  extensive  enough  for  useful  observations  to  be  made. 


Survey  Findings 

The  survey  identified  a vast  amount  of  online  activity  generated  by  universities.  Some  had  comprehensive 
online  programs,  resources  and  links  in  an  impressive  range  of  areas.  However  only  a relatively  small  amount  of 
activity  was  found  to  be  specific  to  our  staff  development  focus.  Every  university  in  this  survey  provided  some  form 
of  in-house  tuition  for  its  staff  and  students  in  using  the  internet  and  substantial  resources  are  dedicated  to  the 
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development  of  software  and  web  skills.  Staff  development  was  offered  almost  exclusively  via  in-house  workshops 
in  all  universities.  Where  online  modules  were  offered,  content  invariably  focused  upon  opportunities  offered  by  the 
web  as  a communication,  authoring  and  publishing  tool. 

Although  North  American  universities  due  to  their  number  and  size  dominate  the  survey,  Australian 
universities  made  a proportionately  good  showing  in  providing  online  staff  development  resources.  Australian 
universities  have  approximately  a 1:4  ratio  of  providing  some  useful  online  resources,  with  North  American 
universities  the  ratio  was  approximately  1:6,  while  in  the  United  Kingdom  the  ratio  was  1:17.  The  ratio  for  North 
America  is  due  to  the  high  use  of  online  resources  which  originate  from  Teaching  Assistant  handbooks. 

The  online  resources  found  are  predominantly  in  learning  and  teaching  with  a strong  emphasis  on  teaching 
tools  and  the  use  of  information  technology.  There  are  very  limited  online  staff  development  resources  in  university 
leadership  and  management.  Minimal  activity  was  identified  in  research  development,  administration  and  specialist 
support,  with  no  activity  in  the  area  of  community  service. 

Apart  from  skill  training  in  how  to  use  the  internet,  the  survey  found  nothing  by  way  of  a single,  cohesive 
package  of  online  staff  development  specifically  designed  for  both  faculty  and  general  university  staff  The  closest 
to  a comprehensive  online  program  in  university  professional  development  was  found  in  online  courses  of 
professional  accreditation. 


The pdonline  Website  at  <www.cpd.mq.edu.au/pdonline> 

Despite  the  overall  bulk  of  material  the  quality  of  resources  varied  widely  necessitating  careful  assessment 
of  the  materials  chosen  for  inclusion  on  the  pdonline  website.  Of  the  385  universities  surveyed  in  detail  only  65 
universities  were  assessed  as  having  produced  suitable  online  resources.  All  sources  of  chosen  links  were  contacted 
for  their  approval  before  the  website  went  online  and  are  acknowledged  on  the  website.  When  maintaining  the 
website  in  late  1999  13  links  were  no  longer  active  and  were  removed. 

Selection  criteria  included  a judgement  of  the  quality  of  theoretical  rigour  and  written  expression,  and  the 
usefulness  of  the  resource  in  staff  development  as  a self-learning  aid,  an  adjunct  to  workshops,  and  as  a support  for 
future  online  development  activities. 

Learning  and  teaching  development  was  the  best  represented  of  all  the  areas  investigated.  Four  main 
categories  were  devised;  (1)  teaching  advice  including  general  teaching  tips,  planning,  lecturing,  teaching  large 
classes,  group  learning,  diversity,  assessment,  (2)  theories  of  teaching  practice,  (3)  teaching  portfolios,  and  (4j 
teaching  using  the  internet.  Resources  originating  from  secondary  sources  were  incorporated  into  each  category  as 
useful  resources. 

Online  resources  in  leadership  and  management  development  were  scarce.  Only  two  links  originated  from 
within  universities  and  other  links  are  to  private  management  training  corporations  and  to  websites  of  individuals. 

The  site  includes  links  to  three  accredited  online  professional  development  courses  in  higher  education. 


Conclusion 

Our  survey  clearly  indicates  universities  everywhere  are  actively  encouraging  the  use  of  online  media  in 
learning  and  teaching.  Ample  resources  are  available  in  how  the  technology  works  however  staff  are  under- 
represented as  learners  and  consumers  in  these  new  processes.  There  was  no  evidence  of  a comprehensive  program 
of  online  professional  development  specifically  designed  for  higher  education  staff. 

We  located  and  produced  a unique  bibliography  of  links  to  the  'best  of  resources  identified  in  the  survey. 
The  authors  offer  the  pdonline  website  to  support  individualised  learning  plans  and  to  provide  supplementary 
resources  for  workshop-based  programs.  Hopefully  this  work  can  prevent  at  least  some  'reinvention  of  the  wheel' 
when  staff  development  centers  produce  their  own  online  resources. 

Interesting  opportunities  exist  in  using  online  resources  in  the  professional  development  of  higher  education 
staff  Online  communication,  activities  and  resources  can  enable  and  support  'just-in-time',  flexibly  delivered  and 
individualised  learning  plans  and  this  is  the  direction  of  the  authors  applied  research.  The  pdonline  website  is  a key 
resource  to  support  this  work. 
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Abstract:  The  UK  government  has  established  an  Institute  for  Learning  and  Teaching  (ILT)  and  will  soon  require 
that  all  professional  teaching  staff  in  Higher  education  (HE)  achieve  membership  through  either  a portfolio  or 
upon  the  successful  completion  of  accredited  courses.  This  development  is  driving  a rapid  expansion  of  coherent 
staff  development  programmes  offering  full  recognition  of  the  professional  development  staff  undertake.  The  aim 
of  this  paper  is  to  examine  ongoing  research  which  will  infonn  future  developments  in  faculty  development  in  the 
use  of  Communications  and  Information  Technologies  (C&IT)  in  teaching  and  learning.  An  analysis  of  responses 
to  an  accredited  staff  development  programme  is  presented  within  the  context  of  the  shifting  role  of  the  university 
teacher.  This  study  will  be  of  interest  to  staff  developers  who  wish  to  offer  this  form  of  support  to  faculty  within 
their  own  institution. 

The  shifting  role  of  the  university  teacher 

Traditionally  staff  development  in  HE  has  been  through  workshops  offered  on  a need-to-know  basis. 
However,  there  is  no  clear  evidence  to  suggest  that  these  have  made  a major  impact  in  changing  the 
nature  of  teaching  and  learning  in  universities  within  the  UK.  In  1997  the  Scottish  Higher  Education 
Funding  Council  (SHEFC)  commissioned  a nationwide  survey  of  staff  development  needs  aiming  “to 
investigate  the  training  requirements  for  staff  in  HEIs  to  enable  them  to  use  the  MANs  (Metropolotan 
Area  Networks)  effectively  and  efficiently  in  teaching,  learning  and  research”  (Higginson  & Tombs, 
1998).  The  results  of  this  analysis  do  not  reflect  an  adherence  to  current  educational  theories 
advocating  the  development  of  students’  adaptive  and  transformative  attributes.  For  example  most 
academics  advocate  using  the  WWW  for  delivery  of  information  and  teacher-initiated  communication, 
corresponding  to  the  conventional  role  of  the  teacher  as  the  provider  of  knowledge  and  information. 
Likewise  student-centred  methods  were  allocated  low  priority  by  respondents. 

These  findings  highlight  the  limited  conception  amongst  HE  teaching  staff  of  how  to  use  the  World 
Wide  Web  (WWW)  effectively  for  teaching  and  learning,  echoed  in  other  studies  (for  example  Thomas 
et  al,  1998).  Littlejohn  and  Stefani  (1999)  reported  that  academics  often  view  the  WWW  as  another 
means  by  which  they  can  ‘deliver  lecture  notes’,  ignoring  the  importance  of  student  dialogue.  This  was 
due  mainly  to  the  misconception  that  the  use  of  new  technologies  involves  transferring  traditional 
teaching  methods  into  an  electronic  format,  with  no  regard  for  the  underlying  pedagogical  implications. 
It  seems  clear  that  knowledge  of  web  authoring  skills  without  pedagogical  design  skills  will  not  lead  to 
effective  online  teaching.  Despite  this  notion,  a recent  survey  conducted  by  the  Scottish  Higher 
Education  Funding  Council  examining  the  present  state  of  ICT  staff  development  in  Scotland  reported 
a significant  gap  between  coherent  courses  which  are  pedagogically  driven  and  a plethora  of  short, 
technology-driven  workshops  (Alexander,  1999).  There  is  a need  to  bridge  this  gap  by  providing  a 
faculty  development  programme  which  models  current  thinking  in  online  teaching  and  places  faculty  in 
the  role  of  an  online  learner. 

In  1998  the  University  of  Strathclyde  piloted  an  innovative,  accredited  programme  of  this  nature. 
Entitled  ‘Advanced  Academic  Studies’,  the  Postgraduate  Certificate/  Diploma  course  offers  modules 
spanning  various  aspects  of  academic  practice  including  Web  Based  Teaching  and  Internet 
Communication  for  Teaching  and  Research.  The  Web  Based  Teaching  module  takes  place  over  a three 
month  period  with  full  day  face-to-face  sessions.  In  the  first  session  Getting  Started  participants 
critically  evaluate  examples  of  web  based  teaching.  The  next  stage  is  Planning  your  Educational 
Website  in  which  participants  storyboard  an  online  learning  activity.  Following  this  Incorporating 
Interactivity  focusses  on  introducing  online  discussion  and  assessments.  Finally  Creating  your 
Educational  Website  is  when  participants  create  the  necessary  webpages  and  present  their  online 
learning  activity  as  their  final  assessment.  A major  feature  of  this  module  is  that  it  places  faculty  in  the 
role  of  an  online  learner  enabling  experimentation  and  evaluation  of  new  technologies  at  first  hand. 


This  gives  faculty  the  opportunity  to  foresee  learning  and  technological  problems  which  their  students 
may  experience  (Ryan,  1998). 

Analysis  of  faculty  responses  to  this  staff  development  programme 


Group  interviews  followed  by  an  online  evaluation  study  took  place  upon  completion  of  the  Web 
Based  Teaching  module  in  May  1999,  Nine  of  the  ten  course  participants  participated  in  the  evaluation: 
four  faculty  members  and  five  support  staff  including  a librarian,  a web  developer,  a learning  support 
officer  and  audio  visual  staff  All  the  respondents  thought  the  module  increased  their  confidence  in 
using  the  Web  for  teaching  and  learning.  Perceived  benefits  included: 

• The  modeling  of  Web  based  teaching  and  learning  placing  faculty  in  the  role  of  online  learners. 

• The  use  of  learning  activities  building  pedagogical  and  practical  skills  simultaneously. 

• Online  publication  of  assignments  enabling  participants  to  exchange  and  evolve  ideas. 

• Dialogue  resulting  from  exchange  of  ideas  online  promoted  the  development  of  a community  of 
scholars  embracing  the  diversity  of  academic  disciplines  represented  on  the  course. 

• Publishing  assignments  on  the  Web  made  peer  critiquing  easy  to  facilitate  compared  with 
traditional  methods. 

Some  drawbacks  were  also  expressed  including: 

• Participants  sometimes  felt  overwhelmed  by  the  continuous  learning  activities.  As  a result  some 
fell  into  the  role  of  the  ‘convergent’  student,  for  example  by  only  completing  activities  which  were 
to  be  assessed, 

• Installing  software  on  their  own  computers  was  occasionally  problematic  for  participants 

• ‘Drop  in’  in  web  authoring  sessions  were  requested 

Most  participants  stated  that  the  module  enabled  them  to  develop  the  required  skills  to  improve  an 
established  class.  However  the  final  submissions  of  online  courses  indicated  a continued  adherence  to 
teacher-centred  teaching.  Major  barriers  voiced  against  using  the  WWW  for  teaching  focussed  around 
three  issues:  negative  attitudes  of  staff  and  students,  lack  of  time  to  implement  web  based  teaching  and 
poor  technical  assistance.  The  most  participants  had  access  to  an  'IT  literate’  person  within  their 
department  who  could  offer  immediate  help  with  technical  problems.  The  wide  range  of  IT  skills 
within  the  class  may  be  alleviated  in  the  future  by  linking  with  supplementary  IT  skills  courses  offered 
within  the  University.  When  the  module  was  repeated  in  January  - March  2000  there  was  stronger 
emphasis  on  the  importance  of  strong  educational  design.  A group  interview  with  1 5 participants 
offered  evidence  that  participants’  views  had  shifted  slightly  towards  developing  more  student-centred 
courses.  Participants  requested  that  a further  session  could  be  offered  in  future  enabling  development 
and  elaboration  of  their  ideas  in  the  form  of  storyboards. 

The  results  of  this  study  emphasise  the  requirement  for  full  integration  of  current  thinking  in 
educational  technology  into  teaching  practice.  The  next  step  is  to  ensure  that  upon  completion  of  the 
course  faculty  are  encouraged  to  continually  reflect  upon  and  improve  their  teaching  practice.  Ongoing 
research  aims  to  further  improve  this  programme  in  an  attempt  to  make  a greater  impact  on  changing 
teaching  practice  both  within  the  University  of  Strathclyde  and  in  the  Scottish  HE  sector  as  a whole. 
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Abstract.  The  dramatic  growth  of  the  Internet  and  subsequent  use  of  the  Internet  for  Electronic 
Commerce  is  fundamentally  changing  the  business  paradigm,  at  both  a local  and  global  level.  The 
emergence  of  the  global  on-line  economy  is  not  only  providing  immense  opportunity  and  threat  for 
small,  medium  and  large  businesses  at  a micro-economic  level,  but  is  posing  similar  opportunities  and 
threats  at  a macro-economic  level,  all  at  'break-neck'  speed!  This  requires  training  students  and 
business  operators  at  a rapid  rate,  which  raises  some  critical  issues  for  educators,  such  as  course 
structure,  delivery  methods,  collaborative  ventures  with  industry  and  keeping  up  with  technology  and 
business  issues. 


Rationale 

“The  uptake  of  electronic  commerce  has  the  potential  to  transform  the  global  economy  more  rapidly  than 
the  industrial  revolution”  (Department  of  Industry  and  Tourism,  1998).  Projections  like  this  have  serious  training 
ramifications.  How  can  large  numbers  of  business  operators  be  quickly  trained  in  on-line  technology  and  business 
strategies  in  order  to  take  advantage  of  this  new  revolution?  The  Australian  Federal  Government  considers  on-line 
and  Ecommerce  as  being  a priority  area  for  future  growth  in  Australia.  The  Department  of  Communications, 
Information  Technology  and  Arts  (1999)  has  identified  the  following  priorities: 

• maximising  the  efficiency  dividend  from  Ecommerce  for  the  economy  at  large; 

• reducing  barriers  for  the  development  of  Ecommerce;  and 

• demonstrating  a business  case  for  the  integration  of  Ecommerce  into  business 

In  order  to  meet  these  priorities  in  a timely  manner,  training  of  both  the  existing  work  force,  and  of  new 
students  entering  the  business  environment  is  required.  University  courses  spanning  three  years  duration  will  not 
provide  the  body  of  knowledge  or  skills  needed  by  Australian  business  to  compete  in  this  rapidly  changing  on-line 
marketplace.  Graduating  university  students  need  short,  bridging  courses  to  account  for  rapid  changes  in  technology 
and  the  impact  that  such  change  will  have  on  business  practice  and  process.  These  courses  need  to  provide  ongoing 
collaboration  and  mentoring  with  industry  to  ensure  the  knowledge  acquired  is  current  and  relevant. 

Case  Study  - Ecommerce  Intensive  Training  Program 

In  September  1999,  ninety-second  year  multimedia  degree  students  from  Edith  Cowan  University 
completed  a 10-day  intensive  Ecommerce  business  planning  course.  They  had  no  previous  instruction  in  business 
planning  or  any  understanding  of  Ecommerce  issues.  They  were  required  to  develop  an  Ecommerce  business  plan, 
which  could  be  sustained  economically  by  a business  to  sell  products  in  an  on-line  environment,  and  possibly  be 
implemented  anywhere  in  the  world,  including  third  world  countries.  For  example,  Amazon.com  in  Pakistan? 

The  course  was  subsidised  by  the  Office  of  Information  and  Communications  (a  State  Government 
Department),  and  developed  by  Dow  Digital  (an  E-commerce  and  on-line  services  consultancy  and  development 
company).  The  program  was  designed  to  educate  students  and  business  representatives  on  how  to  prepare  an 
Ecommerce  strategy  and  business  plan.  The  pilot  course  covered  six  modules  dealing  with  all  aspects  of  creating  an 
Ecommerce  plan  for  small  to  medium  sized  enterprises.  Presentations  were  made  from  Ecommerce  experts  and  local 
businesses  that  had  already  established  Ecommerce  as  part  of  their  business  model.  The  course  culminated  in  the 
students  presenting  their  plans  to  a panel  of  Ecommerce  experts,  members  of  local  industry  and  their  peers  after  the 
10  days  of  intensive  instruction. 

The  program  was  focussed  on  constructivist  learning  theory  which  emphasised  social  interaction, 
communication,  exchange  of  views,  collaboration  and  support  for  learners  to  take  more  responsibility  for  the 
learning  process  through  learner-centred  tasks  (McLoughlin  & Oliver,  1998;  Collis,  1998).  Students  had  to  work 
collaboratively  to  create  the  Ecommerce  plan  and  share  the  expertise  of  the  group.  Students  were  faced  not  only  with 
new  information  and  material,  but  also  with  a collaborative  learning  method  promoted  by  the  tutors,  course 
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materials  and  student  team  leaders.  The  student  team  leaders  were  trained  over  two  sessions  to  develop  an  emphasis 
on  the  required  collaborative  training  process  and  to  familiarise  themselves  with  the  materials.  They  were 
responsible  for  scheduling  tasks,  keeping  the  project  on  task,  solving  problems,  encouraging  collaboration, 
stimulating  discussion,  and  ensuring  that  each  student  was  contributing  to  the  team  effort.  The  students  presented 
their  Ecommerce  plans  on  the  final  day  of  the  program  where  they  were  assessed  by  a panel  of  industry  experts. 

The  course  ran  for  ten  days  and  covered  the  following  sessions:  Ecommerce  - paradigms  and  predictions;  defining 
Ecommerce  product;  the  Ecommerce  transaction  process;  panel  session  of  local  businesses  conducting  Ecommerce; 
panel  session  of  experts  in  Ecommerce  technology;  refining  the  Ecommerce  plan;  practising  Ecommerce;  feedback 
from  tutors  and  experts  on  student  plans;  finalising  the  plan  and  presentation  of  Ecommerce  plans.  The  overall 
objectives  of  the  course  were  to: 

• provide  a,  ’fast  track'  program  which  trained  students  from  any  discipline  in  Ecommerce  business  planning 

• provide  an  training  model  that  promoted  collaboration  between  industry,  government  and  tertiary  providers 

• assess  whether  the  course  model  could  be  applied  to  developing  countries 


Conclusions 

From  the  experience  of  running  this  course  and  from  feedback  gained  from  the  student  questionnaires,  the 
following  recommendations  are  made  with  a view  of  improving  the  course  delivery: 

• a ten-day  intensive  program  for  this  type  of  course  is  too  short  and  a greater  “lead  time”  was  needed 

• more  time  is  required  from  tutors  and  advisers  in  the  tutorial  sessions  to  help  student  teams 

• each  business  planning  team  should  be  working  on  a “real”  industry  product 

• each  team  should  consist  of  approximately  4 student  members  and  one  industry  representative 

• the  course  is  more  suited  final  year  students  who  are  ready  for  employment 

Feedback  from  both  students  and  the  industry  representatives  strongly  supported  a greater  implementation 
of  this  type  of  Ecommerce  training  course.  Opinion  was  strongly  in  favour  of  promoting  an  understanding  of 
Ecommerce  strategic  thinking,,  business  planning,  Ecommerce  technology  issues,  on-line  marketing  and  the 
generation  of  business  ideas  for  the  on-line  environment.  As  indicated  by  the  literature,  business  is  rapidly  moving 
to  a global  economy  and  judging  from  the  student  and  industry  attitudes  gleaned  from  the  evaluation  sheets,  there 
are  still  many  new  and  fresh  business  ideas  waiting  to  be  exploited! 

However,  for  a course  such  as  this  to  be  successful,  it  must  be  seen  as  valuable  for  all  the  “stakeholders”. 
Government  must  see  tangible  benefits  to  business  and  the  community  in  order  to  be  convinced  to  sponsor  such  a 
course.  The  business  sector  will  want  to  be  involved  if  productivity  and  profit  margins  can  be  potentially  improved, 
and  students  will  want  to  be  involved  if  these  skills  provide  a means  of  securing  employment,  or  as  an  opportunity  to 
start  their  own  on-line  business. 

Ecommerce  is  now  offering  “level  playing  fields”  to  new  business  entrants  who  understand  how  to  take 
advantage  of  this  new  business  paradigm.  Existing  and  new  business  merchants  cannot  afford  to  ignore  this  trend, 
which  is  continually  being  highlighted  by  government  and  other  strategic  research  bodies.  Ecommerce  training 
courses  are  needed  for  business  operators  and  graduating  students  in  a timely,  efficient  and  cost  effective  manner. 
These  courses  must  be  well  structured  and  correctly  marketed  to  attract  and  promote  this  new  form  of  trading.  The 
course  implemented  at  Edith  Cowan  University  provides  an  example  of  how  students  and  business  operators  can  be 
quickly  skilled  in  this  rapidly  changing  area. 
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Abstract:  This  paper  represents  initial  exploration  of  the  relative  merits  of  information  literacy 
development  for  university  undergraduates  through  stand-alone  courses  or  an  across-the- 
curriculum  approach.  Reasons  for  preferring  stand-alone  courses  relate  to  the  a)  definition  and 
scope  of  information  literacy;  b)  rapidly  evolving  nature  of  the  Web;  and,  c)  various  levels  of  prior 
exposure  to  Web-based  learning  represented  among  undergraduate  students  in  the  U.S. 


A dilemma  exists  regarding  the  teaching  and  learning  of  information  literacy  among  U.S.  college  and  university 
students.  With  the  increasing  reliance  on  the  Web  as  a source  of  information,  both  to  augment  and  increasingly  to 
supplant  traditional  course  delivery,  a pedagogical  and  policy  issue  is  whether  we  need  stand-alone  courses,  an 
across-the-curriculum  approach,  or  some  combination  of  both,  in  an  effort  to  insure  that  students  are  armed  with 
certain  lowest  common  denominator  skills  and  strategies  for  learning  from  the  Web. 

This  paper  represents  an  initial  exploration  of  the  relative  merits  of  providing  information  literacy  development  for 
undergraduates  through  stand-alone  courses  or  an  across-the-curriculum  approach.  The  stand-alone  course 
approach  provides  assurance  that  the  identified  skills  and  strategies  receive  due  and  explicit  attention.  However, 
the  stand-alone  model  begets  the  transfer  of  learning  problem  (Anderson,  1995),  i.e.,  the  frequent  failure  of 
learners  to  transfer  skills  and  strategies  to  situational  contexts  which  demand  these  abilities. 

Nevertheless,  there  are  three  primary  reasons  for  preferring  stand-alone  courses  at  the  present  time,  relating  to  the 
a)  definition  and  scope  of  information  literacy;  b)  rapidly  evolving  nature  of  the  Web;  and,  c)  various  levels  of  prior 
exposure  to  the  Web  among  U.S.  undergraduates. 

Information  Literacy 

First,  Marchionini  (1999,  p.  18)  defines  information  literacy  as  "a  continuum  of  skills,  concepts,  attitudes,  and 
experiences  related  to  information  access,  understanding,  evaluation,  communication,  application,  creation,  and 
values.”  Similarly,  Shapiro  and  Hughes  (1996)  say  that  information  literacy  consists  of  at  least  seven  sub- 
literacies: 1)  handling  basic  hardware/soflware  (tool  literacy),  2)  mining/managing  databases  and  archives 
(resource  literacy),  3)  recognizing  social  and  institutional  effects  on  information  constructs  (sociostructural 
literacy),  4)  employing  investigational  tools/strategies  (research  literacy),  5)  communicating  findings  (publishing 
literacy),  6)  maintaining  currency  with  advancements  in  information  age  tools  (emerging  technology  literacy),  and 
7)  evaluating  the  worth  of  information  and  information  technologies  (critical  literacy).  How  do  we  know  that  due 
attention  is  being  given  to  this  intricate  array  of  skills  and  strategies  without  offering  dedicated  stand-alone 
information  literacy  courses? 

The  Protean  Web 
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Secondly,  even  a cursory  look  at  the  medium  of  the  Web  itself  reveals  a size  and  complexity  which  beg  for  a grasp 
of  the  whole  as  well  as  of  its  parts.  By  one  estimate  over  800  million  Web  pages  have  been  created  (Rabinovitch, 
1999).  The  search  engines  designed  to  assist  users  with  selection  and  access  seem  increasingly  inefficient  relative 
to  this  proliferation,  although  recent  refinements  promise  improvements  (Greenberg  & Garber,  1999).  The  Web 
also  changes  constantly  with  sites  appearing  and  evaporating  in  relatively  short  periods  of  time.  Many  sites  remain 
in  a limbo  of  partial  construction.  The  validity  of  information  on  websites  as  well  as  the  credibility  of  their  creators 
often  seem  difficult  to  ascertain  with  any  degree  of  confidence  for  one  not  already  an  expert  in  the  domain  of 
knowledge  and  opinion  thereby  represented.  Once  again,  is  it  reasonable  to  expect  that  the  multiple  strategies  and 
skills  needed  to  tackle  questions  of  authority,  accuracy,  and  validity  as  students  make  their  way  through  the 
Protean  entity  of  the  Web  will  be  sufficiently  taught  without  stand-alone  courses  devoted  to  this  purpose? 

Narrowing  the  Digital  Divide 

Finally,  discussions  of  the  digital  divide  (National  Telecommunications  and  Information  Administration  [NTIA], 
1999)  and  technological  apartheid  (Castells,  as  cited  by  Gerstner,  1999)  strongly  suggest  and  the  data  show  (NTIA, 
1999)  that  there  are  wide  gaps  in  the  information  literacy  of  various  groups  of  students  entering  higher  education 
in  the  U.S.  Should  not  courses  therefore  be  provided  which  are  designed  to  level  the  technological  playing  field? 
Stand-alone  courses  in  information  literacy  have  the  potential  to  help  insure  that  the  digital  divide  is  diminished 
rather  than  reinforced  within  institutions  of  higher  education.  Of  course,  stand-alone  study  and  learning  skills 
courses,  to  refer  to  one  example  from  current  practice,  because  they  are  often  seen  as  remedial  in  nature  may  have 
a corresponding  stigma  associated  with  them.  However,  offering  stand-alone  courses  does  not  necessitate  buying 
into  remedial  models  of  intervention.  Such  implementation  mistakes  are  to  be  avoided  in  providing  information 
literacy  courses. 

A Dual  Approach  for  the  Foreseeable  Future? 

While  an  initial  discussion  of  the  relative  merits  of  stand-alone  and  embedded  or  across-the-curriculum  models  for 
teaching  undergraduate  information  literacy  may  raise  more  questions  than  it  answers,  the  level  of  dialogue 
surrounding  the  issue  should  increase.  More  of  us  should  be  debating  the  question:  Should  teaching  information 
literacy  to  undergraduates  be  accomplished  via  stand-alone  courses  or  an  approach  of  developing  information 
literacy  across  the  curriculum,  or  both?  The  already  crowded  curricula  of  undergraduate  programs  lessen  the 
likelihood  of  offering  dedicated  stand-alone  courses.  However,  unless  existing  courses  throughout  the  curriculum 
are  evaluated  for  the  extent  to  which  they  incorporate  information  literacy,  there  are  no  assurances  of  systematic 
teaching  of  critical  thinking  for  Web-based  learning.  A reasonable  approach  may  be  to  advocate  for  both:  stand- 
alone courses  to  narrow  the  digital  divide  and  across-the-curriculum  efforts  to  teach  information  literacy  in  context. 
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Abstract 

The  use  of  Virtual  Reality  (VR)  in  education  has  been  depicted  as  an  interesting  and 
promising  area  where  constructivist  learning  theory  can  be  put  into  practice.  This  paper 
describes  a work-in-progress  project  attempting  to  take  advantage  of  the  current  VR 
technology  for  constructivist  learning  practices  in  the  teaching  of  secondary  school  Physics. 

The  project  involves  the  creation  of  3D  VR  enviromnents  for  use  on  the  World  Wide 
Web  where  learners  are  exposed  to  virtual  physical  environments.  These  environments 
incorporate  rich  and  meaningful  physical  phenomena  and  relationships  so  that  learners  can 
navigate  and  explore  freely,  view  from  different  perspectives  and  visualize  the  various 
physical  concepts  and  relationships  which  otherwise  can  be  difficult  to  visualize  using 
traditional  teaching  material  in  the  form  of  texts  and  graphics. 

This  paper  also  discusses  the  initial  reactions  from  learners  as  well  as  colleagues  teaching 
on  the  subject. 

Introduction 

The  central  tenet  of  Constructivism  is  that  knowledge  of  the  world  is  constructed  by  the  learner  (Boyle 
1997).  Constructivism  proposes  that  courseware  should  be  designed  in  such  a way  to  provide  an  environment 
conducive  to  the  construction  of  knowledge  by  the  individual,  rather  than  supplying  knowledge  directly. 
Constructivism  also  emphasizes  collaborative  learning  and  personal  introspection  into  one's  learning  process 
(Brooks  & Brooks  1993,  1996).  It  has  been  viewed  by  many  (e.g.,  Moore  1995;  Osberg  1997)  that  virtual  reality 
(VR)  offers  promising  grounds  for  creating  and  developing  learning  environments  that  promotes  constructivist 
learning. 

VR  refers  to  computer-generated  3-D  interactive  environments  in  which  the  learner  is  an  active  participant 
(Bricken  1991;  Bricken  & Byrne  1992).  The  learners  interact  with  and  act  directly  upon  the  virtual  environment 
and  influences  the  variables  in  the  environment  (Moore  1995).  Although  some  comment  that  the  VR  technology 
is  still  in  a relatively  rudimentary  stage  of  development,  it  is  our  opinion  that  what  is  currently  available  can 
already  be  applied  in  teaching. 

In  secondary  school  Physics  where  many  topics  to  be  studied,  for  example,  those  in  Mechanics,  involve 
cognitive  visualization  of  distance,  direction,  inclination  and  speed  and  intricate  relationships  between  these 
elements,  it  is  our  belief  that  VR  can  play  an  effective  role  in  creating  3-dimensional  virtual  environments  that 
promotes  learning. 

Rationale 

The  traditional  Physics  class  is  predominantly  filled  with  experiments  with  which  students  observe  physical 
phenomena,  record  data,  analyse  data  and  the  relationships  they  exhibit  in  order  to  verify  established  or  current 
theories.  While  doing  experiments  promotes  the  principles  of  the  Scientific  Method,  the  drawback  of 
conventional  classroom  experiments  is  that  it  is  difficult  to  allow  learners  to  design  their  own  learning  curve.  In 
fact,  under  the  constraints  of  the  physical  settings  of  a school  laboratory  and  the  apparent  authority  of  teacher 
instructions,  learners  typically  pursue  in  their  experimental  work  outcomes  that  are  expected  of  the  experiment, 
either  by  the  teacher  or  by  the  textbook.  The  individual  differences  in  learners  are  rarely  addressed  and 
individuals  rarely  have  the  opportunity  to  develop  unique  ways  of  experimentation. 

Moreover,  the  typical  secondary  school  physics  laboratory  is  simply  not  sophisticated  enough  to  meet  many 
learning  needs.  Observation  and  visualisation  of  complicated  physical  phenomena  is  a virtual  impossibility.  The 
collision  of  gas  particles  in  the  micro  world,  the  circular  motions  of  celestial  bodies  in  the  macro  world,  perfectly 
elastic  collisions,  frictionless  environments  are  some  typical  examples. 

To  teach  concepts  and  theories  related  to  these  physical  phenomena,  teachers  often  depend  on  2-dimensional 
chalk  and  board  or  textbook-based  descriptions  and  explanations.  To  learn  them,  students  have  to  rely  on  their 
own  creative  imagination  to  construct  in  their  mind  a 3-dimensional  space.  Blackboard  descriptions,  and  by  the 
same  token  textbook  descriptions,  only  allow  the  learner  to  view  phenomena  from  a single  rather  than  multiple 
perspectives.  The  use  of  static  images  to  teach  ideas  involving  motion  is  an  understandably  difficult  task.  What 
learners  have  constructed  in  their  imaginary  3-dimensional  world  can  differ  tremendously  from  one  another,  and 
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can  deviate  phenomenally  from  what  the  teacher  is  attempting  to  describe.  This  typical  scenario  presents  great 
barriers  in  learning  and  makes  in-depth  classroom  discussion  and  further  exploration  very  difficult. 

This  project  attempts  to  break  through  this  barrier  by  constructing  an  interactive  web-based  VR  environment 
in  the  secondary  school  Physics  teaching  classroom.  This  environment  is  created  with  a PC-based  3D  virtual 
reality  technology.  In  the  initial  stage  of  this  project,  the  following  areas  targeted  at  Form  4 secondary  school 
students  have  been  selected: 

1 . Relative  Motion:  Ships  in  the  Sea  ; Sun  Rise;  Rotation  of  Planets 

2.  Pressure  and  Collision  of  Particles 

3.  Gas  Pressure  against  Gravity 

4.  Periodic  Motion:  Simple  Pendulum;  Pendulum  of  2;  Pendulum  of  5 

Building  and  using  the  VR 

To  build  the  interactive  environment,  which  we  call  3D  Physics  Worlds  the  following  software  has  been 
used: 

• Superscape  3D  Webmaster  (Evaluation  version) 

• Microsoft  Frontpage 

3D  Webmaster  allows  users  to  create  virtual  reality  environments  called  Worlds.  In  its  Drag  and  Drop 
Warehouse,  different  3D  models  including  buildings,  cars,  plants  and  animals  are  available.  Users  can  create 
their  own  models  or  modify  from  ones  available.  Most  of  the  models  used  in  this  project  were  created  by 
modifying  the  primitive  objects  in  the  Drag  and  Drop  Warehouse  using  the  Shape  Editor  provided.  By  assigning 
these  objects  in  different  initial  positions  and  setting  different  initial  velocities  for  them,  a VR  simulation  was 
thus  created. 

Learners  when  working  in  this  virtual  world  can  observe  the  physical  phenomena  from  any  perspective  and 
any  frame  of  reference.  The  navigational  controls  provided  by  3D  Webmaster  are  intuitive  enough  for  secondary 
school  learners  to  grasp  with  ease  in  a couple  of  minutes.  Once  feeling  comfortable  with  the  navigational  tools, 
the  learner  can  view  the  physical  phenomena  from  a perspective  of  his  own  choice.  He  can  then  initiate 
movements  of  objects  by  dragging  and  dropping  and  observe  the  motion  created  in  different  settings.  He  can  also 
pause  all  motions  to  observe  more  closely  for  as  long  as  he  likes,  or  return  to  the  initial  settings  at  his  discretion 
to  start  all  over  again.  This  kind  of  experience  is  not  what  traditional  2-dimentsional  texts  and  graphics  alone  can 
produce. 

Initial  Observation  and  Feedback 

The  material  developed  in  this  project  was  used  with  a group  of  Form  4 students  in  the  1998-1999  academic 
year.  Students  seemed  to  be  greatly  impressed  with  the  possibility  of  being  able  to  control  and  explore  with 
objects  in  a VR  environment.  There  was  an  obvious  surge  in  interest  in  the  subject  and  in  the  motivation  to  learn. 
Many  were  excited  by  the  hands-on  experience  and  expressed  the  opinion  that  it  was  fun  to  learn  compared  with 
a situation  where  only  the  textbook  and  blackboard  would  be  used.  Peer  discussion  and  share  of  experience 
among  learners  was  also  seen  to  increase  as  excitement  about  the  different  observations  mounted. 

Colleagues  teaching  on  the  subject  also  expressed  positive  comments  after  trying  to  use  3D  Physics  World  in 
class.  They  felt  that  the  lesson  was  conducted  with  less  teacher  talk  and  resulted  in  better  understanding  of  the 
subject  being  taught.  Some  commented  on  the  need  to  design  follow-up  activities  to  go  with  the  Web  material  in 
order  to  maximise  learning. 

Conclusion 

It  is  believed  that  this  project  represents  initial  attempts  by  front-line  secondary  school  teachers  taking 
advantage  of  the  available  VR  technology  in  teaching.  It  is  hoped  that  the  project  could  be  further  developed  in 
the  following  lines: 

• Extension  to  other  topics 

• Post-VR  activities 

More  structured  research  can  be  carried  out  to  study  learner  reaction,  and  learning  outcomes  as  compared 
with  the  more  traditional  chalk  and  board  approach.  It  is  our  hope  that  teachers  teaching  on  the  same  topics  will 
attempt  to  use  3D  Physics  World  posted  on  the  World  Wide  Web  (http : //i . am/3  dph vsi cs vvorl d/)  and  give  us  their 
comments  and  suggestions  for  improvement. 
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Introduction 

The  analysis  and  design  of  information  systems  is  not  only  based  upon  the  understanding  of  an 
organisation’s  objectives,  structure,  and  processes,  but  also  on  the  knowledge  of  how  to  exploit  information  for 
advantage. 

Whilst  traditional  teaching  techniques  can  prepare  our  students  with  the  required  methodologies  and 
techniques  that  they  need  to  design  and  develop  information  systems,  it  is  extremely  difficult  within  the  confines  of 
a lecture  room,  to  convey  any  ‘real’  sense  of  the  business  environments  in  which  they  will  have  to  apply  these 
theories.  Our  challenge  therefore  is  to  find  an  effective  method  of  conveying  to  our  students,  with  little  or  no 
business  experience,  the  analytical  techniques  and  interpersonal  skills  required  to  quickly  understand  the  relevant 
structure,  processes  and  dynamics  of  a business  - in  order  for  them  to  practice  their  problem  solving  skills  and 
conduct  a thorough  systems  analysis. 

This  paper  describes  current  work-in-progress  on  a project  to  design  and  develop  a game-play  Intranet- 
based  Multimedia  Teaching  Support  System  (IMTSS)  for  our  information  systems  students.  We  will  first  describe 
the  background  of  the  project.  Then  briefly  discuss  the  methodology  used,  including  game-play  and  the  active 
learning  approach.  Finally  we  will  give  an  overview  of  the  system  and  highlight  our  conclusions  to  date. 

Background 

The  aim  of  the  proposed  system  is  to  assist  and  enhance  the  ‘Systems  Analysis  and  Design’  and  ‘Group 
Project’  course  modules,  which  are  both  core  subjects  of  the  second  year  Computing  Studies  Information  Systems 
(CSIS)  Degree  course  in  the  Department  of  Computer  Science  at  the  Hong  Kong  Baptist  University. 

In  the  past  the  CSIS  degree  course  was  structured  as  a two-phase  five  year  course,  consisting  of  two  years 
full  time  and  three  years  part-time  study,  the  latter  including  three  years  part-time  industrial  experience.  However  as 
a result  of  the  re-structuring  of  our  CSIS  course  in  1995  to  a three  year  full-time  study  program,  many  of  our 
students  no  longer  have  any  direct  IS  related  work  experience.  This  problem  has  been  further  compounded  by  the 
scarcity  of  summer  jobs  resulting  from  the  Asian  financial  crisis. 

Over  the  last  few  years,  our  department  has  conducted  a series  of  feedback  exercises  such  as  informal 
surveys,  teaching  evaluations  and  student-staff  consultation  meetings.  These  have  revealed  that  whilst  our  students 
believe  that  the  IS  concepts  and  methodologies  taught  are  very  useful,  they  have  found  many  of  the  ideas  rather 
abstract.  They  have  therefore  requested  more  relevant  examples  to  explain  these  concepts.  However  as  most  text 
books  originate  from  North  America  or  Europe,  the  case  studies  used  are  often  not  very  relevant  to  Hong  Kong.  In 
addition  our  teaching  staff  have  indicated  that  our  final  year  project  students  not  only  need  to  improve  their 
analytical  interviewing  techniques,  but  somehow  also  need  to  build  their  confidence  at  interacting  with  business 
users. 

Therefore  the  objective  of  our  IMTSS  project  has  been  to  : 

• provide  a framework  based  on  a game  play  approach,  to  help  students  develop  analytical  and  problem 
solving  skills  through  active  learning 

• make  use  of  multimedia  and  internet  technology  to  help  provide  students  with  a virtual  organisational 
environment  in  which  they  can  conduct  their  systems  analysis 

• incorporate  the  above  into  the  design  and  development  of  an  on-line  bilingual  multimedia  teaching  support 
system 
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Methodology 


With  these  objectives  in  mind,  we  chose  to  adopt  a combined  active  learning  and  game-play  approach  in 
the  hope  that  students  could  develop  analytical  and  problem  solving  skills  through  active  learning,  whilst  stimulating 
and  increasing  their  interests  in  the  subject  through  the  game-play  approach. 

This  is  based  upon  much  historical  research  showing  that  playing  games  can  be  highly  motivational  and 
enhance  learning  through  visualisation,  experimentation,  and  creativity  of  play  (Thomas  and  Macredie  1994,  Betz 
1995-96,  Amory  et  al.  1998).  Other  research  has  shown  that  students  learn  more  when  they  are  actively  engaged  in 
their  own  learning,  which  leads  to  an  increased  ability  to  think  critically  and  to  solve  problems.  Furthermore  this 
active  learning  approach  helps  students  to  learn  content  and  process  at  the  same  time  (Booth  1980,  Page  1990, 
Marlowe  and  Page  1998). 

Our  justification  for  utilising  multimedia  technology  in  the  form  of  streamed  video,  is  that  research  has  also 
demonstrated  that  users  playing  simulation  games  performed  significantly  better  with  highly  graphical  feedback, 
than  when  the  simulation  provided  only  textual  feedback  (Rieber  1996). 

The  IMTSS  System 

For  this  project,  we  have  initially  chosen  a number  of  distinct  case  studies  from  various  Hong  Kong 
business  sectors.  For  each  of  these  cases  our  research  team  have  conducted  a series  of  interviews,  in  order  to  collate 
relevant  material  from  the  chosen  end-users.  This  material  has  then  been  analysed,  in  order  to  create  the  business 
case  structure  and  scripts  for  the  multimedia  teaching  support  system.  These  have  then  been  translated  into 
corresponding  English  and  Cantonese  scripts. 

Each  case  study  is  comprised  of  a series  of  interactive  multimedia  elements,  most  of  which  are  video  clips. 
These  video  clips  were  either  taken  of  business  people  on  their  own  premises,  or  conducted  by  student  actors  at  the 
University.  These  elements  include  explanations  of  the  organisations  background,  current  work  environment, 
business  processes,  and  interviews  with  relevant  staff.  Once  the  system  content  has  been  acquired  and  edited,  it  is 
stored  into  a client-server  database.  The  overall  system  framework  makes  use  of  an  intranet  web  server  to  control 
the  interaction  between  the  user  and  the  multimedia  elements  held  in  the  content  database.  We  have  conducted  much 
research  into  the  investigation  of  optimal  video  delivery  methods,  including  clip  duration,  file  size  and  effective 
delivery  rates. 

For  each  case  study,  the  student  is  provided  with  an  introductory  video  describing  the  relevant  business 
environment,  followed  by  an  introductory  interview  from  the  case  study  sponsor.  This  introduces  them  to  the 
various  staff  that  are  available  for  interviewing.  They  are  then  presented  with  a variety  of  options  to  navigate  their 
way  through  the  whole  series  of  interviews.  For  each  interview  a series  of  questions  may  be  asked,  and  each  of  these 
are  tagged  as  good,  neutral  or  bad.  If  the  student  chooses  a good  question  then  they  are  rewarded  with  a 
complimentary  video  clip  and  given  some  useful  information.  However  should  they  choose  a bad  question  then  they 
may  be  confronted  with  a bored,  sarcastic  or  annoyed  response.  In  this  way  the  students  are  able  to  navigate  their 
way  through  the  various  interviews,  collecting  various  pieces  of  useful  information,  and  identifying  the  users 
requirements.  If  they  run  into  problems  they  are  able  to  return  to  the  sponsor  to  get  some  additional  hints,  although 
this  will  decrease  their  overall  score.  The  progress  of  each  student  is  monitored  throughout  by  the  web  server. 

The  proposed  framework  allows  for  the  number  of  interviews  per  case  study,  and  their  relevant  importance 
to  be  weighted  individually.  It  is  also  designed  to  be  both  flexible  and  scalable,  in  order  to  allow  additional  business 
cases  or  alternative  languages  i.e.  Mandarin  Chinese,  to  be  added  over  time.  Furthermore  this  flexibility  also  allows 
for  existing  multimedia  elements,  that  have  been  identified  as  causing  confusion  for  students,  to  be  easily  updated  or 
replaced. 

Conclusions  and  Discussion 

Although  research  is  still  ongoing,  results  so  far  from  our  prototype  version  of  IMTSS  have  been  extremely 
encouraging,  and  have  clearly  indicated  that  the  system  can  provide  our  IS  students  with  a stimulating  and  valuable 
means  of  bridging  the  gap  between  the  theory  learnt  in  the  classroom,  and  practice  in  the  business  environment. 
Furthermore,  the  use  of  IMTSS  also  appears  to  help  our  students  gain  confidence  to  interact  with  end-users,  which 
in  turn  should  improve  their  final  year  honours  projects,  as  well  as  better  prepare  them  for  their  careers. 

In  addition  we  hope  to  demonstrate  the  potential  benefits  of  this  type  of  flexible  framework  to  complement 
traditional  teaching  methods.  Where  students  can  study  at  their  own  pace,  and  in  the  language  of  their  choice,  free 
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from  the  normal  time  and  geographic  constraints  of  lectures.  Also  we  are  currently  investigating  ways  in  which  the 
system  can  be  further  improved  to  better  monitor  the  students  progress  and  absorption  of  study  material,  as  well  as 
highlight  problem  study  areas. 

Finally,  we  believe  that  the  framework  and  methodology  used  in  this  system  could  also  be  adopted  by  other 
faculties  such  as  Business,  Management,  Accounting  etc.  whose  students  also  need  to  develop  analytical  skills  for 
certain  sections  of  their  courses. 
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Introduction 

The  Computerized  Oral  Proficiency  Instrument  (COPI),  is  a multi-media,  computer-administered  oral 
proficiency  test  that  is  an  adaptation  of  the  tape-mediated  Simulated  Oral  Proficiency  Interview  (SOPI) 
(Stansfield  & Kenyon,  1992).  The  COPI  uses  simulated  real  life  tasks  to  elicit  speech  samples  and  is  rated 
according  to  the  Speaking  Proficiency  Guidelines  of  the  American  Council  on  the  Teaching  of  Foreign 
Languages  (ACTFL)  (ACTFL,  1999).  The  COPI  has  two  parts:  a self-administration  program  for 
examinees  and  a rating  program  for  human  raters. 


Self-Administration  Program 

Steps  in  Taking  the  COPI 

After  being  welcomed  and  introduced  to  the  program,  examinees  perform  the  following:  enter  and  check 
personal  information,  complete  a self-assessment  of  their  proficiency  level,  listen  to  an  adequate  response 
to  a sample  task,  practice  with  the  same  sample  task,  respond  to  performance  tasks  (the  actual  test),  and 
receive  feedback  about  the  levels  of  their  tasks.  While  taking  the  test,  examinees  hear  test  directions  with 
accompanying  text  and  pictures  in  English  and/or  in  the  target  language  (depending  on  the  proficiency  level 
of  the  task)  and  record  their  responses.  Depending  on  the  choices  that  examinees  make,  they  respond  to  7- 
1 1 test  items  and  take  anywhere  from  30-50  minutes  to  complete  the  test. 


Examinee  Control 

The  main  goal  of  the  COPl's  self-administration  program  is  to  use  the  advantages  of  computer  technology 
to  provide  examinees  more  control  over  the  testing  situation.  Our  assumption  is  that  more  test  control  will 
lead  to  anxiety  reduction  and  ultimately  improve  test  performance.  Examinee  control  is  achieved  by  giving 
them  choices  in  the  following:  amount  of  preparation  and  response  time,  speaking  function,  topic,  level  of 
difficulty,  and  language  of  directions.  The  large  pool  of  tasks  (over  100  per  language)  is  also  very 
important  in  ensuring  that  examinees  can  make  these  choices. 

Examinee  Choice 

The  COPI  provides  more  preparation  and  response  time  for  higher-  than  lower-  level  tasks,  but  examinees 
still  have  the  choice  to  use  up  all  the  time  allotted  or  take  less  time.  Also,  after  completing  a task,  they  are 


given  a choice  of  three  new  tasks,  and  an  algorithm  ensures  that  they  get  each  speaking  function  and  topic 
only  once.  Likewise,  the  examinees’  self-assessment  scores  and  the  choices  of  levels  during  the  practice 
task  determine  the  level  of  difficulty  for  the  first  task.  However,  after  the  first  task,  the  program  alternates 
between  giving  examinees  a choice  of  difficulty  level  or  not.  During  the  times  that  examinees  are  not  given 
a choice,  the  algorithm  pushes  them  to  higher  level  tasks  to  ensure  that  they  are  given  an  opportunity  to 
reach  their  performance  ceilings.  Lastly,  lower-level  speakers  are  provided  with  both  English  and  target 
language  directions  whereas  higher-level  speakers  are  given  a choice  of  getting  the  directions  in  English  or 
just  the  target  language. 


Rating  Program 

The  main  goal  of  the  rating  program  is  to  increase  raters'  efficiency,  i.e.,  help  them  rate  as  quickly  but  as 
accurately  as  possible.  The  rating  program  does  this  by  allowing  raters  to  quickly  access  the  examinees' 
audio,  navigate  quickly  from  one  task  to  another,  and  listen  to  the  tasks  in  any  order.  Also,  as  raters  assess 
each  task,  elements  of  the  task,  such  as  its  proficiency  level,  the  picture  accompanying  the  task,  directions, 
and  the  native  speaker  prompt  appear  on  the  screen.  These  elements  give  raters  background  information 
about  the  task  and  facilitate  the  rating  process.  The  program  likewise  allows  raters  to  write  notes  to 
examinees  so  that,  aside  from  providing  a global  rating,  raters  can  give  task-specific  and  overall  feedback 
to  the  examinees.  In  addition,  the  raters  can  write  notes  to  themselves  about  the  examinees'  responses  and 
can  go  back  to  these  notes  when  determining  the  task  rating.  Lastly,  an  algorithm  calculates  the  global 
rating  for  each  examinee. 


Technical  and  Programming  Requirements  and  Specifications 

The  COPI  program  works  well  with  computers  that  have  a Windows  95  operating  system  or  higher,  a 
Pentium  processor  and  64  MB  of  RAM  and  a soundcard.  The  examinees'  responses  can  be  recorded  on 
internal  or  external  zip  drives,  or  the  hard  drive.  We  do  not  recommend  the  use  of  Windows  NT  or  laptop 
computers  because  the  small  memory  space  in  these  computers  makes  recording  difficult.  The  program  was 
developed  using  Authorware  4,0.  The  directions  in  Arabic  script  were  gif  files  inserted  in  the  text  box. 


Future  Directions 

We  just  completed  a study  comparing  student  and  rater  performance  and  affect  on  the  COPI  versus  the 
face-to-face  interview  or  Oral  Proficiency  Interview  (OPI),  and  the  COPI  versus  the  tape-mediated  or 
Simulated  Oral  Proficiency  Interview  (SOPI),  The  results  of  this  study  will  show  us  if  the  COPI  is  a viable 
option  to  the  Oral  Proficiency  Interview  (OPI),  which  can  be  too  expensive  or  inconvenient;  and  if  the 
COPI  is  a significant  improvement  over  other  technology-based  oral  proficiency  tests  such  as  the  SOPI. 
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The  subjects  in  this  research  were  Year  Seven  (12  year  old)  students  (n=  42)  from  two 
primary  schools  in  Western  Australia.  Our  prediction  was  that  temporal  speech  cueing 
would  better  enable  school-age  students  to  focus  their  attention  than  those  in  the  temporal 
text  cueing  group.  Small  sample  sizes  are  not  uncommon  in  experimental  multimedia 
research  (Austin,  1994;  Bridgman,  1990;  Carroll  & Mack,  1987;  Shell,  1990).  Year  seven 
students  were  chosen  for  the  study  because  this  is  the  final  year  of  primary  school  and 
thus  the  results  from  the  study  were  more  likely  to  be  relevant  for  both  primary  and 
secondary  students.  In  addition,  primary  school  students  were  more  suitable  because  the 
primary  school  curriculum  is  more  thematic  and  flexible  readily  allowing  the  addition  of 
content  and  a variety  of  teaching  situations. 

Materials 

The  "Engines"  computer  program  was  developed  as  a sequential  hypermedia  learning 
environment  to  provide  learners  with  a linear  path  through  a set  of  related  nodes,  as 
suggested  in  previous  studies  (Mann,  1993;  1995a;  1997a).  Mann's  (1996)  procedure  for 
adding  sound  prompts  to  hypermedia  learning  environments  was  used  for  the  "Engines" 
program.  "Temporal  speech  cueing"  was  operationalized  as  information  about  future  or 
past  events  digitized  in  enthusiastic  female  speech  prompts.  Also,  future  or  past  event 
speech  or  text  was  implemented  to  elicit  student  writing  in  the  Workbook.  Further,  the 
Engines  program  utilized  Mayer's  (1992)  "description  of  phenomena"  instructional 
approach,  wherein  a scientific  explanation  exists  when  a scientist  observes  events  and 
describes  them.  In  this  study,  the  student-scientists  read  or  listened  to  information, 
observed  animated  graphics  and  movies,  and  described  their  observations  in  a workbook. 
Both  versions  of  the  software,  test,  workbook  and  questionnaire  were  trialled  with  Year 
Seven  students.  The  software,  test,  workbook  and  questionnaire  were  modified  based  on 
their  comments.  Two  versions  of  a simple  tutorial  program  introducing  "Insects"  was 
developed  to  be  used  as  practice  for  the  "Engines"  treatment. 

Design  of  the  Study 

This  study  employed  a repeated  measures  quasi-experimental  design  with  two  treatment 
conditions,  temporal  speech  cueing  and  temporal  text  cueing.  Students  were  randomly 
assigned  to  one  of  the  two  treatment  conditions  based  on  their  gender,  reading  ability  as 
determined  by  their  teachers'  long-term  observations,  perceived  modality  preference  on 
the  auditory-visual-tactile/kinesthetic  learning  style  inventory  (Murphy-Judy,  1998),  and 
students'  attitudes  toward  using  computers  as  determined  by  the  Computer  Attitudes 
Scale  (Loyd  and  Gressard,  1994). 
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Procedure 


The  procedure  in  this  study  followed  those  suggested  in  previous  studies  of  this  kind 
(Knussen,  Tanner,  & Kibby,  1991;  Mann,  1997a).  A number  of  activities  were  done 
simultaneously  and  the  final  schedule  was  established  at  the  mutual  convenience  of  the 
participants  and  the  investigator.  First,  the  consent  of  teacher,  principal,  and  parents  were 
acquired.  Next,  over  the  course  of  a week  of  classtime,  a reading  test,  computer  attitude 
survey,  modality  inventory  were  administered.  Later  that  week  the  computers  were 
installed  and  checked  (19/5/99).  A week  later  the  pre-test  (26/5/99)  was  administered  by 
teacher.  The  two  treatment  groups  were  created  by  the  researchers  based  on  students 
perceived  modality  preference,  gender,  reading  ability,  and  attitude  toward  using 
computers. 

Three  weeks  after  the  pretest  was  administered,  the  treatment  and  posttest  were  given. 
Each  class  was  done  separately  as  was  each  treatment  group.  The  main  intervention  was 
conducted  by  two  of  the  researchers  using  the  1 3 computers  in  the  room  set-up  between 
the  two  classrooms.  The  entire  process  lasted  just  under  40  minutes  and  comprised  the 
following  steps.  On  the  day  of  the  treatment,  subjects  received  information,  training  and 
practice  on  the  "Insects"  program.  The  main  intervention  involved  the  students  in  using 
the  "Engines"  program.  Subjects  were  instructed  to  proceed  through  the  program  while 
following  the  instructions  and  writing  their  answers  in  their  Workbooks.  The  post-test 
was  then  completed  after  they  had  quit  the  "Engines"  program.  Finally  some  free  time  to 
use  the  KidPix  graphics  package.  No  instruction  or  feedback  was  presented  by  the 
investigators  at  this  time.  Notes  of  the  proceedings  were  maintained. 

Results  and  Discussion 

A repeated  measures  ANCOVA  calculated  on  the  data  using  the  pretest  as  a covariate. 
Results  of  the  analyses  showed  that  statistical  differences  between  auditory  and  visual 
groups  were  not  significant.  A correlation  matrix  showed  low  positive  correlations 
throughout  all  the  scales,  with  none  significant.  The  dribble  files  showing  efficiency  with 
the  program  were  also  not  significant.  The  consensus  among  the  investigators  however, 
was  that  the  auditory  group  appeared  to  be  more  focused  on  the  critical  information 
presented  in  the  computer  program  than  the  visual  group.  During  temporal  speech  cueing, 
the  room  fell  to  silence  with  all  eyes  focused  on  their  screens.  We  speculated  that  the 
students  in  this  group  were  more  involved,  because  they  were  listening  to  the  instructions 
and  directions  coming  through  their  headphones.  During  temporal  text  cueing  however, 
the  students  were  restless  in  their  chairs,  frequently  looking  around  the  room;  the 
investigators  having  to  remind  them  not  to  talk  to  their  neighbour  and  to  keep  their  eyes 
on  their  own  computers  and  workbooks. 

Conclusion 

We  think  that  the  significance  of  this  study  resides  in  our  discovery  that  temporal  speech 
cueing  in  educational  multimedia  is  only  necessary,  and  therefore  need  only  be  employed 
on  difficult,  divided-attention  tasks.  Unlike  previous  research,  our  results  appear  to  have 
been  a consequence  of  tasks  that  were  too  easy  or  too  familiar  for  these  students.  You'll 
recall  that  on  easy  tasks  with  familiar  items,  people  use  automatic  processing  which 
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occurs  in  parallel;  that  is,  we  can  process  two  or  more  different  inputs  simultaneously. 
Apparently,  students  in  the  present  study  implemented  pre-attentive  processing  to  work 
on  easy  tasks  or  highly  familiar  items  presented  in  the  "Engines"  program.  We  think  that 
our  conclusion  is  a significant  finding  in  itself,  providing  a new  proviso  into  educational 
media  research,  which  previously  reported  that  the  presence  of  speech  could  improve 
multimedia  learning.  A follow-up  study  has  been  planned  at  a different  campus  of  the 
same  school.  Based  on  the  findings  of  the  present  study,  the  Engines  multimedia 
program  will  be  revised  to  require  serial  processing  from  this  same  population  of 
students. 
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PHASE  THEORY:  A TELEOLOGICAL  TAXONOMY  OF  WEB  COURSE 
MANAGEMENT  FOR  DISTANCE  EDUCATION 

Bruce  Mann 

Memorial  University  of  Newfoundland,  Canada 


Theories  are  meant  to  explain  something  or  to  help  us  understand  the  way  things  are 
within  a certain  domain  (Wilson,  1997).  Phase  theory  is  intended  to  help  educators 
understand  the  way  things  are  for  using  Web  course  management  systems.  A Web  course 
management  system  is  a set  of  software  tools,  customarily  grouped  together  under  a 
course  name  and  protected  by  a password,  and  considered  a system;  a Web  course 
management  system  (WCMS).  Some  examples  of  popular  WCMS’s  include 
LearningSpace  (Lotus  Corporation,  1998),  Virtual-U  (Harasim  & Calvert,  1997), 
TopClass  (WBT  Systems,  1997),  and  If (Goldberg,  1997),  to  name  a few.  Central 
to  phase  theory  is  the  view  that  that  instructors  learn  to  adapt  their  instruction  to  a Web 
course  management  system  in  phases,  namely;  Lesson  Enhancement,  followed  by 
Resource-Based  Teaching  and  Learning,  then  Online  Learning  Environments. 

Lesson  Enhancement 

Lesson  enhancement  is  the  initial  phase  of  Web  course  management  and  is  often 
introduced  to  students  as  an  extracurricular  activity.  Usually  with  help  from  the  resident 
technologist,  the  educator  will  decide  to  introduce  the  Internet  or  Web-supported  material 
as  extra-curricular  activities  to  enhance  the  curriculum.  Three  types  of  lesson 
enhancement  are  classified  for  this  phase,  as  follows:  1)  immersive  collaborative 
environment;  2)  online  self-expression;  3)  online  lesson  assessment. 

Resource-Based  Teaching 

Resource-based  teaching  is  the  second  phase  of  Web  course  management.  "Resources" 
can  be  of  any  form,  namely:  text,  pictures,  video  clips;  and  of  any  knowledge-type- 
declarative,  procedural  and/or  strategic  knowledge  for  subsequent  retrieval  by  students. 
The  mindset  for  resource-based  teaching  is  similar  to  that  of  storing  and  retrieving 
materials  in  a media  repository  or  school  library.  I have  counted  as  many  as  fifteen  types 
of  RBTL  using  a Web  course  management  system  and  operated  directly  by  instructors 
themselves,  namely:  1)  to  provide  content  (i.e.,  online  notes,  online  reader,  or  an  online 
resource  pack);  2)  to  support  a learning  activity  (i.e.,  online  manual,  online  lab  guide,  a 
seminar  guide,  a fieldwork  guide;  online  projects  facts  guide,  or  an  online  work 
placement  guide);  3)  to  support  a learning  process  (i.e.,  online  skills  guide,  skills  profile, 
or  an  online  student  log);  or,  4)  to  build  on  other  resources  (i.e.,  online  textbook  study 
guide,  online  readings  guide,  or  an  online  lesson  outline).  These  types  are  reminiscent  of 
the  conventional  resource-based  learning  categories  described  in  Parsons  and  Gibbs’ 
(1994)  “Developing  Teaching”  series  published  by  Oxford  Brookes  University  Press. 
Resource-based  teaching  may  require  a re-defmition  of  pedagogical  goals,  restructuring 
of  curricular  offerings,  provision  for  educator  training  and  support  material,  and 
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sufficient  online  tools  for  the  collection  of  student  data.  Terms  like  "stockpiling", 
"massing",  "stacking",  "accessing",  "accumulating"  might  well"  define  the  linguistic 
framework  of  the  resource-based  teaching  phase. 

Online  Learning  Environment 

An  “online  learning  environment”  is  the  third  phase  of  Web  course  management  that 
demonstrates  the  application  of  cognitive  psychology  to  education.  A learning 
environment  is  a place  where  learners  work  together  and  support  one  another  as  they  use 
a variety  of  tools  and  information  resources  in  their  pursuit  of  learning  goals  and  problem 
solving  activities  (Wilson,  1997).  The  focus  is  on  meaningful  learning  that  is  active  and 
involves  the  students  in  learning  by  discovery.  Five  components  comprise  the  learning 
environment,  adapted  from  Perkins  (1991),  namely:  1)  information  banks;  2)  symbols 
pads;  3)  construction  kits;  4)  phenomenaria;  5)  online  microworlds,  and;  6)  virtual 
environments. 


Conclusion 

Phase  theory  can  assist  practitioners  in  the  determination  of  their  own  inventions  with 
Web  tools  based  on  the  their  intuition,  personal  preferences  and  prior  educational 
experiences  with  Web  course  tools  (Maim,  1998b).  And  as  a descriptive  taxonomy 
(Reigeluth,  1997),  phase  theory  may  be  applied  as  a descriptive  theory  of  professional 
development  in  Web  course  management.  One  future  direction  would  be  to  implement 
phase  theory  as  an  instructional  strategy  to  train  a different  sample  from  the  same 
population  of  novice  Web  managers.  See  Mann  (2000;  1999a;  1999b;  1999c;  1998b; 
1998e)  for  further  discussion  on  the  first  principles  and  underlying  assumptions  of  phase 
theory. 
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Reconfiguring  the  Academic  Library  for  the  New  World  of  Learning 
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Marcum@postbox.csi.cuny.edu 


Abstract:  Knowledge  modeling,  knowledge-based  systems,  and  other  technology-based 
learning  support  systems  utilized  in  business  training  can  help  the  academic  library 
meet  the  challenges  of  refocusing  attention  from  instruction  to  learning. 


While  today’s  academic  library  is  undergoing  major  changes  in  structure  and  purpose,  it  remains 
fixed  within  a model  of  education  that  steadily  becomes  less  relevant,  the  model  of  instruction  and 
information  processing  and  transfer.  As  attention  shifts  to  what  the  student  actually  learns  (Katz,  1999), 
we  must  rethink  and  reconfigure  the  library  in  order  to  preserve  and  enhance  its  role  in  higher  education. 

The  new  learning  is  influenced  by  the  ideas  of  multiple  intelligences,  learning  styles,  and  the 
psychology  of  individual  differences.  A generation  of  work  in  the  cognitive  sciences  has  demonstrated  that 
the  mind  actively  engages  in  the  creation  of  an  individual’s  “reality”  (Ackerman,  1999).  Yet  many  in 
academe  are  not  yet  cognizant  of  how  individual  learning  profiles,  group  versus  individual  learning 
preferences,  personality  and  trait  factors,  self-monitored  learning,  better  understanding  of  informal 
learning,  and  the  conscious  development  of  learning  strategies  require  new  behaviors  and  new  policies.  We 
must  rethink  and  reconfigure  the  library  to  support  the  emerging  culture  of  learning.  One  option  is  to 
create  a new  environment  where  learning-support  teams  (including  subject  specialists,  information 
specialists,  technical  support,  and  instruction  designers  collaborating  with  expertise  in  personality  and 
intelligence  testing)  maximize  the  possibilities  of  the  new  learning. 

Useful  models  for  emulation  are  available  from  business  and  management  training,  including 
knowledge  modeling,  knowledge-based  systems,  and  other  “learning  strategies.”  For  instance,  “knowledge 
modeling”  stresses  the  role  of  the  “transactional  mind,”  the  central  role  (generally  left  to  chance)  of  other 
influences  upon  the  learner  including  peer  interaction,  self-directed  inquiry,  and  the  context  the  learner 
carries  with  them.  Much  more  is  going  on  in  academia  beyond  the  instructor-guided  model  that  relies  on 
lectures  and  defined  readings  (Greene,  1993,  pp.  513-553).  There  are  other  approaches  to  the  knowledge- 
modeling concept,  running  at  very  different  levels  of  complexity,  from  the  arcane  world  of  artificial 
intelligence  to  a basic  level,  where  knowledge  modeling  can  be  utilized  to  construct  clear,  manageable,  and 
measurable  knowledge  structures  detailing: 

- the  critical  role  of  prior  knowledge  and  context 

- the  differences  between  definitions,  descriptions,  and  comparisons 

- the  nature  of  steady  states 

- cognition,  classification,  and  categorization 

- the  role  of  time  in  the  organizing  of  simple  events 

- steps,  classes,  and  conditions  of  processes 

- knowledge  acquisition  through  accumulation,  tuning,  and  restructuring 

- using  knowledge  models  as  a basis  for  assessing  student  learning  and  knowledge  (Mosenthal  and 

Kirsch,  1991-1992). 

Other  approaches  include  knowledge-based  systems.  These  systems  can  assist  human  operators, 
support  decision-making,  generate,  critique,  and  evaluate  knowledge  structures,  and  monitor 
implementation.  Such  systems  can  address  many  of  the  problems  we  face  today,  including  the  need  for 
individualized  instructional  methods  to  deliver  material  and  dynamically  model  student  comprehension 
(Hayes-Roth  and  Jacobstein,  1 994).  Indeed,  we  must  ask  whether  the  learning  process-focused  world  of 
business  training  enjoys  significant  strategic  advantage  over  the  professor-centered  world  of  academe. 

All  of  these  methodologies  amount  to  the  early  development  of  “learning  support  systems,”  an 
approach  that  stresses  the  process  of  learning,  not  the  mastery  of  traditional,  discipline-bound  information 
content.  Utilized  in  business  training,  learning-support  systems  rely  on  mentors  and  facilitators  to  help 
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learners  understand  their  personal  learning  styles,  consider  performance  standards,  and  test  against 
competency  measures  (Davis  and  Davis,  1998).  A case  study  drawn  from  a software  learning  project 
indicates  that  a variety  of  “lesson”  formats,  the  provision  for  experimentation  and  encouragement  of 
mentoring  are  means  of  bolstering  and  supporting  the  learning  process  (Harp,  1997).  Clearly, 
comprehensive  strategies  for  setting  criteria,  measuring  prior  knowledge,  identifying  learning  traits,  and 
tracking  the  accretion  of  knowledge  and  the  impact  of  learning  can  be  realistically  pursued,  particularly 
considering  the  software  available  today. 

The  potential  of  learning-support  systems  contains  exhilarating  implications  for  academic 
libraries.  Learning  is  not  something  that  is  limited  to  the  classroom;  that’s  just  where  information  and  rules 
are  transferred  and  learning  challenges  issued.  Emergent  learning  occurs  in  groups,  in  study  rooms,  in 
laboratories,  in  individual  study,  and  in  dialogue  (both  face-to-face  and  electronic).  To  date,  academic 
librarians  have  given  little  thought  to  the  learning  process.  Traditional  library  evaluation  focuses  on 
materials,  access,  and  the  usage  of  materials  and  services.  Only  recently  has  concern  about  the  impact  of 
instruction  and  consultation  become  matters  for  measurement  and  evaluation.  One  prescription  for  this 
work  calls  for  cultivation  of  “active  learning,”  that  is,  crystallizing  our  goals,  studying  the  impact  of 
information  technology  on  learning,  and  studying  how  students  actually  learn.  The  key  is  providing 
context,  and  problem  solving  is  one  good  approach  to  that  end.  Academic  librarians  can  and  should  adopt 
the  goal  of  becoming  the  locus  and  advocate  of  technology-based  learning  and  teaching  on  the  campus. 
(Wilkinson,  pp.  181-196). 

What  does  all  this  demand  of  the  library?  First  of  all  it  requires  thinking  beyond  the  “digital 
library”  to  embrace  a range  of  patron  behaviors  - such  a group  learning  and  team  development  - as  well  as 
more  comprehensive  utilization  of  continuous  media.  New  organizational  structures  are  needed  to  utilize 
the  more  varied  professional  skills  demanded  by  this  complex  learning  environment.  Other  requirements 
include  greater  learner  access  to  the  Internet  (via  ubiquitous  data  outlets),  more  powerful  networked  servers 
for  video  storage  and  on-demand  delivery,  experimentation  with  thin-client  and  wireless  configurations, 
and  widespread  use  of  “output”  devices  (such  as  CD-ROM/DVD  printers),  together  with  a variety  of  group 
study  spaces,  computer  labs,  and  support  for  group  inquiry  by  means  of  “collaborative  portal”  technologies. 
All  this  places  enormous  new  expectations  on  an  already  overburdened  institution.  The  challenge  and  the 
opportunity  are  there  if  we  will  but  seize  them. 
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Abstract:  In  the  Summer  of  1999,  a population  of  University  of  Virginia  faculty  completed  a 
survey  regarding  a new  interface  for  the  Faculty  Instructional  Toolkit,  a web-based  application  for 
the  creation  and  management  of  course  web  sites.  Results  of  the  survey,  and  subsequent 
comments,  produced  interesting  insights  regarding  the  identity  of  the  University  of  Virginia’s 
Instructional  Toolkit,  the  mindset  of  University  of  Virginia  faculty,  and  inadvertent  “branding”  of 
web-based  applications  within  the  University  context. 


Background 

In  1996  the  University  of  Virginia  developed  a system  called  “The  Faculty  Instructional  Toolkit.”  The  Faculty 
Instructional  Toolkit  (Toolkit)  is  a Web-based  (http://toolkitvirginia.edu)  application  that  allows  instructors  at  the 
University  of  Virginia  to  easily  create  and  manage  class  home  pages  - selectively  use  electronic  features  such  as 
class  e-mail  lists,  class  rolls,  anonymous  course  feedback,  student  homework  submission,  and  final  grade 
submission. 

While  the  Instructional  Toolkit  system  does  enable  University  of  Virginia  faculty  to  quickly  and  easily  create  course 
web  sites,  faculty  have  an  inability  to  modify  the  appearance  of  course  web  sites.  Basic  Toolkit  navigation  is 
included  on  all  the  class  sites  the  system  creates.  The  Toolkit’s  name  and  icon  identify  the  system  to  students. 

The  interface  of  the  University  of  Virginia’s  Instructional  Toolkit  had  changed  little  since  1996.  Toolkit  support 
staff,  through  interaction  with  faculty  and  students  over  various  issues  regarding  the  increase  in  user  options,  had 
come  to  the  conclusion  that  navigational  issues  could  be  addressed  with  an  interface  update.  In  the  spring  of  1999, 
Toolkit  management  expressed  interest  in  developing  a more  “modem-looking”  Instructional  Toolkit,  in  addition  to 
several  usability  enhancements. 

The  Study 

The  original  Toolkit  interface  contained  an  image  of  the  Rotunda,  the  flagship  building  of  the  University  of  Virginia. 
In  an  attempt  to  better  utilize  the  “toolkit”  metaphor,  an  interface  highlighting  a laborer’s  toolbox  and  a “folksy” 
color  scheme  was  developed.  In  addition,  the  toolkit  metaphor  was  extended  to  the  navigational  choices.  The  use  of 
different  tools,  plans  used  in  building,  supporting  documentation  all  easily  map  to  functions  of  the  Toolkit  system. 

A prototype  showcasing  the  interface  changes  was  developed.  Several  ITC  staff  were  polled  informally  and  their 
response  was  favorable.  In  July  of  1999,  a portion  of  University  of  Virginia  faculty  with  previous  Toolkit 
experience  were  polled  concerning  their  reaction  to  the  new  interface  design. 

Later,  a second  interface  was  developed  which  maintained  the  same  basic  layout  as  the  first  redesign,  yet  replaced 
the  toolbox  and  related  icons  with  images  of  the  Rotunda.  The  Rotunda  images  spanned  the  life  of  the  building  - its 
conception,  construction  and  current  state.  The  original  survey  participants  were  then  again  polled. 
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Findings 


The  response  concerning  the  initial  redesign’s  laborer’s  toolbox  were  mixed.  Only  a little  more  than  the  majority  of 
the  survey  participants  agreed  with  the  statement,  “The  new  colors  and  icons  reinforce  the  Toolkit’  metaphor.”  In 
response  to  the  question,  “Which  best  describes  your  feeling  regarding  the  new  Toolkit  colors  and  icons?”  just  as 
many  participants  selected  “Very  satisfied,”  as  selected  “Not  quite  satisfied.” 

The  census  of  open-ended  responses  was  that  the  interface,  specifically  the  colors  and  icons,  “doesn’t  seem  as 
professional.”  Distress  concerning  the  replacement  of  the  image  of  the  Rotunda,  “that  old  favorite,”  as  it  was 
referred  to  by  one  individual,  was  expressed.  Even  though  “University  of  Virginia,”  was  clearly  stated  it  was 
evident  that  some  users  felt  that  the  laborer’s  toolbox  did  not  sufficiently  tie  itself  to  the  University.  The  icons  made 
the  “toolkit  [part]  bigger  than  the  UVA  part,”  according  to  one  survey  participant.  One  survey  participant  was  very 
clear  that  the  system  now  “looks  ‘canned’”  and  asked  it  if  the  system  could  be  changed  so  that  course  web  sites  do 
not  appear  as  if  they’ve  been  made  with  “cookie  cutters.” 

Individuals  contributing  to  the  second  survey  were  much  more  comfortable  with  the  second  design.  The 
overwhelming  majority  expressed  satisfaction  with  the  colors  and  images  of  the  Rotunda.  The  design  “looks 
classier  and  more  like  UVa,”  according  to  one  individual.  The  images  representing  the  life  of  the  Rotunda 
reinforced  the  toolkit  metaphor,  according  to  some  individuals,  while  others  concluded  that  “toolkit”  is  a poor 
metaphor  for  the  application  and  would  prefer  the  ability  to  drop  the  label. 

Conclusions 

Pinpointing  the  source  of  faculty  dissatisfaction  with  the  toolkit  interface  designs  is  difficult.  One  conclusion  is  that 
branding  inadvertently  occurred  when  the  toolkit  metaphor  was  extended  into  the  application’s  interface  and 
navigation.  According  to  WebMonkey,  a web  development  resource,  the  goal  of  a brand  is  “to  encourage  consumers 
to  feel  a certain  way  about  [a]  product.”  A brand  encourages  association,  but  has  little  to  do  with  usability. 

Prior  to  the  redesign,  the  Toolkit  system  utilized  an  image  of  the  Rotunda,  an  icon  that  gave  context  to  the 
application.  An  attempt  to  give  the  Instructional  Toolkit  its  own  identity  caused  faculty  members  to  focus  on  their 
association  with  the  system.  Paula  Petrik,  a professor  in  the  Department  of  History  at  the  University  of  Maine, 
expressed  discontentment  with  “‘courseware  in  a box’  products,”  finding  them  “unacceptable  in  terms  of  aesthetic 
design.”  Teaching  is  a personal  activity  and  she  wants  her  class  sites  to  “reflect  my  sense  of  myself  as  teacher  and 
scholar.”  Uva  faculty  members  using  the  Toolkit  have  are  unable  to  disassociate  themselves  with  the  Toolkit’s  icon, 
as  it  appears  on  every  course  web  sites  the  system  creates.  An  association  with  an  application  that  fails  to  emphasize 
the  context  of  its  use,  the  University  of  Virginia,  or  the  personality  of  the  course  web  site’s  owner  was  one  that  most 
survey  participants  were  not  eager  to  make. 

Maintaining  the  University  of  Virginia  iconography  allows  users  of  the  Instructional  Toolkit  to  associate  their  work 
with  the  institution.  However,  the  ability  to  personalize  course  sites,  by  choosing  a color  palette  or  incorporating 
their  own  icons  into  their  course,  would  allow  faculty  to  leverage  their  own  identity  with  the  identity  of  the 
application.  Faculty  would  then  be  able  to  associate  with  the  Instructional  Toolkit  because  the  application  associates 
individuals  with  their  own  course  web  site. 

References 

Petrik,  Paula.  (1999,  Oct  14).  Re:  Online  courseware.  Retrieved  Oct.,  1999  from  the  listserv:  H-MMEDIA@h- 
net.msu.edu 

WebMonkey.  (no  date).  Glossary  - Branding.  [WWW  document].  Available: 

http://hotwired.lvcos.com/webmonkev/glossary/branding.html  [ 1 999,  Oct  1 2] 


1455 


The  distance  educator’s  tool  box:  An  overview  of  technologies  for  on-line 

learning 


Thomas  Fox  McManus,  Ph.D. 
Saginaw  Valley  State  University 
7400  Bay  Rd. 

University  Center,  Ml  48710 
Mcmanus@svsu.  edu 


Abstract:  Every  day  new  tools  become  available  to  creators  of  distance  education.  Keeping  up 
with  them  is  almost  impossible.  This  article  gives  a very  short  overview  of  several  of  the  new 
technologies  that  have  become  available  to  distance  educators  in  recent  years,  and  touches  on  their 
instructional  implications.  Tools  mentioned  include  DHTML,  XML,  Flash,  Course  wrap, 

NetMeeting,  ColdFusion,  ASP,  and  Generator. 

Introduction 

Increasingly  educational  institutions  and  organizations  are  looking  to  the  Internet  as  a primary  means  of  delivering 
instruction  to  learners.  Up  until  a few  years  ago,  instructional  designers  and  course  creators  have  been  limited  to 
HTML  and  CGI  or  PERL  scripts  to  create  their  online  instmctional  environments.  Then  content  creation  and 
delivery  tools  like  MacroMedia  Director  Shockwave  and  RealVideo  came  on  the  scene,  and  allowed  designers  to 
improve  their  instructional  environments  by  adding  a bit  of  rich  media  and  interactivity.  But  the  bandwidth  demands 
of  these  tools  still  made  it  difficult  to  have  environments  which  went  far  beyond  the  clickable  book  stage. 

Today,  distance  educators  need  to  harness  the  power  of  a new  generation  of  tools  to  meet  the  growing  demand  of  the 
learners  who  have  become  accustomed  to  business  and  entertainment  sites  that  are  interactive,  personalized,  and 
easy  to  navigate.  Some  of  these  tools  have  been  in  use  in  e-commerce  and  business  for  a few  years,  while  others 
have  been  developed  solely  for  the  distance  education  field.  These  new  tools  affect  all  aspects  of  distance  education, 
from  course  creation  and  communication  to  interaction  and  record  keeping.  Each  tool,  or  class  of  tools,  has 
instructional  implications  of  which  designers  need  to  be  aware. 


Page  Creation  Tools 

Cascading  Style  Sheets  (CSS)  are  design  templates  that  are  used  to  provide  augmented  control  over  presentation 
and  layout  of  HTML  elements,  thus  separating  the  way  you  design  information  from  the  actual  HTML  content. 
DHTML  is  any  combination  of  Style  Sheets,  JavaScript,  Layering,  Positioning,  and  Page  Division  used  to  create 
movement  on-screen  or  user  interactivity.  This  means  that  you  are  able  to  create  complex  interactive  animations 
that  do  not  require  that  learners  have  a particular  plug-in  installed.  The  disadvantages  include  that  DHTML  does 
require  the  user  to  have  one  of  the  newer  versions  of  their  browser,  4.0  or  later,  limiting  the  usefulness  in  some 
educational  environments  with  legacy  systems. 

XML  lets  you  create  your  own  tags  that  describe  the  content  of  your  document  rather  than  its  presentation.  For 
example,  whereas  the  <H1>  tag  in  HTML  specifies  text  to  be  presented  in  a certain  typeface  and  weight,  an  XML 
tag  would  explicitly  identify  the  kind  of  information:  <TESTITEM>  tags  might  identify  evaluation  questions, 
<CORRECTANSWER>  tags  could  describe  feedback  for  correct  answers.  One  example  of  XML  use  with 
instructional  implications  is  the  Mathematical  Markup  Language  (MathML)  which  is  used  to  display  mathematical 
and  scientific  notation  effectively. 
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MacroMedia  Flash  is  used  to  create  highly  interactive  animations  with  file  sizes  equivalent  to  the  current  image 
maps  used  on  many  sites.  This  tool  allows  the  designer  to  finally  break  out  of  the  clickable  book  design  metaphor 
and  present  instruction  in  the  most  contextually  appropriate  fashion.  The  disadvantage  is  that  it  requires  the  learner 
to  install  the  appropriate  plug-in  before  they  can  access  the  instruction.  Flash  can  also  be  used  to  create  small  accent 
animations,  like  rollover  buttons  with  audio  feedback,  for  use  in  more  traditional  web  pages. 

Communication  & Collaboration 

Coursewrap  has  become  the  generic  term  for  software  packages  that  help  the  distance  educator  put  their  instruction 
online,  foster  communication  and  collaboration,  and  to  track  student  progress  through  the  environment.  Examples  of 
this  include  WebCT,  Blackboard.com,  and  PathWare.  Each  package  has  different  strengths  and  weaknesses,  far  too 
many  to  go  into  in  a short  article.  But  anyone  heavily  involved  in  delivering  instruction  online  is  likely  to  use  one. 

Network  Conferencing  software,  like  Microsoft  NetMeeting,  is  another  communication  and  collaboration  tool  that 
has  great  instructional  potential.  NetMeeting  allows  for  multiple  learners  to  work  together  in  a shared  virtual  space. 
Tools  include  chat,  whiteboard,  file  sharing,  and  voice/video  communication.  This  allows  for  everything  you  can  do 
in  a classroom  except  touch.  The  disadvantage  is  that  bandwidth  constraints  can  negatively  impact  sound  and  picture 
quality  and  can  cause  lag  in  the  shared  resources 


Dynamic  Web  Pages  and  Databases 

To  get  truly  interactive  web  pages,  ones  that  will  get  information  from  your  learner  and  return  an  appropriate  page, 
custom  created  from  information  in  your  database,  you  need  to  use  server  side  software.  There  are  a few  of  which 
distance  educators  should  be  aware.  Allaire  ColdFusion  is  a database-to-Web  software  that  has  become  a very 
popular  tool  for  businesses  that  need  to  get  lots  of  data  onto  the  Web  quickly.  Active  Server  Pages  (ASP)  are 
server-generated  pages  which  can  call  other  programs  to  do  things  like  access  databases,  serve  different  pages  to 
different  browsers,  almost  anything  that  can  be  coded.  MacroMedia  Generator  accesses  Internet  optimized 
databases  like  OracleSIi,  which  can  organize  any  type  of  media  file,  to  dynamically  generate  Flash  animations  and 
seamlessly  return  them  to  the  learner's  browser.  These  are  complex  and  expensive  tools,  but  by  allowing  an 
instructor  to  tie  into  the  power  and  organization  of  large  databases,  they  have  tremendous  potential  in  distance 
education.  Some  examples  include  randomly  generated  tests,  with  questions  taken  from  a database  of  items,  or 
instruction  that  is  dynamically  generated  to  meet  an  individual  learner's  needs.  These  needs  might  be  detennined  by 
a customer  preference  tracking  software  like  MacroMedia  LikeMinds. 


Conclusion 

The  potential  of  tools  like  these  to  enhance  distance  education  learning  environment  is  great.  It  is  essential  that 
educators  do  not  fall  behind  in  the  effective  use  of  new  technologies.  Not  every  new  toy  that  comes  along  should  be 
adopted.  But  if  a tool  can  make  a learner's  experience  better  or  an  educator’s  job  easier,  it  should  be  used,  whether  or 
not  it  was  originally  aimed  for  use  in  education. 


For  More  Information 

CNET  Web  Building  - http://www.builder.com 
WebDeveIoper.com  - http://www.webdeveloper.com/ 
WebMonkey  - http.7/hotwired. lvcos.com/webmonkey/index.html 
World  Wide  Web  Consortium  - http://www.w3.org 
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Abstract:  Effective  staff  development  is  the  weaving  together  of  many  strands.  We  need  to 
support  staff  in  their  current  work,  while  providing  them  with  ideas,  incentives  and  resources 
to  look  for  new  ways  to  design  learning  environments  which  will  enhance  student  learning. 
Staff  development  must  be  combined  with  specific  projects  where  change  is  occurring.  Ideas 
are  not  hard  to  find.  Incentives  and  resources  are  another  matter.  The  paper  will  outline 
some  general  principles  for  effective  staff  development.  These  principles  will  be  applied  in  the 
description  of  the  substantial  investment  RMIT  has  made  in  order  to  realise  our  Teaching 
and  Learning  Strategy.  We  have  a model  of ‘grass-roots’  faculty-based  work  funded  by  large- 
scale  corporate  ‘investment’.  ‘Bottom-up’  meets  ‘top-down’. 


Introduction 

Staff  development  can  no  longer  be  a pleasant  ‘cottage  industry'  on  the  fringes  of  academe  or  the  enthusiastic 
enterprise  of  a few  individuals  supported  by  ‘soft’  money.  Effective  staff  development  is  positioned  at  the  centre  of 
university  functioning  and  yet  needs  to  retain  connections  with  the  needs  and  perceptions  of  teaching  staff  This  is  a 
demanding  challenge.  Staff  development  programs  that  are  successful  in  meeting  the  needs  of  complex  modem 
Australian  universities  need  to  be  supported  strategically  (and  financially)  by  their  own  universities. 

McNaught  et  al.  (1999)  lists  six  key  issues  in  staff  development: 

• The  appropriate  balance  point  between  centrally  provided  and  local  staff  development  services  needs  to  be 
determined  in  each  university.  Central  services  can  be  more  clearly  linked  to  university  priorities;  faculty  or 
department  services  can  be  more  in  touch  with  local  needs. 

• As  technology  becomes  more  mainstream,  support  services  need  to  be  scaled  up.  This  involves  deciding  on 
the  level  of  support  that  can  be  afforded  and  the  model  of  support  which  is  most  apposite.  The  educational 
design  and  evaluation,  technical,  and  media  production  support  services  that  universities  currently  have  are 
under  strain.  It  is  unlikely  that  the  existing  examples  of  good  practice  at  each  university  will  be  sufficient  to 
ensure  that  new  or  revised  subjects  will  be  well  designed  and  evaluated.  By  modelling  good  practice 
themselves,  mentors  can  assist  staff  make  optimal  use  of  resources. 

• A follow-on  issue  is  determining  the  optimal  relationship  between  staff  development  and  production 
support  services.  Again,  this  needs  to  be  decided  in  each  university  context. 

• Even  if  an  integrated  model  of  professional  development  is  adopted,  there  are  still  many  professional 
development  providers  at  most  universities.  Mapping  the  services  of  each  provider  and  ensuring  reasonable 
coordination  is  increasingly  important  as  the  need  for  support  services  scales  up. 

• Academic  and  general  staff  work  load  is  a key  issue.  Careful  work  planning  to  ensure  that  staff  have  time 
to  learn  new  skills  and  manage  new  processes  is  essential. 


Learning  Technology  Mentors  at  RMIT  University 

RMIT  University  is  an  ‘old’  (in  Australian  terms;  RMIT  began  in  1887)  technological  university.  It  is  highly  diverse 
- it  is  a cross-sectoral  (includes  vocational  sector)  university  and  has  the  largest  number  of  international  students  of 
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any  Australian  university.  There  are  seven  strong  faculties  which  often  resist  central  directions  (what’s  new?).  The 
Information  Technology  Alignment  Project  (ITAP)  report  (1998)  forms  the  basis  for  a SA50  million  investment  by 
RMIT  over  the  four  years,  1999-2002.  The  report  comprises  several  elements: 

• IT  infrastructure  aligned  with  the  needs  of  education  to  deliver  the  systems  and  hardware  necessary  to 
provide  students  with  an  electronically  connected  learning  environment  and  access  to  computer-based 
learning  resources; 

• a Distributed  Learning  System  (DLS)  compliant  with  the  emerging  Educom/CAUSE  Instructional 
Management  System  (IMS); 

• an  Academic  Management  System  (AMS),  fully  integrated  with  the  DLS  to  provide  enrolment  and  subject 
and  course  progress  records  electronically  accessible  to  academics  and  students; 

• an  extensive  review  of  all  academic  processes  within  the  university  in  a Business  Process  Re-engineering 
(BPR)  project;  and 

• extensive  staff  development. 

We  have  to  deliver  on  our  promise  that  we  can  provide  a flexible  set  of  tools  that  will  enable  staff  who  are  not 
technological  whizz  kids  to  develop  pedagogically  sound,  interesting,  and  relevant  online  courses  in  an  efficient  and 
well  administered  way.  Quite  an  ask.  How  have  we  designed  our  Distributed  Learning  System  (DLS)?  Here  are 
some  of  our  principles: 

• a suite  of  tools,  not  just  one; 

• integrating  educational  principles  into  the  description  of  the  toolset; 

• IMS  compliance  of  all  tools; 

• a team  approach  to  all  online  projects;  and 

• involvement  of  all  seven  Faculties  in  a benchmarking  exercise  to  evaluate  the  toolset  and  the  effectiveness 
of  the  learning  environments  we  are  building. 

Learning  Technology  Mentors  have  been  appointed  in  each  department  of  the  University.  There  are  currently 
approximately  150  Learning  Technology  Mentors  (LTMs)  - two  in  most  departments  of  the  university  and  some  in 
central  areas  such  as  the  Library.  These  are  mostly  academic  and  teaching  staff  who  have  funded  one  day  a week 
time  release  to  develop  online  materials  and  support  their  colleagues  in  their  departments  to  engage  with  online 
teaching  and  learning.  Each  LTM  is  funded  for  26  days  time  release,  and  some  for  longer  periods. 

These  LTMs  undertake  an  extensive  staff  development  program  about  a week  long  which  covers  training  in  the  use 
of  the  DLS  toolset,  key  principles  in  online  educational  design,  and  organisational  learning  within  the  context  of 
RMIT.  Additional  staff  development  sessions  are  run  each  week.  Topics  include  hands-on  training  sessions  on  all 
the  DLS  tools,  quality  improvement  and  evaluation  of  DLS  tools,  student  induction  methods,  assessment  strategies 
for  online  learning,  evaluation  strategies  for  online  learning,  the  role  of  the  Library  in  supporting  online  learning, 
managing  digital  resources  using  metadata,  project  management,  graduate  attributes  and  other  curriculum  matters, 
and  planning  for  online  development  using  the  DLS  support  documents. 

All  LTMs  develop  a work  contract  with  the  first  author;  if  individual  staff  wish  this  can  be  formalised  into 
accreditation  for  a subject  in  a Graduate  Certificate  of  Flexible  Learning.  While  one  day  a week  is  not  a great  deal  of 
time,  it  has  been  enough  to  give  many  staff  a space  in  which  to  learn  new  skills  and  enact  them.  We  are  seeing  that 
staff  development  and  support  for  developing  online  learning  materials  and  strategies  must  become  distributed 
across  the  organisation.  Therefore  the  role  of  the  faculty-based  Faculty  Education  Services  Groups  (FESGs)  is 
pivotal.  Growth  needs  to  occur  in  these  units  rather  than  at  the  centre.  We  have  been  delighted  by  the  enthusiasm  of 
many  Learning  Technology  Mentors.  Have  we  reached  critical  mass  yet,  where  the  appropriate  use  of  technology 
will  roll  out  across  the  University?  Probably  not,  but  we  feel  we  are  on  the  right  track. 
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Abstract:  Database  tools  that  dynamically  generate  Web  pages  have  been  in  use  by  many 
commercial  Internet  sites  for  years.  However,  in  the  filed  of  education,  the  use  of  database 
applications  has  mostly  been  used  to  support  traditional  classroom  learning  activities.  The  use 
of  dynamic  Web-based  databases  as  research  and  communication  tools  in  higher  education  is 
currently  gaining  popularity  as  development  of  online  courses  continues  to  grow.  This  paper 
describes  a theoretical  framework  for  using  Web-based  databases  to  support  constructivist 
learning  and  provides  an  introduction  to  how  these  tools  are  being  used  to  facilitate  the 
construction  of  knowledge  and  support  student  and  faculty  communication  in  an  online 
environment. 


Faculty  and  Student  Concerns  about  Online  Courses 

With  the  rapid  growth  and  acceptance  of  the  Web  as  an  educational  medium,  universities  are  feeling  pressure  to 
produce  online  training  and  instructional  materials.  Faculty  members  at  colleges  and  universities  are  being  told 
almost  daily  that  they  need  to  change  the  way  they  teach,  as  the  value  of  traditional  lecture-based  instruction  is 
being  reevaluated  and  the  promise  of  online  course  delivery  is  being  glorified.  Although  faculty  members  at 
many  institutions  of  higher  learning  have  already  begun  to  create  and  deliver  online  courses,  there  are  few 
standard  procedures  for  doing  so.  At  some  schools,  early  adopters  are  leading  the  way,  but  the  acceptance  of 
emerging  technologies  by  large  numbers  of  faculty  seems  far  away.  One  of  many  reasons  may  be  that  faculty 
who  create  online  courses  are  uncertain  about  how  best  to  proceed.  For  many,  the  development  of  Web-based 
materials  is  time  consuming  and  requires  learning  new  skills,  and  support  systems  are  not  yet  in  place  that  can 
accommodate  large  numbers  of  new  technology-using  educators.  In  addition  to  the  need  for  technical  support, 
faculty  are  also  concerned  about  pedagogical  issues  that  arise  when  distance  education  modalities  are  discussed. 
Designing  a Web-based  course  requires  a different  set  of  instructional  strategies  and  philosophical  orientations, 
and  while  some  faculty  members  see  the  Web  as  an  opportunity  for  implementing  instructional  innovations 
(Khan,  1997;  Owston,  1997),  many  are  largely  uninformed  about  the  advantages  and  disadvantages  of  online 
teaching.  In  addition,  many  are  unclear  about  the  effectiveness  of  online  courses  compared  with  traditional, 
face-to-face  courses  (Reeves,  1997).  Interactions  between  students  and  faculty,  a critical  component  in  any 
course,  change  dramatically  in  an  online  environment  where  such  tools  as  e-mail,  listservs,  and  chat  replace  the 
more  familiar  student  and  faculty  discourse.  Ongoing  discourse  among  participants  is  the  driving  force  behind 
successful  online  courses,  but  unfortunately,  many  online  courses  fail  to  take  advantage  of  these  interactions 
that  can  add  richness  and  depth  to  the  construction  of  knowledge. 


Using  Dynamic  Databases  as  Teaching  and  Learning  Tools 

Many,  if  not  most  Web  pages  simply  duplicate  print-based  materials  in  electronic  form.  In  education, 
converting  existing  course  information  to  a Web-based  format  is  not  enough  to  effectively  produce  meaningful 
online  courses.  Attention  must  be  given  to  the  needs  of  the  students,  including  facilitating  social  and 
instructional  interactions,  creating  a sense  of  a learning  community,  and  delivering  the  most  timely  and  useful 
information.  The  use  of  dynamic  databases  allows  users  to  dynamically  generate  Web  pages  that  present  the 
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exact  information  that  is  being  sought.  This  is  important  because  it  increases  interaction  between  students  and 
the  course  content  and  empowers  students  with  control  of  the  course  environment.  Additionally,  the  body  of 
knowledge  in  the  course  does  not  remain  static  but  continues  to  change  and  grow  without  requiring  extra  work 
on  the  part  of  the  instructor. 


Examples  of  Dynamic  Databases 

Students  in  our  online  classes  use  a variety  of  communication  technologies,  including  a Web-based  discussion 
list.  To  enhance  the  feeling  of  an  electronic  learning  community,  students’  pictures  are  taken  at  orientation  and 
uploaded  to  the  database.  Then,  when  a student  posts  a message  to  the  listserv  through  a Web  form,  the  message 
is  displayed  dynamically  with  their  picture  displayed.  Students  may  also  submit  a variety  of  assignments 
through  a Web  form.  The  forms  may  contain  text  fields,  pull-down  menus,  and  radio  buttons  to  submit  different 
types  of  information.  In  addition,  attachments  may  be  uploaded  and  are  entered  in  the  database.  Students  may 
then  use  the  query  form  to  search  the  database  by  content  area,  audience,  or  author.  After  students  submit  their 
assignments,  the  instructor  may  use  a faculty  review  form  to  provide  feedback  for  the  assignments  that  have 
been  posted.  Since  all  of  the  student  postings  are  available  in  the  database,  this  allows  the  instructor  to  review 
all  of  a particular  student’s  assignments  or  all  of  the  postings  on  a specific  assignment,  utilizing  the  common 
sorting  functions  of  a database  application.  If  desired,  the  database  program  may  also  be  configured  to 
automatically  send  the  feedback  to  a student’s  e-mail  address,  a feature  that  helps  students  feel  a greater 
connection  to  the  course,  the  content,  and  the  instructor. 


Conclusion 

Our  use  of  dynamic  database  tools  has  helped  us  improve  our  online  courses  as  they  give  students  greater 
control  over  the  information  covered  in  the  courses.  However,  some  limitations  to  the  use  of  this  technology  do 
exist.  First,  instructors  who  would  like  to  incorporate  dynamic  database  tools  into  their  courses  face  a steeper 
learning  curve  than  is  required  to  create  traditional  web  pages.  A greater  understanding  of  information  design 
and  information  management  is  required  and  those  educators  who  choose  to  learn  these  skills  must  commit 
more  time  and  effort  to  this  endeavor  than  originally  anticipated.  Additionally,  students  may  require  some 
instruction  in  the  use  of  these  database  tools  before  they  become  completely  comfortable  submitting 
assignments  and  checking  the  progress  of  their  coursework. 
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Abstract:  The  iterative  process  of  an  instructional  designer  working  with  faculty  members  to 
develop  a web-based  module  for  an  interdisciplinary  freshman  studies  course  is  described  in  this 
paper.  During  the  process,  as  it  was  necessary  to  change  pedagogy  to  benefit  from  the  use  of  the 
technology,  teachers  and  students  redefined  their  roles.  The  process  has  occurred  over  3 years.  The 
majority  of  planning,  design  and  development  occurred  in  the  summer  when  faculty  could 
collaborate  with  instructional  technology  staff  and  students  who  helped  create  the  web  pages. 


What  was  Done 

Since  1996,  St.  Edward's  University  has  offered  faculty  members  the  opportunity  to  develop  web-based, 
multimedia  instructional  materials  for  their  classes  during  a 60  hour,  3-week  Summer  Technology  Institute,  Faculty 
members  submit  proposals  and  receive  stipends  to  develop  projects  that  range  from  the  comprehensive  redesign  of 
courses  to  stand-alone  tutorials  to  supplement  instruction.  The  Instructional  Technology  department  collaborates 
with  the  Center  for  Teaching  Excellence  to  offer  these  intensive,  individualized,  hands-on  classes.  Faculty  consult 
with  the  Instructional  Computing  Coordinator  who  acts  as  instructional  designer  and  project  manager. 

In  this  particular  case,  a faculty  member  in  the  School  of  Humanities,  who  did  not  use  his  computer  for 
much  more  than  email,  was  recruited  to  attend  the  summer  institute.  The  proposal  addressed  the  need  to  develop 
instructional  strategies  to  improve  student  mastery  in  a freshman  course.  The  course  was  an  interdisciplinary  one 
(Humanities  and  Rhetoric  and  Composition)  that  had  been  revised  many  times  to  create  consistency  among  the 
many  writing  teachers  affiliated  with  the  course.  Traditionally,  lectures  were  delivered  twice  a week  and  the  other 
times  students  attended  Rhetoric  and  Composition  classes.  The  module  this  professor  taught  was  a 3 hour  lecture  for 
a course  that  was  a general  curriculum  requirement  for  freshmen.  He  would  lecture  in  an  auditorium  of  about  150 
students  twice  a week. 

During  the  first  summer  as  course  material  was  moved  to  the  web  for  delivery  of  instruction,  the  professor 
learned  the  rudiments  of  designing  web  pages,  including  how  to  use  an  HTML  editor,  use  graphics  and  how  to 
manage  online,  asynchronous  group  discussions.  With  the  support  of  student  interns,  a great  deal  of  information 
about  Japan  was  put  on  a web  page  that  was  created  to  support  the  class.  Using  streaming  video  and  audio,  freshmen 
students  could  hear  Japanese  spoken,  they  could  see  a slide  show  of  images  and  listen  to  Japanese  music  with  a 
narration  introducing  the  culture. 


The  Process 

Amidst  the  learning  and  practicing  of  the  technical  skills  involved  with  web  page  production  the  first  year, 
the  faculty  member  and  I consulted  to  focus  the  course  content  by  specifying  the  learning  objectives  for  students. 
We  talked  about  changing  the  manner  of  assessment  that  had  been  in  place  for  so  many  years,  but  we  didn't  get 
around  to  that.  We  consulted  with  the  many  faculty  involved  in  teaching  sections  of  the  course  to  get  their  input.  At 
the  end  of  that  first  summer  we  had  a dazzling  web  page  that  had  lovely  graphics,  lots  of  links  to  supplemental 
information,  audio  and  slide  shows.  That  fall  we  implemented  the  site  and  requested  student  and  faculty  feedback  in 
the  form  of  surveys.  Writing  teachers  and  students  used  the  web  page  as  a resource  to  master  the  course  content.  The 
faculty  member  used  his  lecture  time  to  introduce  additional  course  information. 

When  all  was  said  and  done,  the  feedback  was  far  from  complimentary.  Students  and  faculty  alike 
complained  that  there  was  too  much  information  and  not  enough  clarity  of  learning  outcomes.  No  one  could  figure 
out  "what  was  going  to  be  on  the  test".  We  had  succeeded  in  demonstrating  the  lesson  that  we  had  added 
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technology,  but  not  ’’added  value”!  We  had  not  changed  pedagogy.  We  now  had  a glorified  online  lecture  which  had 
the  Internet  as  it's  boundary.  Links  led  to  links  which  led  students  to  still  other  links  around  the  globe. 

The  second  year,  in  an  effort  to  provide  a structure  or  framework  for  learning  the  information,  we  designed 
a "WebQuest'’.  This  particular  format,  which  is  essentially  the  process  of  a systematic  approach  to  instructional 
design,  was  developed  at  San  Diego  State  University  in  early  1995  by  Bemie  Dodge  with  Tom  March.  The  idea  is 
that  students  take  an  active  approach  to  their  learning.  Learning  activities  give  students  real  world  scenarios  with 
which  to  work  and  experience  collaboration. 

The  development  of  the  WebQuest  included  faculty  other  than  just  the  person  who  had  delivered  the 
lecture.  The  reading  specialists  contributed  as  well  as  other  writing  faculty.  That  fall  the  web-based  module  was 
implemented  with  much  more  success.  Students  and  faculty  felt  that  a framework  had  been  provided.  Surveys  using 
the  Flashlight  Network  toolkit  were  used  to  evaluate  the  use  of  the  web  this  time.  Additionally,  the  professor 
changed  his  assessment  of  student  learning  to  reflect  the  change  in  pedagogy.  Still  a multiple  choice  exam,  it  now 
measured  higher  order  thinking  skills  rather  than  simply  literal  or  factual  knowledge.  Students  need  to  reason  and 
draw  conclusions  using  the  facts  to  answer  test  questions. 

The  plans  are  to  still  revise  the  web  page  yet  again  for  a third  time.  We  want  to  provide  an  interface  that 
uses  real-world  metaphors  and  intuitive  navigation.  For  example,  the  opening  splash  screen  may  be  a student's  room, 
desk,  computer  and  maps  on  the  wall.  Students  will  interact  by  choosing  objects  to  give  them  specific  information. 
For  example,  by  selecting  the  map  of  Japan,  the  student  will  get  information  on  the  geography  of  Japan. 

The  scenario  will  be  changed  again  to  reflect  an  activity  with  an  even  more  meaningful  context  for  the 
student.  Students  will  seek  information  to  answer  questions  related  to  living  abroad  for  a semester.  The  framework 
for  learning  will  focus  on  what  the  student  needs  to  know  to  live  in  Japan  for  a semester  such  as  what  to  expect 
socially  and  culturally.  The  authentic  assessment  will  include  a travel  brochure  on  Japan  compiled  by  teams  of 
students. 


Why  it  is  Important 

The  process  of  creating  instructional  web  pages  is  an  iterative  and  collaborative  process.  The  collaboration 
is  among  faculty  and  instructional  designers  and  students.  It  stands  to  reason  that  many  faculty  are  intimidated  by 
learning  new  technology  skills  and  then  having  to  change  their  instructional  strategies.  Suggestions  made  by 
professional  instructional  technologists  can  not  be  forced  on  faculty.  Faculty  need  multiple  experiences  and 
guidance  to  understand  that  it’s  not  about  the  technology,  it’s  about  how  you  apply  the  technology  to  teaching.  It 
takes  surveying  the  students  who  use  the  web  page  to  understand  their  perceptions  of  how  the  technology  is  being 
used  to  instruct.  It  takes  analysis  of  the  students’  achievements  and  learning  outcomes  to  see  if  objectives  are  being 
reached. 

Not  only  should  revisions  be  made  in  the  instructional  design  using  the  formative  feedback  of  the  primary 
users  of  web  pages,  but  designers  should  be  constantly  aware  of  ways  to  provide  real-  world  applications  for 
students  through  the  use  of  technology.  The  web  is  changing  the  nature  of  learning,  but  we  need  to  quickly  move 
beyond  "the  syllabus  and  Powerpoint  lecture  online”  use  that  is  so  prevalent.  In  a 3 year  period  we  have  been  able  to 
move  toward  a new  paradigm  of  learning  in  a course  that  was  traditionally  lecture-bound.  As  this  process  occurred, 
both  the  roles  of  the  students  in  the  course  and  faculty  member  changed. 

The  lecturer  realized  how  profoundly  the  students’  roles  changed  by  remarking  how  the  students  were 
active  and  engaged  in  their  learning  and  took  on  responsibility.  He  no  longer  felt  his  role  was  to  be  the  "sage  on  the 
stage”.  Although  he  did  still  use  his  lecture  time  to  tell- stories  and  provide  bridges  to  the  content  that  was  on  the  web 
page,  he  enjoyed  his  new  role  as  facilitator  of  knowledge  and  found  that  he  could  lead  students  to  higher 
achievement  with  his  new  manner  of  teaching. 

This  process  of  collaboration  and  solicitation  of  student  feedback  ultimately  seems  to  have  evolved  into  a 
successful  experience.  It  takes  time  for  evolution.  Change  is  slow.  It  takes  team  work.  Hopefully  this  paper  can 
encourage  faculty,  who  may  feel  unsuccessful  in  their  first  attempts  to  add  value  and  technology  to  their  course,  to 
"try  again". 
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Abstract:  Most  web  page  authors  are  completely  unaware  of  accessibility  issues  and  have  little  or 
no  experience  in  making  their  web  pages  usable  by  persons  who  coannot  see  the  screen  or  use  a 
mouse  the  same  way  they  do.  The  World  Wide  Web  Consortium  (W3C)  has  recognized  a disparity 
in  web  accessibility  and  has  developed  a set  of  web  page  design  standards  to  address  the  issue  of 
web  accessibility. 

Introduction 

Overall  use  and  availability  of  information  resources  through  the  Internet  have  continued  to  skyrocket; 
however,  cyberspace  predominantly  has  been  the  domain  of  persons  without  physical  or  mental  disabilities.  While 
sighted  people  hold  the  keys  to  the  virtual  kingdom,  for  all  practical  purposes  the  kingdom  gates  have  been  closed 
and  locked  for  persons  who  are  visually  impaired  or  have  other  disabilities.  According  to  Tim  Berners-Lee,  World 
Wide  Web  Consortium  Director  and  inventor  of  the  World  Wide  Web,  “The  power  of  the  Web  is  in  its  universality. 
Access  by  everyone  regardless  of  disability  is  an  essential  aspect  ” Most  web  authors  develop  web  pages  to  convey 
content  visually  and  do  not  allow  for  the  eventuality  that  some  web  surfers  may  not  see,  hear,  move,  or  process  some 
types  of  information,  may  have  difficulty  reading  or  comprehending  text,  or  may  not  have,  or  be  able  to  use,  a 
mouse  or  keyboard.  The  World  Wide  Web  Consortium  (W3C)  has  recognized  a disparity  in  accessibility  to  the  Web 
between  persons  with  and  without  disabilities  and  has  responded  by  developing  a set  of  web  design  standards  and 
strategies  that  specifically  address  the  issue  of  web  accessibility. 

Problem  Statement 

Web  accessibility  is  an  issue  that  must  be  confronted  by  site  developers  and  page  authors.  In  a recent 
opinion  from  the  U.S.  Department  of  Justice  (see  http://www.intemetlawyer.  com/ada.htm),  the  Americans  with 
Disabilities  Act  also  applies  to  cyberspace.  According  to  this  opinion,  entities  covered  by  the  ADA  that  use  the 
Internet  for  communications  regarding  programs,  goods,  or  services,  must  be  prepared  to  provide  these 
communications  through  accessible  means  as  well.  Additionally,  Section  508  of  the  U.S.  Rehabilitation  Act  requires 
federal  departments  and  agencies  to  evaluate  the  extent  to  which  their  electronic  and  information  technology  is 
accessible  to  and  usable  by  individuals  with  disabilities  (see  http://www.508.org). 

In  April,  1997,  the  W3C  created  the  Web  Accessibility  Initiative  (WAl)  as  an  official  activity  under  the 
Technology  and  Society  domain  of  the  W3C.  The  WAI  is  funded  separately  from  other  W3C  member  activities  and 
is  sponsored  by  representatives  of  web  development  industries,  disability  organizations,  research  organizations,  and 
government.  Some  of  the  WAI  sponsors  include  the  National  Science  Foundation  (NSF),  National  Institute  on 
Disability  and  Rehabilitation  Research  (NIDRR),  Microsoft,  IBM,  Lotus  Development,  and  NCR. 

To  facilitate  the  efforts  for  promoting  Web  accessibility,  the  WAI  joined  forces  with  the  W3C  HTML 
Working  Group  in  the  design  of  HTML  4.0  and  in  December,  1997,  HTML  4.0  became  a W3C  recommendation 
(see  http://www.w3.org/TR/REC-html40/).  Additionally,  in  May,  1998,  an  official  W3C  recommendation  for 
Cascading  Style  Sheets,  Level  2,  (CSS2)  was  issued  (see  http://www.w3.org/TR/REC-CSS2/).  In  May,  1999,  the 
WAI  issued  the  Web  Content  Accessibility  Guidelines  1.0  (see  http://www.w3.org/TR/WAl-WEBCONTENT/). 
These  guidelines  incorporated  the  recommendations  of  HTML  4.0  and  CSS2  and  were  intended  for  use  by  all  web 
content  developers  including  page  authors,  site  designers,  and  developers  of  authoring  tools. 

Developing  Accessible  Web  Pages 

Web  accessibility  is  based  on  design  principles  that  provide  for  the  development  of  web  pages  that 
accommodate  the  needs  of  a broad  range  of  users,  computers,  and  telecommunications  systems  without  regard  to 
age  or  disability.  When  a web  site  is  accessible,  anyone  browsing  the  site  should  be  able  to  gain  a complete 
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understanding  of  the  information  presented  on  the  site  as  well  as  have  an  undiminished  ability  to  interact  with  the 
site.  Since  many  educators  are  commonly  authoring  on-line  courses  or  publishing  research  findings  in  web  pages, 
they  need  to  implement  and  apply  the  design  strategies  of  the  Web  Content  Accessibility  Guidelines  to  the 
development  of  web  pages  that  facilitate  accessibility  for  web  surfers  with  disabilities.  The  Web  Content 
Accessibility  Guidelines  (WCAG)  consist  of  14  guidelines  that  are  formulated  around  two  general  web  page  design 
strategies:  (1)  ensuring  graceful  transformation,  and  (2)  making  content  understandable  and  navigable. 

Ensuring  Graceful  Transformation 

Web  surfers  may  operate  in  contexts  very  different  from  the  one  in  which  a web  page  is  developed.  The 
WCAG  support  and  promote  the  development  of  pages  that  transform  gracefully.  A page  transforms  gracefully 
when  it  remains  accessible  despite  any  constraints  that  may  include  (though  not  be  limited  to)  physical,  sensory,  and 
cognitive  disabilities,  work  constraints,  and  technological  barriers.  For  example,  a web  surfer  may  not  be  able  to  see, 
hear,  move,  or  use  a keyboard  or  mouse,  or  may  have  difficulty  reading  or  comprehending  text.  The  surfer  may  have 
a small  screen,  a slow  Internet  connection,  an  early  version  of  a browser,  a different  browser,  a voice  browser,  or  a 
different  operating  system.  Web  pages  transform  gracefully  when  text  or  text  equivalents  are  provided  because  text 
can  be  rendered  in  ways  that  are  available  to  almost  all  browsing  devices  and  accessible  to  almost  all  users.  Web 
pages  should  not  rely  on  one  type  of  hardware  and  should  be  usable  by  people  without  a mouse,  with  small  screens, 
low  resolution  screens,  black  and  white  screens,  no  screens,  or  with  only  voice  or  text  output.  The  theme  of  graceful 
transformation  is  addressed  primarily  by  Guidelines  1 to  1 1 . 

Making  Content  Understandable  and  Navigable 

Web  page  authors  should  make  page  content  understandable  and  navigable.  The  language  of  a web  page 
should  be  clear  and  simple,  but  also  provide  understandable  mechanisms  for  navigating  within  and  between  pages. 
Providing  navigation  tools  and  orientation  information  in  web  pages  maximizes  accessibility  and  usability.  Not  all 
surfers  can  make  use  of  visual  clues  such  as  image  maps,  proportional  scroll  bars,  side-by-side  frames,  or  graphics 
that  guide  sighted  users  with  graphical  desktop  browsers.  Web  surfers  may  also  lose  contextual  information  when 
they  can  only  view  a portion  of  a page,  either  because  they  are  accessing  the  page  one  word  at  a time  as  with  a 
speech  synthesizer  or  a Braille  display,  or  one  section  at  a time  as  with  a small  or  magnified  display.  Without 
orientation  information,  users  may  not  be  able  to  understand  very  large  tables,  lists,  or  menus.  The  theme  of  making 
content  understandable  and  navigable  is  addressed  primarily  in  Guidelines  12  to  14. 

Principles  of  Accessible  Web  Design 

The  WAI  has  produced  an  extensive  set  of  guidelines  for  authoring  accessible  web  pages.  The  HTML 
Writers  Guild  (see  http://www.hwg.org)  has  proposed  six  principles  of  accessible  web  design  upon  which  the 
WCAG  were  written: 

1.  Create  pages  that  conform  to  accepted  standards.  Use  the  W3C  recommendations  for  WCAG,  HTML  4.0,  and 
CSS2  for  designing  web  pages. 

2.  ICnow  the  difference  between  structural  and  presentation  elements.  Use  HTML  structural  elements  to  convey  page 
content  and  style  sheets  to  convey  page  presentation  and  formatting. 

3.  Use  HTML  4.0  features  to  provide  information  about  the  purpose  and  function  of  elements. 

4.  Ensure  that  pages  can  be  navigated  by  keyboard. 

5.  Provide  alternative  or  text-based  methods  to  access  non-textual  content  that  includes  images,  scripts,  multimedia, 
tables,  forms,  and  frames  for  user  agents  that  do  not  display  them. 

6.  Be  careful  of  common  programming  techniques  that  can  reduce  the  accessibility  of  your  site  such  as  ASCII  art, 
blinking  text,  or  adjacent  links  that  are  separated  by  non-printable  characters. 

Conclusion 

As  the  use  of  the  Web  is  perceived  to  be  an  effective  tool  for  dissemination  of  research  findings  or  for  the 
provision  of  asynchronous  instruction,  the  issue  of  accessibility  of  web  page  information  will  become  more  relevant. 
The  World  Wide  Web  Consortium  has  embraced  the  issue  of  accessibility  through  its  Web  Accessibility  Initiative 
and  essentially  thrown  open  the  gates  to  the  virtual  kingdom  for  persons  with  disabilities.  The  implications  of 
accessibility  features  for  sound  web  page  design  are  a benefit  that  offsets  the  additional  time  and  labor  required  to 
author  accessible  web  pages.  Because  of  the  commitment  of  the  W3C  to  web  accessibility,  the  statutory 
requirements  that  ensure  equal  opportunity  and  access  for  persons  with  disabilities,  and  the  preponderance  of 
resources  available  for  web  authors  to  use  in  the  development  of  accessible  web  pages,  new  and  experienced  web 
authors  have  a compelling  mandate  for  including  accessibility  features  in  web  pages. 
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Abstract  iThere  have  been  recent  advances  in  educational  theory  at  Higher  Education 
level,  reflected  in  Laurillard’s  conversational  framework  [3].  Current  Instructional  Design 
methods  and  their  underlying  educational  model,  are  not  compatible  with  current  views  of 
the  Higher  Education  teaching  and  learning  process.  Therefore  there  is  a need  for  a 
practical  instructional  design  method  which  is  inspired  by  the  Laurillardian  view  of  the 
teaching  and  learning  process.  The  rest  of  this  paper  discusses  the  way  in  which  design 
methods  developed  for  highly  interactive  computer  systems  have  been  employed  to  solve 
this  CAL  design  problem. 


CAL  DESIGN 

In  the  United  Kingdom  a government  white  paper  in  1991  detailed  the  government’s  plan  for  a cost  effective  expansion  of 
the  Higher  Education  system,  this  included  the  use  of  CAL  to  combat  increasing  student  numbers.  Following  the  white 
paper’s  publication  the  government  set  up  a number  of  initiatives  to  develop  CAL  for  Higher  Education,  Computing  in 
Teaching  Initiative  (CTI)  and  the  Teaching  and  Learning  Technology  Programme  (TLTP.)  These  initiatives  resulted  in  large 
quantities  of  CAL  being  produced  across  disciplines.  Investigations  conducted  on  a sample  of  these  resulting  CAL 
packages,  indicated  that  very  little  formal  design  was  carried  out  on  Higher  Education  CAL  [5].  The  only  formal  design 
method  in  evidence,  followed  an  Instructional  Systems  Design  (ISD)  approach  to  design 


THE  CONVERSATIONAL  FRAMEWORK 

ISD’s  underlying  model  of  the  teaching  and  learning  process  is  very  teacher  centred  - however  a student  centred  approach  is 
currently  a much  more  popular  idea  in  the  education  community.  Laurillard's  “conversational  framework"  [3]  falls 
somewhere  in  the  middle  ground.  The  framework  promotes  equal  roles  and  dialogue  between  the  teacher  and  learner  in  the 
complete  teaching  and  learning  process.  The  framework  identifies  twelve  activities  that  Laurillard  believes  must  happen 
[3] — by  traditional  methods  or  by  computer  — in  a given  learning  situation  for  learning  to  take  place. 


CREATING  A DESIGN  METHOD  WITH  AID  OF  SCENARIOS 

At  a British  HCI  group  workshop  in  1997  on  “Usability  and  Educational  Software  Design,”  it  was  stated  that  educational 
design  is  a craft  not  an  engineering  process. 

Schneiderman  [7]  says  that  “Design  is  inherently  creative  and  unpredictable.”  CAL  design  can  be  made  less  dependent  on 
the  particular  skills  and  talents  of  a few  artisans  by  applying  some  science  to  the  design  process,  embodied  in  an  explicit 
design  process.  As  Schneiderman  goes  on  to  say  “in  every  creative  domain,  there  can  also  be  discipline,  refined  techniques, 
wrong  and  right  methods,  and  measures  of  success.”  In  order  to  impose  such  a discipline,  it  was  necessary  to  find  some 
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way  to  use  Laurillard’s  conversational  framework  in  an  instructional  design  setting.  The  discipline  comes  from  the 
imposition  of  the  conversational  framework  on  to  a CAL  design  by  the  instructional  designer.  To  make  this  a reality,  it  was 
necessary  to  work  out  when  and  how  to  use  the  conversational  framework. 

Traditional  methods  for  solving  this  type  of  task  have  relied  on  the  use  of  observational  techniques  such  as  video  recording 
or  interviewing,  to  capture  knowledge  on  the  way  activities  are  performed  [6].  Scenarios  have  in  the  past  been  used  in  place 
of  observational  studies.  A Scenario  is  “a  narrative,”  “it  is  a description  of  context,  which  contains  information  about  users, 
tasks  and  environments.”  [2] 

Jack  Carroll  [1]  says  that  scenarios  can  help  to  ensure  that  computer  systems  are  “easy  to  leam/easy  to  use,”  that  they 

McKerlie  [4]  identified  two  distinct  roles  for  scenarios. 

1)  Supporting  the  generation  of  design  ideas. 

2)  Evaluating  a proposed  design. 

This  work  adds  a new  use  for  scenarios,  an  enhanced  version  of  role  1)  - 

3)  Supporting  development  of  new  design  methods. 

Maclean  & McKerlie  call  type  1)  scenarios  enivisioner  scenarios.  The  fundamental  idea  behind  the  use  of  envisioner 
scenarios  being  that  they  “drive. ..and  contribute  to  the  evolving  design.”  [4].  Although  scenarios  have  been  used  in  the  past 
to  envision  interactions  with  an  implemented  system,  these  statements  seem  also  to  apply  to  the  creation  of  a design 
method. 


THE  DESIGN  METHOD 

From  observation  of  existing  design  methods  and  the  way  people  generally  approach  CAL  design,  identifying  aims  and 
objectives  of  the  package  was  the  place  to  start.  Although  this  is  the  same  starting  place  as  the  traditional  instructional 
design  methods  that  were  earlier  criticized,  the  follow  on  is  radically  different.  The  scenarios  acted  as  a prompter  - what 
way  now?  By  constantly  returning  to  the  scenario,  requirements  for  the  design  method  were  elicited, 
e.g.  Having  identified  the  aims  and  objectives,  what  do  I do  now?  How  can  the  framework  help  me  to  proceed?  It  was  clear 
there  was  a gap.  It  identified  the  need  for  explicit  instruction  to  take  each  objective  and  work  round  the  conversational 
framework.  But  before  this  it  was  clear  decisions  had  to  be  made  with  respect  to  how  each  activity  in  the  conversational 
framework  would  be  implemented  - was  it  a human  to  human  activity  or  human/computer  or  by  some  other  means.  These 
decisions  resulted  in  formulation  of  a table  indicating  the  possible  implementations. 

Use  of  an  enhanced  envisioner  scenario  allowed  the  design  problem  to  be  made  more  realistic  and  concrete.  The  scenario 
provided  a setting  to  explore  requirements  for  a new  design  method  and  to  discover  what  instructional  designers  would 
require  from  such  a design  method.  It  does  however  remain  true  the  method  was  generated  through  a creative  process,  not 
the  mechanical  application  of  a systematic  method.  The  use  of  a scenario  contrasted  with  actual  immersion  in  a real  design 
problem  saved  considerable  time  and  effort.  In  principle,  the  technique  could  also  be  used  to  test  the  method  with  problems 
and  in  contexts,  which  would  be  difficult  or  impossible  to  use  for  real. 
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Abstract:  This  project  utilized  object-oriented  programming  in  Lingo  and  the  Director  authoring 
environment  to  bring  together  video,  audio,  text,  and  graphics  in  an  instructional  CD-ROM 
program  presenting  the  classical  tabla  traditions  of  South  Asia.  Working  in  a highly  collaborative 
framework  in  which  the  subjects  were  partners  as  well  as  informants,  the  developers  were  able  to 
use  interactive  media  to  reflect  the  pedagogic  structure,  mnemonic  techniques,  historical 
development,  and  performance  practices  of  the  oral  traditions  being  represented. 


One  of  the  greatest  challenges  faced  in  the  fields  of  ethnomusicology,  anthropology,  folklore  and  related 
disciplines  is  the  issue  of  using  the  written  word  as  a mode  of  communication  when  describing  traditions  that  utilize 
non-verbal  modes  of  expression.  Even  in  cultures  with  their  own  written  languages  and  elaborate  taxonomies,  we 
find  that  the  simultaneous  presence  of  gesture,  pitch  inflections,  iconography,  color  symbolism,  and  other  forms  of 
communication  serves  to  intensify  and  expand  an  experience  that  is  not  just  verbal,  but  truly  multi-sensory  in  nature. 

Interactive  multimedia,  particularly  when  delivered  in  high-bandwidth  contexts  such  as  CD-ROM 
presentations,  offers  the  possibility  of  transcending  the  limitations  of  writing,  but  without  sacrificing  writing's 
potential  for  conveying  and  promoting  analysis  and  reflection.  In  particular,  the  branching  structures  of  interactive 
presentations  allow  the  designer  to  present  a body  of  material  in  an  organized,  analytic  fashion,  just  as  one  can  find 
in  printed  works,  while  the  media  elements  themselves  offer  a more  profound  experience  of  the  actual  subject 
matter. 

In  the  world  of  music,  the  experience  of  learning  an  instrument  is  almost  always  associated  with  oral 
transmission.  Even  in  cultures  that  make  extensive  use  of  music  notation,  there  are  many  subtleties  of  interpretation, 
many  nuances  of  pitch,  rhythm,  and  tone  color  that  are  learned  only  through  careful  listening,  observation,  and 
imitation.  Often  this  body  of  knowledge  is  passed  on  through  generations  of  musicians,  and  represents  a kind  of 
living  cultural  treasure.  While  nothing  can  replace  the  human  relationships  on  which  this  transmission  is  based,  the 
three-fold  process  of  listening,  observation,  and  imitation  mentioned  above  can  also  take  place  when  the  traditions 
are  conveyed  though  computer-based  media.  If  the  material  being  presented  has  the  required  quality  and  integrity, 
the  sonic  aspects  of  a music  genre  can  be  accompanied  with  the  appropriate  amounts  of  visual  and  textual 
information,  and  the  level  of  learning  can  more  closely  approximate  the  presence  of  a teacher,  a qualified  "carrier" 
of  that  tradition.  This  project  attempts  to  utilize  an  interactive  presentation,  delivered  on  CD-ROM,  to  represent  and 
introduce  the  major  oral  traditions  of  tabla,  the  premier  percussion  instrument  of  Hindustani  classical  music.  The 
single  word  tabla  is  used  to  refer  to  a paired  set  of  drums  that  are  generally  played  by  finger  strokes  rather  than  by 
hand  strokes  or  sticks. 

The  intended  audience  for  the  program  is  quite  broad,  and  includes  learners  with  a background  in  music 
and/or  Asian  studies  as  well  as  those  with  a more  general  or  casual  interest.  Because  we  have  chosen  to  focus  on  one 
instrument,  a comprehensive  view  of  its  diverse  musical  genres  can  be  gained,  and  a practical  sense  of  how  to  play 
the  instrument  can  also  be  acquired,  without  exceeding  the  stringent  storage  limitations  of  the  CD-ROM  format.  In 
the  classical  music  of  North  India,  oral  transmission  of  musical  knowledge  was  originally  achieved  through  the 
system  of  gharanas,  families  of  elite  musicians.  These  families  served  in  the  courts  of  the  emperors,  nawabs,  and 
rajahs  of  the  Hindustani  cultural  region,  which  extended  from  Pakistan  in  West,  Afghanistan  in  the  North, 
throughout  North  India,  and  Bangladesh  in  the  East.  This  entire  region  was  once  dominated  by  the  Mughal  Empire, 


which  dominated  South  Asia  from  the  15th  century  until  the  rise  of  British  colonial  power  in  the  18th  century. 
Because  the  musical  families  often  continued  to  serve  the  same  courts  for  several  generations,  a certain  amount  of 
regional  variation  developed  in  terms  of  instrumental  technique  and  musical  structure.  During  the  20th  century,  with 
the  acceleration  of  communication  and  travel,  the  geographic  isolation  of  the  gharanas  began  to  break  down,  and  the 
context  of  the  music  changed.  It  was  no  longer  heard  exclusively  by  the  elite,  and  became  more  accessible  through 
emerging  media  such  as  records,  tapes,  audio  CDs,  and  broadcast  media.  Among  professional  Hindustani  classical 
musicians,  it  has  become  more  common  to  learn  repertoire  from  outside  one's  own  gharana,  and  the  collective  body 
of  musical  material,  in  spite  of  its  massive  size,  is  now  being  viewed  as  a group  of  bundled,  closely-related 
traditions. 

Therefore  it  is  important  for  the  program  to  include  representative  selections  from  each  of  the  major 
gharanas,  even  though  the  basic  strokes  are  held  in  common.  Working  with  tabla  virtuoso  Enayet  Hossain  and  his 
colleagues,  we  used  a DV  camera  to  videotape  several  selections  from  each  tradition,  and  edited  the  clips  in 
Quicklime  format  for  inclusion  within  the  interactive  presentation.  Along  with  solo  repertoire,  we  recorded  tabla 
duets,  tabla  trios,  and  tabla  in  its  more  common  role  as  an  instrument  that  provides  accompaniment  for  melodic 
soloists.  Through  Mr.  Hossain,  we  also  acquired  additional  material  from  sources  in  India,  and  brought  it  into  the 
program.  These  performances,  however,  include  many  rapid  passages  that  would  be  far  too  challenging  for  most 
beginning  learners. 

In  order  to  accommodate  beginners,  we  have  been  able  to  draw  upon  the  traditional  South  Asian  system  of 
teaching  and  transmitting  musical  patterns.  This  system  evolved  in  India  over  hundreds  of  years,  and  is  learned  by 
dancers,  vocalists,  and  melodic  instrumentalists  as  well  as  percussionists.  In  a lesson  with  a live  teacher,  these 
rudiments  constitute  the  first  material  to  be  learned.  The  system  is  a kind  of  drum  "language"  in  that  every  possible 
sound  made  on  the  tabla  has  a corresponding  verbal  syllable.  Although  it  is  quite  sophisticated,  it  is  not  a language 
such  as  the  drum  languages  of  West  Africa,  because  the  system  is  completely  non-lexical,  and  is  used  only  for 
musical  or  artistic  purposes,  even  though  many  of  its  syllables  are  shared  as  phonemes  in  Hindi  and  other  South 
Asian  languages. 

We  found  that  the  media  capabilities  of  CD-ROM  make  it  an  ideal  vehicle  to  convey  the  actual  sound  and 
display  the  required  technique.  Therefore  a section  was  included  which  focuses  on  the  basic  drum  syllables,  or 
"bols."  Each  spoken  syllable  corresponds  to  a discrete  sound  made  on  the  drums.  Working  with  professional  tabla 
players  to  make  sure  that  the  lexicon  was  complete,  we  represented  each  stroke  with  a high-resolution  audio  clip  of 
each  spoken  drum  syllable,  a small  video  clip  showing  how  each  stroke  is  produced,  and  phonetic  transcription  in 
text.  This  section  serves  both  as  a reference  for  advanced  musicians  and  as  a primer  for  beginners. 

To  communicate  with  the  widest  possible  audience,  we  chose  to  create  a cross-platfomi  CD-ROM  that  can 
be  played  on  both  Macintosh  and  Windows  systems.  As  an  authoring  tool,  Macromedia  Director  enabled  us  to  create 
the  application  rapidly,  so  that  we  could  build  a logical  navigation  structure  and  present  the  layers  of  textual 
information  without  losing  the  multi-sensory  dimension  that  is  so  crucial  to  the  learning  process.  The  video  clips 
were  presented  in  Quicktime  format,  because  Quicktime  is  also  cross-platform  and  can  be  easily  controlled  with 
Lingo,  Director's  scripting  language.  Most  of  the  programming  involved  navigation  control.  Aside  from  simple 
branching  commands  and  navigation  cues,  1 using  Lingo's  object-oriented  features  to  create  a "watcher  object"  that 
would  keep  track  of  the  places  visited  by  the  user  in  any  given  section.  The  places  are  stored  as  a two-dimensional 
array  in  RAM,  and  the  object  increments  the  array  with  each  program  location  visited,  allowing  the  user  to 
backtrack  through  the  program.  Other  programming  allows  the  user  to  play  the  Quicktime  videos  in  single  frame 
slow-motion,  so  that  the  music  students  can  observe  details  of  the  strokes. 

Although  the  main  function  of  this  CD-ROM  is  pedagogical,  there  are  also  documentary  and  historical 
aspects  that  would  be  of  interest  to  more  general  students,  and  to  scholars  in  ethnomusicology  and  Asian  studies. 
Maps  are  provided  to  show  the  original  geographic  locations  of  the  various  gharanas,  full  artists'  profiles  are 
included,  and  a historical  section  is  provides  background  on  each  of  the  various  gharanas,  as  well  as  on  the  general 
development  of  tabla  as  an  instrument.  The  CD-ROM  will  also  serve  as  a kind  of  digital  archive,  preserving  the 
work  of  several  representative  artists  from  living  oral  traditions  at  the  turn  of  the  millennium. 

As  a means  of  conveying  oral  traditions,  well-designed  interactive  multimedia  programs  are  able  to 
combine  the  most  useful  aspects  of  both  print  media  and  linear  media  to  provide  learners  with  a more 
comprehensive  experience.  But  in  order  to  reflect  the  subject  matter  accurately,  the  designers  must  themselves  be 
knowledgeable  of  the  material  being  represented,  and  must  try  as  much  as  possible  to  incorporate  traditional 
pedagogy  as  part  of  that  experience. 
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Abstract:  This  paper  is  fundamentally  about  supporting  learning  in  technology-enhanced 
educational  settings  where  students  are  often  confronted  with  very  large  amounts  of  information, 
and  where  they  are  also  required  to  exercise  a great  deal  of  independence  as  part  of  their  learning 
activity.  First,  it  outlines  very  briefly,  the  critical  attributes  of  such  educational  settings,  the 
challenges  to  learning  that  they  pose,  and  some  ways  of  supporting  students  in  their  learning  as 
they  are  confronted  with  these  challenges.  The  paper  also  discusses  the  pedagogical  foundations  for 
building  these  types  of  cognitive  supports  and  discusses  some  preliminary  work  that  is  being 
carried  out  on  these  lines  at  The  University  of  Melbourne  in  Australia. 


Technology-Enhanced  Learning  Environments 

For  the  purposes  of  this  paper,  technology-enhanced  learning  environments  refer  to  those  educational  settings  where 
some  element  of  instructional  technology  (such  as  CD-ROMS  or  the  Web)  is  used  to  support  the  interaction  between 
the  teacher,  the  learner  and  the  educational  content.  Students  in  these  educational  settings  are  often  confronted  with 
very  large  amounts  of  information  by  way  of  databases  and  other  resources  (making  them  information-dense)  which 
they  are  required  to  use  to  pursue  self-directed  and  open-ended  inquiries.  Interest  in  building  these  educational 
environments,  where  students  are  immersed  in  learning  by  doing  and  where  the  meaning  of  knowledge  and  skills  are 
realistically  embedded  in  authentic  problem  situations,  has  been  growing  in  the  last  decade  (see  Brown,  Collins,  & 
Duguid,  1989).  These  environments  are  designed  not  so  much  to  "instruct"  as  to  provide  contexts  wherein 
understanding  and  insight  can  be  uniquely  cultivated.  Papert's  concept  of  "microworlds  as  incubators  for 
knowledge"  reflects  the  philosophical  bias  of  these  technology-enhanced,  student-centered  learning  environments 
(Papert,  1993,  p.  120).  They  are  called  student-centered  because  the  focus  in  these  learning  environments  is  more  on 
learning,  and  less  on  the  delivery  of  the  content.  Examples  of  these  student-centered  learning  designs  are  goal-based 
scenarios  (Schank,  1997),  and  problem-based  learning  (Koschmann,  Kelson,  Feltovich  8l  Barrows,  1996). 

Challenges  and  Problems  of  these  Learning  Environments 

Technology-enhanced  student-centered  learning  environments  do  not,  in  themselves,  lead  to  learning  efficiency  or 
effectiveness.  Indeed  for  some  learners,  the  use  of  technology  and  the  student-centered  nature  of  these  learning 
environments  can  be  quite  daunting,  and  pose  a real  threat  to  their  success  and  motivation  to  learn.  While  creating 
opportunities  for  student-centered  learning,  these  environments  also  create  demands  for  new  skills  in  managing 
complex  information  and  higher  order  cognitive  processes.  Being  successful  in  such  learning  environments  requires 
the  ability  to  organize,  evaluate,  and  monitor  the  progress  of  one’s  learning.  Not  all  learners  have  the  skills  to 
function  efficiently  and  effectively  in  such  educational  settings.  Students  need  help  with  acquiring  learning 
strategies  to  enable  them  to  organize  and  reflect  on  information  they  have  encountered.  They  have  to  be  taught  the 
learning  skills  and  self-monitoring  (i.e,  metacognitve)  strategies  which  would  in  turn  enable  them  to  take  advantage 
of  rich  databases,  and  open-ended  inquiries  (see  Weinstein  & Mayer,  1986;  Jonassen,  1988).  While  a considerable 
amount  of  work  has  been  done  in  supporting  students’  learning  with  various  types  of  learning  strategies  in  such 
educational  settings  (see  for  example,  Bernard  & Naidu,  1992;  Edelson,  Gordin,  & Pea,  1999),  work  on  supporting 
student  learning  with  software-based  cognitive  tools  is  still  lagging. 
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What  does  this  Work  Involve? 


Weinstein  and  Mayer  (1986)  suggest  that  the  development  of  learning  strategies  (i.e.,  in  learning  how  to  learn)  can 
influence  learner  characteristics.  They  argue  that  employing  these  strategies  and  methods  can  help  with  the  encoding 
process,  which  in  turn  affects  learning  outcomes.  These  authors  have  identified  several  categories  of  learning 
strategies,  namely  rehearsal,  elaboration,  organizational,  self-monitoring  and  motivational  strategies.  These 
strategies  provide  a pedagogically  sound  franiework  for  supporting  "learning  how  to  learn",  and  they  can  also  be 
used  to  guide  work  on  scaffolding  and  supporting  student  learning  in  information  dense,  technology  enhanced 
learning  environments.  They  have  guided  the  research  and  development  work  that  is  reported  in  this  paper. 

As  part  of  our  work  on  the  design  and  development  of  technology-enhanced  learning  environments,  we  have  been 
exploring  both,  the  pedagogical  designs  and  the  technical  architecture  of  a variety  of  these  types  of  learning 
supports.  Some  examples  of  these  are  comparison  frames  to  view  content  elements  in  relation  to  each  other,  clusters 
for  organizing  data  by  themes,  modeling  and  simulations  for  exploring  dynamic  processes,  and  chronological  traces 
for  providing  students  with  representations  of  their  own  actions. 

Comparison  frames  (an  instance  of  an  elaboration  strategy)  are  particularly  useful  in  areas  such  as  medical  and  the 
biological  sciences  where  students  are  often  observing  cells  and  images  of  anatomical  parts  to  determine  unique 
characteristics  and/or  abnormalities.  Clustering  (an  instance  of  an  organizational  strategy)  is  useful  for  organizing 
data  or  information  to  draw  out  and  represent  meaningful  relationships  among  them.  Interactive  modeling  and 
simulation  (instances  of  elaboration  and  self-monitoring  strategies)  are  useful  approaches  to  support  tutorials  in 
applied  domains  such  as  in  mathematics  as  well  as  in  the  natural  sciences.  As  part  of  this,  concrete  dynamic  models 
with  which  the  student  can  interact  and  experiment  are  used  to  represent  abstract  concepts.  Chronological  traces  (an 
instance  of  organizational  and  self-monitoring  strategies)  allow  students  to  observe  their  own  actions  and  problem- 
solving behavior  after  they  have  reached  a solution,  and  compare  it  with  other  solutions. 

The  presentation  of  this  paper  will  focus  on  "The  Skin  Atlas"  which  is  an  instance  of  an  elaboration  strategy.  The 
"Skin  Atlas"  is  a multimedia  module  that  contains  images  of  the  histology  and  pathology  of  human  skin  and 
associated  disorders,  which  allows  students  to  study  specific  images  in  detail  and  to  select  and  compare  images. 

Conclusion 

We  believe  that  the  kind  of  work  that  is  described  in  this  paper  has  the  potential  to  significantly  expand  our 
knowledge  about  scaffolding  student  learning  in  information  dense,  technology  enhanced  learning  environments.  By 
improving  the  scaffolding  of  student  learning  in  these  environments,  we  hope  to  assist  the  efficiency  of  student 
learning.  Furthermore,  a record  of  students’  actions  at  a number  of  levels  (e.g.,  gathering,  comparing,  clustering, 
and  organizing  data,  as  well  as  self-monitoring),  has  the  potential  to  provide  a richer  picture  of  their  learning 
activities.  These  can  be  reflected  upon  by  students  themselves  for  revision  and  group  discussions,  by  the  teachers  for 
assessment  purposes,  and  by  researchers  to  understand  and  improve  student's  learning  experience  in  terms  of  content 
gathered,  and  the  types  of  reflective  learning  activity  in  which  they  have  been  engaging. 
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Abstract:  This  paper  reports  on  the  first  phase  of  a research  project  which  aims  to  support  student 
learning  through  computer-mediated  communication  among  a group  of  about  20  students  enrolled 
in  an  educational  management  course  in  the  Open  University  of  Hong  Kong.  It  describes  the  results 
of  a questionnaire  designed  to  collect  information  on  how  ready  the  students  are  to  access  a Web- 
based  course.  Their  interaction  patterns  in  the  Discussion  Board  during  the  first  few  months  of  the 
implementation  are  analysed.  Some  challenges  for  managing  the  online  communication  activities  in 
the  institutional  context  are  discussed  and  the  next  action  steps  are  then  suggested. 

Introduction 

The  Open  University  of  Hong  Kong  (OUHK)  is  the  only  distance  learning  institution  in  Hong  Kong.  For  the 
April  1999  presentation,  the  OUHK  launched  a pilot  project  to  promote  an  online  course  delivery  system:  the  Online 
Learning  Environment  (OLE).  A Web  course  development  tool  - WebCT  ^ was  selected  as  a unified  platform  for 
delivery.  It  is  intended  that  the  Interactive  Tools  in  the  WebCT  - Email,  Discussion  Board  and  Chat  Room  - will 
create  an  interactive  learning  environment  for  communication  among  students,  tutors  and  the  Course  Coordinator. 

Course  E805  Effective  Leadership  and  Management  in  Education  was  chosen  for  the  pilot.  E805  is  a post- 
graduate level  course  in  the  Master  of  Education,  which  aims  to  improve  participants’  professional  capability  in 
educational  management  through  developing  knowledge  of  theory  and  critical  reflection  on  practice.  An  invitation 
letter  describing  the  nature  of  the  online  mode  of  learning  was  sent  to  all  E805  students  before  the  commencement 
of  the  course.  There  were  70  students  on  the  course  in  total,  and  eventually  20  students  chose  the  online  mode.  A 
videotape  on  setting  up  the  OLE  system  and  a User’s  Guide  to  the  OLE  were  provided  to  these  students. 

In  the  literature,  computer-mediated  communication  (CMC)  has  been  discussed  largely  in  terms  of  providing  an 
additional  medium  to  assist  students’  learning  (Burgstahler  and  Swift  1996).  Claims  have  been  made  that  it  can 
provide  an  environment  through  which  learners  can  interact  and  engage  themselves  in  social  activities  such  as 
discussion  and  debate.  As  the  Course  Coordinator  (CC)  of  the  course,  the  researcher  is  concerned  about  how  the 
CMC  tools  in  the  OLE  system  can  support  student  learning  by  providing  more  opportunities  for  them  to  share  their 
understanding  of  theories  and  their  reflections  on  management  practice  with  their  tutor  and  other  students. 

The  Study 

This  study  is  the  first  phase  of  the  project,  which  aims  to  gain  insight  into  how  students  had  adjusted  to  the  E805 
OLE  during  the  first  five  months  of  implementation.  Students’  views  of  CMC  in  learning  and  how  ready  they  are  to 
use  the  CMC  tools  in  studying  the  course  were  collected  through  a questionnaire  survey.  The  electronic 
communication  messages  over  this  period  were  reviewed  as  a source  of  information  for  understanding  whether  any 
interaction  exists  and  how  it  relates  to  learning. 


Questionnaire  Data 

The  questionnaire  was  sent  to  all  20  online  group  students.  Seven  students  responded,  a response  rate  of  35%. 
The  questionnaire  comprised  26  questions  which  probed  the  following  3 issues: 

Internet  access.  All  of  the  respondents  had  their  own  PCs,  and  were  able  to  access  the  Internet  at  home  or  at  the 
workplace.  When  asked  how  many  hours  per  week  they  used  the  Internet,  students  most  often  responded  1-5  hours, 
and  two  students  indicated  usage  of  more  than  10  hours.  Web  browsing  and  Email  were  the  most  common  functions 
used,  and  the  activities  included  communication  (33%),  online  study  (33%)  and  entertainment  (20%). 

Skills  in  using  the  Internet.  Responses  were  given  on  a 4-point  Likert  Scale  (1  = strongly  disagree  and  4 = 
strongly  agree)  to  indicate  their  skills  in  using  the  Internet.  In  general,  the  participants  showed  competence  in  using 
Email  and  Web-browsing  (mean  D3.0),  but  were  less  competent  in  using  the  Chat  and  electronic  conferencing 
system  (mean  = 2.5)  in  the  Internet. 

Use  of  the  E805  OLE.  At  the  time  the  survey  was  conducted,  all  the  respondents  had  accessed  the  OLE  for  E805. 
Most  of  the  respondents  (86%)  spent  less  than  one  hour  per  week  on  the  OLE,  and  accessed  the  OLE  system  at 
home  (71%)  most  of  the  time.  This  survey  paid  special  attention  to  students’  use  of  the  Interactive  Tools  in  the  OLE. 
Respondents  were  asked  about  the  usefulness  of  each  of  these  features  up  to  that  point  in  their  studies.  Email  was 
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rated  as  at  least  ‘somewhat  useful*  by  more  than  50%,  but  the  Discussion  Board  and  Chat  features  were  not  rated  as 
highly.  Some  students  had  not  used  the  latter  features  at  the  time  the  survey  was  conducted.  Of  those  who  had,  more 
than  half  the  respondents  thought  they  were  not  very  helpful. 

Online  Discussion  Content 

There  were  41  messages  in  the  Discussion  Board  over  the  above  period.  The  items  are  arranged  according  to 
chronological  order.  In  the  column  on  student  postings,  the  alphabetic  letters  indicate  individual  students. 


Item 

no. 

Item  Title 

Themes 

No.  of  postings 

Student 

Tutor 

cc 

Total 

1 

Welcome  from  Course  Coordinator 

Hello  message 

a+b+c 

0 

1 

4 

2 

Welcome  from  tutor 

Hello  message 

c+d 

1 

0 

3 

3 

Web-linlcs  - • ' ■ : ■ ‘ ■ ' 

Course  content  related 

e 

0 

1 

2 • 

4 

Statistical  figures 

Course  content  related 

3e+f 

1 

1 

6 

5 

What  is  management? 

Assignment  related 

2a+e+f 

2 

0 

6 

6 

What  are  taken-for  granted  assumptions? 

Course  content  related 

e 

1 

1 

3' 

7 

Online  survey 

Course  administration 

0 

0 

1 

1 

8 

Course  material 

Course  content  related 

a 

0 

3 

4 

9 

Tutorial  visit 

Course  administration 

0 

0 

1 

1 

10 

Question  in  Section  6 

Course  content  related 

e 

0 

1 

2 

11 

The  reform  of  Education  Department  in:Hong  Kong 

Assignment  related 

0 

0 

1 

1 

12 

Forward  Email 

Course  administration 

2a 

0 

2 

4 

13 

Examples  of  leadership  in  secondary  school 

Course  content  related 

2a 

1 

1 

4 

21 

6 

14 

41 

Table  1:  Summary  of  the  electronic  messages  in  the  Discussion  Board 

Reflections  and  further  actions 

Based  on  the  survey  outcomes,  students’  training  needs  were  further  identified.  Despite  the  fact  that  this  was  a 
self-selected  group  with  a reasonable  level  of  computer  competence,  they  needed  more  guidance  on  using  the 
Discussion  Board  and  Chat  features.  To  help  students  become  more  familiar  with  these  features,  the  researcher  will 
summarize  the  procedures  and  highlight  some  of  their  useful  functions.  Briefing  sessions  will  also  be  arranged. 

The  electronic  messages  showed  the  interaction  patterns  of  the  online  group.  Among  the  13  items,  only  four  of 
them  showed  student-student  interaction  and,  in  general,  students  were  not  involved  as  active  partners.  Only  six 
students  posted  messages,  but  only  three  of  them  were  more  active  members.  To  motivate  students  to  become  more 
involvement  in  online  discussion  activities,  Chat  room  sessions  will  be  organized.  This  is  to  allow  students  to  have  a 
real-time  interaction  to  discuss  their  assignments  or  the  suggested  topics.  The  tutor  will  be  invited  to  join  the  Chat 
room  sessions,  and  the  transcript  of  the  Chat  discussion  will  be  posted  on  the  Discussion  Board  for  further 
examination  with  students’  permission.  Hopefully,  in  such  ways,  students  will  be  encouraged  to  play  a more  active 
role  in  utilizing  the  interactive  features  in  the  OLE. 

It  should  be  noted  that  most  of  the  discussion  topics  were  related  to  course  content  or  assignments.  Based  on  the 
item  titles,  it  is  difficult  to  tell  whether  online  communication  can  promote  improved  learning.  These  messages  need 
to  be  further  reviewed  through  content  analysis  (Mason  1992)  to  understand  whether  any  interaction  exists  and  how 
it  relates  to  learning.  Last,  statistics  on  users’  login  records  and  the  time  spent  on  each  component  in  the  Interactive 
Tools  will  be  collected.  This  will  help  identify  individuals  who  read  the  messages  posted  but  do  not  participate  in 
discussion  (so-called  Murkers’). 
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A Framework  for  Generating  Non-trivial  Interactive  Mathematical 
Exercises  in  the  Web:  Dynamic  Exercises 


Introduction 

Successful  tutoring  is  largely  a matter  of  well-timed  and  focused  feedback.  A computer  program 
can  hardly  replace  a good,  motivated  teacher,  but  it  can  certainly  do  better  than  a non-existing  one. 
Computer- verified  interactive  mathematical  exercises  are  usefial  cognitive  tools  supporting  the  development 
of  personal  mathematical  skills  and  knowledge  in  the  absence  of  human  tutoring.  Our  goal  is  to  find  out  and 
propose  specific  means  supporting  computer-aided  solving  of  computer-generated  mathematical  exercises. 
We  aim  to  achieve  this  goal  by  introducing  a concrete  system  called  Dynamic  Exercises  (DE)  for  doing 
non-trivial  exercises  interactively  in  the  Web. 

Exercises  created  with  DE  are  said  to  be  in  the  Web,  since  they  are  accessed  via  Internet  and 
processed  by  using  basic  WWW  services.  Exercises  are  said  to  be  interactive,  because  the  process  of 
solving  exercises  is  carried  out  in  incremental  steps,  in  an  interaction  with  a computer  program  being 
capable  of  interpreting  and  manipulating  user  input  and  providing  feedback.  Finally  and  most  importantly, 
exercises  are  claimed  to  be  non-trivial  since  exercises  are  designed  and  interpreted  as  multiple  code 
modules  using  DE  extended  Maple  programming  language.  Solving  exercises  does  not  require 
understanding  Maple  language  since  input  can  be  elaborately  processed  using  arbitrary  text  filters. 

This  study  continues  our  previous  line  of  work  of  designing  and  implementing  Internet-based 
learning  environments,  especially  from  the  viewpoint  of  teaching  and  learning  university-level  mathematics 
(see,  e.g.,  (Pohjolainen  et  al,  1999)  and  (Nykanen  & Ala-Rantala,  1998)).  Motivation  of  constructing 
computer-generated  interactive  exercises  originates  largely  from  a recent  survey  conducted  with  our  A&O 
learning  environment  (see  (Pohjolainen  et  al,  2000)).  DE  aims  to  develop  things  further. 

Design  Framework 

The  design  and  implementation  process  of  creating  interactive  exercises  consists  of  three  phases: 
designing  a dynamic  exercise,  implementing  it  as  a set  of  form  template  files,  and  publishing  it  under  a DE 
server,  which  finally  runs  the  exercises  when  requested.  Each  dynamic  exercise  consists  of  one  or  more 
forms.  Forms  are  logical  sub-tasks  of  the  exercise,  compiled  from  form  template  files  on  demand.  Forms 
introduce  mathematical  tasks,  formulae,  textual  descriptions,  variables,  and  graphics  for  the  user,  and 
accept  input  from  the  user. 

User  input  is  interpreted  automatically  by  form's  input  evaluation  programs.  Feedback  is  presented 
by  means  of  textual  or  graphical  output  including  comments,  variable  values,  and  formulae.  In  addition, 
input  evaluators  may  branch  the  execution  flow  to  other  forms,  thereby  providing  entire  forms  as  an  output. 
This  kind  of  branching  of  the  form  execution  makes  it  possible  to  effectively  implement  exercises  with  non- 
linear structure.  For  instance,  branching  enables  recognising  and  accepting  various  kinds  of  different 
solution  strategies  instead  of  a single  one. 

Programs  generating  exercises  and  providing  input-sensitive  feedback  are  written  using  the  DE 
extended  Maple  programming  language  and  are  embedded  in  the  form  template  files.  Program  blocks 
essentially  set  up  task  variables,  produce  graphical  plots,  and  evaluate  user  input  when  present.  Publishing 
an  exercise  means  putting  the  exercise  form  template  files  in  some  appropriate  DE  author  directory.  In 
addition,  classes  of  exercises  can  be  compiled  into  libraries,  the  use  of  which  can  be  optionally  controlled 
and  monitored  using  some  additional  WWW  mechanisms. 

Syntax  for  form  template  files  is  kept  as  straightforward  as  possible.  Each  form  template  file  is  a 
text  file  consisting  of  a logical  combination  of  the  following  DE  elements:  INITIALCODEBLOCK,  TEXT- 
FIELD,  OUTPUTFIELD,  INPUTFIELD,  IMAGELINK,  LOADFORMBUTTON,  and  RESETBUTTON. 
These  elements  (with  proper  attribute  values)  initialise  form's  mathematical  task,  provide  static  text 
descriptions  (optionally  including  other  media  elements),  and  print  dynamic  DE  variable  values  for  the 
user.  In  addition,  elements  accept,  parse,  and  evaluate  user  input,  insert  hypertext  links  pointing  to 
generated  images,  add  buttons  for  manually  launching  other  forms,  and  finally,  provide  a button  for 
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resetting  the  exercise  and  generating  a new  one.  DE  also  enables  generating  demonstration-orientated 
exercises  instead  of  problem-based  ones.  Figure  1 presents  an  example  of  a simple  exercise  in  action. 
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Figure  1:  Solving  a simple  exercise:  interacting  with  a form  via  Web  browser. 

DE  elements  INITIALCODEBLOCK  and  INPUTFIELD  introduce  Maple  programs  needed  for 
initialising  the  form’s  mathematical  problem  and  evaluating  user  input.  From  non-technical  point  of  view, 
the  interaction  scheme  is  quite  straightforward:  first  a set  of  problem  variables  is  generated,  which  are 
presented  to  the  user  along  with  some  static  descriptive  text  fields.  When  user  types  in  an  answer  it  is 
parsed  and  processed  by  the  program  defined  in  the  selected  INPUTFIELD.  Answer  evaluator  produces 
elaborate  feedback  to  the  user  or  jumps  to  another  form  that  is  compiled  and  presented  to  the  user.  After 
one  interaction  step  is  complete,  DE  server  is  ready  to  receive  the  next  input  from  the  user  and  the  process 
continues.  DE  monitors  the  interaction  process  and  maintains  a simple  symbol  table  of  DE  variables 
recording  essential  state  information.  To  get  a better  idea  how  forms  are  designed  and  written  in  practice, 
please  see  the  DE  home  page  at  <http://matriisi.ee.tut.fi/  ~onykane/deserver/>. 

Implementation 

DE  is  based  on  a well-known  CGI-based  client-server  architecture.  Server  software  effectively 
exploits  Mathedge  development  library  providing  full  Maple  V Release  3 Kernel  functionality  (Waterloo, 
1995).  Performance  requirements  for  clients  are  minimal  since  DE  server  carries  out  all  computations. 
Technical  implementation  of  Dynamic  Exercises  follows  guidelines  found  in  (Nykanen  & Pohjolainen, 
1998).  MathEdge  is  a relatively  expensive  product,  available  for  HP9000  and  Sun  platforms,  and  the  large- 
scale  use  of  the  software  is  limited  inside  our  university  with  effective  licensing  policy.  Maple  is  a well- 
known  (commercial)  general-purpose  computer  algebra  system  by  the  Waterloo  Maple  Software  and  the 
University  of  Waterloo.  Maple  includes  a nice  mathematical  programming  language  and  a rich  set  of 
mathematical  packages  and  plotting  facilities  (see,  e.g.,  (Heck,  1993)). 
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Abstract:  During  the  last  four  years,  the  Hellenic  Pedagogical  Institute  (HPl)  has  had  a 
consistent  presence  in  the  WWW  virtual  space,  providing  various  educational  services  to  the 
Greek  schools’  community  nittp://www.pi-schools.gr).  With  the  implementation  of  the  project 
“Added  Value  Services  on  the  Greek  Web  for  Schools”,  the  HPl  aims  to  create  an  integrated 
learning  environment  over  the  WWW,  which  is  going  to  complement  and  extend  the  already 
existing  infrastructure.  This  paper  addresses  the  objectives,  the  design  principles  and  the 
functions  of  the  project. 


Project’s  philosophy  and  design 

During  the  last  three  years,  the  Greek  Ministry  of  Education  has  been  attempting  to  introduce  and 
utilize  Internet  technology  in  the  educational  process.  Through  various  projects,  secondary  schools  around  the 
country  are  supplied  with  advanced  computer  and  networking  equipment  as  well  as  Internet  access.  None  the 
less,  the  use  of  sophisticated  equipment  and  Internet  connection  does  not  necessarily  mean  utilization  of 
Internet  technology  to  the  educational  process.  Having  access  to  the  information  does  not  necessarily  mean  an 
advance  of  knowledge  (Laurilland,  1993,  p.  123).  The  critical  point  for  the  effective  incorporation  of  Internet 
services  in  the  educational  process  is  related  firstly  with  a convenient  access  to  high-quality  content  and 
secondly  with  teachers’  capability  to  use  the  provided  information  and  services  as  structural  materials  and 
components  for  the  design  of  educational  activities. 

The  HPl  with  the  design  and  implementation  of  the  project  “Added-Value  Services  on  the  Greek  Web 
for  Schools”  is  aiming  to  create  an  integrated  environment  for  the  distribution  of  educational  material  and 
services  through  the  Internet,  for  Greek  secondary  schools.  The  HPI’s  objective  is  not  to  enforce  a centralized 
educational  model  on  the  Web,  but  to  establish  a number  of  services  and  design  principles  for  the  creation, 
expansion  and  maintenance  of  the  Greek  Educational  Web.  These  services  are  classified  along  two  main  axes: 

- Implementing  a distance  learning  center  (in  a pilot  mode)  over  the  Internet  for  the  increasing  training  needs 
of  Greek  secondary  school  teachers. 

- Providing  the  schools  with  the  ability  to  promote  their  projects  and  activities  through  the  WWW. 

The  project  creates  an  integrated  learning  environment,  which  is  going  to  cover  the  functionality  and 
activities  for  both  axes.  The  users  of  the  environment  will  have  different  access  levels  to  the  information  and 
services  provided  according  to  their  roles  and  needs.  The  services  and  activities  are  described  as  virtual 
“spaces”  following  the  standard  Internet  terminology. 

The  first  axis  aims  to  create  a flexible  Internet  based  educational  platform  for  the  design,  implementation 
and  delivery  of  teacher  training  services  at  a distance.  The  adopted  training  model  is  based  mainly  on  the 
principles  of  Supported  Self-Learning  model.  In  this  model,  the  trainee  has  the  ability  both  to  use  educational 
material  and  manipulate  its  flow  and  structure  according  to  his/her  learning  goals.  In  addition,  in  order  to  avoid 
feelings  of  isolation  and  inactivity  which  are  common  during  tele-training  programs,  dynamic  communication 
channels  (one  to  one,  one  to  many,  many  to  many)  will  be  established  (Nipper,  1989).  In  the  framework  of  the 
adopted  model,  the  distance-training  center  includes  the  following  virtual  spaces: 

“Space  ” of  training  materia^  which  is  structured  over  two  levels  (level  of  resource-based  training  and 
level  of  structured  training).  The  resource-based  training  level  will  function  as  a reference  “place”  for  the 
teachers  as  well  as  a rich  source  of  educational  material  for  the  enhancement  of  their  work.  The  level  of 
structured  training  aims  to  support  the  implementation  and  delivery  of  structured  training  programs  for  specific 
topics,  making  use  of  flexible  tools  appropriate  for  the  demands  and  features  of  the  Web-based  learning 
environment  (Dabbagh,  1999). 
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Promotion  "Space”  for  the  teachers'  work  The  structure  and  the  function  of  this  space  will  follow  that 
provided  by  online  journals,  which  are  very  common  over  the  Web. 

Teachers'  "Meeting  point'',  providing  the  community  of  teachers  with  both  synchronous  and 
asynchronous  communication  channels. 

The  design  and  implementation  of  the  second  axis  aims  to  offer  to  the  members  of  the  educational 
community  the  opportunity  to  contribute  their  own  work  and  activities  on  the  Greek  Web  for  Schools. 
Additionally,  already  existing  digital  educational  material  can  be  gathered  and  structured  under  thematic 
categories  in  order  to  facilitate  the  search  process  (Luebke  et  ai,  1999;  Collis,  1995,  p.  6-9).  The  second  axis 
includes  the  following  virtual  spaces: 

Space  of  educational  material.  The  material  will  be  annotated  according  to  the  developing 
international  standards  for  educational  metadata.  The  users  will  be  able  to  submit  remarks  and  comments  on  the 
provided  material  along  with  their  own  work. 

Kiosk  for  school  projects.  In  this  virtual  place,  school  projects  and  activities  will  be  accommodated 
under  specific  thematic  structures. 

Exhibition  of  educational  software.  In  this  virtual  space,  informative  material  will  appear  concerning 
evaluated  titles  of  educational  software. 

Schools  ’ "meeting  point '',  providing  the  members  of  the  educational  community  (students,  parents, 
teachers,  managers)  with  both  synchronous  and  asynchronous  communication  channels. 

Operating  specifications  of  the  project 

Online  database  of  training  and  learning  material.  The  database  will  cover  the  needs  of  the 
educational  material  and  software  along  with  those  related  to  the  level  of  resource  - based  training.  Also, 
through  the  database  application,  both  the  needs  of  the  Promotion  “Space”  for  the  teachers’  work  and  the  Kiosk 
for  school  projects  will  be  facilitated. 

Applications  for  the  access  to  the  Learning/Training  material  over  the  Internet.  Through  this 
application,  the  members  of  the  educational  community  are  going  to  dynamically  interact  with  the  material 
through  their  favorite  browser  over  the  Internet. 

Online  courses ' authoring  and  delivery  tools.  These  tools  will  facilitate  instructors  to  create  dynamic 
web-based  courses  and  make  them  accessible  over  the  Web. 

Applications  for  text-based  synchronous  and  asynchronous  communication.  These  applications  are 
going  to  serve  the  needs  for  the  “meeting”  points  of  both  axes. 

On-line  help  desk  and  search  engine.  The  application  will  serve  all  spaces  described. 

Future  plans  for  the  project  involve  the  creation  of  intelligent  help  agents  and  searching  machines 
which  will  be  included  in  the  learning  environment.  Furthermore,  the  potential  and  the  abilities  of  the 
collaborative  virtual  spaces  (MOO)  for  educational  purposes  will  be  investigated.  In  addition,  peripheral  content 
servers  will  be  created  in  order  to  accommodate  issues  of  local  interest.  Finally,  a lot  of  effort  will  be  put  on  the 
enrichment  of  the  environment  with  educational  content.  Towards  this  direction,  all  related  authorities  and 
organizations  have  been  invited  to  participate. 
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Abstract:  This  paper  investigates  the  introduction  of  software  into  an  adult  literacy  program.  Pre  and  posttests, 
questionnaires,  focus  group  mterviews  and  iterative  observation  are  combined  to  form  a multi-method  assessment 
strategy.  The  method  is  used  to  assess  the  satisfaction  that  users  derive  from  computerised  instruction.  It  is  also  used  to 
assess  the  culture  of  learning  that  develops  in  the  classroom.  Six  components  of  user  satisfaction  are  identified:  learning, 
teacher  role,  tedium,  real-world  applications,  software  and  gender  differences.  An  approachable,  friendly  and  interactive 
teacher  is  found  to  be  the  key  component  of  a successful  learning  environment. 


Introduction 

This  project  sought  to  identify  best  practices  for  introducing  technology  into  the  classroom.  It  was  determined  that  this 
matter  would  best  be  investigated  by  answering  the  following  questions,  what  is  the  nature  of  the  satisfaction  that  users  derive 
from  a computerised  adult  literacy  course,  and  what  is  the  nature  of  the  culture  of  learning  that  develops  in  such  a course.  Current 
thinking  has  it  that  multiple  methods  are  needed  to  capture  the  variety  of  outcomes  that  students  derive  from  an  educational 
experience.  A multi-method  assessment  strategy  is  described  below  as  are  the  results  obtained  from  its  application 

The  study 

Five  software  modules  were  developed.  They  were  designed  to  teach  the  following  literacy  skills. 

Module  1:  Reading  words  and  phrases 

Module  2:  Filling  out  forms 

Module  3:  Writing  for  various  purposes 

Module  4:  Following  instructions 

Module  5:  Reading  to  find  information 

Savenye  (1992)  considers  suitable  methods  for  answering  common  research  questions.  Her  suggestions  lead  to  the 
selection  of  the  followirig  methods  for  this  study:  participant  observation  and  focus  group  interviews  to  assess  user  satisfaction  and 
the  culture  of  learning.  Paper  and  pencil  tests  to  assess  learning  outcomes  (the  quantity  of  learning)  and  questiomiaires  to  assess 
attitude  and  socio-demographic  background.  Arrangements  were  made  to  access  two  basic  computer  literacy  classes  that  were  run 
in  the  Toronto  Offices  of  the  United  Food  and  Commercial  Workers  Union.  Each  class  contained  13  students.  The  methods  were 
applied  in  each  class  over  five  consecutive  weeks  in  the  following  manner.  Prior  to  the  first  class  the  attitude  and  socio- 
demographic questionnaires  were  administered.  Pre  and  posttests  were  administered  before  and  after  the  students  studied  each 
module.  Students  were  observed  as  they  worked  with  eacn  module  and  a focus  group  interview  was  run  at  the  end  of  each  class. 
Data  from  the  first  set  of  observations  was  collated  into  groups.  The  name  of  each  group  was  said  to  represent  a component  of  user 
satisfaction.  Coiofirmation  of  the  accuracy  of  the  first  set  of  observations  was  sought  during  the  first  focus  group  interview.  Data 
from  the  interview  was  used  to  update  tne  components  of  satisfaction  as  necessary.  Tlie  observe/interview  process  was  repeated 
for  each  module  and  the  descriptions  of  user  satisfaction  and  learning  culture  were  refined  at  each  iteration.  The  attitude 
questionnaire  was  rerun  at  the  end  of  the  study. 

Results. 

The  attitude  questionnaire  consisted  of  six  questions  that  were  measured  on  a Likert  scale  of  strongly  agree  to  strongly 
disagree.  At  pre-test,  the  participants  answers  to  the  questions  were  that  they:  enjoy  using  computers  to  learn,  think  that  it  is 
important  to  learn  how  to  use  computers,  look  forward  to  using  computers  to  learn,  think  it  is  easier  to  learn  using  computers, 
think  that  computer  use  makes  for  faster  learning  and  that  computers  provide  a better  way  to  learn.  Cochrane's  test  was  used  to 
detect  attitude  shifts  from  pre  to  posttest . Significant  changes  were  detected  for  questions  1 (p  = 0.044),  4 (p  = 0.025),  and  5 (p  = 
0.001).  The  change  for  question  1 was  positive;  the  changes  for  questions  4 and  5 were  negative.  Despite  these  changes  alone  the 
Likert  scale  the  participants'  overall  attitude  (as  described  by  the  above  responses)  did  not  change  over  the  course  of  the  study. 

Three  sets  of  t-tests  were  run  on  the  remaining  data.  Set  one  compared  the  mean  responses  of  the  socio- demographic 
groupings  at  pre-test.  It  indicated  that  females  thought  mat  learning  to  use  the  computer  was  more  important  than  males  (a|  = 

1.26,  X2  = 1.11,  p = 0.02).  No  significant  differences  were  detected  between  the  remaining  socio-demograpliic  indices.  In  set  two, 

pre-test  scores  were  subtracted  from  post-test  scores  to  determine  which  modules  successfully  taught  what  they  were  designed  to 
teach.  Modules  1, 3 and  4 produced  significantly  different  pre-  and  post-test  scores,  (module  1 (X^  = 20.02,  X2  = 65.14,  p = 0.00), 

module  3 (X^  = 20.02,  X2  = 27.31,  p = 0.00),  and  module  4 (X^  = 32.00,  X2  = 39.56,  p = 0.01)).  It  is  not  known  if  modules  2 and  5 

were  inappropriately  designed  or  whether  the  material  they  covered  is  unsuitable  to  computerised  instruction.  The  third  set  of  t- 
tests  examined  the  influence  that  socio-demographics  have  on  learning  outcomes.  No  significant  differences  were  detected  for  the 
socio-demographic  groups. 

The  iterative  ooservation  process  identified  six  components  of  user  satisfaction:  learning,  teacher  role,  tedium,  real  world 
applications,  software,  and  gender  differences.  In  regard  to  learning,  the  students  reported  that  the  computerised  instruction 
simulated  one-on-one  student-teacher  interactions,  inat  is  they  felt  that  the  feedback  provided  by  the  software  enabled  (to  some 
extent)  self  paced  learning.  Teacher  role  was  seen  as  a pivotal  component  of  a successful  computer  course.  A teacher  who  is 
interactive  and  approachable  is  needed  to  alleviate  the  students'  initial  anxiety  toward  computerised  instruction  and  to  provide 
continuing  support  when  problems  arise.  The  above  qualities  develop  a spirit  of  collaboration  in  the  classroom.  Although  the 
students  reported  that  they  enjoyed  using  the  software  they  also  felt  that  it  would  be  tedious  to  take  an  entire  course  on  a 
computer.  The  fourth  component,  "real-world  ^plication"  is  an  inherent  plus  for  computerised  instruction.  Computer  skills  were 
seen  as  a ticket  to  a better  job  and  a better  life.  Tne  software  component  indicated  that  the  skills  of  the  learner  have  to  be  accounted 
for  during  the  design  of  the  software.  At  the  start  of  the  study  students  with  very  little  computer  experience  required  help  from  the 
teacher.  Had  the  disparities  between  their  skills  and  the  software's  operational  demands  been  much  greater  they  might  have 
proved  a barrier  to  learning.  More  experience  computer  users  reported  that  they  would  have  liked  a more  interactive  multimedia 
format.  Designers  can  avoid  these  problems  by  matching  software  to  the  skills  of  the  learner.  In  regard  to  gender  differences,  a few 
of  the  male  participants  expressed  an  interest  in  a home-based  computer  course.  Females  indicatea  that  there  were  too  many 
distractions  at  home  for  them  to  learn  effectively  there. 
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A friendly,  interactive  teacher  who  encourages  camaraderie  and  a fast  pace  of  learning  was  deemed  essential 
to  the  development  of  a positive  culture  of  learning.  Such  a teacher  promotes  a sense  of  "we  are  all  in  this  together."  The  small  class 
size  (13  computers)  with  students  sitting  close  together  was  also  thought  to  influence  the  creation  of  a collaborative  atmosphere. 
The  observer  postulated  that  a positive  class  culture  engenders  faster  learning  as  strong  participants  can  provide  timely  assistance 
to  the  weak.  V^en  students  interacted,  they  interacted  in  small  groups  of  two  or  three.  Operational  problems  of  the  software  and 
the  computer  were  discussed.  By  week  three  participants  had  become  familiar  with  the  software  ana  their  interactions  dropped  off 
considerably.  From  this  time  on  data  was  primarily  obtained  from  the  focus  group  interviews 

Conclusions 

Iterative  observation  affords  repeated  runs  at  a question  so  that  it  can  be  explored  in  considerable  detail.  Multiple 
assessments  mean  that  inaccurate  conclusions  drawn  from  initial  observations  may  be  removed.  Iterative  observation  also  affords 
the  reallocation  of  resources  to  accommodate  factors  that  are  only  discovered  part- way  through  a series  of  observations.  Evaluators 
wishing  to  use  this  technimje  should  expect  that  changes  will  have  to  be  made  to  their  initial  evaluation  strategy.  The  ability  to 
embrace  these  changes  will  determine  the  success  of  the  investigation.  The  participants  change  in  attitude  toward  computer 
learning  may  be  a result  of  the  hyperbole  associated  with  the  computer  industry.  If  participants  entered  the  study  thinking  that 
computers  make  for  effortless,  extremely  rapid  learning  they  would  have  been  disappointed.  Computerised  instruction  is  not 
without  problems,  it  works  best  when  used  in  conjunction  with  a friendly  interactive  teacher  who  instills  a spirit  of  collaboration  in 
the  classroom. 
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Abstract:  This  paper  provides  the  experiences  of  two  faculty  members  learning  technology 
and  how  to  integrate  it.  Both  members  have  received  grants  to  revise  curriculum  to 
integrate  technology.  Based  on  their  experiences  and  the  awarding  of  a technology 
challenge  grant,  faculty  members  are  learning  to  choose  their  tools  wisely. 


Introduction 

Given  the  current  climate  of  technology  integration  in  education,  our  institution  has  begun  to 
encourage  faculty  to  use  technology  within  their  classrooms.  The  mandate  is  clear  from  the  governor  on 
down  to  the  college  levels  that  technology  will  be  used.  Thus,  there  has  been  an  increase  in  the  number  of 
faculty  trying  to  use  technology  in  their  teaching.  The  real  issue,  however,  is  not  using  technology,  but 
rather  using  it  wisely. 

The  experience  of  a lifetime 

Both  authors  have  been  recipients  of  summer  grants  awarded  within  the  state’s  higher  education 
system.  Because  of  these  awards  both  authors  revised  existing  courses  with  the  goal  of  increasing  the 
technological  aspects  of  the  courses.  These  courses  were  requirements  in  the  undergraduate  teacher 
education  program.  The  purpose  of  the  awards  was  to  increase  technology  skills  and  knowledge  by  both 
education  faculty  and  students.  The  grant  covered  salary  and  provided  each  instructor  with  $5,000  to  spend 
for  equipment  and  training  during  the  summer. 

In  each  case  the  authors  begin  the  process  by  investigating  equipment  they  could  purchase.  This 
was  welcomed  because  the  existing  technology  had  not  been  recently  updated  and  therefore  many  of  the 
project  goals  were  not  possible.  One  of  the  barriers  encountered  was  the  system  for  ordering  equipment 
and  software.  The  process  was  a complex  process  in  which  faculty  took  weeks  of  their  project  time  to 
obtain  bids  and  order  the  equipment  they  need  to  accomplish  their  goals. 

Another  issue  that  arose  was  support  in  terms  of  training  on  equipment  use  and  programs.  Faculty 
receiving  the  grants  had  to  actively  seek  campus  resources  to  order  equipment,  gain  training  in  computer 
skills  including  WebPage  design,  course  software  and  presentation  media.  Some  training  was  offered 
through  the  Instructional  Technology  Center  on  campus,  however,  not  all  participants  attended.  Once 
discovered,  the  support  persons  were  available  to  the  awardees  and  provided  guidance. 

During  this  process  the  authors  found  that  the  grant  was  wonderful  for  receiving  new  equipment 
and  learning  skills.  However,  the  actual  course  design  and  integration  was  an  evolving  process.  In  fact,  the 
authors  have  found  the  longer  they  worked  on  their  grant  projects,  the  more  they  realized  that  technology 
integration  is  an  ongoing  process.  Even  though  the  spirit  of  the  grant  proposal  was  accomplished,  some  of 
the  faculty  found  that  more  time,  resources,  and  thought  were  needed  to  go  into  their  projects.  Thus  the 
authors  are  making  significant  revisions  to  their  courses  as  they  realize  that  using  technology  to  facilitate 
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instruction  is  a complex  process.  The  time  it  takes  to  implement  technology  that  is  truly  useful  to  our 
students’  learning  is,  at  the  outset,  tremendous. 

Both  authors  decided  to  use  web-based  materials  to  support  instruction  for  their  grant  projects. 
Besides  the  grant  projects,  the  authors  have  infused  technology  into  other  courses  they  teach.  The  goal  in 
using  web-based  materials  was  not  to  provide  the  lecture  notes  for  their  classes  but  rather  to  extend  learning 
opportunities  within  their  classroom.  This  meant  increasing  access  and  interactivity  between  instructor  and 
student  as  well  as  to  student  to  student  collaboration. 

In  one  project  the  author  used  WebPage  templates  designed  by  the  university’s  instructional 
designer.  The  templates  allowed  the  faculty  member  to  put  materials  such  as  an  online  syllabus,  course 
schedule,  assignments  and  discussion  on  the  web.  As  courseware  became  available  during  the  following 
year  the  information  was  migrated  to  the  new  courseware  application  which  provided  password  protection 
for  the  course  materials  and  interactions.  Another  goal  in  this  project  was  to  increase  the  student’s  use  of 
technology  and  was  accomplished  by  specific  training  and  required  projects  such  as  use  of  the  digital 
camera,  scanners,  and  PowerPoint. 

In  the  other  project,  the  author  took  existing  materials  developed  for  web  publication  and  updated 
the  materials  as  well  as  adding  components  such  as  discussion,  online  surveying  and  project  development. 
Courseware  provided  by  the  campus  was  used  at  first  to  revise  the  existing  course.  This  courseware  was 
not  sufficient  for  the  project’s  purpose  so  the  author  used  grant  time  to  find  other  courseware  options. 

As  the  awardees  learned  skills  and  extended  their  ideas,  they  became  aware  of  the  barriers  in 
integrating  technology  such  as  problems  with  infrastructure,  the  selection  of  the  web-based  courseware  and 
the  ability  to  locate  resources  to  facilitate  web-supported  instruction.  The  other  concern  was  that  while 
increasing  skills  in  technology,  faculty  members  were  not  being  trained  as  to  how  courses  including 
technology  should  be  redesigned.  Technology  integration  requires  that  faculty  rethink  how  the  education 
process  really  happens. 

As  this  realization  hit,  the  college  began  participating  in  a technology  grant.  Through  the  five- 
year  statewide  technology  challenge  grant  entitled  Learning  Organizations  for  Technology  Integration 
(LOFTI),  faculty  members  in  the  college  have  begun  to  receive  both  technical  instruction  and  instructional 
design  instruction.  The  authors  feel  that  the  focus  should  be  on  teaching  our  content  using  technology  to 
enhance  the  learning  process.  Technology  is  a media  decision,  which  should  be  dependent  on  the  faculty’s 
need  to  communicate  with  students,  the  content,  and  the  educational  environment. 

In  the  first  year  of  the  LOFTI  grant,  planning  was  done  by  a group  of  higher  education  faculty, 
students  and  practicing  K-12  teachers.  This  group’s  first  activity  was  an  intensive  week  of  skills-building 
workshop  for  faculty  members.  This  workshop  was  planned  to  concentrate  on  skills  identified  through  a 
needs  assessment  and  included  e-mail,  building  internet  activities,  basic  word  processing,  spreadsheets, 
databases,  ethics  and  standards,  and  WebPages  development.  Faculty  members  chose  the  sessions  they 
wanted  to  attend  and  were  paid  a stipend  based  on  the  number  of  workshops  they  participated  in.  Also  in 
the  first  year,  a request  for  proposals  went  out  among  faculty  calling  for  summer  projects  that  revised 
curriculum  to  include  technology  integration  for  undergraduate  teacher  education  students.  These  projects 
were  intense  in  nature  and,  while  not  providing  the  monetary  value  of  the  previous  statewide  grant 
program,  allowed  faculty  members  to  concentrate  on  a specific  course  revision. 

Currently  in  the  second  year  of  LOFTI,  the  planning  group  has  determined  that  the  focus  of  the 
grant  should  be  staff  development  in  technology  integration,  recognizing  that  skills  training  still  need  to  be 
offered.  One  of  the  projects  is  conducting  college  seminars  on  using  instructional  design  principles  to 
integrate  technology  within  existing  courses.  The  seminars  allow  faculty  an  opportunity  to  discuss, 
brainstorm  and  educate  themselves  about  the  concept  of  using  technology  as  a tool. 

Conclusion 

Through  our  experiences  with  the  statewide  grant  and  the  opportunities  presented  by  the  LOFTI 
grant,  we  hope  to  encourage  appropriate  use  of  technology  to  facilitate  learning.  We  have  discovered  that 
time  and  resources  are  vital  to  the  success  of  technology  integration,  however,  these  are  not  the  only 
variables  at  play.  Good  instructional  design  principles  and  support  and  encouragement  by  fellow  faculty 
members  are  necessary  factors  in  developing  curriculum,  which  uses  technology  that  enhances  the  learning 
process.  Ultimately,  learning  to  integrate  technology  that  strengthens  our  undergraduate  teacher  education 
students  experience  will  provide  a model  for  them  to  take  into  their  classrooms  of  the  future. 
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Abstract:  In  this  session,  we  provide  a brief  guide  for  aspiring  authors  in  the  field  of 

educational  technology.  We  concentrate  on  journal  articles  rather  than  on  books  or 
electronic  publishing.  After  looking  briefly  at  some  of  the  reasons  for  publishing,  we  discuss 
some  of  the  most  popular  types  of  articles  currently  appearing  in  the  literature  and  review 
educational  technology  journals.  Hints  for  creating  publishable  articles  are  also  provided. 


Many  young  professionals  in  the  field  of  Educational  Technology  view  the  process  of  getting  published  as 
one  involving  mystical  wisdom.  Some  find  the  prospect  of  rejection  intimidating  and  others  just  don’t 
know  how  to  get  started.  While  successful  scholars  in  the  field  are  sometimes  characterized  by  an 
extraordinary  intellect,  most  are  primarily  distinguished  by  a willingness  to  be  systematic  and  persistent. 

Why  Publish? 

Although  the  expression  “publish  or  perish”  has  now  become  a cliche  in  academic  circles,  the  importance 
of  its  message  to  faculty  who  desire  to  advance  in  higher  education  remains  as  strong  as  ever.  For  graduate 
students  and  recent  graduates  of  educational  technology  programs  desiring  to  obtain  an  academic  position, 
publications  can  provide  that  extra  edge  that  sets  their  credentials  apart  from  those  of  other  candidates.  But 
even  for  those  whose  career  success  does  not  depend  on  publishing,  there  are  considerable  personal 
rewards  of  seeing  one’s  work  appear  in  print. 

Publication  Outlets 

There  are  many  journals  that  publish  articles  on  educational  technology.  In  addition  to  the  many  journals 
that  deal  directly  with  the  field,  there  are  dozens  of  others  that  publish  articles  in  which  an  application  of 
educational  technology  in  a particular  content  area  is  the  major  theme.  Because  we  can’t  review  every 
educational  journal,  we  have  chosen  to  concentrate  here  on  those  journals  that  deal  specifically  with 
educational  technology. 

A review  of  Cabell’s  Directory  of  Publishing  Opportunities  in  Education,  5'^  edition,  1998  identified  37 
journals  that  deal  primarily  with  educational  computing.  Educational  design,  distance  education,  or  other 
aspects  of  educational  technology.  Seventeen  of  these  are  listed  in  Table  1 showing  circulation  rate, 
acceptance  rate,  the  time  required  for  review  of  manuscripts,  acceptable  manuscript  length,  and  the  email 
address  of  the  publisher.  Information  was  not  reported  for  the  other  20  journals,  so  they  have  not  been 
included.  Readers  should  note  that  the  information  included  here  is  self-reported  and  subject  to  change. 
This  list  is  not  meant  as  a comprehensive  list  but  merely  suggests  the  broad  scope  of  possibilities  for 
publication  in  educational  technology. 


Table  1:  Educational  Technology  Journals 


Title  of  Journal 

Circulation 

Acceptance 

Months  to 

Length 

Email  or  URL 

Rate 

Review 

1482 


Behavior  Research  Methods, 
Instruments,  and  Computers 

2001-3000 

46% 

2-3 

any 

proctor@psych.purdue.edu 

British  Journal  of 
Educational  Technology 

1,300 

4,000 

words 

www.blackwell.com 

Education  Technology 
Research  and  Development 

4001-5000 

11-20% 

1-2 

26-30 

ross.steve@coe.memphis.ed 

u 

Educational  Technology 

3001-4000 

21-30% 

<=1 

any 

edtechpubs@aol.com 

Educational  Technology 
Systems 

1001-2000 

5% 

4-6 

16-20 

TLIAO@ccmail.sunysb.edu 

Information  Technology  & 
Libraries 

5001-10,000 

65% 

2-3 

>30 

dan.marmion@wmich.edu 

International  Journal  of 

Educational 

Telecommunications 

<1000 

NA 

4-6 

35 

pubs@aace.org 

International  Journal  of 
Instructional  Media 

2001-3000 

11-20% 

2-3 

11-25 

NA 

Journal  of  Educational 
Computing  Research 

<1000 

11-20% 

2-3 

26-30 

rseidman@minerva.nhc.edu 

Journal  of  Educational 
Multimedia  and  Hypermedia 

1001-2000 

21-30% 

4-6 

11-30 

pubs@aace.org 

Journal  of  Research  on 
Computing  in  Education 

2001-3000 

50% 

4-6 

10-20 

Dmcgrath®ksu.edu 

Journal  of  Technology  and 
Teacher  Education 

<1000 

21-30% 

4-6 

11-30 

www.aace.ore 

Journal  of  Technology 
Studies 

5001-10,000 

70% 

2-3 

15-25 

Jstreich@bgnet.bgsu.edu 

Learning  and  Leading  with 
Technology(Formerly:  The 
Computing  Teacher) 

10,001- 

25,000 

50% 

4-6 

3-8 

anita_best@ccmail.uoregon 

.edu 

Multimedia  Schools 

10,001- 

25,000 

NA 

4-6 

2400 

words 

veccia®Avell.com 

T.  H.  E.  Journal: 
Technological  Horizons  in 
Education 

>25000 

11-20% 

1-2 

6-10 

www.theJoumal.com 

TechTrends 

5,001-10,000 

40% 

2-3 

6-16 

NA 

It  is  worth  noting  that  the  same  manuscript  must  not  be  submitted  to  more  than  1 journal  at  the  same  time 
(American  Psychological  Association,  1994).  However,  it  is  generally  accepted  as  ethical  practice  to 
present  a paper  at  a conference  and  then  submit  the  paper  or  a revision  of  the  paper  for  Journal  publication. 

Final  Thoughts 

In  conclusion,  learning  to  write  for  publication  can  be  an  exciting  experience,  help  enhance  your  career, 
and  earn  you  the  respect  of  colleagues.  Although  at  times  publishing  may  be  humbling,  most  aspiring 
authors  become  increasing  successful  with  experience. 
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Introduction 

Web-based  publishing  presents  editors  and  graphic  designers  with  novel  problems.  Can  a hypertext 
presentation  deliver  a coherent  narrative?  What  written  style  suits  screen  presentation?  How  can  professional 
standards  of  graphic  and  typographical  design  be  rendered  successfully  in  HTML?  This  paper  suggests  some 
answers  to  these  questions  using  experience  drawn  from  the  presentation  of  a large-scale  web  course  in  the  U.K. 

The  Course 

In  1999  a web-based  course  entitled,  T171:  You,  Your  Computer  and  the  Net  was  presented  to  900  U.K. 
and  European  students  (Weller,  1 999).  This  was  a pilot  presentation,  and  now  in  2000  the  course  is  being  studied 
by  12,000+  students  in  its  first  full-scale  presentation. 

Editorial  Issues 

Publishing  via  the  web  poses  problems  of  narrative  design  and  written  style  that  arise  from  the 
disjointed  nature  of  hypertext  (Plowman,  1998)  and  the  well-documented  difficulties  of  reading  from  screen 
(Jones  et  al.,  1998). 

The  problems  of  constructing  coherent  narrative  in  disjointed  media  are  often  described  in  terms  of  the 
search  for  nonlinear  narrative,  as  if  nonlinear  narratives  are  a new  phenomenon  of  the  digital  age.  Umberto  Eco 
has  pointed  out  (Eco,  1993)  that  in  almost  all  literary  fiction  nonlinear  narratives  are  the  norm. 

Students  of  film  studies  are  similarly  familiar  with  the  scope  for  nonlinear  narrative  in  an  otherwise 
linear  medium  (Chatman,  1989). 

Eco  makes  an  important  distinction  between  narrative  and  exposition.  In  a book  or  film  the  exposition  is 
linear,  but  the  narrative  need  not  be. 

The  issue  with  interactive  digital  media  is  not  so  much  ‘how  can  we  develop  nonlinear  narratives?’,  but 
rather  ‘how  can  we  present  coherent  narratives  in  a medium  in  which  the  exposition  is  both  nonlinear  and  under 
the  reader’s  and  not  the  author’s  control?’ 

The  answer  lies  in  the  scope  web  constructors  have  for  shaping  the  exposition.  In  our  course  we 
exploited  the  advantages  of  the  web  for  both  guided  and  free  exploration,  while  tying  our  students  to  a core 
linear  flow  of  information. 

This  structure  can  be  summarised  as  the  ‘bottlebrush’  narrative  model. 

Reading  from  screen  is  neither  as  comfortable  or  as  efficient  as  reading  from  paper  (Jones  et  al.,  1998). 
There  is  thus  an  advantage  to  writing  in  a concise  style  (Morkes  and  Neilsen,  1997,1998).  In  our  course  the  web 
site  provides  ‘wrap-around’  materials  to  support  two  set  books  (Cringely,  1996;  Hafner  and  Lyon,  1998).  The 
books  in  themselves  carried  strong  narratives,  and  provided  detailed,  discursive  texts.  So  the  web  site  could  be 
presented  in  a more  condensed  style,  without  risking  it  being  either  too  obscure  or  too  light-weight. 

Brevity  leads  to  the  second  style  question:  where  does  one  strike  the  balance  between  asking  your 
reader  to  scroll  through  a long  page,  and  asking  them  to  jump  to  a new  one?  We  suggest  a rough  rule-of-thumb: 
don’t  require  your  readers  to  scroll  for  more  than  about  five  screen  depths,  and  design  your  pages  to  download  in 
a reasonable  time. 
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The  graphic  designers  were  given  the  brief  to  produce  a stylish,  restrained  and  simple  design  that  would 
be  stable  when  viewed  with  any  one  of  the  commonly  available  browsers. 

The  site  was  rendered  and  managed  using  UserLand’s  Frontier.  The  graphic  designers  produced  design 
templates  in  HTML  using  DreamWeaver,  and  the  software  support  team  wrote  scripted  macros  that  Frontier  used 
to  render  tagged  HTML  files  into  finished  designed  pages.  The  generation  of  final  pages  was  thus  automated. 

The  first  major  design  decision  was  whether  to  use  frames  or  not.  The  arguments  against  frames 
included:  the  perceived  difficulty  of  bookmarking  framed  pages,  the  relatively  low  browser  specification  that  we 
had  to  accommodate,  and  the  size  and  complexity  of  the  site.  The  arguments  for  frames  included:  that  the  site 
would  look  more  sophisticated  (for  example,  menu  lists  would  not  scroll  off  the  screen),  and  that  navigation 
would  be  simpler  in  a frames  environment.  The  final  decision,  however,  was  not  to  use  frames. 

We  used  tables  to  implement  the  design.  Tables  allow  for  quite  sophisticated  layouts,  and  it  was  not 
easy  to  decide  how  far  to  go  in  exploiting  the  potential  of  tables  to  deliver  fine-tuned  design.  Compared  to  many 
commercial  sites  the  T171  pages  show  a relatively  simple  design,  but  it  should  have  been  simpler.  A balance  had 
to  be  struck  between  the  achievement  of  particular  design  effects,  and  the  time  expended  to  make  them  happen. 
For  this  balance  to  be  struck  both  graphic  and  software  specialists  had  to  understand  each  other’s  different 
perspectives. 

Seigel  (1997)  recommends  that  an  average  web  page  should  be  around  30k  in  size.  Sometimes  this  can 
rise  to  70k,  but  should  never  exceed  100k.  As  modem  speeds  improve  these  figures  may  need  to  be  revised 
upwards,  but  the  T171  design  took  them  as  benchmarks.  This  led  to  difficult  decisions  about  how  far  to  reduce 
the  visual  quality  of  illustrations  in  order  to  meet  the  benchmark  file  sizes.  A lot  of  the  images  that  provided  the 
basic  design  were  used  many  times,  and  so  we  could  rely  on  them  being  cached  by  the  students’  browsers. 

Conclusion 

The  web  is  a publishing  medium,  and  web-based  learning  materials  can  benefit  from  support  from 
publishing  and  design  professionals.  In  supporting  web-based  educational  publishing  both  editors  and  designers 
have  to  apply  skills  developed  in  the  world  of  print  publishing  to  a dynamic  and  potentially  nonlinear  medium. 

In  developing  coherent  narrative  structures  it  is  useful  to  recognise  the  distinction  between  narrative  and 
exposition,  and  to  employ  narrative  models  that  allow  for  exploration  and  discoveiy  while  maintaining  an 
underlying  narrative  drive.  The  well-documented  problems  associated  with  reading  from  screen  also  demand 
conciseness. 

Good  visual  design  is  important  in  enabling  ease  of  use.  Close  teamworking  between  graphic  designers 
and  software  support  staff  is  essential,  and  both  categories  of  staff  need  a good  understanding  of  the  nature  of 
each  other’s  task,  and  the  constraints  under  which  each  operates. 

Design  decisions  should  be  informed  by  considerations  of  orientation,  navigation,  complexity  versus 
practicality,  and  download  times. 
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Abstract:  This  paper  reviews  the  design,  implementation,  and  testing  of  Project  Links'  WWW-  based 
modules  developed  at  Rensselaer  Polytechnic  Institute  (RPl)  to  support  the  integration  of  mathematics, 
science  and  engineering  concepts  in  undergraduate  teaching  and  teaming.  Using  the  Project  Links 
experience  as  an  example,  the  authors  argue  for  particular  foci,  as  opposed  to  generic  approaches,  and 
formative,  as  opposed  to  summative,  testing  in  the  evaluation  of  technology-based  programs  and  the 
usability  of  particular  software. 


Perspectives 

Project  Links  is  part  of  the  Mathematics  and  Its  Application  Across  the  Curriculum  initiative  funded  by  the  NS F 
and  aimed  at  reforming  undergraduate  mathematics,  science,  and  engineering  education  by  making  clear  the 
connections  between  mathematics  and  its  applications  in  science  and  engineering.  Project  Links  centers  on  the 
development  of  a library  of  WWW-based  hypermedia  modules  that  link  important  topics  in  mathematics  with 
useful  applications  in  engineering  and  science.  Modules  in  the  library  cover  the  mathematical  areas  traditionally 
studied  by  undergraduate  engineering  and  science  majors,  including  calculus,  differential  equations,  mechanics 
and  linear  systems,  advanced  mathematical  methods,  and  probability  and  statistics. 

In  the  Project  Links  modules,  these  ideas  are  both  explored  as  mathematical  concepts  and  contextualized  within 
applied  science  and/or  engineering  situations.  Each  module  consists  of  several  sections  that  are  designed  to  be 
completed  in  a single  class  session  or  in  part  of  a single  class  session.  An  important  feature  of  most  modules  is 
interactive  simulations  designed  to  help  learners  visualize  the  mathematical,  engineering,  and/or  scientific 
concepts  being  illustrated.  Although  they  are  made  so  that  they  can  be  used  in  a variety  of  instructional  settings, 
Project  Links  modules  are  designed  to  support  an  innovative  model  of  undergraduate  education  pioneered  at 
RPl  known  as  studio  teaching  and  learning  (Wilson,  1994).  In  studio  style  teaching  and  learning,  large  lecture 
classes  are  replaced  by  many  smaller,  technology-supported  studio  classes  that  integrate  lecture,  discussion,  and 
laboratory-type  explorations,  many  of  which  are  computer-based  or  computer-supported  (Boyce  & Ecker, 
1994). 

Methodology 

The  authors  reviewed  Project  Links  documents  and  worked  with  the  Links  Project  Directors  to  isolate  a set  of 
defining  factors  that  the  latter  believed  best  encapsulated  what  was  most  important  in  the  project.  These  were: 
the  explicit  linking  of  concepts,  support  for  studio-type  teaching  and  learning,  flexibility  (the  potential  to  be 
used  in  other  sorts  of  teaching  and  learning  situations),  support  for  collaborative  learning,  discovery  orientation, 
and  multi-  modality  (the  use  of  multiple  kinds  of  representations  for  single  concepts).  These  factors  then 
became  the  basis  for  the  formative  evaluation  of  Project  Links  modules.  This  evaluation  is  ongoing  and  has 
taken  multiple  approaches. 

Firstly,  a software  evaluation  protocol  was  designed  to  explore  whether  or  not  the  six  factors  deemed  most 
important  to  Project  Links  were  explicitly  instantiated  in  individual  modules.  Evaluators  were  asked  to  answer  a 
series  of  questions  in  a narrative  format  for  each  of  these  factors,  as  well  as  to  rank  the  modules  using  a scale  of 
one  to  seven.  Two  evaluators  independently  completed  the  software  evaluations  for  each  module,  then  came 
together  to  resolve  any  differences  through  consensus. 


ERIC 


Secondly,  we  developed  a version  of  think  aloud  protocols  (Erickson  & Simon,  1993;  Pressley  & Afferbach, 
1995)  to  explore  the  ways  in  which  students  make  sense  of  the  modules.  Representative  students  were  asked  to 
work  on  particular  modules,  "thinking  aloud"  as  they  worked.  As  they  did  so,  both  their  faces  and  the  computer 
screen  in  which  they  were  working  were  captured  on  a single  videotape  using  a video  mixer  and  picture-in- 
picture technique.  Users  were  also  asked  to  tell  what  they  remembered  from  the  modules  (retrospective  report) 
and  to  answer  specific  questions  concerning  their  behaviors  while  viewing  sections  of  the  tapes  (stimulated 
recall).  Tapes  of  all  three  were  reviewed  for  evidence  of  sense  making,  reflection,  and  usability  issues. 

Finally,  studio  classes  in  which  modules  were  used  were  observed  to  explore  their  actual  use  in  such  setting,  and 
students  within  those  classes  were  interviewed  to  gain  more  detailed  information  concerning  the  software. 


The  software  evaluation  review  protocol  proved  to  be  invaluable  in  the  revision  process  during  the  ongoing 
development  of  Project  Links  modules,  especially  regarding  the  formatting  of  information,  the  design  of 
multimedia  simulations,  and  the  overall  design  of  the  modules.  In  fact,  a template  was  created  to  ensure 
consistency  of  navigation  and  design  as  a result  of  the  review  process. 

In  addition,  the  think  aloud  protocols  demonstrated  that  the  Project  Links  modules  could  be  used  by  four 
categories  of  students  - content  area  novices  and  experts,  and  technology  novices  and  experts.  Usability  testing 
is  ongoing  to  try  and  determine  students'  cognitive  processes  while  interacting  with  the  modules.  Classroom 
observations  have  indicated  that  there  is  still  a need  for  professional  development  concerning  classroom  use  of 
the  Project  Links  software.  In  particular,  cooperative  use  of  the  modules  is  rare.  An  initial  professional 
development  day  was  completed  in  the  spring  and  more  are  planned  for  the  1 999/2000  school  year. 

Educational  Significance: 

Project  Links  is  an  innovative  program  that  is  exploring  new  ways  of  teaching  and  learning  math,  science,  and 
engineering  through  technology  at  the  undergraduate  level  that  integrate  concepts  across  these  disciplines  and 
help  learners  visualize  difficult  underlying  concepts.  This  study  helps  highlight  the  software  features  and 
teaching  practices  that  make  it  successful. 

Additionally,  this  study  demonstrates  the  efficacy  of  contextualizing  software  evaluation  according  to  the 
particular  purposes  for  which  software  is  created.  Such  an  approach,  we  believe,  is  beneficial  for  two  reasons. 
First,  it  only  makes  sense  to  evaluate  educational  software  relative  to  pedagogical  purposes  because  differing 
purposes  are  best  served  in  vastly  different  ways.  Programmed  instruction,  or  objectives  based  programs,  for 
example,  is  well  suited  to  behaviorist  and  perhaps  information  processing  pedagogical  approaches,  but  ill-suited 
indeed  for  constructivist  ones.  Secondly,  assessment  always  guides  practice.  Teachers  teach  to  the  tests  just  as 
students  study  for  them.  Similarly,  software  designers  will  write  and  revise  based  on  assessment  procedures. 
Evaluation  protocols  that  focus  on  the  issues  important  to  a particular  project  or  pedagogical  need  will  help 
insure  that  those  issues  are  held  central. 
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How  to  Make  the  Most  of  Online  Interaction 


Introduction 

The  educational  community  is  finding  itself  on  the  edge  of  a new  era,  online  learning.  Online  learning  has  been  shown  to  be  more  cost- 
effective  and  convenient  than  traditional  educational  environments  as  well  as  providing  opportunity  for  more  learners  to  continue  their  education. 

However,  some  critics  claim  that  web-based  or  online  learning  is  not  as  effective  as  traditional  classroom  learning  experiences  (see  Freeman  and  Capper, 
1999;  Noble,  1996;  Perelman,  1992;  Cuban,  1986).  When  queried  as  to  the  difference  in  the  learning  experiences  across  environmental  format,  many  cite 
the  lack  of  face-to-face  interactions  as  being  the  foremost  deficiency.  However,  when  considering  this  challenge,  researchers  have  to  ask  themselves  if  it  is 
really  the  act  of  viewing,  the  instructor/students,  that  is  an  essential  element  of  learning,  and  if  not,  then  what  element  are  these  critics  referring  to? 

Perhaps  it  is  the  interactions  that  take  place  between  the  student  and  instructor,  but  interactions  can  take  place  in  an  online  environment  as  well  as  in  a 
traditional  classroom.  This  being  the  case,  we  argue  that  the  element  that  critics  are  referring  to  is  that  of  “social  presence.” 

Social  presence  theory,  a sub-area  of  communication  theory,  postulates  that  the  critical  factor  in  a communication  medium  is  its  “social 
presence,”  which  is  defined  as  the  “degree  of  salience  of  the  other  person  in  the  interaction  and  the  consequent  salience  of  the  interpersonal  relationships” 
(Short,  Williams,  and  Christie;  1976,  p.  65).  This  is  interpreted  as  the  degree  to  which  a person  is  perceived  as  a “real  person”  in  mediated  communication. 
Short,  et  al.  further  define  social  presence  as  a quality  of  the  medium  itself  and  hypothesized  that  communication  media  vary  in  their  degree  of  social 
presence.  Social  presence  is  described  as  a construct  that  comprises  a number  of  dimensions  relating  to  the  degree  of  interpersonal  contact  including  the 
concepts  of  “intimacy”  and  “immediacy.”  Thus,  it  can  be  argued  that  social  presence  is  both  a factor  of  the  medium  as  well  as  of  the  communicators  and 
their  presence  in  a sequence  of  interactions  (Gunawardena  and  Zittle;  1997). 

Review  of  the  Literature 

In  the  era  of  online  and  web-based  learning  and  course  offerings,  there  is  an  increasing  need  to  understand  social  and  political  contexts  of 
learning  that  are  created  by  this  type  of  learning  environment.  The  significance  can  be  demonstrated  by  the  growth  of  available  online  courses.  For 
example,  The  SUNY  Learning  Network  (SLN),  the  most  comprehensive  online  course  network  in  the  country,  has  increased  the  number  of  courses  they 
offer  from  4 in  the  1995-1996  academic  year  to  over  1,000  courses  in  the  1 999-2000  academic  year,  a growth  of  more  than  250%  in  less  than  five  years 
(Swan,  Shea,  Fredericksen,  Pickett,  Pelz,  and  Maher,  in  press).  Recent  studies  conducted  by  Jones  (1998),  Baym  (1 998),  Agre  (1998),  Gunawardena 
(1997),  Haraism  (1993),  Warschauer  ( 1 999),  and  Rheingold  (1993)  have  emphasized  the  need  for  further  research  into  the  social  context  of  web-based 
environments  as  it  impacts  learning  and  communication.  To  date  there  has  not  been  much  research  done  on  Social  Presence  Theory  and  computer- 
mediated  communications.  A review  of  the  literature  produced  only  four  articles  pertaining  to  this  area. 

Although  Hackman  and  Zane’s  (1990)  work  examines  a televised  classroom  environment,  it  examines  issues  in  common  with  online  learning 
and  social  presence.  Their  study  investigated  the  effects  of  conveyance  systems  design  and  social  presence,  in  the  form  of  teacher  immediacy  behavior,  on 
perceived  student  learning  and  satisfaction.  The  results  indicate  that  system  design  and  teacher  immediacy  behavior  strongly  impact  student  learning  and 
satisfaction.  In  addition,  system  variables  such  as  interactivity  (e.g.  students  are  able  to  comment  during  lectures)  and  clear  audio  and  video  transmission 
positively  influenced  perceived  learning  and  satisfaction.  Moreover,  it  was  found  that  instructors  that  engaged  in  immediate  behaviors  such  as  encouraging 
involvement,  offering  individual  feedback,  and  promoting  a relationship  with  off-campus  students  were  viewed  more  favorably  (p.  203). 

Demonstrating  the  shift  in  social  presence  theory,  Gunawardena  (1995)  conducted  a two-part  study  examining  whether  social  presence  was  an 
attribute  of  the  communication  medium  or  users’  perception  of  the  medium.  The  studies  were  conducted  in  the  Spring  of  1992  and  Fall  of  1993  with 
students  who  participated  in  a GlobalEd  computer  conference  course.  GlobalEd  “linked  graduate  students  in  several  universities  to  discuss  issues  related  to 
distance  education,  engage  in  collaborative  learning  and  research  related  to  distance  education,  and  experience  distance  education”  by  using  technological 
medium  conducive  to  the  deliverance  of  distance  education  (p.  149).  The  studies  focused  on  students’  perceptions  of  seventeen  five-point  bipolar  scales 
related  to  the  computer-mediated  conference  system.  The  GlobalEd  conferences  were  conducted  using  a listserv,  or  an  electronic  distribution  list.  The 
findings  from  both  of  her  studies  indicated  that  “although  computer-mediated  conferencing  (CMC)  is  considered  to  be  low  in  its  ability  to  convey  social 
presence,  participants  tin  this  conference  rated  CMC  highly  as  an  interactive,  active,  and  social  medium”  (p.  158).  It  is  the  researcher’s  belief  that  these 
findings  were  due  in  past  to  the  social  cohesion  of  the  conference  and  the  individual  input  from  each  student  that  allowed  them  to  create  an  online 
community.  In  other  words,  social  presence  wasn’t  a quality  of  the  medium,  as  Short,  Christie  and  Williams  (1972)  initially  proposed,  but  rather  a quality 
of  how  the  participants  chose  to  interact  with  the  medium.  If  the  participants  had  not  been  committed  to  creating  a cohesive  online  community  then  they 
could  have  fallen  victim  to  the  medium.  As  Gunawardena  explains,  “the  participants  create  social  presence  by  projecting  their  identities  and  building 
online  communities.  In  order  to  encourage  interaction  and  collaborative  learning,  it  is  important  that  moderators  of  computer  conferencing  promote  the 
creation  of  conducive  learning  environments.  CMC  participants  can  be  trained  to  create  social  presence  in  a text-based  medium  and  build  a sense  of 
community”  (p.  163). 

It  is  also  Gunawardena’s  belief  that  “in  reviewing  social  presence  research,  it  is  important  to  examine  whether  the  actual  characteristics  of  the 
media  are  the  causal  determinants  of  communication  differences  or  whether  users’  perceptions  of  media  alter  their  behavior.  It  was  noted  that  social 
presence  can  ‘be  cultured’  among  teleconference  participants,  a position  different  from  the  view  that  social  presence  is  largely  an  attribute  of  the 
communication  medium”  (p.  164).  This  study  is  especially  significant  because  it  demonstrates  that  Short,  Williams  and  Christie  work  has  become 
somewhat  outdated  in  terms  of  defining  social  presence.  Although  to  be  fair,  they  could  not  have  guessed  the  capabilities  of  today’s  technological 
innovations. 

In  a later  study  by  Gunawardena  (1997),  she  examined  the  possibility  of  using  social  presence  as  a predictor  of  satisfaction  within  a computer- 
mediated  conferencing  environment.  The  participants  were  50  graduate  students  from  five  universities  enrolled  in  a “GlobalEd”  conference  course  that 
provided  a forum  for  graduate  students  in  distance  education  to  share  and  discuss  research  and  experience  distance  education  by  using  computer-mediated 
communication.  The  instrumentation  consisted  of  a questionnaire  developed  to  assess  participants  responses  to  CMC,  the  GlobalEd  experience,  and  areas 
of  theoretical  interest  identified  in  the  literature  as  potential  influences  on  CMC  satisfaction;  fifty-two  five-point  Likert-scale  items  were  used  as  the 
measure  (Gunawardena,  1997,  p.  13).  The  results  demonstrated  that  approximately  58%  of  the  variance  was  contributed  by  social  presence,  suggesting  that 
social  presence  is  a strong  predictor  of  satisfaction  in  a text-based  computer  conference.  Moreover,  it  also  “supports  the  view  that  the  relational  or  social 
aspect  of  CMC  is  an  important  element  that  contributes  to  the  overall  satisfaction  of  task-oriented  or  academic  computer  conferences”  (p.  1 9).  In  addition, 
students’  perceived  enhanced  socio-emotional  experience  by  using  emoticons  (symbols  to  express  emotions)  to  express  missing  nonverbal  cues  in  written 
forni.  Overall,  the  findings  imply  the  need  for  designing  academic  computer  conferences  where  equal  attention  must  be  paid  to  designing  techniques  that 
enhance  social  presence  and  instructors  will  need  to  learn  to  adapt  to  the  CMC  medium  by  developing  skills  that  create  a sense  of  social  presence  (p.  19). 

Finally,  Boverie,  Nagel,  McGee,  and  Garcia  (1996)  conducted  a study  to  examine  elements  of  learning  styles,  emotional  intelligence,  and 
social  presence  as  predictors  of  distance  education  student  satisfaction  needs.  The  participants  were  enrolled  in  two  different  class  fonnats:  a 16  week 
traditional  fonnat  teleconference  class  and  a weekend  format  teleconference  class;  e-mail  was  also  used  for  group  discussions  and  learner  support.  The 
results  showed  that  of  the  three  constructs,  only  Social  Presence  was  found  to  be  a significant  predictor  of  satisfaction.  In  other  words,  regardless  of 
student  location  on-site  or  off-site,  or  status  as  a graduate  or  undergraduate  student,  participants  who  perceived  higher  social  presence  also  perceived  a 
higher  level  of  satisfaction  overall  with  the  course.  A secondary  finding  also  indicated  that  those  students  who  perceived  a higher  level  of  social  presence 


, ,or.  BEST  COPY  AVAILABLE 

1489 


also  tended  to  anticipate  a higher  outcome  in  terms  of  grades  (e.g.  A versus  B).  The  researchers  hypothesize  that  the  higher  confidence  levels  may  also 
affect  the  students’  perceived  closeness  with  the  instructor. 

There  is  a need  for  further  research  in  the  area  of  socio-psychological  perspective  in  computer-mediated  communication  as  the  measures  and 
research  methods  to  date  have  been  few  and  they  have  not  been  validated.  In  addition,  the  fact  that  studies  conducted  in  traditional  classrooms  point 
toward  teacher  immediacy  and/or  social  presence  as  a highly  significant  factor  in  improving  instructional  effectiveness,  it  is  necessary  for  research  to  be 
continued  in  these  areas  in  relation  to  online  learning  in  order  to  build  a better  model  of  online  education. 


By  beginning  the  examination  of  available  research  studies  conducted  on  the  construct  of  teacher  immediacy,  it  is  evident  that  this  construct  is 
essential  to  learning  in  terms  of  affective,  cognitive,  and  student  attitudinal  effects.  By  reviewing  the  research,  teacher  immediacy  has  demonstrated  that  it 
not  only  affects  outcomes  but  also  student,  and  possibly  instructor,  satisfaction  with  a course  (Moore,  Masterson,  Christophel,  and  Shea,  1996). 

In  addition,  the  construct  of  social  presence  appears  to  have  subsumed  that  of  teacher  immediacy  while  also  taking  into  consideration  the  fact  that 
some  medium  (e.g.  computer,  satellite  television,  audio-tapes)  is  in  some  way  altering  the  learning  environment.  Moreover,  the  research  on  social  presence 
has  demonstrated  a shift  in  thinking  for  this  construct.  While  social  presence  theory  was  initially  considered  a quality  of  the  medium  (Short,  Williams,  and 
Christie,  1976),  it  has  been  demonstrated  by  research  studies  that  this  may  not  in  fact  be  the  case.  Instead,  recent  researchers  in  the  area  of  online  education 
and  computer-mediated  conferencing  claim  that  social  presence  should  be  considered  to  be  a quality  of  the  users  of  a medium.  Gunawardena  claims  that 
social  presence  can  be  created  or  “cultured”  in  computer-conferencing  environments,  just  as  it  can  be  created  in  a traditional  classroom.  Although 
nonverbal  immediacy  behaviors,  such  as  facial  expressions  and  dress  may  not  be  transmitted  via  computers  or  other  mediums,  the  possibility  exists  that 
users  of  a particular  medium  may  not  only  compensate  for  their  loss  but  also  create  a parallel  set  of  behaviors  (e.g.  emoticons). 

Moreover,  these  findings  have  serious  implications  for  the  development  of  online  courses.  As  other  areas  of  research  have  demonstrated,  it  is 
essential  that  courses  are  created  in  a way  that  demands  interactivity  among  participants  not  only  with  the  material,  but  also  with  each  other  and  instructors 
(see  Irani,  1998;  Hiltz,  1994;  Romiszowski  and  Cheng,  1992;  Haraism,  1987).  This  area  of  research  substantiates  those  claims  by  demonstrating  that  the 
amount  of  social  presence  and/or  teacher  immediacy  behaviors  play  a strong  role  in  the  motivation  and  levels  of  satisfaction  students  derive  from  a course, 
as  well  as  their  learning  outcomes  in  terms  of  cognitive  and  affective  behaviors. 

Finally,  there  is  a significant  need  for  further  research  into  both  teacher  immediacy  and  social  presence  constructs.  Not  only  because  they  have 
thus  far  proven  to  be  essential  to  learning  outcomes  and  student  satisfaction,  but  also  because  there  is  a lack  of  research  in  these  areas  in  both  traditional 
learning  environments  and  especially  the  online  learning  environment. 

Purpose 

XXX.  More  specifically,  the  purpose  of  this  paper  is  to  examine  the  course  activities  and  the  nature  and  frequency  of  instructor  participation/interaction  as 
they  relate  to  students’  perceptions  of  overall  course  satisfaction. 


Methodology 

Participants 

In  the  fall  of  1999,  surveys  were  posted  in  a common  “bulletin  board”  area  and  students  enrolled  in  the  online  courses  were  asked  to  complete  the 
surveys.XX  of  the  XX  students,  only  XX  responded  to  the  survey.  The  low  student  response  rate  is  one  limitation  of  this  study. 

Instrumentation 

One  focus  of  the  posted  survey  was  to  gather  students’  perceptions  regarding  the  social  presence  of  the  instructor,  medium  of  delivery,  and  satisfaction  with 
online  course.  Students  were  asked  to  give  background  information  regarding  their  education  (e.g.  number  of  college  creditsgender,  age,  etc.).  Next,  they 
were  given  13  Likert-type  questions  with  responses  ranging  from  1 to  5 (strongly  agree  to  strongly  disagree).  Finally,  they  were  asked  to  answer  6 open- 
ended  questions  to  gather  further  detail  regarding  the  Likert-type 
analyses  could  be  run. 

Results 

Due  to  the  low  number  of  respondents,  a nonparametric  test  of  bi 
correlations  between  overall  course  satisfaction  and  the  Social  Pi 
significant  correlations: 

• Online  or  web-based  education  is  an  excellent  medium 

for  social  interaction 

• 1 felt  comfortable  conversing  through  this  medium 

• 1 felt  comfortable  introducing  myself  in  this  course 

• The  introductions  enabled  me  to  form  a sense  of 

online  community. 

• 1 felt  comfortable  participating  in  course  discussions. 

• 1 felt  comfortable  interacting  with  other  participants 

in  the  course. 

• I was  able  to  form  distinct  individual  impressions  of 

some  course  participants 

• The  instructor  created  a feeling  of  an  online  community. 

• The  instructor  facilitated  discussions  in  the  course 
(significant  at  the  .05  level,  significant  at  the  .01  level) 

Discussion 

Thus,  the  significant  correlations  indicate  that  there  is  indeed  a strong  relationship  between  the  social  presence  students’  perceive  in  an  online  course  and 
overall  course  satisfaction. 

Subsequent  studies  are  looking  at  these  areas  as  they  relate  to  students’  perceived  learning  and  to  instructor  reports  of  student  learning.  These  studies  are 
informing  our  instructor  training  process  to  emphasize  effective  use  of  "airtime"  in  designing  course  activities  and  in  participation  during  course  delivery. 

Implications  for  future  research 

Future  research  implications  include  the  need  for  further  study  into  the  area  of  social  presence  and  online  courses  to  detenuine  the  best  possible  course 
design  as  well  as  indicators  for  successful  courses. 


questions.  All  of  the  questions,  both  Likert  and  open-ended  were  then  coded  so  that 


/ariate  correlations  (Speanuan’s  rho)  was  mn  and  analyzed.  By  looking  at  the  bivariate 
esence  Scale  (modified  from  Gunawarden  and  Zittle,  1997)  we  found  the  following 
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Introduction 

As  researchers  in  the  relatively  new  dimension  of  online  learning,  we  need  to  gain  a better 
understanding  of  the  different  interactions  that  can  and  do  occur  in  these  developing  learning 
environments,  as  well  as  their  effect  on  students’  levels  of  satisfaction.  The  positive  relationship  between 
students’  satisfaction  with  instruction  and  their  subsequent  success  in  a course  (Pascarella,  et  al,,  1996)  is 
further  linked  with  the  integral  role  that  interaction  plays  in  the  level  of  learning  attained  by  students 
(Vygotsky,  1986;  Maxwell,  Richter  and  McCain,  1995;  Reeves  and  Reeves,  1996),  More  importantly, 

“few  chances  to  interact  with  the  instructor  limits  students’  ability  to  clarify  and  negotiate  instructional 
goals,  explore  alternative  methods,  or  construct  meaning  within  in  a social  context  based  on  personal 
knowledge  (Garrison,  1993).” 

The  importance  of  teacher-student  interactions  in  successful  teaching  episodes  is  supported  by  a 
large  body  of  research,  much  of  which  illustrates  the  belief  that  without  interaction,  knowledge,  whether  it 
be  in  the  form  of  concepts,  skills,  or  strategies,  tends  to  be  inert,  and  learner’s  lack  the  ability  to  apply  the 
new  knowledge  effectively  (see  Laurillard,  1994;  Bennett  and  Dunne,  1991;  Webb,  1991;  Clements  and 
Nastassi,  1988;  Vygotsky,  1978),  To  quote  Oliver  and  McLoughlin  (1997),  “few  dispute  the  advantages 
offered  by  interactive  elements  that  support  dialogue  in  teaching  and  learning.  Communicative 
interactions  can  be  used  to  engage  learners,  to  cause  them  to  reflect  on  and  to  articulate  ideas. 

Interactions  encourage  and  facilitate  cognition  and  play  an  important  part  in  promoting  learners’ 
intellectual  operations  and  thinking  processes  (p,  37),”  In  addition,  when  high  levels  of  interaction  are 
present,  students  report  that  they  experience  more  than  in  situations  that  involve  low  levels  of  instructor- 
student  interactions  (Richard,  Gorham,  and  McCroskey,  1987:,  Gorham,  1988), 

When  dealing  specifically  with  web  courses  or  online  courses,  interaction  and  feedback  have 
significant  impact  on  the  learning  process  since  they  add  value  that  results  in  improving  quality  and 
success  in  those  courses  (Hazari  and  Schnorr,  1999),  In  addition,  Comeaux  (1995)  has  also  found  that 
interaction  and  involvement  lessened  the  psychological  distance  for  students  at  remote  learning  sites. 
Moreover,  the  findings  of  Moore  and  Kearsley  (1996)  and  Cornell  and  Martin  (1997)  show  that 
interaction  and  feedback  from  the  instructor  are  factors  that  not  only  play  a role  in  course  satisfaction  but 
also  in  course  completion.  Similarly,  Barker,  Frisbie,  and  Patrick  (1989)  argue  that  the  effectiveness  of 
distance  learning  is  increased  through  enhanced  instructor-student  interaction.  This  line  of  research  is 
further  validated  by  studies  on  feedback  within  the  distance  education  model  and  its  relation  to  the 
attitudes  of  students  (McCleary  and  Egan,  1989), 

Purpose 

The  purpose  of  this  paper  is  to  determine  if  interaction  in  online  courses  has  an  effect  on 
students’  perceptions  of  that  course.  More  specifically  we  would  like  to  establish  a better  understanding  of 
the  possible  relationships  that  exist  between  students’  perceptions  of  interaction  in  online  courses  and 
overall  course  satisfaction. 


Methodology 
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Participants 


In  the  summer  of  1999,  surveys  were  sent  out  to  students  enrolled  in  online  courses  through  the 
Center  for  Distance  Learning  at  Empire  State  College  for  the  Spring  1999  semester.  Of  the  127  possible 
participants,  37  responded,  giving  us  a return  rate  of  29%. 

Instrumentation 

The  focus  of  the  distributed  survey  was  to  gain  a better  understanding  of  student’s  perceptions  of 
learning,  learning  styles,  instructor  interaction,  and  student-to-student  interaction.  Students  were  asked  to 
give  background  information  regarding  their  education  as  well  as  the  number  of  courses  taken  in  online 
format.  They  were  given  13  Likert-type  questions  with  responses  ranging  from  1 to  5 (strongly  agree  to 
strongly  disagree)  that  dealt  with  interaction  and  course  satisfaction  variables.  Finally,  they  were  asked  to 
answer  6 open-ended  questions  to  gather  further  detail  regarding  the  Likert-type  questions.  All  of  the 
questions,  both  Likert  and  open-ended  were  then  coded  so  that  analyses  could  be  run. 

Results 

Correlations  between  three  of  the  four  interaction  indicators  examined  were  statistically 
significant  at  the  p<.01  level.  An  examination  of  the  analysis  revealed  the  following:  students’  perception 
of  instructor  interaction  (in  general)  and  overall  course  satisfaction  had  a strong  positive  correlation 
(.615);  students’  perceptions  of  instructor  feedback  and  overall  course  satisfaction  had  a strong  positive 
correlation  (.732);  and  students’  perceptions  of  interaction  with  other  students  for  the  purpose  of  offering 
support  and/or  differing  perspectives  and  overall  course  satisfaction  registered  a moderate  positive 
correlation  (.474).  However,  students’  perceptions  of  interaction  with  other  students  (in  general)  and 
overall  course  satisfaction  did  not  indicate  a statistically  significant  correlation  (.07). 

Discussion  and  Implications  for  Future  Research 

The  aforementioned  findings  indicate  that  there  is  indeed  a strong  positive  relationship  between 
students’  perceptions  of  interactions  in  online  courses  and  their  satisfaction  level  with  those  courses. 
Moreover,  these  findings  indicate  instructor  feedback  and  course  satisfaction  are  the  strongest 
relationship,  which  tells  us  that  students  perceive  the  importance  of  scaffolding  their  knowledge  with  that 
if  the  instructor.  Similarly,  while  students’  perceive  interaction  with  other  students  to  also  be  important,  it 
is  not  perceived  to  be  more  important  than  interaction  with  the  instructor. 

Subsequent  studies  are  in  progress  examining  which  course  activities  effect  these  relationships  as 
well  as  how  these  areas  relate  to  students’  perceived  learning  and  to  instructor  reports  of  student  learning. 
Finally,  these  studies  are  informing  the  instructor  training  process  to  emphasize  effective  use  of  "airtime" 
in  designing  course  activities  and  in  participation  during  course  delivery.  These  studies  will  also 
examine  the  demographic  variables  of  the  student  population  to  look  for  potential  effects. 
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Abstract:  Evalutions  are  regularly  used  in  many  universities.  Manual  evaluation  is  very  time- 
consuming,  and  therefore  a natural  candidate  for  delegation  to  computers.  To  avoid  duplicating 
code  or  evaluation  forms,  we  have  developed  a collection  of  tools  that  can  address  all  evaluation 
needs.  Statistics  can  also  be  gathered  on  the  fly  by  accessing  the  tools  online. 


Introduction 

During  the  last  year,  we  have  implemented  WebWorks,  a tool  suite  for  Web  administration  in  teaching.  In  this  paper, 
we  describe  the  subcomponent  for  online  evaluations  of  the  WebWorks  project.  Several  other  components  of  the 
WebWorks  tool  suite  have  previously  been  described  elsewhere  (see  references),  and  include  support  for 
administering  online  knowledge  tests,  text  and  animation  generation,  an  animation  tool,  and  automatic  indexing. 

Evaluations  are  regularly  used  in  many  universities  for  a variety  of  different  courses.  The  basic  evaluation  process 
usually  remains  the  same;  sometimes,  the  only  difference  between  two  evaluations  lies  in  some  questions,  date  and 
title.  Thus,  it  would  be  helpful  to  have  a single  tool  that  can  handle  an  arbitrary  number  of  evaluations  and  is  easily 
customizable.  For  this  purpose,  we  have  developed  a collection  of  Perl  tools  that  support  the  whole  evaluation 
process  over  the  WWW.  One  tool  is  responsible  for  generating  the  evaluation  forms  and  storing  the  entries,  while 
another  tool  handles  the  generation  of  evaluation  reports  in  a variety  of  formats. 

In  the  following,  we  will  briefly  outline  the  tools  and  their  functionality. 


Generating  Online  Forms  Using  the  feedback  Tool 

The  tool  feedback  is  responsible  for  generating  a HTML  form  according  to  the  settings  for  the  current  evaluation 
context  and  parsing  user  input.  The  basic  structure  of  the  feedback  tool  is  shown  on  the  left  side  of  figure  1 . 


Figure  1.  Components  and  Interaction  of  feedback  and  backEval 
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All  entries  apart  from  date  and  title  are  components  allowing  for  easy  reuse  and  dynamic  exchange.  Thus,  it  is  very 
easy  to  reuse  a given  set  of  questions  and  possible  answers  (called  ''catalogue'')  in  a different  context  by  simply 
replacing  the  author  and  course  components,  feedback  currently  supports  standard  text  messages  in  both  English 
and  German,  but  can  easily  be  extended  to  other  languages.  The  formatter  component  can  currently  provide  output 
in  either  ASCII,  HTML  and  L^TeX;  we  are  also  working  on  providing  RTF  output. 

When  a user  accesses  feedback  with  a given  access  key,  the  required  components  are  determined  and  used  to  build  a 
HTML  form.  The  HTML  elements  radio  button,  selection  menu,  checkbox,  text  field  and  text  area  are  supported. 
Upon  submission  of  the  form,  feedback  checks  if  all  entries  marked  as  mandatory  are  given,  and  if  so,  generates  a 
new  entry  into  the  log  file. 

Each  catalogue  consists  of  a set  of  question  entries.  Each  question  entry  contains  a question  text^  type  and  a list  of 
answers.  Furthermore,  the  author  can  mark  any  question  as  mandatory^  and  select  whether  the  evaluation  of  the 
entry  should  be  available  to  all  users,  only  the  author,  or  remain  completely  hidden.  Registering  a new  catalogue 
requires  specifying  the  entries  shown  in  figure  1 and  entering  a single  line  in  the  ASCII-based  registry. 


Automatic  Evaluation  Report  Generation  Using  backEval 

The  backEval  tool  has  the  same  basic  architecture  as  feedback,  as  shown  in  figure  1.  Any  user  may  access  the  tool 
via  the  WWW  to  get  up-to-date  statistics  on  any  given  evaluation;  however,  as  mentioned  before,  authors  may 
restrict  the  availability  of  entries. 

The  generated  page  contains  all  questions  visible  to  the  current  user  including  their  evaluation.  The  evaluation  lists 
the  question,  all  possible  answers  and  how  often  they  were  chosen  by  users.  Text  inputs  resulting  from  text  fields 
cannot  be  displayed  in  this  way,  as  it  is  rather  unlikely  that  two  users  have  entered  precisely  the  same  text.  Thus,  text 
entries  are  shown  in  formatted  lists. 

Statistics  are  generated  in  either  ASCII,  HTML  or  L^TeX  format,  with  a RTF  implementation  under  development. 
Except  for  ASCII  format,  colored  bars  are  used  to  highlight  the  percentage  of  any  given  answer.  It  is  thus  easily 
possibly  for  any  user  to  print  the  current  evaluation  directly  from  their  preferred  browser. 

backEval  has  a number  of  options  for  modifying  the  display  of  the  statistics.  It  can  also  show  the  submitted  entries 
separately  in  the  order  they  were  given,  making  it  applicable  for  tasks  such  as  online  registration.  Due  to  privacy 
concerns,  this  evaluation  can  only  be  selected  by  the  authenticated  author  of  the  given  catalogue. 
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This  paper  describes  effective  ways  that  the  nationally  validated  Information  Technology  Skill  Standards 
developed  by  the  North  West  Center  for  Emerging  Technologies  have  been  used  by  the  Chauncey  Group, 
Divison  of  Educational  Testing  Service  to  develop  vendor  - neutral  certification  exams  in  eight  IT  skill 
cluster  areas.  The  paper  will  also  describe  preparation  workshops  and  educator  versions  of  the  exams  that 
allow  faculty  to  demonstrate  or  certify  their  expertise  in  Information  Technology,  and  how  educators 
benefit  from  the  program. 
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Abstract:  Constructivism  is  learning  which  takes  place  when  learners  actively  construct  their  own  reality 
and/or  knowledge  or  least  interpret  it  based  upon  their  perceptions  of  experience.  Constructivism,  as  theory 
of  learning,  is  a subject  which  educators  are  beginning  to  discuss  in  connection  with  Distance  Education  and 
Computer  - Mediated  Communication.  The  use  of  multimedia  technologies  can  help  the  constructivist  theory 
because  it  is  possible  to  build  a "Learning  by  Doing"  environment  that  combines  multimedia  technologies 
and  the  constructivist  approach.  In  educational  process  the  students  generally  build  hypertexts  and 
hypermedia  on  subjects  which  they  know.  Their  teachers  control  and  give  them  all  information  to  include  in 
hypertext.  Using  this  approach  the  role  of  the  students  is  passive,  because  they  transfer,  on  digital 
documents,  infonnation  furnished  by  the  teachers.  For  this  reason  1 have  thought  to  control  two  different 
samples  of  students  that  have  researched,  selected  infonnation,  and  produced  a hybrid  CD-ROM/Intemet  on 
a subject  which  they  didn't  know  before  (Neural  Networks).  This  paper  describes  this  project  which  1 have 
named  "Learning  by  Doing  and  Creating"  (LDC). 

Scenario 

Two  different  samples  of  students  (aged  17-  20)  of  a High  School  (specialization  course  in  Information 
Technologies),  during  the  school  years  1997-98  (15  students)  and  1998-99  (14  students),  have  researched,  studied 
and  built  a hypermedia  (on  Neural  Networks)  using  the  new  educational  technologies  and  they  will  propose  this  CD- 
ROM  to  other  students.  This  hypermedia  has  been  developed  in  the  Laboratory  of  System  and  Techniques  of 
Transmission  (3  hours  in  the  week  throughout  the  school  year)  in  a technical  institute  in  Italy.  I have  organized  my 
educational  project  in  three  stages. 

First  stage  of  the  project 

In  the  first  stage,  school  year  1997  - 98,  I have  worked  with  15  students  (aged  17  - 19).  1 have  proposed  them  a 
simple  multiple  choices  test  (8  questions)  to  control  the  students'  knowledge  on  the  Neural  Networks.  The  percentage 
of  correct  answers  has  been  the  27,5%  (33  on  120)  , deducing  that  my  students  didn't  know  the  subject  "Neural 
Networks"  and  they  were  a good  sample  to  try  my  "Learning  by  Doing  and  Creating"  (LDC)  approach.  After  the  test, 
the  students  have  researched  in  the  Internet:  information  on  hypertexts  and  on  hypermedia;  (their  history  and  their 
organisation),  how  to  build  a hypertext  (using  HTML)  and  hypermedia  (with  an  introduction  to  Java  Language  and 
Java-Script).  I have  also  explained  them  how  to  use  some  software  and  hardware  tools  to  digitized  the  images,  the 
movies,  the  sounds,  and  the  correct  calibration  of  the  communication  codes  in  educational  hypermedia  [Sala,  1 999a; 
Sala,  1999b].  Inside  this  approach  I have  introduced  some  novelties.  For  example:  a periodical  "time  out"  which  is  a 
stop  of  all  activities  in  the  classroom.  It  is  necessary  to  discuss  the  students'  problem  during  the  project  (e.g.,  the 
coordination  with  the  components  of  the  group,  the  interaction  between  the  groups,  the  organization  of  the  hypertext, 
and  so  forth),  and  a journal  of  the  classroom  where  the  students  resume  all  project'  activities  done  in  a day  (like  a 
"logbook").  The  sample  of  the  students  has: 

• researched  in  the  Internet  some  information  on  Neural  Networks, 

• researched  the  main  bibliography  (using  the  public  libraries), 

• selected  the  information, 

• built  the  storyboard  of  the  hypertext  (they  have  chosen  a hierarchical  hypertext), 

• chosen  other  information  to  insert  in  the  hypermedia  (e.g.,  animations,  movies,  and  so  forth); 

• realized  and  controlled  the  hyperlinks. 

The  hypermedia  has  been  carried  out  using  the  Hypertext  Mark  Up  Language  (HTML)  because  it  is  easy  to  learn,  it  can 
be  integrated  with  Java  Applet  and  Java  Script  to  introduce  the  multimedia  codes  (e.g.,  the  animations,  the  movies),  and 
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we  can  used  a common  browser  without  runtime  module  (e.g.,  Netscape  Navigator™  or  Internet  Explorer™) . 

First  version  of  this  hypermedia,  finished  in  month  of  June  1998,  comprises:  20  pages  of  hypertext,  20  animations 
using  Java  language  and  Java  Script,  2 movies  (one  is  an  interview  to  an  expert  on  Neural  Networks,  the  other  is  on 
the  authors).  After  the  hypermedia  production  my  students  have  studied,  for  six  hours,  the  subject  "Neural  Networks" 
(NN)  using  their  CD-ROM.  During  the  learning  phase  the  students  have  noted  that  it  is  difficult  to  study  using  only  the 
monitor  without  a textbook.  For  this  reason  they  have  selected  the  text  frames  and,  using  "Copy  and  Paste",  they  have 
made  notes.  An  other  problem  is  the  cognitive  overhead,  which  is  avoided  only  by  the  best  students.  1 have  also 
controlled  their  knowledge  using  multiple  - choice  tests  (15  questions,  the  first  8 questions  were  the  same  of  the  first 
test).  The  worst  case  of  this  multiple-choices  test  corresponds  to  the  question  15  ("To  draw  some  model  of  Neural 
Networks")  because  this  subject  is  difficult  to  understand  for  Secondary  School  students  and  it  has  been  proposed  in  a 
long  hypertextual  page  (using  the  scrolling).  The  results  of  these  analysis  have  been  encouraging,  because  the  65.3% 
(which  corresponds  to  the  147  correct  answers  on  225)  of  my  students  have  achieved  the  educational  goals  that  I have 
established.  The  comparisons  between  the  first  test  and  eight  same  questions  of  the  second  have  demonstrated  that  the 
students  have  learnt  the  subject  Neural  Networks  (first  test  27.5%,  second  test  72.5%).  My  contribution  to  the  project 
has  been  well  evaluated.  I have  used  the  negative  data  to  modify  my  interaction  with  the  classroom;  this  feedback  it  has 
been  important  because  all  educational  process  was  under  control. 

Second  stage  of  the  project 

In  the  second  stage,  school  year  1998  - 99,  I have  worked  with  another  sample  of  14  students  (aged  18  - 20)  of 
the  same  technical  school  . have  answered  to  the  entry  test  on  NN  (the  same  of  the  last  sample)  and  after  they  have 
analysed  the  hypermedia  produced.  They  have  evaluated,  answering  to  the  multiple  choices  test,  the  icons  (e.g.,  the 
shape  and  the  image  should  recall  the  linking  function);  the  hot  words,  the  text-  frames,  the  visual  interface,  the 
navigation  in  the  hypermedia.  The  results  of  this  analysis  have  been  used  to  modified  the  first  version  of  the 
hypermedia.  The  second  sample  of  the  students  have  modified  the  contents  of  the  hypermedia  difficult  to  understand 
(e.g.,  some  pages  dedicated  to  the  Hopfield's  model),  and  they  have  added  other  pages  (e.g.,  three  pages  dedicated  to 
the  Kohonen's  model ).  New  version  of  this  hypermedia  comprises  a map  which  performs  the  "here  we  are  function", 
90  pages  of  hypertext,  25  animations  using  Java  language  and  Java  script,  10  digitized  videotaped  segments  (quick- 
time  movies),  30  links  to  other  Internet  sites  on  Neural  Networks.  In  the  last  part  of  this  second  stage,  I have 
controlled  the  students’  knowledge  on  Neural  Networks  using  the  same  multiple  - choices  test  that  1 have  proposed  in 
the  school  year  1997/98.  The  correct  answers  have  been  157  on  210  (74.8%)  1 have  also  made  a comparison  with  the 

two  samples  of  students  using  the  histograms. 

Conclusions 

This  educational  project  is  a "work  in  progress".  In  the  school  year  1999/2000  1 shall  control  a third  sample  of 
students  that  will  study  the  Neural  Networks  using  the  traditional  educational  methods,  and  I shall  compare  the 
educational  results  with  the  other  two  samples.  This  LDC  approach  is  an  example  of  "Learning  by  Doing" 
environment  where  some  students  build  an  educational  hypermedia  and  they  learn  while  build  it.  I have  added  the  term 
"Creating"  because  there  was  also  the  creativity  during  this  hypermedia  development  (e.g.,  the  creation  of  the 
animations,  the  choose  of  the  graphics  interface,  and  so  forth).  Some  researchers,  such  as  Spiro  and  Jehng  (1990), 
have  emphasized  the  active  role  learners  must  play  in  order  to  learn  in  hypertext-based  learning  environments. 
Therefore,  the  multimedia  technologies  can  help  the  teachers  to  complete  this  evolution.  1 think  that  the  "Learning  by 
Doing  and  Creating"  approach  is  in  agreement  with  this  old  Chinese  proverb:  " If  I listen,  1 forget;  if  1 see,  1 
remember;  if  I do,  I understand". 
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Abstract : The  objective  of  this  study  was  to  evaluate  teletraining  in  an  interactive  computer 
mediated  system  through  the  communicative  behavior,  the  specificity  of  each  situation  in 
order  to  better  understand  and  to  optimize  the  communication  in  group.  In  a first  part,  we 
propose  a contextual  evaluation  method  in  order  to  evaluate  perfonnance  reached  by  the 
trainees,  actors  and  socio-technical  system  behavior,  and  satisfaction  of  actors.  We  have 
applied  our  methodology  in  the  evaluation  of  CETTE  (Communication  Environment  for 
teletraining  in  Engineering)  (Gradinariu  1999).  We  present,  then,  some  original  results  of  this 
evaluation  and  their  interpretation  in  lecture,  practical  works  and  cooperative  project  context. 


Introduction 

In  a complex  and  new  situation  it  is  desirable  to  use  several  methods  and  different  sources  of  information  to 
evaluate  the  same  objective  on  populations,  which  carry  out  the  same  task  in  the  same  environment  and  to 
increase  gradually  the  complexity  of  the  task  by  diversifying  pedagogical  strategies. 

An  other  point  which  is  seldom  precise  in  studies  about  the  experimental  evaluation  is  how  to  analyze  series  of 
date  and  observations  to  which  we  attach  different  significance  and  which  have  not  the  same  nature  when  the 
evaluation  methodology  is  based  on  a multiplicity  of  methods. 


Methodology  of  Evaluation 

We  propose  a contextual  evaluation  method  based  on  (see  Sandoz-Guermond  & Beuchot  2000): 

- quantification  of  the  training  outcomes  about  cognitive  activities  : memorization,  understanding  and  mastery 

- analysis  of  communication  types  used  during  each  activity  to  resolve  the  task.  We  suggest  to  conduct 
objectives  observations  made  by  human  evaluators  and  measures  on  the  level  and  the  content  of  the 
communication,  during  each  session  of  apprenticeship.  The  measures  are  collected  by  software  sensors  in  order 
to  evaluate  the  behavior  of  actors  and  the  behavior  of  the  socio-technical  system  during  each  training  session. 

- questionnaires  on  the  perception  and  the  usage  of  the  different  items  of  the  environment  (Rao  1 994)  that  make 
easy  the  communication  during  the  training  session  (interface  quality,  effectiveness  of  new  technology, 
concentration,  interaction,  motivation)  in  order  to  evaluate  the  satisfaction  of  actors  in  the  mediated  environment. 


Data  Analysis  and  Results  Interpretation 

Experimental  procedure 

Experiments  were  conducted  on  two  student  populations  with  a lecture,  practical  works  and  cooperative  project 
on  numerical  command  language  for  mechanical  tools:  19  students  with  the  bachelor  in  manufacturing 
engineering  which  have  a good  background  in  the  training  subject  and  6 undergraduate  students  with  low 
knowledge  in  this  subject.  The  same  kind  of  session  with  the  same  contents  of  the  training  subject  material  was 
carried  in  a traditional  environment  with  8 students  preparing  their  master  in  mechanical  engineering. 
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Training  outcomes 

The  question  is  to  compare  the  training  effectiveness  in  mediated  environment  with  traditional  situation. 

In  an  interpretative  and  comparative  cognitive  perspective  both  on  the  nature  of  the  population  (knowledge, 
capacities)  and  the  technology  (traditional,  multimedia  supported  by  the  computer),  we  determine,  for  each  of 
the  three  cognitive  activities  (memorization  understanding  and  mastery)  similarities,  small  differences,  great 
differences,  surprised  on  an  ordinal  scale  and  report. 

Statistical  results  indicates  that  trainees  in  the  remote  site  achieved  quite  identical  results  than  in  the  traditional 
environment.  Training  outcomes’  standard  deviation  of  the  student  in  the  mediated  environment  is  very  small 
compared  to  the  student  in  the  traditional  environment,  indicating  that  their  grades  were  very  homogeneous. 


Satisfaction  of  the  actors 

In  order  to  analyze  the  data  from  our  study  (Rao  1994),  we  firstly  ran  descriptive  statistics  on  each  of  the  items 
and  their  variables.  The  results  (means  and  standard  deviation)  indicate  that  the  different  groups  have 
appreciated  the  training  progress,  they  are  quite  motivated  to  part  in  it.  The  students  in  the  mediated  teletraining 
system  interacted  more  among  themselves  and  with’  the  professor  than  did  those  of  the  traditional  environment. 
We  observed  a significant  difference  concerning  the  concentration  : the  remote  students  were  also  more 
concentrated  than  were  those  of  the  on-site  location. 

The  second  step  in  the  analysis  of  replies  to  the  questionnaire  is  to  understand  which  items  indicate  a difference 
of  satisfaction  in  the  population.  The  analysis  of  the  variance  of  replies  to  questions  allows  thus  to  define  the 
constructs  of  the  usage  quality  with  their  variables  (components)  as  below  : 

- Acceptability  of  the  communication  technology:  perception  of  the  teletraining  environment,  use  of  computer 
tools,  remoteness 

- Difficulty  to  achieve  their  task  : quality  of  the  interface,  interaction,  accessibility  of  the  information 

- Personal  investment : motivation,  participation,  need  to  solicit  the  trainer,  concentration 

- Quality  of  feedback  of  the  system  : effectiveness  of  new  technology. 


Actors  and  system  behavior 

In  a situation  of  physical  isolation  with  possibilities  limited  of  telepresence,  the  verbal  behavior  is  the  real 
reciprocal  orientation  tool  in  the  cooperative  work.  In  the  cooperative  project,  generally  two  students  (out  of 
four)  are  speaking  simultaneously. 

Three  criteria  allow  to  characterize  the  non-verbal  behavior  : the  mutual  orientation  in  the  shared  work,  the 
behavior  in  critical  situation  and  the  research  of  the  assistance.  According  to  the  trainer,  due  to  the  lack  of 
feedback,  it  is  very  difficult  to  appreciate  the  level  of  comprehension  of  the  trainees  in  a lecture  context. 


Conclusion 

Our  study,  we  retain  a feasibility  of  the  interactive  teletraining  with  computer  mediated  systems.  The  usage  of 
the  new  technologies  seems  as  efficient  as  in  traditional  situation.  There  is  no  significant  difference  during  their 
use.  However,  we  note  a greater  students'  concentration  and  tiredness  during  the  teletraining  session:  the 
duration  of  the  session  must  be  reduced,  the  lecture  re-designed  and  news  pedagogical  strategies  introduced. 
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Abstract:  Our  experience  in  distance  learning  has  shown  us  how  important  it  is  to  have  well  structured  materials 
which  are  designed  from  a pedagogical  point  of  view,  and  keeping  in  mind  what,  how  and  when  we  want  to  teach 
or  encourage  learning.  That  is  why  we  have  created  an  authoring  tool,  which  will  meet  the  needs  of  our 
institution. 


Introduction 

The  Methodological  Resources  Assistant  (MRA)  is  a project  being  developed  by  the  Universitat  Oberta  de 
Catalunya  (UOC),  which  is  a distance  learning  university  based  on  the  concept  of  the  ‘Virtual  Campus’  and  the  use 
of  multimedia  teaching  materials. 

We  must  place  this  project  in  the  context  of  the  task  developed  by  the  UOC  and  other  institutions  and  companies 
created  from  the  Foundation  for  the  Universitat  Oberta  de  Catalunya.  In  only  few  years  the  growth  of  services 
offered  by  our  university  and  its  associated  institutions  has  created  the  need  to  contact  a high  number  of 
collaborators,  subject  specialists,  professors  from  other  universities  as  well  as  professionals  from  the  world  of 
business,  who  could  act  as  authors  of  our  material  or  develop  specific  training  tasks. 

We  have  become  aware  of  the  need  to  train  these  authors  (usually  specialists  in  a field  of  science  or  other  areas  of 
study)  in  how  to  elaborate  teaching  material  for  distance  learning.  This  implies  a training  task,  centred  not  only  in 
the  understanding  of  the  distance  teaching-learning  process,  but  also  in  the  knowledge  of  resources  and  tools  to 
make  the  elaboration  of  material  easier.  The  MRA  was  created  to  answer  the  above-mentioned  needs. 


The  Methodological  Resources  Assistant 
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MRA  main  features 

1.  What  is  the  MRA? 


It  is  a system  for  the  design  and  elaboration  of  methodological  resources  adapted  to  different  objectives  and  training 
needs,  in  a virtual  learning  environment.  Its  aim  is  to  provide  a wide  range  of  distance  learning  methodological 
resources  as  well  as  tools  to  facilitate  material  creation,  for  professors  and  authors. 


2.  What  can  be  found  there? 

The  MRA  enables: 

Looking  for  resources 

Accessing  to  a concrete  resource  report  by  selecting  its  name. 

Acquiring  / Offering  ideas 
Consulting  methodological  aspects 
Creating  activities:  programs  and  templates 
Giving  guidelines  or  advice  of  use 

Every  resource  report  provides  information  about  the  typology  or  category  that  the  resource  belongs  to,  the  goals 
that  can  be  reached  with  it  or  the  activity  given;  the  resource  description  - structure,  stages,  functioning,  etc.-;  the 
application  - guidelines  or  directions  to  improve  the  possibilities  the  resource  offers;  the  work  dynamics  to  follow; 
the  view  of  concrete  examples;  the  elaboration  guidelines  for  the  resource;  the  bibliography  reference;  and  templates 
and  programs  to  create  resources  easier. 


3.  Which  is  its  structure? 

The  MRA  is  structured  as  a system  composed  of  different  elements: 

A communication  space:  a forum,  a mailbox,  FAQ’s. . . 

Help  or  Guidance  about  methodological  aspects 

The  searching  form  (the  tool  that  helps  the  professor  or  the  author  to  know  the  best  resource  features  for  their 
material  elaboration) 

The  result  of  the  searches*  list 
The  resource  reports 
The  examples 
The  templates 

The  data  base  that  supports  the  system 


Conclusions 

• This  space  allows  educators  to  gain  access  to  up-to-date  information  concerning  educational  activities  and  the 
student  body. 

• The  tools  used  in  the  evaluation  process  are  also  incorporated  which  allow  educators  to  introduce  and  consult 
information  concerning  the  progress  and  qualifications  of  students. 

• The  exchange  of  ideas  and  collaboration  are  encouraged  between  all  of  those  people  who  participate  in  the 
teaching  process  by  means  of  the  environments  that  are  reserved  for  telematic  intercommunication. 

• Another  function  of  this  integrated  space  is  to  grant  direct  access  to  the  university’s  administration  services  and 
to  help  in  solving  educators’  problems  and  doubts. 

• Furthermore,  educators  are  involved  in  an  on-going  process  that  is  intended  to  improve  the  educational  model 
through  surveys  and  in  this  way  detect  strong  and  weak  aspects. 
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Abstract 

This  paper  describes  a pilot  research-project  that  will  be  carried  out  during  our  courses 
at  the  Delft  University  of  Technology  in  March  and  April  2000.  After  this  pilot  we  plan 
to  internationalize  the  project  to  interested  foreign  universities. 

We  believe  the  project  '‘How  we  make  our  learners  instructors’’  will  make  better 
learners,  learners  who  learn  to  educate  themselves  and  to  educate  each  other,  and 
learners  who  know  how  important  knowledge  sharing  can  be  in  obtaining  better  results. 
We  facilitate  our  learners  to  define  their  own  learning  paths  through  the  knowledge  and 
additional  information  in  our  learning  environment.  We  focussed  on  the  self-expression 
and  judgement  of  the  learner.  This  resulted  in  a dynamic  knowledge  interaction  between 
the  learners  and  the  instructor,  meanly  led  by  the  learners. 

We  would  like  to  present  the  results  of  this  project  during  our  presentation  at 
Edmedia2000 


As  ICT  practitioners  for  educational  purposes,  we  feel  that  a new  phase  is  coming  up.  The  past  years  the  learners 
learned  to  work  with  the  tools  and  learned  how  to  use  them  in  the  field,  today  our  learners  become  more  and 
more  self-confident  and  self-instructing.  For  educational  purposes  this  means  that  the  discussion  on  ICT  shifts 
from  the  techniques  -how,  to  the  content -what,  and  the  instructors  role  (as  compared  to  the  learner’s  role)  -who. 
In  this  article  we  focus  on  the  role  of  the  learner  and  the  instructor. 

In  the  first  paragraph  we  describe  how  we  educate  our  courses  at  the  Delft  University  of  Technology.  In  the 
second  paragraph  we  will  briefly  describe  what  we  have  done  in  the  past  three  years  to  understand  more  about  the 
roles  of  the  instructors  and  the  learners.  In  the  third  paragraph  we  formulate  the  project  in  2000.  We  will 
summarise  the  results  of  the  project  in  the  last  paragraph. 

Education  at  the  Delft  University  of  Technology,  Business  to  Business  marketing 

In  this  paragraph  we  will  briefly  describe  our  thoughts  about  education  and  the  social  dimension  of  education. 
Education: 

Education  in  our  opinion  is  an  instructor-learner  relation  where  individuals  challenge  each  other  with  new  ideas. 
As  Francois  Jacob  (1991)  has  put  it,  human  beings  are  programmed  to  learn,  they  are  curious  beings,  a 
characteristic  without  they  could  not  know.  So  education  is  learner  orientated  as  well  as  instructor  based.  Our 
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main  question  at  this  point  is:  How  can  we  challenge  the  learners,  in  order  to  make  them  so  curious  that  they 
become  their  own  (and  each  other’s)  instructors? 

Before  the  use  of  ICT  we  graded  our  learners  on  their  problem  solving  capacities,  their  personal  opinion,  their 
theoretical  founding  and  their  knowledge  on  the  subject  itself  Our  experience  with  this  type  of  examination  is 
that  most  students  were  used  to  reproducing  content  an  instructor  or  book  had  told  them,  they  were  parrots.  ICT 
changed  students  from  parrots  reproducing  content  into  supporters  of  the  knowledge.  It  gave  us  possibilities  to 
improve  the  learner  - instructor  relationship.  ICT  offers  solutions  to  warm  up  the  knowledge  flow  between 
learners  and  instructors.  We  are  mainly  talking  about  the  way  knowledge  is  brought  across. 

In  the  next  paragraph  we  will  come  back  on  our  experiences  with  ICT  based  education. 

Social  dimension 

According  to  Josef  Schubert  (1983)  the  activity  specific  to  human  beings  is  the  cooperative  use  of  “tools”  in  the 
production  of  “goods”.  Our  main  question  in  our  courses  is: 

How  can  we  motivate  the  cooperative  use  of  ’’content " in  our  classes  to  create  ’’new  knowledge  ” among  the 
participants  (learners  and  instructors)?  Before  the  use  of  ICT  we  had  no  idea  how  the  learners  cooperated  whit 
in  their  group  to  solve  the  assignments.  In  the  next  paragraph  we  will  come  back  on  how  ICT  enabled  us  to 
monitor  the  participation  of  the  learners  with  the  content  and  with  the  other  learners  in  the  course. 


ICT  based  education  at  the  Delft  University  of  Technology  (1996-1999) 

The  Delft  University  of  Technology,  Business  to  Business  marketing  development  project  on  the  use  of  ICT 
instruments  started  in  1996.  The  first  instrument  was  the  use  of  e-mail  as  a means  of  interactive  communication 
between  instructor  and  learners.  The  second  instrument,  videoconference  (PictureTel),  enabled  us  to  interact  real 
time  with  learners  on  a remote  basis. 

Streaming  media:  In  1998,  the  use  of  real-media  resulted  in  the  first  online  course  (on  the  Internet)  in  the 
Netherlands.  Next  to  the  80  regular  students,  2500  individual  people  logged  in  on  our  course  and  followed  the 
course  Business  to  Business  Marketing  live  on  a remote  basis. 

Learning  environment:  During  the  last  two  years  we  developed  a third  instrument  the  learning  environment 
based  on  WebCT.  (http://webct.io.tudelft.nl:8900/public/B2B  online  in  Dutch).  WebCT  offered  us  an 
environment  where  we  could  easily  upload  our  content  in.  It  also  contained  all  the  tools  we  created  ourselves  for 
our  first  live  online  Internet  course,  like  chat  and  a whiteboard.  Besides  these  tools,  WebCT  made  it  possible  to 
create  student  groups  in  which  the  learners  could  make  their  assignments.  The  most  interesting  tool  of  WebCT  is 
the  student  tracking  option.  With  this  tool  we  can  measure  the  earlier  mentioned  participation  of  the  learners.  We 
can  monitor  the  learning  path  of  each  individual  learner.  How  much  time  did  he  or  she  spend  on  the  course  site, 
how  much  questions  did  the  learner  ask  and  answer?  Did  the  learners  cooperate  online?  Of  course  we  could  not 
monitor  the  face-to-face  discussions  learners  had  about  the  course  material  or  the  time  the  learner  spend  on  a 
document  he  or  she  printed  but  this  monitoring  tool  actually  made  us  change  the  way  we  taught  our  courses  in 
1999. 

Bodies  of  knowledge:  In  December  1998  we  divided  the  course  material  in  100  subjects.  Each  of  the  subjects 
represents  a body  of  knowledge.  We  recorded  a video  clip  for  each  subject  where  the  instructor  explains  the 
subject  in  about  two  minutes,  and  we  started  building  a database  of  related  publications  to  the  subjects.  By  the 
time  our  next  course  started  in  the  spring  of  1999,  all  the  course  content  was  available  online  in  multimedia 
mode.  The  instructor  did  not  teach  in  the  classroom  anymore.  All  we  had  to  do  is  challenging  the  learners  with 
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actualities  and  cases  presented  by  the  instructor  or  invited  speakers  or  actualities  the  students  themselves  heard 
about  in  the  news. 

Dynamic  content:  According  to  their  own  learning  habits,  students  who  use  our  online  learning  environment  are 
free  to  choose  the  medium  that  fits  them  best.  The  content  has  become  dynamic.  Their  options  are:  textbook 
material  on  the  subject,  sheets  (transparencies)  with  streaming  media  explanation  by  instructor,  with  bullets  on 
the  subjects,  articles  (papers)  on  the  subjects,  streaming  media  (realmedia),  video  of  last  years  lectures,  news 
forums,  where  students  can  discuss  a subject,  cases,  other  subjects  (if  any),  e.g.  links  to  companies  working  in  the 
field  or  famous  for  their  marketing  efforts,  or  links  to  other  universities 

In  practice  students  choose  their  own  paths  through  all  this  possibilities,  media  and  content.  The  instructor 
coaches  them,  challenges  them  and  tries  to  motivate  them  to  learn  more  from  the  available  material  or  each  other. 
As  we  now  have  over  80  learning  paths  of  former  students,  we  have  clues  how  to  digitalize  the  content  of  our 
course.  We  feel  that  in  digital  courses  the  non-linearity  of  the  chosen  learning  path  of  learners  and  the  possibility 
to  monitor  this  path  is  very  important.  The  dynamics  are  in  the  path,  not  in  the  content. 

The  multiple  media  approach  has  resulted  in  better  grades  for  our  students  as  compared  to  the  traditional  classes. 
The  commitment,  time  spend  and  the  grades  did  statistically  significant  change  with  previous  years. 


The  ICT  based  education  project  at  the  Delft  University  of  Technology  in  2000 

For  educational  purposes  we  use  ICT  as  an  open  knowledge  transfer  tool,  or  perhaps  better  put:  we  facilitate  the 
learners  to  define  their  own  learning  paths  through  the  bodies  of  knowledge  and  additional  information  in  our 
learning  environment.  The  shift  we  created  in  these  years’  classes  was  a shift  to  dynamic  interaction  between  the 
learners  and  the  instructor. 

So  actually  we  want  the  learners  to  become  instructors  of  each  other. 


The  educational  purposes  of  the  2000  project: 

We  have  different  levels  of  knowledge  in  our  environment:  commodity  data,  commodity  knowledge  and  new 
knowledge.  (Santema,  Genang,  1999)  Actually  all  the  bodies  of  knowledge  are  just  bodies  filled  with  interesting 
data,  or  as  we  call  it  "commodity"  data.  As  Karl  Marx  (1957)  quoted,  a commodity  is  a mysterious  thing,  a 
hierorglyphic  - a picture  thing,  a “beautiful”  system  now  taken  to  be  difficult  or  impossible  to  decode.  Last  year 
we  succeeded  in  motivating  the  learners  to  react  on  this  “commodity”  data  in  order  to  “learn”  it,  to  deepen  it  and 
make  their  own  knowledge  of  it.  But  this  is  not  enough.  The  learner  should  be  able  to  share  his  or  her  knowledge 
with  other  learners  following  the  course.  Can  we  motivate  them  to  state  what  data  they  found  (and  where)  and 
what  they  did  to  process  it  in  order  to  make  it  knowledge  for  them.  And  what  happens  when  they  discuss  it  with 
other  learners.  Will  they  motivate  their  progress,  will  they  change  it,  will  they  convince  others  and  therefore 
deepen  the  knowledge  they  derived  from  the  data?  Can  they  add  new  knowledge? 

In  order  to  achieve  this  we  have  to  reward  the  learners  on  information  and  knowledge  sharing.  We  deliberately 
instruct  groups  of  learners  to  look  at  a subject  from  different  angles  and  using  “knowledge”  out  of  our  learning 
environment  or  any  available  medium  to  substantiate  their  opinions.  The  focus  of  education  shifts  more  to  the 
self-expression  and  judgement  of  the  learner.  (Sakamoto,  1999).  There  will  be  no  instructor-based  lectures  but 
student-based  lectures.  The  lectures  of  the  instructor  are  recorded  and  accessible  through  ICT.  The  time  we  save 
by  not  giving  lectures  anymore  is  now  for  learners.  They  can  discuss  face-to-face  their  learning  moments,  ideas 
with  other  learners  or  experts.  Learners  prepare  their  own  subjects,  they  have  to  use  their  own  hunger  to  learn  to 
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motivate  their  fellow  learners,  they  become  their  own  instructors.  This  is  how  we  motivate  the  cooperative  use  of 
“content”  in  our  classes  to  create  “new  knowledge”  among  the  participants.  We  summarise  this  set  up  in  figure  2, 
model  2000. 


classic  lectures 

Workshops 

Commodity  knowledge 
(teachers  perspective) 

Learning  environment, 
(bodies  of  knowledge) 

Learner  groups 

Commodity  data 
(students  perspective) 

teacher 

▼a 

New  knowledge 

student 

Via 

Learners 

WWW 

instructors 

Discussion  learners 
amongst  themselves,  and 
with  instructor(s)  or 
guestspeaker(s) 

ONLY ICT 

FACE  to  FACE 

Figure  2:  model  2000,  possibilities  for  ICT  based  classrooms,  Santema  Genang,  Delft 


Objectives  of  the  2000  project: 

1 . Monitor  the  interaction/exchange  of  commodity  knowledge 

2.  Monitor  the  interaction/exchange  of  "www"-knowledge 

3.  Monitor  the  interaction/exchange  via  questions  and  discussions 

4.  Monitor  the  use  of  references  stated  by  the  learners 

The  interaction/exchange  of  commodity  knowledge:  We  achieved  that  learners  used  the  course  content 
published  in  the  learning  environment  to  teach  the  other  learners. 

The  interaction/exchange  of  **www**  knowledge:  The  learners  did  look  for  information  outside  our  WebCT- 
based  environment.  They  had  to  find  literature  on  the  topics  in  our  learning  environment 
The  interaction/exchange  of  questions:  Last  year  the  80  students  created  a huge  database  of  questions  on  the 
100  topics  in  our  course.  Each  student  had  to  post  10  questions  and  answer  10  other.  We  were  surprised  of  the 
level  of  the  questions.  Actually  the  students  really  asked  very  difficult  questions  and  the  answers  were  also  good 
to  very  good.  They  used  examples  out  of  multiple  media  resources  to  substantiate  their  answers.  In  the  end  each 
student  could  look  up  their  posed  questions  and  read  the  given  answer.  We  would  like  to  optimise  this  "self- 
examination"  system  more  and  we  will  create  a database  where  students  can  look  up  last  years  questions  and 
answers  by  topic.  An  interesting  experience  we  have  with  questions  posed  by  students  is  our  Frequently  Given 
Answer  (FGA)  list.  Instead  of  making  an  Frequently  given  Answer  list,  which  is  actually  not  a personal  way  of 
communicating,  we  are  able  to  personalize  every  answer  we  give  to  question  of  a student  by  copy-pasting  the 
right  answer  from  our  FGA-list  to  his  question  in  a personalized  e-mail.  The  students  like  this  way  of 
communicating  because  they  feel  they  get  a personalised  answer  to  their  very  important  question. 

Monitor  the  use  of  references  stated  by  the  learners:  The  more  a learner  shares  his  knowledge  or  uses  one  of  the 
earlier  described  ways  to  offer  “new  knowledge”  to  his  fellow  learners,  the  more  points  he  will  get.  In  order  to 
reward  our  students  on  their  information  and  knowledge  sharing,  we  have  to  monitor  the  use  of  references  they 
stated  in  their  new  knowledge;  this  could  be  a reference  to  each  other.  So  the  graduation  principle  is  based  on 


1506 


sharing  knowledge  instead  of  reproducing  it.  At  this  point  we  have  no  idea  how  we  can  monitor  this  and  how 
much  time  an  instructor  has  to  spend  in  this  process,  but  we  think  it  is  worth  trying. 


Summary  of  the  expected  results  of  the  2000  project 

The  project  “How  we  make  our  learners  instructors”  will  make  better  learners,  learners  who  learn  to  educate 
themselves  and  to  educate  each  other,  and  learners  who  know  how  important  knowledge  sharing  can  be  in 
obtaining  better  results.  Especially  this  last  point  is  a very  big  issue  in  companies  around  the  world:  What  to  do 
with  knowledge. 

As  the  course  will  take  place  in  March  and  April  2000,  the  results  will  readily  be  available  at  Ed-Media  2000. 
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Abstract:  Due  to  the  steadily  growing  interest  in  Austrian  Literature,  we  had  initiated  an  Open 
Distance  Learning  (ODL)  course.  For  this  purpose,  we  build  a framework  that  allows  to  record  the 
data  required  for  publication  in  a highly  generic  form.  The  aim  was  to  separate  data  from 
presentation,  to  prevent  information  loss  and  to  gain  flexibility  for  future  use  of  the  data  in  other 
(scientific)  projects.  We  decided  to  store  data  in  a database  using  a GUI  for  easy  access.  The 
database  is  then  transformed  into  XML  using  Java  and  DOM/SAX  libraries  for  flexible  generation 
of  different  presentation  formats  like  HTML  and  PDF.  This  guarantees  usability  in  other  contexts. 
With  this  approach,  we  anticipate  the  possible  direct  use  of  XML  using  XSL  in  the  future. 


Basic  Ideas 

Experiencing  a steadily  growing  interest  in  Austrian  Literature,  Prof.  Zelewitz,  German  language  and  literature 
scientist  at  the  University  Salzburg,  initiated  an  Open  Distance  Learning  (ODL)  course  in  cooperation  with  a team 
of  technicians  in  Vienna.  A set  of  basic  ideas  were  taken  into  consideration  when  starting  the  joint  project: 

Storage  of  generic  information  instead  of  direct  publishing  was  required  for  having  the  option  to  reuse  the  data  for 
future  projects.  So  a highly  generic  representation  of  the  data  with  a minimal  loss  of  information  was  pursued. 
Furthermore,  for  the  processing  of  the  data,  platform-independent  tools  were  preferred  (Java,  XML,  DOM,  JDBC). 
Metadata  initiatives  like  Dublin-Core  and  RDF  should  be  considered  for  easy  interoperability  with  other  information 
systems  (Lassila  1997,  DublinCore  1997). 

Besides  this  set  of  conditions  further  requirements  for  our  co-operative  work  are:  The  two  teams  are  located  in  two 
cities  (Salzburg  and  Vienna)  and  therefore  a good  concept  for  information  exchange  and  documentation  is  needed. 
Additionally  colleagues  from  the  Austrian  Ministry  of  Science  and  from  institutes  of  other  european  countries  were 
interested  in  information  exchange.  Hence  both  results,  discussions  and  also  technical  information  and  instruction 
into  new  software  tools  or  database  interfaces  had  to  overcome  a long  distance. 
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Generic  Concept 


In  our  project  it  was  our  goal  to  avoid  common  problems  of  poor  planning,  i.e.:  Loss  of  information,  Low  flexibility 
(reuse  of  data  is  not  possible  without  a huge  effort),  Layout  and  content  get  mixed  up  (keeping  a consistent  layout  in 
the  whole  website  is  a serious  issue).  We  decided  to  use  a generic  data  concept,  where  Information  is  structured  and 
entered  into  a relational  database  management  system  (RDBMS);  We  try  to  collect  and  structure  not  only  on  the 
surface  “hard”  structured  data,  but  even  “soft”  data  like  lecture  texts.  Following  this  approach  we  avoid  the 
mentioned  problems  and  it  is  again  easier  to  add  meta-information  (metadata)  like:  keywords,  subjects,  abstracts, 
links,  etc.  and  maintain  the  data  (like  URLs,  image-links,  and  so  on).  The  next  step  is  the  generation  of  XML  files, 
which  contain  the  information  for  publication  and  the  meta-information.  The  XML  data  is  then  used  to  generate  the 
files  for  publication  (HTML  and  PDF). 


Data  Structures 

Besides  some  “free”  elements  the  data  in  our  project  could  be  divided  into  a set  of  main  components,  namely 
Biographical  Articles,  the  “main”  information  for  distant  learning:  Lectures,  multimedia  elements,  original  literature 
(Citations,  Full  Text)  and  an  index. 


Technical  Details 

We  decided  to  use  Microsoft  Access  97  as  database  system.  At  the  moment  there  is  no  need  for  a multi-user  solution 
so  Access  is  used  as  frontend  as  well  as  for  data  storage. 

We  recognized  (beside  others)  the  following  advantages  of  XML  for  our  project:  XML  is  a text- format,  hence 
platform  and  system  independent  (Bray  1 996).  This  is  an  important  factor  to  consider  because  we  want  to  emphasize 
durability  in  the  storage  of  information.  XML  offers  the  possibility  to  store  all  our  data  without  a loss  of  information 
as  well  as  handling  flexibility  which  allows  an  easy  publication  of  the  XML  data. 

The  optimal  way  of  publication  would  be  using  XML  directly!  This  seems  possible  as  a part  of  the  XML 
standardisation  efforts  is  dealing  with  style-languages,  particularly  DSSSL  and  XSL.  The  conceived  method  is  to 
keep  the  data  without  information  loss  in  XML  and  publishing  this  data  “through”  a style  “filter”.  The  problem  is, 
that  at  the  moment  neither  Netscape  Navigator  nor  Internet  Explorer  are  capable  to  perform  this  task  in  a stable  way, 
but  it  should  be  possible  with  the  next  browser  generation. 

The  conversion  problem  can  be  divided  (at  least)  into  two  sub-problems:  The  building  of  the  XML  files  out  of  the 
database  and  publishing  the  information  as  HTML  and  PDF.  As  Java  has  adopted  XML  as  a kind  of  standard  data 
format,  lots  of  free  XML  parsers  of  high  quality  are  available  and  also  the  database  linking  using  JDBC  (Java 
Database  Connectivity)  works  reliable,  we  decided  using  Java  as  platform  independent  scripting  language  (Sun 
1999,  Leventhal  1998).  So  the  XML  file  creation  as  well  as  the  HTML/PDF  publication  were  done  using  Java 
programs  and  conventional  libraries  like  JDBC,  LotusXSL,  ProjectX  and  others. 

For  technical  details  please  take  a look  at:  http://www.ifs. tuwien.ac.at/'^aschatt/pubs/ed_media2000/paper.html. 
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Abstract:  This  paper  outlines  a multimedia  based  and  telecommunication  oriented  approach  to  a 
case  study  concept.  The  scenario  consists  of  different  phases  supported  by  online  computer 
conferencing  and  offline  groupware  tools.  It  brings  together  technical  and  didactical  concepts  to 
overcome  educational  lacks  of  traditional  learning  scenarios  and  to  achieve  active  learning 
processes.  This  research  work  is  part  of  a project  to  develop  teleteaching  reference  systems 
supported  by  the  DFN-Verein  (Association  for  Promoting  a Gennan  Research  Network). 


Introduction 

In  higher  education  at  universities  the  transfer  of  how  to  apply  knowledge  is  not  satisfying.  The  challenging 
combination  of  learning  and  practice  is  missing  in  many  educational  scenarios.  Case  studies  are  known  as  elements 
of  a successful  and  efficient  learning  environment  using  special  didactical  strategies.  In  combination  with  new 
telecommunication  technologies  the  case  study  concept  leads  to  a modem  educational  arrangement,  which  we  call 
“telecooperative  case  study”. 

Active  Learning  with  Case  Studies 

As  one  of  the  major  aims  in  pedagogy,  active  learning  has  its  steady  place  in  the  learning  process  (Euler 
1992).  To  create  effective  case  studies,  the  contents  have  to  be  chosen  from  real  situations.  To  match  this 
requirement  cooperation  agreements  with  companies  have  to  be  formed.  Companies  provide  up  to  date  and  real  life 
problems  which  the  students  are  supposed  to  solve  working  together  in  small  teams  (~  5 to  8 students  in  each  team). 
This  kind  of  learning  is  active  because  the  students  need  to  collect  information,  find  solutions  and  present  their 
results.  The  didactical  and  pedagogical  approaches  of  distributed  case  studies  are  based  on  principles  of  the 
curriculum  theory,  which  determine  the  choice  of  contents  within  a case  study.  The  principle  of  active  learning 
closes  the  gap  between  the  case  study  concept  and  other  concepts  of  learning,  e.g.,  learning  by  discovery  or  learning 
by  doing.  A combination  of  face-to-face  and  computer  mediated  communication  is  used  to  create  synchronous  and 
asynchronous  communication  scenarios.  Groupware  tools  offer  the  possibility  to  control  contents,  time  and  sequence 
of  the  computer  mediated  communication. 

Technical  Approach 

The  technical  concept  of  telecooperative  case  studies  supports  a wide  range  of  different  synchronous  and 
asynchronous  teleteaching  scenarios.  Flexibility  and  scalability  are  offered  to  all  participants.  On  the  one  hand 
telecooperative  case  study  environments  have  to  meet  high  quality  standards  that  are  necessary  to  display  all  types 
of  teaching  media  without  loss  of  didactical  effect.  On  the  other  hand,  for  reasons  of  cost  effectiveness  and 
flexibility  in  terms  of  location,  it  makes  sense  to  use  as  much  of  the  infrastructure  as  possible.  At  the  University  of 
Erlangen-Nuremberg  a technical  infrastructure  has  been  set  up  for  multiple  kinds  of  access  (Klein  1999).  The 
infrastructure  supports  low  end  solutions  based  on  the  Internet  protocol  TCP/IP,  medium  quality  solutions  using  one 
or  several  ISDN  channels  and  high  end  solutions  like  transmissions  over  ATM  (Asynchronous  Transfer  Mode)- 
Codecs,  which  use  the  B-WiN  network  (Broadband  Research  Network  of  Germany)  (Hoffmann  1996).  The 
technical  infrastructure  of  the  telecooperative  case  study  concept  has  to  support  the  didactically  coordinated  phases 
of  each  case  . A single  case  study  consists  of  six  phases  (Kaiser  1983).  In  the  first  phase  an  overview  of  the  case  is 
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given.  An  external  presenter  explains  the  important  aspects  of  the  problem  and  provides  the  students  with  additional 
information.  During  the  second  phase  the  students  use  different  information  sources  which  they  discover  by 
themselves  or  by  the  help  of  tutors.  In  the  following  phase  each  student  develops  an  individual  solution.  As  the 
result  of  the  resolution  phase  a team  solution  has  to  be  selected  by  each  group  of  students.  The  teams  present  their 
solutions  during  the  disputation  phase.  Finally  the  students  and  the  external  presenter  compare  the  team  solutions 
with  the  real  life  solution  closing  up  the  case  study.  Each  phase  is  supported  by  the  telecommunication  and 
telecooperation  infrastructure.  A more  detailed  explanation  of  the  realized  multimedia  infrastructure  at  the 
University  of  Erlangen-Nuremberg  is  given  in  (Bodendorf  et.  al.  1996,  Grebner  1997). 

Synchronous  and  Asynchronous  Support  Tools 

To  support  synchronous  presentation  and  videoconferencing  at  the  university  as  well  as  on  the  company 
side  a multimedia  managing  terminal  has  been  developed.  With  a special  pen  and  a touch  screen  the  teacher  can 
write  and  draw  as  he/she  usual  does  on  paper.  The  editing  on  the  screen  is  displayed  by  a local  beamer  and 
transmitted  to  remote  access  points  via  the  network.  The  teacher  can  prepare  computer-based  presentations  with 
animations,  software  demonstrations  as  well  as  hyperlinks  to  web  pages.  Annotations  and  other  presentation 
material  developed  during  a lesson  can  be  saved  and  reused  later.  For  distance  education  scenarios  with  a high 
degree  of  interaction  the  teleteaching  terminal  is  equipped  with  feedback  lamps  beside  the  control  monitors.  If  a 
remote  participant  has  a question,  he/she  indicates  this  through  the  feedback  lamp. 

During  the  asynchronous  phases  of  the  case  study  the  participants  use  information  retrieval  tools  (e.g.  Web- 
browsers)  and  Groupware  tools  to  create  the  team  solutions.  To  support  asynchronous  cooperation  a multimedia 
repository  is  provided.  The  repository  works  on  a database  and  integrates  recorded  lectures  together  with  additional 
learning  materials,  e.g.,  presentation  slides,  graphics,  animations,  lecturer’s  annotations  or  Web  links. 

Conclusion 

The  concept  of  telecooperative  case  studies  offers  various  starting  points  to  improve  educational  efforts. 
The  students  benefit  from  this  form  of  teaching  because  they  gain  real  problem  solving  abilities.  The  flexibility  and 
success  of  this  educational  arrangement  is  strongly  increased  by  adding  telecommunication  technology. 

In  the  near  future  attention  is  being  paid  to  new  stages  of  network  technology.  The  research  on 
telecooperative  case  studies  over  broadband  networks  is  part  of  a research  project  supported  by  the  DFN-Verein 
called  “ Teleteaching/Teleleaming  Reference  Systems  and  Service  Center  within  the  German  Research  Broadband 
Network”.  Within  the  bounds  of  the  project  different  technological  aspects  of  distributed  teaching  and  learning  are 
integrated  and  reference  systems  are  being  developed.  The  service  center  supports  and  evaluates  synchronous  and 
asynchronous  teleteaching  scenarios,  e.g.  telecooperative  case  studies. 
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Introduction 

The  Seminar  in  Business  Communication  course  was  piloted  as  the  first  On-Line  MBA  course 
offered  at  the  University  Of  Wisconsin-Whitewater  in  the  Fall  of  1997.  The  College  of  Business  & 
Economics  has  made  a commitment  to  offer  all  courses  necessary  to  complete  the  MBA  degree  on-line  in  a 
two-year  period.  The  growth  of  the  program  coupled  with  the  required  availability  of  courses  to  enable 
students  to  earn  their  degree  in  two  years  has  resulted  in  1 2 courses  scheduled  to  be  offered  on-line  in  the 
Spring  of  2000.  Nine  more  courses  are  scheduled  in  the  Fall  of  2000. 

In  addition  to  the  On-Line  MBA  program,  UW-Whitewater  is  partnering  with  the  University  of 
Wisconsin-Stout  Hospitality  and  Tourism  Department,  Nottingham  Business  School,  at  Nottingham  Trent 
University  in  the  UK,  the  School  of  Hotel  and  Restaurant  Management  at  Oxford  Brookes  University, 
Oxford,  UK,  and  the  University  of  Paderbom  in  Germany  to  form  an  international  partnership  to  offer  a 
new  concentration  in  Global  Hospitality  Management  for  the  Hospitality  and  Tourism  Masters  Degree 
which  began  in  Fall,  1999. 

We  believe  the  On-Line  MBA  is  quickly  becoming  recognized  as  an  acceptable  method  of  earning 
a quality  education  at  an  AACSB  accredited  school.  The  increased  acceptance  of  on-line  courses  will 
continue  to  increase  demand  for  our  On-Line  MBA  program  into  the  foreseeable  future.  The  following 
case  study  outlines  the  process  used  at  the  University  of  Wisconsin-Whitewater  to  plan,  develop,  and 
implement  the  On-Line  MBA  program. 

Defining  the  customer 


The  benefits  of  distance  education  are  well  documented.  The  most  often  cited  benefits  are  the 
ability  to  break  down  the  geographic  and  time  barriers  required  in  traditional  classes.  When  defining  your 
distance  education  customers,  you  must  analyze  their  needs  and  combine  those  needs  with  your  school’s 
ability  to  meet  those  needs.  To  excel  in  on-line  courses,  students  must  possess  minimal  technical  skills  or 
the  ability  to  learn  these  skills  quickly  and  become  self-sufficient  and  self-motivated  learners. 

Recruitment 

Recruitment  techniques  for  on-line  programs  range  from  contacting  current  and  past  students  to 
placing  banner  ads  on  the  Internet.  An  adequate  advertising  budget  is  required  for  a successful  on-line 
program.  Tough  decisions  must  be  made  on  how  best  to  use  limited  monies  to  recruit  the  most  students 
possible  who  possess  the  capabilities  to  succeed  in  an  on-line  program. 

Software 

One  strong  force  to  push  the  on-line  programs  is  the  popularity  and  gradual  maturity  of  the  World 
Wide  Web.  To  consolidate  information  sharing,  scheduling,  threaded  discussion,  e-mail,  quizzes, 
grad  ebook,  performance  and  participation  tracking,  and  other  teaching  and  learning  activities,  more  and 
more  integrated  Web-based  learning  systems  have  become  available.  At  this  point  in  time,  the  major 
contenders  in  the  academic  market  are  WebCT,  Web  Course  in  a Box,  Courseinfo,  and  Learning  Space. 
Each  of  the  solutions  has  strengths  and  weaknesses,  and  none  is  perfect  for  everybody  to  use.  As  the  cyber 
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evolution  continues,  we  advise  that  you  evaluate  them  and  weight  the  pros  and  cons  for  your  situation,  and 
select  the  system  that  best  fits  your  program  development  needs,  considering  the  scalability  and  technical 
support  accessible  to  the  local  as  well  as  the  remote/on-line  users. 

Hardware  and  Network  Infrastructure 

It  is  essential  that  both  faculty  and  students  should  have  access  to  adequate  computer  systems  to 
handle  the  demands  of  the  software  solutions.  For  those  students  and  faculty  who  do  not  have  access  to  the 
Internet  via  local  area  networks,  a 56K  modem  is  the  minimum  requirement  to  get  on-line. 

Uninterrupted  network  connections  and  communication  channels  are  the  key  to  maintain 
continuous  interactions  among  faculty  and  students.  The  site  where  the  servers  reside  should  have  at  least  a 
T1  (1.5Mbit  date  throughput  rate)  line  to  go  to  the  outside  world.  If  more  and  more  streaming  video  and 
audio  are  requested  to  distribute  course  content  and  relevant  materials,  then  multiple  T1  lines  or  even  faster 
communication  lines  (e.g.,  T3)  should  be  considered.  Carefully  planned  switching  Ethernet  may  be 
implemented  to  relief  the  bandwidth  requirement  for  the  on-campus  network  traffic.  Faster  and  more 
robust  application  and  Web  servers  also  add  importance  to  a reliable  and  speedy  communication  line. 

Teacher  acceptance/resistance  to  change 

Resistance  to  change  is  an  extremely  powerful  force  in  any  organization.  Many  people  fear 
change  for  personal  and  political  reasons.  The  faculty  must  adopt  on-line  programs  as  their  own  if  the 
program  is  to  be  successful.  Training  sessions,  workshops,  individual  faculty  support,  and  peer  support 
network  are  required  to  gain  acceptance  of  the  program.  Most  successful  on-line  programs  have  started 
with  a small  core  of  enthusiastic  faculty  who  were  willing  to  overcome  the  start-up  obstacles  present  when 
launching  any  on-line  programs.  Once  this  core  of  faculty  is  able  to  demonstrate  successful 
teaching/leaming,  other  faculty  will  gradually  follow  to  form  the  critical  mass. 

Technology  Training 

Student  and  faculty  training  are  critical.  One  of  the  most  common  student  complaints  is  the 
frustration  with  using  distance  education  tools.  Student  and  faculty  training  may  take  the  form  of  a video 
presentation,  on-line  presentation,  small  group  training  workshops,  or  individual  training  via  the  phone  or 
Internet. 

Student  Learning 

Effective  distance  learning  depends  on  the  presence  of  self-directed  learning  abilities  of  the 
students  which  can  be  considered  both  a component  and  result  of  distance  education  classes.  While  virtual 
classes  require  independent  thinking  and  exploration,  community  interaction  is  equally  important. 

A question  related  to  the  level  of  student  interaction  is  that  of  asynchronous  and  synchronous 
instruction.  One  major  drawback  of  virtual  classes  is  the  isolation  of  students.  Synchronous 
communication  can  help  overcome  isolafion  and  allow  students  immediate  feedback  from  faculty  and  other 
students. 

A third  question  pertains  to  the  level  of  open-ended  learning  vs.  directed  learning  in  distance 
education  courses.  Faculty  dependence  on  one-way  communication  in  distance  education  systems,  tends  to 
promote  transactional  distance  between  students  and  faculty  creating  little  more  than  a correspondence 
course.  Faculty  must  learn  to  be  the  facilitators  of  learning,  providing  students  with  small  modules  of 
directed  instruction  to  be  followed  by  open-ended  learning  opportunities  for  students. 

Finally,  when  students  are  first  exposed  to  the  non-linear,  individual-mediated  learning 
environments  created  using  distance  education  tools,  they  are  often  confused  concerning  how  to  interact 
with  the  technology.  Many  students  also  require  training  on  how  to  develop  independent  learning  skills. 
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Introduction 

The  fast  evolution  of  today  world  impels  corporations  to  provide  their  employees  just-in-time  training,  adapted 
training  and  more  generally  continuous  education.  To  be  able  to  satisfy  this  demand,  authors  need  means  to  specify 
once  didactic  material  and  then  to  reuse,  select,  adapt  and  distribute  it  to  different  users  in  different  contexts. 

Since  September  1998,  Mediatech  is  developing  the  NURAXI  multimedia  research  platform  aimed  at  the  design, 
generation,  deployment,  management  and  use  of  intelligent  distance  learning  environments.  NURAXI  interacts  with 
the  user  on  the  basis  of  the  competency  assessment  (initial,  on-going  progress  and  final),  the  individual  learning 
style  and  collaborative  learning.  Our  solution  is  departing  from  a document  type  based  organization  of  courses  and 
training  material  towards  a functionality  and  competency  based  model.  We  consider  models  and  structures  for 
information,  knowledge  and  competencies  more  appropriate  to  the  new  on-line  delivery  environment  than  the 
document-based  old  one  (html-based  or  not).  From  a technological  point  of  view,  this  was  made  possible  by  the 
recent  arrival  of  XML  technology,  coupled  with  Java  and  Web  application  technology. 


Services,  Actors,  and  Core  Structures  in  NURAXI 

NURAXI  is  a flexible  platform,  built  in  such  a way  that  the  borderline,  which  separates  tasks  done  by  humans  from 
tasks  done  by  machines,  is  a moving  one  depending  on  the  context  and  on  the  people  needs.  We  use  the  notion  of 
service  to  model  this  moving  frontier.  You  can  see  services  in  NURAXI  as  points  where  humans  can  take  the 
control  over  the  system  in  order  to  inject  more  sophisticated  information.  At  the  same  time,  these  services  release 
authors  and  learners  from  complex  tasks.  For  instance,  an  author  can  decide  to  select  solely  the  competencies  he/she 
wants  to  teach,  letting  NURAXI  decide  the  pedagogical  activities.  Alternatively,  the  author  can  decide  the 
competencies  and  precisely  specify  a path  through  pedagogical  activities. 

NURAXI  supplies  many  services  including:  effective  individualized  courses  based  on  competency  models, 
ontologies  for  documents,  learning  styles,  teaching  strategies,  adaptive  interfaces,  and  dynamic  computation 
according  to  previous  actions;  dialogue  and  communication  support  in  groups;  and  didactic  material  creation  using 
DTD  as  templates  coupled  with  XML  and  XSL. 

Different  actors  interact  with  the  platform:  the  author  of  the  didactic  material,  the  student,  the  guest,  the 
administrator,  the  librarian,  and  the  tutor.  The  two  main  actors  considered  hereafter  are  the  author  and  the  student. 

The  NURAXI  platform  architecture  is  composed  of  various  modules  corresponding  to  actors  and  functionalities 
involved  in  the  learning  process.  Every  action  and  interaction  in  this  process  occurs  around  the  same  core  elements 
that  form  both  the  basic  structures  of  the  teaching  material  and  the  basic  infrastructures  for  the  learning  process.  The 
author  and  the  student  use  these  common  structures  in  a different  way.  The  main  structures  introduced  in  NURAXI 
include:  competence,  knowledge,  course,  pedagogical  activities,  contents  and  student  model. 

A competence  is  the  ability  to  do  something  well  or  effectively.  In  particular,  in  our  pedagogical  context, 
competence  is  an  abstract  concept  which  can  be  reified  through  attributes  or  properties  that  qualify  and  quantify  the 
ability.  Within  NURAXI  competencies  are  classified  according  to  the  domain  area  (e.g.  Computer  Science, 
Languages,  Mathematics)  and  to  Bloom's  learning  outcomes  (Knowledge,  Comprehension,  Application,  Analysis, 
Synthesis  and  Evaluation)  [WestEd  1999],  A competence  is  connected  to  other  structures,  including  the  knowledge, 
and  activities.  When  actuating  a competence  to  perform  a task  an  individual  makes  consciously  or  unconsciously  use 
of  a certain  amount  of  information  and  knowledge, 

A course  is  defined  by  an  author  as  a list  of  learning  objectives  which  aim  to  a particular  pedagogical  goal.  The 
learning  objectives  are  the  same  for  all  the  students  attending  the  same  course;  therefore  they  may  represent  a 
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predefined  shareable  set  of  learning  objectives.  Thanks  to  this  nature,  a course  can  be  certified,  assuring  in  this  way 
the  individual  ability  in  the  specific  field.  The  course  structure  contains  information  such  as  the  list  of  competencies 
that  must  be  the  training  target  for  the  student;  the  audience  to  whom  the  course  is  addressed;  a description  of  the 
certification  that  is  obtained;  the  authors  of  the  material,  and  the  creation  and  last  updating  dates. 

Pedagogical  activities  are  associated  with  each  learning  objective  that  they  aim  to  reach.  At  the  highest  level, 
activities  have  been  classified  into  two  broad  categories:  individual,  e.g.  answering,  problem  solving,  and 
collaborative  activities,  e.g.  brainstorming,  debates,  etc.  A number  of  properties  are  associated  with  each  activity  in 
order  to  specify  learning  outcomes,  learning  styles,  competence  objectives,  complexity,  and  contents. 

The  contents  represent  the  didactic  material  and  constitute  the  basic  bricks  of  the  NURAXI  platform.  The  contents 
are  used  to  generate  the  activities  presented  to  the  student,  and  may  be  used  in  different  ways  in  several  activities. 
Contents  can  be  created  by  authors  or  imported  from  external  sources  and  integrated  into  the  system  by  means  of 
meta  data.  The  innovative  idea  is  to  re-use  the  same  contents,  appropriately  filtered,  in  many  activities.  This  could 
be  easily  achievable  if  the  content  is  formalised  in  XML. 

The  student  model  is  made  of  a static  and  a dynamic  part.  The  static  part  contains  information  such  as  personal 
data.  The  dynamic  part  contains  information  about  the  initial  learning  objectives,  the  learning  style,  the  learning 
status  of  the  student,  and  the  competence  level. 


Processes 

The  author  is  responsible  for  creating  a course.  The  steps  involved  in  the  authoring  process  can  be  summarized  as 
follows:  the  author  will  select  a list  of  competencies  for  a course;  he/she  will  select  or  create  a number  of 
pedagogical  activities  that  the  student  may  undertake  to  get  that  competence,  and  create  or  integrate  the  associated 
didactic  contents.  The  authoring  process  is  supported  by  services  that  the  NURAXI  platform  provides  in  the  form  of 
visual  tools,  such  as  competence,  activity,  and  content  editors. 

The  student  uses  the  platform  with  the  purpose  to  reach  some  learning  objectives,  i.e.  to  acquire  new  competencies 
(learning  process).  These  learning  objectives  can  be  associated  to  a course  (in  this  case  a certification  can  be 
obtained)  or  be  selected  independently  from  a course  by  the  student.  The  most  interesting  aspect  of  the  environment 
is  that  the  training  path  can  be  dynamically  created.  The  training  path  is  a trail  through  pedagogical  activities  that  are 
created  on  the  fly  by  combining  various  contents.  The  path  is  determined  on  the  basis  of  the  student  learning 
objectives,  competence  level,  and  learning  style. 


Conclusions  and  Developments 

The  NURAXI  learning  environment  is  designed  to  provide  content  re-usability,  adaptability,  modularity,  and 
interoperability,  thanks  to  the  integration  of  technologies  such  as  the  XML,  DOM,  XSL,  servlets,  software  agents, 
distributed  databases,  UML  modeling  technique,  JSP  and  JAVA  programming.  Some  of  these  technologies  have 
been  already  tested  in  various  demonstrators  ([Moulin  1999],  [Cenati  and  Sommaruga  1999])  implemented  to  show 
particular  functionalities  or  potential  features  of  the  NURAXI  platform. 
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Abstract:  Three  English  courses  for  first-year  Japanese  science  majors  were  designed  to 
accelerate  reading  skills  and  acquisition  of  science  concepts  by  using  Web- based  materials.  The 
Web  materials  were  selected  especially  for  high  quality  illustrations  (graphics,  pictures, 
simulations)  that  clarified  unfamiliar  scientific  concepts.  To  evaluate  the  effect  of  such  materials 
on  students’  reading  comprehension  in  English,  an  experiment  compared  students’ 
comprehension  results  from  three  types  of  texts:  (a)  paper  text,  no  illustration  (b)  paper  text  with 
an  illustration;  and  (c)  text  on  Web  with  more  elaborate  illustrations.  The  best  comprehension 
scores  were  associated  with  Web  materials.  The  positive  effects  of  the  Web  materials  were 
especially  significant  for  the  most  abstract  concepts. 


Science  concepts  in  a foreign  language 

For  today’s  science  and  technology  students  in  foreign  universities,  English  proficiency  and  Internet  skills  are  now 
almost  as  important  as  quality  training  in  the  students’  respective  majors.  Web-based  English-language  reading 
courses  can  develop  language  and  Internet  skills,  and  simultaneously  provide  students  everywhere  with  the  most 
current  and  authentic  learning  materials  in  their  subject  areas.  However,  because  of  curriculum  structure,  these 
students  may  encounter  unfamiliar  science  concepts  as  freshman  in  their  English  courses  before  they  study  them  in 
their  departmental  programs.  Since  many  science  concepts  are  complex  and  abstract,  reading  about  them  first  in 
English  instead  of  the  native  language  may  be  difficult.  The  potential  obstacles  to  comprehension,  however,  may 
be  offset  by  the  quality  and  array  of  illustrations  available  for  almost  any  science  or  technology  topic.  Well 
designed  graphics  (including  pictures,  diagrams,  photographs,  animations,  simulations,  videos)  often  accompany 
or  link  from  science  articles  on  the  Web.  With  a little  experience,  an  English  instructor  can  assemble  a set  of 
illustrative  resources  for  any  given  topic  in  a modest  amount  of  time.  Nevertheless,  many  instructors  are  reluctant 
to  shift  to  Web  resources,  and  many  feel  that  there  is  very  little  difference  in  comprehension  effect  between 
standard  paper  texts  and  readings  on  the  Web.  Although  this  perspective  may  be  valid  for  some  types  of  reading,  it 
may  not  be  true  for  science  and  other  types  of  technical  material  (Simpson,  1994). 

What,  exactly,  is  the  effect  of  illustrative  material  on  comprehension,  especially  where  unfamiliar  science  concepts 
are  introduced?  Most  science  texts  include  illustrations  of  various  types,  clearly  with  the  objective  of  reinforcing 
difficult  ideas.  If  illustration  does  matter,  the  difference  between  standard  texts  and  the  offerings  on  the  Web  is 
staggering. 

Mental  concepts:  Naive  learners  vs.  experts 


O 
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There  is  usually  a substantial  difference  between  the  naive  learner’s  mental  representation  of  a concept  and  that  of 
the  expert.  This  differential  has  been  studied  extensively  (e.g.,  Kumar,  Helgeson  & White,  1994;  Larkin,  1983; 
McClosky,  1983)  with  a specific  interest  in  bringing  the  learner’s  understanding  into  alignment  with  the  expert’s 
understanding.  Where  concepts  are  particularly  complex  and  abstract,  this  is  not  an  easy  task.  Perhaps  as  a kind 
of  recognition  of  this  problem,  scientists  often  create  metaphoric  labels  that  conjure  up  images:  the  big  bang,  black 
holes,  string  theory  -to  name  a few  of  the  familiar  ones.  But  what  these  metaphors  represent  to  experts  is  quite 
distinct  from  what  they  represent  to  naive  learners.  So  to  further  clarify  ideas  experts  also  include  diagrams  and 
other  graphics  in  instructional  material.  If  graphic  elaboration  of  abstract  concepts  facilitates  learning 
substantially,  then  the  offerings  on  the  Web  easily  justify  the  use  of  Web-based  materials. 

Testing  the  effect  of  Web  materials  on  comprehension 

To  test  the  effects  of  illustration  on  comprehension,  freshmen  majors  in  science  fields  in  a Japanese  university  were 
presented  with  three  types  of  English  texts  in  rotation:  (a)  text  only;  (b)  text  with  picture;  (c)  Web-based  text  with 
selected  and  varied  illustrations  (e.g.,  simulations,  graphics,  photos).  For  each  timed  reading  (e.g,.  Black  Holes) 
one  group  read  (a)-type,  another  group  read  (b)-type,  and  a third  group  read  (c)-type.  The  group  that  read  (a)-type 
for  the  first  topic  rotated  to  (b)-type  for  the  second  topic,  and  (c)-type  for  the  third  reading,.  In  this  way,  all  groups 
read  three  separate  topics  in  three  separate  formats.  Students  indicated  by  1-5  rankings  on  their  test  sheets  the 
level  of  difficulty,  whether  the  topic  was  new  or  familiar,  and  whether  the  topic  was  interesting.  Comprehension 
for  each  reading  was  tested  by  multiple  choice  questions  which  targeted  the  respective  concepts. 

The  results  were  most  interesting  where  readers  were  most  naive  and  topics  were  most  abstract.  For  the  real 
novices,  comprehension  was  substantially  better  for  the  Web-based  material  than  for  either  of  the  standard  written 
materials,  that  is,  (a)- type  or  (b)-type.  Where  readers  were  more  familiar  with  the  target  concept  the  differential 
was  not  so  great.  This  initial  investigation  suggests  that  the  Web  materials  were  most  helpful  in  presenting 
challenging  concepts  to  naive  readers.  Further  refinements  of  the  experiment  are  underway,  which  include  a larger 
set  of  readings  at  two  levels.  This  will  help  to  clarify  the  potential  of  Web-based  materials  for  accelerating 
comprehension  at  different  reading  levels;  that  is,  for  both  the  novice  and  for  students  who  already  have  some 
familiarity  with  a given  concept. 
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Abstract:  The  focus  of  this  paper  is  on  the  preliminary  findings  of  research  into  the 
perceptions  and  experiences  of  educational  technologies  of  teaching  staff  and  students  at 
Sheffield  Hallam  University  in  the  UK.  These  findings  explore  the  relationship  between 
educational  technologies  and  pedagogic  processes  as  well  as  wider  social  and  cultural 
aspects.  Initially  this  research  stresses  the  importance  of  an  ongoing  dialogue  between 
teaching  staff,  students  and  university  management  concerning  the  appropriate,  context 
specific  use  of  educational  technologies,  as  unconsidered  approaches  on  the  part  of 
practitioners  can  impact  negatively  on  the  quality  of  the  student  learning  experience. 


Introduction 

As  a result  of  research  commissioned  by  the  Learning  and  Teaching  Institute  at  Sheffield  Hallam 
University  (SHU),  this  paper  will  present  the  initial  findings  arising  from  research  into  perceptions  and 
experiences  of  staff  and  students  using  educational  technology  as  part  of  their  learning  and  teaching  at 
SHU.  The  primary  objective  of  the  research  is  to  gain  a qualitative  insight  into  the  ‘real  lived’  learning 
and  teaching  experiences  of  students  and  teaching  staff  using  educational  technologies  as  part  of  their 
educational  life  at  Sheffield  Hallam  University.  Fundamental  to  this  task  is  an  investigation  into  social 
and  cultural  factors  that  shape  their  perceptions  of  educational  technology  and  the  pedagogic  processes 
of  teaching  and  learning  within  the  framework  of  specific  learning  contexts. 


Methods 

The  first  key  stage  of  the  research  was  based  around  interviews  of  members  of  teaching  staff  and 
centred  upon  uncovering  and  exploring  learning  and  teaching  within  given  contexts.  These  interviews 
were  structured  so  as  to  allow  the  participant  to  describe  in  an  open  ended  manner  their  experiences 
and  perceptions  of  educational  technology.  The  themes  on  which  the  participants  were  encouraged  to 
talk  included  notions  of  value  of  educational  technology  in  learning  and  teaching;  perceptions  of 
changing  roles  and  relationships  with  students;  views  on  changes  within  higher  education  and  finally, 
perceived  negative  aspects  of  educational  technology  use.  The  findings  of  this  section  of  the  research 
point  to  a multiplicity  of  perceptions  of  educational  technology.  However,  the  overwhelming  theme 
that  emerged,  was  that  considered  and  appropriate  use  of  learning  technologies  should  be  encouraged 
as  the  quality  of  student  learning  can  be  enhanced  and  enriched.  It  was  considered  that  the  view  that 
educational  technologies  as  a ‘quick  fix’  solution  to  the  problem  of  student  numbers  should  be 
challenged  as  this  can  only  demean  the  potential  of  educational  technologies  within  the  learning  and 
teaching  process. 

The  second  stage  of  the  research  is  primarily,  but  not  exclusively,  focussed  on  the  learning  experiences 
and  perceptions  of  students  at  SHU,  with  reference  to  the  use  of  educational  technologies.  Five  separate 
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case  studies  or  ‘contexts’,  which  seek  to  investigate  different  elements  of  the  teaching  and  learning 
from  the  perspective  of  the  student  have  been  developed.  These  research  contexts  are  being  formulated 
in  order  to  investigate  particular  aspects  of  the  teaching  and  learning  experience  of  the  student  body. 
These  contexts  are  centred  around  the  following  areas: 

♦ Web  Based  Material,  ‘Enriching’  the  Student  Learning  Experience. 

♦ Tackling  the  Issue  of  Student  Motivation  with  Educational  Technology:  An  Action  Research  Model. 

♦ Student  Expectations 

♦ Reflections  on  Computer  Conferencing  from  Lecturers  as  Students. 


Conclusion 

The  material  from  the  interviews,  coupled  with  the  four  research  contexts,  provide  a detailed 
introduction  to  a number  of  issues  that  face  students  and  lecturers  when  engaging  with  educational 
technology  in  a learning  and  teaching  context.  The  data  that  will  be  produced  from  this  research  will 
provide  an  insight  into  how  students  and  lecturers  deal  with  educational  technology,  in  terms  of  their 
experiences  and  perceptions  of  educational  technolgy’s  roles  and  functions;  as  well  as  how  it  is 
incorporated  into  pedagogic  processes.  Finally,  an  understanding  of  the  staff  and  student  perspective  of 
educational  technology  in  relation  to  wider  culture  is  generated  from  this  research. 
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Abstract:  As  computers  becomes  more  prevalent  in  the  field  of  education,  educators  are  obligated 
to  plan  and  provide  for  classroom  of  computer  technology.  The  combination  of  media  and 
technology  offer  the  possibility  of  developping  a computerized  virtual  learning  model.  This  paper 
describes  a framework  to  develop  Web-based  instruction  models  (WBI)  to  help  the  teacher  to 
deliver  course  materials  on  site  and  at  remote.  This  WBI  prototype  will  help  the  learners  grasp  a 
rich  learning  content  and  provide  opportunities  to  do  some  collaborative  work. 


Introduction 

Web-based  instruction  could  be  defined  as  the  application  of  a repertoire  of  cognitively  oriented  instructional 
strategies  implemented  with  a constructivist  and  collaborative  learning  environment  using  the  attributes  and 
resources  of  the  World  Wide  Web  (Lebow,  1993;  Perkins,  1991).  The  teacher  becomes  a facilitator  for  finding  , 
assessing  and  making  meanings  form  the  information  discovered  from  a variety  of  media  (Relan,  Gillani,  1997).  The 
Web  can  become  a resource  center  for  the  teacher  and  the  learner. 

The  objective  of  this  paper  is  to  present  a theorical  and  practical  framework  for  creating  an  WBI  model. 

Context 

The  Canadian  Royal  Military  College,  located  in  Kinston,  Ontario,  is  an  academic  institution  which  provides 
education  to  young  Canadians  who  want  to  become  Officers  and  at  the  same  time  obtain  a University  degree. 
Various  academic  programs  in  Engineering,  Science  and  Arts  are  offered.  These  programs,  given  on  site  in 
Kingston,  are  delivered  through  traditional  means  and  methods. 

Cadet  enrollment  for  the  1997-1998  academic  year  in  the  undergraduate  programs  is  approximately  1000  students, 
from  the  age  of  17  to  25.  Graduate  programs  are  offered  to  a small  population  of  400  students.  The  college  is 
planning  to  add  certificate  degrees  to  help  people  in  the  rank  get  a University  degree. 

Military  Men  and  Officers  are  located  in  different  military  units  and  participate  in  military  missions  overseas.  They 
must  to  be  retrained  every  five  years  to  perform  with  a high  degree  of  proficiency  in  their  jobs.  The  Military  College 
will  have  to  use  distance  education  and  other  forms  of  delivery  to  be  able  to  educate  its  work  force.  A challenging 
and  rewarding  avenue  for  this  institution  to  look  into. 

The  vision  of  the  Military  College  is  to  enhance  learning  through  collaborative  educational  technological  tools  in  the 
year  2000.  Every  student  must  be  able  to  learn  at  remote  and  use  interactive  computer  tools  for  a more  enriched 
learning  environment.  This  vision  requires  this  institution  to  define  a new  mission.  The  Military  College,  the  only 
high  level  military  training  in  the  Canadian  Armed  Forces,  will  be  dedicated  to  the  creation  of  knowledge  and  the 
advancement  of  learning  in  the  Armed  Forces. 

The  mission  of  the  Canadian  Royal  Military  College  will  be: 

• To  produce  well  trained  personnel  capable  of  using  computer  tools  and  skills  in  their  working  setup. 

• To  train  professors  in  education  technology  and  distance  education  in  order  to  deliver  rich  training  content 
at  remote. 
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Identified  Need 


We  are  entering  into  the  age  of  communication  rather  than  information.  Databases  are  easily  accessible  at  a very 
minimum  cost  for  our  students.  Computerized  real-time  teaching  is  possible  due  to  a fast  network  infrastructure,  but 
we  need  the  players:  the  teachers  and  the  learners  who  could  use  these  educational  tools  efficiently  and  effectively. 

One  of  the  challenges  we  are  forced  to  face  as  educators  is  that  we  have  to  upgrade  our  teaching  skills  using 
technology.  Enhancing  learning  through  technology  requires  more  than  just  access  to  computers,  software  and 
networks,  it  also  demands  new  ways  of  teaching,  new  roles  for  educators,  new  teaching  and  learning  goals,  and  a 
strong  educational  support  system  for  the  teachers  and  students.  Householder  (1993)  produced  a vision  statement 
which  includes  four  goals  to  guide  our  work: 

• Position  technology  as  a basic  area  for  academic  teaching  and  learning. 

• Provide  leadership  in  developing  curriculum  using  technology. 

• Support  teachers  in  implementing  their  programs  using  new  teaching  methods. 

• Enhance  participation  of  students  in  experimenting  new  educational  learning  media. 

One  of  the  primary  goals  in  the  year  2000  will  be  to  support  the  mission  of  the  Military  academic  institution  through 
successful  integration  of  technology  into  the  educational  process.  The  Military  College  has  to  maintain  educational 
leadership  in  its  environment  in  order  to  receive  the  proper  funding  from  the  National  Defense  Headquarters  to 
achieve  their  mission. 

For  the  purpose  of  this  course,  new  medias  will  be  introduced  through  Internet,  which  will  facilitate,  on  one  hand, 
the  delivery  of  the  contents  by  the  professor  and,  on  the  other  hand,  the  learning  of  the  concepts  of  the  course  and 
the  successful  completion  of  the  Marketing  fundamentals  course  enrolled  by  students. 
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Abstract:  Constructivist  approaches  (Duffy  & Jonassen,  1 992,  Boyle,  1 997)  advocate 
the  provision  of  learning  experiences  which  facilitate  the  students’  learning 
acquisition,  rather  than  the  more  behaviourist  approach  (Skinner,  1 974)  of  deriving 
objectives  and  learning  outcomes  for  the  instructional  material.  However,  there  is 
both  external  pressure,  from  QAA  audits  and  internal  pressure,  from  students,  to 
identify  and  assess  learning  outcomes  and  adopt  a more  behaviourist  practice.  This 
paper  looks  at  a solution  which  seeks  to  accommodate  both  ideals. 


Background 

Constructivist  approaches  (Duffy  & Jonassen,  1992,  Boyle,  1997)  advocate  the  provision  of  learning 
experiences  which  facilitate  the  students’  learning  acquisition,  rather  than  the  more  behaviourist  approach 
(Skinner,  1974)  of  deriving  objectives  and  learning  outcomes  for  the  instructional  material.  The  constructivist 
approach  is  admirable  and  aims  to  transfer  the  emphasis  away  from  lecturer  driven  teaching,  a central  tenet  of 
behaviourism,  towards  more  student  driven  learning.  The  conflict  in  the  adoption  of  this  approach  is  derived 
from  the  lecturer’s  wish  to  facilitate  learning  and  the  student’s  wish  to  be  taught.  Students  admit  to  being 
motivated  by  assessment  and,  to  this  end,  they  perceive  a key  factor  in  their  learning  as  the  provision  of  criteria 
to  underpin  the  assessment  process.  Within  the  modules  undertaken  in  an  undergraduate  or  post-graduate 
course,  the  criteria  are  derived  as  learning  outcomes  which  are  used  to  assess  levels  of  student  performance.  In 
an  effort  to  provide  an  environment  where  more  constructivist  approaches  are  encouraged,  a web  interface  has 
been  developed  which  attempts  to  bridge  the  two  ideals.  One  of  the  aims  of  this  approach  is  to  provide  a gentle 
“nudge”  for  students  who  exhibit  a high  degree  of  dependence  on  lecturer  centred  teaching  towards  a more 
independent  student-centred  learning  experience. 

Using  existing  legacy  material  coupled  to  WWW  facilities,  amalgamating  and  delivering  them  as  resources  for 
an  individual  module,  the  lecturer  and  the  student  progress  at  a pace  which  allows  both  to  feel  secure  with  the 
innovative  approach.  Several  elements  from  the  conventional  teaching  approach,  such  as  lecture  notes  and  slide 
presentations,  can  be  combined  with  WWW  facilities,  such  as  web  pages  via  links  and  email  communication,  to 
provide  a comprehensive  learning  experience. 

Approach 

The  module  chosen  to  pilot  this  approach  was  a level  3 (final  year)  undergraduate  module  entitled  Computer 
Based  learning  (CBL).  Existing  resources  include:  Assignments  (Word  Documents);  Lecture  Notes 
(PowerPoint  slides);  Additional  notes  (Word  Documents).  To  these  were  added  WWW  resources  which 
include:  access  to  the  home  pages  of  staff  teaching  on  the  modules  (HTML);  external  Links  (URL’s);  on-line 
tutorial  systems  for  Authorware  (internal  and  external  links);  sample  CBL  material  (developed  in  Authorware, 
then  “shocked”  to  allow  its  delivery  via  the  WWW).  Email  facilities  for  students  provide  asynchronous 
communication  between  students  and  between  staff  and  students.  The  module  interface  to  the  material  is 
delivered  via  the  WWW,  is  frame  based  and  ensures  that  students  may  access  the  resources  whilst  remaining 
within  the  module  framework.  This  allows  exploration  of  the  resources  without  the  fear  of  getting  “lost  in 
hyperspace”  (Conklin,  1987). 

CBL  Module  Interface 

Figure  1 http://www.comp.glam.ac.uk/pages/stafF/gstubbs/tutorial.gif 
Figure  2 http://www.comp.glam.ac.uk/pages/staff/gstubbs/staffgif 

The  contents  links  frame  provides  selective  access  to  resources,  the  links  form  a typical  hierarchical  structure  as 
illustrated  by  the  indented  points  below: 

• Staff  on  the  module; 


• Documents  available  on  the  server; 

• Tutorial  Exercises; 

• Assignments  and  Lecture  Notes; 

• Papers; 

• Weekly  schedule  (of  lectures); 

• Authorware  Tutorial; 

• Sample  CBL  material; 

• Some  useful  links. 

This  structure  is  sufficiently  flexible  to  be  tailored  to  individual  module  and  lecturer  needs  but  is  simple  enough 
not  to  confuse  students.  The  environment  offers  the  student  access  to  the  assignments  and  hence  the  learning 
outcomes  but  also  provides  additional  learning  opportunities  such  as  on-line  tutorials  or  links  to  external 
resources.  Thus,  providing  constructivist  opportunities  for  students  to  develop  deeper  learning  and  providing  an 
extension  to  the  conventional  “chalk  face”  teaching  and  learning  activities. 

Staff  Issues 

The  implementation  of  this  interface  raised  some  interesting  issues  from  several  members  of  staff  who  viewed 
and  evaluated  the  facility.  The  two  major  issues  raised  were: 

(i)  “1  don’t  want  my  notes  to  be  an  open  access  resource  but  I don’t  mind  the  other  resources  being  open 
access” 

This  concern  can  be  easily  addressed,  solutions  include,  provision  of  resources  via  local  servers  so  that  only 
local  students  may  access  them;  protected  tutorial  groups  where  students  have  a login  name  and  password;  or  a 
combination  of  these  together  with  the  open  access  resources. 

(ii)  “1  haven’t  got  time  to  convert  my  material  to  HTML”. 

This  was  not  a problem  as  no  legacy  material  was  altered.  Staff  were  impressed  that  materials  were  accessible  in 
their  original  format  and  this  engendered  a more  favourable  attitude  to  use  of  the  facility. 

External  Demands 

An  additional  factor  influencing  the  transition  from  traditional  to  more  innovative  delivery  mechanisms  for 
module  resources  comes  from  the  external  audit  body  the  “Quality  Assurance  Agency”  (QAA)  whose  primary 
function  is  “to  review  the  performance  of  universities  and  colleges  of  higher  education”  . To  fulfil  this  role  the 
QAA  visits  Higher  Education  institutions  to  “assess  their  quality  and  standards  of  teaching  and  learning”. 
Institutions  are  rated  as  excellent,  satisfactory  or  unsatisfactory,  these  are  based  on  several  factors  but  the  QAA 
report  of  1994  stated  that  “subject  providers  judged  to  be  excellent,  and  some  others,  were  able  to  utilise  high- 
quality  resources  to  provide  innovative  and  effective  teaching  and  extensive  opportunities  for  independent 
learning  by  students”,  (QO,  1995)  this  area  is,  thus,  seen  as  an  important  factor  in  gaining  an  “excellent”  QAA 
rating. 

Conclusions 

Within  UK  Higher  Education  institutions,  there  is  a move  to  attract  higher  numbers  of  students  and  “to  widen 
access”.  Attracting  students  is  a complex  process  depending  on  location,  courses  and  entry  requirements  but  the 
additional  advantage  of  advertising  a School  as  “excellent”  certainly  confers  a benefit  over  those  rated  as 
“satisfactory”.  Students  are  also  more  familiar  with  technology  in  the  form  of  the  WWW,  email  etc.  and  their 
expectations  on  a Computer  Studies  course  extend  to  the  use  of  these  technologies  within  the  courses,  however, 
in  a learning  environment  they  often  request  the  more  behaviourist  approach  of  providing  learning  outcomes.  In 
the  UK  over  the  last  two  years  the  issue  of  the  abolition  of  student  grants  and  the  introduction  of  student  loans 
has  initiated  a “consumer  approach”  to  education,  the  onus  is  being  placed  on  staff  to  teach  rather  than  students 
to  learn.  If  this  is  continued,  it  will  provide  an  impoverished  learning  experience  for  students  in  general.  In 
order  to  accommodate  the  facilitation  of  learning  whilst  at  the  same  time  fulfilling  the  student  demand  for  both 
technological  innovation  and  provision  of  behaviourist  principles,  care  must  be  taken  to  progress  at  a pace  that 
supports  both  students  and  staff.  The  integration  of  legacy  material  within  a constructivist  environment  seeks  to 
achieve  a satisfactory  outcome  for  the  two  viewpoints. 
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Abstract:  When  undergraduate  wet  lab  experiments  are  converted  to  multimedia  simulations, 
opportunities  become  available  for  addressing  old  problems.  The  simulation  described  in  this  paper 
provides  many  of  the  same  advantages  that  other  lab  simulations  have  in  the  past.  However,  “Frog  Heart 
Physiology  Lab”  goes  further.  Lab  activities  were  redesigned  to  achieve  learning  outcomes  that  were 
previously  elusive.  The  distinguishing  features  of  the  simulation  are  described,  along  with  examples  of 
how  the  lab  is  used.  When  using  the  simulation,  students  are  encouraged  to  generate  and  apply  various 
methods  of  administering  treatments,  and  to  make  better  use  of  the  resulting  data. 


Introduction 

This  paper  discusses  a set  of  biology  laboratory  experiments  that  are  simulated  using  multimedia.  The 
simulation  is  being  used  to  replace  a wet  lab  for  a third  year  undergraduate  physiology  course.  The  “Frog  Heart 
Physiology  Lab”  simulation  shares  advantages  with  other  simulations  of  wet  labs.  It  avoids  the  cost,  effort  and 
ethical  issues  of  acquiring  live  animals  for  testing.  The  redesigned  simulation  gives  students  a look  at  the  work 
involved  in  dissecting  a frog  and  setting  up  the  experiment,  but  avoids  the  technical  problems  which  often  occur 
when  working  with  live  animals.  Simulations  also  provide  opportunities  for  students  to  take  chances  in  a safe 
environment.  In  addition,  it  allows  students  greater  flexibility  to  move  between  activities,  and  allows  them  to 
stop  working  on  the  lab  when  they  want,  and  to  return  to  the  lab  as  often  as  they  want.  Previously,  they  had  to 
do  all  of  their  work  in  one  lab  session  with  one  animal.  There  would  be  many  advantages  if  the  simulation 
simply  replaced  the  wet  lab.  However,  we  wanted  to  go  further. 

We  redesigned  the  lab  in  an  attempt  to  achieve  specific  learning  outcomes  that  were  difficult  to 
achieve  in  the  wet  lab.  The  addition  of  “matching”  tasks  enables  students  to  relate  mechanical  and  electrical 
tracings  to  the  movements  of  a beating  heart.  In  the  simulated  lab,  students  are  encouraged  to  think  critically,  by 
drawing  tracings  to  predict  treatment  effects,  and  deciding  what  is  likely  to  change,  and  by  how  much 
(McCormick,  1998).  In  addition,  students  are  asked  to  justify  their  predictions,  thereby  defining  what  they 
believe  are  the  treatment  mechanisms.  They  are  asked  to  do  this  in  an  attempt  to  dispel  naive  patterns  of 
misunderstanding  which  might  otherwise  persist.  (Perkins  & Simmons,  1988).  The  flexibility  provided  by  the 
simulated  lab  allows  these  activities  to  be  started  a week  or  more  before  the  chemical  and  electrical 
investigations  are  conducted.  This  gives  students  more  time  to  think  about  and  research  their  predictions  and 
justifications.  One  desired  result  of  these  activities  is  that  students’  misconceptions  are  drawn  out  and 
confronted.  Students  are  asked  to  compare  the  variations  they  observe  with  the  data  set  they  constructed,  that  is, 
their  drawings.  This  in  turn  encourages  students  to  explore  alternative  activities,  and  entertain  alternative  ideas. 
Another  desired  result  is  that  students  must  make  use  of  evidence  to  find  relationships  and  support  their 
arguments.  That  means  that  they  are  given  responsibility  for  discovering  “new”  treatment  effects.  This 
approach,  of  asking  students  to  make  inferences  based  on  their  observations,  rather  than  telling  them  what  to 
look  for,  has  been  used  successfully  in  the  redesign  of  other  undergraduate  biology  labs  (Udovic,  1996).  In 
general,  we  wanted  to  encourage  a higher  percentage  of  students  to  adopt  a problem  solving  approach  and  to  be 
more  “playful.” 


1524 


Tales  of  two  labs:  students  then  and  now. 


In  the  wet  laboratory,  somewhere  in  the  middle  of  the  lab  period.  . . 

‘Tracey  and  Ed  read  in  their  manual  that  they  are  to  investigate  the  effects  of  calcium  chloride 
on  heart  function.  Unlike  their  last  investigation  with  epinephrine,  they  are  not  sure  what  they 
are  looking  for.  Ed  applies  one  and  a half  droppers  full  of  calcium  chloride  directly  on  the 
heart  muscle.  After  less  than  a minute,  there  is  a noticeable  difference  in  the  movement  of  the 
heart,  and  the  mechanical  tracing.  Tracey  and  Ed  examine  the  tracings  produced  by  the 
application  of  CaC12.  In  their  lab  report,  they  note  that  calcium  chloride  reduced  the  height  of 
the  tracing,  and  there  was  an  initial  increase  in  heart  rate,  before  heart  rate  slowed.  They  go  to 
the  literature,  and  find  an  explanation  of  the  mechanisms  of  CaC12,  which  they  paraphrase 
and  add  to  their  findings.” 

In  the  computer  simulated  laboratory.  . . 

“Bill  had  written  on  one  of  the  lab  ‘prediction’  worksheets,  ‘the  roof  part  of  each  cycle 
depends  on  the  extracellular  calcium  concentration.  The  more  extracellular  calcium  there  is, 
the  more  Ca2+  will  flow  into  the  cells  and  so  the  longer  the  tracing  in  this  part.’  Bill  had 
traced  over  the  baseline  wave  on  the  worksheet,  making  the  peaks  of  his  new  waves  higher, 
and  the  valleys  between  cycles  narrower.  He  and  Marie  are  now  surprised  again  to  find  that 
the  data  generated  by  the  simulation  don’t  match  their  predictions.  Heart  rate  has  slowed,  and 
amplitude  is  reduced,  not  increased.  They  begin  to  examine  the  data  more  closely,  and  discuss 
the  discrepancy.  Marie  makes  an  observation,  ‘if  calcium  chloride  slows  down  the  heart  and 
shortens  the  height  of  the  waves,  then  it  must  be  pumping  out  less  blood.  So  less  blood  comes 
back  to  the  heart.  And  so  the  atria  move  less,  and  that  part  of  the  tracing  is  smaller,  because 
it’s  not  working  as  hard’  ...” 

Students  who  used  the  simulated  lab  usually  made  mistakes  when  they  first  attempted  the  trace 
matching  tasks.  However  these  students  stated  that  “(The  tasks)  did  help  in  determining  the  contraction  state  for 
the  chemical  additions”,  and  that  “the  exercise  helped  me  realize  the  actual  orientation  of  the  hart  during  the 
procedure.”  Students  also  stated  that  they  “particularly  liked  matching  the  video  images  with  the  tracings”,  and 
that  “one  of  the  most  important  things  learned  from  the  simulation  was  what  parts  of  the  . . . tracings 
corresponded  with  which  contractions  in  the  heart.” 

In  addition  to  mistakes  made  during  the  trace  matching  tasks,  students  stated  that  “my  predictions  were 
also  quite  wrong.  However,  I think  it  was  good  to  have  done  them,  as  it  required  learning  about  the  potential 
effects  . . . before  actually  performing  the  investigations”,  and  that  “discussing  the  tracings  with  other  students 
with  whom  I worked  was  a good  way  to  stimulate  new  questions  about  what  was  occurring  in  the 
investigations.”  Other  students  stated  that  “Predicting  before  performing  the  investigation  really  made  me  think. 
I put  much  more  effort  into  this  lab  because  of  these  instructions”,  and  that  they  “found  making  the  predictions 
very  worthwhile  because  it  made  you  sit  and  think  about  what  was  going  on  . . .” 

It  appears  that  the  simulation  created  an  environment  where  students  enjoyed  learning  from  their 
mistakes. 
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Abstract:  In  this  paper,  we  describe  situated  learning  connoting  the  importance  of 
context  to  meaningful  learning.  In  particular,  based  on  the  writings  of  authors  such  as 
Brown,  Collins,  and  Duguid  (1998),  we  draw  implications  to  instructional  design  and 
attempts  to  recommend  practical  situating  environments  as  part  of  a holistic  learning 
through  the  advent  of  technology. 


The  Situating  Environment 

John  Seely  Brown,  Allan  Collins  and  Paul  Duguid  in  their  paper  ’’Situated  Cognition  and  the  Culture  of 
Learning”  (1998)  said  that  learning  must  involve  activity,  concept  and  culture  as  they  are  interdependent.  The 
model  of  situated  cognition  is  based  upon  the  notion  that  knowledge  is  contextually  situated  and  is 
fundamentally  influenced  by  the  activity,  context,  and  culture  in  which  it  is  used.  Moreover,  such  activity 
contexts  are  mediated  by  tools.  Looi  (1998/9)  suggests  the  supportive  roles  of  technology  for  the  inquiry  process 
(within  the  activity  context)  - as  instructive  tools,  constructive  tools,  communicative  tools  and  as  situating  tools. 
The  instructive  environment  that  holds  the  content,  remains  the  central  part  of  the  learning  environment.  It 
would  start  from  the  didactic  approach  of  posting  the  domain  knowledge  to  the  learner.  As  the  learner  picks  up 
the  knowledge,  construct  his  own  knowledge  from  it,  he  returns  the  newly-constructed  knowledge  back  to  the 
system,  adding  up  to  the  knowledge  base.  Collins  and  Ferguson  (1993)  proposed  epistemic  tools  as  the  basic 
building  blocks  for  knowledge  construction  to  help  learners  to  recognize,  judge  and  organise  patterns  of 
information,  engaging  learners  in  constructive  inquiry.  In  addition,  the  communicative  dimension  of  tools  and 
environments  provides  the  hub  for  community  exchange,  providing  support  to  the  learner,  to  the  learning 
process. 

In  fostering  meaningful  learning  within  activity  context,  Figure  1 below  shows  the  processes.  There  is  a 
deliberate  effort  on  the  part  of  the  Instructional  Designer  to  provide  opportunities  for  the  learner  to  engage,  not 
only  in  the  content,  but  also  in  his  life:  activities  and  situations  of  his  life.  A situating  environment  is  thus  an 
essential  part  of  a total  learning  environment  for  the  learner.  The  application  of  meta-cognitive  skills  will  raise 
the  learner  to  a level  that  he  will  be  able  to  exercise  cognitive  dexterity  in  transferring  the  knowledge  acquired  to 
many  situations  in  his  life. 


Figure  1:  The  learner  and  his  context  for  learning 


In  the  following  sections,  we  describe  what  needs  to  be  situated,  ways  of  situating,  and  we  discuss  implications 
to  the  instructional  design  process. 

On  what  to  situate? 

Context,  Culture,  Activity  are  three  components  as  recommended  by  Brown,  Collins  and  Duguid 
(1989)  to  be  essential  in  learning  as  they  are  interdependent  on  each  other.  To  situate  the  learner  to  the 
"hardware”  and  "software"  of  the  situation,  the  lesson  should  provide  not  only  the  hard  facts  in  the  content,  but 
also  the  peripheral  knowledge.  By  "hardware",  the  author  refers  to  the  tool,  which  the  learner  is  supposed  to 
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learn  to  use.  This  could  be  an  equipment,  a machine,  a word  processing  tool  or  the  control  panel  in  a cockpit. 
By  “software”,  we  mean  the  surrounding  environment,  in  the  form  of  climate,  smell,  heat,  water,  room,  people, 
etc.  This  places  the  learner  in  the  context  and  culture  of  the  ultimate  application  of  the  lesson.  We  attempt  to 
provide  not  only  the  cognitive  situationing,  but  also  (if  possible)  the  physical,  emotional  and  mental 
situationing. 

Ways  to  situate 

Various  ways  have  been  recommended  to  situate  a learner.  The  ’’Adventures  of  Jasper  Woodbury” 
(Pellegrino,  et.  al,  1990)  is  a well-designed  videodisc  program  to  exemplify  situated  learning.  It  requires  the 
learners  to  consider  real-life  needs.  Opportunities  are  embedded  for  the  learners  to  explore,  discuss,  articulate 
possible  problems  and  solutions,  ultimately  to  ’’transfer”  his  classroom  knowledge  to  a near  real-life  situation  in 
the  program. 

To  bring  students  to  a deeper  immersion  to  the  culture,  context  and  activity,  tertiary  education  for  most 
students  (in  Singapore)  have  included  industrial  attachment  as  part  of  their  curriculum.  The  civil  engineering 
students  are  sent  to  work  for  6 months  in  the  construction  site.  Secondary  schools  send  teenagers  to  experience 
the  rigour  of  work  in  the  real  world  outside  school.  Simulators  are  developed  to  bridge  the  gap  between 
textbooks/manuals  and  the  real  environment.  In  computer-based  training,  situating  the  learner  takes  place  by 
presenting  multiple  scenarios  (including  the  use  of  immersive  and  non-immersive  Virtual  Reality)  that  trainee 
would  be  likely  to  encounter.  Variables  in  the  scenario  are  identified  and  changed  for  the  various  scenarios  to 
vary  complexity  and  to  increase  diversity.  Interactive  hardware  is  now  attached  to  desktop  computers  so  that 
the  trainee  can  be  brought  to  a step  closer  to  the  real  situation  by  touching  the  actual  equipment. 


Implications  for  Instructional  Design 

To  bring  the  discussion  to  a practical  level  for  a hypermedia  Instructional  Designer,  how  then  can  we  design 
for  situated  learning?  The  authors  suggest  the  following  considerations  appropriated  from  notions  of  cognitive 
apprenticeship. 

1.  Past  and  present  situations  and  predict  areas  of  future  application 

Presenting  past,  present  and  future  nodes  for  linking  to  the  current  point  of  learning  is  a strategy  for 
situating  to  enculturate  the  learner.  Presenting  a meaningful  relevant  past  node  provides  the  association  from  a 
new  and  unknown  area  to  be  taught  on  a known  area.  The  present  node  would  be  one  that  the  learner  is 
experiencing  and  would  require  the  new  knowledge  to  have  immediate  application.  This  should  bring  about  an 
’uh  huh”  effect  in  the  student.  A future  node  must  be  one  that  is  foreseeable  by  the  learner.  Taking  the  learner  on 
a untouchable  trip,  to  an  unreachable  destination  will  result  in  the  student,  learning  for  learning  sake,  for  the 
sake  of  satisfying  the  teachers’  demand  and  for  passing  examinations  only. 

2.  Complexity  and  diversity  and  multiple  practice  for  each  situation 

Increasing  complexity  can  be  done  by  gradually  increasing  the  variables  in  the  simulation.  In  the  use 
of  equipment,  where  knowledge  is  structured,  multiple  practice  under  different  conditions  should  be  provided, 
so  that  the  students  become  competent  in  the  job.  However,  one  must  be  careful  not  to  over-situate.  (Over- 
situating is  a condition  when  the  learner  is  provided  intensively  with  only  some  very  specific  situations,  where 
he  becomes  very  conversant  under  this  condition.  The  extreme  result  of  over-situating  is  that  the  learner  is  not 
able  to  perform  the  same  activity  given  a changed  condition.)  To  prevent  over-situating,  diversity  in  the 
situations  is  created  through  various  different  situations,  through  the  varying  of  each  dependent  parameter  for 
both  the  ’’hardware”  and  the  ’’software”.  In  the  teaching  of  wave  characteristics,  the  sequence  starts  with  the 
simple  wave  equation  of  C=  X7F.  A picture  to  represent  the  equation  is  shown.  Simulation  is  shown  of  the 
effect  of  a change  in  the  A.,  keeping  C constant,  causing  a change  in  the  F value.  The  student  is  allowed  to 
suggest  any  value  of  the  parameters  to  view  the  effects  on  the  other  parameters.  This  allows  the  student  to 
quickly  grasp  the  implications  of  the  variation.  Then,  the  complexity  is  increased  by  introducing  another  wave 
of  the  same  frequency,  followed  by  the  intervention  of  another  wave  at  different  frequency.  The  learner  would 
see  the  change  in  the  overall  effect.  To  further  increase  the  complexity,  as  in  a real-life  situation,  we  would 
show  the  attenuation  of  the  wave  due  to  interference  caused  by  obstacle  such  as  a tall  building  or  the  occurrence 
of  rain. 

3.  Opportunity  for  learners  to  situate  themselves 

Having  the  student  to  reflect  on  his  own  past  experience  and  associating  with  the  new  knowledge  that 
he  has  Just  picked  up  are  applications  of  the  knowledge  learnt  or  related  to  the  knowledge  learnt.  This  deliberate 
effort  involves  the  student’s  active  participation  in  relating  his  own  experience  to  that  learnt.  He  may  even  be 
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able  to  construct  his  very  own  knowledge  based  on  his  unique  culture  and  context  In  the  study  of  Pythagorus 
theorem,  the  student  may  consider  a route  that  his  father  normally  takes  to  drive  from  school  to  home.  He  can 
calculate  the  amount  of  petrol  saved  if  his  father  uses  the  "hypotenuse”  road  rather  than  the  2 adjacent  roads. 
Real  life  problem  of  having  to  go  through  more  numbers  of  traffic  lights,  become  the  deciding  factor  that  the 
father  did  not  use  the  "hypotenuse  route”  - a peripheral  knowledge.  Allowing  the  students  to  explore  into  the 
various  possible  real-life  situations  also  help  to  elicit  erroneous  concept  of  the  knowledge,  which  the  expert- 
teacher  can  then  correct. 

4.  Situations  in  opening  and  closing  and  within  main  content  of  the  lesson 

Situating  the  learner  at  the  opening  of  a lesson  provides  him  with  an  end  in  mind  as  he  begins  the 
lesson.  Providing  multiple  situations  within  the  main  content  brings  his  focus  to  the  culture,  context  and 
activities  of  his  required  performance,  so  that  the  content  will  not  be  "dead”  knowledge,  but  truly  knowledge 
alive,  used  by  the  experts  and  practitioners.  At  closure,  there  is  a need  to  highlight  as  divergently  as  possible, 
the  many  situations  of  application.  Transfer  of  the  skills  and  knowledge  from  the  simulated  situation  to  real-life 
situation  must  be  emphasised. 

5.  Environment  for  learners  to  learn  from  the  experts  and  practitioners 

In  situating  the  learner  in  an  expert's  environment,  the  learner  is  exposed  to  the  multiple  perspectives 
from  various  experts  in  the  same  area,  providing  him  with  a rich  context.  With  the  advent  of  technology,  we 
throw  open  an  environment  for  the  learner  to  have  access  to  the  interested  experts  and  practitioners.  In  the 
learning  process,  an  environment  where  the  students  can  hold  discussion  with  other  experts  and  practitioners 
should  be  provided.  This  would  help  build  a virtual  community  for  the  students.  Principals  and  teachers  in 
schools  who  are  interested  in  this  area  of  constructive  learning,  situated  cognition,  cognitive  apprenticeship, 
should  also  access  to  this  site  to  collaborate  to  provide  synergy  in  Joint  projects. 


Conclusion 

Situated  learning  is  beneficial  for  the  learner  in  providing  the  culture,  context  and  activities  that  are 
inseparable  from  the  domain  knowledge,  for  meaningful  learning.  Diversity  in  all  the  three  elements  provides 
richness  in  the  learning  environment.  To  ensure  cognitive  apprenticeship,  there  needs  to  be  expert  to  provide  the 
modelling,  coaching,  scaffolding.  Activities  for  collaboration  must  be  designed.  Opportunities  for  learner  to 
articulate  their  learning  and  experience  must  be  provided,  through  writing  their  own  stories  or  observations  that 
he  has  made.  Reflections  on  their  learning,  on  other  expert's  work  must  be  encouraged  and  facilitated.  The 
advent  of  technology  has  compelled  us  to  consider  the  best  use  of  it  for  the  overall  benefit  of  learning 
effectiveness  and  efficiency.  To  be  able  to  implement  all  that  are  described  would  mean  that  the 
teacher/instructor  must  be  an  expert  in  her  areas  of  teaching,  have  experienced  the  richness  of  life  in  the  subject 
matter,  have  many  years  of  experience  in  authentic  activities  and  be  involved  in  current  works  in  the  industry. 
It  is  thus  recommended  that  the  teacher  must  have  rich  experience  in  the  community,  in  the  industry.  Schools 
should  be  encouraged  to  employ  people  from  the  industry  and  these  candidates  should  be  remunerated  more 
than  those  who  have  no  experience  at  all.  . 
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Introduction 

The  suggestion  has  previously  been  made  that  an  instructional  multimedia  (IMM)  application  will  support 
learning  to  the  extent  that  it  offers  support  to  user  performance  speed,  memory,  effort,  comfort,  content  and  course 
management.  (Thomas,  1998).  An  instrument  based  on  this  model  was  then  proposed  (Thomas,  1999).  A 
preliminary  study  using  the  qualitative  measures  of  the  model  is  presented  in  this  paper.  Subjects  evaluated  an 
interactive  multimedia  application  to  teach  children  how  to  draw.  The  results  indicate  that  users  were  quite 
consistent  in  their  evaluation  of  the  application  and  that  these  opinions  did  not  change  a week  later,  after 
discussions  of  good  design  principles.  The  only  exception  was  a change  in  the  perception  of  the  support  of  memory 
provided  by  sound. 

Model  and  Instrument 

In  applying  the  proposed  dimensions  to  evaluating  instructional  multimedia  (IMM)  , speed  is  considered  a 
function  of  how  fast  one  can  learn  the  material  or  search  for  answers.  Support  for  memory  would  be  indicated  by 
the  ease  with  which  encoding  of  information  can  occur  as  a consequence  of  the  integration  of  multimedia.  The 
extent  to  which  images,  sound  and  text  contribute  to  conveying  ideas  or  concepts  to  be  taught  will  determine,  in 
part,  the  efficacy  of  the  application  design.  The  amount  of  effort  which  must  be  expended  in  learning  can  be 
gauged  by  the  structure  and  the  levels  which  must  be  navigated,  that  is,  the  hypermedia  components.  The  comfort 
dimension  measures  the  aesthetic  appeal  of  the  images,  sound  and  text  included  in  an  application.  These  will  be  a 
function  of  the  number  and  quality  of  various  features,  for  example,  resolution,  clarity,  font  style  and  size,  layout, 
navigational  tools,  help  aids,  etc. 

The  instrument,  based  on  the  model,  suggests  that  the  dimensions  can  be  assessed  objectively  as  well  as 
subjectively.  (Scriven,  1990;  Teng  and  Jamison,  1990;  Roberts  and  Moran,  1983;  Holcomb  and  Tharp,  1991; 

Davis,  1989;  Schneier  and  Mehal,  1984;  Murphy,  1992;  Khalifa,  1990;  Poison  and  Kieras,  1985;  Preece  and 
Keller,  1990).  In  this  paper,  the  focus  is  on  the  subjective  or  qualitative  assessments  and  asks  users  to  assess  Speed, 
the  time  needed  to  complete  or  learn  the  application,  as  fast,  moderate  or  slow.  Support  of  memory,  as  a 
consequence  of  integrating  text,  images  and  sound  into  the  application,  may  be  assessed  as  high,  moderate  or  low. 
The  structure  of  an  application  will  dictate  the  effort  required  in  navigating  it.  The  more  links  and  branches  there 
are,  the  more  complex  the  application  and  the  more  effort  required  in  finding  answers  and  in  making  associations 
among  the  various  concepts.  Users  are  likely  to  view  this  effort  as  extreme,  moderate  or  minimal.  The  aesthetic 
appeal  of  an  application,  its  comfort  level,  will  be  a function  of  the  quality  of  good  package  features  included  in  it, 
and  can  be  rated  as  excellent,  good,  or  poor.  The  quality  of  the  content  of  the  application  can  be  assessed  in  terms 
of  completeness  as  complete,  adequate  or  superficial  and,  in  terms  of  relevance  as  relevant,  peripheral  or 
inappropriate. 

The  Study 


The  instrument  was  tested  with  information  systems  students  in  a graduate  introductory  multimedia 
course  who  used  the  instrument  to  evaluate  the  qualitative  aspects  of  an  application  to  teach  children  how  to  draw. 
The  instrument  was  administered  during  the  first  lecture  prior  to  the  presentation  of  any  material,  and  then  a week 
later,  after  discussions  of  good  design  principles.  In  this  preliminary  study,  the  research  questions  investigated 
were: 
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1 . Are  users’  evaluations  consistent  using  the  instrument? 

2.  Do  these  evaluations  change  after  exposure  to  good  design  principles? 

Results 

The  sample  was  mostly  males  in  the  20-29  age  group.  In  total,  there  were  twenty- three  users  filling  in 
evaluations  in  the  first  lecture  and  twenty-one  a week  later  after  discussions  of  good  design  principles.  Most  had 
extensive  computer  experience,  moderate  computer  assisted  instmction  experience  and  minimal  multimedia  design 
experience. 

Users  rated  the  application  as  “Average”  on  most  of  the  dimensions  initially.  Their  opinions  did  not 
change  after  discussions  of  good  design  principles,  except  with  respect  to  support  offered  by  sound  to  memory 
which  went  from  high  to  moderate.  This  is  puzzling.  Perhaps  users  have  more  experience  with  the  use  of  text  and 
images  for  learning,  while  the  purpose  of  sound  may  be  less  evident  without  instruction  as  to  its  use  and  merit. 

Conclusion 

The  instrument  would  seem  to  be  stable  in  eliciting  qualitative  assessments  of  interactive  multimedia  by 
end  users.  This  would  need  to  be  validated  with  a larger  sample  size  and  across  different  applications.  The  results, 
however,  are  encouraging  and  suggest  continued  research  along  these  lines. 
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Abstract:  This  paper  focuses  on  the  qualitative  results  of  a study  that  examined  the 
effects  of  learner  characteristics  and  the  learning  environment  on  students’ 
acceptance  of  control  opportunities  while  using  hypermedia  software.  A covariance 
structure  analysis  revealed  substantial  effects  of  the  school  which  students  attended 
on  their  navigational  choices  within  the  software.  Analysis  of  data  from  student 
interviews  and  audit-trail  data  collected  during  use  of  the  hypermedia  software 
suggest  that  three  possible  factors  may  have  contributed  to  the  strength  of  the 
variable  of  school.  Specifically,  they  may  have  been  the  genders  of  the  students, 
students’  epistemological  beliefs  and  different  the  learning  environments  created  by 
the  cultures  of  the  two  schools  the  students  attended. 

Introduction 

A related  paper  entitled  The  Effects  of  Activity  Structure  and  Learner  Characteristics  on  Acceptance  of  Control 
Opportunities  in  Hypermedia  Environments:  A Report  of  Quantitative  Findings’,  in  this  proceedings,  introduces 
a study  which  explored  the  effects  of  activity  structure  and  individual  characteristics  on  student  acceptance  of 
control  in  an  hypermedia  environment.  The  quantitative  analysis  identified  that  the  school  which  students 
attend  was  a factor  influential  in  navigational  choices  made  by  students.  This  paper  focuses  on  the  qualitative 
findings  of  the  same  study,  identifying  possible  reasons  for  the  strong  influence  of  the  school  variable. 

Analysis  of  the  qualitative  data  and  examination  of  relevant  literature  identified  three  possible  factors  that  may 
have  influenced  the  strength  of  the  school  factor.  Specifically,  these  were  the  gender  differences  of  students  at 
the  two  schools,  the  epistemological  beliefs  of  students  and  the  different  cultures  and  learning  environments 
that  existed  at  the  two  schools. 


Qualitative  Findings 

In  an  attempt  to  determine  the  reasons  for  students’  navigational  choices  and  their  adoption  of  particular 
navigational  strategies  during  software  use,  interviews  were  conducted  with  eighteen  students  directly  following 
computer-based  lessons.  During  the  interviews  students  were  asked  to  explain  their  reasons  for  particular 
navigational  choices  when  required  to  demonstrate  how  they  would  locate  particular  information  in  the 
software.  In  addition  to  the  interviews  a detailed  analysis  of  the  audit-tail  data  of  software  interactions  for  those 
students  interviewed  helped  to  identify  patterns  of  choices  that  represented  particular  navigational  strategies. 

Gender:  Gender  differences  between  the  two  school  populations  is  the  most  obvious  interpretation  of  the  large 
influence  of  the  school  variable.  Analysis  of  audit-trail  data  showed  the  boys  more  likely  to  engage  in  focused 
information  seeking  and  girls  more  likely  to  explore.  Interview  data  also  revealed  gender-related  patterns  of 
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responses,  with  most  girls  expressing  the  opinion  that  they  enjoyed  the  autonomy  offered  in  the  computer-based 
lessons.  In  contrast,  most  boys  indicated  that  the  computer-based  lessons  were  ‘better’  than  normal  lessons. 
Also  consistent  with  the  responses  regarding  autonomy,  most  girls  interviewed  also  expressed  the  opinion  that 
they  would  like  the  opportunity  to  explore  the  software  in  addition  to  completing  worksheets,  while  the  boys 
mostly  considered  the  worksheets  of  prime  importance. 

Epistemological  Beliefs:  Comments  made  by  students  in  interviews  and  the  information  seeking  behaviours 
they  exhibited  when  using  the  software  suggest  that  the  majority  of  students  may  have  possessed  naive 
epistemological  beliefs,  beliefs  that  knowledge  is  absolute,  is  simple,  is  handed  down  by  authority  and  is 
acquired  quickly  or  not-at-all.  Those  exceptions  were  girls  who  were  more  likely  to  make  larger  numbers  of 
choices  and  engage  in  exploration  of  the  software,  suggesting  that  they  sought  more  detailed  information,  and 
possibly  alternative,  supportive  data  for  answers  to  questions  (either  their  own,  or  questions  set  in  the  tasks). 

The  Cultures  of  the  Schools:  A third  potential  area  that  may  offer  explanations  for  the  differences  in  the 
information-seeking  is  the  distinctive  differences  in  the  learning  environments  of  the  two  schools.  While  both 
schools  adopt  a holistic  approach  to  the  development  of  the  person,  the  girls’  school  aims  to  create  an 
environment  in  which  co-operation  and  group  work  are  valued  and  promoted.  The  boys’  school  however,  aims 
to  create  a different  learning  environment  in  which  individual  academic  performance  is  valued,  and  one  in 
which  students  are  encouraged  to  observe  the  discipline  standards  in  willing  co-operation  rather  than  from  a 
fear  of  breaching  them.  These  different  environments  may  have  contributed  to  students’  different  approaches. 

Discussion 

An  important  outcome  of  the  study  is  that  the  findings  of  the  research  have  identified  some  implications 
relevant  to  classroom  practice.  Awareness  of  the  three  identified  factors  will  help  educators  to  consider  the 
possible  effects  of  learning  environments  they  create  and  the  tasks  they  set  students  on  usage  of  complex 
computer-based  software  and  learning  outcomes.  Future  research  could  also  productively  investigate  the  effects 
of  gender,  epistemological  beliefs  and  school  cultures  on  learner  acceptance  of  control  opportunity. 
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Abstract;  In  this  study,  some  figures  are  presented  on  the  presence  of  a home  computer,  frequency 
of  computer  use  and  computer  tasks  among  a large  groi^  of  secondary  school  children  in  Belgium. 
In  particular,  relationships  have  been  sought  between  the  variables  mentioned  above  and  two 
demographic  variables:  gender  and  cultural  background.  In  correspondence  to  previous  studies, 
differences  between  boys  and  girls  in  the  field  of  computing  remained  visible.  Figures  on  cultural 
background  differences  in  computing  were  less  pronounced  than  gender  differences,  but  still 
apparent. 


Background 

In  the  last  two  decades,  a great  amount  of  literature  has  been  produced  on  gender  issues  in  the  field  of  computing. 
Many  researchers  have  pointed  at  differences  between  males  and  females,  with  males  generally  showing  more 
favorable  attitudes  towards  computers,  having  greater  access  to  computers  and  spending  more  time  using  them. 
Gender  inequality  is  discussed  in  many  reviews  of  the  literature  and  meta-analyses  (Sutton,  1991;  Whitley,  1997; 
Kirkpatrick  and  Cuban,  1998;  Liao,  1999).  Although  research  conclusions  are  not  always  similar,  overall,  males 
seem  to  have  more  favorable  attitudes  towards  computers  than  females  (Martin,  1991;  Okebukola,  1993;  Shashaani, 
1997).  For  example,  Okebukola  (1993)  demonstrated  that  boys  in  high  school  show  less  computer  anxiety  and  have 
more  interest  in  computers  than  girls.  Shashaani  (1997)  confirmed  that  male  students  show  more  computer  interest. 

In  contrast  to  the  interest  in  gender  issues  in  computing  research,  less  attention  is  paid  to  the  impact  of  social 
background  on  computer  access  and  use  at  home.  In  a review  of  computer  research  in  the  eighties,  Sutton  (1991) 
stated  that  besides  female  students,  socially  disadvantaged  and  minority  students  had  less  access  to  computers  both 
at  home  and  at  school. 

Methodology 

The  purpose  of  this  study  was  to  examine  the  existence  of  inequalities  between  adolescent  boys  and  girls  and  the 
effect  of  cultural  background  on  the  accessibility  of  computers  and  computer  use  at  home.  A brief  questionnaire  was 
administered  to  1500  children  (age  12-20).  Information  on  demographic  variables  was  collected  (age,  gender, 
cultural  background).  Furthermore,  respondents  were  asked  to  indicate  their  computer  experience,  frequency  of  use 
and  computer  use  for  school  and/or  leisure  activities.  After  removing  missing  data  from  the  sample,  descriptive 
statistics  and  tests  of  significance  were  performed  on  data  from  1305  children. 

Results 

The  average  age  was  about  15  years.  Of  the  sample,  51.6  % were  male  and  48.4  % were  female.  To  indicate  the 
cultural  background  of  the  respondents,  the  nationality  of  the  parents  was  measured.  For  215  respondents  at  least 
one  of  the  parents  were  not  of  Belgian  nationality  (16.5  %). 

In  total,  81.1  % of  the  respondents  owned  a computer  at  home.  The  group  of  children  of  16  years  had  the  highest 
percentage  of  computer  ownership  (87.5%).  A linear  increase  of  computer  ownership  by  age  might  be  expected,  but 
could  not  be  confirmed.  Computer  experience  of  secondary  school  children  in  this  study  was  33  months  on  average. 
Children  of  16,  17  or  18  years  old  had  more  or  less  the  same  amount  of  computer  experience  (40-41  months).  Of  the 
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subgroup  of  children  with  a home  computer,  20.1  % never  of  rarely  used  a computer,  15.3  % used  the  computer 
about  once  a month,  37.9  % on  a weekly  basis,  and  26.6  % daily.  The  older  the  respondent,  the  higher  the  frequency 
of  daily  use  (X^  = 1 3.9  , p < .05). 

Chi^  testing  revealed  significant  differences  between  gender  and  computer  presence  at  home  (X^  = 14.1  , p < .001). 
Of  the  males,  85.0  % had  a home  computer,  as  compared  to  only  76.9  % of  the  females.  Although  more  boys  owned 
a home  computer  in  all  age  groups,  the  gender  gap  was  only  statistically  significant  for  12-,  16-  and  17-year  olds.  So 
the  assumption  that  gender  disparities  are  increasing  with  age  could  not  be  confirmed.  Overall,  girls  had 
significantly  less  computer  experience  (29.3  months)  than  boys  (37.1  months)  (F  = 23.2  , p < .001).  When  controlled 
for  computer  presence  at  home,  this  association  remained  significant  (F  = 10.8  , p < .01).  In  addition,  males  (32.2  %) 
were  found  to  make  more  use  of  a computer  on  a daily  basis  than  girls  (12.0  %)  (X^  = 78.8  , p < .001). 

Results  indicate  that  gender  disparities  depend  on  the  nature  of  tasks  being  performed  with  computers.  The  use  of 
computers  for  leisure  activities  is  clearly  reported  as  being  a masculine  activity.  Significantly  more  boys  (73.7  %) 
reported  to  use  a computer  for  leisure  activities  than  girls  (58.6  %).  This  gap  remained  significant  when  children 
without  a home  computer  were  excluded  from  the  sample:  86.3  % of  the  boys  versus  75.5  % of  the  girls  (X^  = 20.2  , 
p < .001).  The  situation  appears  to  be  different  when  considering  computer  use  for  school  activities.  Similar 
proportions  of  girls  (58.5  %)  and  boys  (58.1  %)  use  a computer  at  home  for  school  activities,  but  if  respondents 
without  home  computer  were  not  taken  into  account,  significantly  more  girls  (75.3  %)  reported  to  use  computers 
than  boys  (67.9  %)  (X^  = 7.17,  p < .01). 

The  presence  of  a home  computer  was  significantly  lower  among  children  of  non-Belgian  origin:  73.9  % versus  82.5 
% (X^  = 8.0,  p < .01).  The  association  was  only  apparent  for  the  oldest  among  the  respondents  (17  years  or  older). 
Overall,  Belgian  respondents  reported  to  have  six  months  longer  experience  with  computers  (F  = 6.8  , p < .01). 
However,  when  the  effect  of  computer  ownership  was  imported  as  a covariate  in  the  test,  no  significance  was  found 
between  the  two  groups  (F  = 1.4  , p = .23).  Moreover,  no  differences  between  groups  could  be  demonstrated  for 
frequency  of  computer  use  variable.  Respondents  of  Belgian  origin  did  not  report  to  use  the  computer  more  often  on 
a daily  basis  than  non-Belgian  respondents.  This  finding  is  confirmed  for  both  males  and  females.  Comparing 
Belgian  and  non-Belgian  respondents,  in  general  Belgian  students  accounted  for  a wider  use  of  computers:  8 % 
more  made  use  of  a computer  for  school  activities  (X^  = 8.8  , p < .05)  and  12  % more  for  leisure  activities  (X^  = 12.3 
, p < .01).  The  presence  of  a computer  at  home  seemed  responsible  for  the  difference  in  use  for  school  tasks.  In  other 
words,  if  respondents  without  a home  computer  were  removed  from  the  sample,  no  significance  was  found.  This 
finding  could  not  be  confirmed  for  leisure  use:  non-Belgian  respondents  made  significantly  less  computer  use  for 
free  time  activities,  even  when  controlled  for  computer  presence  at  home  (X^  =4.1  , p < .05). 

Discussion  and  conclusion 

The  present  findings  confirm  other  studies  which  clearly  have  demonstrated  gender  differences  in  computing. 
Besides,  this  study  suggests  the  existence  of  cultural  differences  in  computing.  Although  there  are  many  social  and 
psychological  factors  responsible  for  gender  and  cultural  inequalities,  the  absence  of  a home  computer  seems  to 
accentuate  the  differences  between  groups. 
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Abstract:  To  evaluate  the  use  of  newly  developed  IT  course  material  in  the  domain  of 
acoustics,  (http://educinno.rug.ac.be/wetenschapsmuseum),  the  students  had  to  perform 
an  open  design  project.  The  way  engineering  students  at  Ghent  University  tackled  the 
problem  and  used  the  available  tools  to  do  so,  is  evaluated  through  their  results  and  their 
opinion  expressed  in  a survey.  At  the  same  time,  their  global  ideas  about  the  usefulness  of 
information  technology  in  education  are  explored. 


Introduction 

To  evaluate  the  learning  environments  from  the  project  “Distributed  Development  of  a Virtual 
Science  Museum  with  Guide  Applied  to  education  in  Acoustics”  and  test  their  effectiveness  in  improving 
students’  insight  in  complex  formulas  and  concepts,  a suitable  student  project  had  to  be  built.  A design 
project  seemed  the  ideal  option  considering  the  numerous  alternatives  and  various  possible  ways  to  achieve 
an  acceptable  result  and  the  different  learning  paths  and  styles  to  be  used.  The  specifications  of  the  project 
corresponded  to  what  an  engineer  can  expect  in  real  life  situations.  Students  were  instructed  to  design 
Helmholtz  resonator:  basic  acoustics  were  known  in  advance,  but  the  students  had  to  explore  more  specific 
information  on  the  Helmholtz  resonator  themselves.  The  IT  project  provided  them  with  learning  objects 
containing  illustration  of  concepts,  mathematics,  models,  model  limitation,  simulation  tools,  etc. 

To  solve  the  problem  the  students  could  follow  five  different  problem  solving  paths.  The  path  of  . . . 

1 . the  rote  experimenter:  look  briefly  at  the  general  operation  principle  and  use  the  virtual  experiment  to 
accidentally  come  up  with  a reasonably  good  solution. 

2.  more  intelligent  trial  and  error:  derive  a mathematical  relation  between  a few  design  parameters  from 
the  general  operation  principle  and  investigate  only  valuable  combinations  experimentally. 

3.  a meaningful  experimenter:  start  with  some  trial  and  error,  realize  there  must  be  a reason  why  some 
parameters  do  not  seem  to  have  any  influence,  turn  to  the  theory  to  find  out  if  there  is  a theoretical 
background  for  this  and  eventually  return  to  experiments  to  determine  the  last  degree  of  freedom. 

4.  a theorist  (pure  mathematics):  combine  the  available  mathematics  to  come  to  a result  possibly  using 
some  numerical  solvers. 

5.  an  expert:  use  the  formula  to  relate  some  parameters,  investigate  more  detailed  theory  to  find  some 
physical  and  technical  restrictions,  solve  the  last  degree  of  freedom  using  mathematics  and  their  own 
software  or  the  applet,  finally  turning  to  the  derivation  to  find  out  if  their  design  is  well  within  the 
limits  of  applicability  of  the  theory. 


Survey  and  exam  results 

The  students  reported  their  ideas  about  the  importance  of  the  several  factors  for  solving  the 
problem:  the  virtual  experiment,  derivation  of  equations,  limits  of  the  theory,  other  Internet  pages, 
communication  with  peers,  literature,  and  their  perception  of  the  insight  they  gained  in  some  acoustic 
concepts.  Despite  a small  statistical  population  of  27  students,  some  interesting  conclusions  can  be  drawn: 

• Part  of  the  goal  of  the  design  project  was  to  urge  students  to  learn  the  theoretical  background  by 
exploration  of  additional  resources.  About  66%  of  the  students  state  that  they  gained  insight  into  the 
theoretical  background.  We  consider  this  a reasonable  success. 
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• Students  focussing  on  experiments  affirmed  they  gained  insight  in  the  problem  at  hand  but  they  did  not 
learn  about  the  theoretical  background.  They  probably  followed  paths  1 or  2 mentioned  above. 

• The  more  students  perceive  the  equations  and  underlying  derivations  as  important  (solution  paths  3,  4 
and  5),  the  more  they  get  the  impression  they  understand  related  theoretical  topics.  However  the 
correlation  with  their  impression  of  understanding  how  a Helmholtz  resonator  really  works  gets  lower 
and  even  becomes  negative,  although  this  tendency  is  not  reflected  in  the  students’  exam  result  of 
questions  related  to  the  topic.  Perhaps  students  only  realize  the  complexity  of  the  matter  when  they 
study  the  topic  thoroughly. 

• There  is  a negative  correlation  between  the  student’s  global  result  and  his  perception  of  insight  gained. 
This  can  mean  several  things:  the  material  is  not  good  enough  and  better  students  are  left  somewhat 
unsatisfied,  or  better  students  do  not  easily  state  understand  something  completely,  or  the  students  who 
perform  well  in  the  current  educational  system  are  not  inclined  to  put  much  effort  in  an  alternative 
project.  One  thing  is  sure:  it  is  a question  of  perception  since  there  is  no  negative  correlation  with  their 
examination  score  on  the  question  related  to  the  subject. 

This  project  forced  students  into  using  IT.  After  this  (for  some  students  ‘first’)  IT-experience  for 

educational  purposes  some  general  questions  about  the  appreciation  of  IT  were  asked.  The  following  trends 

reflect  their  ideas: 

• there  is  a strong  negative  correlation  between  the  wish  to  get  more  interactive  multimedia  material  (as 
was  presented  to  them)  and  the  student’s  performance  in  the  current  system. 

• the  students  almost  unanimously  approve  the  usefulness  of  self-assessment  tools  to  evaluate  the  level 
of  understanding  they  have  reached. 

• applets  allowing  students  to  explore  the  influence  of  parameters  in  a mathematical  theory  are  highly 
appreciated,  as  are  additional  self-assessment  tools. 

• the  possibilities  of  hypertext  to  hide  unessential  information  or  to  cross-link  different  topics  is  not 
estimated  very  high  by  the  average  Flemish  engineering  student. 


Conclusions 

In  general  students  appreciate  the  collaboration  and  the  exploration  type  of  approach  of  a design 
problem  on  a subject  they  had  no  prior  knowledge  of  The  exploration  stopped  at  an  earlier  stage  than 
hoped  for.  This  could  be  due  to  the  way  the  problem  and  the  tools  were  presented,  but  “the  study  culture” 
of  Flemish  engineering  students  probably  explains  a lot.  It  is  not  a habit  of  these  students  to  consult 
literature,  search  the  web,  etc.  When  using  IT  in  education  one  has  to  be  considering  the  study  culture  of 
the  students,  which  is  hard  to  change. 

There  was  an  unexpected  negative  correlation  between  the  general  performance  of  students  in  the 
traditional  educational  model  and  their  appreciation  of  the  project  and  IT  in  education  in  general.  Since  one 
of  the  initial  goals  of  this  project  was  to  make  sure  “better”  students  had  the  opportunity  to  learn  more  than 
in  the  traditional  educational  approach,  some  tuning  seems  appropriate,  in  order  not  to  loose  those  students. 

As  we  expected,  there  is  some  fear  of  loosing  track  and  of  doing  things  that  are  not  really  essential 
in  order  to  be  able  to  pass  the  exam.  In  the  first  semester  of  1999-2000  we  tiy  to  respond  to  this  by 
structuring  the  software  agent  we  called  “the  guide”  in  such  a way  that  a clear  path  through  the  material  is 
outlined  and  that  the  structure  becomes  more  evident.  Hyperlinks  are  applied  in  a more  structured  way  by 
the  use  of  icons  indicating  which  type  of  hyperlink  can  be  expected:  example,  practical  application, 
additional  explanation,  more  background,  etc. 
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Abstract:  The  Smart  School  Project  is  one  of  the  flagships  of  the  Multimedia  Super  Corridor.  The 
Malaysian  government  has  established  partnerships  with  the  private  sector  through  the  Concept 
Request  for  Proposals  (CRFP)  process.  This  paper  will  critically  analyze  the  concept  of  Smart 
School  in  the  Malaysian  environment  and  look  into  the  issues  relating  to  implementation  of  this 
concept  as  per  the  vision  of  the  Ministry  of  Education  Malaysia.  The  paper  will  go  further  to  provide 
empirical  proof  with  regards  to  the  mind  set  and  the  response  to  the  Teaching- Learning  component 
of  the  Smart  School  Concept,  from  the  view  point  of  in  service  teachers. 


Introduction 

In  July  1997  the  Prime  Minister  launched  the  Flagship  Application  documents  to  invite  proposals  for  solutions  from 
the  private  sector  within  and  from  outside  the  country.  The  Education  Ministry  based  on  critical  and  creative 
teaching-learning  approach  planned  the  Smart  School  concept.  Technology  as  an  enabler  became  an  important 
component  of  this  flagship.  The  exercise  is  to  help  the  Malaysian  student  community  to  cope  with  the  Information 
Age. 


The  main  components  in  the  Malaysian  Smart  School  are: 

1 . Teaching  - Learning  Process  - here  self-accessed  and  self-directed  learning  will  be  encouraged 

2.  Management  and  Administration  - the  school  Management  and  Administration  will  be  technology  driven  for 
efficient  and  effective  management  of  resources  and  to  support  the  teaching-learning  process  in  1 above. 

3.  Human  Resources,  Skills  and  Responsibilities  - active  role  play  by  Parents,  Community  and  the  private  sector  as 
Stakeholders 

4.  Processes  - review  of  input  and  output 

5.  Technology  - to  be  used  as  an  enabler  i.e.  as  a means  to  the  end  and  not  as  an  end  to  the  problem 

6.  Policies  - changes  in  existing  policies  and  regulations  over  and  above  new  policies  and  regulations 


Objectives 

The  objectives  of  the  Smart  School  Project  are: 

1 . To  produce  a thinking  and  technology  - literate  workforce 

2.  To  democratize  education 

3.  To  increase  participation  of  stakeholders 

4.  To  provide  all-round  development  of  the  individual 

5.  To  provide  opportunities  to  enhance  individual  strengths  and  abilities 

Implementation 

The  Ministry  of  Education  has  planned  for  a three-year  Smart  School  Pilot  Project  through  the  CRFP  process. 
Ninety  (90)  pilot  schools  have  been  picked  for  the  pilot  stage  which  will  involve  three  levels  of  technology,  i.e.  level 
A,  B+,  and  B. 
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Level  " 

" Total  Number Ibf  GohriiMiter s ^ 

Description-— ' I 'L . ; ■ . . ^ v : 

A 

Primary  - 406 
Secondary  - 479 

Schools  in  this  level  will  be  equipped  with  state  -of-the-art 
technology.  Each  school  will  have  two  computer  labs  and 
each  classroom  and  science  lab  will  be  equipped  with 
computers. 

B+ 

86 

This  level  is  based  on  classroom  model.  This  model  is  based 
on  group-based  activities  and  project  work.  Computers  are 
available  whenever  needed  for  information  access,  writing 
reports,  developing  presentations,  e-mailing  etc. 

B 

42 

The  majority  of  schools  will  implement  technology 
supported  learning  by  use  of  a computer  lab.  Students  will 
use  the  facility  to  access  information,  use  software 
application  for  various  purposes  and  work  on  available 
courseware  in  the  four  subjects. 

The  Government  intends  to  play  a role  as  an  architect  and  driver  for  the  Smart  School  Project.  The  Ministry  of 
Education  will  prepare  guidelines  and  provide  the  basic  amenities  to  schools  according  to  their  individual  needs. 
The  Ministry  is  also  actively  encouraging  schools  to  become  Smart  Schools  on  their  own  initiative  by  using  their 
own  financial  resources  and  expertise.  The  Pilot  Project  period  is  three  (3)  years  from  28th  July  1 999  to  22nd  July 
2002.  It  is  projected  that  all  schools  in  Malaysia  will  be  Smart  Schools  by  the  year  2010. 


Reality 

The  Malaysian  government  is  spending  over  RM.  300  million  for  only  90  schools  to  be  wired,  of  which  RM. 
183,573,737  for  Capital  Expenditure  and  RM.  116,426,263  for  Operating  Expenditure.  As  per  the  time  line  set, 
from  March  -22"^^  March  2000,  the  installation  of  Release  1 software  (Teaching-Learning  Materials  and  the 
Smart  School  management  System)  is  to  take  place. 

However  on  20'*"  February  2000,  The  Star  (National  Daily)  reported  the  Director  - General  of  Education  saying  that 
the  “smart  school”  project  in  90  schools  nationwide  will  be  further  delayed  because  of  hitches  in  the  software 
development  for  the  lessons.  He  had  said  that  the  “slight  delay”  was  due  to  the  fact  that  it  took  time  to  develop  the 
courseware.  The  pilot  project  was  suppose  to  take  off  on  Jan  1999,  but  was  delayed  for  several  reasons  including 
lack  of  funds  and  courseware.  The  infrastructure  for  the  implementation  is  ready  in  '80  of  the  90  schools  for  the  full 
implementation.  The  Education  Deputy  Director  General  stated  that  the  software  for  the  4 subjects  took  time  to 
write,  and  to  evaluate  them.  He  went  on  to  say  that  it  will  take  about  4 years  to  fully  develop  and  implement  the 
program. 

Among  the  reasons  for  the  delay  were: 

♦ lack  of  fully  developed  smart  school  curriculum 

♦ lessons  being  rejected  several  times 

♦ difficulty  in  contacting  evaluators(Ministry  of  Education) 

♦ lack  of  communications 

♦ rejection  after  many  rounds  of  evaluation 

♦ evaluators  not  exposed  to  technological  limitations 

Should  the  consortium  entrusted  with  the  development  of  the  courseware  implement  software  engineering  principles 
in  developing  the  courseware?  If  the  basic  software  engineering  principles  had  been  incorporated,  couldn’t  most  of 
the  above  reasons  for  rejection  be  done  away  with? 
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Introduction 

One  of  the  primary  aims  of  educational  software  is  to  be  used  in  real  school  classrooms.  However,  a lot 
of  educational  software  has  been  criticised  that  has  not  been  designed  to  meet  the  needs  of  real  schools 
settings.  The  above  criticisms  show  that  there  is  a need  to  introduce  research  results  of  educational 
technology  into  the  environment  of  real  classrooms.  A solution  to  this  problem  may  be  the  involvement  of 
school  teachers  in  the  development  of  an  ITS  (Virvou  & Tsiriga  1999a)  and  the  application  of  evaluation 
methods  on  every  stage  of  the  tutoring  system's  life  cycle.  In  this  way  we  can  ensure  usability  and  learning 
effects. 

For  the  purpose  of  this  research,  a multimedia  tutoring  system  was  developed.  The  system  is  called 
EasyMath  and  incorporates  intelligence.  EasyMath’s  model  of  life  cycle  has  been  based  on  Object  Unified 
Process  (Kruchten  1999),  an  object  oriented  process  that  suggests  multiple  iterations  of  the  software 
development  phases.  In  this  way,  we  have  ensured  multiple  iterations  of  evaluation  of  the  design  and 
executable  releases  of  EasyMath. 

Evaluation  of  the  Design 

Evaluation  of  the  design  of  EasyMath  was  considered  very  important  in  order  to  ensure  that  the 
expertise  incorporated  in  the  error  diagnosis  component  would  be  acceptable  by  the  majority  of  school 
teachers.  EasyMath’s  error  diagnosis  and  student  modelling  have  been  based  on  reconstructive  approaches 
which  have  been  used  in  many  ITSs  such  as  (Brown  & Burton  1978).  These  systems  reconstruct  the 
problem  solving  process  and  generate  mal-rules  from  hypothesized  faulty  solution  paths  which  are  used  for 
the  modelling  of  students’  misconceptions  or  procedural  bugs. 

At  this  stage  we  aimed  at  evaluating  the  error  diagnosis  component  in  terms  of  its  cognitive  principles 
and  completeness.  Therefore,  ten  school  teachers  were  given  a computer  based  questionnaire  with  questions 
that  had  been  answered  erroneously.  The  faulty  answers  were  produced  using  the  library  of  the  most 
common  misconceptions  that  were  encoded  in  EasyMath.  The  questions  covered  the  whole  range  of 
exercises  included  in  EasyMath.  The  questionnaire  consisted  of  ten  questions.  In  each  one  of  the  questions, 
each  teacher  had  to  provide  a justification  of  what  s/he  thought  the  underlying  misconception  was.  One 
problem  in  this  stage  of  the  evaluation  process  was  that  many  teachers  did  not  provide  any  explanation  and 
just  gave  the  correct  answer  to  the  question. 

Evaluation  of  a Primary  Executable  Release 

In  an  early  stage  of  the  development,  a prototype  of  the  tutor  was  ready  to  be  used  by  its  end  users. 
This  prototype  was  used  in  order  to  perform  a primary  evaluation  of  the  system.  In  the  phase  of  the  primary 
release  evaluation,  two  school  teachers  along  with  ten  students  were  involved.  The  evaluation  methods 
employed  comprised  qualitative  techniques,  such  as  direct  observation  and  questionnaires. 

The  main  focus  of  this  phase  was  on  the  evaluation  of  the  user  interface  and  the  several  types  of  the 
exercises  provided  by  the  system.  Teachers  and  students  were  asked  to  comment  on  the  working 
environment  of  EasyMath.  Furthermore,  they  had  to  express  their  opinion  about  whether  the  amount  of 
different  exercises  encoded  as  well  as  the  form  of  these  exercises  were  adequate. 

The  teachers  and  the  students  involved  in  this  phase  were  introduced  to  EasyMath  and  were  then  asked 
to  use  it  for  about  an  hour.  During  their  interaction  with  the  system,  an  evaluator  was  present,  trying  to  note 
down  the  difficulties  that  the  users  had  encountered  while  performing  predefined  tasks.  After  the  interaction 
with  EasyMath,  both  the  teacher  and  the  students  were  given  a questionnaire  to  answer.  The  questionnaire 
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was  designed  taking  into  account  a set  of  “usability  heuristics”  presented  in  (Nielsen  1994).  The 
questionnaire  included  questions  such  as: 

1 . Were  instructions  for  use  of  the  system  visible  or  easily  retrievable  whenever  appropriate? 

2.  Were  the  symbols  and  the  words  used  in  the  different  components  of  EasyMath  consistent? 

The  first  question  corresponds  to  the  need  for  recognition  rather  than  recall,  while  the  second  question 
is  related  to  the  need  for  consistency  and  standards.  The  comments  made  during  this  stage  of  EasyMath’s 
development  were  used  as  the  basis  for  the  refinement  of  the  requirements  of  the  system  and  the 
construction  of  the  final  product. 

Formative  Evaluation  Using  a Set  of  “Learning  with  Software”  Heuristics 

The  involvement  of  school  teachers  in  the  phase  of  the  evaluation  of  an  ITS  was  considered  crucial.  In 
addition,  it  is  also  important  to  track  students'  use  of  resource  based  packages  very  closely  to  uncover  the 
problems  and  successes.  For  the  above  reasons,  in  the  phase  of  the  formative  evaluation  of  EasyMath,  10 
school  teachers  as  well  as  240  students  were  involved.  Both  students  and  teachers  were  asked  to  evaluate 
EasyMath  in  terms  of  the  purpose  of  education.  In  addition,  teachers  were  asked  to  evaluate  the  overall 
performance  of  the  system  and  express  their  opinion  about  the  usability  of  such  an  ITS  in  a real  classroom. 

In  the  stage  of  the  students'  evaluation  of  EasyMath,  the  students  were  separated  in  two  portions.  The 
first  120  students  were  introduced  to  EasyMath  and  were  asked  to  use  it  for  an  hour.  Then,  they  were  given 
a questionnaire  to  complete.  For  a more  detailed  description  of  the  results  of  the  questionnaire,  the  reader  is 
referred  to  (Virvou  & Tsiriga  1999b).  As  for  the  rest  120  students,  they  were  taught  half  of  the  syllabus  in 
algebraic  powers  without  the  use  of  EasyMath  and  they  were  given  a written  test.  Then  they  used  EasyMath 
while  being  taught  the  rest  of  the  syllabus.  In  the  end  they  were  given  another  written  test  and  the  grades  of 
their  first  and  second  test  were  compared.  The  results  showed  that  46%  of  the  students  obtained  a better 
grade  in  the  second  test,  43%  obtained  the  same  grade  and  only  1 1%  obtained  a lower  grade  in  the  second 
test. 

The  evaluation  questionnaires  used  in  this  phase  were  constructed  based  on  a set  of  "learning  with 
software"  heuristics,  introduced  by  (Squires  & Preece  1999).  These  heuristics  are  an  adaptation  of 
“usability  heuristics”  (Nielsen  1994),  so  as  to  relate  to  socio-constructivist  criteria  for  learning.  "Learning 
with  software"  heuristics  have  been  suggested  by  their  inventors  as  a method  for  performing  predictive 
evaluation.  However,  we  have  used  them  to  construct  a formative  evaluation  questionnaire,  that  would  give 
us  insight  about  the  integration  of  both  usability  and  learning  issues. 

In  the  phase  of  the  overall  evaluation  of  EasyMath,  the  teachers  who  participated  in  the  evaluation  were 
asked  to  role  play  an  average  student  interacting  with  EasyMath.  Next,  they  were  given  a questionnaire  to 
fill  in.  The  questionnaire  was  carefully  designed  so  as  to  ensure  that  questions  are  related  to  as  many  of  the 
"learning  with  software"  heuristics  as  possible. 

Conclusions 

To  produce  efficient  educational  software,  there  is  a need  to  ensure  the  correctness  of  the  domain 
knowledge  as  well  as  to  evaluate  the  usability  of  the  product.  One  way  to  achieve  the  above  goals  may  be 
the  application  of  evaluation  methods  on  every  stage  of  the  tutoring  system's  life  cycle.  In  addition, 
evaluation  should  involve  both  human  teachers  and  students  in  order  to  maximise  usability  in  real 
classroom  settings. 
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Abstract.  This  work  in  progress  paper  reports  on  whether  hybrid  CD-ROM  technology  used  to  train  or  supplement 
the  training  of  mentors  of  beginning  teachers  in  Vocational  Education  and  Training  is  an  improvement  upon 
traditional  face  to  face  and  print-based  delivery.  It  argues  that  the  shifting  ground  in  pedagogical  models  can  confuse 
educational  software  developers  but  an  understanding  of  a theoretically  driven  level  of  explanation  of  learning  may 
assist  decisions  about  learning  approaches  and  choice  of  technology. 


Introduction. 

Faced  with  a climate  of  change,  educational  institutions  are  under  increasing  pressure  to  explore  new  ways  of  learning.  Emerging  trends  in 
Infonnation  and  Communication  Technologies  (iCTs),  combined  with  national  governments  initiatives  to  use  ICT  and  widen  participation  in  lifelong 
learning  (in  the  case  of  the  UK,  the  influential  Dearing,  Higginson  and  Kennedy  reports),  have  created  a framework  for  supporting  an  increasing 
community  of  participants  in  education  and  training. 

The  development  and  introduction  of  ICTs  into  the  further  and  higher  education  and  training  sectors  has  stirred  debate  about  pedagogical 
perfonnance  versus  educational  efficiency.  In  training,  the  debate  has  focused  on  issues  of  Return  on  Investment  (Hawkins,  et  al  1997).  Factors  of  cost 
and  value  for  money  versus  amount  of  learning  and  quality  of  learning  characterise  educational  and  training  evaluations  A lack  of  separation  of  these 
factors  has  created  a cloud  of  confusion,  and  decisions  about  allocations  of  scarce  resources  are  not  always  made  in  a manner  that  maximises  the 
collective  benefit  of  the  community. 

Many  would  contend  that  ICTs  are  not  only  cost  efficient  (Boulet  & Sene,  1997;  Greenwood  & Recker,  1996)  but  they  also  promote  more 
effective  learning  (Resnick,  1987;  Fjuk  & Dirckinck-Holmfield  1999).  Others  call  for  more  caution  to  be  exercised  about  such  claims.  In  the  UK,  the 
push  towards  the  integration  of  computer-based  education  within  education  and  training  has  been  made  at  the  highest  levels  with  funding  to  match. 
The  premise  is  that  ICT  can  deliver  more  effective  learning  than  traditional  media. 

There  is  no  one  single  recognised  right  way  of  thinking  about  learning;  theories  constantly  change  and  tend  to  reflect  the  attitudes  and 
technologies  of  their  time.  The  literature  points  to  shifting  ground  in  pedagogical  models.(Reiguluth  1999;  Herrington  & Standen,  1999).  Currently 
the  debate  is  influenced  by  ICT. 

It  is  of  some  concern  therefore,  that  educational  software  developers  have  an  understanding  of  a theoretically-driven  level  of  explanation  that 
identifies  the  kinds  of  cognitive  activities  (such  as  learning,  memory  tasks  and  problem-solving)  that  could  take  place  through  the  physical  activities 
provided  by  an  interface.  To  produce  successful  tools  for  learning,  it  is  necessary  to  understand  the  nature  of  the  learning  process  and  the  impact  of 
context  within  which  learning  operates. 

Background 

Tliere  is  a particular  need  at  the  University  of  Greenwich  each  year  to  train  up  to  250  new  and  existing  workplace  mentors  who  support  trainee 
teachers  on  the  Post  Graduate  Certificate  of  Education  (PGCE)  in  Post  16  Vocational  Education  and  Training  (VET)  during  their  teaching  practice. 

Tlie  learning  demands  and  needs  of  these  mentors  are  relatively  different  to  the  more  conventional  learner  and  it  may  be  argued  that  the 
instructional  approach  is  different.  Firstly,  it  concerns  the  training  of  trainers.  Secondly,  there  is  no  set  curriculum  to  follow,  or  assessments  to  be  made; 
thirdly,  there  are  no  prerequisites  other  than  being  a teacher/trainer  within  the  selected  institution  and  lastly,  whilst  the  institution  is  paid  for  the  efforts 
of  their  staff,  the  mentors  personally  go  largely  unrewarded  either  by  time,  cash  or  qualification  inducements. 

Although  some  mentors  may  work  within  the  same  organisation,  many  others  do  not  and  generally  they  work  largely  independently  of  one 
another.  Institutions  span  a large  geographical  area  across  London  and  the  South  East  of  England  and  mentors  rarely  meet  with  one  another  to  discuss 
their  role. 

There  is  no  substantia)  proof  to  suggest  that  a trained  mentor  is  more  effective  than  an  untrained  one  (Jowett  1998),  however,  training  does 
enhance  an  understanding  of  the  stages  of  a relationship,  the  mentoring  process  and  it  enables  critical  reflection  on  professional  practice.  Conventional 
mentor  training,  by  virtue  of  location,  time  and  resources,  can  only  address  a small  number  of  aspects  of  the  mentoring  process  - the  tip  of  the  iceberg. 
Malderez  and  Bodoczky  (1998),  take  the  image  of  the  iceberg  further  and  draw  a parallel  between  the  tip,  which  is  out  in  the  air  and  represents  the 
visible  ‘good  professional’  and  the  main  body  of  the  iceberg,  which  lays  unseen  and  represents  all  that  the  professional  engages  in  as  an  expert  in  their 
subject  and  as  a mentor.  The  submerged  part  represents  the  mentor’s  thinking,  planning,  and  engagement  in  process,  and  draws  upon  their  knowledge, 
understanding,  values,  feelings  and  attitudes.  The  mentor’s  ability  to  respond  to  the  student  teacher’s  professional  development  hinges  on  their 
understanding  of  the  role  and  on  their  own  development  as  mentors.  An  analysis  of  these  elements  forms  the  content  of  the  interactive  tool. 

Project  Objectives  and  Scope. 

In  its  current  form,  training  is  administered  through  a combination  of  print-based  distance  material  and  a single  workshop  session  for  some 
mentors  at  the  beginning  of  the  academic  year.  Our  initial  research  showed  that  the  majority  of  mentors  would  be  willing  to  use  IT  for  training 
purposes,  although  we  identified  a number  of  constraints;  some  were  institutional  such  as  the  limitations  of  the  technical  specification  of  equipment 
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provided  by  colleges  (processor  speed,  CD-ROM  drives,  soundcards,  the  ability  to  play  video,  etc)  and  some  personal  constraints  such  as  knowledge 
about  ICT  and  confidence  in  using  the  computer.  The  pursuit  of  a technological  solution  to  provide  support  and  training,  independent  of  time  and 
location  was  compelling. 

In  1998  the  Mentor  in  cyberspace  project  undertook  to  produce  an  alternative  means  of  supplementing  the  existing  mentor  training  and  support. 
Interactive  multimedia,  it  appeared  to  us,  offered  the  best  chance  of  developing  a range  of  tools  and  resources  to  support  mentor  processes  such  as 
thinking,  creativity,  self-reflection,  problem  solving  and  the  development  of  professional  attitudes  & approaches  over  time.  At  the  start  of  the  project 
only  a small  number  of  academic  institutions  which  were  not  universities  had  organised  individual  internet  access  for  their  staff.  We  considered  the 
lead-in  time  for  the  project  to  be  sufficiently  long  to  utilise  the  hybrid  function  of  CD-ROM  technology  with  its  ability  to  connect  on-line.  Furthermore, 
the  University’s  on-line  Campus  (OLC)  was  under  construction  and  the  idea  of  extending  the  range  and  type  of  tools  available  to  mentors  was 
attractive. 

Learning  approaches. 

Selecting  instructional  methods  on  the  basis  of  the  theories  has  not  easy  because  little  agreement  exists  between  the  main  protagonists.  The 
constructivists  attack  instructivist  design  approaches  as  formulaic  and  mechanistic.  Defenders  of  instructivist  design  have  attempted  to  re-establish  its 
authority  at  the  expense  of  all  other  approaches.  They  take  a staunch  position  against  the  infringements  of,  in  Merill’s  words,  “these  persons  who  claim 
that  knowledge  is  founded  on  collaboration  rather  than  empirical  science,  or  who  claim  that  all  truth  is  relative”  (Merill,  cited  in  Wild  and  Quinn 
1998,  p 74).  In  much  of  the  current  and  recurring  debate  about  the  role  of  learning  theory  in  educational  interactive  multimedia  there  appears  to  be  a 
readiness  to  assert  one  theory  of  learning  over  another  - to  present  one  as  being  deficient  and  the  other  as  the  only  credible  explanation  of  learning. 

The  important  issue  is  to  match  the  instructional  approach  to  learning  need  whatever  the  level,  whether  this  is  the  efficient  dissemination  and 
testing  of  knowledge,  the  creation  of  new  personal  knowledge  or  the  development  of  opportunities  for  networking  and  collaboration.  In  a professional 
context  where  the  goal  is  to  encourage  the  acquisition  of  reproductive  and  productive  skills  and  knowledge,  a balance  must  be  struck  between  those 
methods  that  appear  to  work  best. 

Our  approach  has  been  to  try  to  strike  a balance  between  these  two  ostensibly  opposing  theories  and  adapt  the  technology  to  suit  the  perceived 
learning  needs  of  the  mentors.  On  analysis  of  the  strengths  of  each  learning  approach  in  relation  to  our  purposes,  we  saw  that  instead  of  contradicting 
and  negating  each  other,  these  two  approaches  could  be  complementary.  We  perceived  that  the  use  of  hybrid  CD-ROM  would  combine  both  learning 
approaches.  CD-ROM  is  a relatively  closed  technology,  limited  by  its  content  and  design.  Our  design  here  draws  upon  the  strengths  of  Instructional 
Systems  Design  (ISD):  its  emphasis  on  presentation,  information,  structure  and  reinforcement  is  useful  for  the  reproduction  of  knowledge  (Boyle 
1998)  and  is  suited  to  interactive  multimedia  on  CD-ROM.  Web  technology,  on  the  other  hand,  is  characterised  by  an  open,  explorative  architecture. 
Accordingly  we  have  used  web  technology,  specifically  Computer  Mediated  Communications  (CMC)  which  supports  a more  constructivist  approach: 
it  is  able  to  provide  a set  of  tools  and  methods  for  the  mentor  to  construct  new  knowledge  within  ill-structured,  authentic,  real  contexts  (Brown, J.S., 
Collins,  A.,  & Duguid  1989).  Hybrid  CD-ROM  offers  the  possibility  of  combining  the  reproduction  of  knowledge  with  the  production  of 
‘knowledgeability’  (Guile  & Young  1998). 

This  work  in  progress  will  report  the  findings  of  a pilot  study  ending  in  June  2000  with  approximately  20  mentors  who  will  have  utilised  the 
various  tools  and  approaches  during  this  academic  year.  The  particular  focus  of  the  research  will  examine  whether: 

♦ mentors  preferred  to  use  the  hybrid  CD-ROM  or  the  print  based  material  for  learning  about  their  role 

♦ one  instructional  approach  or  technology  was  more  successful  than  the  other. 

I will  attempt  to  draw  conclusions  about  the  success  of  introducing  interactive  multimedia  and  whether  the  use  of  ICT  in  this  context  is  an 
advantage  for  training  and  supporting  mentors  of  beginning  teachers  in  VET 
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Abstract:  This  paper  proposes  an  open  architecture  of  Visual  Lab  on  WWW  and  focuses  on 
scientific  experiment  for  the  asynchronous  mode  of  distance  learning.  Visual  Lab  is  a virtual 
experiment  environment  by  using  video-media.  A complete  scientific  experiment  in  Visual 
Lab  includes  four  essential  stages:  phenomenon  observation,  experiment  process  observation, 
data  measurement  and  experiment  recording.  In  Visual  Lab,  students  can  cany  out 
experiments  remotely  with  the  toolbox  designed  on  Internet.  A system  architecture  built  on 
World-Wide  Web  demonstrates  the  feasibility  and  flexibility  of  our  idea. 

Keywords:  Distance  Learning,  visual  lab,  scientific  experiment.  World  Wide  Web. 


1.  Analysis  of  Scientific  Experiment  in  Visual  Lab 

For  a full  scientific  experiment,  there  are  several  essential  stages:  phenomenon  observation,  experiment 
process  observation,  data  measurement  and  experiment  recording.  These  essential  stages  are  analyzed  details  in 
the  following. 

Phenomenon  Observation:  With  such  knowledge,  phenomenon  observation  can  be  divided  into  two 
categories 

(a)  Static  Observation. 

(b)  Dynamic  Observation. 

Experiment  Process  Observation:  It  is  very  important  to  practice  with  one's  own  hands.  After 
experimenting  themselves,  students  can  gain  much  impression. 

(a)  Traditional  Process  Observation. 

(b)  Video  Observation. 

Data  Measurement:  Data  measurement  is  the  key  to  prove  a physics  theory. 

(a)  Equipment  Usage  in  a Traditional  Lab. 

(b)  Usage  of  Simulation  Tools. 

Experiment  Recording:  The  formats  of  experiment  records  have  to  be  designed. 

Problems  Analysis  of  Four  Essential  Conditions  on  WWW:  The  formats  of  experiment  records  have  to 
be  designed. 

(a)  Video  Formatj 


Compress 

Network  Transfer 

MPEG 

No 

No 

Real  Video 

No 

Yes 

I*""  # r' 
0^0 


Optional  |No 

Table  1:  Video  formats  comparison. 


AVI 


(b)  Component  Format: 


Download 

Security 

Control  Operation 

Browser  Support 

Java  Applet 

Light 

No 

Can  not  access  local  files 

Both  IE  and  Netscape 

Real  Video 

Heavy 

No 

Can  not  control 

Both  IE  and  Netscape 

Window  Object 
embedded  in  Web  page 

No 

No 

Can  not  control 

IE 

ActiveX 

Heavy 

Yes 

Full  control 

IE 

Application 

Heavy 

No 

Full  control 

Both  IE  and  Netscape 

Table  2:  Comparisons  table  of  web  technologies. 


(c)  Experiment  Record:  Students  can  record  data  what  they  want  to  measure  and  send  out  these  data 
to  teacher.  Then  teachers  can  gather  all  the  student  learning  information  via  the  function  provided 
by  experiment  record. 

2.  Design  the  Platform  of  Visual  Lab 

In  this  section,  several  toolboxes  will  be  discussed.  Each  measurement  and  observe  components  must 
applied  these  toolboxes  in  Visual  Lab  platform. 

Measuring  and  Observing  Tools:  For  a Visual  Lab  System,  measuring  and  observing  tools  are  helpful  to 
both  students  and  teachers 

(a)  Video  Controller:  To  play  some  experiments  video. 

(b)  Scale:  In  order  to  measure  the  actual  length  from  the  experiment  video. 

(c)  Ruler:  Students  can  adjust  the  length  of  this  ruler  tool  to  can  get  the  length  what  they  adjust  when  they 
saw  in  the  video. 

(d)  Protractor:  Students  can  measure  the  angular  magnitude. 

(e)  Calculator:  Students  can  input  the  data  and  get  the  result  just  like  a calculator. 

(f)  Timer:  A timer  is  needed  to  measure  time  in  scientific  experiment. 

Components  of  Visual  Lab:  Each  learning  work  is  implemented  by  an  independent  component,  and  our 
architecture  can  be  illustrated  as  (Fig.  1)  shown. 

3.  Implement  the  Visual  Lab  on  WWW 

We  designed  a platform  using  open  architecture  technology.  Our  purpose  is  that  all  of  the  physics 
experiments  can  apply  in  Visual  Lab  without  any  modification.  As  long  as  teachers  film  the  experiment  and  join 
in  Visual  Lab,  students  can  experiment  in  Visual  Lab  environment  and  record  with  Microsoft  Excel  2000,  as 
show  in  (Fig.  2). 

4.  Conclusion 

This  paper  proposes  an  open  and  reusable  architecture  of  Visual  Lab  on  WWW.  All  of  the  scientific 
experiments  can  apply  to  Visual  Lab  based  on  the  architecture  of  Visual  Lab  had  been  accomplished.  Learners 
can  learn  knowledge  by  experiment  in  the  Visual  Lab  even  they  are  not  at  school.  Solution  for  transferring  video 
stream  through  network  should  be  the  next  research  direction  to  enhance  Visual  Lab. 


Figure  1:  The  architecture  of  Visual  Lab. 


Figure  2:  Scientific  experiment 
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Abstract.  Two  groups  of  teachers,  50  K-12  History  teachers  from  California  and  20 
K-12  teachers  in  a Masters  program  in  Massachusetts,  participated  in  an  online 
experience  to  develop  instructional  materials  which  integrate  technology  into  the 
curriculum.  Each  groups'  level  of  participation  in  online  communication  activities 
and  the  method  that  their  instructional  design  process  was  developed  predicted  the 
quality  of  the  final  materials.  Recommendations  for  structuring  online  collaborative 
activities  for  teachers  to  develop  instructional  materials  are  given. 


Introduction 

This  is  a comparison  of  two  on-line  collaboratives  with  the  same  purpose  — to  have  teachers  develop  K-12 
technology-integrated  instructional  materials  --  but  significantly  different  approaches.  The  two  projects  we  are 
using  for  this  comparison  are  a California  statewide  professional  development  project  in  K-12  History  (CH- 
SSP)  and  an  on-line  course  in  Project  Based  Learning  for  a Masters  degree  program  in  Educational  Technology 
at  Framingham  State  College  in  Framingham,  Massachusetts. 

Key  Factor  - Participation  in  Communication  Activities 

For  all  participants,  developing  instructional  materials  was  a process  that  involved  participant  to  participant,  and 
participant  to  facilitator/instructor  communication  using  an  online  space  and  several  forms  of  electronic 
communication.  There  were  six  different  forms  - online  chat,  face-to-face,  online  submission  of  original 
written  material,  online  response  to  others  written  material,  participation  in  the  online  discussion  forums  and  e- 
mail.  The  greater  the  participation  in  online  activities,  the  more  probable  that  the  lesson  plan  was  completed  and 
the  more  likely  that  those  lesson  plans  were  of  high  quality.  There  were  significant  differences  between  the 
groups  in  their  participation  in  on-line  activities.  The  professional  development  group  participated  significantly 
less  than  the  online  course  group.  There  were  two  primary  reasons  for  this.  Although  both  groups  were 
required  to  participate,  there  was  no  tangible  consequence  for  not  participating  for  the  professional  development 
group.  Second,  technical  resources  and  technical  help  were  more  readily  available  for  the  online  course 
participants  than  the  professional  development  participants. 

The  greatest  disparity  between  the  groups  was  in  the  discussion  area.  The  postings  for  the  professional 
development  group  were  sparse  to  non-existent.  No  discussion  ever  emerged.  For  the  online  course,  this  was 
unquestionably  the  most  successful  communication.  The  discussions  were  interesting,  in-depth,  meaningful  and 
at  the  Masters  level.  The  major  difference  between  these  groups  is  that  the  discussion  was  participant  driven  for 
the  professional  development  group  and  instructor  driven  for  the  online  course  group.  We  find  this  to  be 
problematic,  as  one  of  the  goals  of  the  professional  development  is  to  have  the  participants  drive  the  interaction. 
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Key  Factor  - Instructional  Design  Process 


An  important  component  of  the  CH-SSP  project  was  the  development  of  the  instructional  design  process, 
CHSSP  was  interested  in  having  the  different  grade  level  groups  develop  their  own  process  for  the  development 
of  the  lesson  plans.  In  the  Project  Based  Learning  course,  conversely,  the  instructor  strictly  imposed  the  process 
and  the  students  were,  in  part,  evaluated  on  how  they  used  this  imposed  process.  We  refer  to  this  difference  as  a 
self-generated  process  (CHSSP)  vs.  an  imposed  process  (Framingham), 

The  different  process  experience  led  to  significant  differences  in  the  final  product.  The  self-generated  process 
group  spent  a great  deal  of  time  developing  the  process.  As  this  was  a new  experience  for  them,  there  was 
confusion  regarding  what  was  being  asked  of  them.  This  detracted  from  the  application  of  those  processes  to  the 
instructional  materials  development.  We  still  feel  strongly  that  we  would  like  to  see  the  process  of  materials 
development  arise  from  the  teachers  that  are  using  the  materials.  We  feel  changes  need  to  be  made  to  focus 
more  on  the  development  of  the  materials  themselves. 

Recommendations 

We  have  several  recommendations.  There  were  many  difficulties  with  the  professional  development  institute, 
most  of  them  centered  around  participants  actively  engaging  in  the  online  communications,  and  participants 
understanding  facilitator  expectations  in  relationship  to  the  process  of  instructional  materials  development.  We 
would  prefer,  encourage  and  work  towards  that  process  being  developed  by  the  participants  themselves. 
However,  many  of  the  participants,  from  both  groups,  felt  the  need  for  more  structure  and  more  direction  in 
how  to  go  about  developing  the  materials.  We  do  not  know  if  this  was  because  they  were  using  a new 
environment  and  felt  unsure  about  how  to  proceed,  or  if  they  would  have  difficulty  developing  instructional 
materials  without  an  imposed  process  in  any  environment.  In  the  future.  In  the  professional  development 
project,  some  direction  needs  to  be  provided  for  the  development  process,  however,  this  process  should  remain 
fluid  for  the  participants.  In  the  online  course,  we  would  recommend  that  a portion  of  the  beginning  of  the 
course  be  spent  with  the  participants  examining  their  own  practices  of  instructional  materials  development. 

The  professional  development  institute  needs  to  provide  a more  concrete  reason  for  the  participants  to  actively 
participate  in  the  online  environment.  Providing  participants  with  a 'reward'  at  the  end,  such  as  a stipend. 
University  Extension  credit  towards  salary  points,  or  the  like  may  be  an  important  component  to  motivate 
participants.  Technical  support  is  also  essential.  It  may  be  best  that  technical  support  be  pledged  from  the 
district  as  that  is  frequently  the  ISP  for  the  teacher.  It  also  provides  a district  commitment  to  the  project. 

In  general,  online  instructional  materials  development,  in  a group,  collaborative  setting,  needs  a structure  to 
support  it  which  includes  facilitator  support  for  the  multiple  forms  of  online  communication,  technical  support, 
and  clear  guidelines  for  the  participants  process.  In  addition,  participants  need  a tangible  motivation  for 
completing  the  project,  such  as  salary  points  or  a stipend.  These  online  development  projects  differ  from 
traditional  development  projects  in  that  they  can  be  conducted  asynchronously  and  between  participants  who 
are  at  a geographic  distance.  For  the  future,  we  hope  that  the  communities  that  are  formed  can  continue  on, 
develop  further  materials  and  expand  to  provide  online  support  for  other  teachers  in  their  subject  areas  or  grade 
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Abstract:  This  short  paper  illustrates  a framework  for  examining  the  application,  and  misapplication,  of  media  types  and 
contexts  in  educational  settings.  It  illustrates  a relationship  between  form,  format  and  function  that  is  useful  in  determining  the 
effectiveness  of  new  media  technologies  in  educational  contexts. 


Your  home  has  a drawer  or  box  in  some  convenient  place  where  you  can  always  reach  the  most  useful  tools;  a flashlight,  a ruler,  a roll  of  tape. 

But  there  have  been  occasions  where  rummaging  around  in  that  drawer  didn't  get  you  the  desired  tools.  You  want  to  hang  a picture  on  the  wall 
but  you  can't  find  a nail,  only  some  screws.  You  realize  that  a screw  would  work  for  hanging  the  picture,  only  to  find  you  don't  have  a 
screwdriver.  You  rationalize  that  the  picture  isn't  all  that  heavy  so  you  could  just  pound  the  screw  into  the  wall  and  take  care  of  it,  but  there  isn't  a 
hammer  to  be  found  either,  only  a wrench.  This  tool,  clearly  not  designed  for  the  task,  is  pressed  into  service  and  with  a little  effort,  hopefully 
without  broken  fingers,  you  indeed  hang  the  picture.  You  may  not  have  used  the  right  tool  for  the  job  but  the  job  was  done.  Because  the  picture 
now  hangs  over  and  covers  the  screw  you  are  also  able  to  hide  this  odd  choice  of  a mounting  device  from  visitors  who  would  admire  the  picture. 

This  short  paper  is  meant  as  a starting  point  for  examining  educational  effectiveness  of  technology  applications  by  matching  format,  function,  and 
context.  In  far  too  many  cases  today  the  wrong  tools  are  being  used  in  the  wrong  context,  supporting  the  educational  process  in  ways  that  are  left 
unexamined  by  educational  researchers  and  instructional  designers.  We  are  getting  the  job  done  with  whatever  we  have  available  and  no  one  is 
looking  at  the  inventory  of  tools  in  the  drawer.  The  efficiency  and  effectiveness  of  technology  in  educational  settings  cannot  be  measured  solely 
through  the  final  output  of  an  educational  process.  The  latest  studies  of  the  use  of  computers  for  educational  purposes,  now  focusing  on  the  role 
of  the  Internet,  have  provided  a confusing  mix  of  contradictory  results.  Most  of  the  studies  are  hopeful  in  tone  but  unclear  about  the  benefits,  or 
even  the  overall  value,  of  using  the  Internet  as  a medium  of  education.  It  can  be  used,  so  it  is  being  used. 

Educators  have  a long  history  of  adopting  and  adapting  to  new  communication  technologies  as  they  enter  society-  sometimes  with  great  effect. 
Other  communication  technologies  have  been  misused  in  specific  applications  by  teachers  and  learners  and  abandoned  for  all  other  uses.  The 
awareness  that  a communication  tool  can  be  used  in  an  educational  setting  is  related  to  the  need  that  could  potentially  be  satisfied  by  the  new 
technology.  If  the  tool  does  not  not  immediately  satisfy  the  need,  it  runs  the  risk  of  being  abandoned.  If  the  tool  can  satisfy  an  educational  need 
even  through  a use  that  was  unintended,  even  inefficiently,  it  will  be  adapted  to  suit  the  purpose.  The  effectiveness  of  this  use  can  be  improved 
through  an  understanding  of  the  educational  process,  its  context,  and  the  effects  of  communication  tools  on  teaching  and  learning. 

Communication  formats  and  settings 

Linking  communication  environments  with  educational  settings  is  a valuable  starting  point  for  comparing  the  operational  effectiveness  of  a new 
technology  tool  in  context,  the  following  six  educationai/communication  environments  are  based  on  mass  communication  theories  that  grew  from 
research  efforts  through  the  1940s  and  50s . 

Figure  /.  Formats  & Settings  


Figure  /.  If  the  effectiveness  of  educational  technology  is  to  be  measured  it  must  take  place,  necessarily,  at  one  of  these  levels,  and 
perhaps  be  used  as  a tool  at  several  levels.  The  understanding  that  different  types  of  communication  contexts  offer  differing  levels  of 
interaction  with  educational  material,  and  thus  provide  a variable  basis  for  learning,  is  essentia!  for  measuring  the  effectiveness  of  the 
application  of  technology  tools  at  each  level  of  interaction. 


Figure  2. 
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Figure  2.  The  quality  of  educational  interaction  breaks  down  when  at  lower  levels  of  activity,  and  cannot  be  applied  at  all  at  higher 
levels  of  interactivity.  Interpersonal  communication  cannot  take  place  over  mass  broadcast  technology,  but  lectures  can  be  misapplied 
to  interpersonal  tutorials.  Media  technology  is  most  effective  for  education  when  context  and  type  match  interactivity. 

Media  types  and  technology  tools 

The  historical  development  of  multimedia,  the  combination  of  moving  images,  sound,  and  graphic  information,  begins  with  the  development  of 
motion  pictures  and  extends  to  today's  subscription  television  cable  systems.  The  impact  of  these  multisensory  media  on  viewers  in  the  immersive 
environment  of  a darkened  theater  or  near  the  electronic  hearth  of  individual  homes  is  complex  and  impressive.  Multimedia  technology  has 
become  more  sophisticated  and  multilayered,  allowing  for  multiple  languages,  commentary,  text,  and  audio  input  and  feedback.  This  is  all  the 
more  remarkable  in  view  of  the  almost  total  lack  of  feedback  in  early  silent  movie  production,  mass  broadcast  television  monopolies,  and  linear 
videotape  media. 

Coalescent  media  is  the  result  of  both  technology  and  media  convergence.  It  represents  the  development  of  a new,  digital,  component-based 
medium.  Coalescent  media  have  emerged  from  the  development  of  computational  and  data  management  tools,  publishing  software,  digital  audio 
and  media,  hypertext,  and  hypermedia.  The  linkage  of  different  components  occurs  in  contexts  provided  by  the  user  in  ways  determined  through 
navigational  choices  made  on  the  spot  by  the  user.  Today  web  pages  can  be  built  by  relational  databases  on  demand.  Users  select  options  and  the 
database  engine  places  the  requested  information  in  an  HTML  document  using  preselected  parameters.  The  limitations  of  current  database 
technology  combine  with  bandwidth  restrictions  to  limit  the  potential  of  this  early  technology  to  combine  and  recombine  data,  in  numerous 
forms,  based  on  searches  of  worldwide  resources.  Coalescent  media  will  be  first  relational,  drawn  together  through  user  demand  and  user  selected 
interaction  parameters.  Later  developments  may  find  hypertextual  hypermedia  coalescing  into  forms  of  communication  that  extends  the 
capabilities  of  text,  audio,  or  info-graphics  by  extending  the  interactions  users  have  with  new  combinations  of  that  media.  This  hyperlinking  of 
other  media  types,  in  constantly  shifting  referential  and  contextual  frames,  is  creating  a new  type  of  medium  that  demonstrates  the  characteristics 
of  each  level  of  interaction  and  uses  each  previous  type  of  medium.  As  technology  escapes  the  limitations  of  keyboards  and  screens  we  will  see 
new  types  of  interactions  take  place  in  contexts  wholly  created  and  invented  by  the  user,  not  producer,  of  the  content.  New  technologies  being 
developed  are  allowing  different  types  of  interactions  with  content  in  contexts  not  imagined  by  the  author.  We  are  only  beginning  to  see  the  curve 
rise  in  the  growth  of  computational  efficiency  and  the  rise  of  new  computational  technologies. 

Educational  institutions  are  currently  setting  learning  requirements,  and  designing  the  curricula  to  reach  them,  through  interactions  with 
government,  professional  and  industry  interests.  These  may  be  anything  from  international  professional  societies  to  locally  elected  school  boards. 
What  will  education  look  like  with  information  age  learning  requirements  and  learner-defined  knowledge  building?  If  the  individualization  of 
economic  activity  continues,  what  role  will  educational  institutions  play  in  defining  or  providing  education?  Does  the  personalization  of 
education  erode  or  enhance  the  role  of  educational  institutions? 

Conclusion 

Driven  by  a need  for  improved  commercial  skills  and  abilities  learners  are  rejecting  the  constraints  of  semesters  and  grades  for  the  realities  of 
deadlines  and  competencies.  The  growth  of  asynchronous  distant  learning  systems  has  been  dramatic,  reflecting  both  the  new  availability  of 
learning  through  new  technologies,  and  the  growing  demand  for  training  and  education.  Knowledge  workers  need  to  respond  to  demands  for  new, 
more  valuable,  knowledge  on  an  accelerating  time  scale.  Traditional  educational  institutions  entering  the  distance  education  marketplace  are  now 
faced  with  a similar  dilemma;  the  multifaceted  role  of  information  provider,  curricular  coordinator,  skill  assessor,  and  content  producer. 
Institutions  are  now  faced  with  competition  not  from  other  institutions  or  organizations,  but  from  individuals  operating  outside  of  traditional 
knowledge  production  frameworks.  Without  the  capital  investment  needed  to  create  a scriptorium,  library,  or  research  lab  entrepreneurs  can 
respond  to  the  demands  of  a new  economy  in  ways  that  slow  moving  institutions  cannot.  Research  into  the  improvement  of  learning  using  new 
media  technologies,  in  different  contexts,  is  not  just  important.  It  is  vital  to  the  improvement  of  educational  systems  that  take  advantage  of  each 
medium. 
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Introduction 

A poorly  planned  and  developed  distance  education  program  will  lead  to  negative  learning  outcome.  For  example,  when  a Texas 
college  began  to  offer  on-line  courses  taught  by  instructors  without  the  skills  and  experience  to  teach  on  the  Internet,  only  about 
40%  of  students  who  enrolled  in  the  on-line  courses  completed  them  (Simpson,  McCann,  & Head,  1999).  From  the  same  logic,  a 
poorly  designed  Web  course  interface  will  have  a negative  impact  on  the  entire  distance  education  program. 

This  paper  discusses  the  roles  of  instructional  design  and  interface  design  in  the  development  process  of  a Web  course.  Although 
the  principles  of  both  instructional  design  and  interface  design  emphasize  the  importance  of  formative  evaluation,  formative 
evaluation  (which  should  be  conducted  to  assess  the  quality  of  the  Web  course  instruction  and  interface)  is  often  overlooked  or 
ignored  in  this  process.  Formative  evaluation  involves  (1)  collecting  a third  party  content  expert's  feedback  to  the  instruction  and 
target  learners’  experiences  with  the  instruction  and  interface  and  (2)  taking  these  evaluation  results  into  consideration  when 
revising  the  Web  course.  The  purpose  of  conducting  such  formative  evaluation  is  to  further  improve  the  Web  course  instruction 
and  interface  to  maximize  learner  comfort  and  thus  their  learning  outcome. 

Instructional  Design  and  Interface  Design 

In  the  ideal  context  of  Web-based  instruction  development,  instructional  design  is  a process  in  which  a content  expert  (the 
instructor),  a distance  learning  instructional  designer,  and  a course  production  team  (that  composes  of  an  editor,  a Web 
programmer,  and  a multimedia  developer)  work  together  developing  a pedagogically  sound  Web  course.  According  to  the  Dick 
and  Carey  systems  approach  model  (Dick  & Carey,  1996),  the  instructional  design  process  involves  detemiination  of  the 
instructional  goal,  analysis  of  the  instructional  goal,  learner  and  context  analysis,  development  of  perfomiance  objectives, 
development  of  assessment  instruments,  development  of  instructional  strategy,  development  and  selection  of  instruction, 
fomiative  evaluation,  revision  of  instruction,  and  summative  evaluation. 

In  this  instructional  design  model,  if  the  instructional  strategy  has  been  determined  to  be  Web  delivery,  then  the  next  stage- 
development  and  selection  of  instruction— will  focus  on  the  instructional  content  level  of  the  Web  course,  such  as  choosing  the 
appropriate  instructional  materials  for  the  intended  learners,  modifying  the  existing  format,  structure  and  sequence  of  these 
materials  to  which  learners  are  presented,  and  adding  links  to  Web-based  instructional  resources. 

The  literature  indicates  that  an  important  factor  to  the  success  of  Web-based  instruction  is  the  integration  of  interface  usability 
design  into  the  development  process  (Henke,  1 997).  A Web  course  interface  design  defines  the  surface  level  of  the  Web  course 
(that  is,  the  visible  structure  that  helps  learners  accomplish  the  goal  of  content  learning),  Web  courses  produced  by  using  HTML 
coding,  Web  Course  in  a Box,  WebCT,  or  other  tools  appear  differently,  due  to  their  unique  interfaces.  An  interface  includes  the 
following  features: 

(1)  what  students  will  see  and  hear  on  a web  page  within  the  Web  browser  (e,g.  hyperlinks,  text,  images,  animations,  audio  and 
video  clips,  streaming  audio  and  video  media,  frames),  and 

(2)  any  responses  (e.g.  feedback,  change  of  web  page,  error  message)  to  students'  actions  such  as  clicking  on  a hyperlink  on  a 
Web  page  or  the  "Back"  button  in  the  Web  browser. 

Regardless  which  Web  course  production  tool  one  uses  (HTML  coding  or  Web  course  delivery  software  like  Courseinfo,  Lotus 
LeamingSpace,  Web  Course  in  a Box,  or  WebCT),  it  is  crucial  that  a learner-centered  Web  course  interface  should  address  the 
goals  of  user-interface  design:  time  to  learn,  speed  of  performance,  rate  of  errors  by  users,  retention  over  time,  and  subjective 
satisfaction  (Shneiderman,  1998),  As  mentioned  earlier,  Wedman  is  concerned  of  students'  "subjective  satisfaction"  of  his 
School's  on-line  master's  program. 

Formative  Evaluation 

In  the  Web  course  development  process,  an  in-house  review  of  a Web  course  that  can  be  called  an  alpha  testing  (in  software 
engineering,  the  first  test  that  is  often  perfomied  only  by  users  within  the  organization  developing  the  software)  consists  of 
instructional  designer's  initial  review  and  feedback  to  faculty,  faculty's  revision,  editor's  editing,  instructional  designer's  review 
and  feedback  to  editor,  editor's  re-editing,  programmer's  HTML  coding,  instructional  designer's  review  and  feedback  to 
programmer,  programmer's  revision,  faculty's  final  review,  programmer's  final  revision.  What  is  usually  missing  is  the  beta 
testing  which  requires  a group  of  learners  who  are  representatives  of  the  target  learners  due  to  practical  difficulty.  Beta  tests, 
usability  tests,  or  fomiative  evaluations  (in  the  instructional  design  process)  have  one  thing  similar— the  evaluation  of  the  use  of 
instruction  and  interface  by  actual  learners  or  users. 
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A formative  evaluation  can  be  conducted  to  assess  the  quality  of  the  Web  course  interface  that  includes  the  Web  pages  of  general 
course  information,  and  the  first  two  lessons  (if  time  permits,  the  entire  course).  The  following  is  an  example  of  an  ideal 
formative  evaluation  of  a Web  course  based  on  Dick  and  Carey  systems  approach  model  of  instructional  design  (Dick  & Carey, 
1996):  (1)  subject  matter  expert  evaluation,  (2)  one-to-one  evaluation,  (3)  small-group  evaluation,  and  (4)  field  trial. 

Data  Collection  and  Analysis 

Technology  plays  a number  of  roles  in  the  data  collection  and  analysis  processes  during  the  fomiative  evaluation.  First  of  all,  for 
the  subject  matter  expert  evaluation,  the  subject  matter  expert  may  live  in  a remote  town  away  from  the  institution,  so  frequent 
travel  to  the  institution  will  not  be  a feasible  practice.  As  long  as  the  subject  matter  expert  has  access  to  the  Internet,  he  or  she 
can  review  and  evaluate  the  course  content  on  the  web,  and  make  comments,  corrections,  and/or  questions  in  an  e-mail  message 
sent  to  the  instructor  who  developed  the  course.  Through  e-mail  exchange  between  the  subject  matter  expert  and  the  instructor 
when  discussing  possible  revisions  in  the  course,  the  instructional  designer  will  eventually  be  notified  on  actual  changes  needed 
for  the  course  based  on  these  two  parties  consensus. 

Second,  for  the  one-to-one  evaluation,  the  technologies  used  are  similar  to  an  interface  usability  test.  The  entire  evaluation  will 
be  taped  by  using  a video  camera,  and  the  video  tape  will  be  reviewed  and  studied  to  observe  the  learner's  facial  expression, 
emotion  changes,  body  language,  and  spoken  language  (while  the  think  aloud  protocol  will  be  used)  which  will  be  also 
transcribed  into  words  for  further  qualitative  text  analysis.  The  text  analysis  will  make  use  of  a software  called  HyperQual  or  a 
similar  text  coding/analysis  software. 

Third,  for  the  one-to-one  evaluation,  small -group  evaluation,  and  field  Trial,  a web-based  testing  software  (which  can  be  used 
for  survey  purpose)  like  QuestionMark  Perception  will  be  used  to  collect  both  quantitative  data  and  qualitative  data.  If  a 5-point 
Likert  scale  of  attitude  questionnaire  is  used,  the  questions  can  be  entered  in  Perception.  After  learners  complete  the 
questionnaire.  Perception  is  capable  of  conducting  basic  statistical  analyses  of  the  quantitative  data  collected  such  as  mean 
scores,  medium  scores,  standard  deviations,  and  frequencies.  Such  data  can  be  also  transferred  to  a statistical  software  (e.g.  SPSS 
for  windows)  for  further  statistical  analyses  or  a database  software  (e.g.  Microsoft  Access)  for  data  storage  and  possible  future 
retrieval  for  further  analyses.  Likewise,  if  the  lessons  contain  some  form  of  objective  posttests.  Perception  can  be  used  to 
administer  these  tests,  and  thus  results  of  student  performance  can  be  obtained  by  looking  at  the  test/measurement  analyses 
Perception  conducted  and  by  further  analyzing  the  data  in  other  software  packages  mentioned  earlier. 

Perception  can  also  allow  learners  to  enter  open-ended  information  when  answering  attitude  questions  or  answering  subjective 
lesson  assignment  questions.  The  answers  will  be  either  kept  on  the  Perception  database  on  the  web  or  sent  to  the  instmetor  or 
the  instmctional  designer  for  grading  or  evaluation.  Again,  for  responses  to  attitudinal  questions,  the  responses  can  be  sorted  or 
coded  in  a text  analysis  that  will  make  use  of  a software  called  HyperQual  or  a similar  text  coding/analysis  software. 

Fourth,  for  the  one-to-one  evaluation,  small-group  evaluation,  and  field  Trial,  the  instructional  designer  can  use  e-mail  or  phone 
to  follow  up  with  the  learners  to  check  on  the  accuracy  of  the  designer’s  interpretation  of  their  comments,  suggestions,  criticisms, 
and/or  questions. 

Conclusions 

In  order  for  the  Web-based  instmetion  design  department  to  process  the  same  number  of  courses  with  the  inclusion  of  formative 
evaluation,  more  staff  members  will  need  to  be  employed,  and  the  Web-based  instmetion  design  department  will  need  to  identify 
sources  that  can  allocate  such  money  for  additional  staff  reemitment.  If  it  is  impossible  to  secure  adequate  funding  for  more 
staff,  a compromised  formative  evaluation  will  have  to  be  tailored  (that  is,  to  cut  portions  of  the  evaluation  as  outlined  above)  to 
fit  the  existing  monetary  and  human  resource  stmeture. 
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Introduction 

As  noted  by  Speziale  and  Kachurick  (1996),  "advances  in  technology  have  provided  numerous  opportunities  for  educational 
integration;  however,  inadequate  professional  development  programs  have  seriously  limited  teachers'  knowledge  of  techniques 
for  integrating  instructional  technologies."  In  an  effort  to  solve  this  problem,  instructional  technology  professional  development 
programs  have  been  founded  in  many  major  universities  and  large  community  colleges  in  the  United  States  (Stoloff,  1 999). 

Recognizing  the  emerging  needs  for  distance  learning  and  the  integration  of  instructional  technology.  Southwestern  Illinois 
College  has  launched  a professional  development  program  in  instructional  design  and  technology,  an  innovative  service  to  its 
faculty,  in  the  summer  of  1999.  The  new  program  is  operated  by  the  Instructional  Technology  and  Distance  Learning  department 
under  the  Learning  Resources  Division,  with  a mission  to  assist  members  in  planning,  developing,  implementing,  and  evaluating 
their  instruction  and  to  facilitate  innovative  integration  of  instructional  technology  into  their  on-campus  and  on-line  courses.  This 
institution-specific  case  study  describes  the  implementation  process  of  this  new  program  and  reports  its  recent  perfonuance 
assessment. 

Program  Services  and  Facilities 

The  professional  development  program  is  led  by  the  instructional  technologist/design  specialist  with  assistance  from  and 
collaboration  with  other  staff  members  (including  the  associate  director  of  distance  learning,  the  Internet  systems  manager,  a 
Web-based  instruction  coordinator,  a Web  programmer,  an  Internet  support  technician,  and  a network  technician)  in  the 
Instructional  Technology  and  Distance  Learning  department. 

A.  Facilities:  The  long-term  goal  of  the  program  is  to  implement  a faculty  development  facility  called  the  “Instructional 
Design  & Technology  Center”  that  consists  of  a computer  lab  for  training  purpose  and  a multimedia  work  area  for  production 
purpose.  To  get  things  started,  a computer  workstation  with  a Pentium  II  dual  processor  has  been  built  and  is  available  for 
faculty  use.  Additional  hardware  includes:  a video  capture  card,  a flatbed  scanner,  a CD  writer,  a ZIP  drive,  a JA2  drive,  two 
digital  cameras,  a color  printer,  and  a laser  printer.  Software  titles  installed  are  Microsoft  Windows  NT  4.0  Workstation, 
Microsoft  Office,  Microsoft  FrontPage,  Adobe  PhotoShop,  OCR  Software,  and  other  multimedia  applications.  In  addition,  a 
Macintosh  G3  computer  has  been  recently  included  in  the  Center  with  some  general  purpose  software  such  as  Microsoft  Office. 

B.  Consultation:  The  program  offers  one-to-one  consultation  to  faculty  with  tailored  information  about  theory  and  practice 
in  technology  applications  for  instruction.  A customized  small  group  session  can  be  arranged  when  there  are  two  or  more  faculty 
members  interested  in  a certain  design  and  technology  issue. 

C.  Training:  Hands-on  instmctional  technology  workshops  on  how  to  use  various  instructional  software  are  offered  to 
faculty,  in  addition  to  pedagogical  seminars  for  faculty  to  facilitate  information  sharing  and  discussion  of  current  issues  on 
design,  development,  implementation,  and  evaluation  of  instmctional  technology  applications. 

Program  Performance 

A,  Instructional  Design  Consultation:  To  increase  the  visibility  of  the  innovation— the  new  professional  development 
program,  a small  but  concise  Web  site  was  built  in  the  summer  of  1999  to  publicize  the  services  and  facilities  being  offered  to  the 
faculty.  As  words  about  the  innovation  regarding  the  professional  development  program  spread  out,  the  number  of  faculty  who 
came  to  the  program  for  consultation  on  instmctional  design  and  technology  has  increased: 

June  1999:  1 faculty  on  1 occasion 
August  1999:  2 faculty  on  2 occasions 
September  1999:  2 faculty  on  2 occasions 
October  1999:  7 faculty  on  14  occasions 
November  1 999:  4 faculty  on  9 occasions 
December  1999:  1 faculty  on  2 occasions 
January  2000:  2 faculty  on  3 occasions 

The  average  length  of  a consultation  meeting  is  about  two  hours.  It  shows  faculty's  enthusiasm  and  willingness  to  devote 
their  time  commitment.  Issues  faculty  members  have  brought  to  the  consultation  meetings  are  diverse.  The  following  list 
demonstrates  the  variety  of  such  topics: 

- Developing  web  pages  for  a campus  course  in  FrontPage 

- Working  with  WebCT  for  an  online  course 

- Discussing  design  issues  on  Social  Science  departmental  website 

- Discussing  and  planning  a Faculty  Web  Directory 

- Designing  a welcome  template  for  an  online  course 

- Designing  a telecourse  support  Web  site  in  Frontpage 

- Working  with  PowerPoint  to  develop  a student  orientation  session 
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B.  Training  to  Meet  Facnity  Needs:  In  order  to  understand  faculty  members’  needs  in  instructional  design  and  technology 
and  to  prioritize  offerings  of  workshops  and  seminars,  a needs  assessment  was  conducted  in  the  form  of  a brief  survey  (Dick  & 
Carey,  1996)  to  130  full-time  faculty  members  in  August,  1999.  The  survey  included  questions  asking  faculty  to  rate  (on  a 1-3 
scale,  with  3 being  highly  interested,  2 being  interested,  and  3 being  not  interested)  their  interest  in  a range  of  workshop  and 
seminar  topics  derived  from  the  text  by  Newby,  Stepich,  Lehman,  and  Russell  (2000).  68  faculty  members  returned  the  survey. 
The  response  rate  of  the  survey  is,  therefore,  52.3%.  The  top  rated  topics  are  listed  below  in  the  descending  order  of  the  mean  of 


Rating 

Topic 

2.27 

Using  presentation  software  in  the  classroom 

2.24 

Creating  instructional  web  pages 

2.1 

Integrating  instructional  technology  in  the  classroom 

1.96 

Using  Internet  communication  tools 

1.87 

Digitizing  and  editing  images 

1.83 

Lessons  learned  from  teaching  an  on-line  course 

1.83 

Creating  an  electronic  gradebook 

1.69 

Fair  use  of  copyrighted  materials  in  on-line  instruction 

1.36 

Introduction  to  distance  education  technologies 

Meeting  the  faculty  needs,  the  program  developed  and  delivered  four  hands-on  workshops  and  two  pedagogical  seminars  till 
February  2000.  Workshop  training  materials,  with  step-by-step  instructions  and  screenshot  illustration  on  how  to  use  the 
software  applications,  were  produced  by  the  workshop  trainer  (who  is  the  instructional  technologist/design  specialist)  and  pilot 
tested  by  other  staff  in  the  department.  With  the  intention  to  reduce  faculty's  computer  anxiety  level  and  not  to  overwhelm 
faculty  by  teaching  them  every  single  feature  of  the  software  applications,  the  workshops  and  training  materials  were  tailored  to 
invite  faculty  members  to  practice  some  the  most  commonly  used  functions  to  get  started.  During  the  hands-on  workshops, 
pedagogical  issues  such  as  design  principles  were  discussed  before  the  hands-on  practice  sessions. 

The  following  table  illustrates  a brief  evaluation  of  the  training  delivered  from  September  1999  till  February  2000.  The 
“No.”  column  is  the  number  of  faculty  who  participated;  the  “Overall”  column  is  the  participants'  average  rating  of  the  overall 
evaluation  of  the  workshop  based  on  a 1-5  scale  (with  5 being  the  highest  rating);  the  “Usefulness”  column  is  the  mean  of  their 


Title  of  Training 

No. 

Overall 

Usefulness 

Workshop  - Creating  Instructional  Web  Pages  with  Microsoft  Frontpage  98:  An  Introduction 

30 

4.77 

4.73 

Workshop  - Designing  Classroom  Presentations:  Basics  of  Microsoft  PowerPoint 

23 

4.87 

4.96 

Workshop  - Designing  Classroom  Presentations:  Advanced  Microsoft  PowerPoint 

13 

4.85 

4.77 

Seminar  - Integration  of  Instructional  Technology  into  Teaching  & Learning:  An  Overview 

15 

4.13 

4.40 

Seminar  - Interaction,  Instruction,  and  Research  Via  the  Internet 

12 

5.00 

4.91 

Workshop  - Image  Digitizing  and  Editing 

26 

4.92 

4.77 

More  workshops  and  seminars  are  under  development  and  will  be  soon  offered  to  faculty  members  in  the  future  so  as  to 
address  one  of  the  commonly  complained  drawbacks  of  a professional  development  program  according  to  national  findings 
regarding  to  professional  development:  infrequency  of  training  (Speziale  & Kachurick,  1996). 

Summary 

Instructional  technology  are  great  tools  that  offer  possibilities  for  improving  the  learning  process,  but  such  possibilities  will 
not  be  realized  automatically  unless  the  teacher  will  have  to  integrate  these  tools  into  the  curriculum  in  a way  to  facilitate 
teaching  and  learning  (Heinich,  Molenda,  Russell,  & Smaldino,  1999).  The  professional  development  program  in  instructional 
design  and  technology  fills  this  gap  between  the  tools  and  the  people  by  helping  faculty  use  the  tools  to  teach  in  a more  effective 
and  efficient  manner. 
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Abstract:  Many  recent  web-based  educational  systems  couldn’t  provide  individualized  instruction 
or  interactive  problem  solving,  since  they  are  mostly  built  upon  static  hypertext.  One  possible 
solution  for  this  problem  could  be  adapting  existing  proven  techniques  from  the  stand-alone 
Intelligent  Tutoring  Systems(lTS)  into  the  web-based  educational  systems.  Recent  web-based  ITS 
research  shows  this  direction  by  employing  the  existing  ITS  techniques  minimally.  And  this  needs 
to  be  studied  further  to  support  more  adaptive  instruction.  In  this  paper  we  describe  the 
development  of  a Web  based  Adaptive  programming  Language  Tutoring  System(WALTS)  which 
is  designed  by  ITS  structure  primarily,  and  adapting  many  successful  ITS  techniques  into  the 
system.  In  addition,  the  system  adapted  CORBA  infrastructure  to  support  the  user  more  consistent 
and  reliable  performance.  Together,  the  system  behaves  more  adaptive  and  interactive  than  the 
existing  static  web-based  educational  systems. 


Introduction 

Many  recent  web-based  educational  systems  could  not  provide  an  individualized  instruction  or  interactive 
problem  solving,  since  they  are  mostly  built  upon  static  hypertext.  One  possible  solution  might  be  adapting  the 
existing  techniques  from  the  stand-alone  Intelligent  Tutoring  System(ITS).  However,  most  of  the  recent  web-based 
ITS  research  show  the  efforts  by  employing  the  techniques  selectively(Kay  & Kummerfeld,  1994;  Brusilovsky  et  al. 
1996;  Nakabayashi  et  al.  1997).  And  this  needs  to  be  studied  further  to  enhance  the  overall  capabilities  of  the  system 
at  the  previous  stand-alone  ITS  level.  In  this  paper  we  will  describe  development  of  Web-based  Adaptive 
programming  Language  Tutoring  System(WALTS).  The  main  goal  of  the  development  of  the  system  is  to  provide 
the  first  year  computer  science  student  a comprehensive  instruction  for  learning  C programming  language,  by 
porting  many  successful  ITS  techniques  into  the  system.  In  addition,  the  system  is  designed  to  support  the  user  more 
consistent  and  reliable  performance  by  employing  CORBA(Common  Object  Request  Broker  Architecture)  structure. 


Figurel;  The  main  Architecture  of  WALTS 
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Basic  architecture  of  the  system 


The  basic  architecture  of  WALTS  is  designed  by  typical  ITS  structure  primarily,  including  expert  module,  the 
student  modeler  module,  and  the  instructional  planning  module. 

The  expert  module  consists  of  the  object-oriented  knowledge  base  and  the  problem  solver.  Since  the  domain 
of  the  system  is  the  C programming  language  concepts,  which  may  not  require  any  complex  causal  relationships  and 
only  requires  mostly  simple  concepts,  we  used  the  frame  knowledge  representation.  The  object-oriented  approach  of 
designing  the  knowledge  base  make  it  easy  to  understand  and  can  provide  more  flexibility  for  updating  or 
manipulating  tutoring  strategy  in  the  construction  of  ITS.  The  problem  solver  creates  problem  table  by  referencing 
the  problem  template  from  the  current  lesson  unit 's  content  table,  which  is  the  form  of  hashtable.  The  solver  stores 
the  correct  answer,  and  then  it  presents  the  generated  questions  to  the  user  in  appropriate  HTML  form  through  the 
HTML  generator.  This  method  can  provide  different  styles  of  questions  for  different  users  even  though  they  are 
accessing  the  same  lesson  unit.  Since  the  column  name  of  the  table  is  object’s  name,  the  planner  can  reply  to  the 
user’s  request,  such  as  hint  or  help,  by  referencing  this  table. 

The  method  of  building  the  student  modeler  is  the  simple  overlay.  The  modeler  keeps  all  the  necessary 
administrative  information  on  the  server,  such  as  initial  student’s  ID,  password,  e-mail  address,  the  access  time.  And 
the  information  regarding  the  student’s  learning  process  is  also  stored  in  the  student  model.  And  each  parameters  has 
unique  meanings;  for  example,  the  HelpCount,  and  HintCount  can  be  updated  only  when  the  unitLesson  is  quiz, 
increase  of  ReferenceCount  means  the  user  is  weak  at  the  current  unitLesson,  the  lessonLevel  stores  information 
about  how  many  times  the  user  visited  the  current  unitLesson,  and  so  on. 

The  Instructional  Planning 

The  existing  web  based  educational  systems  mostly  employ  the  hypertext  techniques,  which  is  hard  to  make 
hyperlink  in  every  HTML  pages,  such  that  it  may  be  inappropriate  for  adaptive  learning  environment  Therefore  we 
need  to  devise  a method  that  can  generate  adaptive  lesson  unit  dynamically.  The  instructional  planning  of  the 
WALTS  can  be  further  divided  into  3 steps,  a curriculum  planning,  a lesson  planning,  and  a delivery  planning.  The 
curriculum  planning  of  WALTS  generates  a curriculum  in  tree  structure;  the  curriculum  planner  extracts  information 
from  the  knowledge  base  and  creates  a curriculum  hierarchically  in  the  order  of  prerequisites.  Then  the  lesson 
planning  sets  up  the  lesson  sequence  within  a single  lesson  unit.  The  role  of  delivery  planning  is  limited  to 
presenting  the  selected  lesson  content  to  the  user. 

Conclusion 

The  main  goal  of  this  paper  is  the  adaptation  of  the  existing  ITS  techniques  to  the  web  platform.  Therefore, 
we  have  designed  and  implemented  the  system  based  on  the  major  ITS  architecture,  and  this  brings  us  several 
advantages  over  traditional  HTML-based  educational  systems.  First,  the  main  knowledge  base  is  created  as  an 
object-oriented  concept,  which  can  provide  more  flexibility  for  updating  or  manipulating  tutoring  strategy.  Second, 
the  system  can  generate  a problem  dynamically  by  the  problem  solver  and  also  can  solve  the  problem  intelligently. 
Third,  the  instructional  planning  mechanism  can  generate  an  instaictional  plan  dynamically.  The  secondary  goal  is 
the  designing  of  the  system  as  a distributed  infrastructure  using  CORBA  as  backbone.  This  structure  solves  the 
bottleneck  problem  of  previous  CGI  dependent  systems,  and  also  gives  some  benefits  of  better  performance  and  also 
gives  flexibility  in  the  case  of  further  enhancement  of  the  system. 
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Abstract:  Online  learning  communities  are  an  excellent  means  of  integrating  collaboration  in  online  learning 
environments.  After  a brief  introduction  we  [the  authors]  define  collaboration  and  learning  communities  and  address 
their  theoretical  foundations.  We  then  discuss  the  benefits  of  collaboration  and  learning  communities  in  online  learning 
environments.  Finally,  we  provide  a case  study  of  a new  online  learning  community  at  Western  Governors  University. 

Definitions  and  Theoretical  Foundations 

Learning  communities  are  environments  that  encourage  mutual  exchange  between  the  community  members  to 
support  their  individual  and  collective  learning.  Learning  communities  are  founded  on  the  aspect  of  social 
negotiation  of  meaning.  Collaboration  is  the  key  tenet  of  constructivist  and  smalLgroup  theories  (Springer  Springer, 
Stanne,  & Donovan,  1999;  Grabinger,  1995;  Duffy  & Jonassen,  1992)  that  explains  the  social  component  of  learning 
and  demonstrates  that  “conceptual  growth  comes  from  sharing  perspectives  and  modifying  our  internal 
representations  in  response  to  that  sharing”  (Grabinger,  1995,  p.  669).  Online  learning  communities  provide  the 
collaborative  means  for  achieving  the  desired  “shared  creation”  and  “shared  understanding”  (Shrage,  1991,  p.  40). 

Benefits  of  Online  Learning  Communities 

Collaborative  learning  and  learning  communities  benefit  learners  because  they  encourage  shared  ways  of  knowing, 
promote  active  participation,  improve  achievement,  contribute  to  the  creation  of  knowledge,  and  challenge  learners’ 
cognitive  abilities  (Springer,  Stanne,  and  Donovan,  1999;  Moller,  1998;  Lave  & Wenger,  1991;  Slavin,  1990). 

Through  new  Internet  technology,  synchronous  and  asynchronous  online  collaboration  now  provides  many  of 
the  benefits  that  exist  in  traditional  face-to-face  collaborative  and  learning  community  environments.  Our  recent 
literature  review  produced  evidence  that  online  collaboration  and  online  learning  communities: 

• Allow  learners  and  instructors  to  work  together  to  create  knowledge  constructions  from  their  combined 
experience  (Jonassen,  1998;  Reeves,  1997) 

• Encourage  learners  to  evaluate  complex  issues  from  multiple  perspectives  and  change  their  perspective  based 
on  others’  input  (Sherry,  1998;  Stone,  1996;  Honebein,  Duffy,  and  Fishman,  1993) 

• Hold  learners  individually  accountable  while  striving  toward  group  goals  so  that  students  help  one  another  and 
assess  one  another’s  learning  (Springer,  Stanne,  and  Donovan,  1999) 

• Provide  opportunities  for  learners  to  reflect  on  their  learning  experiences  and  upon  others’  input  (Stone,  1996) 

• Demonstrate  that  meaning  and  reality  is  socially  constructed  and  negotiated  through  the  exchange  of  ideas, 
information,  and  feelings  among  the  members  of  the  community  (Hiltz,  1 998) 

• Improve  student  achievement  through  increased  motivation,  peer  support  and  communication,  and  commitment 
to  participate  and  finish  (Hiltz,  1998;  Sherry,  1998;  Stone,  1996) 

The  above  evidence  demonstrates  that  online  collaboration  and  online  learning  communities  can  impact  and  enhance 
the  learning  process.  We  have  personally  witnessed  the  impact  of  a self-initiated  online  learning  community  at 
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Western  Governors  University.  Although  the  WGU  online  learning  community  is  relatively  new,  we  have  already 
seen  positive  effects  ripple  through  the  community  members. 

Case  Study  of  an  Online  Learning  Community  at  Western  Governors  University 

In  August  1999,  Western  Governors  University  (WGU)  began  enrolling  students  in  its  new  competency-based 
Master  of  Arts  in  Learning  and  Technology  program  (MLT),  A cohort  of  50  students,  primarily  K-12  teachers, 
geographically  dispersed  throughout  Utah  were  some  of  WGU’s  initial  MLT  students.  In  October  1999,  a small 
group  of  15  cohort  students  created  their  own  collaborative  online  learning  community  utilizing  a listserv  since  all 
students  had  ready  access  to  e-mail. 

The  WGU  online  learning  community  provides  a fascinating  case  study  because  it  was  initiated  by  the  students 
themselves  who  by  and  large  had  no  previous  experience  working  together.  Of  the  15  original  members  of  the 
community,  only  five  had  previous  experience  working  with  each  other  on  previous  projects;  the  majority  of  the 
learning  community  members  had  only  met  each  other  briefly  at  a WGU  reception  in  September  1 999. 

As  we  watch  the  WGU  online  learning  community  develop,  we  are  paying  attention  to  the  ways  that  the 
community  members  negotiate  roles  and  establish  rules  and  standards  for  communication,  interaction,  and  entrance 
into  the  community.  We  are  eager  to  compare  learning  community  members’  performance  to  the  performance  of 
students  who  do  not  participate  in  the  online  learning  community.  We  are  also  interested  in  determining  why  some 
learning  community  members  remain  inactive  or  decide  to  leave  the  community. 

Conclusion 

Salomon  and  Perkins  (1998)  state  that  “learning  to  learn...  fundamentally  involves  learning  to  learn  from 
others,  learning  to  learn  with  others,  learning  to  draw  the  most  from  cultural  artifacts  besides  books,  learning  to 
mediate  others’  learning  not  only  for  their  sake  but  for  what  that  will  teach  oneself,  and  learning  to  contribute  to  the 
learning  of  a collective”  (p.  17).  Collaborative  online  learning  communities  provide  community  members  the 
chance  to  learn  from  and  with  others  and  to  contribute  to  others’  learning.  As  a result,  they  are  important  means  of 
integrating  the  social  aspect  of  learning  into  online  learning  environments.  Our  literature  review  and  personal 
research  of  the  WGU  online  learning  community  provide  valuable  insight  into  issues  surrounding  online 
collaboration  and  online  learning  communities’  development,  communication  patterns,  member  roles,  rules, 
flexibility,  sustainability,  and  reproducibility. 
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Abstract:  Encouraging  effective  collaborative  learning  supported  by  computer 
technology  requires  coordination  among  faculty,  administrators,  technical  and 
instructional  support  staff,  and  library  staff.  This  paper  describes  the  roles  of  each  of 
these  groups  in  the  design  and  use  of  computer  technology  to  support  collaborative 
learning.  This  paper  draws  on  collective  experience  from  implementing  Project  Vision 
and  Project  Empower,  two  computer-supported  collaborative  learning  programs  in 
Commonwealth  College,  Penn  State  University. 


Too  often  the  discussion  of  collaborative  learning  and  use  of  computer  technology  to  support  instruction 
focuses  solely  on  the  classroom;  in  fact,  successful  efforts  in  the  classroom  rest  on  a foundation  of 
collaboration  and  support  from  administrators,  technical  and  instructional  support  staff,  and  library  staff. 
Over  the  last  several  years  Penn  State  University  Delaware  County  has  developed  teaching  methods  that 
incorporate  the  best  features  of  distance  education  and  collaborative  learning  in  undergraduate  education. 
The  two  projects  most  directly  responsible  for  the  development  of  collaborative,  technology-enhanced 
instruction  were  Project  Vision  and  Project  Empower,  both  programs  for  freshmen  and  sophomores  in 
general  education  courses. 


Administrators 

Important  administrative  support  issues  include  motivating,  training,  and  supporting  faculty  to  take  on  new 
teaching  roles  and  new  responsibilities,  since  collaborative  teaching  and  use  of  computer  technologies 
differ  significantly  from  the  standard  lecture/discussion  format.  Administrators  must  make  sure  that  the 
reward  system  for  faculty  recognizes  the  effort  necessary  to  change  from  the  old  to  the  new  model;  we 
provided  release  time  and  training  for  faculty  to  learn  new  roles.  Administrators  are  also  responsible  for 
making  sure  that  appropriate  equipment  and  software  for  courses  is  installed  properly  and  works  efficiently. 
Persuading  faculty  to  work  cooperatively  with  librarians  and  with  technical  and  instructional  support 
personnel  is  also  an  administrative  role.  Administrators  may  also  be  responsible  for  the  selection  and 
orientation  of  technical  and  instructional  support  personnel,  who  must  understand  the  academic  mission 
and  the  instructional  process.  Finally,  administrators  must  be  well  versed  in  designing  and  conducting 
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collaborative  learning  in  technology-enhanced  classrooms,  because  they  are  responsible  for  the  evaluation 
of  the  quality  as  well  as  the  economic  viability  of  these  efforts. 


Faculty 

Graduate  training  seldom  includes  a component  on  teaching  collaborative,  technology-enhanced  courses. 
Hence,  faculty  must  learn  how  to  design,  manage,  and  evaluate  collaborative  assignments,  as  well  as  how 
to  choose  appropriate  technologies  to  enhance  learning  in  their  specific  venues.  Faculty  must  be  expert 
users  of  the  technology  in  order  to  explain  the  technology  to  inexperienced  students  who  are  struggling 
concurrently  with  course  content.  Faculty  always  balance  the  advantages  of  new  instructional  media  against 
many  constraints,  including  the  students’  abilities  and  interests,  classroom  facilities,  and  the  quality  of 
technical  support.  They  must  also  balance  the  time  they  spend  in  these  activities  against  other 
responsibilities  of  teaching,  service,  and  research.  Computer  technologies  are  constantly  changing; 
therefore,  faculty  who  are  committed  to  these  pedagogies  know  that  the  process  of  learning  new 
technologies  and  the  balancing  act  are  never-ending. 


Technical/Instructional  Support  Staff 

Technical  and  instructional  support  staff  work  with  faculty  from  a wide  variety  of  disciplines,  as  well  as 
with  administrators  and  students.  Math,  social  sciences,  humanities,  and  business  all  have  different  content 
and  different  pedagogies.  Technical  and  instructional  support  staff  must  understand  a wide  variety  of 
technologies  and  software  programs  appropriate  for  specific  disciplines  to  help  faculty  make  informed 
choices.  Support  staff  often  advise  faculty  in  designing  assignments  that  make  the  best  use  of  collaboration 
and  computer  technologies.  Technical  and  instructional  support  staff  must  understand  students’  abilities 
and  interests  in  order  to  recommend  the  most  appropriate  technologies  for  each  particular  instructional 
situation,  keeping  in  mind  budgetary  constraints.  They  train  faculty  and  students  and  trouble-shoot  daily 
problems  which  inevitably  arise.  Close  collaboration  with  faculty,  administrators,  and  students  requires 
superior  communication  skills.  A sense  of  humor  doesn’t  hurt  either. 

Library  Staff 

The  new  technology-enhanced  and  collaborative  classrooms  are  changing  the  roles  of  librarians  as  well  as 
faculty.  Library  staff  must  constantly  keep  up  to  date  with  emerging  technologies,  a not  inconsiderable 
task  with  the  rapid  development  of  electronic  media.  Librarians  are  responsible  for  designing  systems  for 
accessing  information  that  make  sense  to  users.  Increasingly,  librarians  find  themselves  acting  as  coaches 
and  collaborators  for  instructors  who  are  learning  to  teach  in  collaborative  and  technology-enhanced 
modes.  Library  staff  also  train,  coach,  and  support  students  in  learning  to  use  search  technologies  to  find 
information.  Library  staff  share  responsibility  with  faculty  for  teaching  critical  thinking  skills  as  they  teach 
students  to  evaluate  the  quality  and  usefulness  of  the  information  students  are  using. 


Conclusions 

Commonwealth  College  of  Penn  State  University  created  Project  Vision  to  encourage  faculty  to  redesign 
courses  to  incorporate  collaborative  and  active  learning  strategies  supported  by  appropriate  technology  in 
teaching  undergraduate  education  courses.  The  administration  offered  release  time,  extra  compensation, 
and  training  to  prepare  selected  faculty  for  the  pilot  program  which  was  designed  by  faculty.  Over  the  four 
years  of  this  program,  the  faculty  tried  several  models  for  offering  courses.  Once  all  the  students  took  only 
Project  Vision  courses  for  the  first  year  as  a cohort;  in  other  years,  students  took  some  Project  Vision 
courses  and  some  regular  courses.  For  two  years,  the  faculty  offered  courses  coordinated  across  two 
different  locations.  Usually,  the  courses  were  offered  as  a package.  For  example,  students  took  Speech 
Communication  to  learn  how  to  work  in  groups.  Library  Studies  to  learn  how  to  do  research,  and  Health 
Science  as  the  content  of  the  research  project.  The  technologies  included  FirstClass  computer 
conferencing,  Eudora  e-mail,  PowerPoint,  and  video  conferencing. 
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Introduction 

We  proposed  a method  using  digital  class  video  for  review  and  self-studying  and  applied  it  to  some 
subjects.  We  believe  our  method  is  successful  in  actual  education.  Students  will  make  their  own  notes 
watching  digital  class  video  in  the  same  way  they  make  their  own  notes  during  ordinary  classes.  If  students 
watch  digital  class  video  for  their  study  many  times  over,  their  notes  will  be  essential.  In  addition  to  digital 
class  video,  we  propose  synchronized  notes  for  digital  class  video. 

Digital  Class  Video 

In  many  educational  organizations,  networks  are  used  for  educational  support.  On  the  other  hand,  there  are 
many  traditional  classes  that  have  no  educational  support  using  computer  networks.  When  one  intends  to 
utilize  computer  networks  or  the  Internet  for  traditional  classes,  he/she  will  suffer  from  various  problems 
such  as  time  for  investigating  teaching  materials,  manpower,  arrangements  of  equipment  and  preparation  of 
presentation  data. 

To  solve  those  problems,  we  proposed  a method  using  digital  class  video  for  introducing  educational 
support  utilizing  computer  networks  for  traditional  classes.  Our  five  goals  of  the  proposed  method  are:  the 
purpose  of  the  method  is  review  and  self-studying;  the  method  could  preserve  the  various  teaching  style  of 
teachers;  the  method  should  be  simple  so  that  the  number  of  staffs  and  a budget  should  not  be  large;  the 
method  is  able  to  be  adopted  to  traditional  lectures  and  computer  exercises;  more  than  one  simultaneous 
classes  should  be  supported  using  the  method. 

To  achieve  above  goals, 

• The  scene  of  a class  is  recorded  on  video  tape  as  it  is. 

• The  camera  for  recording  teacher's  movements  has  fixed  view  and  never  follows  of  teacher's  movements. 

• A written  material  or  a computer  screen  of  the  teacher  operates  is  also  recorded  on  another  video  tape. 

These  two  videos  are  synthesized  to  a single  image  for  publication. 

As  experiments,  we  applied  our  proposed  method  to  five  subjects  in  1998.  The  experiments  are  still 
continued  in  1999.  The  number  of  total  access  times  to  the  video  was  1300  in  1998.  We  believe  that  our 
educational  assistance  method  is  successful. 

Synchronized  Notes 

To  study  subjects,  just  watching  class  is  not  enough  for  studying.  Students  ordinarily  make  notes  on  class 
with  a notebook  and  a pen.  (Nowadays,  some  students  use  their  own  lap-top  computer  for  their  notes  in 
classes.)  When  students  watch  class  video  on  the  computer,  it  is  preferable  for  students  to  make  notes  on 
the  computer.  To  make  notes  on  the  computer  while  watching  digital  class  video  is  simple  because  they  can 
easily  use  another  window  for  a text  editor  or  something  like  that.  In  that  way,  students  are  able  to  review 
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their  lessons  again,  by  watching  video  and  their  notes.  As  Notes  is,  however,  "static"  information,  students 
should  synchronize  notes  and  video  by  themselves  during  watching  video. 

To  synchronize  notes  and  video,  we  implemented  a tel  op  function  to  our  MPEG  video  viewer  software. 
Users  are  able  to  put  comments  or  notes  to  video  as  a telop  just  when  they  are  needed.  Once  users  put 
comments  or  notes  to  video,  the  telop  come  up  on  the  screen  at  the  designated  time  whenever  they  watch 
video.  Fig.  1 shows  an  example  of  telop  function. 


Figure  1.  An  example  of  synchronized  notes 


If  user  put  comments  to  video,  a mark  is  shown  on  the  scroll  bar  so  that  users  are  able  to  find  where  they 
put  their  comments.  Scroll  bar  of  the  MPEG  viewer  is  used  to  forwarding  and  rewinding  video. 

The  mark  on  the  scroll  bar  is  also  used  to  find  important  points  of  the  video.  In  our  method,  the  entire  class 
is  recorded  on  the  video.  Usually  teachers  use  a lot  of  time  for  thinking,  joking  or  something  like  that.  The 
mark  is  used  to  avoid  overhead  of  watching  unnecessary  parts  of  video. 

In  the  same  way,  not  only  users  (students)  but  also  teachers  are  able  to  put  comments  to  video.  After  the 
class,  teachers  sometime  notice  there  were  mistakes  or  insufficient  explanation  during  the  class.  In  such 
case,  teachers  are  able  to  add  additional  explanation  or  corrections  to  video  as  a telop. 

Conclusion 

We  implemented  synchronized  notes  to  MPEG  viewer.  Users  are  easily  able  to  add  comments  and  notes  to 
class  video  for  their  self-studying.  After  the  class,  teachers  are  also  able  to  add  additional  comments  or 
explanation  to  their  class  video.  The  softwares  are  implemented  as  WindowsNT  and  UNIX  (FreeBSD)  X- 
window  applications  so  that  students  are  able  to  study  on  any  plathome. 
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Abstract:  In  recent  years,  popularized  technologies  such  as  “Multimedia”  and  “Internet"  came  to  be  used  in 
various  fields.  The  field  of  education  is  not  an  exception  either,  and  these  technologies  are  adopted  by  CAL 
(Computer  Assisted  Learning).  Especially,  the  successful  introduction  of  these  technologies  into  Japanese 
education  is  being  looked  forward  to.  Since  Japanese  as  the  2nd  language  is  said  to  be  high  in  the  degree  of 
difficulty  in  acquisition,  an  improvement  in  the  study  efficiency  through  adopting  such  technologies  is  expected. 
This  paper  focuses  on  Kanji  of  Japanese  language  education  and  discusses  the  multimedia  Kanji  dictionary. 


Introduction  & Background 

At  present,  there  are  many  non-Japanese  who  are  learning  Japanese  in  the  world.  However,  with  a limited 
time  allocated  for  study  in  classes,  the  present  condition  does  not  offer  sufficient  education  for  mastering  Japanese.  For, 
not  only  are  there  problems  related  to  the  basic  grammar  such  as  the  word  order  in  Japanese  is  sometimes  different  and 
the  use  of  the  auxiliary  verb  is  more  complicated  in  comparison  with  other  languages  but  also  there  are  three  kinds  of 
characters  used:  "Hiragana,"  "Katakana"  and  "Kanji."  Especially,  quantity  in  Kanji  is  voluminous  and  the  shapes  are 
complicated.  Even  for  Japanese,  it  takes  6 years  to  learn  over  1000  Kanji  characters.  Furthermore,  so  much  information 
is  contained  in  Kanji  (for  example,  reading,  meaning,  radical  and  the  stroke  count,  etc.).  Therefore,  the  more  efficient 
Japanese  education  system  is  necessary  because  it  is  excruciating  for  people  in  the  countries  where  Kanji  are  not  used  to 
learn  them  in  a short  period  of  time.  When  people  in  the  countries  where  Kanji  are  not  used  learn  their  mother  tongue, 
they  emphasize  "conversation."  Therefore,  even  when  they  learn  Japanese,  they  emphasize  "conversation."  In 
comparison  with  Japanese,  a one  byte  code  language,  in  the  first  place,  has  a fewer  number  of  characters  but  has  a 
number  of  types  of  pronunciation.  Therefore,  such  a language  can  be  mastered  mainly  by  improving  conversation 
ability. 

From  such  a reason,  for  people  of  the  countries  where  Kanji  are  not  used  inevitably  neglect  learning  of  Kanji. 
To  effectively  learn  Kanji,  the  multimedia  Kanji  dictionary  was  developed.  Since  the  electronic  dictionary  facilitates 
the  understanding  of  Kanji  by  using  animation  and  sound,  it  can  improve  learning  efficiency  much  more  than  a 
conventional  paper  dictionary.  Also,  the  characters  of  the  index  are  so  small  that  we  sometimes  become  fed  up  with 
looking  up  Kanji  in  the  paper  dictionary.  However,  people  will  become  able  to  search  Kanji  more  easily  by  just 
clicking  if  they  use  the  multimedia  Kanji  dictionary.  This  electronic  dictionary  is  functioning  as  the  dictionary  module 
of  an  integrated  Japanese  learning  system  called  "Terakoya." 

URL:  http://www.narita.elec.waseda.ac.ip/-fuiita/terakova.html 


Kanji  Dictionary 

This  dictionary  system  can  be  divided  into  the  Kanji  searching  part  and  dictionary  database  part.  In  the 
searching  part,  there  are  three  kinds  of  functions  "Read,"  "Stroke  count"  and  "Radical."  Here,  we  can  find  a Kanji 
character  for  which  we  are  searching,  and  we  can  read  out  detailed  information  from  the  dictionary  database  that  is 
stored  as  a CSV  file.  On  the  one  hand,  this  system  contains  detailed  information  such  as  radicals  and  the  stroke  count 
for  people  who  want  to  learn  detailed  information.  On  the  other  hand,  this  system  facilitates  easy  learning  of  Kanji  by 
including  items  for  children  such  as  pictures  and  figures  that  look  similar  to  Kanji.  This  system  has  become  an 
interactive  system  that  has  more  mobile  capability  by  using  animation  and  sound,  which  is  in  contrast  to  the 
conventional  dictionary  system  that  emphasized  text  data.  Animation  is  used  to  display  the  order  of  strokes  of  a Kanji, 
and  voice  is  used  to  let  the  learner  hear  the  data  of  "Reading"  and  "Examples."  Furthermore,  in  order  to  have  the  users 
learn  a large  quantity  of  Kanji,  this  system  understands  Kanji  through  graphic  means  by  recognizing  each  Kanji  as  a 
picture  or  a figure  to  improve  learning  efficiency. 

And,  because  this  system  is  using  WWW,  it  is  not  dependent  on  a particular  type  of  platform.  A learner  can 
use  it  through  the  Internet  from  PC  regardless  of  whether  the  person  is  at  home  or  in  a class.  Even  if  there  was  no  such 
environment,  in  such  case  information  of  Kanji  can  be  read  by  printing  the  contents  out  on  paper  as  a conventional 
dictionary.  Although  this  system  is  developed  with  Shockwave  of  Macromedia  Company,  the  browser  is  not  able  to 
function  as  a printer  when  the  movie  that  was  made  with  Shockwave  is  loaded  into  the  browser.  Therefore,  a page 
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where  Kanji  information  is  transformed  into  an  HTML  has  been  made.  If  the  hard  copy  of  Kanji  information  can  be 
made,  it  can  be  used  at  places  where  there  is  no  computer  nor  a network  environment.  This  function  comes  from  the 
requests  of  the  Japanese  language  teachers  and  also  of  Japanese  language  learners. 


Future  Plans 

Although  this  dictionary  system  has  searching  functions  of  "Read,”  "Stroke  count"  and  "Radical,"  these 
searching  methods  are  used  conventionally  in  the  conventional  paper  dictionary.  However,  there  must  be  cases  where  a 
beginner  of  Kanji  learning  has  no  idea  about  the  reading,  the  stroke  count  nor  the  radical  of  the  Kanji  that  he/she  wants 
to  find  out.  To  this  end,  we  are  studying  the  functions  by  which  a Kanji  can  be  searched  through  handwritten  letter 
recognition.  If  this  searching  method  is  created,  a learner  can  search  for  the  Kanji  by  the  action  that  is  similar  to 
drawing  a picture  of  the  Kanji  whose  detailed  information  is  not  known  to  the  learner.  Also,  although  the  present 
system  has  registered  80  Kanji  characters  in  the  database,  in  order  to  maintain  coordination  with  the  "Terakoya"  system, 
it  is  necessary  to  classify  Kanji  by  the  degree  of  difficulty  or  by  the  use  frequency  in  daily  life  to  offer  data  to  the  user  in 
an  easy-to-understand  manner.  Furthermore,  because  there  are  some  cases  where  the  requests  of  teachers  differ  from 
the  contents  of  a dictionary,  we  should  add  such  function  that  enables  the  teachers  to  freely  select  the  words  and 
pronunciations,  etc.  that  he/she  wants  to  use  in  the  lesson.  If  there  is  such  a function,  a lesson  will  become  very 
effective.  At  last,  we  need  to  evaluate  this  Kanji  dictionary.  This  system  will  be  tested  by  the  Center  for  Japanese 
Language  of  Waseda  University  and  the  Richmond  school  in  Portland.  And,  we  want  to  advance  to  further 
improvement  by  asking  advices  from  teachers  and  students. 
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Abstract:  The  purpose  of  this  study  was  to  identify  the  similarities  and  differences  of  online  and 
traditional  foreign  language  courses  and  to  determine  the  advantages  and  disadvantages  of  the 
online  courses.  Using  a random  sample  of  20  courses,  ten  on-line  and  ten  traditional,  this  study 
examined  the  online  and  traditional  courses  in  terms  of  five  constructs  and  compared  the  methods 
and  strategies  used  in  online  courses  with  regular  classroom  instruction.  With  a rubric  developed 
for  the  study,  the  online  and  traditional  course  were  rated  in  a 4-point  scale  and  analyzed  with 
independent  t-tests.  The  findings  of  the  study  indicated  that  online  and  traditional  courses  had  about 
the  same  weight  in  teaching  foreign  language  and  suggested  that  they  each  had  its  advantages  and 
disadvantages.  The  study  concluded  that  traditional  courses  had  better  defined  and  more  specific 
goals  and  objectives  for  both  teaching  and  learning  and  was  more  effective  in  teaching  conversation 
and  listening  comprehension,  while  in  online  courses,  students  could  learn  at  varying  rates  and 
explore  material  to  whatever  depth  they  desire,  and  Internet  technology  made  it  possible  for 
students  in  different  areas  and  from  different  countries  to  access  online  courses. 


Introduction 

With  the  rapid  development  and  fast  growing  adoption  of  technology,  the  use  of  technology  in  education  has  been 
dramatically  increasing  over  the  past  few  years.  In  the  area  of  language  education,  various  technologies  are  being  utilized  by 
language  teachers  and  learners  in  and  out  of  the  classroom  (Warschauer,  1996).  These  uses  would  include  the  traditional  use  of 
audio  and  films;  however  they  are  now  being  supplemented  by  computer-assisted  instruction  and  interactive  media  technologies 
(Richards,  1996).  Computer-based  programs,  as  well  as  other  computer-related  interactive  technologies,  have  been  specifically 
developed  to  provide  students  with  visual  and  audio  support.  These  applications  have  shown  potential  for  helping  students 
develop  their  language  abilities  and  improve  classroom  performance  (Pennington,  1996). 

Among  a variety  of  instructional  delivery  media,  the  Internet,  especially  the  World  Wide  Web,  has  been  commonly 
adopted  by  many  educators  in  higher  education  (Khan,  1997;  Porter,  1997).  However,  little  research  results  have  shown  to 
support  claims  for  the  effectiveness  of  Web-based  instruction  (Nunan,  1999).  The  focus  of  this  study  was  an  analysis  of  existing 
web-based  foreign  language  courses  and  a comparison  of  the  methods  and  strategies  used  in  online  courses  and  regular  classroom 
instruction  to  determine  the  common  and  unique  methods  and  strategies  used  to  achieve  the  educational  objectives  in  each 
environment. 

Purpose  of  the  Study 

The  purpose  for  conducting  this  study  was  to  identify  the  similarities  and  differences  of  online  and  traditional  courses 
of  teaching  foreign  languages  and  to  determine  the  advantages  and  disadvantages  of  the  online  courses.  Methods,  strategies,  and 
technologies  used  in  online  foreign  language  courses  were  examined  and  compared  with  those  of  the  traditional  foreign  language 
courses.  In  order  to  achieve  the  purpose  of  the  study,  the  current  study  addressed  the  following  research  questions: 

1.  Based  on  the  courses  examined,  what  methods,  techniques  and  strategies  were  used  for  teaching  online  foreign  language 
courses  and  traditional  foreign  language  courses? 

2.  What  were  the  differences  between  traditional  foreign  language  courses  and  online  foreign  language  courses  in  terms  of 
goal/objectives,  strategies,  interaction,  feedback,  and  resources? 

3.  What  were  the  advantages  and  disadvantages  of  the  online  courses? 

Method 

The  sample  for  this  study  consisted  of  twenty  foreign  language  courses,  with  ten  on-line  courses  and  ten  traditional 
courses.  Thirty  online  courses  were  first  identified  directly  from  foreign  language  courses  available  on  the  Internet,  which 
included  listening,  speaking,  reading,  and  writing  courses.  The  following  systematic  procedure  was  used  to  randomly  select  ten 
courses  from  the  thirty  previously  identified  online  courses:  First,  a random  list  of  the  thirty  online  courses  was  generated.  Then 
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a start  point  for  the  selection  was  randomly  chosen  on  the  list.  From  that  point  every  third  course  was  selected  until  a total  of  ten 
online  courses  were  identified  for  the  study.  Among  the  ten  courses  that  were  chosen  were  two  listening  courses,  one  speaking 
course,  two  reading  courses,  two  grammar  courses,  and  three  writing  courses. 

With  the  same  procedure,  the  ten  traditional  courses  used  for  this  study  were  randomly  selected  from  foreign  language 
courses  offered  by  the  Intensive  English  Programs  and  foreign  languages  departments  at  various  universities.  The  course 
materials  of  these  ten  courses  were  available  at  the  University  of  Louisville,  the  University  of  Alabama,  Southeast  Missouri  State 
University,  and  North  Carolina  Wesleyan  College.  They  were  three  listening  courses,  two  speaking  courses,  two  reading  courses, 
one  grammar  course,  and  two  writing  courses,  which  comprised  the  traditional  foreign  language  course  group  for  data  analysis. 
With  a rubric  developed  for  the  study,  the  online  and  traditional  course  were  rated  on  a 4-point  scale  and  analyzed  with 
independent  t-tests. 

Results  and  Discussion 

The  comparison  by  means  of  the  rubric  indicated  that  the  online  courses  and  the  traditional  courses  had  about  the  same 
weight  in  teaching  foreign  languages  in  general.  Online  courses  had  a total  score  of  147,  while  the  score  of  the  traditional 
classroom  teaching  was  153.  However,  the  independent  t-tests,  which  were  performed  to  identify  any  possible  significant 
difference  between  the  online  and  traditional  courses,  found  that  significant  differences  existed  between  the  two  types  of  courses 
in  two  of  the  five  constructs  measured. 

Conclusion 

A foreign  language  acquisition  process  consists  of  listening,  speaking,  reading,  and  writing  skills.  In  order  to  facilitate 
the  success  of  second  language  teaching  and  learning,  modem  technologies  have  been  used  in  the  classroom  to  promote  teaching 
perfonnance  and  student  learning.  Online  courses,  which  have  been  developed  for  that  purpose,  have  proved  their  advantages  in 
foreign  language  teaching  and  learning.  However,  when  comparing  traditional  classroom  teaching  with  online  courses,  the  results 
of  this  study  suggest  that  each  had  its  advantages  and  disadvantages.  For  traditional  courses,  better  defined  and  more  specific 
goals  and  objectives  were  given  for  both  teaching  and  learning.  Delivering  course  material  through  the  Web,  on  the  other  hand, 
afforded  some  advantages  over  traditional  classroom  teaching.  In  online  courses,  students  could  learn  at  varying  rates  and 
explore  material  to  whatever  depth  they  desire.  There  were  new  ways  to  guide  students  through  tutorials  and  to  provide  them 
with  instant  feedback.  Using  Internet  technology  made  it  possible  for  students  in  different  areas,  different  countries  to  use 
varying  computer  types  to  have  access  to  the  course. 

However,  as  far  as  a foreign  language  is  concerned,  it  is  obvious  that  traditional  classroom  teaching  was  better  in 
teaching  conversation  and  listening  comprehension  courses  when  students  needed  face-to-face  interactions  with  the  instructor  and 
instant  verbal  feedback  from  the  instructor.  Computer-mediated  communication  had  been  criticized  as  lacking  the  human  social 
interaction  necessary  for  human  learning  (Gilbert,  1996)  although  Lemke  (1993)  claimed  that  "cyberspace  will  be  a virtual  place 
FOR  human  social  interaction."  It  seems  that  web-based  courses  might  not  be  able  to  provide  as  effective  or  efficient  a means  of 
learning  a language  as  one-on-one  human  interaction.  Without  such  non-verbal  cues  as  body  language  and  the  tone  and  pitch  of 
one’s  voice,  it  is  possible  to  misconstrue  messages  (Berge  & Collins,  1995;  Gilbert,  1996). 

In  summary,  neither  of  the  two  types  of  courses  is  absolutely  good  or  faulty.  Each  had  its  own  advantages  and 
disadvantages.  As  an  instructional  designer,  we  should  be  aware  of  the  advantages  and  disadvantages  of  both  types  of  courses 
and  make  full  use  of  their  advantages  in  the  design  and  development  of  the  courses,  based  on  different  student  needs,  student 
characteristics,  and  skill  requirements,  to  achieve  the  goals  of  our  education. 
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Abstract:  The  paper  proposes  and  explains  five  common  mistakes  which  institutions  often 
make  in  implementing  distance  education:  using  packaged  courses  and  low-level  instructors, 
using  one  technology  to  deliver  all  distance  education,  not  monitoring  student  satisfaction  and 
retention,  assuming  that  all  courses  can  and  should  be  offered  via  distance  technology,  and 
focusing  marketing  only  on  the  technology-literate. 


Racing  along  the  Road 

The  shape  of  distance  education  is  changing  daily  as  more  and  more  institutions  embrace  the  technologies 
now  available  to  deliver  remote  instruction.  One  estimate  by  Robert  Tucker  (1997)  quoted  recently  in 
Forbes,  is  that  about  90%  of  US  four-year  institutions  will  have  technology-based  distance  learning 
offerings  by  2000.  Even  in  1996,  over  a million  students  took  distance  learning  courses  of  the  14  million 
attending  post-secondary  educational  institutions.  And,  although  these  courses  use  many  different 
technologies,  there  is  rapidly-increasing  focus  on  internet  delivery.  In  this  rush  to  get  on  the  bandwagon, 
the  authors,  who  have  been  involved  with  emerging  distance  education  in  Texas  for  the  last  ten  years,  have 
observed  several  common  assumptions  which  cause  great  concern.  The  focus  of  this  paper  will  be  to 
identify  and  call  into  question  some  frequent  phenomena  which,  we  believe,  threaten  to  undermine  the 
success  of  cyberl earning. 

Wrong  Turn  One:  Buy  a packaged  course  and  hire  a TA  or  two  to  run  it 

One  of  the  driving  forces  behind  the  charge  to  cyberleaming  is  the  common  perception  that  cyberleaming 
will  be  a cash  cow  for  traditional  institutions,  allowing  them  to  cut  costs.  In  a rush  to  milk,  institutions  are 
sometimes  choosing  to  dispense  with  home-grown  courses  run  by  local  professors.  This  choice  threatens  to 
result  in  a world  of  cyberleaming  where  there  are  many  institutions  to  choose  from,  but  all  are  teaching  the 
same  course.  Losing  the  personal  enrichment,  the  personal  mentoring  and  the  constant  updating  which  an 
experienced  teacher  puts  into  a course  is  equivalent  to  losing  the  personal  voice  or  identity  of  an  institution. 
Cyberleaming  should  increase  the  diversity  of  the  educational  ecosystem,  not  decrease  it. 

Wrong  Turn  Two:  Buy  one  technology  and  use  it  for  everything. 

Because  the  costs  for  establishing  technology-based  distance  learning  can  be  large,  institutions  are  prone  to 
make  an  initial  commitment  to  a technology  (VHS  with  two-way  audio,  or  TV,  or  compressed  video,  or 
web)  then  force  all  course  offerings  to  fit  themselves  to  that  technology.  Institutions  have  been  known  to 
simply  buy  whatever  was  pitched  to  them  most  intensely  in  the  hopes  of  not  having  to  face  the  bewildering 
array  of  choices  again  for  quite  some  time.  However,  close  analysis  of  student  needs,  course  content,  and 
educational  effectiveness  has  led  us  to  believe  that  hybrid  technologies  evolved  over  time  and  respondent  to 
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individual  courses  and  student  audiences  produce  superior  educational  results.  Educational  needs  and  goals 
should  drive  technology,  not  vice  versa. 

Wrong  Turn  Three:  Don't  worry  about  student  satisfaction;  the  learning  is  fine 

Indeed,  much  past  research  on  distance  learning  using  all  kinds  of  different  delivery  modalities  from 
correspondence  courses  to  web  has  indicated  that  students  seem  to  learn  as  much  from  distance  courses  as 
they  do  from  face-to-face  instruction  (Russell  1996).  However,  we  cannot  ignore  that  cybercourses  vary 
enormously  in  learner-satisfaction  and  quality.  In  an  ideal  educational  cyber-marketplace,  learner-centered 
course  design  will  eventually  prevail.  Courses  which  use  technology  well  and  flexibly  to  deliver  their 
course  content  and  achieve  course  goals,  and  courses  which  attend  to  a wide  variation  in  student  learning 
styles  will  have  better  success.  Convenience  of  access  is  not  enough;  quality  matters,  too. 

Wrong  Turn  Four:  Move  everything  onto  the  cyber-track. 

With  the  emergence  of  virtual  universities  and  entire  degree  plans  available  by  remote  technology,  the 
instinct  of  many  institutions  is  to  think  of  distance  learning  offerings  as  a substitute  for  face-to-face 
instruction,  rather  than  an  adjunct  to  it.  However,  a closer  look  shows  quite  clearly  that  cyberl earning  has, 
thus  far,  leaned  heavily  in  the  direction  of  engineering,  business,  computer  science  and  health  sciences. 
(Miller  1998).  According  to  Miller,  all  other  subject  matters  account  for  only  10%  of  cybercourses.  In  spite 
of  the  pronouncement  of  Peter  Drucker  (1997)  that  "Universities,  as  we  know  them,  won't  survive,"  almost 
all  lower-division  and  humanities  courses  continue  to  be  offered  in  traditional  fashion.  One  reason  for  this, 
we  believe,  is  the  need  for  greater  flexibility  in  adapting  to  the  goals  of  such  courses  and  the  needs  of 
students.  Cyberleaming  should,  we  believe,  offer  additional  choices  to  students  and  teachers— not  force 
them  into  a course-delivery  system  alien  to  them.  In  fact,  we  go  so  far  as  to  argue  that  cyber  courses  should, 
in  most  cases,  have  live,  face-to-face  versions  running,  too. 

Wrong  Turn  Five:  Just  market  the  courses  and  the  students  will  come 

Certainly  convenience,  accessibility,  and  low  cost  are  strong  incentives  for  students  to  engage  in 
cyberleaming.  However,  issues  of  equity  and  market  penetration  will  become  ever  more  pressing  as 
institutions  seek  to  draw  increasing  numbers  of  students.  Access  to  computers  is  increasing  daily,  as  is 
internet  access;  however  individuals  without  home  computers  or  internet  access  may  be  precisely  the  ones 
who  most  need  access  to  "anytime,  anywhere"  post-secondary  education.  Institutions  must  attend  to 
insuring  access  at  public  libraries,  K-12  schools  and  community  centers  in  order  to  insure  equity  for  all 
ranks  of  society  and  to  avoid  widening  the  digital  divide.  Moreover,  some  segments  of  the  market  will  need 
additional  technology  training  in  order  to  effectively  use  the  access  which  is  available  to  them,  and 
institutions  must  provide  appropriate  training  in  technology  if  they  are  to  draw  high-school  dropouts  and 
retired  persons  into  cyberleaming. 

In  a dynamic  growth  period  such  as  we  face  now,  the  drive  to  implement  cyberleaming  will 
benefit  many  students  and  institutions,  but  only  if  decision-makers  tread  the  path  thoughtfully  and  with 
their  eyes  open  for  wrong  turns. 
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Abstract:  This  paper  sought  to  identify  learner-demonstrated  learning  patterns  when  undergraduate  students  were  learning  to  use  a computer-based  presentation 
program  in  a multimedia  learning  environment  without  time  constraints.  Using  analysis  of  patterns  in  time  (APT),  a methodology  that  can  code  event  changes  over 
time,  the  frequencies  and  the  amounts  of  time  of  the  five  learning  patterns  were  calculated.  The  data  derived  from  the  APT  scores  indicated  two  major  findings:  (1) 
The  amount  of  time  used  by  the  participants  ranged  from  20  to  87  minutes.  The  amount  of  lime  spent  did  not  predict  mastery  of  the  post  test.  (2)  ‘Following 
through  error  corrections’,  ‘confirming  actions’,  and  ‘trying  new  steps’  were  the  patterns  that  did  appear  to  predict  mastery,  and  ‘hand  on  mouse*  and  ‘keeping  up 
step-by-step  with  the  video  instruction’  were  indicators  of  learners’  task  engagement.  The  findings  suggest  that  it  is  important  to  design  flexible  instruction  to 
facilitate  repeated,  persistent,  and  successful  practice  in  order  to  achieve  mastery  in  a self-paced,  computer-based  learning  environment,  regardless  of  the  amount  of 
time  spent. 


Introduction 


This  study  sought  to  investigate  the  existence  of  any  significant  temporal  patterns  of  learner-demonstrated  sequencing  as  learning  styles  that  are  related  to  the  mastery  of  learning  a procedural 
computer-based  multimedia  task. 

The  impact  of  how  different  learning  styles  affect  human  learning  has  concerned  educational  researchers  for  decades  (Bruner,  1966;  Gagne,  1967;  Kolb,  1984;  Wang  and  Walberg,  1985;  Tobias, 
1981,  1987).  According  to  Cronbach  and  Snow  (1977),  learning  style  can  be  used  to  predict  what  kind  of  instructional  methods  or  strategies  would  be  most  elTective  for  a given  individual  and  learning  task. 
Such  attempts  to  discover  the  predictors  for  content  mastery  were  typified  in  Aptitude  Treatment  Interaction  (ATI)  research.  In  ATI  studies,  aptitude  was  defined  as  any  learner  characteristic  related  to 
instructional  outcome,  and  treatment  was  defined  as  any  variation  in  instruction  to  the  learner  (Cronbach  and  Snow,  1969).  However,  research  to  date  on  this  pmblem  has  not  identified  many  robust 
relationships  between  learning  style  and  instructional  method  that  are  needed  in  order  to  generalize  the  assignment  of  instruction  to  different  learners  (Snow,  1989).  Could  this  lack  of  robustness  be  caused  by 
overlooking  the  means  in  favor  of  the  ends  of  learning?  Or  perhaps  the  notion  of  aptitude  needs  to  be  defined  differently  when  the  means  of  learning  (or  learning  process)  is  considered  as  a critical  component 
in  a learning  experience. 

Frick  (1990;  1992)  indicates  that  one  of  the  fundamental  problems  of  ATI  is  that  the  linear  model  approach  (LMA)  employed  by  ATI  cannot  explicitly  demonstrate  the  different  means  of 
learning  because  the  LMA  was  designed  to  estimate  parameters  of  mathematical  models  and  is  not  adequate  in  dealing  with  occurrences  and  relative  frequency  of  temporal  events.  LMA  is  best  for  explaining 
the  relation  between  two  or  more  independently  measured  variables,  but  it  cannot  measure  the  frequency  or  duration  of  occurrences  of  a temporal  path  (Frick,  1983).  This  observation  directly  implies  that  there 
is  a need  for  a methodology  other  than  LMA  when  we  investigate  the  temporal  patterns  in  a learning  process,  especially  with  a concurrent  attempt  to  examine  aptitude  in  a different  way. 

Learning  takes  time.  Learners  take  different  amounts  of  lime  to  perceive  information  in  order  to  acquire  knowledge.  Knowledge  is  actively  constmeted  through  temporal  sequencing  of 
meaningful  interactions  between  the  individual  and  the  learning  context  (Piaget,  1973;  Dewey,  1916;  Bmner,  1973;  von  Glasersfeld,  1991).  But  how  can  we  move  the  learning-styles  research  away  from  the 
instruction-centered  “how  one  should  learn”  to  the  learner-centered  "how  one  does  learn”?  In  order  to  make  instmctional  systems  more  adaptive,  researchers  need  to  pul  more  emphasis  on  leamer-demonslrated 
learning  styles,  where  the  relationships  between  the  sequencing,  time  duration,  and  nature  of  the  media  should  be  carefully  examined.  This  need  is  especially  important  now  that  multimedia,  productivity 
software  programs,  and  the  explosive  growth  of  the  Internet  have  entered  our  educational  settings  and  homes  (Nielsen,  1995),  and  the  technological  capabilities  have  caused  us  to  reconceptualize  the  learning 
process  and  to  design  new  instmctional  approaches  (Duffy  and  Jonassen,  1992). 

To  understand  the  need  to  redefine  learning  styles,  it  is  necessary  to  comprehend  the  nature  and  limitations  of  conventional  learning  styles.  Sewall  (1986)  compares  four  popular  instmments  to 
measure  learning  styles:  the  Myers-Briggs  Type  Indicator  (Myers,  1962),  the  Kolb  Learning  Style  Inventory  (Kolb,  1976),  Canfield’s  Learning  Style  Inventory  (Canfield,  1980),  and  Gregorc’s  Type  Indicator 
(Gregorc,  1984).  He  finds  that  all  of  these  instmments  fall  short  generally  in  four  areas:  (1)  no  appropriate  normative  base  for  the  valid  interpretation  of  scores;  (2)  limited  validity  due  to  constmetion  problem 
with  the  instmments;  (3)  reliability  estimates  are  unstable,  though  one  can  argue  that  the  dynamic  nature  of  learning  style  makes  high  test-retest  reliability  unnecessary;  (4)  the  normative  frame  of  reference 
makes  the  interpretation  of  scores  very  difficult.  Among  his  suggestions  for  future  research,  one  suggestion  particularly  interests  the  present  writer: 

“Future  research  could  also  explore  the  issue  of  whether  an  individual’s  preferred  learning  style  is  modified  by  the  educational  environment.  For  example,  do  adult  students  learn  better  when  instmetion 
is  adapted  to  their  learning  style  preferences?  Can  people  be  trained  to  adopt  a particular  learning  style?  Do  learning  styles  remain  stable  over  time  in  the  adult  population?  Docs  a significant  change  in  life 
situations  result  in  changes  in  learning  styles?”  (p.6l .) 

It  is  interesting  to  notice  that  all  four  of  the  above-mentioned  learning-style  instmments  use  lists  of  verbal  statements  or  different  visual  forms  prepared  for  people  to  choose  or  rank.  This  isolated 
“one  size  fits  all”  approach  is  decontcxtualized  and  artificial,  and  apparently  biased  on  language  skills  and  cultural  backgrounds.  The  predefined  assessment  tools  require  people  to  fit  in  the  limited  and  selective 
choices  with  “I  think  this  is  who  I am”  or  “I  think  this  is  what  I will  do  or  how  I will  react,”  which  may  not  predict  how  one  actually  reacts  to  or  interacts  with  a problem  in  a specific  learning  situation.  This 
predetermined  and  convergent  assessment  approach  may  be  the  fundamental  reason  that  only  limited  reliability  and  constmet  validity  have  been  found  in  existing  learning-style  studies. 

Maybe,  as  Sewall  suggested,  human  learning  styles  arc  constantly  modified  by  the  educational  environment  and  social  context  and  arc  too  dynamic  to  measure  on  a predetermined  and  predictable 
scale.  This  notion  leads  the  present  writer  to  believe  that  to  survey  fully  the  dynamic  nature  of  learning  styles,  rather  than  using  artificial  instmments,  we  can  observe  and  faithfully  record  the  event  changes  of  a 
Icamer-demonstratcd  process  over  time.  In  other  words,  the  result  of  this  study  may  not  verify  that  learning  style  is  a stable  trait  (Bateson,  1979). 

Research  Questions 

I.  In  the  midst  of  the  information  explosion,  multimedia  has  become  the  new  learning  environment.  Learning  how  to  interact  with  multimedia-based  information  or  inslmclions  has  become  our  daily 
experience.  It  is  important  to  note  that  multimedia  is  a combination  of  three  forms  of  content:  (1)  abstract  (e.g.,  verbal  - either  text  or  audio  - explanation  in  a computer  program  window),  (2)  iconic 
(c.g.,  the  screen  capture  of  a computer  program  window  printed  on  paper),  and  (3)  concrete  (e.g.,  the  actual  computer  program  windows  displayed  on  a monitor  screen).  The  main  questions  addressed  in 
this  study  were  as  follows:Given  the  choice  of  instmctional  materials  in  different  forms  of  content,  what  learning  sequences  are  selected  by  the  learners? 

1. 1 . If  the  sequences  differ,  how  do  they  differ? 

1 .2.  Arc  any  of  these  sequences  especially  likely  to  result  in  overall  mastery? 

1 .3.  Is  there  a difference  in  the  amount  of  lime  spent  learning  for  those  learners  who  achieved  mastery  and  those  who  did  not? 

In  order  to  increase  both  reliability  and  validity  in  identifying  the  learning  patterns  (Denzin,  1970;  Foreman,  1948),  a triangulation  inethoil  was  useil  that  involved  observation  of  the  learning 
process  and  think-aloud  interview.  “Tri angulation"  is  to  use  multiple  methods  in  data  collection  (Denzin,  1970).  Merriam  (1988)  indicates  that  “methodological  triangulation  combines  dissimilar  methods  such 
as  interviews,  observations,  and  physical  evidence  to  study  the  same  unit.”  An  observation  system  designed  and  based  on  the  analysis  of  patterns  in  lime  (APT)  approach  was  used  to  code  how  and  how  long 
students  use  and  switch  the  instmctional  materials,  and  their  engagement  during  the  processes  of  learning.  Both  the  learning  processes  and  the  think-aloud  interviews  were  recorded  on  videotapes  for 
examination  and  analysis. 

Twelve  participating  students  from  a midwest  regional  university  were  selected  through  a screening  process  using  Kolb’s  Learning  Style  Inventory  (Kolb,  1976)  where  three  students  were 
selected  extreme  cases  in  each  of  the  four  learning  styles  identified  by  Kolb  (converger,  accomodalor , di  verger,  and  assimilalor).  The  participants  were  expected  to  create  three  electronic  slides  using  Astound, 
a multimedia  presentation  software,  during  the  learning  process.  They  were  told  that  there  was  no  lime  constraint  and  that  they  could  freely  use  the  printed  jumpslart  inslmclions,  the  video  demonstration 
parallel  to  the  printed  inslmclions,  and  the  user  manual  and  tutorial.  They  then  created  a fourth  slide  without  instmetion  as  the  post  lest.  Learning  sessions  were  recorded  on  video,  and  participants  subsequently 
watched  their  own  videos  and  were  interviewed  using  think-aloud  methodology.  Multiple  observers  watched  the  videotapes  of  the  learning  performances  for  patterns  of  interaction,  which  were  illuminated  by 
the  think-aloud  interviews. 

In  this  study,  the  interview  was  semistmclured.  The  interviews  followed  the  observations,  hence  the  interviews  supplement  and  complement  the  outcomes  of  the  observations,  rather  than 
fonning  a dilTerent  set  of  data  not  related  to  that  from  the  observations.  The  questions  asked  were  open-ended  so  the  students  could  explain  fully  and  interpret  any  specific  events  that  occurred  in  their  learning 
performance. 

As  soon  as  the  raw  data  from  the  interviews  were  collected,  the  data  gathered  from  both  the  observation  and  the  interview  on  each  student  were  arranged  into  one  case  for  analysis.  As  Merriam 
(1988)  suggests,  the  “simultaneous  analysis  and  data  collection  allows  the  researcherlo . . . develop  a data  base  that  is  both  relevant  and  parsimonious.” 

Next,  the  data  collected  from  both  the  observation  and  the  interview  were  sorted  into  any  repeated  patterns  that  are  common  to  more  than  one  person  or  identifiable  categories  or  themes.  Some 
patterns  and  their  variations  were  spotted  faster  than  others.  Some  unexplainable  events  or  sequences  from  the  observation  were  explained  when  data  from  the  interview  were  reviewed  and  compared.  This  step 
was  repented  until  all  12  cases  were  analyzed.  Then,  an  analysis  to  reduce  and  refine  the  categories  or  patterns  among  the  cases  was  conducted. 

Result 


The  narrative  temporal  descriptions  of  each  of  the  1 2 cases  were  compiled.  The  descriptions  included  answers  to  the  following  questions;  Who  was  the  person  (uge,  gender,  race,  academic  major, 
Kolb’s  learning  style,  mastery  of  the  post  test,  prior  computer  experience).  What  did  s/he  do  with  the  instructional  materials  during  different  parts  of  the  learning  session?  What  comments  (direct  quotes  from 
the  think-aloud  videos)  did  the  person  make  about  what  s/he  was  doing?  These  thick  descriptions  (Lincoln  and  Guba,  1985)  illustrated  how  the  participants  actually  interacted  with  the  instmctional  materials 
during  the  learning  sessions.  These  thick  descriptions  (Lincoln  and  Guba,  1985)  illustrated  how  the  participants  actually  interacted  with  the  instmctional  materials  during  the  learning  sessions.  To  demonstrate 
the  uniqueness  of  these  participants,  the  following  is  a complete  set  of  descriptions  recorded  from  one  of  the  1 2 students. 

Student  F 


Student  F was  a twenty-two-year  old  white  male.  He  majored  in  Secondary  Education  and  had  had  four  years  of  computer  experience.  He  was  a converger  (with  a tendency  toward  active 
experimentation  and  abstract  conceptualization),  according  to  Kolb’s  learning  style.  He  was  judged  a master  of  the  post  test.  He  spent  a total  of  42.5  minutes  in  completing  the  learning  tasks  and  the  post  test. 

Student  F spent  nine  minutes  on  Task  Ml.  He  began  his  learning  session  by  playing  the  video  demonstration  while  reading  the  instmetions.  Student  F is  rather  reserved  but  well-mannered.  Of 
his  choices  of  instmctional  materials,  he  said,  “I  didn’t  mind  it.  1 like  to  have  something  to  follow  along  with  [the  printed  instmetions],  and  1 can  stop  [the  video]  whenever  I want,  so  that  makes  it  a lot  easier 
for  me.”  When  asked  how  he  used  the  instmctional  materials,  he  answered,  “I  started  by  reading  the  [printed]  instmetions  and  watching  the  videotape  step-by-step.  But  after  a while,  I was  not  even  watching  it, 
‘cause  I know  pretty  much  what  to  do  by  listening  to  it.”  He  said  of  his  reading  strategy,  “After  a while,  I was  just  reading  ahead,  and  make  sure  that  I wa 
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In  a careful  review  of  all  ihe  think-aloud  interviews  in  which  the  learners  answered  open-ended  questions  to  explain  their  learning  performances  while  watching  the  video  recording  of  their 
learning  Astound,  five  learning  patterns  emerged:  (I ) following  through  error  correction,  (2)  confirming  actions,  (3)  hand  on  mouse,  (4)  keeping  up  step-by-step  with  the  video  instruction,  and  (5)  trying  new 
steps.  Using  analysis  of  patterns  in  time  (APT),  a methodology  that  can  code  event  changes  over  time,  the  frequencies  and  the  amounts  of  time  of  the  five  learning  patterns  were  calculated.  The  data  derived 
from  the  APT  scores  indicated  two  major  findings:  (I)  The  amount  of  time  used  by  the  participants  ranged  f^m  20  to  87  minutes.  The  amount  of  time  spent  did  not  predict  mastery  of  the  post  test.  (2) 
‘Following  through  error  corrections’,  ‘confirming  actions’,  and  ‘trying  new  steps’  were  the  patterns  that  did  appear  to  predict  mastery,  and  the  other  two  patterns  were  indicators  of  learners’  task  engagement. 
The  findings  suggest  that  it  is  important  to  design  flexible  instruction  to  facilitate  repeated,  persistent,  and  successful  practice  in  order  to  achieve  mastery  in  a self-paced,  computer-based  learning  environment, 
regardless  of  the  amount  of  time  spent. 

Suggestions  to  Practitioners 

First  and  foremost,  the  implications  these  dynamic  learning  patterns  brought  to  the  instructional  designers  and  teachers  are: 

(1)  Ideally,  there  should  not  be  any  time  constraints  in  a self-paced  computer-based  learning  environment.  However,  in  classroom  settings  and  even  in  corporate  training  settings,  there  are  lime 

limitations.  As  teachers  and  instructional  designers,  we  should  try  to  mitigate  the  way  the  timed  classroom  hour  shapes  learning.  For  example,  in  a traditional  classroom  hour.  Student  H,  a master  who  took  87 
minutes  to  finish  the  task,  would  have  failed  to  master  the  task  if  she  had  had  to  stop  after  50  minutes.  We  need  to  develop  new  ways  of  structuring  the  time  of  the  class  so  that  such  failures  would  not  be  built 
into  the  classroom  framework.  Nevertheless,  the  immediate  value  of  self-paced  learning  will  be  outside  of  the  classroom  where  the  learners  or  trainees  can  determine  or  have  control  over  the  time  they  use  to 
leam  the  tasks. 

(2)  Those  learners  who  have  difficulty  with  following  through  error  correction  may  have  problems  with  a lack  of  lower-level  computer  skills.  The  present  writer  recommends  that  under  such 
circumstances  that  the  instructional  designer  needs  to  develop  a mechanism  to  detect  the  problem  quickly  and  refer  the  learner  to  simple  optional  lessons  on  the  conventions  of  the  operating  system  (e.g., 
Windows  95  in  this  study)  and  the  basic  features  of  the  application  software  program  (e.g.,  the  tools  featured  in  Astound  in  this  study). 

(3)  If  those  learners  who  choose  to  use  exclusively  the  text-only  printed  instructions  and  have  difficulty  finding  the  features,  tools,  push  buttons,  or  have  difficulty  finding  “where  they  are’’  from 

just  a verbal  description,  then  if  space  and  format  allow,  the  related  graphical  icons  of  the  tools  and  the  appropriate  screen  captures  should  be  added  to  the  printed  instructions. 

(4)  Making  the  printed  (verbal)  and  the  video  versions  of  the  instructions  as  well  as  the  computer  with  the  software  program  available  as  options  at  the  sumd  location  can  accommodate  most 
learners’  preference  in  terms  of  forms  of  content  and  sensory  modalities. 

(5)  For  those  learners  who  like  to  try  new  steps,  the  learning  lessons  should  be  ftexible.  However,  for  those  who  do  not  like  to  explore  new  things,  more  structured  jumpstart  lessons  at  the  same 
or  at  difterent  levels  of  complexity  should  be  provided. 

(6)  Many  participants  in  this  study  strongly  suggested  that  more  "context  sensitive’’  hints  or  notes  should  be  made  available  on  both  the  printed  and  video  instractions  because  the  just-in-time 
hints  or  notes  given  immediately  before  or  after  the  “problematic  spots”  helped  some  learners  recover  from  errors  quickly  without  getting  frustrated.  Therefore,  it  is  important  to  observe  where  in  the  lesson 
learners  tend  to  have  problems  with  something,  then  design  and  position  the  “anticipatory  fee 

Discussion 


In  this  study,  there  were  five  leamcr-demonstraled  patterns  identified  in  a self-paced  environment  in  learning  to  take  part  in  a computer  software  program.  Although  generalizations  cannot  be 
made  for  all  people  in  all  kinds  of  learning  because  of  the  relatively  small  and  selected  sample,  the  trends  found  in  the  data  do  suggest  that  a number  of  these  learning  patterns  would  be  predictive  of  mastery  in 
procedural  learning.  Future  studies  with  a larger  randomly-selected  sample  size  will  further  test  the  character  of  these  trends. 

This  present  study  used  Kolb's  Learning  Style  Inventory  (LSI)  as  the  screening  tool  to  select  samples  because  the  LSI  is  a popular  inslrume 
weaknesses  as  a learner”  (Kolb,  1976)  and  used  widely  for  career  predictions.  Comparing  the  activities  from  the  emerging  learning  patterns  identified  in  this  study  with  Kolb’s  four  learning  abilities,  we  can 
draw  some  parallel  relations  between  them:  reading  printed  instructions  is  an  example  of  abstract  conceptualization  (AC);  watching  a video  demonstration,  an  example  of  reflective  observation  (RO);  doing  the 
steps  as  described  in  the  instructions,  an  example  of  concrete  experience  (CE);  and  trying  new  steps,  an  example  of  active  experimeniatiun  (AE). 

Kolb’s  learning  styles  are  generated  from  two  combination  scores  (AC  - CE  and  AE  - RO)  that  indicate  the  extent  to  which  an  individual  emphasizes  abstractness  over  concreteness  and  action 
over  reflection.  However,  the  similarities  between  Kolb’s  learning  styles  and  the  identified  learning  patterns  end  here.  One  fundamental  difterence  between  these  two  is  that  Kolb  assumes  the  learning  styles 
arc  stable  personality  traits,  whereas  the  learning  patterns  identified  in  this  study  are  dynamic  in  nature.  For  example,  while  student  F was  designated  as  a converger  with  tendency  to  be  abstract  and  active, 
according  to  Kolb,  he  actually  demonstrated  in  his  learning  performance  that  he  engaged  in  all  four  learning  abilities  over  lime.  Suidart  was  watching  the  video  demonstration  (reflection)  and  following  each 
step  as  it  was  described  on  the  video  (concreteness).  He  also  frequently  checked  with  the  printed  instructions  to  see  if  he  did  the  steps  right  (abstractness).  After  getting  more  familiar  with  the  computer 
program,  he  started  to  try  new  steps  that  were  not  included  in  the  instructions  (action).  Over  all.  Student  F showed  a dynamic  learning  sequence  that  involves  several  learning  patterns  that  even  he  himself  could 
not  have  predicted  or  repealed. 

In  the  present  study,  the  major  finding  is  that  regardless  of  the  amount  of  time  spent,  repeated,  persistent,  and  successful  practice  lends  to  lead  to  the  mastery  of  the  post  test.  This  finding  echoes 
the  result  found  by  Sze  (1988)  that  repeated  masteries  during  practice  strongly  affect  student  learning  achievement  and  altitude  in  an  adaptive  computer-guided  practice  environment.  Also,  it  concurs  with  the 
Academic  Learning  Time  (ALT)  research  reported  by  Berliner  (1979)  that  when  students  spend  more  lime  engaging  in  a particular  curriculum  content  area,  they  are  more  likely  to  attain  higher  academic 
achievement.  It  is  important  to  note  that  the  above-mentioned  findings  strongly  support  Carroll’s  (1989)  argument  that  aptitude  is  primarily  a measure  of  time  required  to  leam.  This  argument  is  in  contrast  with 
the  classic  models  (based  upon  theories  of  intelligence)  in  which  all  learners  are  given  the  same  amount  of  time  to  leam,  and  the  focus  is  on  differences  in  ability. 
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Does  the  labeling  of  links  assist  user  orientation? 


Rolf  Zajonc  & Lutz  Schon 
Humboldt-University 
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10115  Berlin,  Germany 
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Abstract:  Hypermedia  learning  material  often  makes  broad  use  of  links  to  infonnation  which  goes 
beyond  the  learning  objectives.  But  instead  of  being  additionally  motivated  some  students  rather 
feel  too  much  asked  of.  Within  the  Lilienthal  project  twww.pilotschool.neO  links  that  don’t  follow 
the  linear  navigation  are  labeled  with  respect  to  their  relevance  for  the  learning  objectives.  First 
evaluations  prove  that  the  students  welcome  this  assistance. 


Introduction 

“Information  overflow“  or  “lost  in  hyperspace”  are  keywords,  which  also  are  frequently  used  to  describe  the 
difficulties  of  students  using  hypermedia  for  learning  purposes.  Indeed  they  give  a good  impression  of  what  may 
hinder  students  from  enjoying  the  new  technical  possibilities  offered  by  the  internet  in  contrast  to  their  teachers 
expectations.  When  creating  webpages  instructors  believe  they  can  motivate  their  students  by  offering  many  links  to 
further  interesting  or  related  aspects  on  the  specific  subject  matter.  But  e.g.  many  lecturers  report,  that  rather  the 
opposite  effect  is  shown  by  their  students:  learners  tend  to  complain  about  the  number  of  webpages  they  are 
supposed  to  visit  (Dunlap  1999). 

The  problem  faced  is,  that  while  the  lecturer  wants  to  offer  the  students  the  best  chances  to  learn  the  subject  matter 
as  well  as  possible,  the  student  also  has  in  mind,  that  he  needs  to  achieve  certain  learning  objectives  within  a certain 
amount  of  time,  e.g.  to  be  prepared  on  time  for  an  examination.  Consequently  students  feel  overwhelmed  and 
frustrated  by  the  amount  of  information,  because  they  cannot  differentiate  between  more  and  less  important  links  as 
their  teachers  can.  One  possible  solution  to  this  problem  might  be  the  labeling  of  the  links  according  to  their 
relevance  for  the  required  learning  objectives,  e.g.  by  the  use  of  small  icons  in  front  of  links. 

Labeling  of  links  according  to  relevance 

This  directly  leads  to  the  first  question:  How  to  decide  on  the  relevance  of  a link?  The  answer  can  be  easy,  if  the 
creator  of  the  webpages  and  the  corresponding  examination  responsible  are  the  same  person.  But  in  many  cases  the 
learning  material  is  programmed  by  a team  whose  members  can  have  different  opinions  on  this  question.  As  this 
also  affbcts  the  pages’  content  in  general,  it  is  necessary,  that  the  complete  team  agrees  on  a set  of  well  formulated 
(e.g.  Mager  1978)  detailed  learning  objectives  before  the  actual  programming  takes  place.  Afterwards  the  links  can 
be  labeled  with  respect  to  these  detailed  learning  objectives. 

In  order  not  to  confuse  the  students  within  Lilienthal  it  has  been  decided  to  restrict  the  labeling  to  three  icons: 
-mandatory^  e.g.  a link  leading  to  a formula  which  the  student  must  know  by  heart; 

-recommended,  e.g.  a link  leading  to  an  interesting  example  which  illustrates  the  effects  of  the  formula  in  a certain 
context;  or 

-nice  to  know,  e.g.  a link  leading  to  an  abstract  about  how  this  formula  had  been  discovered  historically. 

Of  course  it  still  remains  the  student’s  decision  whether  he  follows  a link  and  for  how  long  he  stays  on  the 
corresponding  pages. 

Research 

Hypermedia  material  for  roughly  80  hours  of  learning  time  has  been  distributed  to  pilot  students  of  four  European 
flightschools.  The  material  had  been  programmed  after  a Delphi-process  (Linstone  & Turoff  1978)  which  lasted  one 
year,  during  which  several  subject  matter  experts  of  all  involved  flightschools  had  discussed  and  finally  agreed  on  a 
set  of  detailed  learning  objectives,  each  describing  5 minutes  of  learning  time.  On  this  basis,  the  links  within  the 
material  have  been  labeled  as  described  above.  The  number  of  links  vary  per  module  (9  DLO  in  average). 
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At  this  point  in  time  (March  2000)  evaluation  of  the  various  data  (tests,  questionnaires,  protoquols)  has  just  started. 
A first  evaluation  of  208  questionnaires  on  1 1 different  modules  (average  45  minutes)  not  only  proved  that  the  big 
majority  of  students  found  the  labeling  of  links  useful  (compare  figure  1),  but  also  that  there  is  a strong  correlation 
(p  <0.01)  to  the  statement  “the  module  is  easy  to  operate”.  A reasonable  interpretation  of  this  correlation  is  that 
those  students  who  made  use  of  the  additional  info  of  the  labeling  icons,  also  felt  they  had  a better  control  in  general. 


The  labeling  of  links  (m,  r,  n)  is  useful! 
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Figure  1 : Evaluation  of  208  student  questionnaires  on  1 1 different  modules  for  the  statement  The  labeling  of  links  is  useful! 

Mevertheless  further  investigations  are  required,  e.g.  because  in  the  pre-evaluation  (in  total  19  students)  14  out  of  all 
17  students  who  marked  “(strongly)  agree”  almost  immediately  added  “...but  still  you  click  on  every  link  anyway”. 
This  shows  a well  known  behavior  of  hypermedia  users,  who  only  have  little  navigation  experience.  The  assumption 
is,  that  the  labeling  will  have  a stronger  influence  on  the  ‘click  or  no  click’-decision  in  the  long  run. 

In  this  context,  although  in  our  case  the  pilot  students  certainly  form  a rather  homogenous  group  (typically  male,  age 
19-23,  open-minded  for  technically  related  matters,  good  marks  in  physics  and  mathematics),  it  is  still  expected,  that 
students  are  influenced  differently  by  this  labeling.  E.g. 

A students  still  click  on  every  link, 

B students  select  recommended  and  in  particular  nice-to-know  links  according  to  the  indicated  topics, 

C students  avoid  most  recommended  and  all  nice-to-know  links. 

In  addition  it  might  well  be  possible,  that  a student  changes  his  attitude  in  the  courses  process.  As  has  been 
mentioned  before,  in  particular  novices  might  rather  show  A behavior,  but  after  a while  they  are  expected  to  realize, 
that  they  need  to  work  more  efficiently  and  become  B or  C students. 

But  the  labeling  should  not  only  influence  the  decision  whether  to  click  but  also  the  navigation  on  the  following 
pages.  A student  should  be  more  likely  to  really  explore  a webpage  if  he  knows  its  content  belongs  to  the  mandatory 
level.  In  contrast  he  might  show  scanning  navigation  (Canter,  Rivers  et  al,  1985)  if  he  knows  that  he  needn’t  follow 
the  link  before  at  all.  Since  so  far  almost  no  experiences  are  reported  in  this  context,  these  are  Just  reasonable 
assumptions,  which  need  to  be  investigated. 
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When  is  an  Interactive  Learning  Scenario  a Matter  of  Interface  Design? 


Christian  M.  Adriano,  Adriane  M.  Adriano,  Ivan  L.  M.  Ricarte 
State  University  of  Campinas,  Brazil 

Interaction  is  an  important  feature  a learning  activity  must  promote.  Parker  (1999)  outlines  benefits  pro- 
vided by  interaction:  “Understanding  and  long-term  internalization  can  only  be  acquired  by  communication,  recon- 
struction and  reconciliation  of  information;  interaction  enables  socialization,  promotes  motivation,  and  learning 
commitment;  interactive  processes  foster  comprehension  and  consensus  on  universal  truths”. 

In  previous  research  (Adriano  et  al.I999),  we  implemented  a computer  supported  learning  scenario  by 
producing  content,  modeling  evaluation,  and  choosing  a proper  pedagogical  orientation.  Our  effort  resulted  in  the 
first  version  of  the  CALM  platform,  which  implements  a learning  scenario  inspired  by  the  Goal  Based  Scenario 
approach,  provides  an  intelligent  tutor,  and  hosts  material  for  learning  Java.  In  order  to  guide  further  effort  on  in- 
teraction, the  following  problem  was  posed:  When  is  an  interactive  learning  scenario  a matter  of  interface  design? 
We  propose  two  situations  in  which  developing  interactive  learning  scenarios  incurs  in  interface  design.  First, 
when  concern  fits  on  one  of  the  items  proposed  by  Marcus  (1994):  Mental  and  Navigation  Models,  Appearance, 
Interaction,  or  Metaphor.  Second,  when  the  learning  scenario,  implemented  by  the  resulting  interface  design,  must 
accomplish  complementary  definitions  for  the  concept  of  "interaction". 

Metaphors  are  fundamental  concepts  communicated  through  words,  images,  and  sound.  Mental  models 
are  data  structures,  functions,  tasks,  roles,  jobs,  and  people  in  organizations  of  work.  Navigation  is  the  process  of 
moving  through  the  mental  models.  Interaction  is  all  input-output  sequences  and  means  for  conveying  feedback. 
Appearance  relates  to  visual,  verbal  and  acoustic  information. 

The  designed  metaphors  (Adriano  et  al.  2000)  were  annotation,  document,  and  annotation-place.  The  de- 
sign of  the  interaction  item  resulted  in  three  annotating  paradigms  represented  by  specific  annotation-places: 
Bridge,  Inline,  and  Tree.  The  Bridge  place  represents  the  idea  of  annotating  as  adding  links.  The  Inline  place  sup- 
ports the  paradigm  of  writing  over  the  text  and  having  the  annotation  content  visible.  The  Tree  place  organizes 
annotations  in  a newsgroup-like  fashion.  The  mental  model  comprises  the  annotation-place  model  and  organiza- 
tion of  roles.  In  the  former,  a document  may  contain  many  annotation-places  and  an  annotation-place  may  contain 
many  annotations.  The  organization  of  roles  depends  on  each  learning  scenario  specification.  The  navigation 
model  comprises  resources  to  manipulate  metaphors.  Such  resources  correspond  to:  annotation  editing  interfaces,  a 
search  tool,  a document  composition  tool  (annotation-places  are  implemented  as  applets  within  Web  pages),  a tool 
for  repositioning  annotation-places  due  to  changes  in  the  original  Web  page,  and  a tool  to  generate  workbook-like 
compilation  of  annotations.  The  appearance  item  relates  to  types  of  annotations  and  annotation-places.  The  anno- 
tation metaphor  has  four  visually  different  types:  doubt,  suggestion,  commentary  and  mark. 

The  discussion  scenario  comprises  theme  presentation,  individual  annotation,  group  annotation,  and  dis- 
cussion summarization.  Roles  are  “moderator”  and  “commenter”.  The  theme  presentation  is  a notification  that  the 
Web  page  hosting  the  text  for  discussion  is  available  for  commenting.  After  notification,  starts  the  individual  an- 
notation phase  in  which  commenters  read  and  make  personal  annotations  on  the  text.  In  the  next  phase  com- 
menters  may  request  the  annotations  of  their  peers  and  discuss  opinions  and  acknowledgements,  which  involves 
commenting  on  each  other  annotations.  When  discussion  is  finished,  due  to  a deadline  or  a common  agreement, 
the  moderator  compiles  annotations  into  a second  text  elaborating  a summary  of  the  discussion. 

The  co-authoring  scenario  includes  goal  definition,  authoring,  reviewing,  voting,  and  consolidation.  Roles 
are  “co-author”,  “reviewer”,  and  “author”.  Co-authors  annotate  using  inline  annotation-places  and  may  define 
when  an  annotation  is  ready  to  be  seen  by  others.  Reviewers  may  comment  on  annotations  made  by  co-authors.  As 
a co-author  decides  that  an  annotation  should  become  part  of  the  underlying  text,  he/she  requires  the  author  to  con- 
solidate the  annotation.  If  the  author  agrees,  the  annotation  is  incorporated  to  the  text.  The  author  can  also  ask  for 
co-authors  to  vote  whether  an  annotation  should  be  consolidated.  An  annotation  has  states  such  as  ready,  removed, 
in  consolidation,  being  voted,  and  not  available. 

Some  interaction  definitions  were  collected  from  Vigotsky,  Habermas,  de  Certeau,  and  C harder.  Vigotsky 
(1978)  deals  with  the  cultural  dimension  of  learning.  Apprenticeship  is  result  of  interactions  among  historical 
subjects  and  the  world.  Signs  and  symbols  are  the  condition  for  these  interactions.  People,  instruments  and  signs 
mediate  relations  between  a subject  and  physical/social  environment.  Signs  correspond  to  a representation  of  real- 
ity and  refer  to  remote  times  and  spaces.  Interaction  is  conceptualized  as  relations  among  historical  individuals. 
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signs,  and  instruments.  Such  definition  is  satisfied  by  interfaces  that  relate  individuals  as  role  players,  signs  as 
metaphor  representations  and  instruments  as  tools  of  the  navigation  model. 

Certeau  (1974)  states  that  interaction  is  based  on  a metaphor  of  landmark  and  on  distinctions  between 
place  and  space.  In  short,  place  is  static  and  permanent,  while  space  is  dynamic  and  transient.  Landmarks  delimit 
regions  (spaces  or  places),  being  also  classified  as  dynamic  or  static.  Static  landmarks  are  frontiers,  like  rivers  and 
fences,  delimitating  places.  Dynamic  landmarks  are  bridges,  like  doors  and  windows,  delimitating  spaces.  Bridge- 
like landmarks  represent  encounters  of  action.  Moreover,  there  will  be  so  many  bridges  as  the  number  of  “inter- 
actions”. Interaction  is  defined  as  a bridge  separating  spaces  of  action.  Such  concern  inspired  the  design  of  anno- 
tation-places, whose  types  provide  different  bridging  between  the  document  space  and  the  annotation  space. 

As  quoted  by  Prestes  (1997),  Habermas  argues  that  the  communicative  action  has  a kind  of  anchor  con- 
stituting background  knowledge.  Such  knowledge  fosters  interpretations  for  participants  of  an  interaction.  The 
background  knowledge  relates  to  a world  of  life  {Lebenswelt\  the  horizon  on  which  we  move.  It  is  pre-scientific, 
intuitive,  not  expressed  and  not  questionable.  It  constitutes  itself  not  only  by  cultural  convictions,  but  also  by  insti- 
tutional organizations  and  structures  of  personality.  Habermas  uses  the  daily  communicative  praxis  to  guarantee 
the  symbolic  reproduction  of  three  spheres;  culture,  society,  and  personality.  Culture  refers  to  knowledge  acervus, 
over  which  we  formulate  interpretations  to  understand  the  world.  Society  refers  to  legitimated  organizations  that 
regulate  our  bindings  with  social  groups.  Personality  refers  to  competencies  that  enable  subjects,  capable  of  lan- 
guage and  action,  to  take  part  in  an  understanding  process.  The  following  citation  of  Habermas  clarifies  these  three 
aspects:  **ln  respect  to  the  functional  aspect  of  understanding,  the  communicative  action  promotes  tradition  and 
cultural  renovation.  In  respect  to  coordination  aspects,  the  communicative  action  promotes  social  integration  and 
creation  of  solidarity.  In  respect  to  socialization,  the  communicative  action  promotes  formation  of  individual  iden- 
tities". Interaction  is  a space  of  conflict  between  a pre-comprehension  of  world  of  life  and  the  communicative  ac- 
tion. Therefore,  when  and  where  interaction  happens,  the  stability  of  the  world  of  life  is  disturbed  by  arguments  of 
the  communicative  action.  Annotation  interfaces  enabling  discussion  was  a mean  to  accomplish  such  vision. 

Following  a historical  point-of-view,  Chartier  (1997)  provides  a study  of  how  mankind  has  been  interact- 
ing with  and  by  means  of  the  written  letter.  Medium  has  changed  from  papyrus,  to  codex,  to  Gutenberg  press,  and, 
nowadays,  to  digital  text.  Each  medium  required  a proper  interaction  paradigm  suggesting  that  interaction  be  also 
defined  by  the  enabling  technological  support.  In  other  words,  medium  defines  interaction.  Such  concept  for  inter- 
action was  accomplished  by  supporting  audio  and  written  media  for  annotation. 

In  addition  to  the  five  components  proposed  in  Marcus  (1994),  we  suggest  a careful  study  of  the  interac- 
tion concept.  The  presented  discussion  stems  from  project  meetings  between  educators  and  software  developers.  A 
recurring  problem  faced  was  the  lack  of  consensus  on  concepts,  accompanied  by  concerns  of  presupposing  things 
that  needed  better  understanding  before  implementation.  Interaction  conceptualization  is  one  these  concerns.  We 
hope  the  ongoing  research  will  collect  further  insights  to  the  design  of  interactive  learning  scenarios  as  a coopera- 
tive entrepreneurship  of  educational  and  computer  scientists. 
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An  Interactive  System  Model  of  an  Online  System  for  Teaching  & 
Learning  Developed  as  a Scoping  / Costing  Instrument 
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Although  there  are  strong  reasons  for  purchase  of  an  off-the-shelf  system  for  online 
course  delivery  (including  the  competitive  advantages  of  reliability,  the  comfort  of  a 
large  client  base  and  scalability),  many  universities  develop  a system  themselves.  These 
home-grown  systems  are  often  more  responsive  to  specific  needs.  When  James  Cook 
University  undertook  to  investigate  available  systems  for  university  wide  online  course 
delivery  it  was  important  to  explore  costs  of  building  a system  using  university  resources. 
To  this  end  a system  model  was  designed  as  an  interactive  diagram  reflecting  each  part  of 
the  system  in  prototype  http://www.tsd.icu.edu.au/icudevelop/icuproto/icuonline 
The  model  is  both  an  innovative  example  of  organisational  design  method  and  a useful 
means  of  testing  user  pathways  through  the  system,  (use-case),  which  can  assist  costing 
and  project  management  of  the  development  of  such  a system. 


As  part  of  the  search  for  the  most  suitable  system  for  use  at  JCU  we  set  out  to  determine  the  costs  of 
building  a system  using  university  resources: 

• there  was  a high  level  of  interest  in  the  JCU  community; 

• the  possibility  of  economies  of  production; 

• a collaborative  venture  of  this  kind  with  the  potential  for  cross-institution  partnerships  could  be  of 
value  to  the  university  in  the  future. 

To  this  end  a system  model  was  designed  as  an  interactive  diagram  reflecting  each  part  of  the  system,  in 
prototype  (at  http://www.tsd.icu.edu.au/icudevelop/icuproto/icuonline). 

The  model  is  both  an  innovative  example  of  organisational  design  method  and  a useful  means  of  testing 
user  pathways  through  the  system,  (use-case),  which  can  assist  costing  and  project  management  of  the 
development  of  such  a system. 

Discussions  were  held  with  the  schools  of  IT  and  Computing  Science.  The  interactive  system  model  was 
felt  to  be  helpful  in  this  process  in  that  it  provided,  by  visible  and  testable  means,  a common  understanding 
of  the  scope  and  direction  of  the  exercise.  Costing  was  based  in  staged  development  from  base  level, 
adding  functionality  as  we  saw  a need. 

There  was  found  to  be  no  case  for  developing  DIY  for  immediate  implementation: 

« the  real  costs  of  DIY  infrastructure  are  high  for  a one-off  development  which  is  not  amortised  over 
other  products; 

• time  of  development  has  to  be  compressed  to  keep  pace  with  user  needs  and  technical  innovation:  to 
compress  time  requires  commitment  of  personnel  which  increases  costs; 

• when  a university  implements  its  own  system,  support  for  users  becomes  an  internal  budget  decision 
not  externally  (market)  driven:  support  is  therefore  subject  to  internal  budgetary  pressures  rather  than 
the  wider  driving  client  base. 
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Abstract:  This  paper  provides  a theoretical  framework  for  the  design  of  a successful 
web-based  course  in  Personal  Finance.  Using  an  open  learning  model  based  on 
constructivist  principles  that  draw  from  the  literature  in  cognition  and  the  social 
sciences,  this  paper  sketches  an  instructional  design  model  and  specifies  online  tools 
that  mediate  between  course  objectives  and  desired  learning  outcomes.  In  order  to 
validate  the  model,  data  generated  from  tool  use  and  from  student  performance  will  be 
analyzed  using  confirmatory  factor  analysis. 

Mixed  Messages  with  Multimedia 

A growing  body  of  research  is  exploring  the  learning  effects  of  multimedia  designed  environments.  Some 
of  this  research  has  attempted  to  incorporate  educational  concepts  in  the  design,  process  and 
implementation  of  course  production  in  technology-rich  platforms.  However,  the  results  are  mixed. 
According  to  The  No  Significant  Difference  Phenomenon  (Russell,  1999)  which  is  based  on  355  research 
reports,  summaries  and  papers,  the  conclusions  mainly  indicate  that  the  learning  outcomes  of  students 
using  technology  enriched  courseware  are  similar  to  those  of  students  who  participate  in  conventional 
classroom  instruction.  Yet,  universities  are  allocating  substantial  resources  towards  the  development  of 
distance  education  courses  and  programs  (Brahler,  Peterson  et  al.,  1999)  to  increase  access  to  learning, 
with  the  hope  that  the  quality  of  education  will  be  damaged  little,  if  at  all  (Ehrmann,  1999)  Questions  and 
concerns  relating  to  technology  integration  are  being  raised,  but  as  yet  have  had  little  systematic  impact 
on  the  policies  that  guide  educators,  administrators  and  practitioners  (Merisotis  and  Phipps,  1999;  Welte, 
1997)).  Faculty  realize  that  there  are  certain  benefits  of  technology-based  distributed  learning 
environments,  but  put  up  considerable  resistance  to  the  development  of  such  environments  (Rickard, 
1999). 

Mixing  Cognitive  and  Soda)  Science  Approaciies 

This  working  paper  presents  a theory-based  design  of  an  undergraduate  web-based  course  in  Personal 
Finance  with  the  aim  of  addressing  some  of  the  issues  addressed  above  in  the  context  of  better 
understanding  student  learning.  Concerns  relating  to  technology  integration  in  educational  practice,  and 
technology’s  intersection  with  learning  theory  are  addressed  in  a design  framework  that  aligns  learning 
objectives,  course  activities  and  assessment  methods.  This  alignment  simultaneously  considers  the 
selection  of  online  learning  tools  that  mediate  between  design  components  for  Personal  Finance.  The 
selection  of  learning  tools  also  plays  an  important  role  in  terms  of  how  they  will  be  meaningfully  used  by 
students,  and  how  they  will  affect  student  performance. 

Web-based  instruction  that  is  designed  on  accepted  theoretical  principles  in  cognitive  psychology  opens 
new  doors  to  designers,  learners  and  practitioners.  This  is  simply  because  we  know  very  little  about  best 
practices  arising  from  the  design  of  courseware  that  primarily  uses  this  medium.  However,  in  contrast, 
there  is  an  abundance  of  theoretical  studies  in  the  cognition  literature  that  have  increased  our 
understanding  of  competence  in  understanding  a given  discipline  (See  for  example,  Anderson,  Corbett  et 
al.,  1995;  Ausubel,  1968;  Hatano,  1996;  Means,  1993;  VanLehn,  1996;  and  Winn,  1990).  Similarly, 
studies  from  the  literature  in  social  sciences  have  proposed  interactive  socio-cultural  contexts  as  a basis  for 
understanding  how  learning  can  be  constructed,  scaffolded  and  facilitated  (See  for  example,  Bandura, 
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1986;  Brown,  Collins  et  al.,  1989;  Derry  and  Lesgold,  1996;  Lave  and  Wenger,  1991;  Moore,  1994; 
Shuell,  1993;  and  Richey,  1994.  In  his  1999  AERA  Presidential  Address,  Alan  Schoenfeld  describes  the 
split  between  cognitive  and  social  science  research  with  references  to  studies  in  cognition  that  excel  in 
fine-grained  data  analysis  but  suffer  from  tunnel  vision  and  points  to  as  many  sociological  studies  that  are 
rich  in  scope  but  are  weak  with  respect  to  validity  and  reliability  constructs.  His  suggestion  to  bridge  the 
gap  was  to  “build  something  to  see  if  it  works  and  then  study  the  hell  out  of  it”. 

This  study  seeks  to  validate  a non-traditional  design  based  on  theoretical  findings  from  the  cognitive  and 
social  sciences  literature.  The  world-wide-web  is  an  obvious  choice  as  the  platform  for  building  a design 
prototype  because  learners  recognize  that  it  is  more  accessible  and  convenient  than  traditional 
opportunities  afforded  in  the  classroom.  Combining  the  web  platform  and  a theoretical  basis  into  a 
prototype  design  becomes  meaningful  when  specific  learning  goals  are  specified  with  evidence  to  link 
these  goals  with  performance  outcomes  that  are  assessed.  These  links  are  critically  explored  and  are  the 
focus  of  this  study. 

Challenges  in  the  Design  Process 

The  term  technology  is  used  broadly  to  refer  not  only  to  tools  but  also  processes  and  is  closely  identified 
with  the  Greek  form  techne  (art,  craft,  or  skill),  which  was  closely  associated  with  episteme  (systematic  or 
scientific  knowledge).  Its  educational  meaning  is  historically  derived,  bound  by  specific  psychological 
theories  and  philosophical  underpinnings  (Saettler,  1990).  Visions  of  technology  look  very  different 
depending  on  the  explicit  theories  of  learning  that  guide  their  design  and  implementation.  According  to 
The  Cognition  and  Technology  Group  at  Vanderbilt  (1996),  the  challenge  for  instructional  design  based 
on  significant  technology  integration  is  that  learning  theory,  educational  practice  and  technology  interact 
with  each  other.  To  understand  the  role  of  technology,  one  must  examine  a simultaneous  intersection  with 
student  learning  and  educational  practice.  This  challenge  is  increased  since  all  three  areas  have  seen 
dramatic  change  in  the  past  ten  years.  Thus,  changes  in  theories  of  learning  affect  uses  of  technology,  but 
new  technologies  such  as  the  world  wide  web  also  make  new  kinds  of  interactions  possible  and  hence 
affect  theories  of  learning  (Kozma,  1994;  Saloman,  1993).  Knowledge  acquisition  in  constructivist 
environments  engages  learners  in  drawing  meaning  from  multiple  perspectives  and  through  collaborative 
efforts.  Engagement  in  real  world  or  authentic  tasks  provides  a context  for  the  learner  to  construct 
meaning  from  his/her  experiences.  Thus,  learning  is  situated  in  a community  of  learners  who  construct 
personal  meaning  through  dialogue  and  socialization  (See  Brown,  Collins  et  al.,  1989;  Jonassen,  1998;  & 
Wilson,  1995). 

Sketching  the  Design  Components 

Given  these  perspectives,  different  views  of  how  humans  learn  and  how  designers  approach  instruction 
are  incorporated  under  the  umbrella  of  knowledge  acquisition  and  knowledge  construction.  In  addition, 
aspects  of  situated  cognition,  which  emphasizes  social  and  ecological  interactions  is  also  included.  Parts 
of  the  design  relies  on  an  information  processing  view  emphasizing  course  content  that  hierarchically 
structures  the  sequence  of  information.  Using  Hannafin,  Land  et  al.’s  (1999)  open  learning  model  which 
emphasizes  diversity  and  choice,  course  material  was  selected  to  reflect  a variety  of  print-based, 
multimedia  and  interactive  web-based  tools  that  could  best  enable  the  learner  to  attain  the  learning  goals 
set  for  the  course.  The  importance  of  constructive  alignment  between  different  design  components  is 
emphasised  by  Biggs  ( 1 999);  they  become  inextricably  linked,  so  that  the  course  becomes  greater  than  the 
sum  of  its  parts.  These  components  include  a textbook,  short  vignettes  with  practionners  who  introduce 
each  topic,  simulations  that  forecast  your  net  worth,  or  your  tax  bill,  interactive  practice  tests  including 
pre-tests,  review  exercises,  short  cases,  etc  and  communication  software  that  allows  for  threaded 
discussion,  chats,  critiquing,  posting  and  other  activities  consistent  with  best  practices  in  effective 
learning  communities.  Technology  has  been  nested  within  a constructivist  framework  to  raise  the 
intellectual,  social  and  ethical  standards  of  learners,  by  empowering  them  to  question,  participate,  and 
practice  authentic  issues  in  personal  finance.  Jonassen,  Davidson  et  al.  (1995)  summarize  this 
approach:**.  ..learning  is  necessarily  a social  dialogical  process  in  which  the  communities  of  practitioners 
socially  negotiate  the  meaning  of  phenomena”  (p.9).In  addition,  feedback  opportunities  are  provided  by 
experts  (the  instructor  as  well  as  practionners)  to  ensure  accurate  knowledge  acquisition.  Finally, 
communication  strategies  are  designed  to  facilitate  elaboration  of  content  and  solicitation  of  responses  that 


reflect  what  students  are  thinking.  Here,  the  tools  employed  include  interactive  tests,  e-mail  and  bulletin 
boards,  as  well  as  audio  conferencing. 

Data  reflecting  student  use  of  designed  tools  and  their  performance  on  assigned  tasks  will  be  collected  to 
confirm  the  validity  of  the  design  model.  Evidence  of  learning  using  Bloom’s  (1956)  taxonomy  will  reveal 
the  general  factors  associated  with  the  model.  Path  analysis  will  be  used  to  confirm  the  relationship 
amongst  the  design  components  as  they  fit  within  the  instructional  design  model. 
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Abstract:  Multimedia  instruction  must  follow  instructional  design  principles  that  are 
based  on  learning  theories.  This  session  will  share  an  effective  model  for  designing 
learning  materials  and  will  describe  the  strategies  that  should  be  used  when  designing 
multi-media  learning  materials. 


Pre-instructional  Activities 

Before  teaching  new  materials  to  students,  the  lesson  must  overview  the  materials  so  that  students  can  form 
a mental  set  for  the  materials  and  become  motivated  to  learn  the  materials.  The  different  types  of  pre- 
instructional  activities  include  content  map,  advance  organizer,  pre-test,  learning  objectives,  and  rationale. 

Prepare  Learning  Activities 

Conventional  instructional  methods  tend  to  use  passive  instructional  strategies.  It  is  important  to  use 
strategies  that  will  keep  students  active  so  that  they  will  be  motivated  and  will  process  the  information  in 
greater  depth.  To  encourage  active  learning,  it  is  important  to  use  discovery  strategies  so  that  students  can 
find  out  things  for  themselves.  Instead  of  teaching  a large  amount  of  information  to  students,  we  should 
give  students  the  task  and  then  guide  them  to  find  and  apply  the  information.  Multi-media  and  on-line 
instruction  are  excellent  methods  to  promote  active  learning. 


Elaborative  Strategies 

To  make  the  instructional  process  more  effective  and  efficient,  the  lesson  must  provide  activities  for 
students  to  process  the  content  at  a deep  level.  The  deeper  the  information  is  processed,  the  better  it  is 
learned  or  remembered.  As  a result,  the  instruction  should  contain  elaborative  strategies  to  help  facilitate 
deep  processing.  Some  common  elaborative  strategies  are  asking  questions  during  the  instruction,  ask 
students  to  summarize  segments  of  the  instruction,  requesting  students  to  make  their  own  notes,  and  asking 
students  to  underline  or  highlight  important  information.  The  elaboration  process  facilitates  efficient 
storage  in  long-term  memory.  This  is  important  since  the  ease  of  retrieval  will  depend  on  the  way  the 
information  is  stored  in  long-term  memory. 

It  is  known  that  when  new  learning  materials  are  encountered,  they  are  attended  to  and  transfer  to  short- 
term memory  storage.  After  entry  into  short-term  memory,  the  information  must  be  transferred  to  long- 
term memory  immediately  since  short-term  memory  has  limited  capacity.  As  a result,  students  should  be 
given  the  opportunity  to  elaborate  on  small  segments  of  instruction  so  that  the  materials  can  be  transferred 
to  long-term  memory  immediately. 


Presentation  Strategies 
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Most  instruction  is  done  in  an  expository  fashion  where  the  learning  material  is  presented  in  its  final  form 
to  students.  The  most  common  expository  methods  are  rote,  inductive,  and  deductive  learning. 

The  rote  method  consists  of  presenting  the  materials  to  students  and  then  asking  them  to  recall  the  materials 
as  presented.  For  example,  the  lesson  may  present  a rule  and  then  ask  students  to  recall  it.  The  deductive 
method  occurs  when  students  are  given  a rule  followed  by  examples  to  explain  the  rule.  The  inductive 
method  involves  giving  examples  on  a certain  rule  followed  by  the  rule.  However,  to  make  learning  more 
efficient  and  effective,  discovery  strategies  must  be  used.  With  the  introduction  of  new  instructional 
technologies  and  multi-media,  it  is  possible  to  design  instruction  that  contains  discovery  strategies. 

Practice  Exercises 

Another  important  component  of  instruction  is  practice  with  feedback.  After  the  presentation  of  the 
materials,  students  should  be  given  the  opportunity  to  practice  what  they  have  learned  so  that  they  can  get 
feedback  on  how  they  are  performing  and  at  the  same  time  prepare  them  for  the  final  test  in  the  lesson.  In 
addition,  the  practice  test  acts  as  a good  learning  strategy  since  test  questions  usually  consist  of  general 
ideas  which  act  as  cues  to  recall  specific  details  and  skills. 

Summarize  the  Lesson 

After  the  lesson  is  presented  to  students,  they  should  be  given  a summary  so  that  they  can  get  a sense  of 
closure  for  the  lesson  and  gives  a general  review  of  the  lesson. 
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Abstract:  Training  preservice  education  students  on  the  Internet  requires  the  course  to  be  more 
structured  than  would  be  required  to  teach  the  same  course  without  an  electronic  component 
(Gillette,  1996).  Education  students  have  consistently  expressed  the  need  to  see  technology  used 
by  their  university  instructors  if  they  are  to  use  these  tools  effectively  in  their  own  teaching  (Larson 
& Clift,  1996).  Due  to  both  a lack  of  suitable  instruments  and  the  importance  of  being  able  to  assess 
web-course  effectiveness,  the  authors  developed  the  Web-Based  Survey  of  Instructional 
Effectiveness  (WebSlE)  to  meet  this  need.  Forty-one  students  enrolled  in  a web-based  course  were 
given  the  WeBSlE  prior  to  and  after  the  completion  of  the  course.  Pretest  to  posttest  comparisons 
showed  students  improved  in  their  self-reported  web-based  skills  and  attitudes  in  the  use  of  the 
internet,  integration  of  the  internet  into  instruction,  general  computer  use/attitude  and  using  a 
computer  for  communication.  Future  studies  need  to  replicate  these  findings  and  investigate  the 
effects  of  the  integration  of  technology  into  more  and  varied  teacher  training  courses. 


The  effective  use  of  technology  includes  rethinking  traditional  forms  of  instruction.  Instructors  teaching  online  need 
to  maintain  the  perspective  that  we  learn  best  “with”  technology  rather  than  “from”  it  (Johnson,  1996).  This 
perspective  allows  instructors  to  engage  students  in  active  (Copley,  1992;  Reeves,  1996;  Yakimovicz  & Murphy, 
1995),  authentic  (Copley,  1992),  challenging  tasks  (Copley,  1992;  Reeves,  1996),  and  collaborative  efforts  (Reeves, 
1996;  Yakimovicz  & Murphy,  1995). 

A computer-specific  introductory  course  is  usually  insufficient  in  modeling  for  preservice  teachers  the  use  of 
technology  as  a tool  for  teaching  and  learning  (Topp  et  al.,  1994).  Education  professors  from  various  disciplines 
must  begin  to  use  technology  as  a cognitive  tool  to  mediate  and  transform  the  training  experiences  of  teacher 
education  students  (Larson  & Clift,  1996). 

Instructional  Design 

Training  preservice  education  students  on  the  Internet  requires  the  course  to  be  more  structured  than  would  be 
required  to  teach  the  same  course  without  an  electronic  component  (Gillette,  1996).  The  structure  that  is  embedded 
and  remains  consistent  throughout  each  unit  of  study.  The  structure  of  an  introductory  special  education  course 
taught  online  will  be  elaborated  on  in  this  section  by  clustering  the  components  into  categories  and  providing  a brief 
description  of  each. 

1 . Instructional  Design  Framework 

(a)  Each  unit  of  study  is  divided  into  six  instructional  steps. 

(b)  Each  of  the  six  steps  are  divided  into  instructional  components. 

(c)  Reference  numbers  are  assigned  to  each  of  the  instructional  components.  The  reference  numbers 
are  depicted  using  three  digits  separated  by  decimals  (e.g.,  2.3.4  represents  Unit 
2.Step3.Activity4).  The  reference  numbers  are  used  when  discussing  activities  via  e-mail, 
newsgroup,  and/or  chat. 

(d)  Buttons  are  located  at  the  bottom  of  each  of  the  six  instructional  step  pages  so  that  students  can  go 
to  the  “next  step”  or  return  to  the  “unit  index”,  which  provides  links  to  all  six  steps. 

2.  Curriculum  Development 

(a)  The  curriculum  is  built  upon  the  principles  of  active  learning,  cooperative  groups,  peer-mediated 
instruction,  and  content  experts 
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(i)  Active  learning  is  integrated  into  the  course  through  the  use  of  interactive  forms  and  case 
studies  found  on  the  Internet 

(ii)  The  class  is  divided  into  cooperative  groups  of  two  based  on  where  students  live  and  their 
educational  major, 

(iii)  Peer-mediated  instruction  is  used  through  the  creation  of  student  homepages  and  peer 
expertise  pages.  These  pages  allow  students  to  share  their  work  and  ideas  with  each  other, 

(iv)  Content  experts  are  utilized  by  incorporating  parents  of  students  with  disabilities  and 
personnel  in  special  education  field  in  local  public  schools  into  the  class  discussions  (e,g,, 
e-mail,  chat,  and  newsgroup), 

(b)  All  activities  taught  in  the  traditional  course  on  campus  are  incorporated  into  this  online  course 
through  the  use  of  e-mail,  chat,  newsgroup,  electronic  handouts,  forms,  and  web  sites. 

Teaching  in  this  virtual  classroom  in  comparison  with  teaching  in  the  traditional  classroom  requires  much  more 
learning  and  much  less  teaching  (Peterson  & Facemyer,  1996),  The  dynamic  learning  environment  available  online, 
which  is  expanding  at  exponential  rates,  provides  opportunities  for  students  to  become  explorers  and  discoverers 
(Gates,  1996), 

Education  students  have  consistently  expressed  the  need  to  see  technology  used  by  their  university  instructors  if  they 
are  to  use  these  tools  effectively  in  their  own  teaching  (Larson  & Clift,  1996),  All  students  enrolled  in  this  online 
course  had  completed  at  least  one  required  computer-specific  course  prior  to  the  beginning  of  this  online  course.  The 
development  of  a survey  of  instructional  effectiveness  was  developed  by  the  authors  to  measure  the  effects  of 
learning  educational  content  through  the  use  of  web-based  technology, 

Web-Based  Survey  of  Instructional  Effectiveness 

Due  to  both  a lack  of  suitable  instruments  and  the  importance  of  being  able  to  assess  web-course  effectiveness,  the 
authors  developed  the  Web-Based  Survey  of  Instructional  Effectiveness  (WebSIE)  to  meet  this  need.  Expert  opinion 
from  individuals  with  both  academic  credentials  (Ed,D,  or  Ph,D,)  and  web-based  course  experience  were  sought  to 
develop  an  original  pool  of  58  items.  This  pool  of  items  was  subjected  to  factor  analysis,  A five-factor  model  was 
adopted  retaining  30  items:  Use  of  the  Internet  (8  items);  Integration  of  the  Internet  into  Instruction  (8  items); 
General  Computer  Use/Attitude  (4  items);  Using  a Computer  for  Communication  (5  items);  and  web-based  Course 
Effectiveness  (5  items). 

Using  this  30  item  survey  authors  administered  313  to  preservice  education  students  and  practicing  teachers, 
Cronbach’s  alpha  and  split-half  reliabilities  were  calculated  for  the  five  scales.  Alpha  for  the  whole  30  item 
instrument  was  ,803,  and  the  five  scales  were  Use  = ,557;  Instruction  = ,531;  Computer  Use/Attitude  = ,415; 
Communication  = ,653;  and  Course  Effectiveness  = ,419,  Split-half  reliabilities  for  the  five  scales  were  Use  = ,762; 
Instruction  = ,750;  Computer  Use/Attitude  = ,726;  Communication  = ,823;  and  Course  Effectiveness  = ,672, 

Forty-one  students  enrolled  in  a web-based  course  were  given  the  WeBSIE  prior  to  and  after  the  completion  of  the 
course.  Single-sample  Dependent  T-tests  were  performed  on  all  five  scales.  An  alpha  level  of  ,05  was  used  for  all 
statistical  tests.  Four  of  the  five  scales  were  significant:  Use,  t4i  = 4,873,  p_<  ,001;  Instruction,  t4i  = 4,143,  p_<  ,001; 
Computer  Use/Attitude,  t4i  = 3,797,  p_<  ,001 ; and  Communication,  t4i  = 2,401,  p_<  ,02, 

Pretest  to  posttest  comparisons  showed  students  improved  in  their  self-reported  web-based  skills  and  attitudes  in  the 
use  of  the  internet,  integration  of  the  internet  into  instruction,  general  computer  use/attitude  and  using  a computer  for 
communication.  These  statistically  significant  gains  in  web-based  skills  and  attitudes  were  noted  after  one  education 
methods  course  that  modeled  the  use  of  the  Internet  in  instruction.  Future  studies  need  to  replicate  these  findings 
and  investigate  the  effects  of  the  integration  of  technology  into  more  and  varied  teacher  training  courses.  Additional 
effects  of  the  integration  of  technology  into  teacher  training  programs  needs  be  studied  after  these  preservice 
teachers  become  practicing  teachers,  focusing  on  how  often  and  in  what  ways  they  integrate  technology  into  their 
own  teaching. 
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Abstract:  This  paper  reports  the  findings  of  a study  conducted  with  an  experimental 
undergraduate-level  class.  It  investigated  the  differences  between  learners  studying  with  the 
teacher  face-to-face  and  learners  studying  at  distance  in  terms  of  three  dimensions;  © learners’ 
sense  of  distance,  © interaction,  and  ® achievement. 


Introduction 

One  of  the  most  important  innovations  resulting  from  the  remarkable  advances  in  communication  and 
Internet  technologies  is  '‘face-to-face  at  distance  education  \ which  is  forcing  us  to  adapt  new  definitions,  theories 
and  practices.  Physical  distance  is  no  longer  the  limiting  factor;  "psychological  distance”  is.  Drawing  on 
Moore’s  work  redefining  “distance”,  “interaction”  and  “transactional  distance”  (Sewart,  1988  & Moore  1991), 
the  concept  of  psychological  distance  (PSD)  refers  to  how  close  the  participants  within  the  educational 
environment  feel  to  each  other  in  terms  of  emotional  comfort  and  satisfaction. 

As  Holmberg  (1988)  observes  “real  learning  is  primarily  an  individual  activity  and  attained  through  an 
internalization  process;”  thus,  learning  is  the  product  of  internal  interaction  between  the  learner’s  previous 
experiences  and  new  information  obtained  from  external  interaction  with  the  other  components  of  the  educational 
situation,  including  contents/media,  other  learners  and  the  teacher/instructor/coach. 

The  present  study  focuses  on  this  interaction  within  the  educational  environment,  in  terms  of  the  learners  ’ 
sense  of  distance"'  (SoD)‘,  as  a direct  indicator  of  PSD.  Specifically,  the  main  objective  of  the  experiment  is  to 
examine  the  relationships  that  exist  between:  a)  “the  learner's  sense  of  distance”,  b)  “interaction  within  the 
educational  environment”  and  c)  “learners’  achievement”. 

The  Study 

Twenty-two  (only  the  data  from  19  participants  was  usable)  Japanese  undergraduate-student  volunteers, 
from  a women’s  university  attended  four  seminars  entitled  “Introduction  to  Public  Speaking”  during  spring, 
2000.  The  participants  were  divided  into  two  equal  groups,  matched  for  age  and  scores  for  English  and  Japanese 
language;  a "Hocal  group"  (LG),  which  studied  with  the  teacher  face-to-face  and  a ‘‘distant  group"  (DG),  which 
studied  with  the  same  teacher  at  distance.  An  ISDN-based  TV  conferencing  system  was  used  to  mediate 
communication  between  the  two  groups,  so  that  they  can  study  in  different  places  at  the  same  time.  Achievement 
was  evaluated  by  pre-tests  and  post-tests.  The  participants  were  asked  to  evaluate  their  “sense  of  distance”  from 
others  (i.e.,  the  teacher,  other  learners  in  the  same  group  and  other  learners  in  the  other  group)  on  a 5-point  scale, 
with  1 indicating  very  close  and  5 very  far.  The  participants  also  recorded  both  how  often  they  actually  interacted 
with  others,  and  how  often  they  wanted  to,  as  an  indication  of  the  interaction  within  the  educational  environment. 
The  classes,  as  well  as  short  interviews  after  each  class  to  regarding  technical  aspects  such  as  sound  and  picture 
quality,  were  also  videotaped  for  further  analysis  and  to  improve  the  seminar  contents. 


Sense  of  distance  is  contrasted  with  “real  distance,”  as  a measure  of  the  distance  learners  feel  to  exist  between  themselves 
and  other  components  of  the  educational  setting,  irrespective  of  the  actual  distance. 
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Findings 


The  differences  between  pre-test  scores  concerning  the  participant’s  knowledge  of  public  speaking  for  the 
two  groups  were  not  significant  (average  scores  were  4,5  and  5,22  for  the  LG  and  DG  conditions  respectively). 
The  scores  for  the  post-test  were  16,4  for  the  LG  and  18,66  for  the  DG,  indicating  significant  improvements  in 
knowledge  concerning  public  speaking  for  both  groups  (p  < 0,0001).  The  average  scores  for  both  groups  also 
indicate  that  there  were  no  significant  differences  between  the  two  groups  in  terms  of  achievement  at  the  end  of 
the  experiment. 

In  terms  of  learners’  SoD  to  the  teacher,  the  average  evaluation  rating  for  the  LG  condition  fell  from  3,166 
after  the  first  class  to  1,7  after  the  last  class.  In  the  DG  condition,  this  fell  from  4,7  to  3,00,  Although  analyses 
(Wilcoxon  Signed  Ranks  tests)  indicated  that  these  changes  were  significant  (p  < 0,025  and  0,015  for  LG  and  DG 
respectively),  further  analyses  (Mann-Whitney  U test)  also  indicated  that  there  were  significant  differences 
between  the  two  groups  after  both  the  first  and  last  classes.  In  other  words,  although  the  learners’  sense  of 
distance  to  their  teacher  declined  significantly  for  both  groups,  those  in  the  LG  condition  felt  much  closer  to  the 
teacher  then  those  in  the  DG.  Reviewing  the  videos  for  both  sites,  we  found  that  although  the  teacher  almost  had 
the  same  amount  of  interaction  with  both  groups  during  the  class,  he  did  spend  more  time  talking  and  explaining 
to  the  LG  both  before  and  after  the  classes,  and  that  he  was  able  to  give  more  individualized  attention  to  the 
learners  in  the  LG  condition,  which  may  have  contributed  in  making  them  feel  closer  to  him. 

The  data  indicated  no  significant  changes  in  the  learners’  sense  of  distance  to  other  learners  in  the  same 
groups  at  both  sites,  which  might  be  due  to  the  fact  that  the  students  came  from  the  same  university.  Examining 
the  learners’  sense  of  distance  to  the  learners  in  the  other  group,  the  evaluation  rating  fell  from  4,62  after  the  first 
class  to  3.00  after  the  last  class  for  the  DG  condition  (significant  at  4,6%  level).  However,  there  were  no 
significant  changes  in  the  SoD  for  learners  in  the  LG  condition.  This  might  be  due  to  the  initial  setting  at  the  LG 
site,  which  did  not  allow  learners  at  the  DG  site  to  see  their  counterparts  at  the  LG  site  during  the  first  class, 
which  was  adjusted  for  later  classes. 

The  correlation  between  the  learners’  SoD  to  their  teacher  and  the  records  of  interactions  was  -0,287  (sig, 
at  the  .05  level).  Although  no  significant  correlation  was  found  between  the  learners’  SoD  to  their  teacher  and 
achievement,  the  learners’  SoD  was  highly  correlated  (0,636  sign,  at  the  ,01  level)  to  how  important  the  learners 
regarded  the  contents,  and  how  much  they  felt  they  had  learnt  (0,496  sign,  at  .1  level). 

Conclusions 

The  results  of  this  study  suggest  that  it  might  be  very  difficult  to  acquire  the  same  quality  of  interaction 
with  physically-distant  learners,  and  that  it,  therefore,  takes  more  effort  in  order  to  reduce  their  SoD  compared  to 
learners  studying  face-to-face.  Despite  this,  however,  the  more  learners  interact  with  their  teachers  the  closer  they 
feel  to  them  regardless  actual  physical  distances.  Moreover,  as  learners  feel  closer  to  their  teachers,  they  tend  to 
regard  the  study  as  more  important  and  to  feel  that  they  understand  it. 
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Abstract:  Pedagogical  agents  offer  great  promise  to  improve  the  learning  experience  of  students 
interacting  with  educational  software  systems.  This  paper  describes  work  in  progress  on  a generic 
pedagogical  agent  architecture  and  an  authoring  system  to  support  the  creation  of  custom  agents  by 
teachers  and  other  domain  experts  using  a natural,  conversational  metaphor. 


The  field  of  pedagogical  agents  is  a recently  emergent  one  which  draws  from  previous  work  done  in  intelligent 
tutoring  systems,  autonomous  agents,  and  educational  theory  (Johnson  et  al.  2000).  Pedagogical  agents  have  the 
potential  to  offer  a wide  spectrum  of  educational  interactions  with  students,  including  opportunistic  instruction,  and 
have  been  shown  to  have  positive  affective  as.  well  as  cognitive  influences  on  students  (Lester  et  al.  1997). 

Several  successful  pedagogical  agents  have  been  designed  for  a variety  of  learning  environments,  ranging  from 
skill-based  training  systems  to  more  open-ended  constructionist  environments  (Johnson  et  al.  2000).  Currently, 
however,  the  task  of  building  pedagogical  agents  is  a non-trivial  one  which  falls  within  the  domains  of  researchers 
and  professional  programmers  — not  those  of  teachers,  curriculum  designers,  and  other  non-programmers. 
Authoring  tools  designed  for  non-programmers  to  create  customized  content  for  particular  agents  are  under  way,  but 
little  attention  has  been  devoted  to  the  question  of  authoring  tools  for  creating  customized  agents,  given  a particular 
computer-based  learning  environment.  It  is  to  this  question  that  we  turn  our  attention  in  this  paper.  We  will 
describe  work  in  progress  on  a project  which  aims  to  allow  teachers  and  other  non-programmers  to  author 
customized  pedagogical  agents  for  interactive  learning  environments.  We  first  discuss  the  design  of  a generic 
pedagogical  agent  architecture,  suitable  for  use  in  constructionist  learning  environments.  Secondly  we  discuss 
techniques  we  are  developing  to  allow  teachers  to  author  agents  based  upon  this  architecture.  We  require  that  the 
architecture  be  suitable  for  constructionist  learning  environments  due  to  their  open-ended  nature,  which  raises  a 
greater  challenge  for  the  agent  designer. 

Our  generic  agent  architecture  (see  figure  at  http://www.cs.washington.edu/homes/jbaer/pedagent/agentarch.jpg)  is 
built  around  the  Agent  Control  Module.  This  module  is  responsible  for  processing  events  received  from  the  learning 
environment,  updating  its  student  and  session  models  based  on  those  events,  initiating  interactions  with  the  student, 
and  potentially  sending  commands  back  to  the  learning  environment.  The  appearance  and  user  interface  aspects  of 
the  agent  have  been  separated  out  into  a distinct  module.  This  allows  for  maximum  flexibility  in  the  appearance  of 
the  agent.  Either  the  agent  can  appear  in  its  own  window,  or  the  learning  environment  itself  can  implement  this 
module,  allowing  the  agent  to  appear  within  the  main  window  of  the  environment,  as  do  many  of  the  existing 
pedagogical  agents  described  by  Johnson  and  Lester  (Johnson  et  al.  2000)  do. 

The  interaction  between  the  agent  and  the  learning  environment  consists  of  the  agent  receiving  events  from  the 
environment  and  requesting  state  information  from  the  environment  or  sending  commands  to  the  environment.  It  is 
up  to  the  environment  to  define  the  set  of  events  that  will  be  sent  to  the  agent,  as  well  as  the  state  and  control 
commands  that  it  wishes  to  expose.  A distinguishing  feature  of  this  agent  model  is  that  we  do  not  require  the  agent 
to  have  complete  access  to  the  state  of  the  learning  environment.  While  this  can  limit  the  information  which  the 
agent  has  to  work  with,  it  allows  for  interoperation  with  a wider  variety  of  learning  environments,  all  of  which  may 
represent  their  state  in  different  ways. 

While  the  Agent  Control  Module  is  the  center  of  operations  for  the  agent,  it  does  not  contain  any  built-in 
intelligence  pertaining  to  any  specific  learning  environment.  Instead,  it  relies  on  the  Agent  Behavioral  Control 
Description  (ABCD)  for  this  information.  The  ABCD  is  divided  into  four  types  of  objects:  potential  student  actions, 
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agent  interventions,  pedagogical  goals,  and  student  concepts.  Potential  student  actions  are  actions  or  classes  of 
actions  which  the  student  might  take  within  the  environment,  and  which  the  agent  should  recognize.  These  actions 
are  comprised  of  a series  of  events,  each  of  which  may  contain  a parameter  sent  from  the  learning  environment  to 
the  agent.  The  sequence  of  events  and  their  parameters  can  be  generalized  using  regular  expressions.  Agent 
interventions  may  be  of  three  distinct  types  - information  presentation  using  text  and  images,  agent-initiated  dialog 
with  the  student,  or  agent-initiated  intervention  of  arbitrary  complexity  by  loading  and  executing  an  externally 
authored  program.  Pedagogical  goals  represent  general  learning  objectives,  for  example,  what  the  most  important 
concepts  are  for  a student  to  learn,  what  interaction  styles  to  prefer,  etc.  Student  concepts  represent  correct  or 
incorrect  conceptions  that  students  may  hold.  Our  notion  of  student  modeling  is  based  upon  the  work  of  Minstrell  et 
al  on  educational  facets  (Minstrell  1 992),  and  our  student  concept  database  is  a facet  base,  in  their  terminology.  The 
student  model  the  agent  constructs  consists  of  a collection  of  student  concepts;  the  agent  infers  these,  and  each  is 
weighted  with  a confidence  factor. 

Since  it  is  the  Agent  Behavioral  Control  Description  (ABCD)  which  makes  a particular  agent  unique,  we  must  allow 
non-programmers  to  author  it.  Its  rules  are  similar  to  rules  in  a declarative  programming  language.  However,  we  do 
not  want  to  require  teachers  to  think  like  programmers.  Our  solution  is  to  have  the  agent  guide  the  author  through 
the  process  in  a natural,  conversational  manner.  The  agent  directs  the  authoring  process  by  asking  a series  of 
questions.  As  the  author  responds  to  these  questions,  the  agent  modifies  the  conversational  flow  and  builds 
components  of  its  ABCD,  based  on  the  author's  input.  In  addition  to  direct  question  and  answer,  the  agent  utilizes 
generalization  techniques  from  machine  learning  to  allow  for  authoring  by  demonstration  in  some  circumstances. 

This  authoring  technique,  which  we  call  "conversational  authoring",  bears  a resemblance  to  the  command  "wizards" 
which  are  found  in  a variety  of  applications.  However,  there  are  a number  of  factors  that  distinguish  this  authoring 
paradigm  from  application  command  wizards.  One  factor  is  that  the  interaction  between  the  author  and  the  system 
may  be  long-lived  and  span  multiple  authoring  sessions.  Because  of  this,  a need  exists  for  the  agent  to  maintain 
continuity  between  sessions  by  being  able  to  summarize  the  previous  session  and  describe  its  current  state  to  the 
author  in  a conversational  manner.  This  self-description  is  also  important  for  purposes  of  agent  validation.  The 
conversational  authoring  tool  mns  concurrently  with  the  agent  and  learning  environment,  so  the  author  can  test  the 
behavior  of  the  agent  at  any  point  in  the  authoring  process  by  using  the  environment  from  the  perspective  of  a 
student.  However,  simply  testing  the  agent  may  not  provide  enough  validation  that  the  desired  ABCD  structure  has 
been  created.  The  ability  of  the  agent  to  conversationally  summarize  its  state  at  any  given  point  in  the  authoring 
process  adds  another  tool  that  the  author  can  use  to  validate  the  authoring  outcome. 

The  work  described  in  this  paper  is  currently  in  progress.  The  generic  agent  architecture  has  been  implemented  in 
Java  and  integrated  into  a constructionist  learning  environment  called  PixelMath.  PixelMath  is  a recent  outgrowth 
of  research  at  the  University  of  Washington  designed  to  teach  mathematical  concepts  through  digital  image 
processing.  Additionally,  a custom  agent  has  been  built  on  top  of  the  generic  architecture  by  hard  coding  the  ABCD 
in  Java.  We  are  currently  in  the  process  of  implementing  the  conversational  authoring  tool,  which  will  be  tested  for 
effectiveness  by  teachers  familiar  with  the  classroom  use  of  tools  like  PixelMath.  A complete  description  of  this 
project,  including  current  status,  is  available  at  http://www.cs.washington.edu/homes/Jbaer/pedagent/index.html. 

References 

Johnson,  W.L.,  Rickel,  J.,  and  Lester,  J.  (2000).  Animated  Pedagogical  Agents:  Face-to-Face  Interaction  in  Interactive  Learning 
Environments.  International  Journal  of  Artificial  Intelligence  in  Education,  (2000),  1 1,  to  appear. 

Lester,  J.,  Barlow,  T.,  Converse,  S.,  Stone,  B.,  Kahler,  S.,  and  Bhogal,  R.  (1997)  The  Persona  Effect:  Affective  Impact  of 
Animated  Pedagogical  Agents.  Computer-Human  Interaction,  1997.  Assoc,  for  Comput.  Machinery,  New  York,  NY.  359-366. 

Minstrell,  J.  (1992).  Facets  of  students  knowledge  and  relevant  instruction.  Proceedings  of  an  international  workshop  - Research 
in  physics  learning:  Theoretical  issues  and  empirical  studies.  The  Institute  for  Science  Education.  110-128. 

Acknowledgements 

The  authors  wish  to  acknowledge  the  contributions  of  Chenoah  Morgan  to  the  creation  of  the  PixelMath  learning  environment 
and  the  preparation  of  this  manuscript.  Partial  support  for  the  student  modeling  part  of  this  work  from  NSF  Grant  CDA-96 16532 
is  gratefully  acknowledged. 


15.90 


From  “Learning-by-Viewing”  to  “Learning-by-Doing”: 
A Video  Annotation  Educational  Technology  Tool 

Ofer  Bergman,  Ruth  Beyth-Marom,  Ahuva  Leopold,  The  Open  University  of  Israel 
Doron  Hadar,  Ben  Gurion  University,  Israel 
Amnon  Dekel,  Interdisciplinary  Center,  Herzliya,  Israel 
Synopsus@netvision.net.il 


Abstract:  Video  films  are  particularly  suitable  as  a way  of  experiencing  the  world 
because  they  provide  a dynamic,  although  vicarious,  encounter  through  sound  and 
vision.  However,  the  use  of  video  films  in  their  present,  mostly  passive,  form,  do 
not  succeed  in  contributing  enough  to  the  learning  process.  SYNOPSUS  was 
designed  to  cause  a shift  from  the  passive  “leaming-by-viewing”  model  to  the 
“leaming-by-doing”  model  while  studying  with  educational  films.  By  utilizing  a 
simple  user-friendly  interface,  the  tool  allows  students  to  engage,  on  a computer, 
in  active,  reflective,  and  interactive  learning  with  existing  video  films.  The  tool 
and  some  preliminary  results  of  an  evaluation  study  will  be  presented. 


Introduction 

video  films  are  particularly  suitable  as  a way  of  experiencing  the  world  because  they  provide 
a dynamic,  although  vicarious,  encounter  through  sound  and  vision  (Collis,  1 996).  Moreover,  they  use 
a number  of  technical  devices  to  manipulate  that  experience.  Salomon  (1979)  called  these  devices 
“supplantations”,  in  the  sense  that  they  supplant  a cognitive  process. 

However,  the  use  of  video  films  in  their  present,  mostly  passive,  form,  do  not  succeed  in 
contributing  enough  to  the  learning  process  (Clark  & Craig,  1992).  In  active  learning  situations 
students  are  able  to  reflect,  construct  and  interact  (Laurillard,  1993,  Bates,  1997).  In  such  learning 
environments,  students  are  expected  to  comprehend  the  structure  of  the  lesson,  integrate  its  parts  and 
construct  their  knowledge  through  experienc-based  actions.  Present  video-film  learning  environments 
do  not  succeed  in  providing  such  a model. 

SYNOPSUS  was  designed  to  improve  the  passive  “leaming-by-viewing”  model  by 
reinforcing  a shift  to  “leaming-by-doing”  while  studying  with  educational  films. 


By  utilizing  a user-friendly  interface,  the  tool  allows  students  to  engage,  on  a computer,  in 
active,  reflective,  and  interactive  learning  with  existing  video  films.  By  allowing  students  to  actively 
summarize  a video  film,  the  tool  encourages  them  to  engage  in  active  learning.  While  summarizing 
the  film,  students  are  creating  their  own  personal  model  of  the  movie,  its  structure,  and  its  contents. 

When  the  observer  decides  to  summarize  the  film  (which  is  seen  on  the  left  side  of  a 
computer  screen),  he/she  begins  typing  the  summary.  Immediately,  the  film  stops  running;  the  written 
summary  appears  at  the  right  side  of  the  screen  and  the  last  video  frame  is  attached  as  an  icon  to  the 
beginning  of  the  text.  Pressing  the  Esc  function  resumes  the  film,  and  the  summary  with  its  icon  are 
framed  in  a box.  Each  time  the  observer  begins  typing,  the  film  stops  running  and  another  summary 
appears  with  its  relevant  icon  on  the  right  side  of  the  screen.  At  the  end,  the  student  has  a list  of 
blocks  in  each  of  which  there  is  an  icon  representing  the  place  of  summary  in  the  film,  and  a written 
text  which  can  be  printed.  This  list  of  blocks  can  be  arranged  according  to  their  order  in  the  film,  or 
according  to  the  order  in  which  they  were  created. 

By  automatically  attaching  relevant  video  icons  next  to  a summary  block,  the  tool  provides 
an  easy  way  of  visually  remembering  and  keeping  track  of,  and  returning  to,  important  points  in  the 
film.  This  capability  encourages  students  to  engage  in  reflective  thinking  since  relevant  sections  of 
the  film  can  be  found  and  seen  by  clicking  on  an  associated  video  icon  that  immediately  moves  the 
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film  to  the  intended  section.  Reflection  is  also  aided  by  allowing  immediate  access  to  relevant  sections 
in  the  film  via  simple  word  searches  on  the  textual  summaries. 

The  tool  also  allows  instructors  to  prepare  interactive  questions  and  assignments  pertaining 
to  the  film  watched.  The  questions  and  assignments  pop  up  when  the  student  reaches  a pre-specified 
point  in  a the  film. 

SYNOPSUS  can  be  used  for  distance  learners  and  for  face-to-face  teaching  during  frontal 
teaching  sessions.  By  using  the  presentation  mode  of  the  tool,  instructors  can  show  predefined 
segments  of  a film  along  with  their  own  headers  and  annotations. 

An  Evaluation  Study 

The  Open  University  of  Israel  (OUI)  is  a distance  learning  institution.  Some  of  the  courses 
offered  to  students  are  telecourses  which  include  as  learning  material  a textbook,  26  video  films  and  a 
student  guidebook.  A telecourse  in  psychology  was  chosen  for  a study  of  the  new  tool.  Out  of  400 
registered  students,  100  volunteered  to  try  the  software.  They  received  two  CDs:  one  contained  the 
software  and  the  other  contained  a thirty-minute  educational  film.  The  first  60  students  who  were 
willing  to  participate  in  the  study  (after  experimenting  successfully  with  the  software),  were  divided 
into  two  groups  of  30  students:  In  group  1 students  received  films  1 to  6 on  CDs,  and  in  group  2 
students  received  films  7 to  12  on  CDs.  Both  groups  received  all  other  films  on  video  cassettes.  The 
two  groups  were  similar  in  average  age,  percentage  of  males  and  number  of  previous  course  credits. 
This  design  allows  for  between  and  within  comparisons  regarding  the  experience  with  the  software 
and  CDs  on  the  one  hand  and  video  films  on  the  other  hand,  A third  group  of  eight  students  with 
learning  disabilities  served  for  an  exploratory  investigation  into  the  possible  advantages  of  the 
software  for  students  with  such  difficulties.  They  received  the  same  material  as  group  1 , 

A questionnaire  will  be  sent  to  all  three  groups  at  three  points  in  time:  (a)  after  the  allocated 
learning  period  for  the  first  six  films  (CDs  for  groups  1 and  3 and  videos  for  group  2);  (b)  after  the 
allocated  learning  period  for  films  7-12  (CDs  for  group  2 and  videos  for  groups  1 and  3)  and  ( c)  after 
the  allocated  learning  period  for  films  13-18  (videos  for  groups  1,  2 and  3).  Group  3 subjects  will  be 
interviewed  individually  regarding  their  experience  with  the  software  and  its  possible  help  in 
overcoming  some  of  their  special  learning  difficulties.  At  the  end  of  the  semester  students  will  be 
asked  to  send  their  summaries  to  the  research  team. 

The  purpose  of  the  experiment  was  to  test  learners’  attitudes  towards  the  learning  tool  as 
well  as  their  learning  processes  while  using  it.  The  design  of  the  experiment  permits  absolute 
evaluations  as  well  as  comparisons  with  learners  who  watch  the  films  on  the  TV  screen  without  the 
tool.  The  following  hypotheses  will  be  tested: 

1.  SYNOPSUS  facilitates  the  process  of  summarizing  the  film, 

2.  Subjects  summarize  more  and  better  with  SYNOPSUS  than  without  it, 

3.  SYNOPSUS  facilitates  concentration  while  watching  the  film, 

4.  SYNOPSUS  facilitates  the  process  of  locating  a specific  subject  in  the  film, 

5.  Subjects  go  backward  and  forward  in  the  film  in  order  to  access  specific  segments  of  it  more  often 
while  learning  with  SYNOPSUS  than  while  learning  without  it, 

6.  Subjects  enjoy  the  film  more  on  a television  screen  without  SYNOPSUS  than  on  the  PC  screen  with 
SYNOPSUS. 

7.  Subjects  feel  they  learn  more  from  a film  they  watched  on  a PC  screen  with  SYNOPSUS  than  from 
one  they  watched  on  a TV  screen  without  it, 

SYNOPSUS  will  be  presented  along  with  some  of  the  experiment’s  results  that  will  be 
available  at  the  time  of  the  conference. 
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Abstract:  The  purposes  of  the  Internet  are  many;  commercial,  recreational,  informational, 
educational,  communication al,  relational.  The  Internet  is  a paradoxical  universe  of  structure  and 
anarchy:  producing  some  good  and  some  bad,  some  beautiful,  and  some  ghastly.  There  are  many, 
educationally  relevant  Internet  resources,  “by  the  year  2000,  anyone  in  western  civilization  will  be 
able  to  get  the  answer  to  any  question  that  has  an  answer”  according  to  influential  Byte  columnist, 
Jerry  Poumelle.  In  the  1997  Current  Population  Survey  by  the  US  Census  Bureau  computer  use 
was  surveyed.  The  study  showed  that  three  quarters  of  the  children  in  the  USA  have  access  to 
computers  at  home  or  school  - significantly  more  at  school  than  at  home.  Children  most  frequently 
used  home  computers  for  educational  purposes  (93.3%)  and  games  (83%).  One  fifth  of  the  children 
who  have  home  computers  used  them  to  access  the  Internet.  For  adults,  1 8 and  over,  92  million 
people  used  the  computer  at  home  or  work.  More  than  half  these  adults  accessed  the  Internet  from 
home,  with  80%  of  them  using  the  web  to  access  government,  business,  health  or  educational 
information.  Significant  increases  on  the  order  of  5-15%  were  recorded  for  every  category  of 
questioning  over  the  5 years  span  between  surveys.  In  all,  one  of  five  Americans  (ages  3 and 
higher)  used  the  Internet  in  1997^. 

This  presentation/essay  concerns  controlling  a kid’s  access  to  “morally  undesirable”  information. 
But  before  we  can  get  to  that,  it  is  important  to  consider  the  need  for  such  control.  In  the  schools  of 
our  nation,  our  children  are  using  the  Internet  to  reach  out  of  the  walls  and  experience  the  universe. 
Free  ranging  access  to  the  wonders  of  science  and  humanity  can  be  had  through  simply  searching 
the  Internet  using  search  programs.  Any  teacher  or  student  deserves  these  resources  and  they 
deserve  to  follow  the  curiosities  that  are  a contingent  element  of  human  intelligence.  The  question 
that  arises  involves  the  various  interests  of  the  purveyors  of  these  archives.  Some  of  these  interests 
are  illegal  and  even  more  are  morally  undesirable. 


Introduction 

The  purposes  of  the  Internet  are  many;  commercial,  recreational,  informational,  educational,  communicational, 
relational.  The  Internet  is  a universe  of  anarchy  and  chaos:  producing  some  good  and  some  bad,  some  beautiful,  and 
some  ghastly.  There  are  many,  educationally  relevant  Internet  resources,  “by  the  year  2000,  anyone  in  western 
civilization  will  be  able  to  get  the  answer  to  any  question  that  has  an  answer”  according  to  influential  Byte 
columnist,  Jerry  Poumelle.  In  the  1997  Current  Population  Survey  by  the  US  Census  Bureau  computer  use  was 
surveyed.  The  study  showed  that  three  quarters  of  the  children  in  the  USA  have  access  to  computers  at  home  or 
school  - significantly  more  at  school  than  at  home.  Children  most  frequently  used  home  computers  for  educational 
purposes  (93.3%)  and  games  (83%).  One  fifth  of  the  children  who  have  home  computers  used  them  to  access  the 
Internet.  For  adults,  18  and  over,  92  million  people  used  the  computer  at  home  or  work.  More  than  half  these  adults 
accessed  the  Internet  from  home,  with  80%  of  them  using  the  web  to  access  government,  business,  health  or 
educational  information.  Significant  increases  on  the  order  of  5-15%  were  recorded  for  every  category  of 
questioning  over  the  5 years  span  between  surveys.  In  all,  one  of  five  Americans  (ages  3 and  higher)  used  the 
Internet  in  1997^ 

This  essay  concerns  controlling  a kid’s  access  to  “morally  undesirable”  information.  But  before  we  can  get  to  that,  it 
is  important  to  consider  the  need  for  such  control.  In  the  schools  of  our  nation,  our  children  are  using  the  Internet  to 
reach  out  of  the  walls  and  experience  the  universe.  Free  ranging  access  to  the  wonders  of  science  and  humanity  can 
be  had  through  simply  searching  the  Internet  using  search  programs.  Any  teacher  or  student  deserves  these  resources 
and  they  deserve  to  follow  the  curiosities  that  are  a contingent  element  of  human  intelligence.  The  question  that 
arises  involves  the  various  interests  of  the  purveyors  of  these  archives.  Some  of  these  interests  are  illegal  and  even 
more  are  morally  undesirable. 
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Illegal  interests  are  pursued  and  prosecuted  in  the  United  States  by  state  and  local  law  enforcement,  the  FBI,  the  US 
Marshall  or  the  Federal  Trade  Commission  depending  on  which  jurisdiction  the  activity  entails.  The  international 
nature  of  the  Internet  may  bring  other  law  enforcement  agencies  into  the  investigation  and  prosecution.  In  the  US 
there  are  specific  police  resources  devoted  to  cyber-crime. 

On  the  morally  undesirable  side,  there  is  no  such  concerted  effort  - in  fact  there  seems  to  be  a fair  level  of 
disagreement.  While  there  is  much  media  attention  given  to  (thus  far  ineffective)  efforts  aimed  at  outlawing  these 
undesirable  Internet  activities,  there  is  a strange  avoidance  of  discussion  into  exactly  which  information  should  be 
restricted.  Consensus  has  arisen  around  some  pseudo-illegal  activities  such  as  distribution  of  child  pornography. 

But  aside  from  that. . .almost  certainly  the  literature  simply  starts  off  with  the  assumption  that  children  need  to  be 
protected  from  whatever  morally  reprehensible  activity  the  writer  is  imaging  but  not  really  describing.  And, 
honestly,  aside  from  acknowledging  this  vagueness,  this  writer  is  not  much  different. 

The  generally  agreed  philosophical  rationale  is  that  children  from  ages  5 to  1 8 must  be  protected  from  this 
information  because  they  are  not  ready  to  contextually  understand  these  darker  fantasies  and  features  of  our  culture. 
(In  extreme  cases  — 1 am  wondering  who  is?)  Kids  are  ignorant  to  the  dangers  that  may  be  encountered  through  their 
venturing  into  knowledge  unconditioned  by  experience.  A loud  debate  rages  about  “how  to  stop  access”  to  these 
dark  media  resources.  When  the  next  kids-killing-kids,  school  shooting  incident  occurs,  our  attention  will  be  brought 
sharply  back  to  the  studies  that  widely  implicate  media’s  influence  on  a child’s  moral  development.  This  research  is 
not  clear  on  the  nature  of  the  developmental  harm  associated  with  a premature  familiarity  of  the  darker  side  of 
humanity.  We  could  wish  for  more  specifics  as  to  which  type  of  media,  how  it  specifically  affects  development  and 
when  in  the  child’s  development  are  they  vulnerable  to  such  influence. 

For  purposes  of  this  essay,  it  will  be  assumed  that  kids  need  protection  from  morally  undesirable  elements  of  our 
culture  even  if  adults  do  not.  That  further,  the  special  media  circumstance  of  Internet  delivery  of  these  morally 
undesirable  archives  directly  into  our  schools  and  homes  deserves  our  special  attention.  We  will  assume,  as  do  many 
others,  that  an  inadvertent  curriculum  can  formulate  around  these  dark  and  undesirable  interests. 

Since  these  fascinations  with  the  darker  elements  exist  in  our  culture,  there  is  necessarily  a developmental  time 
when  youth  should  gain  controlled  experiences  in  them.  The  school  however  is  excluded  from  that  objective  - it  is 
left  to  parents  to  control  this  curriculum.  There  are  logical  flaws  in  this  plan  (e.g.  that  parents  often  do  not  address 
these  legitimate  needs)  will  also  be  excluded  from  this  essay. 

This  essay  will  assume  that  someday  these  “morally  undesirable”  graphical  and  information  archives  can  be 
identified  through  public  discourse  and  reasoning.  Something  beyond  the  “we  will  know  them  when  we  see  them” 
criteria.  Though  there  is  no  theory  on  whether  it  really  is  undesirable  for  children  to  be  exposed  to  any  of  the 
following  areas.,  .and  though  there  is  no  sharp  delineation  of  the  level  at  which  information  becomes  damaging, . .the 
general  content  areas  are  well  elaborated  in  the  literature  and  involve  concern  about  the  following  issues. 
Inappropriate  sexual  displays. 

There  is  a well-publicized  and  very  active  pornographic  sector  accessible  through  the  Internet^.  These 
resources  may  involve  text  or  graphical  displays  that  are  developmentally  inappropriate  for  children  (and 
frankly  adults!).  This  area  receives  much  attention  by  the  naysayers  of  open  Internet  access.  It  is  said  to 
represent  a significant  but  small  proportion  of  Internet  informational  traffic  (I  have  seen  figures  of  less  than 
10%  - 1 cannot  seem  to  find  the  source  right  at  this  moment  though)  Additionally,  these  sites  often  involve 
unscrupulous  schemes  to  charge  the  viewer  — while  claiming  to  be  free  of  charge.  It  is  a common  family 
dilemma  to  have  a surprise  encounter  with  the  financial  consequences  of  a child’s  experimenting  with 
Internet  sex  sites.  Again,  research  is  disappointing  here,  but  in  my  circle  of  friends  and  family,  children 
have  spent  thousands  of  dollars  in  this  way.  Anecdotally,  costs  associated  with  these  episodes  are 
confabulated  with  similar  costs  charged  by  cable  and  phone  sex  experimentation. 

Predatory  persons 

There  is  a danger  that  children  may  be  lured  into  meeting  some  predatory  person,  whose  desire  is  to  do 
them  physical  harm.  While  this  danger  is  theoretically  real,  there  is  no  frequency  data  available  in  the 
literature.  It  can  be  assumed  though  from  reports  of  adults  being  lured  out  to  meet  predatory  persons,  that 
children  would  be  also  similarly  vulnerable  to  such  risks^^. 
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Financial  Scams 

There  are  a number  of  financial  schemes  communicated  through  the  Internet.  Pyramid  schemes  are  popular 
— so  are  offers  “too  good  to  be  possible.”  Low  interest,  poor  credit  credit  card  offers  are  also  frequently 
offered  on  the  Internet.  It  can  be  assumed  that  the  limited  experience  of  kids  would  make  them  targets  for 
these  deceptive  or  illegal  activities.  Again,  there  is  no  data  readily  available  on  the  incidence  of  children 
being  victims.  It  is  sufficient  to  say,  that  there  are  many  adults  that  get  scammed.  We  can  be  sure  that  if 
there  were  unlimited  access  to  children  for  the  scam  artists  and  a source  of  credit  for  the  children,  there 
would  be  a significant  level  of  risk  to  the  financial  well  being  of  families  and  children 
Privacy  concerns 

The  issue  here  is  that  children  might  give  out  information  that  could  physically  target  them  for  predators. 
Another  concern  may  be  that  the  child,  not  being  aware  of  the  risks,  may  innocently  or  out  of  bravado 
arrange  a connection  with  an  unsavory  character^*’^^. 

System  integrity  and  security  issues 

There  might  be  risks  to  the  stability  of  the  network  due  to  downloading  programs  that  originate  from 
certain  Internet  sites.  There  is  a fair  amount  of  Internet  traffic  visible  on  lists  associated  with  “hacking.” 
(search  keyword  “crackers”)  The  substance  of  these  messages  often  reflects  an  interest  in  illegally  obtained 
software  and  passwords  to  various  nefarious  sites.  An  easy  and  more  popular  invasion  of  networks  involves 
reflecting  junk  e-mail  (“spam”)  from  the  users  domain  so  that  it  will  appear  to  be  from  your  computer 
system.  1 work  for  a women’s  college  in  the  upper  Midwest  and  we  discovered  that  advertising  for  a 
pornographic  site  was  using  our  domain  as  a reflector. . .to  the  extent  that  in  one  night  60,000  emails  had 
been  distributed!  This,  of  course,  was  unacceptable  to  our  network  administrators.  Protection  from  this  type 
of  hacking  has  been  installed.  Another  concern  is  viruses  and  virus  hoaxes.  Aside  from  the  damage  an 
actual  virus  can  do  to  networks,  the  volume  of  messages  that  virus  hoaxes  can  generate  is  in  itself,  a plague. 
Harassment 

It  is  possible  for  a predatory  person  to  stalk  and  harass  a person  via  the  Internet.  Especially  if  used  in 
combination  with  other  modes  of  information  accessible  through  the  Internet  or  other  information 
providers.  Phone  numbers,  addresses,  maps  may  assist  the  stalker  to  find  their  target  repeatedly.  It  is  more 
and  more  difficult  to  “disappear”  if  one  needs  to.  One  source  listed  the  costs  for  information  on  the  a 
person  as  $15  for  social  security  number,  to  $150  for  a complete  background  check. 

Internet  ‘‘addiction” 

While  dubious  as  a real  disorder,  some  people  spend  inordinate,  obsessive  amounts  of  time  on  the 
Intemet^’'**’^’*^’*^’*'*’'^  Sometimes  this  interferes  with  normal  activities  of  daily  living.  The  fear  is,  that 
introducing  Internet  use  in  schools  give  tacit  approval  for  obsessive  use  of  the  Internet.  There  is  no 
empirical  data  available  in  the  literature,  but  based  on  anecdotal  evidence  (again  from  families  and  friends) 
teenagers  are  known  to  sacrifice  duties  and  sometimes  sleeping  in  order  to  use  the  Internet.  Recently,  the 
American  Psychological  Association  gave  some  context  to  the  question  during  a symposia  on  the  topic. 
They  concluded  that  pathological  internet  use  was  possibly  an  addiction,  a disorder  or  a symptom  of 
depression. 

Abuse  of  credit 

A problem  not  exclusive  to  the  Internet,  there  cannot  be  any  doubt  that  the  Internet  is  a good  place  to  spend 
a lot  of  money.  Many  Internet  retail  and  service  sites  exist  and  accept  credit  cards.  It  is  easy  to  rack  up  an 
impressive  bill  in  a few  hours  of  Internet  shopping.  Amazon.com  for  books,  records,  presents;  Quill  for 
office  supplies;  LandsEnd  for  clothing;  EggHead  for  software;  Del!  for  a computer;  Auto.com  for  a car; 
ebay  for  a boat;  pretty  soon  you’ve  spent  some  bucks! 

Depression 

There  have  been  articles  in  the  psychology  literature  about  a relationship  between  Internet  use  and 
depression^’^.  The  debate  rages  on  with  both  side  positing  credible  pros  and  cons.  The  cmx  of  the  issue  here 
is  the  chicken/egg  problem.  That  is  did  the  depression  precede  the  use  of  the  Internet  or  was  depression  a 
result  of  Internet  use?  Currently  the  theory  that  seems  most  favored  is  that  use  of  the  Internet  is  more  a 
symptom  than  a cause  of  depression. 

Academic  integrity 

A virtual  industry  has  evolved  selling  term  papers  to  students.  Keyword  search  on  “term  papers”  revealed 
100s  of  sites  devoted  to  free  and  for  sale  papers.  Many  services  offered  custom  services.  According  to 
“Paper-Masters”  their  custom  written  term  papers  are  meant  to  be  research  to  assist  you  writing  your  own 
paper.  They  conclude  that  it  is  not  in  any  way  unethical  to  use  their  services. 
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Identity 

The  relative  anonymity  provided  by  the  Internet  is  a challenge  to  individual  identity.  This  feature  also 
allows  people  to  fabricate  an  identity  that  is  quite  different  from  their  own.  The  concern  is  that  this  fantasy 
identity  may  interfere  with  the  identity  forming  process  that  is  an  important  in  the  child^"^. 

Community 

There  are  fears  that  rather  than  encouraging  community  and  multicultural  exchange  the  Internet  actually 
encourages  isolation.^ 

Intellectual  property 

Many  teachers  report  a cut  and  paste  form  of  writing  where  the  students  simply  search  out  information  on 
the  Internet  and  then  copy  it  directly  into  their  papers  without  quotation.  Additionally  copyright 
infringement  rules  are  very  difficult  to  understand  within  the  context  of  the  Internet. 

Social  accountability 

The  anonymity  and  2 dimensionality  of  the  Internet  does  not  foster  authentic  relationships.  This  issue  is  the 
converse  of  the  safety  afforded  by  the  electronic  distance  of  the  Internet.  One  concern  for  kids  is  the 
negative  role  model  that  this  demonstrates.  The  superficiality  of  complex  individuals  as  represented  on  the 
Internet  is  also  an  impression  that  is  unrealistic.  On  the  Internet,  as  with  other  media,  the  distinction 
between  real  an  imaginary  is  difficult  to  detect.^^ 

Review  of  the  literature 

While  it  this  essay  is  focused  on  educational  issues  in  American  schools,  it  is  interesting  to  observe  that  in 
Europe  several  countries  (GDR,  GB  and  others)  have  also  sought  to  control  certain  types  of  Internet 
communications.  Great  Britain  actually  has  a complaint  bureau.  The  Internet  Watch  Foundation”,  where 
consumers  can  expect  to  see  prosecution  of  child  pornography,  and  other  harmful  activities.  The  bureau 
expressly  stays  clear  of  controversies  into  items  of  political  or  social  question  though.  Internet  access  is 
severely  limited  at  the  source  - the  Internet  access  hub,  in  several  Middle  Eastern  countries  (Saudi  and 
Egypt).  All  accessible  sites  are  government  approved  even  email  is  scrutinized. 

In  the  United  States,  a first  Amendment  disagreement  has  arisen  over  Internet  control.  The  US  Supreme 
Court  has  made  four  determinations  in  favor  of  free  Internet  access  for  anyone  able  to  get  there.  Though 
there  are  plenty  of  details  left  to  interpretation,  the  court  has  made  sweeping  statements  about  the  freedom 
of  the  Internet.  Congress  continues  to  contemplate  legislative  action,  intended  to  circumvent  the  Supreme 
Court's  First  Amendment  rulings  and  control  some  types  of  Internet  communications.  As  stated  before, 
illegal  activities  are  prohibited  and  prosecuted  in  multiple  ways,  the  debate  concerns  pseudo-legal  and  legal 
but  controversial  Internet  resources  such  as  child  pornography,  bomb-building  directions,  racism,  slander, 
misinformation. 

Strategies  for  limiting  the  access  of  kids  to  morally  undesirable  Internet  archives 
Free  access  to  all: 

It  would  seem  that  this  is  what  we  have.  This  is  a common  misconception  of  the  Internet.  Certain  types  of 
information  cannot  (currently)  be  transmitted  via  the  Internet.  The  interface  is  restricted  to  materials  that 
are  comprised  of  an  audio,  visual  and/or  digital  nature.  Certainly  this  covers  a lot  of  ground,  but  in  reality 
this  is  a significant  limitation  from  the  start.  Not  everything  the  human  has  a capacity  to  sense  can  be 
transmitted  by  the  Internet. 

Additionally,  and  of  more  relevance  to  the  theme  of  this  essay;  illegal  activities  are  prohibited  on  the 
Internet.  As  stated  above,  several  governmental  policing  agencies  monitor  and  in  fact  utilize  the  Internet  to 
investigate  and  prosecute  criminal  activities”.  Email  threats  are  taken  quite  seriously  — the  appearance  of 
treasury,  secret  service  officers  at  a local  high  school  last  year  to  arrest  a student  for  having  sent  a death 
threat  to  President  Clinton  is  lesson  enough  to  prove  this  point.  Drug  and  fireami  enforcement  agencies  also 
monitor  the  Internet  for  signs  of  criminal  activity. 

Direct  adult  supervision: 

There  is  a tacit  rejection  of  direct  adult  supervision  as  a control  strategy  in  almost  all  the  literature^”^.  It  is 
felt  unrealistic  that  an  adult  mentor  could  monitor  a kids  Internet  adventures,  or  help  them  gain  context  if 
they  bump  into  something  morally  undesirable.  The  literature  in  this  area  occasionally  mentions  that  this  is 
an  option,  but  that  then  goes  no  further  than  to  state  it  is  simply  not  feasible  for  presumably  practical 
reasons.  There  is,  however,  agreement  that  this  method  of  control  would  be  desirable  and  effective.  This 
writer  is  not  so  sure  that  this  is  the  contingent  difficulty  with  the  idea.  Challenges  here  may  be  more 
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associated  with  the  problems  of  communicating  with  our  youth.  Adults  are  really  hesitant  to  talk  about 
morally  challenging  topics  - a sudden  view  of  human  sexual  interaction  might  initiate  questions  - and  that 
would  lead  to  a difficult  conversation.  Teachers  should  be  allowed  to  have  this  kind  of  conversation  with 
our  youths,  but  the  onus  of  responsibility  is  on  the  family  where  it  <dismay>  rarely  occurs. 

User  Self  Policing  - Moral  development  and  the  Internet 

It  could  be  left  to  the  user  to  restrict  their  own  access  by  virtue  of  their  own  experiences^^’^^.  This  highly 
improbable  suggestion  centers  on  development  of  a personally  held  and  derived  dislike  for  morally 
undesirable  Internet  resources.  It  can  happen,  and  most  probably  does,  after  the  thrill  is  gone  so  to  speak. 
The  impractical  part  of  this  suggestion  is  not  the  philosophically  sound  grounding  that  exists  in  allowing 
people  to  morally  develop,  but  rather  the  political  impossibility  of  implementing  such  a plan  in  a country 
founded  by  the  Puritans,  Additionally,  there  is  much  to  be  said  for  the  idea  that  a staged  approach  with 
contextulization  would  need  to  be  implemented  to  coincide  with  the  developmental  capacity  of  each 
individual  kid.  Given  the  performance  record  of  our  public  schools,,, it  is  utopian  to  imagine  such  a strategy 
could  be  harmlessly  implemented. 

Legislation: 

Communication  Decency  Act  (CDA)  and  Son  of  CD  A - overturned  by  the  US  Supreme  Courts,  Ruled  as 
unconstitutional  by  virtue  of  their  violation  of  First  Amendment  rights'”.  Politicians  continue  to  bring  up 
legislation  that  the  American  Civil  Liberties  Union  defeats  in  court  challenges.  Literally  millions  of  dollars 
have  been  expended  in  the  quest  for  legislation  that  can  get  through  the  first  amendment  tests.  Often  the 
problem  is  that  the  definitions  for  morally  undesirable  materials  cannot  be  specifically  measured,  and  could 
thus  be  applied  too  widely. 

Acceptable  Use  Policies  (AUP): 

As  an  aide  to  self  control,  a signed  statement  of  appropriate  internet  use  AND  the  consequences  of  abuse  of 
privileges  is  used.  Samples  of  AUPs  are  available  on  the  Internet,  AUPs  are  used  in  nearly  all  networked 
computer  labs.  It  is  necessary  that  some  person  observe  the  Internet  use  in  such  situations.  This  can  be  done 
by  walking  around  the  lab  or  by  looking  at  the  log  of  Internet  addresses  that  has  been  accessed  on  a 
particular  computer.  It  would  seem  very  possible  to  automate  this  observation  of  a lab  by  sharing  the  log 
with  a central  teachers  workstation,  but  there  was  no  mention  of  this  strategy  in  the  literature.  See 
references^'  for  a link  to  an  example  provided  by  Rice  University, 

Rating  Systems: 

A strategy  familiar  to  moviegoers  is  common  of  the  Internet  - warning  the  consumer  of  adult  content.  Most 
commercial  (even  free)  adult  sites  require  the  user  to  physically  confirm  select  their  intent  to  utilize  the 
morally  undesirable  archives.  Some  Internet  businesses  have  arisen  that  claim  to  restrict  the  access  of 
minors  to  their  sites  by  virtue  of  credit  card  credentials.  Of  course,  a credit  card  has  little  to  do  with 
verifying  age  - these  sites  are  more  interested  in  connecting  interested  parties  (and  their  credit  cards)  with 
their  cadre  of  pornographic  vendors.  The  intent  is  to  protect  the  inadvertent  viewing  of  these  resources,  but 
of  course,  many  curious  15  year  olds  are  undeterred  by  these  warning  and  click  right  through. 

Screening  Software: 

It  would  be  nice  if  software  could  be  developed  to  filter  morally  undesirable  Internet  content,  but  thus  far 
efforts  have  been  only  partially  successful, Current  filtering  software  can  be  grouped  based  on  the 
strategy  used  to  filter.  One  group  uses  keywords,  taken  from  a ready  made  list  of  dirty  words,  to  eliminate 
access  to  undesirable  Internet  archives.  Often  the  list  can  be  modified  by  the  administrator  to  add  new 
words  or  to  add  in  a miss  spelling  of  the  particular  dirty  word  of  concern.  This  is  effective  but  suffers  from 
the  problem  that  sometime  dirty  words  also  have  morally  desirable  alternative  uses.  For  instance  “breast” 
as  a keyword  will  also  eliminate  sites  concerned  with  breast  cancer.  Another  approach  is  to  keep  a list  of 
either  acceptable  and  unacceptable  sites.  For  instance  sites  ending  with  the  “,gov”  suffix  are  approved, 
Disney, com  is  approved,  a list  of  sites  that  are  not  approved  is  provided  and  updated  by  the  software 
company  and  can  be  modified  by  the  administrator.  Unfortunately,  the  undesirable  element  of  the  Internet 
moves  around  a lot,  and  new  morally  undesirable  sites  are  launched  every  day,  so  it  is  easy  to  imagine  that 
something  will  get  past  the  filter. 

Network  Filter  programs: 

Similar  to  screening  software,  more  sophisticated  filters  can  be  installed  at  the  network  hub^”.  This  is  the 
strategy  used  by  those  Middle  Eastern  countries  to  prevent  access  to  certain  internet  resources. 

Intranet  strategies: 

One  strategy  taken  by  some  schools  has  been  to  simply  offer  only  internal  versions  of  Internet  available 
resources.  It’s  a sort  of  smaller,  internally  stored  version  of  the  Internet,  WebWhacker^”  whacks  entire  sites 
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out  of  the  Internet  and  stores  them  on  your  computer  network.  This  strategy  is  fine  as  long  as  the  learner 
only  asks  questions  available  within  the  limits  of  the  stored  information.  An  encyclopedic  resource,  and 
other  informative  resources  are  made  available.  Often  access  to  the  external  Internet  is  possible  for  special 
situations,  and  can  be  more  effectively  monitored. 

Internet  watch  groups: 

Another  way  used  in  concert  with  screening  software  is  to  have  a group  of  evaluators  monitor  the  moral 
permissibility  of  Internet  sites^^’^^  What  happens  is  that  a software  company  or  a school  enlists  the  help  of 
a screening  committee.  Their  function  is  to  expand  the  list  of  forbidden  and  permitted  Internet  resources. 

Free  market  pressure  and  industry  self  regulation: 

Especially  during  the  legislative  debate  on  CD  A,  many  Internet  Service  Provider  companies  became 
concerned  over  the  trouble  it  might  be  if  they  could  be  held  responsible  for  Internet  content.  America 
Online,  the  largest  Internet  Service  provider,  initiated  a program  of  parental  controls  that  allow  various 
levels  of  access  control  over  the  Internet  and  AOL  network  resources* Also,  they  developed  a system 
of  “moderators”  to  monitor  kid  chatrooms,  as  well  as  a responsive  complaint  system.  In  other  words,  they 
felt  it  was  good  for  their  business  interests  to  allow  the  user  to  determine  limits  on  access. 

Conclusions 

So  what  is  the  big  picture  here?  First,  this  is  a new  experience  to  all  of  us,  second  the  Internet  needs  to  be  a 
vital  part  of  the  learning  experience  for  kids.  Third  we  will  not  be  able  to  bring  ourselves  to  restrict 
anything  but  illegal  activities  from  the  Internet  - and  even  here  it’s  a post  hoc  analysis. 

Clearly  the  best  choice  is  to  design  a system  where  Internet  use  is  closely  monitored,  but  not  for  punitive 
purposes,  but  for  contextualizing  by  adults  who  are  skilled  and  vested  in  helping  kids  see  our  world,  all  of  it 
at  the  right  time.  Since  this  is  so  impossible  given  our  current  factory  model  schools,  I would  recommend 
the  strategy  where  much  of  the  information  needed  by  students  is  anticipated  and  locally  stored  on  school 
friendily  servers  — a second  network  soto  speak.  When  kids  need  to  venture  onto  the  Internet  let  them,  and 
be  there  for  them. 

There  is  much  research  that  needs  to  be  done,  but  some  of  the  important  questions  IMHO  (in  my  humble 
opinion)  have  to  do  with  cultural  changes  that  will  come  as  a result  of  this  new  dimension.  Questions  about 
moral  development,  community  and  diversity  seem  largely  unstudied.  Philosophical  and  psychological 
issues  such  as  what  really  is  inappropriate  and  at  what  developmental  age  are  barely  disturbed.  Finally,  1 
wonder  what  the  future  will  hold  for  new  ways  to  communicate  human  sensations  and  feeling  will  be. 
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Using  Speech  Recognition  to  Support  Online  Learning  of 
Second/Foreign  Languages 


Francis  Bonkowski,  1C  Education,  Inc.,  Canada 
Quinn  Cheung,  IC  Education,  Inc.,  Canada 
Liam  Cheung,  1C  Education,  Inc.,  Canada 


This  paper  presents  IC  Education’s  state-of-the-art  pedagogy  and  technology  to  support  online  language 
learning.  1C  Education  has  developed  speech  recognition  over  the  Internet  using  a new  client/server 
application  that  recognizes  the  voice  of  user’s  without  having  to  install  complicated  software  on  a personal 
computer.  The  system  entails  minimal  software  installation  by  the  user  and  provides  centralization  of  the 
speech  recognition  engine.  This  innovative  technology  is  a powerful,  albeit  simple  tool  used  to  teach 
learners  proper  pronunciation  and  new  vocabulary. 
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A BUILDING-BLOCK  METHOD  FOR  COURSEWARE 
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Abstract;  Although  courseware  has  the  potential  to  increase  the  efficiency  and 
effectiveness  of  courses,  until  now  the  results  of  its  application  is  often  disappointing.  In 
our  research  concerning  courseware  development  within  the  Royal  Netherlands  Army 
(RNLA),  we  investigated  some  of  the  underlying  reasons  for  this,  and  specified  an 
approach  to  promote  its  applicability.  One  of  the  main  problems  is  the  high  cost  of 
courseware  development,  resulting  from  the  long  development  time  and  the  specific 
expertise  required.  We  constructed  a ‘building-block’  approach,  which  enables 
instructors  with  limited  multi-media  expertise  to  rapidly  develop  courseware  for  simple 
learning  objectives,  but  with  the  use  of  advanced  multi-media  technology,.  This  approach 
consists  of  templates  that  incorporate  sound  didactical  principles,  lower  the 
technological  complexity  of  courseware  development,  and  can  be  used  within  modern 
authoring  tools.  Currently  we  are  implementing  the  method  and  are  studying  how  to 
enlarge  the  scope  of  the  method  towards  more  advanced  learning  goals. 


The  use  of  courseware 

Recently,  the  interest  for  the  use  of  courseware  within  many  educational  and  corporate 
organizations  is  increasing  in  a rapid  pace,  as  it  has  the  potential  to  increase  the  efficiency  and 
effectiveness  of  courses  significantly.  Organizations  want  to  develop  courseware  in-house,  as  in  most 
settings  Commercial  Of  The  Shelf  (COTS)  courseware  is  not  specific  enough  and  currently  modem, 
powerful  authoring  tools  are  available.  However,  in  many  cases  such  attempts  have  been  disappointing 
in  terms  of  contributing  to  efficiency  and  effectiveness  of  (the  development  of)  training  programs.  For 
instance,  in  the  last  few  years,  the  Royal  Netherlands  Army  (RNLA)  has  spent  considerable  time  and 
effort  in  developing  courseware  for  its  Training  Centers,  This  courseware  mainly  directed  at  concept 
learning  and  procedural  tasks.  However,  the  application  of  courseware  has  not  yet  lived  up  to  its 
expectations.  It  has  never  been  very  cost-effective  and  (therefore)  organizational  acceptation  or  broad 
adaptation  of  courseware  within  training  programs  is  still  not  accomplished. 

Bottlenecks  in  developing  courseware 

TNO-HF  has  been  asked  to  investigate  this  problem  and  to  formulate,  implement  and  test 
solutions  for  it.  We  found  a number  of  problems  concerning  courseware  development  (Boot  & Van 
Rooij,  1999).  The  development  of  courseware  is  mostly  done  by  Subject  Matter  Experts  (SME’s)  and 
instructors,  using  modem  authoring  tools.  Despite  the  efforts  and  promises  of  these  tools  to  reduce 
development  time  and  the  level  of  required  expertise  (e.g.  Asymetrix,  1998),  it  still  takes  considerable 
time  (and  therefore  costs)  to  produce  courseware.  We  found  that  considerable  programming  knowledge 
is  required  for  anything  more  that  very  simple  courseware.  Also,  much  time  is  required  to  familiarize 
with  the  tools  and  its  many  features  and  the  translation  of  didactical  models  into  courseware  is 
complicated.  In  the  RNLA,  the  SME’s  and  instructors  lack  the  didactical-,  multi-medial-  and 
programming  expertise  to  make  optimal  and  efficient  usage  of  these  tools.  One  can  conclude  that  the 
development  of  anything  more  advanced  than  very  simple  courseware  programs,  remains  costly  and 
will  be  only  cost-effective  for  large  target  groups. 

The  building-block  approach 
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If  one  aims  at  courseware  with  advanced  technological  features  (in  terms  of  multi-media 
usage  and  interactivity),  developed  by  people  with  limited  programming  expertise,  a so-called 
‘building-block’  approach  can  be  a solution.  This  approach  pre-structures  the  development-process  by 
offering  templates  within  authoring  tools.  Templates  consist  of  empty  courseware-structures  to  be  filled 
with  content.  These  structures  incorporate  sound  didactical  principles  and  advanced  technological 
features.  Once  connected  to  each  other  (hence  the  name  building-block  approach)  and  filled  with 
content  (the  learning  materials),  they  constitute  a complete  courseware  program.  We  investigated  some 
build-in  pre-structuring  features  of  modem  authoring  tools  that  make  the  same  claims,  but  they  are  too 
limited  for  our  purposes.  Therefore,  our  templates  should  be  more  advanced  and  comprehensive,  in 
order  to  promote  proper  implementation  of  didactical  models  and  lower  the  technological  complexity. 
As  standardization,  re-usability  and  exchangeability  of  course ware(elements)  is  a key  issue,  these 
templates  should  be  embedded  within  COTS  authoring  tools. 

Specification  and  implementation  of  the  approach 

We  specified  a building  block  approach  for  the  RNLA,  with  templates  for  four  levels  of 
courseware-structure  (Boot  & Van  Rooij,  2000); 

1 . At  the  top  level,  templates  provide  different  kinds  of  navigation  through  a number  of  lessons. 

2.  At  the  second  level,  templates  provide  different  kinds  of  navigation  of  learning  activities 
within  lessons. 

3.  At  the  third  level,  templates  provide  interactivity  within  learning  activities. 

4.  At  the  bottom  level,  templates  provide  the  presentation  of  multi-media  elements  within 
learning  activities. 

Altogether,  they  contribute  with  standardized  screen-elements  to  the  complete  user-interface 
of  the  program.  The  didactical  functionality,  embedded  in  the  templates,  is  directed  at  relative  simple 
learning  goals;  learning  facts,  concepts,  procedures  and  principles.  The  instructional  strategies  that  are 
embedded  are  based  on  theories  like  Gagne’s  Nine  Events  and  Merrill’s  Component  Display  Theory  as 
found  in  Reigeluth  (1983).  Technically,  the  templates  arc  build  using  Knowledge  Objects®  within  the 
authoring  tool  Authorware®  Version  5,  and  are  provided  with  Wizards  for  guiding  the  interaction  with 
the  developers.  For  reasons  of  acceptance  and  commitment  of  the  future  developers,  we  have  involved 
them  during  the  specification  process  of  the  building-block  approach.  We  found  that  it  is  important  and 
difficult  to  balance  well  between  how  much  templates  should  prescribe  and  which  freedom  remains  for 
the  developers. 

Currently,  we  are  in  the  implementation  phase;  external  software  developers  are  building  the 
templates,  after  which  they  will  be  used  by  some  developers  within  the  RNLA.  We  will  start  a number 
of  pilot  projects  to  evaluate  the  application  of  the  approach  and  the  resulting  courseware.  Based  on 
these  results,  the  templates  can  be  improved  and  the  building  block  approach  will  be  applied  on  a wider 
scale. 

Future  developments 

One  important  aspect  of  our  research  is  yet  unresolved;  how  to  specify  a building-block 
approach  that  is  also  directed  to  more  complex  courseware?  During  specifying  the  described  approach, 
we  found  that  finding  a sound  theoretical  basis  is  troublesome.  It  is  a challenge  to  embed  didactical 
models  and  instmctional  principles  into  software-templates  that  are  content-empty.  Our  future  research 
will  be  directed  at  embedding  didactical  models  for  advanced  learning  goals,  using  principles  from 
learning  theories  like  constructivism  and  situated  cognition.  Our  goal  is  to  formulate  concrete 
guidelines  for  the  problem-based  or  case-based  approaches  from  such  theories,  and  implement  and  test 
these  within  an  advanced  version  of  our  current  building-block  approach. 
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Abstract:  A concept  has  been  developed  to  achieve  various  forms  of  collaborative  teleteaching, 
such  as  sharing  ideas  about  teaching,  using  learning  scenarios,  and  engaging  in  joint  teaching 
actions.  Professors  play  a key  role,  since  they  are  instrumental  in  integrating  new  technologies  into 
teaching  An  environment  is  being  developed  to  encapsulate  an  instructional  design  technique  to 
facilitate  sharing  processes.  A database  technology  provides  distributed  services  over  the  Internet. 


From  Teaching  to  Teleteaching 

University  teaching  has  traditionally  been  defined  as  an  on-campus  activity  where  all  players  (1  professor  and  X 
students)  are  in  the  same  place  at  the  same  time,  with  or  without  any  media.  In  fact,  all  teaching  has  been  based  on 
these  parameters.  New  technology  (NT)  brings  many  potential  changes  to  these  parameters,  since  it  creates  an 
opportunity  for  innovative  distributed  teaching  modes.  This  potential  has  increased  the  need  for  instructional 
design,  and  consequently  the  progressive  transformation  of  traditional  teaching  modes  into  instructional  design- 
based  teaching.  Integrating  NT  into  teaching  implies  changing  teaching  practices.  Integration  has  taken  various 
forms  in  higher  education,  such  as:  Web-based  instruction,  online  education  (www.teleleam.ca);  videoconference- 
based  teaching  (Bourdeau  & al.,  1998);  the  virtual  campus  (www.tecfa.unige.ch),  or  integrative  distance  learning 
environments  (www.licefteluq.uquebec.ca).  Satisfaction  levels  in  students  and  faculty  vary  from  enthusiastic  to 
negative.  Faculty  is  being  pressured  to  digitize  material  in  the  form  of  electronic  books,  or  add  electronic 
communication  to  their  classroom  teaching.  However,  teaching  continues  to  be  based  on  the  synchronization  of 
professor  and  students  in  a given  space.  There  is  room  for  integration  modes,  which  could  be  better  anchored  in 
current  practices,  while  simultaneously  fostering  greater  faculty  involvement. 

PETRUS:  The  Concept 

Petrus  is  a new  concept  developed  to  integrate  new  technology  into  university  teaching.  The  fundamental 
characteristic  underlying  the  idea  is  to  consider  professors  the  driving  force  behind  educational  integration,  since 
they  are  the  agents  who  design  teaching  specifically  for  their  own  students.  The  integration  of  new  technology 
derives  its  meaning  (direction  and  semantics)  from  their  teaching  and  personal  interpretation.  The  second  one  is  to 
consider  new  technology  a catalyst  rather  than  a tool:  the  spirit  of  the  Internet  is  all  about  sharing  and  cooperation. 
A good  number  of  professors  are  clearly  prepared  to  integrate  this  spirit  into  their  teaching.  And  lastly,  the  third 
characteristic  of  Petrus  is  to  provide  professors  with  a computerized  environment  where  the  groundwork  for  an 
instmctional  design  technique  has  already  been  laid  through  scenario-building.  Depending  on  their  preferences  or 
needs,  professors  can  use  the  ScenarioBuilder  in  Petms  to:  1)  assemble  dynamic  instmctional  scenarios,  2)  gain 
shared  access  to  comprehensive,  validated  scenarios,  3)  select  individual  components  from  scenarios. 

Implementation  Plan  and  Work  in  Progress 

Implementing  sharing  practices  requires  sharing  processes  and  an  environment  designed  to  support  these  processes. 
Scenario-building  skills  can  be  acquired  through  a high-quality  system  where  instmctional  principles  have  been  built 
into  an  expert  environment.  Professors  can  access,  consult,  edit  or  assemble  scenarios  in  the  most  flexible  ways. 
They  also  have  the  option  of  designing  their  own  teaching  environment  and  learning  interface  for  students. 
Teleteaching  services  naturally  have  to  be  anchored  in  cun-ent  teaching  practices  and  jointly  designed  with 
professors.  The  following  plan  was  designed  for  a course  on  learning  theories:  1)  Collect  and  analyze  a series  of 
syllabi  on  learning  theories;  2)  Extract  and  classify  items  according  to  objectives,  content,  leaming/evaluation 
activities  and  material;  3)  Conduct  a needs  assessment  of  useful  teleteaching  services;  4)  Choose  software 
engineering  solutions;  5)  Develop  and  test  a prototype  for  one  course;  6)  Develop  and  test  a prototype  for  five 
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courses  on  distributed  sites;  7)  Develop  a full  environment  including  rights  and  privileges.  The  plan  is  currently 
under  way  after  a team  of  professors  agreed  to  jointly  design  and  test  the  scenario-builder.  Steps  1 through  5 have 
been  completed,  and  a Web-based  prototype  environment  is  being  developed.  The  environment  will  be  served  by  a 
database  that  dynamically  feeds  information  into  the  system.  First  results  will  provide  information  on  the  quality  and 
level  of  interest  in  collaborative  teleteaching  for  a course  on  learning  theories  in  teacher  education  programs. 

Related  Projects 

To  the  best  of  our  knowledge,  the  concept  that  comes  closest  to  Petrus  is  ARIADNE,  which  stands  for  Alliance  of 
Remote  Instructional  Authoring  Distribution  Networks  for  Europe  (http://ariadne.unil.ch/).  The  project  focuses  on 
the  development  of  tools  and  methodologies  for  producing,  managing  and  re-using  computer-based  educational 
components  and  curricula.  Collaborative  approaches  are  encouraged,  as  is  the  share-and-reuse  principle  (Duval, 
1 999).  Petrus  is  radically  different  in  that  professors  are  at  the  centre  of  all  processes;  they  are  the  agents  through 
whom  everything  passes,  and  both  their  personality  and  teaching  style  are  incorporated  into  the  design.  Petrus  is  not 
looking  to  set  standards,  but  seeks  diversity  instead  in  terms  of  intellectual  perspectives  and  teaching  styles. 
Secondly,  Petrus  contains  a scenario-builder  that  incorporates  teaching  strategies  which  can  also  be  shared.  The 
STAR.Legacy  project  (http://peabody.vanderbilt.edu/ctrs/ltc/)  at  Vanderbilt  University  has  some  features  similar  to 
those  of  Petrus:  the  learning  cycle  resembles  the  Petrus  Scenario-Builder  and,  and  the  “multiple  perspectives” 
activity  in  the  STAR  Legacy  learning  cycle  is  characteristic  of  a team-teaching  situation.  The  working  hypothesis 
underlying  the  Petrus  project  is  as  follows:  the  successful  integration  of  digital  technologies  into  university  teaching 
must  go  through  the  professor  - an  expert  in  a particular  domain  who  is  responsible  for  conveying  information, 
inspiring  and  coaching  students  to  help  them  achieve  their  personal,  academic  and  career  goals.  Professors  need  to 
customize  the  learning  environment  for  their  students,  and  change  it  as  needed.  Professors  would  ultimately  be 
engaging  in  team  teleteaching.  Videoconferencing  and  video  servers  would  provide  a multitude  of  perspectives  for 
students,  thus  enabling  them  to  acquire  greater  discernment  and  critical  skills. 

Sharing,  Collaboration  and  Teleteaching 

In  the  context  of  computer-supported  collaborative  work,  sharing  is  seen  as  a sub-process  of  collaboration,  which  is 
itself  a combination  of  communication  and  coordination.  Coordination  science  defines  coordination  as  managing 
interdependencies,  and  this  concept  can  be  extended  to  the  management  of  interdependencies  among  actors 
(Bourdeau  & Wasson,  1997).  In  the  Petrus  project,  sharing  processes  can  be  identified  together  with  their  associated 
interdependencies.  The  software  environment  would  be  built  around  these  interdependencies  and  provide  the  means 
to  manage  them.  Examples  of  sharing  processes  include:  sharing  ideas,  scenarios,  teaching  or  coaching  activities 
and  lessons  learned.  Mentoring  (at  the  novice  or  expert  level)  can  also  be  considered  a sharing  process.  Examples 
of  sharing  principles  include:  reciprocity,  privileges,  task  distribution,  resource  allocation,  and  so  on.  Software  to 
manage  and  support  sharing  processes  will  be  designed  when  sharing  processes  are  identified  and  validated,  and 
means  for  managing  interdependencies  are  developed.  Ethical  and  legal  issues  will  also  be  taken  into  account. 
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Abstract:  This  paper  describes  the  work  of  the  Carlos  111  University  in  the  project  SEBASTIAN 
(Educational  System  based  on  Internet  Technology).  The  aim  is  to  develop  a tele-educational  tool, 
providing  to  educators/trainers  with  a simple  mechanism  that  makes  possible  the  use  of  the  network 
for  the  broadcast  of  educational  content  and  enabling  the  creation  of  multilingual  courses  based  on 
XML  (Bray,  1998)  and  SMIL  (Stephan,  1998)  technologies. 


Introduction 

As  soon  as  web  technologies  first  became  available,  they  were  adopted  for  educational  purposes.  Hypertext 
technologies  had  already  been  used  in  learning  systems,  and  the  Internet  facilitated  the  overcoming  of  distance. 
Higher  education  is  now  undergoing  structural  changes  in  terms  of  the  composition  of  student  populations,  learning 
paradigms  and  curricula.  As  distance  education  becomes  an  integral  part  of  higher  education,  student  bodies  are 
expanding  to  include  more  non-traditional  students  and  the  contents  of  courses  are  richer  than  before,  and 
incorporate  more  interactivity. 

In  this  new  world,  students  participate  more  actively  in  class  and  collaboration  becomes  a more  important 
component  of  the  learning  process.  These  changes  have  stimulated  the  development  of  new  ways  to  represent 
information  (for  example:  animations,  simulation,  hypertextual  navigation  and  so  on).  In  particular,  multimedia 
content  has  been  introduced  via  the  medium  of  the  WWW.  However,  there  are  problems  associated  with  the  variety 
in  present  day  formats.  Facing  up  to  this  situation.  The  Carlos  III  University  of  Madrid  is  working  on  a project 
named  SEBASTIAN. 

System  Architecture 

The  system  serves  to  integrate  multimedia  learning  experiences  into  the  network  enabling  it  to  be  visualized 
according  to  the  preferences  and  characteristics  of  the  student.  The  courses  generated  are  described  in  EML  2.0 
(Educational  Markup  Language  2.0).  This  language  is  based  on  XML  and  includes  spatial  information  (multimedia 
objects  place,  presentation  size,  etc.),  temporal  information  (duration  of  multimedia  objects,  scenes,  etc.),  and 
information  about  students  profiles  (learning  styles).  The  professor  will  eventually  have  one  tool  with  three 
integrated  functionalities  for  the  creation  of  courses,  see  [Figure  1].  The  functions  are: 

• LG  (Layout  Generation):  Allowing  the  description  of  the  spatial  distribution  of  multimedia  elements  that 
comprise  each  scene  of  the  presentation.  A description  language,  SLML:  Space  Layout  Markup  Language 
(based  on  XML),  has  been  designed  for  this  purpose.  This  functionality  will  provide  the  creation  of 
multimedia  templates.  Initially,  some  predefine  templates  will  exist,  but  the  object  is  that  professor 
generated  its  own  templates. 
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STG  (Synchronous  Text  Generation):  The  files  generated  are  based  on  a new  language  STML: 
Synchronization  Text  Markup  Language,  and  provide  textual  content  in  several  languages.  These  files 
contain  synchronization  labels  which  synchronize  the  text  with  audio  or  video,  as  in  a karaoke  session. 
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may  bo  presented  to  different  students. 


Figure  1:  Contents  generation,  distribution  and  visualization  process. 


• CG  (Course  Generator):  Integrates  multimedia  files  and  files  generated  by  the  two  previous  applications. 
The  result  is  a presentation  composed  of  a sequence  of  "abstract  scenes".  Each  scene  is  a compound  of  the 
different  presentations  that  may  be  presented  to  different  students.  The  view  presented  to  each  student 
depends  on  their  preferences  and  pedagogic  profile.  These  profiles  are  generated  automatically  in 
accordance  with  the  Learning  Style  Model  (Richard  Felder,  1996),  the  PAPI  (Public  and  Private 
Information,  2000)  and  the  LOM  (learning  Object  Metadata  1 999)  specifications. 

Acknowledgments 

The  work  reported  in  this  paper  has  been  partially  funded  by  the  project  SEBASTIAN  07T/00 15/ 1997  of  the 

Spanish  CAM  and  the  project  fNFOMEDIA  CICYT  TEL  1999-0207.  We  wish  to  acknowledge  fruitful  discussions 

with  our  colleagues  in  the  Department  of  Telematics  Engineering  in  the  Carlos  III  University  of  Madrid. 

References 

- Bray,  T.  (1998).  Extensible  Markup  Language  (XML).  W3C  REC-xml- 199802 10 

- Stephan,  L (1998).  Synchronized  Multimedia  Integration  Language  I.O  (SMIL).  W3C  REC-smil-1 998061 5 

- Felder,  R.  (1996).  Learning  Styles.  North  California  State  University 

- Hodgins,  W (2000).  Learning  Object  Metadata.  IEEE  P1484.12/D4.1 

- Farance,  F (1997).  Public  and  Private  Information.  IEEE  PI 484.2 


BEST  COPY  AVAILABLE 


1608 


The  Construction  and  Evaluation  of  an  Internet-Based  Knowledge 
Integration  System  (IKIS)  For  a CAI  Graduate  Course 


Mei  Chen  & Tanya  Scott 
Department  of  Education 
Concordia  University 
1455  Maisonneuve  Boulevard  W. 

Montreal,  Canada,  H3G  IMS 
MeiChen@vax2.concordia.ca 
TScott@Education.concordia.ca 

Abstract:  This  paper  describes  an  on-going  project  that  involves  the  constmction  and  evaluation  of 
an  Internet-based  Knowledge  Integration  System  (IKJS)  developed  to  support  a Computer-assisted 
Instmction  (CAI)  graduate  course.  The  IKIS  integrates  cognitive  theories  of  learning  and 
instmction,  computer  networking  technologies,  and  face-to-face  classroom  teaching  in  order  to  (a) 
help  students  build  coherent  knowledge  representations  of  a great  variety  of  knowledge  and  skills 
that  are  required  to  design  and  produce  instmctional  software  prototypes  and,  (b)  create  a 
community  of  practice  to  aid  students  in  making  the  decisions,  concerning  the  design  and 
production.  Our  discussion  will  be  focused  on  two  important  system  features  of  the  IKIS  (i.e.,  the 
information  architecture  and  communication  facilities)  that  enable  us  to  achieve  such  goals.  A plan 
for  evaluating  the  IKIS  will  also  be  briefly  described. 

In  this  paper,  we  will  describe  an  on-going  project  that  involves  the  construction  and  evaluation  of  an  Internet- 
based  Knowledge  Integration  System  (IKIS)  developed  to  support  a Computer-assisted  Instruction  (CAI)  graduate 
course  in  which  students  will  develop  knowledge  and  skills  for  designing  and  producing  instructional  software 
prototypes.  Understanding  of  various  design  concepts  and  approaches,  specification  techniques,  authoring  tools,  and 
development  methods  is  emphasized  within  this  course.  Whereas  the  instructor  of  the  course  provides  face-to-face 
lectures,  demonstrations,  discussions,  and  laboratory  work,  the  complex  nature  of  the  subject  can  still  pose  a 
challenge  for  students  to  assimilate  and  integrate  such  knowledge.  Furthermore,  traditional  classroom  instruction 
remains  an  inadequate  method  in  providing  timely  instruction  to  each  individual  student.  The  IKIS  for  (CAI)  was 
constructed  to  (a)  provide  students  with  a rich,  well-structured  knowledge  base  in  order  to  ensure  their  active 
assimilation  and  integration  of  a variety  of  knowledge  and  skills  to  be  learned,  and  (b)  create  a community  of 
practice  to  aid  students  in  making  the  decisions  concerning  the  design  and  production  of  software  prototypes. 

The  Design  of  the  IKIS 

The  IKIS  integrates  cognitive  theories  of  learning  and  instruction,  computer  networking  technologies,  and 
face-to-face  classroom  teaching  to  enable  students  to  use  the  well-structured  knowledge  base  and  diverse  tutorial 
resources  to  build  coherent  knowledge  representations  of  the  subject-matter  knowledge  to  be  learned.  The  design  of 
the  IKIS  is  based  on  the  cognitive  theories  of  text  comprehension  that  emphasizes  the  importance  of  structured 
knowledge  presentations  in  the  construction  of  coherent  knowledge  representations  (Kintsch,  1988;  1998;  Mayer, 
1999;  1997),  and  on  situated  theory  that  views  social  interaction  and  collaboration  as  crucial  factors  in  the 
development  of  scientific  understanding  (Lave  & Wenger,  1991;  Resnick,  Levine,  & Teasley,  1991).  In  addition,  the 
cognitive  apprenticeship  model  proposed  by  Collins,  Brown,  and  Newman  (1989)  also  informed  the  design  of  the 
IKIS.  Five  principles  are  articulated  to  guide  the  design  of  the  IKIS:  (a)  providing  a rich  knowledge  base  to  reflect 
expertise  in  the  domain  of  instructional  software  design;  (b)  providing  a well-organized  and  transparent  knowledge 
structure  to  help  students  construct  a coherent  conceptual  representation  of  the  subject-matter  knowledge,  (c) 
employing  an  effective  model  of  instruction,  (d)  building  a community  of  expert  practice  and  providing  multiple 
perspectives  towards  the  solutions  and,  (e)  facilitating  collaborative  learning  among  students.  The  theoretical 
foundation  and  design  principles  of  the  IKIS  have  been  described  in  details  elsewhere  (Chen,  1999;  Chen,  2000).  In 
this  paper,  our  discussion  will  be  focused  on  two  important  system  features  of  the  IKJS,  namely  the  information 
architecture  and  communication  facilities  that  reflect  the  principles  described  above.  In  addition,  an  evaluation  that 
we  will  conduct  in  the  near  future  will  also  be  proposed. 

The  information  architecture  incorporated  into  the  IKIS 

The  CAI  graduate  course  is  focused  on  the  user-centered  design  for  Computer-supported  Collaborative  Learning 
(CSCL).  A series  of  topics  were  identified  as  important  for  successful  design  and  implementation  of  instructional 
software  prototypes.  In  order  to  help  students  construct  coherent  knowledge  representations,  the  IKIS  incorporates 
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an  information  architecture  (i.e.,  a hierarchical  organizational  scheme)  to  represent  two  “layers”  of  the  knowledge  to 
be  learned:  the  first  layer  consists  of  various  topics,  which  range  from  learning  theories  to  web  authoring  techniques 
whereas  the  second  layer  is  a breakdown  of  knowledge  on  each  topic  which  includes  concept,  rationale,  goals, 
methods,  example,  related  issues,  trouble  shooting  and  tips,  and  reflective  questions  and  exercises.  Such  a 
knowledge  base  is  expected  to  enable  the  students  to  answer  questions  concerning  what,  why,  how,  to  evoke 
reflective  thinking,  and  ultimately,  to  create  a rich  context  for  students  to  engage  in  productive  group  discussions  on 
the  topics  via  computer-mediated  conferencing.  Furthermore,  some  Internet  resources  and  lab  tutorials  that  are 
relevant  to  specific  topics  are  also  provided  to  extend  the  knowledge  base  of  the  IKIS.  Students  can  also  access  a 
perquisite-course  website  to  acquire  background  knowledge  if  they  wish.  The  information  architecture  of  the  IKIS 
enables  the  students  to  learn  the  topics  in  a logical  manner,  meanwhile  this  architecture  is  flexible  enough  to  allow 
each  individual  to  take  a different  navigation  path  based  on  his  or  her  own  selection.  There  is  a navigation  bar 
located  on  the  top  of  the  screen  that  allows  students  to  maneuver  through  the  course  topics  and  breakdown  by 
following  either  a pre-determined  sequence  or  a selected  path.  In  addition,  a bottom  navigation  bar  is  available  all 
time  to  enable  the  students  to  access  a computer-mediated  conferencing  facility,  newsgroup  discussions,  and  e-mail 
addresses  in  the  context  of  reviewing  the  course  topics  and  breakdown.  This  bottom  navigation  bar  also  provides 
accesses  to  the  resources  and  lab  tutorials  outside  of  the  instructional  content  that  is  defined  in  the  IKIS.  An  on-line 
help,  site  map,  and  search  facility  are  also  provided  by  the  bottom  navigation  bar  to  enable  students  to  navigate 
efficiently  through  the  site. 

The  communication  facilities  utilized  in  the  IKIS 

There  are  numerous  decisions  that  one  must  make  in  the  processes  of  designing  and  producing  instructional  software 
prototypes.  No  matter  how  rich  the  knowledge  base  is  and  how  well  such  knowledge  base  is  structured,  students 
may  still  have  unanswered  questions.  The  purpose  of  proving  the  accesses  to  a computer-mediated  conferencing 
facility,  newsgroup  discussions,  and  e-mail  addresses  is  to  enable  students  to  discuss  such  unanswered  questions 
with  their  peers  or  experts  outside  of  their  physical  environments.  Three  communication  channels  are  established  to 
support  students*  knowledge  construction  and  decision-making:  (a)  a channel  for  integrating  with  the  peers:  the 
FirstClass^^  software  program  is  used  to  provide  computer-mediated  conferencing  and  chat  facilities  to  facilitate  the 
discussions  among  students.  Each  student  is  required  to  provide  answers  to  the  reflective  questions  that  are  proposed 
at  the  end  of  each  topic.  Students  are  also  encouraged  to  bring  up  issues  that  they  wish  to  clarify  and  respond  to  each 
other’s  questions.  Students  can  also  the  FirstClass^^  to  exchange  electronic  files  that  they  produced  for  their  group 
work.  The  instructor  and  teaching  assistant  will  monitor  the  discussion  and  provide  directions  and  clarifications  if 
needed;  (b)  a channel  for  participating  in  the  communities  of practice:  The  IKIS  provides  students  with  access  to  the 
Internet  newsgroups  that  are  most  relevant  to  the  topics  discussed  in  the  course.  These  newgroups  are  organized  in  a 
way  that  is  sensitive  to  the  course  topics  so  that  the  students  can  be  exposed  to  multiple  perspectives  on  a given 
issue.  Students  will  also  have  the  opportunities  to  share  their  experience  and  expertise  with  people  who  face  similar 
problems  and  understand  the  concerns  of  the  practitioners  in  the  community;  (c)  a channel  for  communicating  with 
experts:  In  addition  to  the  communication  with  the  instructor  of  the  course,  students  can  ask  questions  to  some 
design  experts  who  will  serve  as  the  mentors  during  the  course  of  the  semester.  These  experts  will  help  the  students 
diagnose  some  "unsolvable"  problems  and  provide  suggestions  via  e-mail.  In  addition,  a list  of  e-mail  addresses  of 
the  experts  in  the  fields  relevant  to  CAI  design  will  also  be  provided  to  allow  students  to  contact  these  experts.  The 
communication  with  these  experts  may  provide  students  with  insights  into  how  experts  think  and  practice.  Thus,  The 
communication  facilities  offered  by  the  IKIS  will  break  the  boundary  of  time  and  location  that  is  inherited  in 
traditional  teaching. 

The  Evaluation  of  the  IKIS 

The  IKIS  will  be  evaluated  both  formatively  and  summatively.  In  the  formative  evaluation  phase,  a few 
representative  students  will  be  asked  to  “walk  through”  the  environment  and  provide  “think  aloud”  protocols  for 
making  diagnosis  of  the  system  in  terms  of  its  usability  and  leamability.  Following  the  necessary  modifications,  a 
summative  evaluation  will  be  conducted  to  determine  the  effectiveness  of  the  IKIS.  Two  groups  of  students  will  be 
compared:  one  that  use  the  IKIS  and  another  that  use  a print-based  version  of  the  same  material.  An  attitude 
questionnaire  will  be  administered  before  and  after  the  course  to  determine  the  changes  in  student  attitudes.  In 
addition,  students’  on-line  discussions  and  course  work  will  also  be  analyzed  to  determine  the  levels  of  expertise 
developed.  The  ultimate  goal  of  the  project  presented  here  is  to  understand  the  ways  that  computer  technologies  can 
be  used  to  enhance  student  learning  of  complex  subject-matter  knowledge  and  to  develop  a model  of  successfully 
integrating  technology  into  the  classroom  teaching. 
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1.  Introduction 

Although  the  increasing  sophistication  of  computer-based  learning  systems  has  enabled  gains  to  be  made  in 
tenns  of  their  pedagogical  effectiveness,  an  undesirable  consequence  of  this  sophistication  has  been  a significant 
increase  in  development  cost.  The  high  cost  of  these  systems  has  been  identified  as  a principal  barrier  to  their 
widespread  deployment.  Increased  complexity  in  learning  systems  has  also  led  to  lengthier  development  cycles, 
which  has  constrained  the  rate  of  design  evolution  and  innovation.  As  a result,  there  is  a pressing  need  for 
instructional  designers  to  focus  on  the  efficiency  of  their  development  processes  as  well  as  the  effectiveness  of  the 
outputs  that  are  produced  by  them. 

A possible  strategy  for  reducing  the  time  and  cost  of  system  development  is  to  create  a context  wherein 
whole  systems  would  not  have  to  be  designed  largely  from  scratch.  A key  aspect  of  such  a strategy  could  be  the 
establishment  of  cross-organizational  standards  that  would  allow  developers  to  reuse  content  components  from  other 
systems,  both  from  within  their  own  organization  and  from  external  sources.  In  essence,  this  would  represent  an 
attempt  to  leverage  the  productivity  gains  from  reusability  that  have  been  achieved  in  software  engineering  into  the 
domain  of  instructional  content  development.  Over  the  past  year,  coordinated  proposals  have  rapidly  emerged  from 
several  bodies  (e.g..  Advanced  Distributed  Learning  Initiative,  2000;  Anderson,  T.,  McArthur,  D.,  Griffen,  S.  & 
Wason,  T.,  1999)  as  candidate  standards  for  this  purpose.  As  these  proposals  are  vetted  through  their  initial  trials  in 
actual  learning  systems,  it  will  be  important  to  observe  how  standards  for  content  reusability  affect  instructional 
content  development  methodologies.  Will  processes  primarily  developed  in  the  context  of  authoring  content  in 
small-scale  closed  systems  evolve  smoothly  into  processes  for  architecting  large-scale  distributed  learning 
environments  where  the  number  of  distinct  content  components  may  be  in  the  hundreds  or  thousands?  Conversely,  if 
new  processes  must  be  developed,  which  ones  will  be  required,  and  on  what  basis  do  we  start  their  development? 

A fundamental  issue  that  needs  to  be  addressed  in  this  regard  is  the  broad  function  of  instructional  design 
(ID)  in  the  development  of  systems  that  have  a substantial  information  technology  (IT)  component.  Where  does  ID 
stop  and  IT  start?  Consider  the  development  of  a set  of  interactive  practice  exercises  that  include  variable  remedial 
feedback  based  on  learner  response  patterns.  A design  decision  could  be  made  to  use  a database-driven  approach 
wherein  feedback  is  assembled  from  low-level  content  components  that  are  sourced  from  a database.  If  these  low- 
level  components  include  subject-matter  terms  and  their  definitions,  the  same  database  might  be  used  to  drive  a 
glossary  function  in  the  learning  environment,  serving  the  objective  of  content  reusability.  In  designing  such  a 
system,  we  would  likely  not  expect  an  IT  professional  to  author  the  subject-matter  definitions  that  lie  within  the 
database.  Just  as  we  would  not  expect  the  ID  professional  to  design  the  software  that  allows  the  database  to 
communicate  within  a client-server  environment.  Appropriate  allocation  of  responsibility  for  many  other  design 
issues,  however,  is  not  as  clear-cut.  For.  example,  is  one  of  these  professionals  primarily  responsible  for 
decomposing  the  feedback  into  suitable  lower-level  components,  or  is  this  a Joint  responsibility  - and,  if  so,  how  do 
these  professionals  collaborate  on  this  task  effectively?  Of  specific  concern  are  the  pedagogical  implications  of 
successful  technical  convergence  (e.g.,  the  use  of  database-driven  content  systems,  or  the  adoption  of  XML  as  a 
markup  language):  does  this  form  of  standardization  support  research  on  and  development  of  pedagogical  strategy, 
or  does  it  sacrifice  flexibility  at  this  level  to  achieve  the  goal  of  technical  convergence?  The  absence  of  a clear 
articulation  of  the  interface  between  pedagogy  and  technology  in  practical  terms  may  prove  problematic. 

2.  Common  architectural  issues  in  the  design  of  computer-based  learning  systems  using 
modular  content 


To  begin  assessing  the  viability  of  current  content  development  process  models  in  the  context  of  large-scale 
distributed  learning  environments,  we  can  examine  common  categories  of  design  decisions  that  occur  in  smaller- 
scale  development  projects  and  assess  the  possible  ramifications  of  a significant  change  in  scale  on  them: 

• Modularity:  What  is  a module,  in  terms  of  instructional  content?  What  are  the  quality  criteria  for  the 
definition  of  ‘good’  modules?  Do  these  criteria  change  as  scale  increases?  How  do  we  use  partitions 
between  modules  to  manage  risk  in  content  development?  For  example,  how  do  we  partition  a system  in 
order  to  increase  our  ability  to  test  numerous  content  components  as  they  are  being  developed? 

• Communication  and  information  flow:  What  types  of  information  get  communicated  between  modules  in  a 
single  system?  Where  are  the  architectural  boundaries  in  an  open  content  system?  How  does  the 
information  flow  in  a learning  environment  instantiate  a pedagogical  theory  of  control? 

■ Object-  and  component-based  approaches  to  the  development  of  instructional  materials:  How  do  we 
determine  the  right  grain  sizes  for  content  development?  What  is  the  relationship  between  grain  sizes  for 
authoring  and  the  units  of  analysis  employed  during  front-end  analysis?  How  can  the  choice  of  content 
units  facilitate  collaboration  with  software  developers  during  design  and  production? 

■ Reusability:  How  is  the  design  of  a single  reusable  content  component  different  from  the  design  of  an 
instructional  product  out  of  reusable  components?  What  parts  of  a learning  environment  can  be  reused? 
How  do  we  move  from  ad-hoc,  opportunistic  reuse  to  systematic  re-use?  What  are  the  tradeoffs  involved? 
Can  there  be  too  much  reuse? 

• Interoperability:  At  what  levels  is  it  feasible  to  make  content  interoperable?  What  goals  are  achieved  by 
making  content  interoperable?  What  is  the  relationship  between  reusability  and  interoperability? 

• System  maintenance  and  evolution:  How  do  we  design  system  architecture  so  that  it  can  survive,  and 
indeed  facilitate,  bottom-up  evolution  of  the  system?  What  kind  of  evaluation  data  should  the  system 
collect,  and  how?  How  do  we  document  system  architecture  so  that  it  is  understandable  by  those  who  will 
be  responsible  for  its  maintenance  and  evolution? 

• Coordination:  What  happens  when  we  move  from  managing  individual  content  development  projects  in 
relative  isolation  to  managing  integrated  content  development  programs?  Which  design  assumptions  are 
challenged  when  discrete  learning  products  become  parts  of  continuous  learning  systems? 

3.  The  identification  and  categorization  of  design  decisions 

When  a design  domain  is  relatively  stable,  the  explicit  articulation  of  design  decisions  is  less  important, 
since  decisions  and  their  consequences  are  reasonably  predictable  based  on  past  experience.  Process-level  issues 
can  generally  be  resolved  on  a case-by-case  basis  in  such  circumstances.  As  the  transition  is  made  to  developing 
instructional  content  components  for  use  in  distributed  learning  systems,  instructional  designers  may  face  a period 
of  domain  instability  where  novel  and  complex  interactions  force  many  tradeoffs  to  be  made  unwittingly. 
Reassessing  and  evolving  processes  in  light  of  these  challenges  will  require  an  understanding  of  the  new  design 
space  that  instructional  content  developers  are  working  within.  In  part,  the  opportunities  and  constraints  associated 
with  this  new  context  will  have  to  emerge  through  hard  experience.  A viable  approach  for  articulating  the 
boundaries  and  substance  of  this  space  in  advance,  however,  is  to  identify  and  categorize  the  macro-  and  micro- 
decisions that  are  made  in  developing  small-scale  modular  content,  and  examine  these  decisions  in  terms  of  their 
scalability  to  larger,  more  integrated  systems.  This  is  the  subject  of  my  current  work-in-progress.  By  examining  the 
development  process  from  the  perspective  of  decisions  that  are  made  during  design,  a top-down,  principle-based 
approach  to  learning  system  architecture  can  be  combined  with  a bottom-up  awareness  of  the  individual  decisions 
designers  make  on  actual  projects.  This  is  a key  stepping-stone  in  the  analysis  of  the  cohesiveness  and 
comprehensiveness  of  a given  set  of  instructional  development  processes.  The  design  decisions  can  also  serve  to 
frame,  in  tangible  terms,  the  relationship  between  a given  pedagogical  strategy  and  its  technical  implementation. 
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Abstract:  Concerned  with  the  relationship  between  interaction  and  learning,  this 
paper  aims  at  exploring  relations  between  interactive  flows  in  an  EFL  course 
offered  on  the  Internet  and  language  learning.  The  course  was  produced  and  is 
run  by  the  research  team  of  EDULANG,  an  ongoing  research  project  in 
language  learning  on  the  Web  and  teacher  development  for  the  Web.  In  this 
research  context,  interactive  flows  are  being  used  to  record  and  analyze  the  rate 
and  type  of  interaction  in  the  course,  identify  units,  tasks  and  course  spaces  that 
do  not  promote  expected  levels  of  interaction  and  plan  and  promote 
improvement  in  course  structure,  course  content  and  teaching  strategies. 

Aims  and  Context 

This  paper  derives  from  ongoing  research  in  the  context  of  EDULANG,  a team  project  that  aims  at  investigating 
multiple  aspects  of  language  learning  and  teaching  in  courses  on  the  Internet.  The  paper  is  more  specifically 
concerned  with  the  relationship  between  interaction  and  learning  and  aims  at  exploring  relations  between  interactive 
flows  and  language  learning.  An  interactive  flow  is  here  defined  as  a longitudinal  profile  depicting  amount  (no.  of 
messages  or  interactive  turns)  and  direction  (sender  and  recipient)  of  synchronous  and/or  asynchronous 
participation  in  an  online  course.  The  context  of  the  research  is  an  EFL  course  offered  on  the  Internet 
(www.cogeae.uol.com/sal).  The  course  was  produced  and  is  run  by  the  research  team  of  EDULANG.  Aimed  at 
adults  with  a basic  command  of  English,  its  general  social  aim  is  to  offer  students  the  opportunity  to  make  new 
acquaintances  on  the  Internet.  This  general  aim  unfolds  into  specific  social  and  language-specific,  communicative 
functions.  The  course  officially  lasts  8 weeks  but  can  in  fact  reach  12  to  13  weeks.  The  interactive  spaces  make  it 
possible  for  participants  to  a)  communicate  through  messages  posted  to  a virtual  Bulletin  Board  (BB)  subdivided 
into  three  forums  (Classroom,  Cafe  and  Technical  Support),  b)  display  (and  re-edit)  their  interactive  production  to 
course  peers  in  a Production  Panel,  c)  send  progress  tasks  and  tutorial  exercises  to  the  teacher  through  mail-to 
devices  on  the  course  pages  and  d)  participate  in  online  Chats.  All  communication  is  carried  on  in  English,  except 
for  messages  left  in  the  Technical  Support  forum.  As  an  essential  element  of  learning  and  knowledge  construction, 
interaction  must  be  evaluated  for  type,  effect  and  efficiency  in  courses  on  the  Internet.  In  this  research  context, 
interactive  flows  are  being  used  to  record  and  analyse  the  rate  and  type  of  interaction  in  the  course,  identify  units, 
tasks  and  course  spaces  that  do  not  promote  expected  levels  of  interaction  and  plan  and  promote  improvement  in 
course  structure,  course  content  and  teaching  strategies. 

Method 

Data  was  collected  from  indicators  about  sender,  recipient,  date  and  subject  of  6 groups  of  students  and  their 
respective  teachers,  in  1998  and  1999,  and  include  a)  messages  posted  in  the  Bulletin  Board,  b)  messages  with 
ongoing  tasks  sent  to  the  teacher,  c)  texts  displayed  in  the  Production  Panel  and  d)  participation  in  online  chat 
sessions.  All  indicators  were  available  within  the  course  system. 

Some  Findings 

Results  were  organized  in  order  to  reveal  graphic  flows  of  interaction  involving  students  and  peers,  students  and 
teacher,  teacher  and  students.  These  flows  reveal  trends  within  each  group  as  well  as  across  groups.  Interpreted  in 
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the  light  of  the  course  context,  the  graphic  flows  below  have  offered  support  for  pedagogical  and  managerial 
decisions,  as  well  as  for  changes  in  design.  Figure  1 indicates  that  in  this  course,  the  amount  of  student  talk  tends  to 
be  equivalent  to  or  higher  than  teacher  talk.  This  may  mean  that  in  an  interactive  internet  course,  students  engage  in 
more  interaction  than  students  in  traditional  face-to-face  EFL  courses.  Figure  2 shows  that  different  groups  present 
different  types  of  flows,  either  with  a focus  on  teacher-student  and  student-teacher  interaction  or  with  a focus  on 
communication  among  students.  In  groups  where  the  teacher  talks  less,  interaction  among  students  tends  to  be 
higher.  In  this  sense,  the  amount  of  teacher  interaction  with  students  influences  the  amount  of  students’  interaction 
with  their  peers.  Additionally,  Figure  2 shows  that  groups  A,  B,  C and  D (taught  in  1 998)  communicate  a lot  less  in 
the  BB  than  groups  C and  D (taught  in  1999).  This  suggests  a development  of  familiarity  and  comfort  with 
educational  contexts  on  the  Web  for  both  students  and  teachers.  However,  relatively  close  averages  of  participation 
in  the  1998  groups  (11.07,  5.53,  8.45  and  1T95)  and  equivalent  participation  in  the  1999  groups  (28.28  and  28.38), 
suggest  that  students  seem  to  have  a quantitative  communicative  quota ^ that  they  strive  to  use,  regardless  of  the 
interlocutors  they  choose  to  have  (the  teacher  or  a peer).  Figure  3 allows  an  interpretation  of  the  interactive  flows  in 
the  BB  in  the  light  of  course  content  distribution  (weeks  1-3  for  unit  1,  weeks  4-5  for  unit  2 and  weeks  6-8  for  unit 
3).  A higher  state  of  motivation  seems  to  be  associated  to  specific  phases  of  the  course  when  participants  are  all 
getting  to  know  each  other  (the  beginning)  or  students  are  trying  to  close  their  participation  successfully  (the  official 
end,  the  8'^  week).  Mid  course  activities  have  not  been  able  to  sustain  adequate  levels  of  interaction  (notice  the 
relative  drop  both  in  the  98  and  99  groups)  and  are  now  being  thoroughly  reformulated.  Figures  4 and  5 offer  data 
for  interesting  observations.  Groups  A and  B seem  to  reinforce  the  quantitative  communicative  quota  hypothesis, 
with  task  completion  and  BB  message  flows  in  a complementary  relationship  (lower  averages  in  one  flow  type 
complement  higher  averages  in  the  other).  Groups  C and  D,  on  the  other  hand,  present  opposite  results  in  both  types 
of  flows  and  suggest  that  teacher  management  has  a big  influence  on  task  completion  and  student  participation  in  the 
BB.  Figure  6 reveals  that  among  students  who  completed  at  least  50%  of  tasks  there  are  more  poor  “chatters”  (23) 
than  good  “chatters”  (13).  However,  among  the  best  “chatters”  (more  than  30  turns)  were  the  best  task  completion 
rates.  This  suggests  that  completing  tasks  (a  traditional  activity)  is  still  favored  over  other  web-typical  types  of 
participation  and  that  an  intensive  participation  in  chat  sessions  is  usually  associated  to  intensive  task  completion. 


Fig  1:  Interactive  Flows  in  the  Bulletin  Board  between 
Teachers  and  Students  in  6 Groups 


□ taach/slud 
Slttiid/all 


Figure  1 : BB  Messages  between  Teachers  and  Students 


Figure  2:  BB  - teachers/students  and  student/student 


Fig  3:  Interactive  Flows  of  6 groups  in  the  Buiietin  Board 
during  1 5 weeks 
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Figure  3:  Flows  of  6 groups  in  the  BB  during  15  weeks 


Figure  4:  Task  Completion  Flows  in  4 groups  (averages) 
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Abstract:  This  paper  describes  a study  that  is  being  developed  to  explore  online  collaboration  between 
undergraduate  journalism  students  at  universities  in  Australia,  Canada,  New  Zealand,  the  United 
Kingdom  and  the  United  States.  The  premise  of  the  study  is  that  by  bringing  students,  instructors  and 
experts  together  to  examine  different  perspectives  of  international  news  stories,  learners  will  develop  a 
stronger  understanding  of  stories  that  get  international  coverage.  The  goal  of  the  research  is  to  develop  a 
consensual  and  collaborative  module  that  explores  international  perspectives  on  global  news  and  to 
present  this  module,  online  ,to  teams  of  students  at  the  partner  universities.  Data  will  be  gathered  from 
online  interviews  and  questionnaires  given  before  and  after  the  module. 


Introduction 

In  the  past  decade  there  has  been  a growing  amount  of  research  in  journals,  books  and 
conference  papers  in  the  area  of  online  learning.  Online  learning,  for  the  purpose  of  this  research, 
refers  to  study  that  is  done  using  computer  technologies  with  the  learner  at  the  same  location  or  at  a 
distance  from  the  instructor. 

Many  established  universities,  such  as  Stanford  in  the  United  States,  Deakin  in  Australia, 
University  of  Twente  in  the  Netherlands,  offer  online  classes  to  students  who  live  at  a distance  from 
campus,  to  learners  studying  part-time  and  to  professionals  engaged  in  life  long  learning.  These  classes 
may  have  an  international  reach  but  are  run  and  controlled  entirely  by  individual  universities.  The 
Open  University  in  the  United  Kingdom  and  the  University  of  Phoenix  in  the  United  States  are  two 
examples  of  universities  that  are  completely  online.  All  students  study  from  a distance  but  students  are 
still  studying  at  one  university  and  working  within  the  structure  and  policies  of  that  system. 

Most  of  the  online  classes  described  above,  involve  students  working  alone  online  with  some 
support  from  instructors  or  tutors  via  e-mail  but  with  little  collaboration  amongst  students  themselves. 
Collaborative  learning,  as  defined  for  this  study  is:  learning  in  an  educational  program  created  by  a 
group  of  partner  universities  and  in  which  students  and  instructors  get  together  to  examine  course 
content  issues.  This  sort  of  interactive,  online  learning  is  becoming  more  widely  practised.  Medical 
schools  throughout  Germany  (Friedl,  99)  share  online  resources  to  study  cardiac  surgery.  An  online 
training  for  trainers  course  involved  learners  in  22  European  countries  (Leino,  99).  Organizations  such 
as  the  United  Nations  are  using  online  modules  to  give  professionals  around  the  world  a chance  to 
study  copyright  law  (Durant,  99). 

So  far,  in  this  researcher’s  literature  search,  most  examples  of  online  collaboration  involve 
engineering,  science,  medical  and  business  programs  and  most  of  the  collaborations  are  developed  at 
one  university.  Telg  (96)  noted  that  in  the  United  States  “few  collaborative  distance  education  efforts 
have  taken  place  between  American  universities  and  universities  in  other  countries”. 

In  this  researcher’s  area  of  interest,  broadcast  journalism,  there  is  little  documentation  in  the 
literature  of  any  programs  using  online  collaboration  as  a learning  tool. 


Study 


It  is  becoming  easier  and  easier  for  journalists  to  quickly  gather  information  from  any  part  of 
the  world  and  disseminate  it  to  any  other  part  of  the  world  but  McGregor  (98)  asks  is  “the  news  is 
better  reported  with  more  understanding  as  a result  of  the  new  technologies”.  We  are  sending  young 
broadcasters  into  a world  of  instant  information,  instant  analysis  and  instant  conclusions  about  global 
news  issues.  One  of  the  questions  raised  by  this  study  is:  do  those  young  journalists,  and  the 
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experienced  story  tellers  for  that  matter,  have  the  enough  understanding  of  global  issues  to  provide 
clear,  non-biased  reports? 

The  premise  of  this  study  is  that  by  linking  students,  instructors  and  broadcast  experts  in 
online  discussion  and  by  working  in  international  teams  on  assignments  and  presentations  concerning 
global  news,  the  learners  will  develop  a stronger  understanding  about  stories  that  gain  international 
media  attention.  The  researcher  is  working  in  consensus  with  partner  instructors  in  Australia,  Canada, 
New  Zealand,  the  United  Kingdom  and  the  United  States  to  develop  a collaborative  module  that  will 
allow  students  to  examine  and  discuss  international  perspectives  on  global  news  stories.  Students  will 
gather  at  the  module  web  site  that  includes:  structured  and  unstructured  forums  for  discussion  and 
debate,  a location  for  students,  instructors  and  industry  professionals  to  interact,  links  to  broadcast 
networks  worldwide  plus  visual  and  written  content  that  takes  the  learner  through  the  module  and  its 
assignments  step  by  step. 

Data  will  be  gathered  using  a quiz  to  examine  students’  knowledge  of  global  news  issues, 
before  and  after  the  module  is  run.  Learners  and  instructors  will  respond  to  questionnaires  online 
before  and  after  the  module  and  will  participate  in  structured  and  informal  taped  interviews  to  elicit 
personal  and  anecdotal  information.  At  least  ten  students  from  each  partner  university  will  participate 
in  the  first  run  of  the  module  in  April  2001. 

The  result  of  this  study  will  be  to  establish  an  international  circle  bringing  learners  with 
different  backgrounds  and  understandings  of  international  news,  into  an  ongoing  discussion.  The 
timing  is  right  to  examine  how  online  collaborative  learning  functions  within  the  untested  area  of 
broadcast  journalism  and  to  determine  if  there  is  a benefit  for  students  in  using  this  new 
teaching/leaming  method.  This  study  aims  to  bring  issues  of  online  learner  effectiveness  to  light  to 
provoke  further  discussion  research  and  practise  in  this  area. 


References 

Bonk,  C.J.  (1999).  Breakout  from  Learner  Issues,  InternationalJournal  of  Educational 
Telecommunication,  5 (4),  387-410 

Durant,  F.T.  (1999)  WlPO  Academy’s  Online  Courses  bring  Intellectual  Property  to  Worldwide 
Audiences,  Online  EDUCA  Berlin,  5^*^  International  Conference  on  Technology  Supported  learning 

Ewing,  J.M.,  Dowling,  J.D.,  Doutts,  N.,  (1999).  Learning  Using  the  World  Wide  Web:  A Collaborative 
Learning  Event,  Journal  of  educational  Multimedia  and  Hypermedia,  8 (1),  3 - 22 

Friedl,  R.  (1999  Teleteaching  in  Cardiac  Surgery:  An  Innovative  concept.  Conference  Paper,  Online 
EDUCA  Berlin,  5^^  International  Conference  on  Technology  Supported  learning. 

Leino,  A.  (1999)  Virtual  United;  Online  courses  to  22  Countries  by  an  international  team.  Online 
EDUCA  Berlin,  International  Conference  on  Technology  Supported  learning 

McGregor,  B.  (1998)  Live  Direct  and  Biased:  Making  television  news  in  the  Satellite  Age,  Arnold, 
London 

Oliver,  R.,  Omari,  A.  (1998)  Exploring  Student  Interaction  in  Collaborative  world  wide  Web 
Computer-Based  Learning  Environments,  Journal  of  Educational  Multimedia  and  Hpyermedia,  1 (2/3), 


Parker,  R.  (1996^  Mixed  Signals:  The  Prospects  for  Global  Television  News,  20^^  Century  Fund,  New 
York 

Telg,  R.  (1996)  Instructional  Design  Considerations  for  Teaching  International  audiences  via  Satellite, 
International  Journal  of  Instructional  Media,  23  (1)  209  - 223 


263  - 285 


CHANGES  IN  CANADIAN  HIGHER  EDUCATION  ICT  AND  SUPPORT, 

2000  TO  2003 


Carl  Cuneo,  McMaster  University  and  EvNet,  Canada  (cuneo@mcmaster.ca) 
Brain  Campbell,  Mount  Allison  University  and  EvNet,  Canada  (bcampbell@mta,ca) 
Campus  Computing  International  (Canada) 


Over  the  next  decade,  there  will  be  an  increasing  digital  gap  between  higher  education  institutions  offering  and  not 
offering  online  courses.  This  will  result  from  the  lack  of  planning  by  offline  institutions  in  information  technology 
policies,  networking  and  technical  infrastructure,  and  access  to  remote  electronic  library  resources.  This  is  not  an  all 
or  nothing  situation.  Some  offline  institutions  have  laid  plans  for  a digital  learning  future.  But  compared  to  online 
institutions,  they  have  less  extensive  plans.  It  takes  several  years  of  laying  down  the  technical  infrastructure  before 
we  see  the  results  in  suites  of  online  courses.  An  institution  cannot  decide  one  year  to  start  putting  in  place  such  an 
infrastructure,  and  then  expect  to  see  the  results  the  following  year  in  a dramatically  increased  online  learning. 


We  mailed  8 questionnaires  to  310  colleges,  CGEPS,  and  universities  in  English  and  French  Canada.  134 
institutions  returned  at  least  one  questionnaire  from  June,  1999  to  April,  2000,  for  a response  rate  of  43%.  The 
response  rate  was  good  among  the  60  universities  (89%),  but  poorer  among  colleges  (35%)  and  CGEPS  (25%). 

This  is  due  to  the  inclusion  of  many  very  small  colleges  and  CGEPS  that  had  difficulty  filling  out  complex 
questionnaires.  The  questionnaires  were  completed  by  8 senior  officials  responsible  for  policy  and  implementation 
of  information,  communication,  and  learning  technologies  (questionnaires  returned  are  in  parentheses):  VP  or  most 
senior  officer  responsible  for  information  and  communication  technology  policy  (82/134  =61%);  manager  or 
director  of  networking  and  technical  infrastructure  (72/134  = 54%);  head  of  computer  support  (84/134  = 62%); 
director  of  audio-visual  services  (68/134  = 51%);  head  librarian  (68/134  = 51%);  head  registrar  (69/134  = 52%); 
director  of  instructional  development  or  academic  or  teaching  technologies  center  (75/134  = 56%);  and,  president  of 
the  faculty  or  teachers'  association  (47/134  = 35%).  The  following  comments  and  analyses  are  based  on  these  data. 


INFORMATION  TECHNOLOGY  POLICIES 

VPs  of  Information  Technologies  at  institutions  offering  online  courses,  compared  to  those  not  offering  online 
courses,  placed  more  emphasis  over  the  next  3 years  in:  instructor  training  in  teaching  technologies;  recruitment  of 
more  technical  staff;  cooperation  with  other  institutions  in  software  licensing;  encouragement  or  requirement  of 
student  computer  ownership,  and  provision  of  open  access  computers  to  students.  If  we  expect  the  information 
technology  gap  to  close  between  online  and  offline  institutions,  institutions  not  offering  online  learning  should 
emphasize  these  policies  at  least  to  the  same  degree,  if  not  more,  than  institutions  offering  online  courses.  But  this  is 
not  so.  Over  the  long  term  this  will  lead  to  a growing  gap  between  online  and  offline  institutions. 


NETWORKING  AND  TECHNICAL  INFRASTRUCTURE 

We  asked  Managers  of  Networking  and  Technical  Infrastructure  how  much  they  emphasize  servers,  networking, 
student  computers,  and  wireless  devices  in  their  plans.  The  data  on  servers  reflects  the  declining  use  of  UNIX 
(despite  its  history  and  stability),  and  the  rise  of  Linux  and  Microsoft  NT  servers.  But  institutions  offering  online 
courses  are  more  likely  to  have  all  three  server  types  in  their  plans  than  institutions  not  offering  online  courses.  This 
may  reflect  the  greater  demand  on  servers  among  institutions  that  offer  online  learning.  However,  online  courses  are 
not  the  only  demand  for  servers;  they  are  used  for  a wide  variety  of  administrative  and  educational  functions.  Our 
question  did  not  tie  the  server  type  to  a particular  online  learning  use.  The  data  probably  reflect  a more  general 
difference  in  information  technology  configurations  between  online  and  offline  institutions.  Online  institutions  are 
also  much  more  likely  to  expand  wired  and  wireless  networking,  fast  ethnemet  connections,  notebook  and  other 
mobile  computing  devices,  and  student  computer  lab  capacity.  The  differences  are  so  large  that  this  kind  of  weak 
infrastructure  in  offline  institutions  will  likely  undermine  their  inability  to  offer  future  online  learning. 


DIGITAL  LIBRARY  RESOURCES 

If  institutions  not  offering  online  courses  expect  to  catch  up  to  institutions  offering  online  courses,  they  should 
reduce  their  expenditures  on  hard  copy  print  library  resources  as  much  as  or  more  than  institutions  that  are  heavily 
into  online  learning.  In  fact,  that  is  not  the  case.  Offline  higher  education  institutions  plan  to  spend  more  on  hard 
copy  print  library  resources.  A greater  percentage  of  head  librarians  of  institutions  offering  online  courses  plan  to 
expand  electronic  library  resources  over  the  next  three  years  in  contrast  to  offline  institutions.  This  reinforces  the 
view  that  institutions  already  offering  online  learning  are  forging  ahead  with  additional  plans  to  develop  and  extend 
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their  information  technology  infrastructure,  in  this  case  the  library  infrastructure,  in  contrast  to  institutions  not 
offering  online  learning.  Our  data  also  suggest  that  most  libraries  are  moving  quickly  into  the  electronic  age  on 
almost  any  information  technology  question  we  asked.  But  only  about  one-third  of  head  librarians  plan  to 
emphasize  student  e-mail  access  over  the  next  three  years.  Libraries  do  not  view  themselves  as  centers  for  providing 
students  with  e-mail  access;  such  services  and  access  are  available  elsewhere  on  campus  and  from  off-campus. 

SUPPORT  FOR  ELECTRONIC  COURSEWARE 

Directors  of  Academic  Technology  and  Instructional  Development  Centers  at  institutions  offering  online  courses 
plan  to  emphasize  the  following  between  2000  to  2003  in  contrast  to  institutions  not  offering  online  courses: 
funding  for  teaching  support  and  teaching  technology  support  units;  funding  for  free  instructor  workshops  in 
academic  technologies;  training  instructors  in  computer  literacy  and  web  design;  funding  for  instructional 
technology  projects  proposed  by  instructors,  academic  programs,  and  departments;  web  course  outlines;  interactive 
multimedia  web  courseware  with  full  instructor  participation  to  teach  the  courses;  standalone  interactive  web 
courseware  with  minimal  instructor  intervention;  e-mail  in  courses;  computer  conferencing  in  courses;  and, 
computer-supported  collaborative  learning  (CSCL)  in  courses.  However,  those  items  receiving  the  greatest 
endorsement  over  the  next  three  years  among  institutions  offering  online  courses  are:  course  outlines  on  the  web; 
computer  conferencing  in  courses;  computer  supported  collaborative  learning  in  courses;  and,  training  instructors 
in  web  design.  The  least  emphasis  in  plans  over  the  next  three  years  are  in  user-pay  instructor  workshops  in 
academic  technologies,  and  funding  for  instructor  release  time  to  work  on  teaching  technologies. 

REGISTRAR  PLANS  FOR  SUPPORTING  ONLINE  LEARNING 

Among  head  registrars  at  Canadian  higher  education  institutions,  there  is  a moderately  positive  association  between 
the  number  of  online  courses  offered  and  plans  over  the  next  three  years  in:  student  web  registration;  electronic 
delivery  of  student  academic  counseling;  web  course  evaluations  and  course  outlines;  courses  geared  to  Job  skills; 
distance  education  courses;  computer  conferencing,  video-conferencing  and  e-mail  in  courses;  computer-based 
training;  and,  linking  electronic  classrooms  between  institutions.  There  are  no  plans  among  registrars  in 
institutions  with  many  online  courses  to  move  into  general  paperless  office  procedures  and  web  calendars  over  print 
versions  (already  done);  student  computer  competencies  (these  skills  are  assumed  or  picked  up  as  students  take 
online  courses);  instructor  online  access  to  class  lists  and  online  submission  of  student  grades;  and,  information 
technology  in  face-to-face  courses  . There  are  three  areas  in  which  registrars  with  low  numbers  of  online  courses 
plan  to  emphasize  over  the  next  three  years,  while  registrars  at  institutions  with  many  online  courses  will  definitely 
not  emphasize:  part-time  student  enrollment;  continuing  education;  and,  instructor  access  to  student  academic 
records.  It  is  often  assumed  that  distance  online  learning  goes  hand-in-hand  with  continuing  education  and  part-time 
studies.  This  may  have  been  the  world  of  traditional  distance  education.  Online  learning  has  moved  out  of 
continuing  and  part-time  studies  into  full-time  studies.  We  are  probably  witnessing  here  a transition  in  the  academic 
programming  of  distance  education  as  it  moves  from  offline  print  correspondence  to  online  learning. 

DEVELOPMENT  AND  DELIVERY  TOOLS  OVER  NEXT  THREE  YEARS 

We  asked  directors  of  Instructional  Development  and  Teaching  Technology  Centers  the  emphasis  that  their 
institutions  will  place  on  development  and  delivery  software  over  the  next  three  years.  Institutions  offering  online 
courses  indicate  that  they  will  move  more  strongly  in  the  direction  of  the  top  software  in  the  market.  Over  three 
quarters  (78%)  of  institutions  offering  online  courses  indicated  that  they  will  move  even  more  strongly  in  the 
adoption  and  implementation  of  courses  using  Web  CT.  Even  among  institutions  not  offering  online  courses,  almost 
one-third  (30%)  plan  to  adopt  WebCT.  No  other  software  comes  close  to  WebCT  in  the  future  plans  of  institutions. 
However,  there  is  significant  growth  potential  in  Lotus  Notes,  Centrinity’s  FirstClass,  Macromedia  Director  and 
Authorware,  Asymmetrix's  Toolbook,  and  the  Web  Board.  There  does  not  seem  to  be  much  growth  potential  in 
Virtual-U,  Question  Mark,  Allaire  Forum,  Norton  Connect,  and  Web  Course  in  a Box. 

CONCLUSION 

There  is  little  evidence  to  suggest  that  institutions  not  offering  online  courses  will  soon  catch  up  to  institutions 
currently  offering  online  learning.  They  have  not  prepared  their  technical  infrastructure  for  offering  online  courses. 
Their  main  hope  might  be  closer  collaboration  with  neighboring  institutions  that  have  prepared  their  own 
infrastructure  for  a digital  future  in  online  learning. 
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Abstract:  In  this  paper  we  focus  on  the  use  of  computer  supported  collaborative  learning 
(CSCL)  in  an  organizational  setting.  The  program  we  use  is  Web- Knowledge  Forum. 
Although  there  is  a lot  of  experience  in  using  Knowledge  Forum  in  a classroom  setting,  little 
is  known  about  how  to  implement  and  work  with  this  program  in  an  organization  other  than 
schools.  The  problem  investigated  in  this  pilot  study  is  how  a community  of  practice  uses 
Web-Knowledge  Forum  as  an  online  learning  environment  to  discuss  work-related  problems. 
This  study  focuses  on  role  of  the  worker  as  a learner  in  an  unstructured  process  of  negotiation 
of  meaning  in  a community  of  practice.  In  the  study  eight  police  officers  participated  by 
forming  a heterogeneous  group  initiated  by  the  Information  Management  Approach  of 
Criminal  Investigation  in  Police  Education  (ABRIO)  a subdivision  of  the  Dutch  police  force. 


Introduction 

A learning  organization  stimulates  the  ability  to  learn  individually,  in  groups  and  to  learn  by  the 
organization  as  a whole.  The  aim  in  this  is  to  support  continuous  learning  to  combine  and  develop  knowledge  in 
the  organization  in  order  to  respond  flexible  to  changes  that  occur  in  the  market  (Bolhuis  and  Simons,  1999). 
There  is  a tendency  to  organize  this  learning  as  close  to  the  workplace  as  possible  instead  of  sending  people  to 
courses  given  outside  the  workplace  (Van  der  Krogt,  1995;  Nonaka  & Takeuchi,  1997;  Bolhuis  & Simons, 

1999).  Revans  argued  for  action  learning.  He  formed  learning  teams  to  work  on  real  organizational  problems  and 
to  structure  their  experience  in  such  a way  that  both  useful  solutions  to  these  problems  emerge  and  substantial 
learning  occurs  for  participants  (Vaill,  1996),  So  the  employees  are  recognized  as  an  important  resource  to  the 
organization.  According  to  Wenger  (1999)  people  in  organizations  form  communities  of  practice  by  helping  out 
each  other  and  discussing  the  latest  developments.  These  communities  of  practice  are  bound  by  a shared  practice 
related  to  a set  of  problems.  In  these  communities  they  share  and  create  knowledge;  they  learn  through 
participation  (Wenger,  1998).  Membership  to  these  communities  of  practice  is  voluntary,  these  communities  are 
not  bound  by  organizational  affiliations. 

In  large  organizations  like  the  Dutch  police  force  online  communities  have  an  advantage  in  bringing 
people  together  independent  of  time,  space  and  local  cultures.  CSCL  makes  it  possible  for  people  to  participate 
in  communities  of  practice  and  work  at  their  own  pace  and  time.  One  of  the  programs  that  support  this  kind  of 
collaboration  is  Web-Knowledge  Forum.  Web-Knowledge  Forum  is  a discussion  program  designed  to  form  a 
learning  community  over  the  Internet.  It’s  a product  of  the  Computer  Supported  Intentional  Learning 
Environments  (CSILE)  family,  developed  at  the  Ontario  Institute  for  Studies  in  Education  (OISE)  to  support  the 
collaborative  construction  of  knowledge  (Scardamalia  & Bereiter,  1992).  The  participants  operate  in  a shared 
workspace  in  which  they  read  and  write  notes.  A note  is  a contribution  that  can  contain  text,  pictures  and  links  to 
documents,  html  pages  or  other  notes  in  the  shared  knowledge  workspace.  Working  with  this  program  stimulates 
the  participants  to  talk  about  the  subject,  read  relevant  resource  materials,  pose  questions,  offer  theories,  conduct 
experiments,  and  work  together  to  make  sense  of  new  ideas.  By  working  together  participants  develop  greater 
competence  in  a particular  subject  area,  using  what  group  members  already  know  as  an  important  component 
and  co-constructing  plans  of  action  to  extend  that  knowledge  (Hewitt  & Scardamalia,  1998).  The  creation  of 
knowledge  therefore  is  seen  as  a social  product. 

Originally  this  program  was  designed  for  use  in  the  classroom  to  support  the  construction  of  knowledge 
in  a social  context.  The  aim  in  the  development  was  to  support  an  environment  that  will  make  it  possible  for 
schools  to  function  as  knowledge  building  communities  (Scardamalia  & Bereiter,  1 992).  Within  knowledge 
building  communities  the  focus  is  on  knowledge  construction.  It’s  a knowledge-centered  community  of  practice. 

The  problem  we  face  is  how  to  facilitate  the  creation  and  support  of  communities  of  practice  in 
organizations  that  work  in  a Web  based  environment  and  make  knowledge  building  the  core  of  their  activity.  In 
the  first  place  there  has  to  be  identified  a ‘real’  and  meaningful  problem  that  exists  in  the  organization.  A 
problem  that  is  owned  by  the  participants  and  in  which  solution  they  are  willing  to  put  effort.  Second  the 
members  of  the  organization  who  feel  affiliated  by  this  problem  have  to  form  a community  in  which  they  can 
participate  on  a voluntary  and  functional  basis.  In  this  paper  we  focus  on  the  possibilities  of  Web-Knowledge 
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Forum  to  serve  as  a meeting  place  for  such  communities.  A place  where  participants  can  work  together, 
undertake  collaborative  learning  activities,  sharing  their  knowledge,  aimed  at  deepening  their  expertise  in  the 
problem  to  be  solved. 

Our  question  is:  Can  groupware,  e.g.  Web-Knowledge  Forum,  support  the  negotiation  of  knowledge 
and  understanding  in  communities  of  practice  in  an  organization? 

We  focus  on  the  role  of  the  worker  as  a learner  in  an  unstructured  process  of  negotiation  of  meaning  in  an  online 
community  of  practice. 


The  study 

This  study  was  conducted  to  gain  experience  with  participants  who  were  engaged  in  an  online  learning 
community.  The  community  consisted  of  eight  participants  who  voluntary  joined  in  the  community.  They 
responded  to  a letter  that  was  sent  by  the  ABRIO  to  several  police  departments  explaining  the  problem  that  has 
to  be  solved.  The  problem  was  about  how  to  identify  and  describe  general  work-processes  used  in  the  field  of 
criminal  investigation.  As  a whole  the  participants  formed  a heterogeneous  group  (policy  makers,  criminal 
investigators  and  experts). 

During  a period  of  two  months  they  worked  together  using  Web-Knowledge  Forum.  There  was  much 
uncertainty  about  how  to  identify  work-processes,  therefore  the  participants  agreed  to  start  with  an  open 
discussion  on  the  subject  ‘work-processes’,  instead  of  following  a structured  plan  of  action  to  tackle  the  problem. 
Web-Knowledge  Forum  played  a central  role  in  supporting  the  discussion  because  all  the  written  contributions 
are  stored  as  notes  in  a shared  database  available  to  all  the  participants.  The  discussion  was  divided  into  certain 
subjects  called  views  in  which  the  participants  contributed  a note  or  comment  on  a note  they  had  read,  by  writing 
a build-on  note. 


Instruments 

The  way  people  participated  and  interacted  with  each  other  provides  information  about  the  activities  of 
such  a community.  Web-Knowledge  Forum  is  provided  with  an  analytic  toolkit  (ATK)  that  analyses  the 
activities  of  the  members  of  the  community  in  the  database.  It  creates  log  files  of  all  the  users  about  how  many 
times  they  have  read,  write  or  edited  a note.  How  many  notes  are  linked  to  each  other  and  how  many  ‘build-on’s 
there  are  made. 

Web-Knowledge  Forum  is  designed  to  facilitate  cognitive  and  metacognitive  activities  by  providing 
opportunities  to  give  your  opinion,  to  give  a comment,  or  by  making  suggestions  or  providing  new  information. 
Because  of  these  possibilities  it  is  important  to  know  more  about  the  nature  of  the  content  of  the  material  the 
community  has  created  together  and  what  kind  of  activities  the  participant  undertake.  Are  they  tiying  to  advance 
their  knowledge?  Veldhuis-Diermanse  (1999)  developed  a coding  scheme  to  gain  information  about  the  content 
of  the  written  notes  based  on  the  constructivist  view  of  learning  in  an  educational  setting.  This  coding  scheme  is 
still  in  a design  stage  and  it  is  the  first  time  this  scheme  is  applied  to  an  organizational  community.  This  coding 
scheme  consists  of  three  main  categories:  1 Cognitive  activities,  2 Metacognitive  activities,  and  3 Affective 
activities. 

1 Cognitive  activity:  Cognitive  activities  are  used  to  process  and  acquire  insight  in  the  information  being 
discussed.  Veldhuis-Diermanse  recognized  three  subcategories  of  cognitive  activities:  1 debating,  the  accent  is 
on  arguing,  presenting  new  ideas  or  thoughts  about  the  subject;  2 using  external  information,  the  accent  is  on 
referring  to  information  found  in  other  sources  than  the  database;  and  3 linking  or  repeating  internal  information, 
the  accent  is  on  referring  to  information  found  in  the  shared  database. 

2 Metacognitive  activities:  Veldhuis-Diermanse  (1999)  describes  metacognitive  activities  as  activities 
undertaken  to  regulate  each  other’s  learning  process  or  to  regulate  the  goals  and  direction  of  the  discussion. 

3 Affective  activities  are  used  to  cope  with  feelings  occurring  during  the  discussion  among  the  participants 
(Veldhuis-Diermanse,  1999). 

At  the  end  of  the  pilot-study  we  gave  the  participants  a questionnaire  to  gain  information  about  their 
experiences  working  with  Knowledge  Forum. 


Results 

The  log  files  generated  by  ATK  gives  a description  of  the  activities  that  has  taken  place  in  the  database 
(Tab.  1).  The  participants  contributed  98  notes  in  the  database.  That’s  an  average  of  12,25  notes  per  participant. 
56%  percent  of  the  notes  have  been  read.  This  means  the  amount  of  notes  that  have  been  opened  by  the 
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participants.  So  this  might  exaggerate  the  actual  reading  that  has  been  done.  83%  percent  of  these  notes  are 
linked,  also  called  build-on.  The  log-file  only  records  the  activities  of  the  participants,  a build-on  activity 
therefore  does  not  have  to  be  content  related.  Table  1 shows  that  there  are  substantial  differences  between  the 
participants,  both  in  writing  and  reading.  Notice  that  build-on  notes  are  also  a part  of  the  written  notes. 


Written 

Build-on 

Read 

N 

8 

8 

8 

N 

8 

8 

8 

Mean 

12,25 

10,75 

232,62 

Mean 

12,25 

10,75 

232,62 

Std. 

9,54 

8,80 

147,51 

Deviation 

Std. 

9,54 

8,80 

147,51 

Deviation 

Minimum 

1,00 

1,00 

113,00 

Minimum 

1,00 

1,00 

113,00 

Maximum 

30,00 

26,00 

552,00 

Maximum 

30,00 

26,00 

552,00 

Table  1.  Participation  in  the  database 


The  content  analysis  of  the  notes  reflects  the  following  cognitive  and  metacognitive  activities  (Fig.  1). 


debating  linking  affective 

external  information  regulation 
information 

activities 


Figurel.  Activities  in  the  database 


These  results  indicate  that  the  participants  discuss  a lot  about  the  subject.  The  participants  present  a lot 
of  ideas  of  there  own  and  start  discussing  about  that  in  the  community.  The  participants  do  not  bring  in  much 
new  information;  only  seven  times  they  referred  to  information  that  can  be  found  outside  the  database  (e.g,  book 
pages).  Also  they  did  not  make  many  content  related  references  to  the  contributions  of  other  participants  in  the 
database.  Although  according  to  the  ATK  83  % of  the  notes  are  linked.  This  difference  is  explained  by  the  fact 
that  the  coding  scheme  only  refers  to  written  comments  to  other  contributions  in  the  database.  The  participants 
show  quite  a lot  of  regulative  activities,  but  in  fact  two  participants  are  responsible  for  66%  of  the  regulative 
activities.  There  are  quite  a lot  of  affective  contributions  in  the  database.  The  explanation  for  it  is  that  this  way  of 
working  was  new  to  them  and  therefore  they  were  regularly  asking  for  feedback  and  gave  a lot  of  general 
reactions  to  the  other  participants.  This  category  shows  that  they  did  not  use  this  medium  for  chatting  or  sending 
‘e-mails’  to  each  other,  that  are  not  related  to  the  discussion.  Most  contributions  even  the  affective  ones  were 
subject  related. 

The  questionnaire  provided  information  about  the  way  participants  worked  with  Web-Knowledge 
Forum,  five  out  of  eight  were  returned.  60%  of  the  participants  agreed  on  the  question  if  they  were 
collaboratively  building  new  knowledge  about  ‘work-processes’,  but  they  pointed  out  that  they  need  to  grow 
more  into  building  upon  the  ideas  of  others.  Also  they  mentioned  that  there  was  a lot  of  confusion  about  the 
concepts  being  used  and  that  they  need  to  clarify  the  goal  of  their  study,  to  give  more  direction  to  the  discussion. 
80%  of  the  participants  indicated  that  they  were  satisfied  with  the  opportunities,  provided  by  the  program,  to 
discuss  the  subject  together.  The  participants  (80%)  notified  to  have  enough  information  to  be  able  to  take  part  in 
the  discussion.  Answers  to  the  question,  what  they  do  if  they  lack  certain  information,  are  searching  for  relevant 
information,  consulting  colleagues  at  work,  and  trying  to  stimulate  the  other  participants  to  explain  certain 


issues.  Results  of  the  question  about  what  they  thought  of  the  quality  of  the  written  notes,  varies  from  good  to 
reasonable.  In  general  the  quality  is  good  but  the  discussion  became  more  silent  later  on,  “There  is  to  little 
structure  to  guide  our  discussion,  the  notes  contain  valuable  information  but  what  does  it  bring  to  us?”  40%  of 
the  participants  indicated  that  there  was  to  less  coordination  during  the  discussion,  60%  of  the  participants 
pointed  out  that  a more  structured  or  goal  directed  approach  is  necessary.  They  argue  that  this  will  help  them  to 
achieve  agreement  and  build  on  to  that. 


Conclusion  and  Discussion 

Summarizing  the  results  it  can  be  seen  that  the  participants  show  a lot  of  activity  in  the  database 
although  they  read  much  more  than  they  contribute.  Related  to  the  content  analysis  of  the  contributions.  It  seems 
that  the  activities  the  participants  undertook  are  more  discussion  oriented  than  building  knowledge  upon  already 
contributed  or  new  information.  However  in  their  perception  of  working  with  groupware  they  appreciate  the 
possibility  of  knowledge  sharing.  On  the  first  glance  people  are  willing  to  work  in  a groupware  environment  and 
share  knowledge  together.  This  is  promising  for  the  support  of  working  in  communities  of  practice  in 
organizations.  The  analysis  also  shows  that  sharing  of  knowledge  does  not  happen  automatically.  Although  there 
are  a lot  of  regulation  activities,  it  turns  out  that  they  were  mostly  carried  out  by  two  participants.  The 
questionnaire  reveals  that  the  community  members  desire  more  structure  and  support  to  direct  the  knowledge 
building  activities  of  the  community  as  a whole.  The  use  of  groupware  in  communities  of  practice  in 
organizations  seems  promising.  The  participants  report  that  this  tool  is  useful  for  knowledge  sharing  but  that  the 
actual  knowledge  building  activities  needs  to  increase.  They  had  trouble  clarifying  the  goals  and  direction  of  the 
discussion.  Also  they  discovered  that  there  was  quite  a lot  of  confusion  about  the  concepts  used  by  the 
participants.  The  main  concern  was  lack  of  coordination  during  the  discussion.  In  an  unstructured  process  of 
negotiation  nobody  feels  direct  responsibility  to  organize  and  structure  the  content  of  the  problem  studied.  The 
lack  of  the  learning  ability  in  the  sense  of  regulating  the  content  and  community  processes  seems  to  be  crucial 
for  people  to  become  used  to  share  knowledge,  deepening  their  own  and  common  understanding  and  creating 
further  insights.  To  compensate  this  lack  one  can  structure  the  negotiation  of  meaning  by  making  a learning 
agenda.  By  which  participant  express  their  goals,  divide  certain  tasks,  and  divide  responsibilities  (e.g, 
coordinator  of  the  content,  someone  who  keeps  the  community  together  or  invites  new  participants  when  needed, 
technical  assistance).  One  can  also  introduce  a didactic  approach  of  inquiry  analogue  to  phases  of  the  scientific 
enquiry  process  or  problem  solving. 
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Abstract:  Federal  mandates  enacted  on  August  7,  1998  revealed  that  "Effective  2 years  after  the  date  of 
enactment... any  individual  with  a disability  may  file  a complaint  alleging  that  a Federal  department  or  agency  fails  to 
comply  with  subsection  (a)(1)  in  providing  electronic  and  infonnation  technology.  (Section  508.  "Electronic  and 
Information  Technology)  "The  new  rules  will  apply  within  a few  months  to  all  web  sites  operated  by  government 
agencies.. .By  August  7,  2000,  they  will  extend  to  vendors  doing  business  with  the  federal  government.  "Possibility 
soon  afterward  they  may  extend  to  every  Web  site  posted  in  the  U.S."  (Adam  Clayton  Powell  III).  Are  ready  for 
August  7,  2000?  If  not,  this  paper  can  provide  ideas  to  help  you  get  started. 


Introduction 

In  PC  Week  Online,  Vaas  (2000)  notes  that  between  95  and  99  percent  of  web  sites  are  inaccessible  to  the  visually, 
hearing  -and/or  mobility  impaired.  Many  web  developers  are  ignoring  the  accessibility  issue  or  just  do  not  know 
how  to  making  their  sites  universally  accessible.  They  are  bewildered  in  the  face  of  the  vast  information  available  in 
terms  of  how  to  make  it  applicable  to  their  situation.  Three  areas  can  help  focus  the  journey  to  understanding. 

What’s  What 

The  law  is  the  driving  force  behind  the  need  to  have  universal  web  sites.  There  are  already  been  law  suits  against 
AOL  and  Universities  for  not  providing  accessible  web  access.  Equally  access  to  information  is  the  key  reason  that 
the  focus  is  on  web  sites.  Individuals  cannot  fully  participate  in  today’s  complex  society  based  on  technology-based 
information  if  they  do  have  access  to  the  same  information.  There  are  several  references  to  the  law  at  the  following 
web  site: 

http://www.svvt.edu/~dfl  2/summit/LAW.HTM 

Who’s  Who? 

This  paper  will  mention  just  a few  of  the  leaders  in  the  field  who  are  trying  to  make  electronic  information  globally 
accessible.  The  first  is  W3C.  Their  mission  states  "The  W3Cs  commitment  to  lead  the  Web  to  its  full  potential 
includes  promoting  a high  degree  of  usability  for  people  with  disabilities.  The  Web  Accessibility  Initiative  (WAI), 
in  coordination  with  organizations  around  the  world,  is  pursuing  accessibility  of  the  Web  through  five  primary  areas 
of  work:  technology,  guidelines,  tools,  education  & outreach,  and  research  & development." 

(http://www.w3.org/WAI/)  W3C  focus  is  for  experienced  programmers  and  guidelines  can  be  found  at  the  following 
address: 

http://www.w3.org/TR/1999/WAl-WEBCONTENT-1999Q5Q5/ 

Accessible  Web  Authoring  Resources  and  Education  (AWARE),  mission  is  to  serve  as  a central  resource  for  web 
authors  for  learning  about  web  accessibility.  The  AWARE  Center  was  launched  in  April  1999  as  part  of  the  HTML 
Writers  Guild’s  (HWG)  annual  Web  Accessibility  Month,  a special  focus  on  the  importance  of  designing  for 
universal  accessibility.  The  Center  is  supported  by  the  Guild’s  staff  and  volunteers,  and  is  designed  as  a resource  for 
all  web  authors.  Ken  Barlett  co-founded  the  HTML  Writer's  Guild  with  Martin  Bayne.  The  HWG  is  the  world’s 
largest  international  organization  of  Web  authors  with  over  13Q,QQQ  members  in  more  than  13Q  nations  worldwide. 
The  HGW  exists  to  assist  its  members  in  developing  and  enhancing  their  capabilities  as  web  authors,  to  compile  and 
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publicize  information  about  standards,  practices,  techniques,  competency,  and  ethics  as  applied  to  web  authoring, 
and  to  contribute  to  the  development  of  the  web  and  web  technical  standards  and  guidelines. 

Disability  Access  Information  and  Support  (DAIS).  Jane  Jarrows,  President,  states,  " But  after  we  got  the  ramps  and 
elevators  installed,  we  could  think  about  how  to  make  the  teaching  that  had  always  gone  on  in  those  classrooms 
fully  available  to  all  students.  The  parallel  ends  here.  While  we  are  scrambling  to  get  the  technological  "ramps  and 
elevators"  in  place,  we  are  actively  creating  new  barriers  to  access  on  a regular  basis.  At  the  risk  of  stretching  the 
analogy  a little  too  thin,  if  we  don't  be  careful  we  will  find  that  when  those  technological  ramps  and  elevators  we 
installed  are  finally  in  place,  they  only  allow  for  a better  view  of  the  opportunities  passing  by  us.  We  cannot  wait  to 
get  the  technology  in  place  before  we  start  trying  to  impact  on  the  application  of  that  technology 

Microsoft,  The  Microsoft  Senior  Initiative  is  a program  aimed  at  bridging  the  digital  and  generational  divides  and 
ensuring  that  seniors  are  not  left  behind  on  the  information  superhighway.  By  providing  access  to  information 
technology  and  PC  literacy  training,  the  Microsoft  Senior  Initiative  introduces  the  exciting  possibilities  of 
technology  to  senior  citizens.  The  Seniors  and  Technology  Web  site  (http://www.microsoft.com/seniors/)  is  a 
resource  for  seniors,  their  families  and  communities  about  the  exciting  possibilities  that  can  be  realized  through  the 
use  of  technology. 

How  to?  For  people  who  are  just  learning  how  to  develop  web  pages,  the  easiest  way  is  make  a page  using  a 
"what  you  see,  what  you  get  editor,"  then  save  the  web  page  as  text  and  htm.  The  problem  with  this  method  is  that 
that  many  people  do  not  do  an  adequate  job  of  keeping  both  pages  updated.  Software  companies  are  starting  to 
make  it  easier  to  make  web  pages  accessible  through  what  is  called,  style  sheets,  which  control  the  presentation  of  a 
document.  It  is  often  easier  to  work  in  pairs  or  teems  of  two  to  learn  this  process.  There  are  often  many  free 
workshops  on  the  web  and  in  your  local  area.  Perhaps  partner  with  someone  who  can  pay  to  have  a consultant. 
Design  is  the  first  part  of  "how  to."  The  second  part  is  getting  your  page  validated  by  a reliable  source.  Sites  can  be 
easily  validated  using  on-line  features  which  require  that  you  paste  your  URL  name  in  the  box  and  almost 
immediately  you  will  learn  if  you  have  accessibility  problems.  The  top  two  sources  of  validation  are: 

Bobby  Accessibility  Validator 
http://www.cast.org/bobby/ 

W3C  HTML  4.0  Validator 
http://validator.w3.org/ 

The  complexity  of  implementation  of  this  mandate  can  not  be  fully  address  with  this  in  progress  paper;  however, 
there  will  a much  greater  discourse  at  the  Universal  Web  Accessibility  Symposium  2000  (at  WebNet  2000)  --  San 
Antonio,  Texas,  October  31,  2000. 
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Introduction 

Many  educators  are  experimenting  with  flexible  on-line  learning;  consequently  academics  have  to 
adapt  their  delivery,  preparation  and  course  administration.  This  study  shares  staff  and  student 
experiences  of  participating  in  a completely  remote  on-line  course,  compared  to  a flexible  on-line 
course  where  students  can  attend  classroom  sessions. 

The  issues  we  encountered  included  ones  related  to  hardware  and  software,  submission  of  assessments, 
student  communication  with  each  other  and  the  lecturers,  as  well  as  student  and  staff  motivation  over 
the  holidays. 

Existing  research  documents  the  pitfalls  of  teaching  on-line  and  we  used  these  lessons  to  anticipate 
problems  as  we  planned  our  on-line  course.  This  study  documents  the  problems  we  encountered  but 
hadn’t  anticipated. 

THE  COURSE 

Internet  and  Web  Design  (IWD)  is  a popular  first  year  introductory  course  in  the  Bachelor  of 
Computing  Systems  degree.  It  is  normally  taught  flexibly  with  lectures  and  practical  classes  as  well  as 
a self-paced  on-line  course  comprised  of  notes,  exercises,  self-assessment  questions  and  a discussion 
forum  (1).  In  the  1999-2000  Summer  School,  the  course  was  offered  completely  remotely  and 
condensed  from  14  to  11  weeks.  There  were  no  face-to-face  sessions  except  an  orientation  session,  the 
project  presentations  and  final  exam.  If  students  had  problems  they  were  requested  to  post  their 
question  on  a discussion  forum  and  had  no  option  to  ask  the  lecturers  face-to-face. 

DISCUSSION 
Student  issues 

No  time  for  a Summer  break,  condensed  time  frame,  lack  of  personal  tutor-student  contact,  no  option 
of  attending  classroom  sessions  for  personal  contact  with  other  students  and  no  opportunity  to  take  on  a 
vacation  job.  Perhaps  because  of  the  above  issues,  there  was  a higher  level  of  drop  outs  than  normal. 
Reasons  for  students  dropping  out  included:  getting  a job,  workload  too  heavy  and  no  computer  access 
at  home 

Staff  issues 

Lack  of  a gap  between  semesters  to  attend  to  the  course  administration  for  the  previous  course.  We 
undertook  the  course  as  an  extra  to  our  workload,  meaning  no  summer  break  to  recharge  and  prepare 
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for  the  next  semester.  Most  of  the  workload  consisted  of  administration  and  marking  and  we  felt  the 
remuneration  we  received  did  not  reflect  the  associated  workload. 

Bulletin  board  issues 

If  students  had  problems  they  were  requested  to  post  their  question  on  the  discussion  forum  rather  than 
e-mail  the  lecturers,  (so  that  we  could  avoid  duplication  of  lecturer  responses  and  attempt  to  mirror 
classroom  communication  by  allowing  students  to  answer  each  others’  questions  when  lecturers  were 
not  around).  Despite  our  attempts  to  encourage  participation,  the  use  of  the  bulletin  board  was  fairly 
low,  with  only  140  posts  over  the  whole  Summer  School,  Many  students  continued  to  email  the 
lecturers  rather  than  use  the  bulletin  board  for  questions.  Perhaps  this  was  because  they  didn’t  want  to 
look  unknowledgeable  in  front  of  their  peers,  or  because  we  didn’t  assess  the  students’  contributions 
(2,3),  Lecturers  accessed  the  bulletin  board  every  other  day  to  check  for  questions  to  answer.  The 
lecturer’s  often  interrupted  their  vacations  to  make  special  trips  to  cyber-cafes,  only  to  find  that  no 
students  were  seeking  help. 

Assignment  submission  issues 

Students  didn’t  email  their  assignments  to  the  correct  address,  often  sending  them  to  our  home  email 
addresses  and  multiple  addresses.  Issues  resulting  from  this  included:  lack  of  software  at  lecturer’s 
home;  lecturer’s  vulnerable  to  the  risk  of  virus  infection;  and  duplication  of  administration  effort.  There 
was  a significant  administrative  burden  for  the  lecturers  who  had  to  confirm  receipt  by  email  and  print 
out  the  assignments.  This  was  exacerbated  by  the  often  inappropriately  large  file  sizes  of  the  email 
attachments,  (e.g,  one  Storyboard  Word  document  was  3.5Mb). 

Group  work  issues 

Generally,  students  find  it  difficult  to  work  in  groups.  This  is  exacerbated  in  an  on-line  course  by 
having  to  use  the  printed  word  rather  than  face-to-face  interaction.  The  condensed  timescale  of  the 
course  made  this  even  more  of  an  issue  and  one  of  our  groups  suffered  a complete  breakdown  in 
communication.  During  a Semester  course  it  is  easier  to  identify  problems  with  group  dynamics  and 
deal  with  them  before  they  escalate. 

CONCLUSIONS 

The  students  had  a generally  favourable  impression  of  on-line  learning,  but  with  reservations  about  the 
lack  of  personal  contact  and  the  high  workload  due  to  the  collapsed  course  time-scale.  Reasons  for  the 
course’s  success  included:  anticipating  possible  problems  and  devising  strategies  to  deal  with  them; 
and  the  considerable  amount  of  user-testing  that  took  place  in  the  prior  flexible  on-line  versions  of  the 
course,  when  semantic  problems  could  be  cleared  up  easily. 

However  unexpected  problems  were:  inter-group  conflicts;  a higher  than  normal  drop-out  rate  that 
impacted  on  other  group  members;  higher  than  normal  administrative  burden  on  the  lecturers;  and 
serious  practical  problems  with  the  publishing  of  student  web-sites  using  FTP  (4). 

It  is  important  to  spend  time  forming  project  groups,  to  let  the  students  get  to  know  each  other’s  skill- 
set  and  interests  thereby  avoiding  problems.  This  is  perhaps  even  more  important  in  a fully  on-line 
course. 
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Abstract:  Offering  video  and  other  rich  media  content  over  the  Internet  is  an  emerging  growth  area  and 
represents  a new  model  and  opportunity  for  education.  As  broadband  access  increases  and  advances  are  made  in 
network  caching,  streaming  servers,  video  and  audio  compression,  and  multimedia  databases,  delivering  streaming 
video  claims  unprecedented  potential  for  education.  Universities  and  corporations  will  continue  to  look  to 
technology  to  deliver  distance  education  and  web  based  courses.  Determining  the  pedagogical  applications  of 
streaming  video  is  an  important  area  of  research.  Identifying  the  strengths  and  weaknesses  from  an  educational 
point  of  view  provides  an  important  context  to  the  practical  matters  of  technical  implementation.  This  research 
intends  to  investigate  both  the  pedagogical  implications  and  the  technical  implementation  of  streaming  video. 


Overview 

Offering  video  and  other  rich  media  content  over  the  Internet  is  an  emerging  growth  area  and  represents  a 
new  model  and  opportunity  for  education.  As  broadband  access  increases  and  advances  are  made  in  network 
caching,  streaming  servers,  video  and  audio  compression,  and  multimedia  databases,  delivering  streaming  video 
claims  unprecedented  potential  for  education.  Universities  and  corporations  will  continue  to  look  to  technology  to 
deliver  distance  education  and  web  based  courses.  Determining  the  pedagogical  applications  of  streaming  video  is 
an  important  area  of  research.  Identifying  the  strengths  and  weaknesses  from  an  educational  point  of  view  provides 
an  important  context  to  the  practical  matters  of  technical  implementation.  This  research  intends  to  investigate  both 
the  pedagogical  implications  and  the  technical  implementation  of  streaming  video. 

Often  when  new  technologies,  like  video,  are  applied  to  education  they  are  seen  as  having  the  ability  to 
make  us  all  learn  better  and  faster.  Some  of  the  questions  that  need  to  be  answered  regarding  pedagogy  include: 

• Does  streaming  video  over  the  web  offer  any  more  promise  than  standard  videotape,  the 
Laser  Disc,  or  the  CD-ROM? 

• Can  the  metadata  applied  to  streaming  video  offer  learning  strategies  and  information  that 
was  previously  unavailable? 

• Does  the  distribution  medium,  the  Internet  and  a computer,  improve  the  quality  or  access 
to  instruction? 

• What  are  the  ideal  tools  and  methods  to  develop  content  and  sufficient  metadata  that 
make  streaming  video  effective  in  teaching  and  learning? 
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In  addition  to  the  pedagogical  perspective,  there  is  a highly  technical  aspect  of  streaming  video  that  affects 
price,  availability,  usability,  performance,  and  quality.  The  following  technical  components  will  be  investigated: 

• Copyright,  Security,  and  Intellectual  Property 

• Multimedia  Databases 

• Servers 

• Clients 

• Compression 

• Network  Caching  and  Bandwidth  Issues 

Finally,  partnerships  with  industry  suppliers,  distributors,  faculty,  and  technology  solutions  will  be 
investigated  in  an  effort  to  provide  business  models  that  can  recover  the  costs  of  implementing  and  delivering 
streaming  video  content  for  universities. 

Currently,  the  pedagogical  potential  of  streaming  video  has  not  been  demonstrated,  the  technology  is  so 
new  that  it  remains  largely  untested  in  a learning  environment,  and  content  development  tools  are  crude  or  non- 
existent. Answering  the  various  pedagogical  and  technical  questions  are  important  outcomes  of  this  research 
project.  Performance  indicators  of  this  research  will  include  the  application  of  streaming  video  technologies  to 
multiple  content  areas  - including  campus  and  off-campus  users.  In  addition,  the  partnerships  developed  with 
industry,  such  as  Akamai  or  other  industry  partners,  will  be  measured  against  the  benefits  to  the  institution. 

Detailed  Description 

Applications  for  streaming  video  are  various.  They  include:  webcasts  of  special  lectures;  access  to  a rich 
educational  video  library;  video  and  audio  conferencing;  the  integration  of  video  into  course  web  pages;  and  the 
creation  of  video-based  educational  objects.  The  effectiveness  of  this  technology  to  teaching  and  learning  is  largely 
untested,  but  none  the  less  the  demands  to  incorporate  it  into  teaching  and  learning  are  starting  to  be  met  by  the 
technology.  Only  recently  has  a distribution  medium  become  available  that  allows  full  text  searching  of  video,  the 
application  of  metadata,  interactivity,  and  other  media  in  ways  that  make  the  use  of  video  in  education  more  flexible 
than  ever  before.  Research  into  the  tools,  usability,  educational  effectiveness,  and  various  solutions  benefit  current 
and  future  teaching  and  learning  at  our  post-secondary  institutions. 

The  focus  is  on  three  main  aspects  of  streaming  video.  These  are  quality  and  performance,  pedagogy,  and 
business  models.  Compression  algorithms,  the  end  user  software,  or  client,  networking  technologies  and  server 
solutions  impact  quality  and  performance.  In  order  to  study  quality  and  performance,  various  partners  around  the 
world  are  providing  and  testing  content.  Other  partners,  that  can  provide  performance  enhancements  to  streaming 
video  including,  cable  companies,  phone  companies,  and  broadband  access  providers  is  under  investigation. 
Various  solutions’  quality  and  performance  will  be  compared  by  individual  reviewers  and  focus  groups  in  several 
locations.  Quality  and  performance  will  be  tested  given  different  conditions,  including  connection  speed,  time  of 
day,  distance  from  server,  end-user  platform,  server  characteristics,  compression  algorithm,  and  client.  Akamai’s 
Free  Flow  server  technology  will  be  tested  to  compare  performance  enhancements  at  a regional  and  international 
level.  Campus  and  off-campus  user  tests  will  determine  quality  and  performance  standards  for  the  main  streaming 
video  solutions. 

Streaming  video,  in  itself,  is  like  any  other  video  format  from  a pedagogical  point  of  view.  Video,  in 
general,  is  considered  linear  and  lacking  interactivity.  If  putting  learners  in  front  of  a video  feed  was  effective,  we 
would  learn  from  home,  in  front  of  our  televisions,  watching  a video  taped  lecture.  Clearly,  streaming  video  of 
lectures  over  the  web  does  not  represent  any  educational  advantage  over  distributing  VHS  copies  of  lectures. 
However,  the  potential  for  the  video  and  audio  to  contain  additional  information  may  provide  an  educational 
advantage.  Using  video  taped  scenarios  that  demonstrate  procedures  and  principles  can  be  an  effective  use  of  video. 
Marking  up  that  scenario  with  additional  information,  or  metadata,  may  demonstrate  clear  advantages  for  teaching 
and  learning.  Creating  interactivity,  allowing  the  learner  to  make  decisions,  search  text,  navigate  non-linearly  , and 
interact  with  the  video  may  allow  a learner  more  effective  access  to  learning  using  video  than  previously  possible. 
Demonstrating  the  possibilities  is  not  simple.  Tools  that  develop  the  metadata,  and  procedures  that  accommodate 
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advanced  content  development  for  video  are  crude  and  difficult  to  use.  In  fact,  although  the  need  and  technology 
exist,  there  no  cost  effective  methods  for  creating  a transcript  of  a video  without  the  use  of  an  individual  typing  what 
they  hear  into  a computer.  The  expense  of  adding  this  metadata  discourages  its  inclusion.  The  goals  here  are  to 
demonstrate  its  potential  and  to  discover  processes  and  tools  that  permit  the  inclusion  of  metadata  for  streaming 
video.  It  is  anticipated  that  this  research  will  more  clearly  specify  the  need  for  content  development  software  to  be 
developed. 

Associated  with  content  development  for  the  metadata  of  video  are  the  data  stores  for  the  video  and  its 
metadata.  There  are  several  commercial  database  solutions  currently  available  and  establishing  the  criteria  that 
matches  sound  pedagogy  for  the  database  is  an  important  outcome  of  this  study.  One  of  the  issues  is  extensibility  of 
the  database.  Scalability,  compatibility  and  reusability  are  all  related  to  extensibility.  The  momentum  of 
educational  objects  and  IMS  metadata  standards  for  education  dictate  the  need  for  the  open,  flexible  standards  and 
an  object  oriented  approach.  A solution  where  costs  do  not  prohibit  university  access  to  necessary  features,  while 
offering  security  and  intellectual  property  protection  will  be  sought. 

The  third  aspect  of  this  research  focuses  on  the  costs,  partnerships,  infrastructure  requirements,  and 
opportunities  of  streaming  video  for  institutions.  In  essence,  a business  model  that  can  sustain  the  delivery  of 
streaming  video  for  the  institution.  Costs  associated  with  video  production,  editing,  and  compression  are  high, 
adding  in  metadata  and  storing  in  it  all  in  large  database  applications  can  be  very  high.  Pricing  varies  widely 
between  suppliers  and  solutions  and  an  investigation  into  the  details  will  lead  to  recommendations  for 
implementation.  Other  costs  associated  with  streaming  video  include  servers,  licensing  fees,  and  Internet 
infrastructure.  A detailed  comparison  of  these  costs  against  the  performance  and  quality  testing  will  lead  to 
additional  recommendations. 

In  addition  to  the  cost/benefit  analysis,  the  opportunities  for  collaboration  with  the  private  sector  is  being 
investigated.  Opportunities  exist  that  allow  the  university  and  its  faculty  to  recover  royalties  to  its  intellectual 
property  and  charge  for  access  to  its  servers.  Streaming  video  over  the  web  could  prove  to  become  the  best 
distribution  medium  available  for  the  thousands  of  hours  of  video  already  produced  and  owned  by  our  institutions. 
Developing  the  types  of  partnerships  and  business  models  that  could  help  support  delivery,  provide  new  content, 
build  infrastructure,  reach  broader  audiences,  and  maintain  hardware  and  software  is  being  prototyped  at  the 
University  of  Calgary. 


Summary 

The  various  complex  issues  from  network,  to  compression,  multimedia  databases  and  usability  require 
research,  development,  and  testing  to  determine  the  best  solutions  for  the  institution.  Guided  by  sound  pedagogy, 
the  framework  suggested  above  is  meant  to  get  to  some  of  the  most  important  and  practical  issues,  enabling  the 
recommendation  and  adoption  of  solutions  for  the  use  by  a post-secondary  institution. 
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Abstract:  The  availability  of  computers,  the  emergence  of  new  visualisation  and  authoring  tools 
and  techniques,  together  with  the  expansion  of  web  based  learning  environments  and  CD-based 
multimedia  applications,  offer  chemistry  teachers  new  possibilities  for  bridging  the  gap  in  students’ 
knowledge  and  understanding  of  abstract  and  concrete  chemical  concepts  and  processes.  This  paper 
shows  by  example  how  the  advantages  of  both  the  Internet  and  the  interactive  CD-ROMs  can  be 
used  to  improve  students'  learning  at  both  macro-  and  microscopic  level.  We  present  results  of  a 
study  of  the  effects  of  interactive  multimedia  teaching  units  on  the  quality  of  knowledge. 


Introduction 

The  old  Chinese  saying,  "a  picture  is  worth  a thousand  words",  has  been  proven  many  times  by  modem 
neuropsychological  studies.  Human  beings  can  memorize  20%  of  what  they  read,  and  30%  of  what  they  hear,  but 
40%  of  what  they  see,  55%  of  what  they  say,  and  60%  through  active  involvement;  however,  when  all  of  the  above 
sensory  channels  are  used  simultaneously,  this  results  in  the  memorization  of  90%  of  the  content  (Rose,  1993). 
According  to  the  Gardner  theory  of  multiple  intelligences  (Veenema  & Gardner,  1996),  we  possess  a number  of 
relatively  independent  types  of  intelligences:  linguistic,  logical,  musical,  kinesthetic,  spatial,  inter-  and  intrapersonal. 
Each  individual  has  the  natural  ability  to  use  some  of  his  sensory  systems  better  than  others.  This  shows  that  many 
different  communication  channels  have  to  be  used  to  achieve  similar  objectives  with  most  of  the  learners.  Spatial 
intelligence  is  commonly  linked  to  visualisation  and  can,  like  all  intelligences,  be  improved  by  various  activities 
(e.g.  in  chemistry  through  use  of  models  of  chemical  structures),  which  should  be  increasingly  incorporated  in  the 
teaching  and  learning  process. 

In  chemistry,  three  levels  of  visualisation  are  defined,  which  we  explain  by  the  example  of  chemical  experiments.  At 
the  macro  level,  the  real  world  is  representable  by  photographs,  drawings  or  video  clips  of  the  laboratory 
experiments.  The  3D  models  of  chemical  structures  or  animations  of  chemical  changes  represent  the  micro  level.  At 
the  symbolic  level,  chemical  change  is  visualised  by  chemical  equations  or  graphs.  In  order  to  achieve  the  learning 
objective,  all  three  levels  have  to  be  correlated. 

In  this  paper  we  present  our  work  on  the  design,  development  and  evaluation  of  interactive  multimedia  teaching 
units  and  web  based  teaching  resources  to  be  used  at  the  primary  and  high  school  level.  They  can  be  used  either  for 
individual  learning  or  in  parallel  with  the  traditional  teaching  tools  in  the  classroom  situation. 

Development  of  multimedia  chemistry  teaching  materials 

In  the  design  and  implementation  phase  of  the  underlying  CD-ROM  applications  of  teaching  materials  all  three 
different  levels  of  visualisation  in  chemistry  are  taken  into  account.  By  using  video,  animations,  and  simulations 
coupled  with  interactive  questions,  games  and  problem-solving  tasks,  the  users  have,  e.g.  in  the  case  of  a chemical 
reaction,  the  possibility  to  observe  chemical  changes  in  reactants  and  products  as  well  as  energy  changes,  to  follow 
them  step-by-step,  and  then  evaluate  their  observations.  We  try  to  simultaneously  activate  as  many  of  their  receptor 
channels  as  possible,  and  the  use  of  nonvisual  and  static  elements  is  kept  to  the  minimum;  the  learning  and  testing 
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segments  are  tightly  interlaced.  Through  communication  - either  live  discussion  with  a teacher  or  via  guided  online 
discussion  groups  - the  concepts  learned  are  further  clarified  and  relevant  information  is  shared  among  the  users. 

The  use  of  existing  multimedia  teaching  materials  in  schools  is  supported  by  the  intensive  in-service  training 
seminars  for  chemistry  teachers.  Teachers  become  prepared  for  active  involvement  in  the  development  of  teaching 
materials,  produced  by  the  University.  To  facilitate  the  needs  of  teachers  and  students,  a specialized  chemistry 
website  named  Keminfo  was  established.  It  includes  a module  on  the  visualisation  of  chemical  structures  and 
processes,  together  with  tutorials  for  existing  high  quality  educational  multimedia  CD-ROMs.  It  also  provides  the 
users  with  original  interactive  teaching  materials,  prepared  either  by  teachers,  through  small-scale  projects,  or  by  the 
university  lecturers.  The  teaching  materials  vary  in  size  and  complexity,  the  emphasis  being  on  the  applications 
using  visualisation  techniques  (e.g.  interactive  maps  of  chemical  reactions)  and  interactivity  (collections  of 
interactive  exercises  and  games). 

Evaluation  of  multimedia  chemistry  teaching  materials 

In  spite  of  the  rich  assortment  of  educational  CD-ROMs  for  chemistry,  very  little  has  been  published  regarding  the 
impact  of  multimedia  on  the  knowledge  and  motivation  of  students,  let  alone  the  cost-effectiveness  of  these 
products.  In  a research  project,  our  own  interactive  multimedia  CD-ROM,  "Light  and  Chemical  Change",  was 
evaluated.  The  impact  of  multimedia  on  cognitive,  motivational  and  motoric  development  of  students  was 
investigated  in  a qualitative  study.  The  research  involved  a sample  of  50  third-year  secondary  school  students,  who 
were  divided  into  two  proportional  groups,  namely  the  experimental  and  the  control  group.  Both  groups  were  pre- 
tested a week  prior  to  the  experiment  through  multiple-choice  and  open-answer  written  tests,  so  that  the  initial 
difference  between  the  groups  could  be  established.  The  experimental  group  then  worked  with  the  interactive  CD- 
ROM  in  pairs,  and  they  were  given  three  hours  to  learn  a specific  content,  presented  by  video  clips,  animations  and 
simulations,  accompanied  by  interactive  questions  with  intelligent  feedback.  This  group  did  not  use  any  other  source 
for  knowledge  consolidation.  Students  from  the  control  group  were  also  given  three  hours,  with  several  course 
books  available,  starting  with  the  teacher’s  explanation  of  the  subject  topic.  A week  after  the  experiment  both  groups 
took  a post-test  focused  on  understanding  the  concepts  at  the  microscopic  level  of  visualisation.  The  performance  of 
both  groups  before  and  after  the  learning  session  was  compared.  Whereas  the  groups  did  not  significantly  differ  at 
the  beginning,  the  results  of  the  post-test  show  significant  difference  in  favor  of  the  experimental  group,  which  can 
be  largely  attributed  to  the  visualisation  elements  included  in  the  CD-ROM  and  its  interactivity.  In  our  ongoing 
research  project,  a study  aimed  at  improving  the  efficiency  of  visual  elements  used  in  multimedia  teaching  units  is 
being  conducted.  We  are  investigating  the  correlation  between  the  spatial  abilities  of  the  individual  students  and 
their  ability  to  correctly  interpret  different  chemical  processes  visualised  by  application  of  the  diverse  tools. 

Conclusion 

Active  involvement  of  teachers  in  developing  high  quality  multimedia  teaching  materials  is  the  key  factor  for  their 
regular  use  in  schools.  The  research  on  the  effects  of  such  materials  is  needed  to  provide  the  developers  with  the 
feedback.  The  use  of  visualisation  elements  at  all  three  levels  is  crucial  in  chemistry  teaching,  however  the  learning 
process  has  to  be  supported  by  discussion  and  testing. 
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Abstract: 

A unique  service  organization,  consisting  exclusively  of  student  employees,  exists  within  the  University  of 
Wisconsin’s  Information  and  Media  Technologies  Division.  Known  as  Student  Technology  Services  (STS),  this 
student  managed  organization  is  tasked  with  the  operation  of  various  computer,  media  and  technology  related 
campus  services.  STS  employees  work  in  all  areas  of  I&MT  including  the  Campus  Computer  Labs,  Classroom 
Support,  Desktop  Support,  Help  Desk,  Network  Services,  Peer  Training,  Photo  Services,  Printing  Services, 
Shortcourses,  Technical  Resources,  Video  and  Multimedia  Production  Services,  TV  Engineering,  Distance 
Education  Technologies,  Visual  Design  Services,  and  Web  Maintenance  Services.  More  than  300  students  are 
expected  to  be  employed  in  STS  by  Fall  2000. 

Student  Technology  Services  (STS): 

Student  Technology  Services  (STS)  is  a student  staffed,  student  managed  organization  responsible  for  delivering 
technology  and  media-related  services  to  the  campus  community  of  the  University  of  Wisconsin-Milwaukee.  While 
empowered  student  IT  workers  are  employed  to  provide  a service,  they  are  at  the  same  time  engaged  in  a learning 
experience.  They  are  preparing  themselves  for  their  future  careers  through  "experiential  learning". 

Student  employees  of  STS  come  from  virtually  every  academic  major.  Whereas  many  universities  employ  primarily 
technical  majors  under  the  mistaken  belief  that  only  they  can  satisfy  their  campus  community's  technology  needs, 
STS  instead  looks  for  interpersonal  skills,  drive,  and  willingness  to  learn.  STS  has  discovered  that  customer  service 
and  management  skill  sets  have  become  as  important  as  technical  skills  in  the  delivery  of  information  and  media- 
related  technology  services.  Because  most  STS  student  employees  do  not  have  technical  backgrounds,  a carefully 
tailored  training  curriculum  was  developed  and  implemented.  The  purpose  of  the  curriculum  is  to  empower  STS 
student  employees  by  giving  them  the  technical  skills  they  will  need  to  excel  in  their  jobs  as  well  as  important 
professional  and  life  skills  they  will  need  to  excel  in  their  future  careers. 

The  STS  training  curriculum  becomes  an  education  when  the  learning  is  demonstrated  through  experience  (i.e., 
implemented  on  the  job)  and  when  the  experience  is  reflected  upon  (i.e.,  in  an  extensive  oral  and  written  evaluation.) 
Because  completing  the  multiple  levels  of  the  curriculum,  using  the  skill  sets  on  the  job  and  participating  in  the 
evaluation  process  are  all  mandatory  conditions  of  employment,  STS  provides  the  setting  for  experiential  learning  at 
institutions  of  higher  education.  Many  of  the  classes  in  the  STS  curriculum  are  developed  and  taught  by  STS 
students  themselves.  These  STS  Peer  Trainers  do  not  impose  upon  the  institution's  academic  teaching  mission,  yet 
are  an  integral  part  of  adding  value  to  the  learning  experience  of  STS  students.  STS  Peer  Trainers  are  evaluated  after 
each  class  they  teach  and  are  charged  with  making  the  necessary  adjustments  to  their  lesson  plans. 

In  addition  to  the  experiential  learning  provided  by  the  curriculum,  STS  students  are  encouraged  to  transfer  to  other 
departments  each  year  in  order  to  learn  and  practice  new  skills.  The  exposure  they  gain  in  these  other  areas 
strengthens  their  learning  experience  and  enables  them  to  identify  their  personal  and  professional  strengths  and 
weaknesses.  STS  students  are  required  to  maintain  a portfolio  of  their  diverse  work  experience  and  training.  The 
portfolio  serves  as  the  reflection  piece  to  the  classroom  and  on-the-job  experience,  thereby  further  adding  value  to 
TS'  philosophy  of  experiential  learning. 

STS’  focus  on  experiential  learning  has  been  met  with  welcomed  enthusiasm  at  UW-Milwaukee.  STS  student 
employees  are  sought  after  not  just  for  their  learned  technical  skills,  but  also  for  the  maturity  and  professionalism 


that  they  acquire  through  on-the-job  experience  and  reflection  pieces.  UWM  department  heads  specifically  request 
STS  student  employees  to  be  placed  in  their  departments  to  troubleshoot  and  to  maintain  department  web  pages. 

STS’  example  of  experiential  learning  is  now  being  introduced  to  the  26  other  campuses  within  the  University  of 
Wisconsin  System.  Wisconsin's  governor  recognized  the  benefits  of  experiential  learning  as  a means  of  preparing  a 
qualified  workforce  in  the  state  that  can  remain  competitive  even  in  the  fluid  environment  of  information 
technology.  The  Governor's  Office  recommended  increased  funding  for  training  IT  student  workers  throughout  the 
entire  University  of  Wisconsin  system.  His  decision  was  based  solely  on  the  UWM  STS  model  and  its  success.  The 
biennium  budget  was  signed  into  law  in  October,  1999  thus  propagating  the  STS  model  of  experiential  learning 
throughout  the  state. 

The  STS  program  has  generated  strong  interest  from  many  academic  and  non-academic  partners.  We  are  exploring 
relationships  with  public  school  systems  and  with  leaders  in  private  industry  to  provide: 

• A pre-college  program  to  create  work  experience  and  training  in  technology  for  high  school  students. 

• A similar  work  program  focused  on  the  two  year  technical  college  in  Milwaukee 

• Summer  internships  and  potential  scholarships  for  the  STS  student  employees  at  UWM 

• Meaningful  industry  relationships  for  students  leading  to  employment  upon  graduation. 

We  have  met  with  enthusiasm  and  willingness  by  all  participants  to  work  together  on  these  new  relationships.  We 
look  forward  to  a greatly  enhanced  student  empowerment  program  when  the  new  partnerships  are  in  place. 

Internet  Connection 

For  additional  information,  please  visit  the  STS  web  site  at:  www.uwm.edu/IMT/STS 
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Abstract:  This  paper  is  a report  of  an  ongoing  longitudinal  evaluation  assessing  the  effect  of 
level  of  technology  training  on  teachers’  attitudes  and  comfort  toward  using  and  integrating 
technology  into  the  curriculum.  Multivariate  and  univariate  analyses  were  used  on  an  initial 
sample  of  54  K-12  public  and  private  school  teachers  from  a Northeast  state  undergoing  two 
levels  of  technology  training  who  participated  as  part  of  a Technology  Literacy  Challenge 
Grant.  Initial  findings  support  and  extend  previous  research  that  shows  teachers  become  more 
comfortable  with  using  and  integrating  technology,  and  possess  more  positive  attitudes  toward 
technology,  as  the  extent  of  training  increases. 


Introduction 

Although  over  8.5  million  computers  are  being  used  for  instruction  in  the  U.S.,  and  85%  of  K-12 
teachers  have  access  to  computers  and  other  technology  (Survey  finds...,  1995),  only  half  of  all  teachers  report 
using  computers  for  instruction  (Hunt  & Bohlin,  1995).  Social  learning  theory  (Bandura,  1977)  predicts  that 
failure  of  teachers  to  use  computers  will  vicariously  teach  students  not  to  use  computers.  This  lack  of  computer 
use  may  be  related  to  attitudes  (e.g.,  anxiety)  toward  technology  (Pina  & Harris,  1993).  Although  !4  of  all 
people  who  use  computers  experience  some  degree  of  “computerphobia”  (Rosen  & Maguire,  1990),  when 
teachers  experience  it,  this  interferes  with  their  ability  to  integrate  technology  into  the  curriculum  (Hunt  & 
Bohlin,  1995).  The  strongest  predictor  of  computerphobia  is  experience  (Rosen  & Weil,  1995).  Limited 
research  has  shown  anxiety  towards  computers  decreases  after  training  and  practice  (Cambre  & Cook,  1 987), 
yet  only  29%  of  teachers  report  having  more  than  five  hours  of  technology  training  in  the  past  year 
(Technology  Counts  '99,  1999). 


The  Study 

The  purpose  of  this  study  was  to  investigate  the  effect  of  level  of  technology  training  on  teachers’  attitudes 
toward,  confidence  using,  and  comfort  with  integrating  technology.  Attitudes  were  represented  by  three  constructs: 
anxiety,  avoidance,  and  fear  of  stereotyping.  Confidence  was  measured  in  three  areas:  software,  hardware,  and 
communication  tools.  Comfort  with  integration  was  assessed  by  participants  indicating  one  of  six  stages  (awareness, 
learning  the  process,  understanding  and  application,  familiarity  and  confidence,  adaptation  to  other  contexts,  and 
creative  application  to  new  contexts).  Training  occurred  at  two  levels  (Level  I;  5 days  training  on  using  Power  Point  or 
HyperStudio  in  the  classroom:  Level  II;  Level  I training  plus  3 additional  days  training  in  Web  page  design  for 
curriculum  use).  Participants  were  54  public  and  private  school  educators  participating  in  a regional  Technology 
Challenge  Grant  training  program  in  a Northeast  state.  Twenty-six  educators  experienced  Level  I training  while  28 
educators  experienced  Level  II  training.  Over  300  educators  are  expected  to  be  trained  at  both  levels  by  the  end  of  this 
grant  year.  An  end-of-year  paper/pencil  survey  was  administered  to  the  first  cohorts  of  educators  experiencing  both 
Level  1 and  Level  II  training  so  far  this  school  year.  Ten  items  measured  attitudes  toward  technology  (anxiety, 
avoidance,  and  fear  of  stereotyping)  on  a six-point,  Likert-type  scale  (l=strongly  agree,  6=strongly  disagree).  Eighteen 
items  measured  confidence  in  using  technology  (hardware,  software,  communication  tools)  on  a six-point  scale 
(l=extremely  confident,  6=not  at  all  confident).  The  Stages  of  Adoption  Scale  (Knezek  & Christensen,  1999) 
measured  comfort  with  using  and  integrating  technology  into  the  curriculum. 
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Findings 


A multivariate  analysis  of  variance  (MANOVA)  was  used  to  analyze  the  data.  The  fixed,  categorical 
independent  variable  was  level  of  technology  training  (Level  1 and  Level  II).  The  continuous,  random 
dependent  variables  were  confidence  with  using  technology  (software,  hardware,  and  communication  tools), 
and  attitudes  toward  technology  (anxiety,  fear  of  stereotyping,  and  avoidance).  Findings  indicated  no 
significant  difference  for  level  of  training  on  confidence  in  using  technology  (F=1.26;  df=3,37;  p>.05). 
However,  a significant  difference  was  found  for  the  effect  of  level  of  training  on  the  construct  of  attitudes 
toward  technology  (F=2.83;  df=3,50;  p<.05).  Examination  of  the  univariate  Fs  indicated  a significant  portion 
of  the  construct  was  avoidance  (F=6.78;  df=3,50;  p<.05)  and  anxiety  (F=6.6I;  df=3,50;  p<.05),  accounting  for 
15%  of  the  total  variance  (Wilkes  lambda=.85).  A univariate  analysis  of  variance  (ANOVA)  was  used  to 
analyze  the  effect  of  the  fixed,  categorical  independent  variable  (level  of  training)  on  the  continuous,  random 
dependent  variable  of  comfort  using  and  integrating  technology  (six  stages:  awareness,  learning  the  process, 
understanding  and  application,  familiarity  and  confidence,  adaptation  to  other  contexts,  and  creative  application 
to  new  contexts).  A significant  difference  was  found  for  level  of  training  (F=5.00,  df=l,50,  p<.05)  on  stage  of 
comfort.  Strength  of  association  showed  that  nine  percent  of  the  variability  in  level  of  comfort  could  be 
accounted  for  by  level  of  training.  Those  with  more  training  were  more  comfortable  with  integration. 


Conclusion 

These  findings  support  previous  research  that  shows  teachers  who  receive  training  in  using  and 
integrating  technology  into  the  curriculum  experience  less  anxiety  toward,  and  more  comfort  with,  that 
technology.  This  study  adds  to  existing  literature  by  showing  that  additional  training  with  more  advanced 
technological  software  further  decreases  computer  anxiety  and  increases  comfort  with  integrating  technology 
above  and  beyond  less,  and  lower-level,  training.  If  teachers  are  expected  to  integrate  technology  into  the 
curriculum  and  model  appropriate  technology  utilization  behavior  for  their  students,  they  need  more  training 
than  the  levels  currently  being  received  by  most  educators. 
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Abstract:  The  Vocational  Education  and  Training  (VET)  sector  in  Australia  is  coming  under 
increasing  pressure,  both  nationally  and  internationally,  to  deliver  courses  in  flexible  ways  that 
incorporate  the  latest  in  information  and  communications  technologies  such  as  the  Internet  and 
the  World  Wide  Web.  LearnScope  is  a national  initiative  funded  by  the  Australian  National 
Training  Authority  (ANTA).  Its  aim  is  to  provide  professional  development  support  to 
individuals  and  work  groups  in  order  to  develop  skills  and  capabilities  for  the  online  environment 

A preliminary  study  of  LearnScope’ s Virtual  Learning  Community  (VLC)  was  undertaken  using 
a questionnaire  based  survey  plus  analysis  of  transcripts  of  forum  discussions  and  published 
stories.  The  results  highlight  the  complexity  of  the  role  of  providing  professional  development 
for  establishing  online  learning  communities.  The  major  areas  of  Technical  Skills  and  Computer 
Literacy,  Writing  and  Communication  Skills,  issues  of  Interactivity,  and  the  development  of 
Self-related  attributes  are  identified  as  important  if  the  VET  sector  is  to  build  a critical  mass  of 
creative,  capable  instructors  to  work  in  the  online  environment. 


The  Changing  Context  of  Vocational  Education  and  Training 

As  the  geographical  boundaries  of  Vocational  Education  and  Training  (VET)  in  Australia  become  blurred,  both 
nationally  and  internationally,  the  competitive  edge  of  providers.  Registered  Training  Organisations  (RTO’s), 
will  not  only  be  the  courses  and  training  programs  they  offer  but  the  quality  of  instructor  support  and  setwicc 
provided  to  the  learner.  Information  and  communications  technologies  (ICT)  such  as  the  Internet  and  the  World 
Wide  Web  has  seen  online  interactivity  emerge  as  a critical  factor  in  delivering  learning  materials  and 
supporting  learners  and  instructors.  A series  of  Australian  National  Training  Authority  (ANTA)  reports  (1994, 
97  and  98)  provide  details  to  developments  in  this  area. 

Traditionally  VET  practitioners,  in  particular  those  in  Technical  and  Further  Education  (TAPE)  Colleges,  have 
been  provided  with  professional  development  to  update  their  skills.  In  recent  years  this  has  been  through  work 
based  learning  using  a range  of  processes  such  as  mentoring,  coaching,  workshops,  discussion  groups, 
conferences  and  site  visits.  In  the  area  of  ICT,  professional  development  programs  have  primarily  focussed  on 
technical  skills  development,  that  is,  learning  how  to  use  the  technology  without  focusing  on  the  ‘deeper’  level 
of  critically  understanding  the  benefits  and  restrictions  of  using  these  tools  in  the  learning  environment. 

LearnScope  and  its  Virtual  Learning  Community 

LearnScope  is  a national  professional  development  initiative  that  is  available  to  work  teams  and  individuals 
within  RTO’s.  It  is  focussed  on  the  development  of  skills  to  deliver  training  in  increasingly  flexible  ways  using 
new  ICT’s.  The  LearnScope  home  page  provides  more  details:  http://www.leamscope.anta.gov.au/cover.html 

For  the  individual  instructors  it  is  LeamScope’s  Virtual  Learning  Community  (VLC)  that  provides  a range 
of  activities  in  several  zones.  These  activities  include: 

■ About  LearnScope  ~ an  information  about  the  national  LearnScope  project,  guidelines  and  application 
forms  and  contact  people  within  the  project. 

■ Key  Issues  - an  information  area  which  provides  clustered  groupings  of  key  issues  relating  to  flexible 
delivery.  This  includes  resources,  text,  access  to  software,  people  or  web  sites. 

■ Project  Gallery  - the  area  where  LearnScope  teams  publish  their  plans,  interim  and  final  reports. 

■ Expert  Spruik  - an  area  to  introduce  key  people  who  will  contribute  to  the  LearnScope  VET  community. 


■ RantZone  - an  area  for  people  to  practice  their  publishing  skills,  request  information  or  make  a statement. 

■ Evaluation  - an  area  for  the  LeamScope’s  project  evaluator  to  submit  information  about  evaluation  issues 

■ What 's  New  - a frequently  changing  area  which  provides  a summary  of  key  information  for  all  users. 

■ Forums  - an  area  for  both  public  and  private  discussions. 

■ Chat  Room  - an  area  for  planned  synchronous  discussions. 

While  the  VLC  is  primarily  for  the  support  of  LeamScope  project  teams,  anyone  interested  in  the  application  of 
new  learning  technologies  to  achieve  more  flexible  learning  and  professional  development  within  the  vocational 
education  and  training  sector  is  welcome  to  register  and  participate. 

The  Survey:  Methodology  and  Sample 

Data  was  gathered  in  1999  during  two  rounds  of  LeamScope  projects  and  in  two  formats:  a questionnaire,  and 
forum  discussions  and  published  stories  (Weathereley,  2000).  A total  of  58  questionnaires  were  distributed  and 
33  were  returned  giving  a response  rate  of  57%.  A total  of  153  postings  to  5 forums  (average  14,  range  10  to  21 
participants  per  fomm)  as  well  as  336  published  stories  were  also  collected.  These  data  was  then  used  to; 

(i)  Identify  the  reactions,  feelings  and  thoughts  to  using  a virtual  learning  community  in  the  context  of  the 
national  VET  system  and  to  gain  insight  as  to  the  level  of  experience  of  users. 

(ii)  Identify  how  VET  practitioners  used  an  asynchronous  fomm  area  set  up  specifically  for  learning. 

Findings  and  Recommendations 

It  is  recommended  that  for  instmctors  or  professional  development  planners  who  are  engaging  in  developing 
online  learning  plans  that  the  following  areas  of  skills  development  and  capabilities  must  be  addressed: 

(i)  Technical  Skills  and  Computer  Literacy;  To  achieve  the  best  possible  learning  environment  it  is  necessary 
to  go  beyond  the  simplistic  operational  understanding  of  ICT.  It  is  important  to  have  the  capability  to  make 
critical  judgements  about  its  appropriateness  and  to  develop  ideas  about  teaching  and  learning  within  the 
environment.  The  challenge  for  the  national  VET  system  is  getting  more  people  involved  in  a professional 
community  which  as  Spitzer  and  Wedding  (1995)  says  “...requires  the  active  and  willing  participation  of 
teachers  in  the  process  of  examining,  reflecting  on,  experimenting  with,  and  ultimately  changing  the  way  they 
practice  - in  the  context  of  a professional  community  that  allows  them  to  enrich  their  understanding  of  subject 
matter  and  to  consider  issues  about  how  students  learn.” 

(ii)  Writing  Skill  and  Communication  Style:  For  online  learning  communities  to  be  successful,  the  writing 
and  language  skill  must  be  conducive  to  the  audiences  ability  so  that  group  interaction,  sharing  and  participation 
can  lead  to  an  environment  based  on  relationship  development. 

(iii)  Interactivity 

(iv)  Understanding  the  range  of  interactive  possibilities  and  promoting  an  environment  which  encourages  active 
interaction  online  between  people  is  vital.  Understanding  the  significance  of  the  human  medium  and  the  levels 
of  interactivity  will  provide  teachers  with  the  potential  to  make  critical  judgements  about  student  learning. 

(v)  Self-related  attributes;  Developing  self-related  attributes  is  significant  in  professional  development  for  the 
online  learning  environment.  Teachers  need  to  identify,  develop  and  model  these  attributes.  Self-related 
attributes  should  also  be  an  important  aspect  of  learner  development. 

(vi)  Teaching  and  Learning:  Knowledge  building  in  order  to  gain  maximum  benefits  of  an  online  community 
and  to  provide  a quality  learning  environment,  requires  a range  of  strategies.  Based  on  the  constructivist  model 
where  learning  builds  on  previous  professional  and  personal  experiences,  the  importance  of  linking  practice 
with  discussion,  dialogue  and  debate  in  this  new  learning  field  is  critical.  Professional  development  activities 
need  to  incorporate  pedagogical  dimensions.  These  recommendations  must  be  incorporated  into  instructor’s 
professional  development  plans  otherwise  a quality  online  learning  environment  will  not  be  achieved. 
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Abstract:  The  purpose  of  this  study  is  to  explore  students’  perceptions  of  the  use  of  World  Wide  Web  (WWW)  technology  as  an 
instructional  resource  specifically  customized  to  support  teaching  and  learning  practices  on  campus-based  classes.  While  there  are  many 
efforts  to  design  meaningful  learning  environments  there  is  very  little  current  instructional  design  literature  on  fostering  and  support  of 
Web-based  environments.  This  study  examined  college  students’  perspectives  on  the  usefulness  of  specific  Web  site  functions  used  for 
teaching  and  learning  purposes,  in  an  effort  to  optimize  the  use  of  Web-based  instructional  support  resources. 

Review  of  Relevant  Literature 

The  1999  Campus  Computing  Project’s  (CCP)  national  annual  survey  of  information  technology  in  higher 
education  reported  that  many  higher  education  institutions  are  providing  more  services  via  the  WWW  (CCP,  1999), 

For  example,  more  than  half  (69,5%)  of  these  institutions  provide  online  undergraduate  applications  on  their  Web 
sites.  More  than  three-fourths  (77,3%)  make  the  course  catalog  available  online,  A quarter  (25%)  of  the  institutions 
make  library-based  course  reserves  readings  available  online  and  almost  a half  (46,5%)  offer  one  or  more  full 
college  courses  online  via  the  Internet  and  the  WWW.  Similarly,  the  percentages  of  college  courses  using  the 
WWW  in  the  syllabus  rose  from  7%  in  1995  to  28%  in  1999  (CCP  1996).  This  number  quadrupled  by  1999 
meaning  that  more  than  one  in  every  four  classes  has  a Web  page.  In  fact,  compared  to  e-mail  and  general  use  of 
Internet  resources,  use  of  Web  pages  has  shown  the  highest  rate  of  increase  over  the  six-year  period.  E-mail  had 
already  been  in  use  in  1994  rising  to  54%  by  1999  up  from  44%  in  1998,  and  20%  in  1995,  Khan  (1997)  provided  a 
definition  of  Web-based  instruction  used  for  the  purpose  of  this  study. 

Web-based  instruction  (WBI)  is  a hypermedia-based  instructional  program  which  utilizes  the 
attributes  and  resources  of  the  World  Wide  Web  to  create  a meaningful  learning  environment 
where  learning  is  fostered  and  supported  [italics  added],  (p.  6) 

At  a university  level,  the  Syracuse  University  Faculty  Syllabus  Web  site  available  at  http://svllabus.syr.edu  is  an 
example  of  a growing  list  of  campus-based  course  Web  sites.  For  the  purposes  of  this  study,  a preliminaiy  analysis 
of  1 75  Syracuse  University  Web  sites  was  conducted  during  the  fall  1 998  semester. 

Theoretical  Framework 

A conceptual  framework  was  developed  to  ground  and  guide  the  research  conducted  in  this  study.  There  is  evidence 
from  research  findings  that  certain  principles  exist  in  most  successful  teaching  and  learning  practices  (Chickering  and 
Gamson,  1987,  1991).  Second,  there  is  evidence  in  the  literature  that  the  application  of  instructional  design  principles  to  the 
teaching  and  learning  practices  leads  to  more  effective  learning  (Gagne’  1975,  Gagne',  Briggs,  & Wager,  1992).  One  of  the 
important  measures  of  effective  instruction  is  how  students  feel  about  the  instructional  process  (Dick  & Reiser,  1989),  Based 
on  the  above  premises,  one  can  make  the  assumption  that  a well-designed  instructional  Web  site  can  support  and  facilitate 
good  teaching  and  learning  practices  and  therefore  can  be  perceived  as  a valuable  instructional  resource  for  both  faculty  and 
students. 

Purpose  of  Study  and  Research  Questions 

The  purpose  of  this  study  was  to  examine  the  opinions  and  perceptions  of  college  students'  use  of  the  course  Web 
site  as  an  instructional  resource  for  classroom-based  teaching  at  a private  university  in  the  northeast  of  the  United  States.  The 
focus  of  the  study  was  on  the  identification  of  Web  site  functions  that  students'  perceived  as  supporting  and  enhancing  of 
their  learning  experiences  by  exploring  students’  perceptions  of  the  usefulness  of  the  various  Web  site  functions.  The 
following  question  was  posited:  How  useful  did  students  perceive  their  course  Web  sites? 
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142  primarily  undergraduate  students  enrolled  in  nine  courses  that  employ  Web  sites  as  course  resources  responded 
to  a questionnaire  on  their  use  of  their  respective  course  Web  sites.  Initially,  all  Web  sites  were  first  screened  using 
an  instrument  developed  by  the  researcher  to  evaluate  the  Web  sites'  instructional  quality  and  design  based  on 
principles  of  instructional  design.  On  the  student  Web  site  perception  questionnaire  (SWPS),  functions  pertaining  to 
the  Usefulness  dimension  were  construct  validated  using  confirmatory  factor  analysis.  Two  indexes  were 
constructed  for  the  Usefulness  dimension  using  items  that  loaded  (>0.60)  on  these  two  factors:  GUIDE  and 
PREPARE.  The  first  represents  practice,  elicitation,  and  feedback  functions.  The  second  represents  functions  that 
facilitate  recall  of  prior  learning  and  enhance  retention  and  transfer  through  communication,  enrichment  and 
remediation.  All  functions  are  derived  from  Gagne’,  Briggs,  and  Wagers’  (1992)  nine  events  of  instruction  and 
Chickering  and  Ehrmann's  (1996)  principles  of  good  teaching  and  learning  practices  using  technology  as  lever. 

Results 


Results  showed  that  most  functions  were  useful  with  a mean  score  for  the  PREPARE  index  (n=132, 
M=1.87,  SD=0.74«  scale  0-4  where  4=highly  useful)  and  a higher  one  for  GUIDE  (n=131,  M=2.20,  SD=Q.7K  scale 
0-4  where  4=highly  useful). 

The  PREPARE  index  consisted  of  functions  including;  (a)  the  use  of  pictures,  tables,  diagrams,  etc.  to 
recall  or  present  new  information  (n=  78,  M=2.79,  SD=1 .00);  (b)  the  additional  links  and  information  on  the  course 
Web  site  for  further  study  (n=  96,  M=2.43,  SD=1.16k  (c)  the  links  to  review/pre-requisite  material  to  help  recall 
(n=91,  M=1.87,  SD=0.74),  and  (d)  overall  usefulness  of  the  course  Web  site  (n=132,  M=2.63,  SD=1 .13). 

The  GUIDE  index  consisted  of  functions  including:  (a)  the  opportunity  to  ask  questions  online  (e-mail, 
listservs,  hypermail,  etc.  (n=  112,  M=2.74,  SD=1.00):  (b)  posting  graded  assignments/Home  Work  on  the  Web 
(n=64,  M=2.66,  SD=1.30):  (c)  online  practice  assignments  (n=47,  M=2.64,  SD=0.85);  (d)  online  feedback  on 
various  assignments  (n=  73.  M=2.60.  SD=1.00):  (e)  situations,  case  studies,  formulas,  problems,  simulations,  etc. 
posted  on  the  course  Web  site  to  improve  learning  (n=  65,  M=2.52,  SD=0.99):  (f)  online  self-assessment  quizzes  (n= 
49,  M=2.5 1,  SD=1 .23);  and  (g)  instructions  on  how  to  navigate  the  Web  site  and  download  software,  get  an  account, 
subscribe  to  a listserv  (n=  100,  M=2.30,  SD=1.10). 

Discussion 

Key  findings  from  the  study  are  indicated  below  using  Chickering  and  Ehrmann’s  (1996)  seven  principles  of  good 
teaching  and  learning  practices  for  undergraduate  education  and  Gagne’,  Briggs,  and  Wager’s  (1992)  nine  events  of 
instruction  as  the  framework  for  discussion.  Findings  for  the  PREPARE  index  related  to  assisting  students  to  recall 
information  and  to  transfer  new  knowledge  were  supported  by  functions  representing  two  principles:  (a)  Good 
practice  gives  prompt  feedback  and  (b)  Good  practice  encourages  contacts  between  students  and  faculty  in  that  the 
most  useful  functions  were  those  that  enabled  students  to  communicate  with  their  instructors  and  to  receive  timely 
feedback.  Electronic  mail,  computer  conferencing,  and  the  WWW  increase  opportunities  for  students  and  faculty  to 
interact  and  communicate  much  more  efficiently  than  before.  In  addition,  the  nature  of  communication  becomes 
more  thoughtful,  deliberate,  convenient  and  safe.  Moreover,  the  negative  aspects  of  face-to  face  confrontation  are 
lessened.  Findings  from  the  GUIDE  index  indicated  that  Gagne’s  et.  al.  (1992)  instructional  events  pertaining  to 
presentation,  practice,  feedback,  and  elicitation  of  new  knowledge  and  skills  were  overall  most  useful  functions. 
The  multimedia  capabilities  of  the  Web  allow  for  an  almost  seamless  integration  of  email  capabilities,  video  clips, 
sound,  images,  and  animation  thereby  allowing  for  multiple  modes  of  communication  through  feedback  and 
assessments.  In  conclusion,  in  terms  of  theoretical  implications,  the  study  supported  the  usability  of  Gagne's  events 
of  instruction  in  particular  the  ones  related  to  presentation,  practice,  elicitation,  and  feedback,  as  a viable  means  of 
designing  course  Web  sites  to  support  teaching  and  learning  practices.  The  study  also  provided  empirical  findings 
to  support  Chickering  and  Ehrmann’s  (1996)  principles  of  good  teaching  and  learning  practices  in  particular  the  ones 
related  to  prompt  feedback  and  student-faculty  interactions.  It  is  recommended  that  course  Web  site  design  could  be 
optimized  to  include  more  instructional  functions  such  as  elicitation  of  responses,  practice,  feedback,  and  increased 
student- faculty  interactions  using  the  interactive  and  visual  and  auditory  display  capabilities  of  the  medium. 
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Abstract;  This  work  in  progress  short  paper  is  a report  on  subjects'  evaluations  of  a 
web-based  language  course's  motivational  aspects  through  WebMAC,  Website 
Motivational  Analysis  Checklist,  developed  by  Small  and  Amone  (1998).  The  subjects 
were  26  adult  students  who  participated  in  a basic  online  English  course  for  Internet 
users,  focused  on  informal  asynchronous  and  synchronous  writing  and  communication 
skills,  called  Surfing  and  Learning.  This  course  is  designed  by  teachers  and  researchers 
of  the  EDULANG  research  group.  The  instrument  measured  the  level  of  four  motivation 
factors  which  provided  the  results  for  two  dimensions  within  an  Expectancy-Value 
framework:  Value  and  Expectancy  for  Success.  The  evaluation  results  showed  a high 
level  of  Value  dimension  and  Expectancy  for  Success,  although  factors  linked  to 
Expectancy  for  Success,  such  as  Organization  and  Ease  of  Use  showed  more  variation 
in  scores  than  those  linked  io  Value,  such  as  Meaning.  Although  WebMAC  seems  to  be 
sensitive  to  student  motivation,  other  kinds  of  data  are  required  to  a complete  course 
evaluation.  It  will  continue  to  be  used  for  the  ongoing  evaluation  of  Surfing  and 
Learning  and  other  courses  offered  by  EDULANG. 


Introduction 

Most  researchers  agree  that  motivation  is  a difficult  construct  to  work  with  (Weiner,  1990). 
Wlodkowski  (1999)  says  that  beyond  the  general  understanding  of  motivation  as  a concept  that  explains 
why  people  think  and  behave  as  they  do,  "any  specific  discussion  of  the  meaning  of  motivation  brings  in  a 
cornucopia  of  differing  assumptions  and  terminology"(p.l).  According  to  Domyei  (1998)  motivation  is  a 
"process  whereby  a certain  amount  of  instigation  force  arises,  initiates  action,  and  persists  as  long  as  no 
other  force  comes  into  play  to  weaken  it  and  thereby  terminate  action,  or  until  the  planned  outcome  has 
been  reached."  (p.l  18).  He  also  agrees  with  the  idea  that  motivation  is  a multifaceted  factor  and  no  theory 
has  yet  been  able  to  represent  it  in  its  total  complexity.  Despite  difficulties,  instructional  designers 
continue  to  seek  motivational  frameworks  in  order  to  achieve  a better  design  in  tecnology  based  learning 
environments  such  as  the  WWW  (Cornell  & Martin,  1997).  Duchastel  (1997)  elects  two  main  models. 
One  is  Keller’s  ARCS  model,  considering  four  factors  in  motivation  to  learn:  (a)ttention,  (r)elevance, 
(c)onfidence,  and  (s)atisfaction.  The  other  one  is  Malone’s  CFC  framework  for  instrinsically  motivated 
inscruction,  which  involves  three  factors:  (c)hallenge,  (f)antasy,  and  (c)uriosity.  ARCS  guided 
interventions  reduced  the  drop  out  rate,  by  half,  of  a Master's  degree  distance  course  intended  to  prepare 
adult  students  for  careers  in  the  areas  of  instructional  design,  job  performance  improvement,  etc. 
(Chyung,  Winiecki  and  Fenner,  1999).  Also  influenced  by  the  ARCS  model,  the  Website  Motivational 
Analysis  Checklist  - WebMAC  (Small  & Amone,  1998,  1999)  was  designed  to  assess  the  motivational 
quality  of  a website.  Small  (1997)  claims  that,  although  the  ARCS  model  is  rooted  in  a number  of 
motivational  theories,  its  main  basis  is  the  expectancy-value  framework  in  which  motivation  to  behave  is 
the  sum  of  individual’s  expectancy  for  success  in  a given  task  plus  the  value  attributed  to  success  in  the 
task.  Website  quality  is  measured  within  expectancy-value  framework:  the  user  is  motivated  to  remain  in 
the  website  if  it  has  value  and  if  the  user  expects  to  be  successful  in  the  website  environment.  Domyei 
(1998),  surveying  motivation  in  second  and  foreign  language  learning  (L2)  research  shows  that 
researchers  have  adopted  and  empirically  validated  some  concepts  originally  introduced  by  motivational 
approaches  in  mainstream  psychology,  such  as  expectancy-value  theories,  goal  theories  and  self- 
determination  theory.  Therefore,  WebMAC  was  chosen  here  to  evaluate  motivational  aspects  in  a L2 
web  based  instructional  design.  This  evaluation  is  part  of  a major  research  project  in  which  the  analysis  of 
interactive  flow  is  also  considered  (see  Collins'  paper). 
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The  subjects  were  26  students,  male  and  female,  age  29  to  64  , who  participated  in  a web-based 
language  course  in  1999.  The  course,  Surfing  and  Learning  (http://cogeae.uol.com.br/sal)  offers  a basic 
English  for  adult  Internet  users,  focusing  on  informal  asynchronous  and  synchronous  writing  and 
communication  skills.  It  was  designed  by  English  language  teachers,  members  of  EDULANG  research 
group,  under  the  coordination  of  Heloisa  Collins,  in  the  Graduate  Program  in  Applied  Linguistics  of  the 
Catholic  University  of  Sao  Paulo  (LAEL/PUCSP).  WebMAC  versions  3.1  and  4.0  were  translated  into 
Brazilian  Portuguese  and  edited  into  a HTML  format,  so  that  the  students  answered  the  checklist 
anonymously  at  the  end  of  the  course  and  submitted  it  through  a CGI  implemented  web  form.  The  answers 
were  scored  according  to  WebMAC  scoring  guidelines.  Both  versions  were  used  and  the  scores  obtained 
from  3.1  version  were  adjusted  to  version  4.0  factors.  WebMAC  is  composed  of  assertions  or  list  items  to 
be  rated  by  an  evaluator  on  a 4- point  level  of  agreement  scale.  The  32  items  in  version  4.0  measured  4 
factors:  1)  Stimulating  ; 2)  Meaningful;  3)  Organized  and  4)  Easy  to  use.  The  sum  of  first  two  equals  the 
Value  dimension  level;  the  sum  of  the  other  two  equals  the  Expectation  for  Success  dimension  level.  Zero 
to  8 points  for  a factor  means  the  site  needs  much  improvement  on  this  factor;  9 to  16,  it  needs  some 
improvement,  17  to  24  means  the  site  is  highly  motivating.  Version  4.0  contains  four  additional  unscored 
items:  two  yes/no  assertions  and  two  open  questions.  Most  of  list  items  are  the  same  in  both  versions. 
However,  they  are  not  related  to  the  same  factors.  In  both  versions,  each  subject  provides  two  main  scores 
that  are  plotted  on  a bidimensional  grid:  1)  Low  Value  to  High  Value  and  2)  Low  Expectation  for  Success 
to  High  Expectation  for  Success.  The  score  position  reflects  the  motivational  status  of  the  site  in  terms  of 
Value  and  Expectation  for  Success.  (fig.l)[figures  at  http://www.anise.f2s.com/anise-edmedia.html] 

Preliminary  flndings 

The  results  (fig.  1)  show  the  scores  on  the  average  to  high-level  area  of  Expectation  for  Success 
and  Value.  Only  two  students  showed  low  Expectation  for  Success  and  only  one  gave  low  scores  to  Value 
dimension  factors.  The  individuals'  scores  under  each  of  the  four  factors  (fig. 2 ) show  less  variation  for 
the  Meaning  factor  than  Organized  and  Easy  to  Use  factors.  The  answers  to  WebMAC  open  questions 
presented  in  4.0  version  revealed  some  students'  satisfactions  and  dissatisfactions.  Despite  high  levels  of 
Value  and  Expectation  for  Success,  the  amplitude  of  scores  in  each  of  the  four  factors  suggests  that  the 
course  can  be  improved  in  Organization  and  Ease  of  Use  factors.  The  interaction  in  communication  areas 
of  the  course  was  pointed  out  by  students  as  a particularly  strong  aspect  of  the  course  design.  The 
complaints  about  tools  and  dictionaries  seem  to  confirm  comparatively  lower  scores  of  Organization  and 
Ease  of  Use  factors.  Future  work  should  consider  the  use  of  WebMAC  during  the  course.  These  results 
can  be  discussed  in  terms  of  guidelines  for  motivating  language  learners  (Domyei,  1998).  Although  the 
present  instrument  seems  to  be  sensitive  to  student  motivation,  other  kinds  of  data  are  required  to  complete 
the  course  evaluation,  such  as  the  interaction  flow  analyses.  The  ongoing  evaluation  of  Surfing  and  Learning 
and  other  courses  offered  by  EDULANG,  such  as  Reading  and  Listening  Comprehension  for  public  high 
school  teachers,  will  continue  to  use  WebMAC. 
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Abstract:  U Wired  created  the  web-based  Catalyst  initiative  to  support  innovation  in  teaching 
through  technology.  To  scale  beyond  early  adopters.  Catalyst  has  developed  a new  strategy 
centered  on  collaborative  partnerships  with  campus  teaching  practitioners — learning,  technology, 
and  teaching  research  centers,  libraries,  and  departments — ^to  disseminate  innovative  uses  of 
technology  in  teaching  through  the  Catalyst  website. 


The  Problem  of  Supporting  Innovative  Teaching  with  Technology 

In  1994,  three  top-level  administrators  at  the  University  of  Washington  were  charged  by  the  Provost  to  “do 
something  about  technology.”  Collaborating  with  faculty,  librarians,  technologists,  and  students  they  quickly  moved 
beyond  a small  pilot,  rounding  up  partners  and  gathering  resources  from  wherever  they  could  to  focus  on  bringing 
technology  into  the  service  of  teaching  and  learning.  Today  the  organization  they  formed — U Wired — has  a 
permanent  budget,  a staff  of  eight  professionals  and  over  60  students,  and  has  extended  services  and  support  to 
thousands  of  students  and  faculty. 

As  higher  education  institutions  have  scrambled  to  keep  pace  with  rapid  changes  in  information  technology,  many 
campuses  have  established  faculty  technology  centers,  offering  workshops  and  training  to  interested  educators  and 
perhaps  even  supporting  faculty  technology  projects  to  transform  particular  courses.  These  efforts  have  generally 
targeted  "early  adopter"  faculty  who  are  eager  and  willing  to  teach  themselves  how  to  use  and  implement  new 
technologies.  Resources  are  thus  devoted  under  the  assumption  that  early  adopters  will  play  the  role  of  Johnny 
Apple-Seed,  spreading  innovation  throughout  their  home  departments.  Indeed,  UWired  inaugurated  its  Center  for 
Teaching,  Learning  and  Technology  (CTLT)  to  foster  and  support  instructional  innovation  in  this  manner. 

For  the  most  part,  however,  this  strategy  has  had  a negligible  impact  on  educational  transformation  through 
technology.  Beyond  the  few  early  adopters  are  legions  of  "wary  adopters"  who  look  for  easy  ways  to  improve  their 
teaching  through  technology  but  are  unwilling  to  match  the  time  commitment  of  the  “early  adopters.”  The  classic 
early  adopter  is  drawn  to  technology  for  its  own  sake,  reveling  in  the  "gee  whiz  factor"  of  new  technologies,  but 
"wary  adopters"  are  pragmatic.  Representing  the  mainstream  of  higher  education  faculty,  "wary  adopters"  will  use 
technology  only  when  it  can  be  demonstrated  that  doing  so  will  add  value  to  their  teaching  without  significantly 
reducing  time  already  allocated  to  teaching,  research,  and  service  activities.  Needless  to  say,  the  CTLT  found  it 
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quite  difficult  to  scale  up  support  services  while  also  ensuring  that  these  services  meet  the  often  different  needs  of 
"wary  adopters." 

Creating  a Catalyst  for  Innovative  Teaching  with  Technology 

The  need  to  expand  support  to  a much  wider  audience — and  to  do  so  without  a major  infusion  of  new  staff  or 
relocation  to  a larger  facility — was  clear.  How  to  do  so  was  not.  In  fact,  the  CTLT  faced  four  major  obstacles  posed 
by  the  needs  of  "wary  adopters":  (1)  the  absence  of  a coherent  framework  for  faculty  Web  publishing,  (2)  the 
absence  of  well  explained,  compelling  examples  of  enhanced  teaching  and  learning  through  the  Web,  (3)  the 
absence  of  convenient,  in-person  support  located  in  departments,  and  (4)  the  absence  of  a clear  incentive  for  faculty 
to  explore  the  use  of  technology  in  their  teaching.  After  an  intense  period  of  reflection  and  wide  ranging 
conversations  with  campus  stakeholders  and  faculty  focus  groups,  the  CTLT’s  leadership  team  developed  a new 
framework  for  faculty  support.  The  framework  depended  upon  an  instructional  support  Web  site  that  would  provide 
faculty  with  anytime-anywhere  resources  and  also  serve  as  the  foundation  for  redesigned  instructional  workshops 
and  one-on-one  consulting. 

The  Web  site.  Catalyst,  was  launched  in  1999  to  provide  this  support  while  placing  pedagogy  and  student  learning  at 
the  center  of  the  discussion  about  technology.  Catalyst  is  based  upon  four  assumptions:  (1)  faculty  want  just-in-time 
learning  and  support;  (2)  they  prefer  to  learn  at  their  own  pace,  in  their  local  environment;  (3)  the  Web  is  or  will 
become  the  vehicle  of  choice  for  just-in-time  information  and  learning;  and  (4)  distributed  support  personnel  are 
best  suited  to  make  the  critical  decisions  about  local  infrastructure  and  local  support.  Given  this,  the  key  function 
for  Catalyst — the  value  add — is  in  capturing,  focusing  and  disseminating  ideas,  resources,  and  tools  that  allow  both 
faculty  members  and  local  support  personnel  to  make  innovative  use  of  new  technologies  in  teaching  and  learning 
with  a minimum  of  duplicative  effort. 

Scaling  Innovation  through  Partnerships  and  Collaboration 

Through  Catalyst’,  U Wired  can  easily  identify  promising  practices  from  within  the  campus  and  then  use  them  to 
address  enterprise-wide  educational  needs.  As  an  easily  navigable  clearinghouse  for  information  on  the  use  of  the 
Web  in  teaching  and  learning,  the  site  has  met  faculty  needs  and  drawn  repeat  visits,  creating  a center  of  gravity  for 
innovation  in  teaching  with  technology  that  far  exceeds  the  physical  reach  of  U Wired  and  the  CTLT.  Catalyst  has 
four  basic  types  of  content:  (1)  profiles  of  educators  and  programs  that  provide  a vehicle  to  share  ideas  and 
experiences  with  technology,  humanizing  its  use  and,  hopefully  diffusing  teaching  innovation;  (2)  guides  to 
instructional  methods  and  technology  tasks  that  provide  a map  to  a wide  range  of  individual  documents  geared 
towards  improving  learning;  (3)  dynamic  content  and  frequently  updated  information  on  news  and  events  relevant  to 
teaching  with  technology;  and  (4)  instructional  tools  that  provide  a standard  mechanism  for  faculty  to  create 
interactive,  Web-based  instructional  modules  in  any  computing  environment  using  only  a Web  browser. 

As  envisioned,  departments  and  college  support  personnel  who  have  developed  materials  specific  to  their  local 
computing  environments — for  instance  how-to  documents  on  a particular  Web  editor  or  "one-off  Web-based 
instructional  tools — have  been  eager  to  see  their  work  repackaged  and  made  available  to  a much  larger  audience. 
More,  through  a partnership  with  a university-wide  initiative,  the  Program  for  Educational  Transformation  through 
Technology  (PETTT),  UWired  now  has  the  capacity  to  conduct  research  on  the  science  of  learning  and  disseminate 
information  via  Catalyst,  thus  promoting  the  thoughtful  exploration,  development,  assessment,  and  dissemination  of 
next-generation  technologies  and  strategies  for  teaching  and  learning. 

Catalyst  not  only  extends  UWired’s  reach,  but  it  also  improves  the  overall  quality  of  faculty  support,  providing 
department  support  personnel  with  a set  of  resources  they  can  draw  upon,  freeing  them  individually  tailor  services  to 
their  faculty.  By  aggressively  profiling  the  efforts  of  campus  educators.  Catalyst  makes  faculty  innovation  visible 
and  contributes  directly  to  the  vitality  of  the  teaching  community  on  campus.  In  its  first  year,  the  Catalyst  Web  site 
logged  half  a million  page  views;  it  now  averages  100,000  page  views  per  month. ^ On  the  UW  campus,  over  1000 
faculty  and  instructors  from  departments  ranging  from  Anesthesiology  to  Urban  Horticulture  have  implemented 
Catalyst  tools. 


’ See  http://depts.washington.edu/catalyst/home.html 
^ See  http://depts.washington.edu/catalyst/2000_report.pdf 
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Abstract:  Social  pressures  are  creating  the  necessity  for  a system  of  lifelong  learning.  Electronic 
distance  education  appears  to  be  the  optimal  system  for  meeting  the  educational  requirements  of 
learners.  While  educational  needs  have  shaped  the  development  of  the  electronic  learning 
environments  there  has  been  little  attention  paid  to  the  physical  and  psychological  health  issues  for 
the  electronic  learner.  Athabasca  University  (AU)  has  used  questionnaire  and  focus  groups  to 
examine  the  implications  for  learners  using  AU's  electronic  environments  for  learning. 


INTRODUCTION 


Distance  education  is  often  identified  as  a panacea  for  adult  learners  as  it  overcomes  geographical  barriers  that  often 
prevent  students  from  attending  universities  to  study.  With  the  advent  of  the  Internet  as  a readily  accessible  medium, 
web-based  courses  have  become  immensely  popular  with  providers  of  distance  education  around  the  world.  It  clear 
by  the  enthusiasm  with  which  computer-based  distance  delivery  has  been  embraced  that  society  is  willing  to  rely  on 
computer  technology  for  learning. 

At  Athabasca  University,  where  the  mandate  is  distance  education  , students  are  distributed  not  only  across  Canada 
where  the  University  is  located,  but  also  across  North  America  and  the  rest  of  the  world.  For  the  past  five  years 
Athabasca  University’s  Centre  for  Computing  Information  Systems  and  Mathematics  (CCIS)  has  offered  20 
undergraduate  Computing  Science  courses  over  the  World  Wide  Web  (WWW).  In  2001  it  will  be  offering  a Master 
of  Computing  and  Information  Systems  over  the  WWW.  The  Center  for  Nursing  and  Health  Studies  (CNHS)  has 
offered  a Master  of  Health  Studies  (MHS)  degree  over  the  WWW  since  September,  1999  and  in  less  than  a year  has 
over  200  graduate  students  enrolled  in  the  program. 

The  needs  and  capabilities  of  the  learners  are  our  first  considerations  in  the  development  of  an  electronic  learning 
environment.  In  our  developments  we  have  focussed  on  involving  the  learners  to  optimize  the  environment  for 
learning  (Holt,  Stauffer  & Jelica,  1999).  However,  until  recently  we  did  not  look  seriously  at  health  related  issues. 
Marcoulides,  Mayes,  and  Wiseman  (1995)  have  suggested  that  the  use  of  computers  in  the  workplace  can  result  in 
anxiety.  Similarly,  technology-based  learning  may  result  in  added  stress  among  learners.  There  are  other 
potentially  negative  implications  for  students  who  rely  on  computers  for  learning.  According  to  Faulkner  (1998) 
these  may  include  psychological  factors  such  as  alienation,  inadequacy,  lack  of  privacy,  loss  of  responsibility  and 
damage  to  self-image.  Furthemore,  Dix  et  all  (1998)  noted  that  physical  problems  include  repetitive  motion  strain, 
postural  fatigue,  eye  fatigue  and  less  frequently,  skin  rash,  reproductive  disorders,  and  cataracts. 
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In  order  to  examine  health  issues  related  to  the  use  of  computers  for  learning,  we  decided  to  survey  graduate 
students  in  our  Master  of  Health  Studies  Degree  and  undergraduate  students  in  Computing  Science,  Questionnaire 
data  has  been  collected  on  general  computer-related  habits,  any  health  problems  encountered,  and  their  ergonomic 
awareness  related  to  design  of  workspace,  posture,  usage,  and  exercise.  This  was  a large  exploratory  study  with  a 
questionnaire  of  19  items  distributed  to  1310  CCIS  Students  and  145  CNHS  students).  More  indepth  data  will  be 
gathered  by  focus  groups  which  are  underway— the  results  and  will  be  reported  more  fully  at  EdMedia  2000,  The 
more  interesting  questionnaire  results  analyzed  to  date  are  presented  in  this  session, 

CNHS  students  and  CClS  students  differed  on  a number  of  dimensions  including  graduate  versus  undergraduate, 
continuous  monthly  enrollment  versus  semester,  discipline,  and  different  WWW-based  learning  platforms.  Since  the 
purpose  of  the  study  is  to  consider  ergonomic  factors  in  the  design  of  electronic  curricula  for  both  CCIS  and  CNHS 
students,  we  have  not  focussed  on  inferential  statistics  and  differences  between  the  groups  but  upon  useful 
descriptive  statistics  within  each  group. 

The  return  rate  was  low  for  CCIS  students  (150  of  1310  questionnaires).  It  was  higher  for  CNHS  students  (41  of  145 
questionnaires).  The  low  return  rate  makes  the  results  more  difficult  to  interpret  as  students  with  health  related 
problems  may  be  more  likely  to  respond  giving  us  a biased  sample.  Nevertheless,  computer  usage  was  reported  to 
be  very  high  among  respondents.  Both  CNHS  students  (80,5%)  and  CCIS  students  (85,3%)  had  high  rates  of 
computer  use  in  general  and  relatively  high  use  for  education  other  than  the  current  courses  (43,9%  and  72,3%), 

The  mean  time  using  computers  was  24,2  hours  per  week  for  CNHS  students  and  32,2  hours  for  CCIS  students. 

There  was  a wide  range  of  usage  but  it  would  appear  these  are  high-use  groups  at  risk  for  computer  related  health 
problems, 

CNHS  students  and  CCIS  students  rated  themselves  respectively  as  somewhat  or  informed  (75,6%  and  63,4%)  about 
ergonomic  issues.  However,  a majority  of  all  students  did  not  make  use  of  special  ergonomically  sound  furniture 
and  at  least  half  of  both  CNHS  students  (53,7  %)  and  CCIS  students  (50,0%)  did  not  take  any  regular  breaks  while 
working  at  the  computer.  As  high  usage  groups  these  students  ought  to  be  more  aware  of  ergonomic  issues  and  in 
ideal  circumstances,  practice  more  ergonomically  sound  computer  usage. 

Both  CNHS  and  CCIS  students  reported  some  physical  problems  related  to  computer  use  (26,6%  and  26,8% 
respectively).  Problems  reported  include  carpal  tunnel  syndrome,  eye  strain,  and  back  strain.  Ironically,  although 
both  groups  reported  low  rates  of  psychological  problems  (9,8  % and  6%),  they  did  report  considerable  levels  of 
anxiety  about  their  use  of  computers.  Interestingly,  the  most  intense  anxiety  was  associated  with  issues  of  privacy 
and  security  with  70,7%  of  the  CNHS  students  and  65,1%  of  the  CCIS  students  reporting  slight  to  high  levels  of 
anxiety  in  this  area.  Other  areas  of  anxiety  included  the  cost  of  computer  technology  (46,3%  and  57,7%);  keeping 
current  with  software  (46.3%  and  40,3%);  loss  of  data  (43,9%  and  60,8%);  keeping  current  with  technology  (34,1% 
and  40,3%;  and  keeping  up  with  the  information  glut  on  the  Internet  (36,6  and  47,7).  High  computer  usage  does 
appear  to  affect  these  students’  social/family  life— 36,6%  of  the  CNHS  students  and  34,7%  had  complaints  from 
family  members  about  their  computer  usage.  This  finding  is  hard  to  interpret  in  lack  of  comparison  with  students 
learning  by  other  methods  of  education  but  it  certainly  warrants  further  attention. 

Of  the  CNHS  students  32.5%  reported  that  the  electronic  courses  aggravated  old  problems,  50.0%  said  they  created 
new  problems,  and  51.3%  had  suggestions  for  course  design.  Of  the  CCIS  students  12,7%  reported  that  the 
electronic  courses  aggravated  old  problems,  16,7%  said  they  created  new  problems,  and  30,7%  of  CCIS  had 
suggestions  for  course  design.  When  students  listed  concerns  most  of  them  were  related  to  computer  use  or  course 
design  rather  than  to  health  related  problems  explicitly  (other  than  perhaps  related  to  stress).  This  could  be  a 
function  of  students  being  so  engrossed  in  their  use  of  computers  for  learning  that  they  are  relatively  unaware  of 
health  problems.  We  are  conducting  further  focus  groups  to  further  investigate  the  factors  and  underlying  situations 
that  contribute  to  students  being  in  undesirable  physical  and  /or  psychological  danger  and  other  ameliorative 
approaches  they  might  use.  On  the  basis  of  findings  to  date  we  are  implementing  a Java/XML  based  printing  utility 
and  highlighting  ergonomic  tips  in  the  course  materials. 
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Abstract:  The  Columbus  Zoo  has  a Distance  Learning  Interactive  Field  Experience 
Teleconferencing  program  (CZDLIFE)  to  provide  a vehicle  to  enhance  standards-based  math  and 
science  instruction  in  Ohio’s  schools.  The  use  of  interactive  technology  in  schools  can  increase  the 
ability  to  use  contextual  settings  in  education  which  add  a level  of  reality  to  classroom  learning. 
Making  connections  through  planned  partnerships  outside  the  traditional  classroom  promotes 
curiosity,  creativity,  and  exploration.  Learning  in  an  informal  setting  helps  students  connect  and 
transfer  their  knowledge  base  as  they  engage  in  new  learning  opportunities.  Zoos  can  provide  an 
excellent  informal  context  to  ground  distance  learning  interactive  field  experiences. 


The  Project 

The  goal  of  the  Columbus  Zoo  Distance  Learning  Interactive  Field  Experience  (CZDLIFE)  is  to  offer  educational 
and  scientific  resources  to  enhance  standards-based  math  and  science  instruction  using  two-way  audio/video 
interactive  technologies.  The  CZDLIFE  provides  an  exciting  and  meaningful  curriculum  that  incorporates  a variety 
of  activities  for  all  students  at  the  distant  site.  The  instructional  design  of  the  project  was  grounded  in  contextual 
teaching  and  learning  theory  and  situated  learning  environments  (Brown  et  al.  1989,  Balas,  2000).  The  design 
emphasizes  the  use  of  several  pedagogical  strategies  blended  with  contextual  experiences  that  promote  real-time 
problem-solving  interaction.  These  experiential  activities  are  designed  to  promote  knowledge  creation  through 
interaction  with  zoo  facilitators  and  other  learners  (Thomson,  B.,  et  al.  1999).  The  field  experience  was  constructed 
using  six  key  areas:  1)  considerations  to  connections  to  the  curriculum,  2)  connection  to  multiple  content  areas,  3) 
inclusion  of  non-formal  educators  in  the  design  phase,  4)  developmentally  appropriate  activities,  5)  professional 
development  activities,  and  6)  follow-up  activities  (Francis,  1999). 

Research  Problem  and  Question 

The  Columbus  Zoo  has  experienced  a great  deal  of  success  implementing  on-site  and  outreach  educational 
programs.  The  recent  advent  of  the  use  of  two-way  audio/video  interactive  teleconferencing  as  a means  of  sharing 
the  zoo  resources  has  added  a new  and  complex  medium  for  educational  content  delivery.  Interactive 
teleconferencing  is  a unique  medium  that  requires  different  and  diverse  pedagogical  approaches  than  those  regularly 
used  with  on-site  visitors  and  outreach  populations.  The  main  benefit  of  two-way  interactive  teleconferences  is  that 
the  nature  of  the  system  provides  a level  of  intimacy  in  communication  that  is  not  apparent  in  other  forms  of 
distance  learning.  Two-way  real  time  video  transmission  of  information  implies  a new  definition  of  real-world 
context,  in  which  the  learning  environment,  although  video-mediated,  is  indeed  the  actual  environment  with  which 
the  learner  interacts  (Jonassen  et.  al.  1995).  Another  strength  of  the  technology  lies  in  its  synchronous  nature;  the 
teacher  and  learners  experience  parallel  delivery  and  reception  of  information  without  a time  delay. 

In  order  to  add  to  research  based  design,  the  main  question  of  the  study  is  How  do  you  design  effective  standard- 
based  two-way  synchronous  teleconferences  from  an  informal  education  setting? 
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Research  Design  and  Procedures 

Background  research  in  the  development  of  interactive  teleconferences  in  contextual  settings  such  as  the  zoo  is 
almost  non-existent.  The  study  involves  the  Columbus  Zoo,  in  Columbus  Ohio,  and  four  distance  learning  sites 
throughout  the  state.  The  four  sites  were  divided  among  three  schools  in  New  Lexington,  Stark  County,  and 
Richfield,  Ohio;  and  one  community-based  after-school  enrichment  program,  which  used  the  Cincinnati  Zoo  as  their 
teleconferencing  facility.  Five  elementary  school  teachers,  one  principal  , four  technology  coordinators  and  seven 
classes  were  used  for  data  collection.  The  student  population  ranged  from  second  to  fifth  grade  with  a combined 
total  of  140  students.  This  population  is  representative  of  Ohio’s  urban,  suburban  and  rural  students. 

Qualitative  data  was  gathered  through  surveys,  interviews,  observations,  field  notes,  and  researcher  reflections. 
Surveys  were  collected  immediately  after  the  CZDLIFE,  and  interviews  were  conducted  with  the  teachers  and 
technology  coordinators  prior  to  and  after  the  CZDLFE  teleconference.  The  qualitative  data  was  interpreted  through 
the  development  of  common  themes  generated  from  the  interviews  and  is  being  synthesized  into  assertions  about  the 
nature  of  the  experience. 

Preliminary  Results 

The  CZDLIFE  teleconferences  use  a view  of  learning  that  encourages  knowledge  creation  rather  than  a knowledge 
delivery.  For  most  learners  having  tactual  contextual  opportunities  is  critical  to  accommodate  for  their  learning 
styles  (Thomson,  B.,  et  al.  1999).  The  CZDLIFE  program  has  been  effective  in  providing  an  opportunity  to: 

1.  Bring  to  the  zoo  a wider  student  audience. 

2.  Help  meet  the  needs  of  students  who  are  unable  to  attend  on-site  programs. 

3.  Provide  real  time  face-to-face  interaction  between  classrooms  and  zoo  educators. 

4.  Involve  outside  speakers  who  would  otherwise  be  unavailable  to  regular  zoo  visitors. 

5.  Provide  an  opportunity  to  “export”  unique  exhibits  such  as  the  coral  reef  or  some  of  the 
endangered  animals  that  can  not  be  used  on  regular  zoo  outreach  programs. 

6.  Allow  “real  time”  2 way  audio  and  video  contact  between  students  and  the  facilitators  at 
the  zoo. 

7.  Incorporate  the  use  of  experiential  learning  activities,  live  action  and  pre-recorded  video 
to  present  interactions  carried  out  at  the  zoo  which  are  not  available  to  the  general  public. 

Implications 

The  results  and  conclusions  of  this  program  help  provide  a better  understanding  of  design  process  for  interactive 
distance  learning  field  experiences  in  informal  science  settings.  The  study  also  serves  as  one  of  the  few  studies  that 
examine  the  outcomes  of  using  two-way  interactive  distance  learning  between  elementary  classrooms  and  an 
contextual  learning  setting. 
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Abstract:  This  presentation  reports  on  our  current  and  on-going  R&D  efforts  towards  the 
design  and  development  of  multimedia,  web-based  courses  suitable  for  use  through  the 
Internet.  We  discuss  the  process  that  has  been  adopted  in  designing  the  interface  and  the 
tools  used.  An  important  issue  for  the  implementation  of  the  courses  is  the  use  of  streaming 
technology  for  the  reliable  transmission  of  multimedia  content  material  over  the  Internet. 
This  technology  allows  us  to  transmit  multimedia  data  in  a more  efficient  way  and  to 
introduce  interactive  elements  using  SMIL  (Synchronized  Multimedia  Integration  Language). 


Introduction 

Until  now,  the  traditional  educational  system  has  assumed  that  all  students  learn  identical  materials 
via  the  same  delivery  mode.  However,  it  is  known  that  each  student’s  intelligence  profile  and  learning  style  is 
unique  and  therefore,  the  traditional  lecture-based  method  of  classroom  instruction  cannot  accommodate  the 
large  number  of  diverse  learning  preferences.  In  this  context,  the  Internet  has  emerged  as  a new  means  for 
meeting  the  needs  of  individual  learning  diversity  and  has  introduced  significant  opportunities  for  the 
improvement  of  the  educational  process. 

However,  although  significant  investments  have  been  made  for  the  technological  support  of  this  new 
educational  environment,  similar  steps  have  not  been  taken  towards  the  creation  of  noteworthy  educational 
software.  Thus,  although  a vast  amount  of  electronic  educational  material  currently  exists  on  the  Internet,  it  is 
usually  in  the  form  of  simple  text  occasionally  combined  with  some  graphics.  Undoubtedly,  there  is  a lack  of 
genuinely  qualitative  educational  content.  This  is  due  mainly  to  the  fact  that  the  current  Internet  infrastructure 
is  incapable  of  transmitting  high-quality  multimedia  content  (a  problem  which  can  partially  be  addressed  by 
the  use  of  streaming  technology).  In  addition,  the  significant  expense  and  time  required  for  the  development  of 
such  material  are  serious  prohibiting  factors. 

This  paper  discusses  our  efforts  towards  the  design  and  development  of  interactive  multimedia,  web-based 
courses  suitable  for  use  through  the  Internet.  The  primary  characteristics  of  the  material  developed  are  the 
following: 

1 . It  is  dynamic  and  interactive. 

2.  It  supports  the  concept  of  collaborative  learning  i.e.  it  offers  opportunities  to  simulate  real  situations  of 
collaboration  among  various  groups  of  users. 

3.  It  can  be  transmitted  efficiently  to  users. 


Designing  the  interface 

The  appropriate  design  of  the  interface  depends  on  the  specific  context  that  it  is  intended  for.  To  this 
purpose,  an  effort  was  made  to  create  an  interface,  which  would  be  used  by  adult  graduate  and  post-graduate 
students  and  would  use  unambiguous  “metaphors”  for  the  effective  and  consistent  delivery  of  specific  concepts 
to  learners.  The  screen  was  divided  into  three  vertical  frames,  the  left  of  which  always  displays  the  contents  of 
the  course.  The  middle  frame  presents  each  individual  course  in  the  form  of  text  while  in  the  third  frame, 
accompanying  graphics,  video  or  animation,  can  be  viewed  by  the  user,  depending  on  the  requirements  of  the 
specific  material  being  worked  on.  It  is  important  to  emphasize  that  that  each  lesson  is  available  in  several 
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different  forms  of  delivery  i.e.  text,  narration,  etc. 


Implementation  of  the  educational  material 

On  a technical  level,  the  implementation  of  the  courses  was  based  on  the  use  of: 

a)  Streaming  technology 

b)  HTML,  JavaScript,  VBScript,  XML  (SMIL),  ASP 

Streaming  technology  for  the  efficient  transmission  of  multimedia  elements 

One  of  the  basic  problems  encountered  during  the  implementation  of  a multimedia,  web-based  course  is 
undoubtedly  the  limited  bandwidth  of  the  Internet.  This  is  perhaps  one  of  the  main  factors  that  justify  the  lack 
of  extremely  high-quality  multimedia  material.  The  greatest  amount  of  educational  software  existing  presently 
on  the  Internet  is  in  the  form  of  simple  text,  frequently  enriched  with  some  graphics.  This  form,  however, 
proves  to  be  inadequate. 

Streaming  technology  addresses  this  problem  in  the  following  way: 

Although,  the  conventional  way  of  transmitting  data  through  the  Internet  requires  that  the  entire  file  (e.g.  a 
video  file)  be  received  before  it  may  be  viewed  by  the  user,  this  is  not  a necessity  when  streaming  technology  is 
the  adopted  method.  In  the  latter  case,  the  information  is  presented  to  the  user  (e.g.  the  video  file  begins  to 
“play”)  even  though  the  remaining  information  is  still  being  received  and  it  becomes  possible  to  view  the 
multimedia  elements  almost  simultaneously  with  the  appearance  of  the  rest  of  the  HTML  document. 

A primary  advantage  of  this  kind  of  technology  is  that  it  is  possible  to  transmit  very  large  multimedia  files  in 
an  efficient  and  reliable  way  thus  resulting  in  the  preservation  of  the  learner’s  concentration. 

Streaming  technology  makes  use  of  the  RTSP  (Real  Time  Streaming  Protocol)  instead  of  the  HTTP  protocol. 
The  product  selected  to  deal  with  the  difficulties  associated  with  the  network  infrastructure  of  the  Internet  was 
the  RealServer  created  by  Real  Networks.  RealServer  makes  it  possible  to  incorporate  high-quality  multimedia 
elements  (RTSP  is  designed  to  stream  clips  that  have  time  lines)  within  a web-based  course.  Whether  the 
learner  has  an  ISDN  line  or  accesses  the  Internet  through  a low-bandwidth  connection,  s(he)  may  still  view  a 
course  that  comprises  a variety  of  multimedia  elements.  There  is  also  a possibility  for  the  dynamic  detection  of 
different  connection  speeds  between  the  client  and  the  server  so  as  to  achieve  improved  transmission. 

Emphasis  should  be  given  to  the  fact  that  it  is  possible  to  include  additional  interactive  elements  within  the 
educational  material  (sound,  video,  animation)  through  the  use  of  the  SMIL  language. 

Interactivity 

An  additional  item  of  concern  was  that  the  educational  material  should  be  enriched  with  a variety  of 
interactive  elements.  To  this  extent,  SMIL  (Synchronized  Multimedia  Integration  Language)  presents  notable 
interactive  characteristics. 

Based  on  XML,  this  language  reacts  upon  the  multimedia  elements  of  the  course,  which  are  transmitted 
through  the  use  of  streaming  technology. 

The  main  reasons  that  make  SMIL  so  important  are: 

a)  It  can  integrate  and  co-ordinate  many  diverse  types  of  multimedia  information.  We  can  capture  events  that 
are  generated  from  the  interaction  between  the  learner  and  the  multimedia  element.  This  is  achieved  through 
the  use  of  callback  methods. 

c)  It  can  be  considered  as  an  “open”,  platform-independent  technology,  which  is  based  on  W3C  XML. 
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Abstract:  Based  on  case-studies  of  edutainment  Multimedia  programmes  this  study  is  exploring 
the  affordances  of  the  contextual  elements  of  narrative  in  relation  to  interactive  Multimedia 
programmes,  the  possible  cognitive  costs  that  lack  of  canonical  narrative  elements  may  have  for 
the  user  and  the  particular  possibilities  they  may  offer  in  a learning  environment  in  which 
meaning,  experience  and  motivation  for  further  exploration  are  key  elements. 


Narrative  and  the  user's  construction  of  meaning 

Educational  or  edutainment  Multimedia  programs  are  offering  new  environments  for  learning  and  communication 
(Gjedde  1999a)  but  often  these  programs  are  not  employing  the  canonical  narrative  elements  that  can  be  found  in 
popularization  of  complex  information  in  television,  magazines  or  in  books.  Instead  there  has  been  a natural  interest 
in  exploring  the  non-linear  possibilities  that  the  interactive  media  is  offering.  This  has  been  done  while  letting  go  of 
the  canonical  narrative  elements  and  schemata’s  that  typically  has  a beginning  including  a notion  of  time,  place  and 
a protagonist,  a development  and  an  end  (Bartlett  1932  1995;  Labov  1972).  Research  on  the  relationship  between 
narrative  structure  and  cognitive  processes  like  memory  and  comprehension  has  indicated  that  there  tends  to  be 
better  memory  and  comprehension  for  texts  that  employ  a narrative  structure  than  texts  that  do  not  use  a narrative 
structure.(Thomdyke  1977;  Mandler  1984).  Narrative  theories  are  furthermore  pointing  to  its  importance  for 
experience,  identity  and  communication.  (Schank  1990;  Schankand  Abelson  1995;  Schank  1997). 

The  structuring  and  communicative  elements  of  narrative  are  often  not  used  in  educational  or  edutainment 
Multimedia  programs,  which  are  often  encyclopaedic  in  structure  and  if  using  other  structures  as  a motivating  factor, 
tend  to  use  games  that  often  do  not  have  the  basic  narrative  elements.  They  are  not  embedding  the  information 
offered  in  the  context  and  structure  of  a narrative,  but  it  is  instead  presented  in  an  abstract  cyberspace,  or  in  an 
encyclopedic  structure  that  does  not  provide  the  same  cognitive  and  experiential  level  of  support  for  the  user. 

This  lack  of  narrativity  can  have  consequences  for  the  ability  of  the  user’s  to  construct  meaning  from  the  seeming 
chaos  gathered  by  experiencing  information  that  is  presented  out  of  a context  or  a known  and  comprehensible 
universe. 

This  can  be  found  in  the  edutainment/educational  genre,  where  the  user  may  be  particularly  vulnerable  to  the  lack  of 
cognitive  support  for  the  organization  of  the  material,  that  a narrative  structure  may  provide. (Plowman  1996; 
Plowman  1998) 

The  user  may  also  lack  the  involvement,  which  the  narrative  can  create  through  an  experiential  dimension,  through 
the  unfoldment  of  a narrative,  with  the  possibilities  for  emotive  and  identificatory  involvement  and  a narrative  logic 
with  an  explicit  causal  structure.  These  factors  may  be  of  importance  for  the  learner’s  motivation  and  construction  of 
meaning,  particularly  the  possibility  for  making  a connection  between  the  information  they  may  gather  through  the 
program  and  their  present  experiences  and  knowledge  structures.  (Sarbin  1986;  Schank  1990;  Gjedde  1998b) 
Previous  research  in  this  area,  has  looked  at  various  CD-ROM’s  in  the  edutainment  and  educational  genre 
(Laurillard  and  Taylor  1994;  Plowman  1998).  There  are  indications  that  a non-narrative  or  encyclopeadic  structure 
tends  not  to  support  the  user’s  construction  of  knowledge  or  perception  of  global  coherence  to  the  extent  that  the 
narrative  that  may  be  found  in  books  and  television  typically  does. 

An  empirical  research  project  with  an  experimental  design,  the  MENO-project  (Multimedia  in  Education  and 
Narrative  Organisation)  has  experimentally  employed  different  types  of  structure  in  an  interactive  educational 
program.  (Laurillard  and  al.  1995)  (Plowman,  1999  ).  It  was  however  focused  on  the  structural  manipulations  and 
not  on  a narrative  embedding  of  the  material  through  a consistent  main  character  that  provided  a focus  for 
identification,  or  the  point  of  view  of  a narrator. 
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This  type  of  narrative  embedding  may  be  found  in  several  successful  popular  science  productions.  An  example  of 
this  is  the  best-selling  novel  by  Jostein  Gaarder  about  the  History  of  philosophy;  Sophies  World  (Gaarder  1992), 
which  became  the  world's  most  sold  book  in  1995.  A CD-ROM  production  was  made  on  the  basis  of  this  book 
within  the  genre  of  edutainment,  which  was  based  on  some  radically  different  stmctures.  In  the  CD-ROM  version 
some  of  the  canonical  narrative  elements  have  been  replaced  by  a NLP-generator,  which  is  capable  of  having  a 
primitive  dialogue  with  the  user;  and  with  a number  of  game  elements. 

Based  on  a pilot  project  with  a case-study  of  some  users  of  this  CD-ROM  I have  identified  some  potential  problems 
that  will  be  further  explored  in  a study  that  will  involve  other  CD-ROM's  of  the  same  genre  that  are  using  narrative. 
The  results  from  the  pilot  study  are  indicating  that  it  might  have  implications  for  the  user's  experience  when  the 
basic  narrative  element  are  replaced  by  other  elements,  which  do  not  support  the  process  of  narrative  construction, 
by  the  user.  The  narrative  is  seen  as  fundamental  for  the  user's  experience  of  meaning  and  global  coherence.(Bruner 
1990;  Schank  1990;  Schank  and  Abelson  J.1995)  the  narrative  is  furthermore  of  importance  for  the  dimension  of 
experience.(Gerrig  1993). 

Without  the  narrative  which  affords  an  embedding  context  the  user  tends  to  experience  the  options  offered  in  the 
interactivity  as  lacking  in  coherence  and  meaning.  This  must  be  seen  in  relation  to  the  genre  of  edutainment  and  that 
the  target  group's  users,  are  not  supposed  to  have  much  previous  knowledge  about  the  particular  field  that  the  CD- 
ROM  is  presenting. 

The  study  is  being  done  in  a naturalistic  mode,  with  a high  degree  of  ecologic  validity,  using  natural  users  of  the 
CD-ROM's  with  a mobile  usability  lab.  This  set  up  allows  for  a video-recording  to  be  done  with  a video  of  the  user 
as  well  as  a documentation  of  the  activity  on  the  screen  through  a scan- converter.  The  study  uses  a qualitative 
methodology  that  involves  user's  narratives  about  their  experiences  with  the  program  and  their  construction  of 
meaning  from  it.  Using  both  video-observation  and  qualitative  interviews  as  well  as  a questionaire  allows  for  a 
comparison  between  the  different  sets  of  data.. 

The  focus  of  the  project  is  to  explore  the  user's  experience  through  narrative  and  how  it  translates  into  a potential 
greater  sense  of  motivation  and  possibly  cognitive  capability  for  construction  of  meaning,  in  relation  to  the  non- 
linearity of  the  media. 


The  aims  of  the  project: 

• To  provide  a greater  understanding  for  the  qualitative  aspects  of  narrative  in  relation  to  interactive  Multimedia 
and  the  particular  possibilities  they  offer  in  a learning  environment  in  which  meaning,  experience  and 
motivation  are  key  elements. 

• To  explore  the  relationship  between  the  contextual  elements  of  a narrative  . In  particular  these  which  are  offered 
by  the  elements  that  constitute  a beginning  in  a canonical  narrative  (Labov  1967.1997;  Mandler  1984)  - and  the 
cognitive  implications  for  the  user  in  terms  of  experience,  understanding  and  interest  for  the  subject  and 
motivation  for  further  explorations  into  the  subject. 

• To  look  at  possible  connections  between  the  narrative  elements  in  the  interactive  program  and  the  user's  process 
of  narrative  construction  and  focus  on  the  relationship  between  the  narrative  organization  of  the  material  and 
user's  level  of  processing.(Laurillard  et  al.  1984,  1997) 

• To  explore  gender  specific  preferences  in  the  relation  to  narrativity  (Gjedde  1998b)  and  multimedia.(Adam 
1998). 

• To  inform  the  producers  of  educational  multimedia  about  the  implications  for  using  narrative  structure  and 
other  narrative  elements  in  relation  to  specific  groups. 
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Abstract:  The  aim  of  the  current  project  is  to  design,  develop  and  implement  a teaching  strategies 
model  and  a multimedia  template  which  crosses  over  curricular  boundaries  and  responds  to  a wide 
variety  of  learning  styles  of  students.  The  Conversational  Framework  model  as  proposed  by 
Laurillard  formed  a starting  point  in  examining  the  place  of  a multimedia  template  in  Academic 
Learning  while  the  design  of  the  template  has  been  guided  by  the  need  to  establish  reusable 
resource  materials.  Key  characteristics  of  such  a template  were  deemed  to  be  accessibility, 
cohesiveness,  supportiveness,  flexibility  and  quality. 


Background 

According  to  Laurillard,  the  teaching  - learning  process  is  an  interaction  between  teacher  and  student  operating  at 
two  intermingling  levels: 


Level  1 - Discursive  Level  - the  level  of  theory 

4 stages  to  reach  consensus: 

Level  2 - Interactive  level  - the  level  of  practice 

The  way  the  student  acts  in  the  world 

1 . Teacher  articulates  the  subject  matter 

1 . Teacher  sets  task 

2.  Student  joins  dialogue  - asks  questions,  practices  etc 

2.  Student  acts 

3.  Teacher  re-expresses  their  point  to  clarify  and 
elaborate.  * Teacher  uses  student  understanding  to 
adapt  interactive  activities  and  elaborate  theory  to  those 
appropriate  to  student  needs 

3.  World  responds  - includes  teacher,  other  students, 
experts  etc.  * Teacher  uses  student  understanding  to 
adapt  interactive  activities  and  elaborate  theory  to  those 
appropriate  to  student  needs 

4.  Student  has  another  attempt  at  representing  the  theory 

4.  Student  modifies  action  to  improve  their  performance 
( adaptation  and  reflection). 

The  teaching  model  is  designed  to  provide  a framework  for  the  teacher  and  student  for  all  steps  in  both  levels  whilst 
the  multimedia  matrix  plays  a role  in  only  part  of  the  total  process.  A balanced  mix  of  multimedia  resources,  face 
to  face  interaction,  online  communication  and  practical  activity  could  form  the  components  of  a complete  learning 
experience. 

Teaching  Plans 

It  is  the  intent  of  the  designers  that  the  multimedia  template  act  as  a guiding  mechanism  for  the  teacher  in  order  to 
facilitate  the  mapping  and  selection  process  for  the  teaching  sequence.  Through  thorough  analysis  of  the  learning 
objectives,  the  resource  selection,  interaction  planning  and  assessment  strategies  are  developed  and  a mapping  of 
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strategies  is  undertaken.  Once  all  aspects  of  the  teaching  mix  are  determined,  then  the  multimedia  elements  are 
produced  and  assembled  and  supporting  technologies  are  weaved  into  the  virtual  learning  environment. 

A typical  analysis  by  a teacher  in  planning  the  learning  experiences  may  be  as  follows: 

* Concept  Map  is  produced  showing  the  interconnecting  curriculum  and  content  elements 

* Identification  of  feasible  interactions  that  best  achieve  learning  outcomes 

* Development  of  plans  for  each  interaction  eg  multimedia  matrix,  face  to  face  discussion,  practical  excursion  etc 

* Topic  in  Context  material  is  assembled  and  Themes  and  associated  Activity  sequence  is  determined  and  designed 

* Media  Resources  are  gathered  for  supporting  problem  solving  as  required  by  activities 

* Computer  tools  are  assembled  for  use  by  students  in  carrying  out  activities 

Delivery  proceeds  with  monitoring  and  feedback  loops  built  in  to  the  interactions  - Evaluation  data  is  gathered  and 
used  in  review  of  the  model  and  matrix. 

In  teaching  academic  staff  to  utilise  the  matrix  tool,  the  above  sequence  forms  the  basis  for  a workshop.  This 
enables  the  intertwining  of  a pedagogical  and  a technological  focus  and  is  punctuated  by  a management  theme  ie  all 
three  focuses  become  a part  of  the  teaching  plan. 

The  Multimedia  Matrix 

The  matrix  template  consists  of  a number  of  "cells"  which  may  be  populated  with  content  or  resources  which  are 
required  for  the  student  to  carry  out  their  study  of  a particular  topic.  These  are  as  follows: 

A)  Topic  in  Context  - an  overview  of  the  topic  and  its  place  in  the  whole  curriculum.  The  relevance  of  the  topic  to 
the  profession  is  also  a key  aspect. 

B)  Topic  resources  - these  are  not  directly  linked  to  any  one  theme  or  activity  but  rather  are  more  global  to  the  topic 
eg  Extended  readings  or  digressions  may  be  placed  in  the  resource  area 

C)  Themes  - In  order  to  create  a single  environment  for  students  that  is  self  contained  but  has  interconnecting  parts, 
the  template  allows  for  the  creation  of  several  themes.  A concept  map  highlights  the  links  between  themes. 

D)  Activities  - these  are  the  problems  and  instructions  that  guide  students  through  the  resources  and  learning 
experiences  designed  for  them. 

E)  Tools  - These  include  Glossaries,  Concept  Map  generators.  Text  editors.  Analysis  tools  and  a launch  pad  for 
applications  that  are  common  place  such  as  word  processors  and  spreadsheets. 

F)  Media  - Video,  Audio,  Graphics  and  Text  elements  can  be  used  in  the  template  - these  are  dropped  into  any 
matrix  and  become  a part  of  the  application. 

Progress  to  date 

The  use  of  the  matrix  in  teaching  the  concepts  of  movement  through  the  specific  idea  of  "gait"  has  been  planned.  A 
plan  for  the  main  phases  of  teaching  the  sequence  has  been  established  and  each  matrix  in  a sequence  multimedia 
support  applications  is  being  developed.  These  include  Biomechanical,  Bioenergetics,  Diversity  and  Neurologic 
perspectives  on  gait.  Rather  than  examining  gait  in  each  of  these  aspects  separately,  the  planned  sequence  moves 
from  an  exploratory  view  through  each  of  these  perpectives  in  order  to  establish  core  understanding  and  then 
evolution  of  ideas  both  quantitative  and  qualitative  from  a combined  view.  To  illustrate,  a typical  activity  may 
require  a student  to  make  a series  of  observations  that  will  lead  to  understanding  of  both  diversity  and  biomechanical 
principles  and  their  interrelation.  Use  of  a common  visual  resource  in  supporting  such  an  activity  ie  the  same  video 
sequence,  also  reinforces  this  cross  curricular  notion.  The  template  itself  is  in  BETA  phase  and  has  been  used  to 
assemble  the  first  of  these  applications. 

The  staff  development  program  that  is  essential  in  establishing  the  model  for  achieving  maximum  effectiveness  of 
implementation  of  the  Conversational  Framework  model  has  begun  and  a network  of  colleagues  from  one  faculty 
group  who  are  in  close  proximity,  has  been  set  up.  This  is  a key  feature  for  staff  development  in  an  area  that 
requires  complete  rethinking  and  restructuring  of  teaching  and  its  planning.  Collegiality  and  support  cannot  be 
underestimated  in  their  power  to  catalyse  best  practice  teaching.  We  envisage  that  the  notion  of  a learning  matrix 
both  in  terms  of  the  multimedia  application  itself  and  of  the  curriculum  structure  will  provide  a clearer  view  for  both 
students  and  teachers.  In  this  way  the  path  through  the  study  of  a discipline  may  be  made  more  transparent  and 
flexible.  In  supporting  a student-centred  approach  to  teaching,  the  initial  effort  has  been  significant,  however  the 
revolution  that  will  follow  will  show  energy  levels  that  pale  these  efforts  into  insignificance. 
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Abstract;  The  purpose  of  this  study  is  to  investigate  which  experiences  of  adult  learners 
in  an  online  cooperative  learning  environment  contribute  to  the  expected  evolution  from 
an  aggregate  of  strangers  to  a group  capable  of  high  performance.  There  is  a clear  need 
for  a thorough  qualitative  study  focusing  on  the  development  of  online  relationships 
required  for  successful  cooperative  learning  groups.  . The  interview,  observation,  and 
journaling  methods  will  provide  information  to  address  impression  formation,  social 
presence,  group  development  and  the  conditions  of  ambiguity  and  detraining.  In  addition, 
the  results  of  this  study  will  help  provide  a palette  of  best  practices  for  designers  of  online 
learning  environments  utilizing  similar  instructional  strategies. 


Introduction 

Providing  opportunities  for  students  to  take  distance  education  courses  is  becoming  commonplace 
in  higher  education.  It  should  come  as  no  surprise  that  this  growth  fosters  change  and  with  change  comes 
resistance  and  criticism.  Yet,  perhaps  it  is  time  to  listen  to  that  criticism.  Historian  David  Noble,  professor 
at  York  University,  wrote  a series  of  articles  criticizing  what  he  characterized  as  the  unmindful  haste  by 
higher  education  institutions  to  adopt  the  Internet  and  World  Wide  Web  for  course  delivery  creating 
nothing  more  than  a technologically-slick  updated  version  of  the  correspondence  course  (summarized  in 
Noble,  1998).  Nobles’  primary  fear  is  that  we  are  creating  courses  devoid  of  any  true  interaction  between 
the  student  and  instructor. 

Course  developers  and  instructors  recognize  the  importance  of  designing  interactions  into  courses 
offered  by  distance  education  means.  The  importance  and  validity  of  these  interactions  is  amplified  in 
numerous  studies  in  which  students  reported  that  their  individual  success  was  due,  in  large  part,  to  a high- 
level  of  interaction,  typically  learner-instructor  (Eastmond,  1995;  Wegerif,  1998).  Likewise,  many  learning 
theorists  do  describe  the  integral  role  that  interaction  plays  in  the  level  of  learning  attained  by  students  (see 
Reeves  & Reeves,  1996,  for  review).  Most  studies  on  communication  using  computer-mediated 
communication  (CMC)  or  asynchronous  learning  networks  (ALN)  have  examined  interactions  resulting 
from  tasks  requiring  some  type  of  online  discussion  (Eastmond,  1995;  Hiltz,  1994;  Wegerif,  1998)  Yet, 
very  few  studies  have  looked  at  the  social  aspects  of  group  processes  in  online  environments  using 
cooperative  learning  techniques. 

Reports  from  distance  learning  students  in  collaborative  or  cooperative  learning  groups  using 
computer-mediated  communication  (CMC),  specifically,  asynchronous  learning  networks  (ALN),  indicate 
that  there  is  a crucial  moment  in  their  experience  in  which  they  cross  a threshold  from  outsider  to  insider  or 
from  newcomer  to  old-timer  (Eastmond  1994;  Wegerif,  1998).  Crossing  this  threshold  is  crucial  for 
increased  positive  attitude  leading  to  continuation  of  the  course  and  higher  achievement.  Failure  to  cross 
this  threshold  often  results  in  the  student  dropping  out  of  the  course.  How  does  a collection  of  strangers 
move  from  being  an  aggregate  of  people  to  a group  capable  of  high  performance  using  computer-mediated 
communication  (CMC),  specifically  asynchronous  learning  networks  (ALN),.  as  the  sole  means  of 
communication? 

It  stands  to  reason  that  understanding  interpersonal  communication  across  asynchronous  learning 
networks  is  crucial  to  designing  and  implementing  effective  cooperative  learning.  Results  from  laboratory 
research  led  to  a set  of  interpersonal  CMC  categorized  as  the  “Cues-Filtered-Out  Approach”  (see  Walther, 
1994,  for  review).  Essentially  those  theories  postulated  that  the  reduction  of  nonverbal  cues  leads  to 
communication  that  is  task  oriented  and  impersonal,  resulting  in  more  equal  communication  among  group 


1659 


members  but  the  possibility  of  very  negative  interactions  (‘flaming’).  Initially,  Walther  (1992)  stated  that 
the  communication  process  observed  within  CMC  follows  that  found  in  face-to-face  groups  only  the  time 
required  for  certain  behaviors  to  emerge  is  of  much  longer  duration.  In  addition,  an  individual’s  perception 
of  anticipated  future  interaction  with  group  members  leads  to  more  social  relational  behavior.  His  most 
recent  work  showed  that  computer-mediated  communication  can  run  the  gamut  from  impersonal  to 
interpersonal,  and  finally  to  what  he  calls  hyperpersonal  (Walther,  1996). 

There  are  major  discrepancies  for  prescriptions  on  how  to  establish  and  manage  groups  and 
cooperative  groups,  whether  face-to-face  or  online.  For  example,  the  need  for  an  individual  to  identify  with 
the  group  as  a prerequisite  for  intimacy,  affection  and  social  orientation  (Bales,  1950)  appears  to  be 
contrary  to  the  identification  of  heterogeneous  groups  as  the  most  successful  for  cooperative  learning 
(Johnson,  Johnson,  & Holubec,  1 994).  Furthermore,  there  are  conflicting  theoretical  approaches  and  even 
haziness  in  single  theories  about  the  ongoing  development  of  thriving  groups  in  CMC.  Specifically,  there 
are  conflicting  indications  of  how  one  might  go  about  creating,  facilitating,  and  managing  online  groups 
using  cooperative  learning  communicating  via  ALN/CMC.  How  can  these  discrepancies  be  resolved? 

Proposed  Research 

This  study  examines  the  key  aspects  to  consider  when  using  cooperative  group  instructional 
strategies  for  distance  learning  mediated  by  text-based  Internet  communication.  Based  on  components 
derived  from  research  on  small  group  development,  cooperative  learning,  and  relational  behavior, 
instructional  strategies  are  recommended  and  a set  of  research  questions  are  proposed. 

Understanding  the  cooperative  group  processes  when  using  CMC  that  provide  for  the 
development  of  successful  interpersonal  communication  leading  to  close  interpersonal  relationships  will  1) 
provide  information  for  instructional  designers  and  instructors  of  online  courses  to  guide  their  design  and  2) 
delivery  efforts  in  terms  of  group  composition,  group  activities,  and  facilitation  strategies,  and  guide  design 
of  better  interfaces  and  software  to  enhance  effective  communication  interaction. 

References 

Bales,  R.F.  (1950).  Interaction  process  analysis:  A method  for  the  study  of  small  groups.  Cambridge,  MA: 
Addison-Wesley. 

Eastmond,  D.V.  (1995).  Alone  but  together:  Adult  distance  study  through  computer  conferencing. 
Cresskill,  NJ:  Hampton  Press,  Inc, 

Hiltz,  S.R.  (1994).  The  virtual  classroom:  Learning  without  limits  via  computer  networks.  Norwood,  NJ: 
Ablex  Publishing  Corporation, 

Johnson,  D.W.,  Johnson,  R.T,,  & Holubex,  E.J.  (1994).  Cooperative  learning  in  the  classroom.  Alexandria, 
VA:  Association  for  Supervision  and  Curriculum  Development. 


Walther,  J.B.  (1992).  Interpersonal  effects  in  computer-mediated  interaction.  Communication  Research, 
19(1),  52-90, 


Walther,  J.B.  (1994).  Anticipated  ongoing  interaction  versus  channel  effects  on  relational  communication 
in  computer-mediated  interaction.  Human  Communication  Research,  40,  473-501 . 

Walther,  J.B.  (1996).  Computer-mediated  communication:  Impersonal,  interpersonal  and  hyperpersonal 
interaction.  Communication  Research,  23,  3-43, 

Wegerif,  R.  (1998).  The  social  dimension  of  asynchronous  learning  networks.  Journal  of  Asynchronous 
Learning  Networks,  2,  34-49, 


Keep  facilitating  pupils  and  English  teachers  appropriately 


Bo  Han 

Foreign  Languages  Department 
National  Teachers’  College 
Inner  Mongolia  P.R.  China 
hanbo49@hotmail . com 


Abstract:  This  paper  describes  a research  project  on  primary  school  English  teaching  and 
teacher  training  in  Inner  Mongolia  P.R.China  where  the  financial  situations  of  the  schools  and 
pupils  vary,  and  English  teachers’  levels  differ  greatly.  The  research  group  has  developed  a 
course,  which  has  a set  of  kernel  textbooks,  and  there  are  eight  different  combinations.  This 
article  describes  why  certain  media  are  chosen  and  how  they  are  used  in  order  to  make  sure 
that  teachers  can  be  trained  properly  and  the  pupils  can  get  the  most  possible  benefit. 


Introduction: 

There  is  no  doubt  the  use  of  technologies  in  learning  is  gathering  pace  around  the  world.  The  use  of  the 
multimedia  as  leaming/teaching  is  changing  the  way  the  world  thinks  and  acts.  Multimedia  has  the  potential  to 
make  teaching  and  learning  more  effective  but  can  we  use  them  everywhere?  This  article  describes  the  research 
project  on  primary  school  English  teaching  and  teacher  training  in  Inner  Mongolia  and  why  certain  media  are 
chosen  and  how  they  are  used.  Future  plan  is  mentioned  and  suggestions  for  improvement  are  longed  for. 


Analysis  and  schemes: 

The  demand  for  learning  English  is  greater  and  greater  in  this  fast  developing  world.  This  research  project  is 
aimed  to  meet  the  need  of  this  demand.  To  plan  for  flexibility  and  adaptation  right  from  the  start  is  very 
important,  if  we  aim  to  get  good  learning  results  for  a long  period  of  time  (Griffiths,  Heppell,  Millwood, 
&Mladenova,  1994),  The  following  aspects  are  what  we  considered  when  we  tried  to  work  out  a proper  plan: 

There  are  not  enough  English  teachers  in  Inner  Mongolia,  Some  schools  may  have  one;  some  other  schools  may 
have  more.  The  teachers’  English  levels  and  teaching  experience  vary;  Inner  Mongolia  is  a region  with  different 
nationalities.  The  main  languages  are  Mongolian  and  mandarin  which  have  sharp  differences,  Mongolian  pupils 
learn  Mongolian  and  mandarin  in  primary  school,  so  they  can  only  learn  English  for  one  or  two  years  in  primary 
schools  to  make  sure  that  they  may  not  be  overloaded;  Some  schools  may  have  tape  recorders,  others  may  have 
overhead  projectors.  Video  player,  etc;  locations:  Inner  Mongolia  is  a vast  area.  Primary  schools  scatter  all  over 
the  area.  To  train  all  the  teachers  face  to  face  is  impossible;  Some  pupils  cannot  afford  expensive  textbooks  and 
tapes.  Some  schools  cannot  afford  expensive  teaching  aids;  Although  Inner  Mongolia  is  a backward  region,  it  is 
also  changing  rapidly.  The  numbers  of  English  teachers  in  each  school  will  change,  the  facilities  in  schools  will 
change,  the  home  appliances  will  change;  How  can  we  use  different  media  for  the  pupils  and  teachers  in  the 
most  appropriate  way?  (Collis  & Deboer,  1998;  Griffiths  et  al,,  1994) 

After  all  the  above  aspects  besides  teaching  methodology  being  considered,  certain  media  are  used  in  the  way  as 
the  following:  Kernel  textbooks  and  supplementary  textbooks  have  been  developed  to  make  sure  that  the  pupils 
who  can  only  study  one  or  two  years  may  have  basic  knowledge  of  English  before  they  go  to  middle  schools 
and  other  pupils  who  have  the  opportunity  to  learn  more  in  primary  school  may  well  informed.  Eight  teaching 
plans  with  scientifically  related  teaching  materials  have  been  designed  to  make  sure  that  all  the  pupils  can 
benefit  most  efficiently.  The  textbooks  are  designed  black  and  white  and  special  revision  exercises  are  designed 
to  make  pupils  paint  the  textbooks  with  colorful  pencils  by  themselves.  Therefore  the  textbooks  not  only  attract 
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pupils  attention  more  and  the  pupils  may  remember  more  of  the  teaching  materials  but  at  the  same  time  the  price 
of  the  textbooks  is  cut  down;  Two  sets  of  tapes  are  designed.  One  set  is  that  one  tape  is  offered  each  term  to  help 
the  pupils  to  learn  at  home  and  to  keep  a low  price  for  those  pupils  who  cannot  afford  two  tapes.  The  other  set  is 
that  two  tapes  are  offered  each  term.  This  is  to  make  sure  that  there  are  enough  feedbacks  and  interactive 
exercises  to  do,  and  it  makes  preview  and  review  convenient.  At  the  same  time,  these  tapes  can  also  be 
convenient  for  teachers  to  prepare  their  lessons.  A special  tape  is  designed  only  for  teachers  to  check  pupils 
assignment.  In  this  way  pupils  do  not  have  to  buy  that  tape  and  teachers  will  not  have  trouble  with  the  questions 
and  answers.  Transparent  sheets  for  overhead  projectors:  Two  sets  of  transparent  sheets  are  designed.  One  is 
black  and  white;  the  other  is  with  color  to  meet  the  need  of  different  schools.  In  this  way,  teachers  may  have 
teaching  aids  even  in  schools  with  limited  budgets.  We  made  videotapes  for  pupils  and  teacher  training  so  that 
teachers  from  remote  areas  can  also  have  the  opportunity  to  watch  other  teachers’  teaching  and  understand  the 
theories  of  teaching.  Schools  with  video  players  can  play  the  videotapes  to  their  pupils  and  pupils  with  video 
players  at  home  can  watch  the  videotapes  to  learn  better. 

Learning  should  be  immediate  feedback  as  many  educators  have  already  recognized.  However,  the  above  media 
which  have  been  used  are  quite  limited  in  the  aspect  of  interactive  activities.  This  means  that  a lot  of  potentials 
from  the  teachers  and  pupils  have  not  been  cultivated.  What  should  we  do  to  accomplish  our  goal  then?  In  order 
to  work  out  our  future  plan,  let’s  first  see  what  will  happen  in  this  part  of  world  in  the  near  future? 


The  future  plan: 

In  the  near  future  more  and  more  schools  in  Inner  Mongolia  will  also  have  computers.  Since  Internet  connection 
in  China  is  still  quite  expensive,  most  of  the  families  in  Inner  Mongolia  will  not  be  able  to  get  an  access  very 
soon.  But  the  perspective  of  using  Internet  in  schools  is  great.  A significant  number  of  schools  will  have  the 
capability  to  interact  online.  It  means  that  teaching  materials  designed  to  be  used  with  computers  may  help  more 
and  more  teachers  and  pupils  to  teach  and  learn  more  effectively.  The  key  to  successful  interactivity,  is  not 
simply  offering  educational  access  to  preexisting  networked  materials.  Rather,  a more  valuable  role  is  to  offer  a 
collaborative  environment  where  users  can  create,  and  share  information  of  their  own  with  other  users.  It’s  an 
opportunity  that  many  teachers  won’t  let  slip  by  them.  It  means  that  we're  reaching  a point  where  teachers  can’t 
be  competitive  in  other  words,  prepare  today’s  children  for  the  university  and  the  modem  workplace  without  the 
Web.  Therefore,  our  future  plan  is  divided  into  three  steps: 

1 . To  put  the  teachers  books  on  Internet  first  and  build  a forum  for  teachers  to  share  their  teaching  experience. 
In  this  way,  teachers  will  be  well  informed  and  share  their  experience  in  teaching,  and  with  the  latest 
version  of  teachers’  books,  they  can  always  try  to  teach  their  pupils  better. 

2.  To  put  teaching  materials  with  more  interactive  activities  on  CDs.  In  this  way  schools  with  computers  and 
pupils  with  computers  at  home  can  use  these  materials  to  teach  and  learn  more  efficiently. 

3.  To  establish  on  line  courses  for  pupils  to  learn.  Besides  textbooks  with  interactive  activities,  a reading 
series  can  be  developed.  In  this  way,  pupils  with  internet  access  not  only  can  learn  the  textbooks  well  but 
can  also  learn  by  their  own  pace  to  develop  their  abilities  in  reading,  listening  , speaking  and  writing. 

The  above  are  our  plan  for  the  near  future.  We  are  eager  to  ask  you  the  specialists  in  the  conference  to  offer  us 
suggestions  in  order  to  make  this  ideal  into  reality.  All  your  advice  would  be  greatly  appreciated. 
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Abstract:  Developers  of  educational  material  like  web-based  training  and  online  courses  face  well  known 
aspects  of  classic  software  development  processes:  Reusability  of  course  units;  Supporting  cooperation  and 
resource  sharing  between  the  members  of  the  design  process,  like  content  authors,  designers,  multimedia 
producers  and  quality  managers  and  Quality  management.  Therefor,  software  engineering  techniques  like 
object-oriented  analysis  (00 A)  based  on  a central  database  to  be  used  in  iterative  design  processes  will  also 
be  useful  for  the  development  of  educational  material.  This  considerations  are  the  background  for  our  work 
and  experiences  in  the  project  Virtual  University  as  described  in  this  paper. 


INTRODUCTION 

The  Project  “Virtual  University  of  Applied  Science” 

This  paper  reports  work  done  in  the  project  “Virtual  University  of  Applied  Sciences”  (period  of  duration  1 998-2003,  see 
http://vfh,de).  It  aims  at  establishing  a location  independent  university  with  a curriculum  for  computer  science  of 
multimedia  systems  and  for  business  engineering  (Bachelor,  Master),The  authors  of  this  paper  are  involved  in  the 
production  of  web-based  courses,  in  the  design  of  user-adequate  learning  spaces  and  in  the  support  of  the  design  process. 
Their  focus  is  on  usability  recommendations  and  quality  management  for  the  course  material  during  the  development 
process.  Other  aspects  like  teaching  strategies,  learning  processes  or  technical  issues  concerning  the  course  production  are 
supervised  by  other  dedicated  consulting  groups  within  this  project, 

Adressed  Problems 

During  the  first  months  of  course  production  for  the  virtual  university  several  problems  got  obvious: 

• The  lack  of  a appropriate  context  of  use  description. 

Practice  showed  that  the  developers  of  course  material  had  only  an  incomplete  mental  model  about  the  needs  and 
abilities  of  the  intended  user  groups,  like  previous  knowledge,  learning  style,  preferences  in  teaching  mode,  motivation 
and  other  information  that  would  be  important  to  adapt  the  teaching  strategies  and  delivery  of  hypermedia  structure  and 
content  accordingly  (Kritzenberger  2000),  The  mental  models  about  this  user  group  and  the  assigned  attributes  even 
differed  during  development  phases, 

• No  stable  base  for  the  quality  management 

Quality  management  (e.g.  of  learning  processes,  functional  correctness,  usability)  suffers  from  a lack  of  basic 
information  about  design  and  implementation  requirements. 

® Consistency  of  content  information  and  re-use  of  contents 

Another  problem  is  that  the  content  or  the  structure  of  the  course  units  may  get  inconsistent  during  workflow.  While 
the  content  author  has  her  or  his  version  (e.g.  a linear  MS-Word-Document),  the  hypermedia  producer  has  to  transfer 
content  and  structure  to  web  formats  (HTML  and  sometimes  JAVA  Add-Ons).  Changes  in  the  linear  version  have  to  be 
manually  transferred  to  the  hypermedia  version.  This  is  a source  of  errors  and  often  prevents  maintainance,  because  of 
the  extra  effort  it  will  take.  Keeping  such  a training  unit  up  to  date  often  requires  to  start  all  over  again  (e.g.  with 
updating  the  Word-Document)  and  repeat  former  development  steps. 

DESCRIPTION 

The  main  idea  is  to  overcome  the  problems  mentioned  above  by  introducing  a central  information  base  which  is  used  by  all 
participants  during  the  design  process:  content  author,  designer,  producer  and  quality  manager  (see  figure  1).  Using  a 
simple  relational  database  combined  with  a powerful  web  frontend  seems  to  be  adequate.  This  database  should  include  all 
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the  context  information  as  well  as  the  content  of  the  teaching  units.  In  a first  step  these  informations  are  only  presented  to 
the  involved  persons  but  no  contents  are  automatically  generated. 

For  database  organization  object  oriented  techniques  are  used.  Classes  of  information  like  “user  attributes”,  “organisational 
requirements”  “content”,  etc.  are  identified  and  then  may  be  freely  combined  into  “views”.  These  views  are  role  dependent, 
for  example  the  content  author  may  concentrate  on  the  analysis  data  and  the  contents.  Whereas  the  producer,  who  has  to 
implement  the  learning  unit,  may  need  additional  design  rationales  added  by  the  designer.  A quality  manager  can  base  his 
evaluation  of  the  system  on  the  requirements  identified  and  documented  during  the  analysis  phase. 


As  the  size  and  the  number  of  the  learning  units 
is  difficult  to  handle,  techniques  of  the  OOA  are 
used;  abstraction,  inheritence  and 
generalization.  If  for  instance  many  different 
user  groups  are  to  be  considered,  they  may  be 
ordered  hierarchically;  “All  users**  ^ “users  of 
the  teaching  unit”  ^ “A  special  group  within 
these  user  group”.  Attributes  of  the  most 
general  object  (“All  users”)  are  inherited  by  all 
following  objects  and  then,  e.g.  the  designer’s 
view  includes  all  attributes,  from  the  general  to 
the  specific  ones,  OOA  techniques,  like 
underspecification  and  refinement  make  the 
handling  of  large  object  sets  easier  and  allow  all 
participants  to  start  with  rather  raw  data  and  to 
refine  them  during  the  lifecycle  of  the  developed 
system  using  OOA  techniques  based  on  a 
database  has  several  advantages.  The  database 
supports  the  complete  lifecycle  of  the  course  unit 
and  makes  all  information  and  design  rationales 
available  again  for  maintaining  or  updating  the 
course.  As  the  contents  are  HTML-based  they 
can  be  included  into  the  courses  with  less 
manual  effort.  Furthermore,  the  connection  of 
each  unit  to  the  position  and  role  in  the  database 
is  kept,  nection  to  the  database.  Additionally 
for  each  production  phase  there  is  appropriate 
additional  information  available,  like  design 
rationales  and  the  related  context  informations. 
Updating  a content  in  the  database 
automatically  updates  the  course  and  avoids 
inconsistencies. 

Current  and  future  work 
Currently  two  major  tasks  are  to  be  solved:  First 
the  different  object  classes  (content  and  context 
objects)  and  views  needed  for  the  educational 
systems  must  be  identified,  validated  and 
implemented.  Second  the  database  tool  has  to  be 
implemented  and  tested. 

As  the  described  method  is  similar  to  a special  OOA  technique  currently  under  development  in  our  institute  as  well 
(project  “TAToo”)  see  (Herezeg  1999),  we  reuse  implementation  resources  from  that  project.  Actually  the  database  frontend 
and  the  needed  JAVA/JDBC/JSP-Interface  will  be  the  same  in  both  projects. 
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Figure  I:  An  iterative  development  process  for  educational 
svstems  (eg.  web  based  training  units).  Please  note  that  all  these 
steps  are  to  be  handled  iterativly,  that  means  the  whole  (low 
must  be  done  several  times. 
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Abstract:  In  an  Internet  environment,  users'  expectations  are  diverse  due  to  its  on-line 
characteristics.  Internet  can  be  assessed  by  any  user  of  any  age,  group,  race,  nationality,  and  gender 
all  over  the  world  at  any  time  and  at  any  place.  Considering  these,  web  site  designers  must  adhere 
to  certain  design  standards  and  properly  evaluate  their  sites  to  ensure  its  usability  and  effectiveness. 
Despite  abundance  of  studies  on  web  usability,  very  few  of  them  focus  on  the  political  web  sites. 
Using  political  web  sites  in  Malaysia  as  an  empirical  base,  this  research  intends  to  identify  factors 
affecting  the  use  of  political  web  sites;  and  then  based  on  the  analysis,  develop  a framework  for 
evaluating  the  usability  of  political  web  sites.  The  main  purpose  is  to  contribute  towards  the 
improvement  of  cyberdemocracy  in  the  information  age. 


Introduction 

The  central  focus  of  the  debate  on  the  success  of  a democratic  political  system  is  people  participation  in  election, 
political  discussion,  and  the  process  of  governance.  In  order  to  participate,  people  should  possess  sound  knowledge 
about  political  and  social  issues  affecting  them.  Traditionally,  people  are  mostly  dependent  on  mass  media  mainly 
newspapers,  magazines,  television  and  radio  as  sources  of  political  information.  Indeed,  mass  media  has  been  very 
influential  in  shaping  public  opinion  about  political  and  social  issues  in  any  democratic  country.  Today,  the  advent 
of  the  Internet  has  revitalized  the  whole  concept  of  communication  media.  Ideally,  it  does  not  only  provide  an 
alternative  source  of  political  information,  but  it  can  also  be  used  as  an  effective  political  communication  medium 
between  and  among  citizens,  public  leaders  and  political  parties.  This  development  certainly  provides  opportunities 
for  the  enhancement  of  democracy  whose  main  emphasis  is  on  citizens’  freedom  to  actively  participate  in  politics  by 
debating  and  exchanging  views  on  key  social  and  political  issues. 


Cyberdemocracy  and  Current  Practice 

There  is  little  agreement  on  the  exact  definition  of  cyberdemocracy  because  of  its  uncommon  usage.  Clift  (1997) 
states  that  cyberdemocracy  is  about  the  use  of  online  communication  tools  for  many-to-many  civic  discussions 
among  citizens  and  public  leaders.  It  is  based  on  the  belief  that  opens  communication  and  participation  is  the 
foundation  of  democracy.  According  to  Alexander  and  Grabbs  (1998),  cyberdemocracy  refers  to  the  optimized  use 
of  internet-based  communication  technologies  by  government  agencies,  interest  groups  and  non-profit  organizations 
to  promote  public  participation  in  the  process  of  governance.  In  general,  cyberdemocracy  is  defined  as  the  use  of  the 
Internet  technology  such  as  World  Wide  Web  (WWW),  File  Transfer  Protocols  (FTP),  electronic  mail  (email), 
conference  facilities  and  net  chatting  for  dissemination  of  information  and  exchange  of  ideas  between  and  among 
government,  political  parties  and  public  leaders  with  the  general  public.  There  are  a number  of  examples  of  efforts 
carried  out  by  government,  political  parties  and  voluntary  bodies  to  establish  cyberdemocracy.  Minnesota  e- 
democracy  project  (http://www.e-democracy.org),  established  in  1994  by  a group  of  volunteers,  held  the  first  online 
debate  via  email  among  candidates  for  governor  and  United  States  Senate  and  launched  the  MN-POLITICS  email 
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discussion  forum.  Another  example  is  United  Kingdom  Citizen  Online  Democracy  (http://www.democracy.org.uk), 
a non-partisan  effort,  that  began  work  before  the  1997  national  election.  It  hosted  a number  of  topical  events  on  such 
topics  as  European  monetary  union  efforts  and  online  delivery  of  government  services,  and  it  held  an  all-party 
debate  during  the  election. 


Web  Usability  and  Research  Focus 

Usability  is  basically  a very  broad  concept  in  system  design  and  difficult  to  define.  In  this  research,  usability  refers 
to  how  easy,  effective  and  useful  it  is  for  a person  who  uses  a political  web  site  to  achieve  their  goals.  According  to 
Nielsen  and  Mack  (1994),  and  Shneiderman  (1998),  there  are  5 main  aspects  of  usability:  ease  of  use,  efficiency, 
satisfaction,  rate  of  users'  error  and  retention  over  time.  The  primary  focus  of  this  research  will  be  on  the  usability 
issues  of  web  sites  used  as  political  communication  medium.  These  issues  are  grouped  into  seven  main  factors: 
Screen  design.  Content,  Accessibility,  Navigation,  Media  Use,  Interactivity,  Consistency  — abbreviated  by  the 
researcher  as  SCANMIC.  These  are  the  generic  factors  of  web  usability  based  on  an  analysis  of  current  web  design 
guidelines  and  literature. 


Research  Question 

In  an  Internet  environment,  users'  expectations  are  diverse  due  to  its  on-line  characteristics.  Internet  can  be  assessed 
by  any  user  of  any  age,  group,  race,  nationality,  and  gender  all  over  the  world  at  any  time  and  at  any  place. 
Considering  these,  web  site  designers  must  adhere  to  certain  design  standards  and  properly  evaluate  their  sites  to 
ensure  its  usability  and  effectiveness.  Despite  abundance  of  studies  on  web  usability,  very  few  of  them  focuses  on 
the  political  web  sites.  Yet,  usability  has  become  one  of  the  most  important  aspects  that  determine  the  success  of 
web  sites  regardless  of  their  category.  With  this  in  mind,  this  research  will  address  three  major  questions; 

a)  What  are  factors  that  affect  the  usability  of  political  web  sites? 

b)  How  to  evaluate  the  usability  of  political  parties’  web  sites? 

c)  What  is  the  level  of  the  usability  of  web  sites  in  current  cyberdemocracy  efforts  in  Malaysia? 


Conclusion 

The  above  research  questions  will  be  used  to  achieve  the  following  objectives: 

a)  To  identify  factors  that  affect  the  usability  of  political  parties’  web  sites. 

b)  To  design  a framework  for  evaluating  the  usability  of  political  web  sites.  The  framework  is  designed  for  both 
technical  and  non-technical  people  who  are  involved  in  political  web  site  development. 

c)  To  test  the  applicability  of  proposed  framework. 

d)  To  propose  a guideline  for  developing  a usable  political  web  site. 
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Abstract:  Biggs’  (1996)  proposed  a pedagogical  model  where  “performances  of  understanding” 
specified  in  course  objectives  are  used  to  systematically  align  teaching  methods  and  assessment 
practices.  The  model  was  applied  in  a plant  physiological  ecology  course  delivered  at  a distance 
over  the  web.  Students  and  instructor  thought  the  course  format  helped  learning  and  that  the 
assessment  model,  the  SOLO  Taxonomy,  was  good.  Everyone  also  concluded  that  the  course 
workload  was  too  great  in  spite  of  ample  feedback  to  discourage  such  excess.  The  course  may  be 
viewed  at:  http://www.biology.ualberta.ca/courses.hp/bot43 1 .hp/bot43 1 hp.html 


Evidence  based  good  teaching  practice  seeks  to  enhance  “deep  learning”  by  emphasizing  active  learning 
strategies,  high  expectations,  time  on  task,  prompt  feedback,  respect  for  diverse  ways  of  learning  and  student-student 
and  instructor-student  interaction.  Deep  learning  is  supported  by  a constructivist  pedagogy  that  argues  that  it  is  the 
learner’s  activities  that  aid  their  construction  of  meaning  to  bring  about  learning. 

Biggs  (1996)  has  articulated  a “Constructive  Alignment”  model  that  is  the  marriage  of  two  ideas: 
constructivism  as  a framework  with  the  learner's  activities  as  central  in  creating  meaning  and  the  instructional 
designers  emphasis  on  the  relationship  between  the  learning  objectives  and  the  targets  for  assessing  student 
performance.  In  his  model,  the  "performances  of  understanding"  nominated  in  the  objectives  are  used  by  the 
instructional  designer  to  systematically  align  the  teaching  methods  and  the  assessment  activities. 

Biggs  and  Collis  (1982)  previously  developed  a ‘Structure  of  Observed  Learning  Outcomes  (SOLO) 
Taxonomy’  that  analyzes  the  cognitive  level  of  students’  written  work.  It  identifies  five  categories: 

1)  prestructural  - no  understanding  of  the  topic, 

2)  unistructural  - nominal  understanding  of  one,  or  a few,  points, 

3)  multistructural  - several  aspects  understood  but  used  separately, 

4)  relational  - components  integrated  into  coherent  wholes, 

5)  extended  abstract  - the  integrated  whole  is  reconceptualized  at  a higher  level  of  abstraction. 

The  taxonomy  provides  a scheme  that  served  as  a basis  for  giving  feedback  to  students  on  their  written  work. 

The  Constructive  Alignment  model  was  applied  in  a distance  delivered  web  based  course  for  senior 
undergraduate  and  graduate  students.  For  each  week  of  the  course,  the  course  web  site  gave  details  of  the  learning 
objectives,  required  reading,  optional  supplemental  reading  and  web  hyperlinks  to  resource  sites.  The  web  site  also 
provided  an  extensive  background  on  the  pedagogical  principles  behind  the  course,  the  nature  of  the  SOLO 
Taxonomy,  sample  responses  to  a problem  and  their  assessment  according  to  the  Taxonomy,  and  advice  on 
preparing  portfolios  of  evidence  that  the  learning  objectives  had  been  achieved. 

Students  were  invited  to  assemble  their  own  portfolios  of  evidence  that  they  had  achieved  the  learning 
objectives.  Portfolios  were  submitted  by  e-mail  and  assessment  feedback  was  provided  in  the  terminology  of  the 
SOLO  Taxonomy.  Students  had  access  to  a password  protected  conferencing  system  that  allowed  them  to  interact 
with  each  other,  the  teaching  assistant  and  the  instructor  in  a virtual,  asynchronous  group  environment.  An  on-line 
chat  room  was  also  available  through  the  conference  system.  Private  communication  between  participants  was 
available  through  e-mail. 

The  course  opened  with  a telephone  conference  to  allow  the  students  to  meet  each  other  and  interact  with 
the  teaching  assistant  and  instructor.  The  students  and  the  teaching  assistant  then  moved  to  a computer  lab  where  all 


students  had  the  opportunity  to  demonstrate  that  they  were  familiar  with  the  computer  and  software  skills  required  to 
complete  the  course  work. 

Student  assessment  in  the  course  was  based  on  the  submission  of  three  portfolios  of  evidence  of  meeting 
the  learning  objectives  through  the  term.  Take  home  mid-term  and  final  exams  were  also  given.  These  exams  were 
integrative  of  the  material  in  the  portfolios  and  gave  the  students  considerable  discretion  in  selecting  the  context  in 
which  the  questions  were  answered.  Exam  feedback  was  again  based  on  the  SOLO  Taxonomy.  As  the  eight 
students  in  the  course  had  no  previous  experience  of  distance  education  or  portfolio  assessment,  they  were  invited  to 
maintain  a reflective  journal  that  was  read  three  times  during  the  course  by  the  instructor.  Marks  for  keeping  the 
journal  were  given  on  a credit/no-credit  basis. 

At  the  end  of  the  course  students  were  asked  to  evaluate  the  course  and  their  learning  experience. 
Evaluations  were  submitted  to  the  TA  who  cut  and  pasted  the  anonymous  student  comments  into  an  e-mail  message 
to  the  instructor. 

The  evaluations  showed  that  the  students  thought  the  course  format  helped  learning,  they  developed 
independent  learning,  time  management  and  communication  technology  skills,  that  the  web  materials  on  the  SOLO 
Taxonomy  were  good,  that  the  web  material  on  the  course  philosophy  was  good  but  only  in  retrospect  and  that  the 
learning  journal  material  was  very  helpful. 

In  the  future  the  students  thought  that  workload  in  the  course  should  be  reduced,  feedback  increased,  course 
notes  provided,  more  texts  should  be  placed  on  reserve,  a lab  should  be  included  and  a TA  with  a background  in 
physiology  would  be  preferable. 

Students  thought  that  future  offerings  of  the  course  should  retain  the  chat  room,  group  camaraderie, 
learning  objectives,  the  SOLO  Taxonomy,  and  the  instructor’s  understanding. 

The  instructor’s  evaluation  concluded  there  should  be  a reduced  workload,  more  frequent  feedback,  greater 
use  of  an  on-line  seminar  facility  and  continued  use  of  the  SOLO  Taxonomy  and  learning  journals. 

Workload  was  an  issue  for  eveiyone.  Despite  the  feedback  given  on  their  portfolios,  the  volume  of 
evidence  provided  on  each  of  the  three  submission  dates  increased  steadily.  Students  were  spending  more  time 
assembling  evidence  and  the  instructor  was  spending  more  time  assessing  it.  Conversations  in  the  chat  room 
emphasized  the  concern  that  everyone  shared  and  a phone  conference  was  organized  to  discuss  the  problem  in  real 
time.  Despite  encouragement  by  the  instructor  to  pay  attention  to  the  quality  of  the  portfolio  contents  rather  than  the 
quantity,  the  problem  persisted. 

Providing  more  regular  feedback  would  have  spaced  out  the  assessment  workload  for  the  instructor  but  it  is 
not  clear  that  it  would  have  helped  reduce  the  pressure  the  students  placed  on  themselves  to  submit  a large  mass  of 
evidence  of  their  learning.  The  conferencing  tool  (WebTeach)  had  a seminar  facility  where  students  could  have 
posted  questions  for  their  peers  relating  to  the  learning  objectives.  Responses  to  questions  posted  by  their  peers 
were  not  readable  until  participants  had  submitted  their  own  answer.  This  facility  remained  unused  although  it 
might  have  been  a vehicle  to  model  the  depth  of  material  that  was  reasonable  in  a portfolio.  It  is  also  possible  that 
students  would  see  use  of  the  seminar  facility  as  even  more  work. 

Everyone  involved  in  the  course  was  satisfied  with  the  SOLO  Taxonomy  as  a framework  for  assessment.  It 
allowed  the  students  to  align  their  portfolios  to  the  learning  objectives  and  the  assessment  criteria.  Students  found 
preparing  a learning  journal  to  be  useful.  Evidence  of  concern  over  workload  was  recorded  in  the  journals  along 
with  recognition  of  the  feedback  they  were  getting  on  what  was  anticipated.  However,  recognition  of  the  issue  and  a 
discussion  of  solutions  did  not  solve  the  problem. 

The  learning  objectives  were  well  received  by  the  students.  However,  some  continued  to  think  that  course 
notes  would  have  helped.  It  seems  that  even  these  senior  students  were  uncomfortable  in  assuming  a large  degree  of 
control  over  their  own  learning.  It  is  worth  noting  that  some  students  commented  that  the  material  on  the  pedagogy 
behind  the  course  was  only  useful  “in  retrospect”.  If  these  various  opinions  were  expressed  by  the  same  set  of 
individuals,  it  might  suggest  that  more  quickly  coming  to  terms  with  the  pedagogical  background  might  have 
compensated  for  the  lack  of  the  course  notes. 

In  conclusion,  Biggs’  pedagogical  model  of  Constructive  Alignment  is  suitable  for  application  in  a web 
environment. 
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Abstract:  This  paper  introduces  an  attempt  to  make  use  of  Information  Technology 
to  facilitate  teaching  and  learning  in  a teacher  education  institute.  A multimedia 
video  bank  was  being  developed  to  provide  with  teacher  educators  and  student 
teachers  necessary  video  clips  that  could  be  made  use  to  produce  computer-  assisted 
instruction  programs  that  fit  the  local  context.  All  the  videos  are  intentionally  to  do 
without  any  narration  or  background  music  to  allow  flexibility  for  the  subject 
teachers  to  present  their  materials  and  to  better  communicate  with  the  students. 


Background 

In  the  First  Policy  Address  of  the  Chief  Executive  of  the  Hong  Kong  Special  Administrative  Region 
(HKSAR)  Government,  a firm  devotion  of  the  Government  to  develop  information  technology  (IT)  in 
education  was  first  revealed.  However  the  education  community  in  HK  is  still  striving  to  seek  the 
direction  of  IT  in  education  that  matches  the  local  context,  a unique  culture  of  the  East  and  the  West. 
Most  of  them,  especially  the  primary  school  teachers  are  puzzled  simply  because  they  do  not  know 
how  to  start  with  teaching  with  IT. 

Objective 

With  the  aim  to  facilitate  teaching  and  learning  with  IT  that  fits  the  local  context,  this  project  attempts 
to  build  up  a multimedia  resources  bank  (MMRB)  for  teacher  educators  and  student  teachers  in  the 
Hong  Kong  Institute  of  Education  (a  publicly-funded  teacher  education  institute)  to  develop  their  own 
computer-assisted  instruction  (CAI)  programs  or  to  use  as  part  of  the  teaching  materials.  It  works 
towards  a direction  to  encourage  and  promote  innovative  teaching  approaches  with  an  effective  use  of 
IT  in  classrooms. 

Multimedia  Resources  Bank  in  General  Studies 

Despite  of  the  resources  put  in  acquiring  computer  hardware  and  software,  there  is  hardly  any  quality 
interactive  CAI  program  in  “General  Studies”  being  developed  because  there  are  difficulties  facing  by 
the  primary  school  teachers: 

[1]  The  heavy  workload  does  not  allow  them  to  have  location  shooting  and  video  editing  that  are  both 
needed  in  developing  the  CAI  programs  in  “General  Studies”. 

[2]  It  is  not  worthwhile  for  the  primary  school  teachers  to  heavily  involve  in  video  production,  as  it  is 
not  their  profession  in  this  aspect. 

However,  there  are  enthusiastic  primary  school  teachers  who  are  willing  to  produce  their  own 
multimedia  teaching  materials  provided  that  there  is  enough  support  (Pow,  So  and  Hung,  2000).  With 
this  in  mind,  we  therefore  tried  to  build  a MMRB  in  “General  Studies”  for  the  primary  schools  teachers 
and  let  them  to  make  use  of  the  resources  to  tailor  make  their  own  CAI  programs  to  facilitate  teaching 
and  learning.  With  the  cooperation  between  a lecturer  from  Department  of  Science  and  education 
technology  professionals  from  the  Centre  for  Learning,  Teaching  and  Supervision  in  Hong  Kong 
Institute  of  Education,  the  team  has  produced  about  a hundred  video  clips  according  to  the  syllabus  of 
the  “General  Studies”  to  be  put  on  the  MMRB.  We  chose  “General  Studies”  as  the  starting  point  for 
MMRB  as  it  is  a new  subject  that  integrates  science,  social  study  and  health  education  in  primary 
school  curriculum,  and  hence  at  hand  resources  are  still  lacking. 
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The  content  of  the  video  clips  covers  a broad  area  of  topics  from  Hong  Kong  community  services, 
transportation,  housing,  etc.  to  scientific  experiments  such  as  the  property  of  magnet,  heating,  etc.  The 
videos  are  classified,  edited  into  a reasonable  length  and  stored  onto  a video  server  for  easy  retrieval 
and  viewing.  All  the  videos  are  intentionally  to  do  without  any  narration  or  background  music  since  we 
would  like  to  allow  flexibility  for  the  subject  teachers  to  present  their  materials  and  to  better 
communicate  with  the  students.  Although  it  is  the  Institute  that  owns  the  copyrights,  staff  and  students 
can  use  it  freely  in  their  own  works  or  teaching.  Staff  and  students  can  instantly  view  the  video  clips  of 
their  choice  at  any  computer  that  is  linked  up  with  the  Institute’s  Intranet  or  download  these  media 
clips  for  their  own  use  later  on.  This  MMRB  is  expected  to  be  able  to  save  up  valuable  time  and 
resources  for  those  who  want  to  try  out  teaching  with  IT  but  do  not  have  enough  time  and  resources  to 
produce  the  media  on  their  ovm.  It  is  hoped  that  this  could  contribute  partly  to  promote  innovative  use 
of  IT  in  teaching  and  learning,  not  just  within  our  Institute  but  also  to  the  education  community  at 
large. 
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Abstract:  The  purpose  of  this  study  is  to  examine  the  effects  of  learner’s  PBL 
authenticity  perception  on  achievement  and  motivation.  There  are  two  major 
research  questions.  First,  is  there  a difference  on  learner's  achievement  between  high 
and  low  perception  level  of  authenticity?  Second,  is  there  a difference  on  learner's 
motivation  between  high  and  low  perception  level  of  authenticity?  The  results  show 
the  following:  (1)  There  is  a significant  difference  in  the  learner's  achievement 
between  low  and  high  level  of  authenticity  perception  (t=-3.42,  p<.05)  (2)  There  is 
a significant  difference  in  learner's  motivation  (t=-3.34,  p<.05). 


Introduction 

In  recent  years,  with  the  advent  of  the  Information  Society,  the  focus  of  educational  paradigm  has  been 
shifting  from  "teaching"  to  "learning".  This  signifies  a move  to  a learner-centered  education  where  the 
student  becomes  an  active  participant  in  the  learning  process.  Korea's  educational  reform  committee  has 
established  a goal  to  build  a leamer-centcred  educational  environment  to  prepare  Korea’s  students  to  be 
competitive  in  2U*  century. 

“Web’s  Cool”  program  has  been  planned  since  1998  to  support  the  learner-centered  education  system 
in  Korea.  Subject  areas  such  as  language  arts,  math,  science  and  ESL(English  as  a Second  Language) 
for  middle  school  students  have  been  developed  on  the  Web.  Since  the  Web  was  considered  to  be  an 
effective  medium  for  learner-centered  approach  by  applying  the  constructivistic  principles, 
PBL(Problem  Based  Learning)  strategy  were  applied  to  provide  students  opportunities  to  participate  in 
the  learning  process  actively. 

While  the  Web’s  Cool  has  been  developed,  we  wanted  to  investigate  the  fact  that  whether  the  problem 
we  were  using  as  an  authentic  task  has  any  effect  on  learner’s  motivation  and  achievement.  In  order  to 
measure  the  authenticity  of  the  task,  the  expert’s  as  well  as  student’s  perception  level  of  the  authenticity 
toward  the  task  has  been  scored  by  using  a 5 point  scale.  Two  tasks  were  selected  as  high  and  low 
authenticity  level  of  perception  by  both  10  experts  and  10  students  to  investigate  the  effects. 

The  purpose  of  this  study  is  to  examine  the  effects  of  the  authenticity  perception  on  learner's  motivation 
and  achievement.  There  are  two  major  research  problems.  First,  is  there  a difference  on  learner's 
motivation  between  high  and  low  perception  level  of  authenticity?  Second,  is  there  a difference  on 
learner’s  achievement  between  high  and  low  perception  level  of  authenticity? 

Research  Method 

Subjects 
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Subjects  for  this  study  are  51  students  of  the  middle  school  students.  Two  learning  tasks  perceived  low 

and  high  authenticity  are  selected. 

Results 

The  results  are  as  follows: 

(I ) There  is  a significant  difference  in  the  learner's  achievement  in  general  between  low  and  high  level 
of  authenticity  perception  (t=-3.42,  p<  ,05).  A significant  difference  was  found  in  the  learner's 
authentic  achievement  between  low  and  high  level  of  authenticity  perception  (t=-4,54,  p<  .05). 
However,  no  significant  difference  was  found  in  the  learner's  logical  achievement  between  low  and 
high  level  of  authenticity  perception  (t=-1.75,  p<  .05). 


Achievement 

Authenticity  perception  level 

Mean  SD  t p 

Authentic  Achievement 

High 

1.58  .77  -4.54*  .00 

1.05  .88 

Low 

Logical  Achievement 

High 

1.69  1.01  -1.75  .09 

1.34  1.03 

Low 

Total  Achievement 

High 

3.27  1.56  -3.42*  .00 

2.42  1.99 

Low 

( * : p < .05 ) 


(2)  There  is  a significant  difference  in  learner's  motivation  of  a total  (t=-3.34,  p<.05),  attention(t=-3.16, 
p<.05),  relevance(t=-2.55,  p<.05),  confidence(t=-2.85,  p<.05)  and  satisfaction(t=-2.34,  p<,05) 
between  low  and  high  level  of  authenticity  perception. 


Motivation 

Authenticity  perception  level 

Mean 

SD 

t 

P 

Attention 

High 

3.68 

.57 

-3.16* 

.003 

Low 

3.44 

.52 

Relevance 

High 

3.64 

.50 

-2.55* 

,014 

Low 

3.47 

.53 

Confidence 

High 

3.47 

.42 

-2.85* 

.006 

Low 

3.29 

.41 

Satisfaction 

High 

3.40 

.46 

-2.34* 

.023 

Low 

3.25 

.45 

Total 

High 

3.45 

,42 

-3.34* 

.001 

Low 

3.27 

,40 

( * : p < .05 ) 


Conclusions 

Based  on  the  results  of  the  study,  following  suggestions  were  made:  (1)  Measuring  scale  of  authenticity 
of  the  learning  task  has  to  be  further  validated  through  many  different  situations  and  various  subjects 
(2)  Authentic  assessment  of  the  authentic  learning  environment  has  to  be  studied  further. 
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Abstract:  Wireless  Application  Protocol  (WAP)  provides  a universal  open  standard  for  bringing 
Internet  and  Intranet  content  and  advanced  Value  Added  Services  to  mobile  phones,  PDA’s  and 
other  wireless  devices  [1],  The  production  of  WAP  applications  and  content  requires  new  software 
professionals  on  the  area.  In  this  paper  we  describe  our  experiences  from  the  multimedia  course 
where  students  developed  WAP  applications  as  their  project  work[5]. 


Background 

WAP  is  an  industry  standard  whose  purpose  is  to  achieve  universal  mobile  access  to  Internet  based 
information  services.  The  application  is  independent  of  the  underlying  digital  wireless  network  technology  [1], 
WAP  applications  are  developed  with  Wireless  Markup  Language  (WML)  and  WML  Script.  [2],  WML  applications 
can  be  developed  with  special  toolkits,  for  example  Nokia  WAP  Development  Toolkit , Ericsson  WAP  development 
kit  or  UP. PHONE  development  environment . These  toolkits  include  smart  editors  and  debuggers,  phone  simulators 
and  help  system.  However,  its  is  possible  to  develop  your  WML  applications  using  only  a simple  ASCII  editor.  The 
students  developed  simple  WAP  applications  in  the  laboratory  work  and  implemented  a larger  project  work  in  the 
end  of  the  course. 

Wireless  Markup  Language  Specification  and  WML  Script 

The  Wireless  Markup  Language  specification  describes  the  markup  language,  WML,  including  the  document 
semantics,  Document  Type  Definition  (DTD),  and  encoding  extensions.  WML  is  a markup  language  based  on 
Extended  Markup  Language  (XML)  and  is  intended  for  use  in  specifying  content  and  user  interfaces  for  narrowband 
devices,  including  cellular  phones  and  pagers.  All  information  in  WML  is  organized  into  a collection  of  cards  and 
decks  [3].  WML  Script  is  a scripting  language  that  makes  it  possible  to  implement  dynamical  elements  in  the  WML 
application. 

Student’s  WAP  projects 

The  students  were  required  to  design  and  implement  a real  WAP  software  and  document  the  code  in  the  end  of 
the  course.  The  topic  of  the  project  was  free  and  many  of  the  students  chose  to  implement  a game  in  WAP  phone.  In 
this  chapter  we  will  present  three  WAP  games,  namely  Moonlanding  game.  Lucky  Wheel,  and  Jackpot.  The  idea  of 
the  Moonlanding  game  is  to  land  to  the  surface  of  the  moon  by  controlling  the  fuel  consumption  of  the  rocket  (Fig. 
3).  In  the  Lucky  Wheel  game  your  mission  is  to  find  out  a secret  word  or  words  by  guessing  the  missing  characters. 
Jackpot  is  the  WAP  implementation  of  the  old  casino  game  (Fig.  3).  Some  of  these  applications  are  being  developed 
furthermore  and  will  be  published  in  the  commercial  WAP  service  by  the  national  mobile  phone  operator. 

The  first  step  in  the  project  was  the  learning  of  the  WAP  development  environment.  The  Nokia  Wap  software 
toolkit  was  used  in  the  projects.  In  general,  programming  in  WML  and  WMLS  is  similar  than  programming  in 
almost  any  other  environment.  It  is  possible  to  use  the  toolkit  as  a stand-alone  environment  or  with  Nokia  WAP 
gateway  in  order  to  simulate  real  wireless  connections  [6].  In  our  case,  it  was  not  required  for  the  students  to  test 
their  software  in  real  phones.  During  the  development  of  their  software  the  students  find  out  several  problems.  One 
of  the  most  significant  problem  was  the  differences  between  the  real  phone  and  the  simulated  phone.  Of  course  the 
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toolkit  itself  was  beta-level  software  and  was  not  very  reliable.  In  addition  the  real  WAP  phone  does  not  full  fill  the 
WAP  specification  [4].  For  example,  the  Nokia  7110  WAP  phone  does  not  support  table  feature  in  the  WML 
content  and  the  WAP  applications  can  use  only  a little  memory  - the  memory  size  in  the  phone  is  appx.  2 KB. 

Figure  3.  The  moonlanding  game  and  the  Jackpot. 


Positive  aspects  in  the  project  were  that  the  project  was  very  motivating,  the  simulator  makes  it  possible  to  develop 
software  without  the  real  phone,  it  was  easy  to  learn  to  program  with  WML  and  WMLS  because  they  are  similar 
than  HTML  and  JavaScript  and  students  used  actively  WAP  discussion  groups  in  internet. 

Conclusions 

Increasing  global  competition,  digitalization  of  content  the  general  use  of  HTML  and  WML  languages  and  the 
adoption  of  mobile  broadband  data  transfer  will  require  software  specialists  with  new  skills.  In  the  future  content 
producers,  telecommunication  companies,  IT  companies  and  media  companies  need  workers  with  good  knowledge 
technological  convergence  and  good  ability  to  use  the  new  content  producing  technologies  like  WAP.  This  is  a 
demand  for  the  educational  institutes  and  universities.  We  should  provide  up-to-date  technological  education  and  be 
prepared  to  modify  our  curriculum  to  follow  the  latest  technological  innovations.  In  this  paper  we  have  discussed 
about  WAP  as  a new  technology  to  implement  wireless  content  services.  The  students  at  Tampere  University  of 
Technology  studied  the  WAP  technology  and  implemented  simple  WAP  applications  as  their  project  work.  The 
applications  were  developed  using  freely  downloadable  Nokia  WAP  development  toolkit.  The  most  significant 
problem  was  the  differencies  between  the  real  WAP  phone  and  the  simulated  WAP  phone.  From  our  point  of  view 
the  projects  succeeded  well:  some  of  these  applications  are  being  developed  furthermore  and  will  be  published  in  the 
commercial  WAP  service  by  the  national  mobile  phone  operator. 

References 

1.  Wireless  Application  Protocol  Forum  Ltd.  WAP  Architecture, 

http://www.wapforum.org/what/technical/SPEC-WAPArch-19980430.pdf,  version  30.4.  1998  (26.1.2000) 

2.  Wireless  Application  Protocol  Forum  Ltd  (1999).  Official  Wireless  Application  Protocol  The.  Complete  Standard 
with  Searchable  CD-ROM  Published  by  John  Wiley  & Sons,  Inc. 

3.  Pekka  Niskanen  WAP  Ohelmoijan  Kdsikirja,  IT  Press  1999  (Finnish) 

4.  Wap  Forum  (2000)  http://www.wapforum.org/  (24.1 . 2000) 

5.  Keiho  H.,  Koivisto  H.,  Jaakkola  H.,>4  new  business  strategy  for  teleoperators.  Opportunity  for  modern 
broadband  access  technologies.  Proceedings  of  the  lEMC  '98,  Puerto  Rico.  IEEE.  pp.  399-402. 

6.  Getting  Started  Guide:  Nokia  Wap  Server  7.0  November  1999.  Nokia  Mobile  Phones 


BEST  COPY  AVAILABLE 


1674 


Enhancing  Total  Patient  Management  Skills  in  Dentistry  with  Interactive  Multimedia  Simulation 

Mike  Keppell 

Biomedical  Multimedia  Unit,  Faculty  of  Medicine,  Dentistry  & Health  Sciences 
The  University  of  Melbourne,  Australia 
e-mail:  m.keppell@medicine.unimelb.edu.au 

Karen  Kan  & Louise  Brearley  Messer 

School  of  Dental  Science,  The  Faculty  of  Medicine,  Dentistry  & Health  Sciences 
University  of  Melbourne,  Australia 
e-mail:  k.kan@dent.unimelb.edu.au,  l.messer@dent.unimelb.edu.au 

This  paper  examines  a project  which  aims  to  improve  dental  clinical  management  through  case 
management  of  medically  compromised  dental  patients.  During  the  past  few  years,  it  has  become 
increasingly  difficult  to  access  suitable  patients,  especially  child  patients,  for  dental  students  and 
consequently  for  them  to  become  sufficiently  experienced  in  treating  patients  with  medical 
conditions.  Multimedia  simulation  provides  a means  to  fill  this  gap  and  allows  interactive  learning  in 
a non-threatening  environment.  This  paper  examines  the  design,  development,  project  management 
and  evaluation  of  the  first  module  - congenital  heart  disease. 

Teaching  dental  students  to  treat  patients  safely  and  efficiently  presents  many  challenges.  The  declining 
availability  of  suitable  patients  attending  the  teaching  clinics  of  the  Royal  Dental  Hospital  of  Melbourne 
prevents  students  from  developing  a range  of  experiences  in  total  patient  management.  A disadvantage  of 
traditional  preclinical  laboratory  teaching  is  that  students  are  not  able  to  integrate  theoretical  and  practical 
skills.  Consequently  there  are  concerns  that  dental  students  are  not  competent  in  combining  preventive  and 
restorative  management  philosophies  while  integrating  diagnosis  and  treatment  planning  (Suvinen,  Messer, 
Franco,  1998).  Multimedia  case  simulations  were  considered  to  be  a viable  alternative  as  they  replicate  the 
dental  clinic  without  requiring  ‘live’  patients.  These  modules  provide  an  opportunity  to  develop  and 
consolidate  the  concept  of  integrated  patient  care. 

During  1999  the  first  module,  congenital  heart  disease  (CHD)  was  designed  and  completed.  During  2000 
further  funding  will  allow  three  other  modules  to  be  developed:  diabetes,  Down  syndrome  and  cleft  lip  and 
palate.  These  medical  conditions  were  chosen  as  they  are  the  most  likely  medical  conditions  to  be 
encountered  and  are  least  understood  by  a dentist  in  general  practice  treating  children.  Each  condition  will 
follow  the  existing  template  developed  for  the  CHD  module  except  for  the  diabetes  module  which  will  use 
a 3-D  Dental  surgery  to  enhance  interactivity.  The  CHD  module  contains  a description  and  review  of  the 
medical  condition,  considerations  of  medical  management  and  dental  management.  The  student  is  then 
presented  with  a case  scenario  of  a child  with  CHD.  The  student  constructs  a treatment  plan  and 
management  approach  after  considering  a range  of  treatment  options.  The  module  series  will  replace  some 
tutorial  and  lecture  content  for  both  4^*^  and  final  year  dental  students.  The  modules  will  not  replace 
valuable  clinical  practice;  but  prior  learning  from  the  modules  will  enhance  students’  ability  to  perform 
clinical  work.  The  modules  will  also  be  used  for  post-graduate  students  and  in  Continuing  Dental 
Education  programs  for  dentists. 

Design  and  Development 

In  designing  and  developing  the  CHD  module,  the  project  team  followed  the  model  utilised  by  the 
Biomedical  Multimedia  Unit  (Keppell,  1998).  The  design  phase  of  the  project  focussed  on  developing  the 
macro-structure  for  the  module  in  the  form  of  a concept  map  (Novak  & Gowin,  1984)  and  completing  the 
micro-structure  in  the  form  of  a ‘planning  grid’.  The  content  experts  for  the  project  were  experienced 
specialists  and  educators  in  the  field  of  paediatric  dentistry.  They  worked  with  the  instructional  designer  to 
develop  the  concept  map  and  planning  grids  for  the  project.  Figure  1 http://www.incdfac.iiniinclb.cdu.au/Ed- 
Media2000/  Outlines  the  resultant  concept  map  which  provided  a scaffold  for  the  development  of  the  planning 
grids.  The  process  used  to  develop  the  concept  map  and  planning  grids  followed  the  Content  Production 
Process  (Keppell,  1999).  A specific  form  of  storyboard  or  ‘planning  grid’  has  been  developed  to  provide  a 
‘communication  tool’  for  interfacing  between  the  instructional  designer,  content  expert  and  graphic 
designer/programmer  (Keppell  & Buschgens,  1995).  The  planning  grid  is  useful  in  streamlining 
communication  and  providing  a common  ground  for  discussing  the  multimedia  project.  The  planning  grid 
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is  analogous  to  an  ‘architectural  blueprint’,  which  can  be  applied  or  ‘engineered’  by  the  graphic 
designer/programmer  (see  Figure  2 http://www.medfac.uninielb.edu.au/Ed-Medla2000/).  The  graphic  designers 
developed  the  interface  for  the  congenital  heart  disease  module  and  the  subsequent  modules  (see  Figure  3. 
http://www.medfac.unimeib.edu.au/Ed-Media2000A 

Project  Management  & Evaluation 

The  project  management  focussed  on  carefully  scoping  the  project,  providing  a detailed  planning  grid  to 
the  graphic  designer/programmer  and  establishing  the  “look  and  feel”  of  the  interface.  England  and  Finney 
(1999)  suggest  that  each  project  will  define  its  “own  quality  priorities”  (p.l7).  In  this  project  the  priorities 
of  target  audience,  subject,  budget  and  the  time  available  impacted  on  the  project  management  of  the 
project.  England  and  Finney  (1999)  also  suggest  that  “design  quality  for  media  projects  = content  and 
treatment  agreement”  (p.  1 7).  These  treatment  parameters  were  carefully  considered  in  the  initial  stages  of 
the  project  conceptualisation.  The  first  module  was  targeted  to  be  completed  within  an  hour  by 
undergraduate  users.  The  macrostructure  for  integrating  the  subsequent  modules  was  also  considered  as  an 
essential  characteristic  of  the  project.  In  order  to  analyse  project  costs  and  as  a means  to  plan  for  the  future 
modules,  the  total  work  hours  were  tracked  for  this  module.  A total  of  390  hours  was  required  to  produce 
the  CHD  module  (see  Table  1 http://www.medfac.unimelb.edu.au/Ed-Media2000A.  The  CHD  module  was  completed 
in  November,  1999  and  was  evaluated  with  a group  of  15  recent  graduates  (interns  and  postgraduate 
students)  at  the  School  of  Dental  Science  in  December  1999,  using  a Likert-style  questionnaire.  Table  2 
him://www.mcdfac.unimc]b.cdu.au/Ed-Mcdia2000/  shows  the  distribution  of  responses  to  questions  concerning 
content.  The  majority  of  all  responses  were  either  “strongly  agree”  or  “agree”,  with  most  modal  responses 
being  “strongly  agree”  (Table  3).  http://www.medfac.unimelb.edu.au/Ed-Media2000/  Most  respondents  correctly 
identified  the  deliberately  placed  distractor,  “confusing”.  In  general  appraisal,  the  majority  of  respondents 
rated  the  module  as  “good”  or  “excellent”,  and  “very  easy”  to  operate  and  navigate  (Table  4). 
htip://www.mcdfac.unimclb.cdu.au/Ed-Mcdia2000/  Of  interest,  One  respondent  rated  the  module  “very  poor”  and 
“difficult”  on  these  points.  The  module  is  currently  undergoing  beta  testing  at  three  national  and  three 
international  locations,  in  departments  of  pediatric  dentistry. 

Discussion 

Computer-assisted  education  in  dentistry  is  in  its  early  stages  (Yip  and  Barnes,  1999).  Developments  are 
occurring  very  rapidly,  and  educators  must  evaluate  modules  developed  internally  or  purchased  externally 
(Mercer  and  Ralph,  1 998).  Sound  instructional  design  indicates  the  goals  of  instruction  must  be  made  clear, 
so  that  users  know  what  is  expected  of  them  in  operating  and  navigating  a module.  This  may  not  have  been 
so  in  the  instance  of  one  respondent.  Nonetheless,  it  is  concluded  that  computer  assisted  instruction  was 
enthusiastically  received  by  most  of  the  recent  graduates.  Experience  in  focus  groups  has  shown  that 
participants  valued  the  tutor  discussions  that  were  held  at  the  end  of  the  module,  which  allowed  extension 
of  learning  and  further  professional  role-modelling. 
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Abstract:  At  the  George  Washington  University  teacher  candidates  in  the  master’s  program 
learn  to  apply  technology  in  teaching  and  learning  within  the  context  of  methods  and 
curriculum  courses  and  school-based  practicum  and  internships.  Teacher  candidates  acquire 
skills  and  understanding  as  they  prepare  presentations,  develop  curriculum  materials,  teach 
lessons,  assess  student  learning,  and  reflect  on  practice.  This  short  paper  describes  the  ways 
in  which  teacher  candidates’  development  of  multimedia  electronic  portfolios  to  assess  their 
ability  to  implement  a variety  of  instructional  models  assists  in  learning  to  use  a variety  of 
hardware  and  software  for  instructional  purposes.  It  is  expected  that  teacher  candidates  will, 
on  some  level,  transfer  their  proflciency  in  using  technology  to  the  secondary  students  with 
whom  they  work  during  their  teaching  internships. 

Why  Electronic  Portfolios? 

Performance  assessment  in  teacher  education  programs  commonly  includes  development  of  a teaching 
portfolio  during  the  teacher  internship.  The  purposes  of  the  portfolio  are  to  document  professional  growth 
(Campbell,  et.  al.  2000)  and  to  cultivate  outstanding  teaching  and  learning  (Wolf,  1996).  Fulfilling  these  purposes 
requires  that  the  portfolio  is  a dynamic,  flexible  documentation  of  student  growth.  Emerging  technologies  facilitate 
creating  electronic  portfolios  that  support  flexibility  rather  than  traditional  portfolios  constructed  in  3-ring  binders 
that  tend  to  be  collections  of  artifacts.  The  electronic  format  offers  several  advantages:  the  ease  of  storage  usually  on 
a computer  hard  drive  or  some  sort  of  removable  media  (floppy  disk,  Zip  disk,  CD);  easy  access  to  information; 
portability  is  enhanced;  multiple  copies  can  be  created  at  low  cost;  the  opportunity  to  add  sound,  pictures,  graphics, 
and  even  video  make  an  electronic  portfolio  more  attractive  and  interactive  than  traditional  style  binders;  in  the 
process  of  developing  portfolios,  students  gain  valuable  technology  skills  as  they  create  and  edit  this  multimedia 
document.  These  skills  can  be  transferred  to  6-12  students  during  the  internship  and  as  the  successful  teacher 
candidate  moves  into  classroom  teaching. 

Development  of  the  Electronic  Portfolio 

A group  of  teacher  candidates  in  the  secondary  education  graduate  program  at  The  George  Washington 
University  are  expanding  their  technology  proficiencies  through  developing  an  electronic  portfolio.  The  primary 
purpose  is  to  provide  teacher  candidates  with  technology  related  skills  that  they  can  transfer  to  middle  and  high 
school  students  and  use  to  support  their  teaching.  A secondary  purpose  is  to  enable  teacher  candidates  to 
demonstrate  their  teaching  competencies  using  an  alternative  method  to  the  binder-style  portfolio  currently  in  use  in 
many  teacher  preparation  programs. 

The  conceptual  framework  used  to  develop  the  portfolios  is  based  on  the  Interstate  New  Teacher 
Assessment  and  Support  Consortium  Standards  (Darling-Hammond,  1992)  and  the  National  Board  for  Professional 
Teaching  Standards  (1991).  The  domains  addressed  include:  1)  commitment  to  students  and  their  learning,  2) 
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knowledge  of  the  subjects  taught  and  how  to  teach  those  subjects  to  students,  3)  establishment  and  management  of 
student  learning  in  a positive  environment,  4)  assessment  of  student  progress  and  adapting  instruction  to  improve 
student  learning,  5)  commitment  to  reflective  practices,  and  6)  professionalism.  Several  indicators  are  provided  to 
students  within  each  domain  specifying  ways  in  which  each  is  exemplified  in  teaching  or  in  course  work. 

Those  participating  are  currently  enrolled  (spring  2000)  in  a class  that  teaches  application  of  instructional 
and  classroom  management  models.  The  instructional  models  reviewed  include  Direct  Instruction,  Inductive, 
Inquiry,  Integrative,  and  Cooperative  Learning.  Each  of  the  models  represents  a cognitive  perspective  of  learning 
which  views  learners  as  active  investigators  of  their  environment.  In  these  models,  teachers  are  directly  involved  in 
guiding  learning  through  questioning  and  discussion.  Candidates  construct  a number  of  products  to  show  their 
understanding  of  concepts  and  theory  and  their  ability  to  apply  them  in  a teaching-learning  situation. 

Specifically,  teacher  candidates  learn  how  to  use  various  technologies  as  they  learn  pedagogy.  In  the 
course,  they  will  practice  using  technology  to:  present  information;  research  content;  organize  and  store  data;  track 
and  record  their  development  as  teachers;  and  reflect  on  their  teaching  behaviors  as  a process  for  solving  problems 
and  growing  professionally.  For  example,  they  will  learn  to  use  PowerPoint  in  the  context  of  learning  how  Howard 
Gardner’s  Theory  of  Multiple  Intelligences  applies  to  the  teaching  of  science,  math,  English,  social  studies  or  ESL. 
Each  candidate  will  design  a presentation  that  outlines  how  each  of  the  8 intelligences  is  used  to  frame  curriculum 
development  of  a specific  unit  topic.  Candidates  select  topics  according  to  their  content  specialization.  Each  will 
focus  on  the  utility  of  the  theory  in:  1)  designing  curriculum,  2)  engaging  and  motivating  students,  and  as  3) 
assessing  students’  learning. 

In  demonstrating  their  understanding  of  and  ability  to  apply  instructional  models,  candidates  will  teach 
lessons  they  develop  to  other  members  of  the  class.  They  will  be  videotaped  using  a digital  video  camera.  After 
receiving  feedback  from  peers  and  the  course  instructor,  the  teacher  candidates  will  view  the  video  and  reflect  on  the 
effectiveness  of  their  teaching.  As  a part  of  the  reflective  process,  they  will  edit  their  video  to  capture  those  portions 
that  exemplify  what  they  have  learned  about  their  teaching  and  will  include  this  shortened  version  in  the  electronic 
portfolio.  In  the  process  of  creating  the  portfolio,  teacher  candidates  will  learn  how  to  create  a personal  website. 
The  goal  is  to  expand  their  portfolios  as  they  move  through  the  graduate  program.  Unlike  what  typically  occurs  in 
the  binder  approach  to  portfolios  in  which  students  organize  a collection  of  artifacts  during  the  teaching  internship 
which  illustrate  their  best  work,  the  approach  taken  here  will  concentrate  on  documenting  growth  as  teacher 
candidates  develop  their  knowledge  and  skills  in  both  pedagogy  and  content.  This  means  that  candidates  will  begin 
assembling  their  work  during  their  initial  experiences  in  the  program  and,  the  work  included  in  the  portfolio  will  not 
necessarily  showcase  their  best  products.  Rather,  the  final  product  will  reveal  the  ways  and  extent  to  which  each 
candidate  has  grown  as  a beginning  teacher. 

Conclusion 

One  major  area  of  emphasis  in  the  teacher  preparation  program  at  the  George  Washington  University  is  to 
prepare  teachers  who  will  serve  as  leaders  in  the  schools  where  they  teach.  An  important  aspect  of  that  leadership 
role  is  technology.  As  our  students  develop  their  proficiency  in  this  area,  they  will  be  encouraged  to  explore  the 
connections  to  teaching  in  middle  or  high  school.  The  ways  in  which  technology  use  can  enhance  teaching  and 
learning  is  of  particular  importance.  Specifically,  through  the  development  of  electronic  portfolios,  these  teacher 
candidates  will  consider  how  the  use  of  technology  can  assist  secondary  level  students  in:  learning  content; 
presenting  information;  conducting  research;  and,  reflecting  on  their  development  as  learners. 
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Abstract:  This  paper  is  based  on  experiences  with  the  production  of  web-based  training  (WBT)  courses  for  a 
course  of  studies  in  medical  computer  science.  Course  production  is  regarded  as  software  development  process, 
which  starts  with  conceptual  models  shaping  the  design  and  defining  the  way  the  software  product  is  presented 
to  the  user.  This  paper  reports  experiences  with  conceptual  design  models  and  shows  the  danger  of  being 
insufficient  or  incorrect. 


INTRODUCTION 

Our  project  “Distance  Education  in  Medical  Computer  Science”  (started  in  January  1999)  aims  at  providing  a 
complete  course  of  studies  for  the  specialization  of  students  in  medical  computer  science,  offered  at  a virtual 
university.  Our  responsibility  is  to  transfer  the  linear  text  documents  (mostly  MS-Word  format)  into  hypermedia 
networks  and  multimedia  courses,  which  means  practicing  reverse  engineering  for  WBT  course  development. 

PROBLEMS  WITH  CONCEPTUALIZING  WBT  COURSES 

The  transfer  of  a text  document  to  hypermedia  means  transformation  of  format,  gain  of  flexibility,  interaction 
capabilities  and  so  on.  Hypermedia  and  multimedia  courses  can  be  used  in  ways  beyond  the  intentions  of  the 
content  author.  On  the  other  side,  people  occupying  other  roles  in  the  development  process,  like  multimedia 
designers,  or  quality  managers  do  not  know  about  the  content  domain  or  didactic  aspects.  The  static  version, 
which  the  content  author  had  in  mind,  has  only  one  dimension  of  presentation.  Accordingly,  the  semantic 
(content)  structure  and  the  didactic  structure  of  the  course  are  intertwined.  Most  authors  are  not  able  to  separate 
both  structures  in  a way  applicable  for  hypermedia  production  and  vice  versa  the  multimedia  experts  are 
normally  no  experts  in  the  semantic  domain  or  in  didactics,  therefore,  they  can  hardly  explain  their  needs  to  the 
content  authors.  This  lack  of  knowledge  in  each  others  domain  results  in  unintended  course  structures, 
unnecessary  restrictions  or  other  mismatches.  A similar  problem  arises  if  the  author  for  example  has  to  start  with 
specifying  a semantic  model  and  then  would  have  to  design  for  different  learner  groups. 

Furthermore,  we  found  in  our  projects,  that  media  designers,  producers,  and  content  authors  often  have 
insufficient,  incomplete,  and  partly  incorrect  models  about  the  learners,  the  task  and  the  situation  of  use.  The 
models  vary  at  different  times  during  the  period  of  course  production. 

COURSE  PRODUCTION  AS  SOFTWARE  DEVELOPMENT  PROCESS 

Our  approach  to  course  production  as  practised  in  our  projects  is  to  regard  the  production  of  hypermedia  and 
multimedia  WBT  courses  as  a software  development  process.  As  for  many  other  software  systems,  it  is  also  true 
for  the  development  of  WBT  courses:  To  realize  a user  and  task  adequate  human-computer-system,  developers 
have  to  take  into  consideration  all  system  components:  user,  task  and  situation  in  order  to  form  a conceptual 
model  that  is  as  adequate  as  possible  [1].  The  conceptual  model  of  the  developers  shape  the  software  product,  in 
our  case  the  WBT  course,  and  therefore,  will  decide  how  usable  and  user-friendly  the  WBT  course  will  be  and 
how  adequately  it  will  support  learning  processes. 

Figure  1 shows  the  different  models  we  have  to  deal  with  in  the  context  of  WBT  course  production.  First  of  all 
there  is  the  developer’s  mental  model  the  design  process  starts  with.  In  the  case  of  WBT  course  development  we 
have  to  deal  with  diverse  developers.  In  our  project  there  is  the  conceptual  models  of  the  content  author,  of  the 
designer,  of  the  HT/MM  producer  and  of  the  quality  manager.  Furthermore,  there  is  the  learner’s  mental  model 
about  the  application  domain,  how  the  WBT  course  is  implemented  and  how  it  supports  learning  processes.  And 
last  but  not  least,  there  is  the  system’s  model,  which  contains  the  way  the  application  domain  is  implemented 
and  sometimes  information  on  the  user’s  interaction  with  the  system.  Each  model  represents  the  complex  domain 
of  interaction  with  WBT  courses  from  a certain  perspective  and  in  a specific  way. 
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Figure  1 : Models  in  the  context  of  WBT  course  production 


Mismatches  between  these  models  lead  to  usability  problems  of  the  software  product.  Above  all,  the  importance 
of  the  developer’s  conceptual  model  for  shaping  the  software  product  has  been  noticed  for  other  kinds  of 
software  applications  for  a long  time  [2],  [3].  According  to  our  view,  this  holds  true  for  WBT  courses  as  well. 

CONCEPTUAL  MODELS  OF  DEVELOPERS 

A fundamental  problem  for  the  development  process  is,  that  the  developers  need  substantial  and  relevant 
information  about  the  user,  the  task  and  the  situation  of  use.  They  need  this  information  from  the  very  beginning 
of  the  design  process  in  order  to  specify  the  requirements  and  shape  the  WBT  course  adequately.  However,  in 
practice  content  authors  start  course  production  in  the  traditional  way  and  produce  a linear  document  that 
includes  their  conceptual  model  about  domain  knowledge  and  about  teaching  strategies.  This  production 
methods  corresponds  to  the  production  of  traditional  lectures  for  classroom  teaching.  These  courses  will  not 
match  the  situation  of  use  in  the  virtual  university  of  the  future.  Lectures  on  demand  have  to  serve  the  needs  of 
different  user  groups  and  different  situations  of  use.  The  attributes  for  users,  the  task  and  the  situation  of  use  will 
differ  [4]  and  should  have  an  adequate  representation  in  the  conceptual  model  of  all  authors.  A severe  drawback 
with  this  situation  is,  that  it  is  impossible  for  a content  author  (and  also  for  the  other  developers)  to  take  into 
consideration  all  these  attributes  at  one  time. 

In  some  way,  of  course,  the  author  is  aware  of  the  situations  of  use  and  the  complexity  involved.  This  might  be 
the  reason,  why  the  conceptual  model  about  the  user  group  and  the  assigned  attributes  differ  from  phase  to  phase 
of  the  development  process.  This  is  a problem  as  well.  If  the  conceptual  model  of  the  developers  includes  a 
prospective  mental  model  of  the  user  and  if  the  design  is  based  on  this  model,  then  it  will  be  clear  that  varying 
the  conceptual  model  during  the  development  process  will  result  in  an  inconsistent  WBT  course,  which  will  be 
hard  to  understand  and  will  not  support  learning  processes. 

FUTURE  WORK 

The  conceptual  models  of  the  diverse  developers  shape  the  WBT  course  and  determines  the  way  the  educational 
system  is  presented  to  the  user  and  determines  if  it  supports  learning  processes.  Therefore,  it  is  important  for  the 
developers  to  have  sufficient  and  appropriate  knowledge  about  the  user,  the  task  and  the  situation  and  it  is  also 
important  that  the  conceptual  models  of  the  diverse  authors  can  be  communicated  and  complement  each  other 
during  the  design  process. 

With  this  background  we  are  working  on  methods  to  adequately  represent  conceptual  knowledge  for  the  design 
process  [5].  We  also  want  to  offer  ways  to  communicate  relevant  knowledge  components  between  all  developers 
through  the  whole  development  process. 
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Abstract:  The  goal  of  this  work  was  to  develop  a set  of  recommendations  for  the  use  of 
videoconferencing  in  large  classrooms.  The  students'  acceptance  of  synchronous 
videoconferencing,  the  role  of  the  instructors  as  well  as  the  influence  of  videoconferencing  on 
motivation  and  interactivity  were  in  the  focus  of  this  study. 

Based  on  the  statements  of  178  evaluated  students  there  are  three  main  recommendations  that 
can  be  made:  (1)  The  didactic  approach  of  the  course  must  concentrate  on  an  more  active 
involvement  of  the  participants.  (2)  The  instructor  must  be  willing  to  adapt  his  teaching  style  to 
the  integration  of  new  media.  (3)  Technic  has  to  be  so  perfect,  that  it  guarantees  the  first  two 
aspects:  an  undisturbed  transmission  with  possibilities  to  interact  for  all  participants  and 
support  of  the  instructor  to  make  teaching  easy. 


Introduction 

A lot  of  studies  report  about  the  benefits  of  videoconferencing  in  the  context  of  small  classes  with  active  student 
participation.  Videoconferencing  in  small  classes  has  been  shown  to  increase  student  motivation  as  well  as 
interaction  among  students  and  lecturers  often  leading  to  a redefinition  of  teaching  (among  others  Abotte,  Dallat, 
Livingston,  & Robinson,  1994;  Bramble  & Martin,  1995).  Experiences  with  videoconferencing  in  the  large 
classroom  context  however  differ  at  least  in  part  in  some  of  these  results.  Increased  inhibition  to  speak  in  front  of 
others  and  lowered  spontaneity  seemed  to  be  real  problems  (Knox,  1997).  Further,  an  improved  motivation  or 
satisfaction  was  not  found  (Pillion,  Limayen,  & Bouchard,  1999).  The  purpose  of  this  study  was  to  find  out, 
which  aspects  are  the  most  crucial  for  the  use  of  videoconferencing  in  large  classrooms. 


Method 

Equipement 

The  videoconferencing  facility  used  in  this  evaluation  is  a synchronous  system  based  on  ATM.  Its  theoretical 
bandwidth  of  up  to  155  Mbits/s  allows  the  simultaneous  transmission  of  three  audio-  and  video-streams  with 
high  quality  and  perfect  audio-video  synchronisation  between  two  and  more  connected  sites.  The  system 
provides  full  interaction  possibilities  to  all  participants  at  all  connected  sites. 

The  setting  in  the  examined  lecture  is  a two-point  configuration  consisting  of  the  following  components: 

Remote  classroom:  Three  screens,  projected  by  beamers,  with  1.)  the  instructor,  2.)  the  PowerPoint  slides  and  3.) 
the  other  class. 

Local  classroom:  Two  screens,  projected  by  beamers,  with  1.)  the  PowerPoint  slides  and  2.)  the  remote  class.  For 
further  technical  details  see  (Walter  & Hanni,  1998). 

Participants 

The  examined  sample  consisted  of  totally  1 78  participants  of  a two  year  course  in  pharmaceutical  chemistry  (86 
students  in  the  locat  classroom;  92  students  in  the  remote  classroom).  Gender  representation  was  not  equal  (139 
female  and  39  male). 

Control  data  (normal  course  without  videoconference)  were  collected  from  students  of  both  locations  during 
summer  1999.  The  experimental  data  resulted  from  the  first  half  (13  weeks)  of  the  academic  year  1999/2000. 

The  constructs  and  its  components  consisted  of  items  which  were  measured  using  a five-point  Likert-type  scale 
(l=strongly  disagree  to  5=strongly  agree). 
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Results  and  discussion 


The  following  presented  results  were  based  on  a minimum  level  of  significance  of  p<0.001. 

Acceptance  and  Motivation 

As  expected,  students  in  the  local  classroom  accepted  with  M=3.3  the  videoconference  better  than  remote 
students  (M=2.7).  Furthermore  there  was  a significant  difference  in  the  acceptance  which  is  correlated  with  the 
infrastructure:  While  in  location  1 the  auditorium  used  for  the  transmission  consisted  of  a permanent  equipped 
room,  location  2 had  to  build  up  all  equipment  for  the  lecture  every  week. 

In  contrast  to  studies  who  reported  about  increased  motivation,  the  measured  motivation  construct  (consisting  of 
subject  interest  and  self-efficacy)  remained  the  same  with  versus  without  videoconference.  Nevertheless,  with 
r=0.57,  motivation  was  one  of  the  strongest  predictor  of  the  acceptance. 

Influence  of  the  quality  of  the  transmission 

The  participants  rated  the  quality  of  audio  (M=3.9),  the  level  of  synchronisation  (M=3.7)  as  well  as  the  quality  of 
the  transmitted  videos  (picture  of  the  lecturer  (M=3.8)  and  projection  of  the  slides  (M=4.1))  more  than 
satisfying.  Only  the  video  of  the  remote  auditorium  was  assessed  average  (M=2.9).  The  quality  of  transmission 
seems  to  be  so  good,  that  there  were  no  correlations  greater  than  r=.34  with  the  acceptance  or  the  motivation. 

Role  of  interactivity 

The  largest  impact  on  the  acceptance  of  a videoconference-classroom  exerted  the  possibilities  of  interaction. 
Students  assessed  this  kind  of  lecture  in  comparison  with  a normal  classroom  less  stimulating  for  discussions 
between  them  and  the  lecturer  (M=2.5).  This  variable  had  with  r=.61  thew  strongest  correlation  with  acceptance. 
The  hypothesis,  that  students  were  more  inhibited  to  ask  questions  during  the  class,  can  be  accepted.  However, 
the  level  of  inhibition  was  lower  than  expected  (M=3.5).  With  r=-.27  (p<0.002)  the  correlation  between 
inhibition  and  acceptance  never  reached  the  expected  level. 

The  possibilities  of  spontaneous  intervention  were  even  rated  slightly  higher  than  in  the  traditional  classroom 
setting  (M=3.2)  and  did  not  coincide  with  the  students’  actual  hesitance.  This  high  rating  is  most  likely  due  to  the 
lecturers'  efforts  to  integrate  prepared  questions  in  their  slides.  Its  correlation  of  r=.43  with  the  acceptance 
underlined  the  importance  of  the  possibility  to  interrupt  the  speaker  at  any  time. 

Role  of  the  instructors 

The  correlation  of  r=.38  between  the  acceptance  of  videoconferencing  and  the  performance  rating  of  the 
lecturers  (with  M=3.6  both  were  assessed  above  average)  emphasized  the  important  role  of  the  instructors. 

It  is  encouraging  to  note,  that  the  implementation  of  videoconferencing  had  positive  side  effects  on  teaching:  The 
cooperation  of  the  two  involved  professors  lead  to  an  overall  improvement  of  the  curriculum  with  better 
structure,  more  goal  orientation  and  the  integration  of  prepared  questions. 


Recommendations 

Based  on  these  results,  the  following  guidelines  are  proposed  for  the  use  of  videoconference  systems  in  the 
lecture  context  with  groups  consisting  of  more  than  20  participants  on  each  side: 

1. )  Lecturers  have  to  consciously  plan  possibilities  to  interfere  and  to  create  an  atmosphere  that  motivates 
students  to  undertake  a more  active  part. 

2. )  The  instructor  must  be  above  average  and  be  willing  to  accomplish  an  extra  effort. 

3. )  The  technical  infrastructure  must  guarantee  a clear  transmission  of  video  and  audio  as  well  as  the  possibility 
of  interaction  at  anytime  for  all  participants. 
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Abstract:  This  study  is  intended  to  understand  collaborative  learning  in  a context  that  enables 
communication  by  computer  text  messaging.  Particularly,  the  study  seeks  to  determine  the  level  of 
collaboration  in  a classroom  that  uses  computer-mediated  communication  (CMC),  as  well  as  to 
examine  the  relationship  between  the  language  ftinctions  associated  with  CMC  generated  messages 
and  the  collaborative  learning  process.  In  this  study,  "language  tunctions  refer  to  ways  we  can  use 
language  to  achieve  a communicative  purpose"  (Olsen,  1992).  It  is  what  people  "can  do  with 
language"  (McDonell,  1992),  or  "what  people  want  to  do  with  the  language"  (Finocchiaro  & 
Brumfit,  1983,  p.l3),  such  as:  requesting  information,  requesting  confirmation,  making  suggestion, 
apologizing,  etc. 


The  study  is  prompted  by  following  questions 

1 . What  are  the  levels  of  collaboration  (in  terms  of  interactivity)  in  the  classroom  that  uses  CMC? 

2.  Are  there  any  patterns  in  people  using  of  language  functions  in  relation  to  the  degree  of  collaboration  (in 
terms  of  interactivity)  in  the  context  of  CMC?  If  yes,  what  are  the  patterns? 

a) .  Are  there  any  patterns  in  people  using  "requesting  information"  in  relation  to  the  degree  of  their 
collaboration  (in  terms  of  interactivity)? 

b) .  Are  there  any  patterns  in  people  using  "requesting  confirmation"  in  relation  to  the  degree  of  their 
collaboration  (in  terms  of  interactivity)? 

Computer-mediated  communication  (CMC),  is  an  effective  electronic  means  of  connecting  geographically  dispersed 
learners  using  computers,  in  order  that  they  may  collectively  participate  in  a learning  experience  where  the 
interaction  is  mediated.  Despite  it  relatively  short  history  as  a distance  medium,  a great  deal  of  interest  has  been 
shown  in  this  technology. 

Courses  offered  by  CMC  are  gaining  favor  in  schools  and  institutions  of  higher  education.  Students  can 
interact  at  a time  and  in  a place  that  is  personally  convenient,  discuss  issues  as  fully  as  desired  without  constraints 
(Henri,  1995).  CMC  promotes  interactive  activities  such  as  active  dialogue  and  sharing  of  information  and 
knowledge,  which  help  develop  a sense  of  community  among  participants  (Burge,  1993).  The  CMC  can  be  used  as 
a powerful  tool  for  group  communication  and  collaborative  learning  (Crook,  1995).  The  collaborative  learning 
environment  promotes  active  mental  engagement  in  the  experience  (Grabowski,  1 990). 

Despite  the  large  body  of  literature  on  peer  learning  and  teaching,  there  are  few  acknowledgments  of  how  well  CMC 
may  support  collaborative  learning.  Although  cognitive  and  social  psychology  have  already  provided  much 
information  about  how  students  communicate  and  its  relationship  with  collaborative  learning,  the  context  has 
usually  been  the  traditional  face-to-face  setting  with  its  paralinguistic  cues  to  facilitate  communication.  What  we  do 
not  yet  know  in  published  detail  are  how  people  learn  collaboratively  and  how  this  collaborative  learning  relates  to 
the  communication  in  the  CMC  context.  In  addition,  O'Malley  (1995)  calls  for  studies  of  collaborative  learning  that 
"focus  more  on  the  processes  involved  in  successful  peer  interaction,  rather  than  just  on  learning  outcomes". 

Therefore,  this  study  seeks  to  understand  how  people  learn  collaboratively  and  how  their  collaborative 
learning  relates  to  the  communication  in  the  context  of  CMC.  By  asking  these  research  questions,  I assume  that  if 
we,  as  educators,  know  more  about  the  process  of  collaborative  learning  in  a computer-mediated-communication 
environment,  we  have  a better  chance  of  developing  practical  guidelines  for  facilitating  learning.  In  addition,  the 
information  obtained  from  the  study  may  contribute  to  the  present  understanding  of  students  learning  in  CMC 
environment  and  may  suggest  future  studies  in  the  area. 

For  instance,  numerous  studies  (Johnson  & Johnson,  1989)  dealing  with  collaborative  learning  have 
suggested  that  students  need  appropriate  social  skills  in  order  to  be  successful  in  their  collaborative  learning. 
Johnson  and  Johnson  (1989)  emphasize  that  collaboration  is  the  most  important  and  basic  form  of  human 
interaction,  and  the  skills  of  collaborating  successfully  are  the  most  important  skills  anyone  needs  to  master.  Coelho 
(1992)  explains  that  good  social  skills  can  promote  collaborative  learning  and  some  negative  behaviors  can  shut  off 
future  collaboration.  In  addition,  the  literature  on  collaborative  learning  emphasizes  the  need  to  teach  social  skills. 
Students  need  assistance  in  acquiring  those  social  skills  and  many  of  these  social  skills  can  be  regarded  as 


communication  skills.  For  example,  Olsen  and  Kagan  (1992)  state  that  "social  skills  include  ways  students  interact 
with  each  other  to  achieve  activity  or  task  objectives  (e.g.,  asking  and  explaining)  and  ways  students  interact  as 
teammates  (e.g.,  praising  and  recognizing)". 

However,  all  these  conclusions  are  based  on  the  works  that  have  been  done  in  traditional  face-to-face 
settings.  Are  the  conclusions  still  true  in  the  CMC  environment?  This  remains  to  be  answered  and  it  raises 
important  questions  in  the  field  of  collaborative  learning.  This  study  will  try  to  understand  how  students  go  about 
communicating  and  learning  in  the  CMC  environment.  Once  we  have  this  understanding,  we  might  then  compare 
the  CMC  learning  strategies  with  those  reported  for  face-to-face  classroom  contexts  in  order  to  search  for  new 
practices  to  facilitate  collaborative  learning  and  better  suited  to  the  CMC  context.  And  only  when  we  have  a better 
understanding  of  computer  mediated  learning,  we  can  make  a better  use  of  CMC. 

The  subject  of  this  study  will  be  a grade  5-6  classroom  in  an  inner  elementary  school  in  Toronto. 
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Introduction  and  Overview 

Two  distinct  fields  of  educational  activity  should  inform  instructional  design  of  courses  using  teaching 
technologies.  These  are  cognitive  theory  and  instructional  theory.  Cognitive  psychologists  tend  to  analyze 
cognitive  structures  and  processes  at  a conceptual  level  without  due  regard  to  how  these  may  be  implemented  in 
the  instructional  setting.  Conversely,  content  designers  and  instructors  do  not  often  recognize  the  value  of 
undertaking  the  analysis  of  content  and  teaching  practices  from  a cognitive  analytic  perspective  (Reusser,  1993). 
We  propose  that  one  means  to  combine  the  best  practices  of  both  fields  in  teaching  and  learning  is  to  ensure  that 
the  instructional  design  process  integrates  both  perspectives.  Three  principles  inform  the  proposed  framework  of 
design. 

• Selected  models  of  instruction  that  best  support  the  teacher’s  purpose  for  mastery  of  content,  and  the 
selection  of  enabling  technologies  that  mirror  the  selected  model  of  instruction  are  combined  in  the  design 
process. 

• Models  of  instruction  and  enabling  technologies  are  selected  according  to  the  nature  of  the  content  to  be 
learned,  the  teaching  styles  and  preferences  of  instructors,  and  the  learners  concerned. 

• Promoting  and  supporting  development  of  the  active  nature  of  the  learner  underscores  the  entire  design 
process. 

The  development  of  this  framework  took  place  through  a recursive  process  of  experimentation  and  teaching  in 
two  different  university  courses.  The  campus  course  uses  concept-mapping  software  as  a teaching  aid  and  as  a 
study  aid  for  students  needing  to  master  the  more  difficult  and  abstract  content  areas  in  the  study  of  German 
Thought  and  Culture.  The  software  used  was  chosen  for  its  ability  to  mirror  the  ‘meaningful  learning’  model  of 
instruction  developed  by  (Ausubel,  Novak  & Hanesian,  1978)  and  (Novak,  1998).  The  online  course. 
Introduction  to  Academic  Writing,  uses  a Web  site  as  the  primary  teaching  tool  and  its  design  mirrors  the  ‘expert 
instructional  scaffolding’  model  of  learning  developed  by  (Hildyard,  1 996).  This  model  supports  the  need  to 
ladder  content  into  instructor  supported  incremental  steps  that  allows  modeling  of  best  writing  practices  through 
interactive  feedback  processes. 


UsingVisual  Organizers  and  Meaningful  Learning  to  Teach  Cultural  History 

Students  unfamiliar  with  cultural  history  are  frequently  presented  with  a huge  number  of  facts  and  dates  that 
have  little  significance  to  them,  encouraging  - if  not  forcing  - them  to  acquire  as  much  as  they  can  through  rote 
learning.  The  nature  of  the  assignments  and  examinations  often  further  reinforces  rote  learning  and  tends  to 
discourage  the  synthesis  and  creative  application  of  the  material.  The  use  of  visual  organizers  coupled  with 
instructional  design  based  on  a model  of  meaningful  learning  permits  the  student  to  see  how  the  information 
communicated  in  the  course  is  interrelated  internally,  and  also  how  it  is  connected  with  students’  prior 
knowledge.  Relationships  and  interrelationships  can  be  seen  and  apprehended  quickly  and  easily  when 
represented  visually. 

Concept  mapping  conveys  information  by  reducing  it  to  concepts  (perceived  regularities  in  objects  or  events) 
and  propositions  (significant  combinations  of  concepts),  which  are  then  structured  hierarchically  and  linked  by 
means  of  lines  representing  the  relationships  between  concepts  and  labeled  accordingly.  Each  concept  is  thus 
already  presented  in  a contextual  relationship  with  other  concepts.  This  device  can  be  used  either  as  a visual  aid 
in  teaching  or  more  actively  as  an  aid  to  learning.  Novak’s  strategy  of  concept  mapping  is  based  on  Ausubel ’s 
principles  of  meaningful  learning  designed  to  promote  retention  and  subsequent  related  learning.  These 
principles  include  the  following: 
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• The  learner  must  have  prior  knowledge  related  to  the  new  information  to  be  learned 

• The  new  material  must  be  relevant  to  other  information  and  must  contain  significant  concepts 

• The  learner  must  choose  to  learn  meaningfully,  rather  than  by  rote. 

We  focus  on  concept  mapping  as  the  most  viable  strategy  to  link  prior  knowledge  with  relevant  new 
conceptual  material,  using  the  CMapTools  software  package  to  support  this  strategy.  The  software  has  the 
features  needed  for  the  strategic  learning  practices  outlined  above;  it  is  downloadable  as  freeware  and  it  is  easy 
to  learn  and  use. 


Using  ‘Expert  Scaffolding’  and  the  Web  to  Teach  Academic  Writing  Skills 

The  purpose  of  this  course  is  to  provide  the  foundation  for  scholarly  writing  that  students  can  build  on  through 
further,  related  course  work  and  through  application  of  their  writing  skills  in  other  university  courses.  Because 
writing  skills  are  best  acquired  sequentially,  the  material  in  this  course  is  scaffolded  into  linked,  incremental 
units  that  build  on  each  other.  Throughout  the  course,  the  instructor  provides  expert  modeling  through  feedback 
as  evaluative  discourse.  Using  WebBoard  as  a vehicle  for  threaded  discussion,  students  interact  with  an 
experienced  writer  who  supports  them  until  skills  are  mastered  at  each  level.  Drawing  from  (Danger  and 
Appleby,  1986),  four  design  principles  for  scaffolding  were  used  to  inform  teaching  practice  and  design  of  the 
Web  site 

• Students  were  encouraged  to  assume  ownership  of  their  writing  tasks 

• The  assignments  were  designed  to  address  students’  incremental  levels  of  knowledge 

• The  assignments  were  designed  to  support  the  sequence  of  skills  acquisition  in  the  course 

• The  instructor  and  tutors  supported  students  in  a facilitative  manner 

Successful  teaching  in  a course  of  this  nature  requires  providing  ongoing  feedback  using  multi-faceted  dialogue 
with  students  through  commentary  on  writing  and  involvement  in  peer  editing  sessions.  Feedback  comes  from 
the  instructor  and  tutors  and  from  other  members  of  the  student  group.  The  course  begins  with  a shared  basis  of 
readings  - reading  groups  which  present  their  interpretation  of  texts  - and  continues  with  discussion  groups 
which  assist  writers  to  refine  and  elaborate  their  thoughts,  and  concludes  with  peer  revision  and  editing  groups. 
Thus  evaluation  is  embedded  in  the  design  of  the  course  and  specifically  with  the  design  of  the  assignments.  The 
assignments  and  final  project  include  interactive  exercises,  instructor  and  peer  commentary  and  editing,  and 
online  areas  for  discussing  course  content. 


Conclusion 

To  date,  we  have  successfully  demonstrated  in  two  courses  that  instructional  theory  and  enabling  technologies 
can  be  combined  in  a useful  framework  to  inform  instructional  design  of  a given  course.  The  next  step  is  to 
design  research  studies  that  address  the  impact  of  using  this  framework  on  teaching  and  student  learning. 
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Conceptual  approach 

The  TeleCabri  practices  can  be  articulated  with  the  whole  of  the  experimental  studies  led  these  ten  last  years  on 
remote  teaching  (Dessus  et  al.  97)  in  term  of  situated  learning.  The  teachers  seldom  intervening  in  the 
realization  of  the  computer-based  environments  that  they  use  to  teach  remotely,  the  purpose  of  this  research  is 
to  specify  the  cognitive  influence  (Balacheff  91)  of  the  mediation  of  these  environments  on  the  student/teacher 
interaction.  The  computer-based  didactic  systems,  with  the  constructivist  meaning  of  the  didactic  theory  of  the 
situations  (Brousseau  86),  are  its  object  of  study.  It  is  built  according  to  the  interactionist  approach  in  HCI 
(Pochon  & Grossen  97)  considering  these  systems  as  a semiotic  mediator  by  way  of  the  didactic  concept  of 
computer-based  milieu  (Che vail ard  92).  A computer-based  milieu  is  regarded  as  knowledge  carrier  aimed  by 
the  learning.  We  are  particulary  interested  in  the  milieu  which  include  microworlds  or  simulators  characterized 
by  a significant  epistemological  validity  and  a capacity  to  support  a learning  guided  by  the  discovery  (Masseux 
& Michau  96).  It  can  thus  include  a computer-based  tool  identified  like  software  or/and  hardware  support  of  a 
learning.  Here,  we  center  our  study  on  the  environments  including  several  educational  applications.  This  study 
aims  at  specifying  the  strategies  of  didactic  regulation  engaged  by  the  teacher  modelled  before  in  the  theory  of 
the  didactic  situations.  These  strategies  are  analyzed  using  cognitiv  models  (Anderson  et  al.  90)  of 
action/feedback  between  a user  and  an  application  who  are  declined  for  the  student  and  the  teacher.  The  finality 
of  this  research  is  to  develop  tools  which  help  the  teacher  to  diagnose  in  real  time  the  learning  activity  of  the 
student.  It  lies  within  the  scope  of  study  of  computational  didactics  applied  to  mathematics  (Balacheff  & Kaput 
97). 


Functional  integration  of  a platform  to  teach  remotely 

The  functional  integration  of  the  platform  is  based  on  the  capacity  of  a system  to  bring  answers  to  the  specific 
constraints  of  the  students  and  teachers.  The  functionalities  of  the  platform  bring  a technological  response  to  the 
complexity  of  management  of  distributed  human  resources  and  of  which  the  availability  is  unforeseeable. 

The  student/teacher  interaction  through  PCs  articulates  three  types  of  functionalities: 

- functionalities  allowing  an  autonomous  work  of  the  student  (educational  applications) 

- functionalities  of  synchronous  collaboration  (video,  audio,  shared  applications) 

- asynchronous  functionalities  of  interaction  (email,  sending  of  files) 


Cognitive  influence  of  a multiapplication  environment  on  the  remote  student/teacher 
interaction 

With  TeleCabri,  the  student  and  the  teacher  interact  remotely  by  way  of  three  applications,  each  one  considered 
here  as  didactic  milieu  and  each  one  characterized  by  an  epistemological  validity  specific  to  the  stake  of 
learning: 

- an  oral  milieu,  supported  by  the  videocommunication, 

- a milieu  Web  on  the  geometry,  including  a base  of  exercises,  definitions  and  properties. 
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- a milieu  (CabriGeometre)  microworld  of  direct  interaction  with  geometrical  objects  provides  as  a solving 
board  of  problem. 


To  control  the  student/environment  interaction 

The  regulation  which  the  teacher  carries  out  is  conditioned  by  the  sequential  organization  of  the  various  stages 
of  use  of  the  platform  by  the  student. 

This  sequential  organization  can  be  schematized  as  follows: 

- an  autonomous  stage  of  work  for  the  student  which  precedes  the  stage  by  videocommunication  with  the 
teacher, 

- a stage  of  videocommunication  during  which  the  student  requests  the  assistance  of  the  teacher  to  solve  his 
difficulties  by  sharing  its  applications 

The  regulation  processes  of  the  teachers  on  the  student/environment  interaction 

The  observation  underlines  the  didactic  contribution  of  the  property  multiapplication  of  the  environment.  This 
contribution  is  observed  through  the  diversity  and  the  accuracy  of  the  regulation  processes  carried  out  by  the 
teachers.  Those  try  to  exploit  the  epistemological  complementarity  of  the  three  milieu  within  the  limit  of  their 
own  functional  and  cognitive  control  of  the  environment. 

In  short,  to  control  the  student/environment  interaction,  the  teachers  proceed  according  to  two  types  of 
interaction: 

- co-operative  interactions  on  the  geometrical  objects  with  the  interface  of  CabriGeometre  when  the  teacher 
focuses  himself  on  the  diagnosis  while  the  student  concentrates  on  the  autonomous  experimental  solving  of  the 
problem, 

- collaborative  interactions  to  create,  to  modify  and  to  remove  objects  with  the  interface  of  the  microworld  when 
the  teacher  wants  to  destabilize  a balance  of  action/feedback  of  the  student  with  the  environment. 

The  process  of  destabilizing  used  by  the  teacher  consists  in  questioning  the  student  on  the  inaccuracies  or  the 
cognitive  disjointednesses  which  arise  from  the  interaction  of  the  student  with  the  three  milieu.  The  regulation 
in  the  oral  milieu  in  term  of  formulation,  reformulation  in  the  linguistic  reference  frame  of  the  Euclidean  theory, 
takes  into  account  the  collaborative  interactions. 

References 

Anderson,  J.R.,  Boyle,  C.F.,  Corbett,  A.  & Lewis,  M.  (1990).  Cognitive  modeling  an  intelligent  tutoring. 
Artificial  Intelligence,  42,  7-49. 

Balacheff,  N.  (1991)  . Contribution  de  la  didactique  et  de  I’epistemologie  aux  recherches  en  EIAO.  Actes  des 
XIII°  Journees  Francophones  sur  Vlnformatiqiie.  Grenoble  - IMAG.  Ed.  Bellissant. 

Balacheff,  N.  & Kaput,  J.  (1997).  Computer-based  Learning  Environments  in  Mathematics.  International 
Handbook  in  Mathematics  Education.  A.  Bishop  et  al.  (eds.),  Dordrecht:  Kluwer  Academic  Publisher.  469-501 . 
Brousseau,  G.  (1986).  Fondements  et  methodes  de  la  didactique  des  mathematiques.  Recherche  en  Didactique 
des  Mathematiques,  1 (2),  33-1 15. 

Chevallard,  Y.  (1992).  Integration  et  viabilite  des  objets  informatiques  dans  I’enseignement  des  mathematiques. 
B.Comu  (d.).  L’ordinateur  pour  enseigner  les  mathematiques,  Nouvelles  encyclopedie  Diderot,  Paris  : PUF. 
183-203. 

Dessus,  P.,  Lemaire,  B.  & Bailie,  J.  (1997).  Etudes  experimentales  sur  I'enseignement  a distance.  Sciences 
et  Techniques  Educatives,  4 (2),  137-164. 

Hoyles,  C.  & Noss,  R.  (1992).  A pedagogy  for  mathematical  microworlds.  Educational  studies  in  Mathematics, 
23  (1),  31-57. 

Masseux,  N.  & Michau,  F.  (1996).  Decouverte  guidee  sur  simulateur  pour  I'apprentissage  de 
I'automatique  par  des  eleves-ingenieurs.  Sciences  et  Techniques  Educatives,  3 (1),  77-100. 

Pochon,  L-0  & Grossen  M.  (1997).  Les  interactions  homme-machine  dans  un  contexte  educatif:  un 
espace  interactif  heterogene.  Sciences  et  Techniques  Educatives,  4(1). 


1689 


The  Total  Student  Experience 


Paul  McKey 
CTO  NextEd  Ltd 
paul.mckev@nexted.coni 


Abstract 

With  the  advent  of  online  education  the  Universities  of  the  world  are  for  the  first  time 
exposing  their  wares  in  a public  space,  the  Internet,  in  direct  competition  and  comparison 
to  similar  commercial  offerings. 

While  the  argument  for  the  University  offering  as  being  one  of ‘accredited  academic 
quality’  is  easy  to  win  against  its  unaccredited  training  and  professional  development 
competitors  it  is  not  necessarily  as  easy  to  suggest  the  university  offering  is  always  a 
better  ‘product’. 

Universities  typically  have  little  experience  selling  their  wares  in  a commercial 
environment  where  price,  service  and  immediacy  matter.  It  then  comes  as  no  surprise  that 
when  a mature  adult  learner  is  shopping  around  universities  are  often  being  overlooked 
for  education  alternatives  more  in  tune  with  the  requirements  of  a modem  lifestyle. 

The  Total  Student  Experience  offers  a conceptual  framework  for  the  discussion,  delivery 
and  measure  of  an  online  educational  system  from  the  student’s  perspective.  It  suggests  a 
model  with  four  distinct  layers,  which  individually  and  collectively,  effect  the  learner’s 
overall  satisfaction  with  the  system.  The  four  layers  are  presentation,  function,  education 
and  administration  with  a number  of  key  variables  occurring  within  each  layer? 

This  paper  will  discuss  the  use  of  this  framework  and  the  importance  of  the  underlying 
assumption  that  the  university  ‘product’  in  the  online  environment  will  be  measured  not 
only  by  its  traditional  hallmarks  but  also  by  the  key  elements  of  the  medium  in  which  it  is 
offered.  When  that  medium  is  the  Internet  these  elements  are  convenience  and  relevance, 
service  and  value,  and  the  antithesis  of  the  mass  production  era  of  education, 
personalisation. 

The  complete  version  of  this  paper  is  available  at 
http://www.nexted.com/news/papers/tse  nickev.html 
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Abstract:  This  paper  is  on  the  Telematic  Education  (TE),  which  is  not  the  mere 
use  of  the  television  set  as  a teaching  tool/aid.  . It  should  be  viewed  and  managed  as 
a strategic  resource.  TE  refers  to  a comprehensive  system  of  flexible  learning.  It 
emphasises  the  use  of  technology  that  will  enhance  the  teaching  and  learning 
environment.  Nurses/lecturers  have  been  confronted  with  the  fast  development  of 
technology  for  many  years,  but  mostly  in  clinical  practice.  The  Department  of 
Nursing  Science  of  the  University  of  South  Africa  is  providing  in  the  learning  needs 
of  professional  nurses  in  certain  rural  areas  of  the  country  by  means  of  TE. 


1.  Introduction: 

Broadcast  television  has  been  a solution  to  educational  requirements 
such  as  the  widely  distributed  campuses  of  some  countries  such  as 
Australia  and  Canada,  or  even  the  widely  distributed  student 
populations  of  universities  (Laurillard.  1993:  113).  To  be  of  any 
value  to  a student  television  education  must  be  interactive  in  nature. 
This  implies  that  student  and  lecturer  must  be  able  to  communicate 
during  a lecture  that  is  being  broadcast  by  means  of  television.  This 
brings  to  mind  the  concept  of  Telematic  Education  (TE),  which  is 
not  the  mere  use  of  the  television  set  as  a teaching  tool/aid. 


2.  University  of  Pretoria  (UP)  : 

No  university  is  immune  to  the  impact  of  technological  change. 
Organisations  can  benefit  tremendously  by  exploiting  technological 
change.  It  should  be  viewed  and  managed  as  a strategic  resource. 
The  deployment  of  the  Technology  Plan  of  the  UP  has  already 
borne  fiuit.  The  establishment  of  the  unit  for  Telematic  Teaching 
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and  the  subsequent  virtual  campus  initiative  are  direct  benefits  of 
the  Plan. 


3.  Telematic  Education: 

Before  one  should  attempt  to  understand  what  (TE)  is,  it  is 
important  to  know  what  is  meant  by  ‘flexible  learning’.  According 
to  Dr.  Tom  Brown  of  the  Telematic  Education  Department  of  the 
University  of  Pretoria,  TE  refers  to  a comprehensive  system  of 
flexible  learning.  It  emphasises  the  use  of  technology  that  will 
enhance  the  teaching  and  learning  environment.  TE  is  mostly 
delivered  over  a distance.  The  dependent  learner  will  be  more 
directed  by  the  lecturer  whereas  the  autonomous  student  who  are 
capable  of  self-study  will  be  able  to  employ  the  higher 
technological  learning  facilities.  In  the  educational  setting,  all  the 
dynamics  of  the  traditional  classroom  are  preserved,  regardless  of 
the  distance  separating  the  locations”.  This  is  the  ideal  situation  to 
which  finances  and  available  sources  not  always  lend  it. 


4.  Interaction: 

The  structure  of  a typical  teaching  session  must  have  two  parts. 
During  the  first  part  of  the  lecture  the  lecturer  delivers  the  lecture, 
i.e.  passing  on  knowledge  to  the  students.  The  second  part  of  the 
lecture  can  be  in  the  form  of  a discussion/question  session  with  high 
interaction  between  the  lecturer  and  the  students  (Jameson,  Hanlon, 
Buckton,  & Hobsley,  1995: 114).  The  need  for  students  to  have  back 
up  videos/copies  of  broadcast  stems  from  the  fact  that  some 
students  might  not  be  able  to  attend  broadcast  sessions  due  to 
employer  demands,  technical  failure  or  such  like  reasons  (Jameson 
et  al.  1995:45). 


5.  The  Nursing  Education  Situation: 

Nurses/lecturers  have  been  confronted  with  the  fast  development  of 
technology  for  many  years,  but  mostly  in  clinical  practice. 
Telematic  education  (TE)  is  employed  in  nursing  education  (NE) 
for  post-basic  programs.  Post-basic  programs  imply  that  people 
who  have  already  registered  as  professional  nurses  at  the  South 
African  Nursing  Council  (SANC)  are  continuing  and  furthering 
their  skills  and  knowledge  regarding  certain  specialities  in  the 
profession.  Continued  education  for  post-basic  nursing  requires  a 


distance  education  method  that  will  serve  as  an  academic  solution  to 
professional  nurses  in  rural  areas  of  South  Africa.  The  Department 
of  Nursing  Science  of  the  University  of  South  Africa  is  providing  in 
the  learning  needs  of  professional  nurses  in  certain  rural  areas  of  the 
country.  Jack  Yensen  (1996:213),  who  calls  this  method  of  teaching 
‘telenursing’,  is  of  the  opinion  that  the  two  key  dimensions  are 
distance  and  electronic  mediation. 


6.  The  Lecturer: 

Lecturers  who  teach  by  means  of  telematic  education  must  be 
skilled,  self-confident,  understand  the  uniqueness  of  the  rural 
students  and  should  be  flexible.  They  should  keep  the  concepts  of 
adult  learning  in  mind  and  respect  the  authority  of  experience  that 
many  students  bring  to  the  classroom.  Lecturers  who  have  not 
taught  by  means  of  this  method  need  sufficient  training.  They  do 
not  need  technical  courses,  but  training  on  how  to  adjust  their 
method  of  teaching  as  well  as  teaching  aids  to  that  of  the  telematic 
medium  (Yeaworth.  1996:150).  The  use  technology  influences 
educational  innovation. 


7.  Closing  Remarks: 

The  twenty-first  century  will  see  many  changes,  the  first  of  which 
will  be  the  end  of  extensive  emphasis  on  classroom  teaching. 
Teachers,  students,  and  preceptors  will  come  together  in  dialogues 
to  question,  to  demonstrate,  and  to  participate  actively  in  the 
learning  process.  Nursing  will  hopefully  develop  a ‘world  view’. 
“Remember,  the  information  revolution  will  continue  to  roll  on  - 
with  or  without  you.  So,  get  on  board  and  begin  changing” 
(Simpson.  1997:27). 
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Abstract;  The  greater  prevalence  of  computer  technology  in  the  daily  lives  of  middle  school  students  is 
impacting  their  learning  envrionment.  Students  of  both  genders  and  different  levels  of  socio-economic 
status  are  well  on  their  way  to  becoming  part  of  the  digital  culture.  This  paper  suggests  that  the  gaps  that 
once  existed  with  regard  to  computer  access,  use,  and  perceived  expertise  are  narrowing  significantly. 
These  findings  are  based  upon  survey  data  from  512  middle  school  students  in  three  areas:  (a)  self- 
perception of  computer  skills  and  their  acquisition;  (b)  exposure  to  technology  at  home  and  at  school; 
and  (c)  media  style  and  content  preferences. 


Introduction 

Media  and  digital  technology  are  pervasive  in  our  lives.  The  advent  of  technology  into  the  lives  and  culture 
of  adolescents  is  not  one  of  choice  but  of  certainty.  If  we  understand  the  influences  at  work  among  our  youth,  then 
perhaps  we  can  address  their  educational  needs  in  ways  that  maximize  what  we  already  know  about  how  to  structure 
effective  content.  Are  there  any  signs  of  technology  acculturation  at  work?  Have  computer  presence  and  increased 
proficiency  translated  into  any  appreciable  closing  of  the  previously  documented  digital  divide  with  regard  to  gender 
and  socio-economic  variables  among  middle  school  students? 

The  Study 

Based  on  a series  of  small  focus  groups  with  middle  school  students,  the  authors  compiled  a 68-item 
questionnaire,  both  closed  and  open  answer,  to  cover  students’  preferences  and  practice  using  different  types  of 
media.  Students  answered  the  paper  and  pencil  questionnaire  during  either  a science  or  computer  technology  class 
period.  Typically,  the  questionnaire  took  30  minutes  to  complete. 

Between  October  1998  and  April  1999,  568  middle  school  students  were  surveyed  from  eight  different 
metropolitan-area  public  and  private  middle  schools  covering  four  different  school  districts.  The  students  ranged  in 
age  from  1 1 to  15  years  (M  = 12.59,  ^ = 0.66).  Students  who  did  not  report  essential  demographic  information 
were  dropped  from  the  analysis  for  a final  sample  of  512  students.  Of  the  final  sample,  43.4%  were  male  (n  = 222) 
and  56.6%  were  female  (n  = 290).  Schools  were  recruited  to  obtain  a diverse  student  population  representative  of 
urban  and  suburban  schools,  as  well  as  all  ranges  of  socio-economic  status  (SES).  The  final  sample  contained  31% 
high  disadvantage  students  (n=158),  33%  middle  disadvantage  students  (n=170),  and  36%  low  disadvantage 
students  (n=184).  Within  each  SES  category,  male  and  females  were  represented  nearly  equally. 

Findings 

The  digital  divide  separating  the  confidence  levels  of  male  and  female  adolescents  with  regard  to  computer 
use  is  narrowing.  While  some  significant  differences  emerged,  the  patterns  of  computer  use  and  the  purposes  of  use 
were  remarkably  similar  for  males  and  females  in  our  sample.  As  recently  as  1994,  one  study  by  Sakomoto  indicated 
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that  among  fourth  through  sixth  grade  students  considered  "heavy  users"  of  computers,  the  ratio  of  boys  to  girls  was 
4 to  1 . Our  results  indicate  that  this  gap  is  closing  as  we  found  no  gender  difference  among  those  who  could  be 
identified  as  heavy  users  of  computers.  Additionally,  when  looking  at  the  use  of  computers  for  identified  school 
subjects,  we  found  similar  levels  of  self-reported  computer  use  between  males  and  females. 

In  contrast  to  previous  research,  both  males  and  females  indicate  that  games  dominate  the  home  use.  Even 
in  the  school  environment,  games  were  one  of  the  prime  uses.  Games  remain  an  important  part  of  adolescents’ 
computer  repertoire.  These  findings,  in  particular,  argue  for  ways  in  which  to  infuse  the  gaming  format  with 
sufficient  intellectual  content  to  make  the  “game”  a valuable  learning  tool.  Some  educators  have  already  seized 
upon  this  notion  and  are  producing  more  substantive  Internet  games  that  incorporate  accepted  learning  theory.  The 
theoretical  framework  of  curriculum  built  around  problem-based  or  case-based  learning  could  lend  itself  to  a gaming 
environment.  Crucial  issues  of  how  intrinsic  the  problem  is  to  the  game  scenario  and  how  well  do  learning 
objectives  translate  into  an  appealing  hypermedia  format  are  all  ones  that  would  need  to  be  explored. 


Conclusions 

While  the  content  of  educational  web  sites  targeted  for  middle  school  students  is  still  an  issue  for  debate, 
the  data  from  this  survey  point  clearly  toward  web  sites  as  an  effective  delivery  mechanism.  Access,  use,  and 
perceived  self-confidence  in  navigating  the  Internet  are  less  constrained  by  gender-related  concerns  than  they  were 
less  than  a decade  ago.  In  addition  to  the  rapid  rise  in  adolescent  female  participation  in  technology,  the  data  also 
echo  the  research  of  others  in  highlighting  the  prominence  of  all  types  of  media  in  the  life  of  adolescents. 

The  recent  Kaiser  survey  (Kaiser  Family  Foundation,  1999)  found  that  the  average  amount  of  time  each 
day  spent  using  all  types  of  media  for  children  from  8 to  1 8 years  olds  was  6.43  hours.  The  influence  of  all  media  in 
youth’s  life  was  reinforced  by  our  survey  participants’  choices  of  “your  favorite  famous  person.”  Of  the  512 
responses,  74%  were  performers  or  musicians.  The  remaining  types  of  persons  named  could  be  categorized  as  1 8% 
athletes,  3%  historical  figures,  2%  religious  figures,  and  4%  other. 

Media  of  all  types  are  playing  a larger  role  in  children’s  lives;  however,  it  is  mostly  a solitary  experience. 
Our  data  indicate  that  children  are  largely  using  the  Internet  and  computers  at  home  free  of  supervision.  It  also  may 
not  be  the  case  that  the  computer  is  replacing  other  forms  of  media  in  adolescents’  lives,  but  rather  it  is  being 
“added”  to  the  existing  repertoire  among  those  who  are  already  “heavy  users”  of  media. 

One  of  the  challenges  before  us  is  the  creation  of  sufficient  quantity  and  quality  of  digital  materials  to 
attract  and  retain  adolescents’  interest  in  an  unsupervised  environment.  The  materials  must  be  free  of  gender-bias 
and  bridge  the  gulf  that  currently  exists  between  how  kids  learn  and  communicate  on  their  own  and  how  they  are 
taught  in  school.  Can  the  production  of  high-quality,  educationally-grounded,  and  content  significant  materials 
attract  adolescents  to  the  world  of  learning  via  the  Web? 
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Abstract 


This  evaluation  project  examines  the  required  elements  for  a standardized  instrument  for  the  evaluation  of 
educational  technology.  The  project  will  survey  current  users  of  educational  technology  and  seek  common  threads 
and  patterns  in  a successful  evaluation.  This  study  will  attempt  to  answer  this  question:  What  should  we  be 
collecting,  measuring  and  examining  when  evaluating  technology  in  an  educational  setting?  This  project  is  the  first 
step  in  a multi-stage,  two-institution  evaluation  study  that  will  develop  a standardized  evaluation  instrument  for  use 
in  the  evaluation  of  education  technology.  The  project  will  enlist  the  assistance  of  two  post  secondary  faculties, 
clients,  developers  and  students.  Each  group  will  be  surveyed  using  an  online  form.  A representative  sample  from 
each  group  will  be  interviewed  in  a focus  group  setting.  Expected  outcomes  of  this  first  stage  include  the  collection 
of  responses  from  users,  developers  and  clients  of  educational  technology.  The  survey  responses  (n=81)  have 
provided  a unique  opportunity  to  share  and  compare  educational  strategies  and  concepts  between  post  secondary 
institution  faculty,  users  and  developers.  Common  patterns  were  evident  within  and  between  groups  relative  to 
concepts  of  educational  technology  evaluation  and  important  aspects  of  instructional  design. 


This  evaluation  project  will  examine  the  required  elements  for  a standardized  evaluation  instrument  for  educational 
technology  tools  and  applications.  The  project  will  survey  current  users  of  educational  technology  and  seek  common 
threads  and  patterns  amongst  user  groups  for  the  design  of  a successful  evaluation  instrument.  The  goal  of  this 
project  is  to  determine  what  users  and  developers  deem  important,  valuable  and  relevant  with  respect  to  instructional 
design  and  the  evaluation  of  technology  in  the  post  secondary  arena.  The  survey  data  collected  will  be  used  to 
create  a standardized  evaluation  tool.  This  evaluation  tool  will  assist  developers,  instructors,  teachers,  and  learners  in 
the  design,  construction  and  use  of  an  instructional  design  model  and  template  suitable  for  use  in  the  post  secondary 
field  of  education. 


Project  Rational  and  Purpose 

First,  the  research  in  this  area  has  focused  on  the  application  of  technology.  Evaluation  studies  on  educational 
technologies  have  traditionally  focused  on  the  particular  technology  tool  and  ifs  impact  on  the  learning 
environment.  While  useful  in  producing  data  for  that  tool  or  application,  generalizability  to  other  technologies  or 
application  is  not  possible  due  to  the  design  of  the  instrument.  Typically  these  instruments  survey  specific  program 
issues  and  fail  to  consider  the  larger  pedagogical  issues  of  technology  (this  link  provides  examples  of  about  50 
instruments  of  evaluation  used  in  educational  technology)  *.  Such  programs  as  the  Annerberg's  Flashlight  Project^ 
and  McM aster  University’s  EVNet^  have  been  leaders  in  educational  technology  project,  but  have  failed  to  develop  a 
standardized  instrument,  useful  in  evaluating  all  types  of  projects,  across  multiple  users  in  various  circumstances. 

Secondly,  this  project  is  the  first  step  in  a multi  stage;  two-institution  evaluation  study  dedicated  to  the  development, 
testing  and  dissemination  of  a standardized  instrument  for  educational  technology  evaluation.  The  first  stage  is  a 
necessary  step  in  determining  what  should  be  evaluated  when  examining  technology  in  the  education  field  of  study. 


Introduction 


Several  studies  have  identified  survey  instruments  and  other  collection  tools  for  use  in  educational  technology"*  ^ 
While  useful  in  assessing  individual  programs  for  a specific  outcome,  these  types  of  instrument  are  difficult  to 
generalize  to  educational  technology,  regardless  of  platforms,  users  or  programs.  Determining  what  should  be 
evaluated  in  a format  that  can  transferred  across  technologies  and  application  is  a critical  next  step  in  valuing 
educational  technology's'  contribution  to  learning. 

Conclusions 

The  first  step  of  this  multi-stage  evaluation  project  produced  an  extensive  database  of  information  on  what  users, 
developers,  clients,  and  educators  think  about  educational  technology  and  its  application  to  learning.  The  collection 
of  data  ranges  from  attitudes  towards  technology  to  perceptions  of  technology  and  its  affect  on  learning.  It  also 
provided  a resource  in  the  development  of  a standardized  tool  for  evaluation  educational  technology. 

From  this  study  sample,  it  is  clear  that  respondents  agree  on  a number  of  issues  in  terms  of  importance  and  relevance 
to  instructional  technology.  It  is  also  important  to  note  that  there  are  some  differences  between  respondent  groups 
relative  to  the  evaluation  process  for  educational  technology.  This  information  supports  previous  research  in 
educational  technology  that  has  identified  key  concepts  and  factors  of  educational  technology. 

This  first  stage  of  the  project  provided  an  opportunity  to  validate  the  survey  instrument,  by  seeking  some  responses 
to  confirm  that  the  evaluation  of  educational  technology  is  an  important  aspect  of  the  instructional  design  process. 
These  findings  can  be  used  to  development  a survey  instrument  designed  to  evaluate  the  important  constructs  of 
instructional  design  with  the  use  of  technology. 

This  first  stage  of  this  project  provided  a unique  opportunity  to  share  and  compare  educational  strategies  and 
concepts  between  two  post  secondary  institutions.  It  is  hoped  that  the  two  institutions  will  share  a common  ground 
with  respect  to  the  development,  application  and  delivery  of  educational  technology. 
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Introduction 

This  paper  investigates  how  continuous  access  to  laptop  technology  and  telecommunication  tools  can  promote  a 
student-centered,  inquiry-based,  problem-solving  teaching  practice.  The  purpose  of  the  study  is  to  deepen  our 
understanding  on  how  technology  and  Internet  tools  can  best  be  utilized  in  K-12  classrooms.  We  present  an  Internet- 
based  project  that  utilizes  laptop  computers  in  a sixth  grade  classroom. 


Research  Questions 

We  explore  how  laptop  computers  and  Internet  tools  are  used  by  a sixth  grade  teacher  who  has  extensive  experience 
with  technology  and  multiple  opportunities  for  professional  development.  The  overall  questions  that  guide  this 
research  are  as  follows:  a)  What  are  the  teacher’s  beliefs  about  teaching  and  learning?  b)  What  is  the  nature  of 
students’  assignments?  c)  What  is  the  role  of  the  teacher  and  the  student  in  the  classroom?  d)  How  are  laptops  being 
utilized  in  the  curriculum?  e)  How  is  the  Internet  being  used  in  the  classroom? 


Methodology 

This  study  began  in  Septemberl998  and  takes  place  in  urban  elementaiy  and  intermediate  schools  in  New  York 
City.  The  primary  data  collection  techniques  that  are  utilized  include  classroom  observations,  teacher  interviews, 
and  document  analysis.  These  activities  have  been  carried  out  during  school  visits  since  the  academic  year  of  1998- 
99.  Participants  for  this  initial  research  effort  include  a sixth  grade  teacher  and  her  students.  All  students  in  the  class 
are  minority  students;  the  teacher  is  a white  female. 

The  classroom  is  part  of  the  Laptop  Project,  which  was  initiated  by  the  school  district  in  1 997.  All  students  in  the 
project  have  been  issued  a laptop  computer.  The  classroom  is  also  part  of  the  Eiffel  Project,  a major  effort  of  the 
Institute  for  Learning  Technologies  (ILT)  at  Teachers  College,  Columbia  University  to  assist  schools  and  teachers 
integrate  computers  and  telecommunication  tools  effectively  in  their  classrooms.  ILT  provides  teachers  with 
technological  and  pedagogical  support,  as  well  as  with  opportunities  for  communication  and  collaboration  with  other 
teachers  and  researchers  involved  in  the  Eiffel  Project. 


Project  Design  and  Preliminary  Results 

Mary,  the  sixth  grade  teacher  participating  in  our  experiment,  started  teaching  for  first  time  using  laptop  technology 
during  the  academic  year  of  1997-98.  That  first  experience  provided  her  with  valuable  training  on  the  use  of 
computers  in  teaching.  Mary  was  ready  to  try  new  ideas,  experiment  with  different  projects,  and  go  beyond 
traditional  instruction.  Therefore,  she  volunteered  to  participate  in  the  Reach  the  World  (RTW)  project. 

RTW  (www.reachtheworld.org)  is  a virtual  expeditionary  venture  that  uses  the  world  circumnavigation  of  Makulu 
II,  a sailboat,  as  an  interactive  tool  for  classroom  teaching  in  the  United  States.  RTW  utilizes  the  Internet  and  other 
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telecommunication  tools  to  bridge  the  student  classroom  with  different  cultures  of  the  world.  The  crew  of  the 
Makulu  II  communicates  with  students  regularly  via  satellite  email.  Since  September  1998,  Mary  accommodated  her 
social  studies  curriculum  to  the  RTW  project:  Soon  after  she  started  participating  in  the  project,  she  became  the  most 
active  teacher.  Some  of  the  lesson  plans  she  developed  are  posted  on  the  RTW  web-site  as  examples  for  other 
teachers. 

Nature  of  Student  Assignments 

Planning  an  Itinerary.  In  this  project  students  were  asked  to  form  groups  and  conduct  research  in  order  to  identify 
possible  sites  that  the  Makulu  II  should  visit  on  their  trip  to  Singapore,  Each  group  researched  different  attractions 
and  recommended  sites  of  interest.  Students  used  travel  brochures  to  locate  tourist  attractions,  maps  to  figure  out 
distances  and  ways  of  transportation  in  Singapore,  and  historical  information  from  the  Internet  to  identify  museums 
of  interest.  When  the  final  itinerary  was  put  together  it  was  sent  out  via  email  to  the  Makulu  II  crew.  The  Makulu  II 
crew  completed  the  tour  and  sent  back  relevant  information  and  pictures  to  the  students  via  email  and  air-mail. 

Comparing  religions  of  the  world.  Students  studied  different  religions  using  information  and  videotapes  that  were 
sent  by  the  Makulu  II  crew.  In  sequence,  students  formed  groups  and  selected  one  religion  to  study  in-depth.  Final 
reports  were  presented  to  other  sixth  grade  students  in  the  school  using  PowerPoint  slides. 

The  Role  of  the  Teacher  and  the  Students 

Teaching  was  organized  around  two  types  of  pedagogies:  Collaborative  knowledge  construction  and  direct  verbal 
instruction.  Direct  instruction  did  not  represent  the  dominant  mean  of  communicating  knowledge  to  students. 
Students  were  rarely  asked  to  memorize  facts.  They  were  instead  asked  to  locate  facts  using  CD-ROMs,  books,  the 
Internet,  and  other  resources.  The  information  was  then  analyzed  in  order  to  complete  projects  and  solve  problems. 

The  teacher  was  mostly  a facilitator  of  learning  and  a "guide  on  the  site".  Students  were  given  opportunities  for 
self-directed  inquiry.  In  many  cases  they  were  asked  to  generate  their  own  questions  and  search  for  the  answers. 
Laptops  were  very  helpful  because  they  put  a lot  of  the  resources  into  the  hands  of  the  students. 

Utilization  of  Laptop  Computers  and  Telecommunication  Tools  in  the  Classroom 

Laptops  were  a vehicle  to  support  the  curriculum  and  expand  the  classroom ’s  walls.  Students  typically  used 
application  programs  such  as  a word  processor  for  typing  and  editing  documents,  databases  for  organizing 
information,  and  graphics  packages  for  developing  multimedia  presentations.  Email  was  also  used  extensively  in  the 
classroom  for  communication  purposes.  Moreover,  all  students  used  First  Class,  an  Intranet  service  that  allowed 
them  to  communicate  with  their  classmates  and  teacher  after  school  hours. 

Laptops  were  effective  in  promoting  collaborative  work  and  problem  solving.  Very  often  students  were  required  to 
work  in  groups  and  create  projects.  In  collaborative  projects,  students  were  required  to  perform  research,  collect 
information,  analyze  it,  synthesize  it,  and  produce  a final  product. 


Conclusions 

This  paper  presented  an  effort  to  integrate  technology  in  the  classroom.  Laptop  computers  and  telecommunication 
tools  were  used  to  provide  students  with  authentic,  meaningful,  and  motivating  experiences.  Learning  became  a 
challenge  and  students  exhibited  increased  levels  of  control  and  creativity.  Successful  use  of  technology  in  the 
classroom  depends,  however,  to  a great  extend,  on  the  teacher.  Therefore,  in  any  implementation  effort,  adequate 
time  must  be  devoted  in  helping  teachers  become  comfortable  with  the  equipment  and  absorb  the  new  teaching 
principles  associated  with  the  use  of  the  new  technologies. 

We  currently  work  with  a number  of  teachers  in  the  New  York  City  area  assisting  them  with  their  technology 
implementation  efforts.  Many  teachers  use  laptop  computers  and  Internet  tools  for  instructional  purposes.  Therefore, 
we  continue  our  research  by  observing  what  is  happening  in  these  classrooms  and  analyzing  changes  in  the 
instructional  beliefs  and  practices  of  the  teachers. 
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Abstract:  Scenario  building  is  a process  that  can  be  effectively  used  to  design  a distance- learning 
program.  It  begins  with  the  identification  of  the  key  stakeholders,  who  are  typically  students  or 
participants,  course  developers,  teachers  or  facilitators,  support  staff,  and  administration.  Then,  for 
each  perspective,  the  relevant  characteristics  are  identified,  followed  by  a determination  of  what  is 
required  to  meet  the  needs  of  a person  in  that  category.  The  process  also  has  the  benefit  of  being  a 
group  activity  that  can  facilitate  team-building  and  commitment.  We  have  used  this  process  with 
multiple  organizations  and  report  on  these  case  studies. 


Introduction 

Distance  learning  programs  are  designed  in  many  ways:  some  evolve  from  individual  efforts,  some  are  designed 
using  available  technological  capabilities,  and  others,  more  recently,  are  purchased  from  or  co-branded  with  e- 
leaming  providers.  Scenario  building  provides  a systematic  process  for  the  group  of  people  involved  in  planning  and 
implementing  a program  to  work  as  a facilitated  team  to  design  a program.  The  process  can  be  used  for  any  type  of 
organization,  including  a university  or  business.  It  involves  a number  of  steps:  determining  who  will  be  involved  in 
developing,  delivering,  supporting,  administrating,  or  taking  courses,  and  prioritizing  these  to  order  the  next  phase. 
Starting  with  the  most  important  perspective,  usually  that  of  the  student  or  participant,  the  relevant  characteristics  of 
members  of  that  category  are  determined  by  the  group.  This  may  include  computer  and  Internet  literacy,  work 
environment,  and  time  commitments.  The  group  then  determines  what  happens  when  a representative  of  that 
category  carries  out  their  role,  i.e.,  taking,  teaching,  or  supporting  a class.  This  results  in  a list,  which  is  then 
prioritized,  of  the  needs  and  requirements  from  that  perspective  for  the  distance  learning  program.  Subsequent 
perspectives  are  usually  easier  to  consider  after  the  first,  and  the  entire  process  leads  to  requirements  analysis  and 
planning  for  the  program.  This  process  has  been  used  for  a number  of  organizations  that  are  planning  distance 
learning  projects,  including  the  University  of  Puerto  Rico. 

The  University  of  Puerto  Rico 

The  Puerto  Rican  government  funded  an  ambitious  distance-learning  program  at  The  University  of  Puerto  Rico.  To 
help  get  the  program  underway,  the  university  called  on  a team  of  EDS  consultants  to  structure  and  support  the 
program's  essential  planning  and  implementation  processes.  The  team  began  with  strategic  planning  to  determine  the 
tasks  and  responsibilities  needed  to  ensure  the  program's  on-schedule  readiness.  During  this  process,  it  became  clear 
that  the  distance-learning  concept  conjured  widely  different  images  for  members  of  the  steering  committee 
depending  on  their  knowledge  of  its  theory  and  practice.  Many  assumptions  were  based  on  the  association  of 
distance  learning  with  the  room-based  videoconferencing  that  dominated  distance  education  prior  to  the  introduction 
of  Web-based  technologies  in  the  mid-1990s.  Some  of  these  images  did  not  account  for  the  level  of  interactivity  and 
engagement  that  students  and  teachers  experience  using  state-of-the-art  distance-learning  technologies  or  failed  to 
accommodate  the  important  differences  between  learning  and  teaching  in  traditional  and  technology-mediated 
settings. 

The  team  proceeded  with  scenario  building,  an  activity  designed  to  help  the  program's  steering  committee 
understand,  envision,  and  specify  technology's  role  in  the  new  distance-learning  offering.  Scenario  planning  began 
by  determining  the  roles  that  people  will  play  in  the  university's  distance-learning  program;  those  roles  are  student, 
professor,  teaching  assistant,  and  a number  of  support  and  administrative  positions.  The  scenario-planning  work  led 
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to  each  role  being  assigned  characteristics  that  are  relevant  to  the  how  a person  in  that  role  operates  in  a distance 
learning  program  or  that  give  the  role  dimension  and  interest.  Characteristics  of  a student,  for  instance,  included  age, 
family/home  situation,  educational  background,  technology  literacy,  Spanish  and  English  literacy,  and  typing  skills. 

The  goal  was  to  agree  upon  the  role  of  technology,  the  specific  types  of  technology,  and  the  support  requirements 
for  technology  in  the  distance-learning  program.  To  achieve  this  goal,  we  developed  a prototypical  member  of  each 
role  in  order  to  envision,  as  a group,  what  a day  in  that  person's  life  would  be  like.  We  carried  this  to  the  level  of 
detail  of  which  relevant  activities  they  were  engaged  in,  where  they  carried  out  the  activities,  and  what  technological 
and  human  support  was  required.  There  were  implications  from  some  of  the  characteristics  of  a role;  to  discuss  the 
activities  of  a person  in  a day,  we  needed,  for  instance,  to  know  if  there  was  access  to  a computer  and,  if  so,  under 
what  conditions  or  with  what  possible  distractions  it  would  be  used.  If  there  was  no  access,  then  a staffed  computer 
room  must  be  available  during  reasonable  hours.  Additionally,  we  considered  the  implications  of  atypical  but 
plausible  cases  for  each  characteristic,  such  as  the  ramifications  if  a student  was  older  and  returning  after  a period 
out  of  school. 

Conclusions 

The  scenario-building  process  uncovered  many  issues,  concerns,  assumptions,  and  previously  unarticulated 
misconceptions,  which  led  to  constructive  discussion  that  culminated  in  a consensus  about  technology's  role  in  the 
project  and  about  the  people  who  would  manage  and  support  it.  Equipped  with  the  scenarios'  insights,  we  completed 
a thorough  analysis  of  the  program's  specific  technology  needs,  identifying  and  prioritizing  the  components  of  the 
supporting  technology.  In  subsequent  discussions,  we  continued  to  refer  to  the  prototypical  characters,  and  found 
that  they  continued  to  help  clarify  issues  since  the  group  could  look  at  that  person's  specific  (imagined)  needs. 
Ultimately,  the  process  of  identifying  each  role’s  relevant  characteristics  and  the  hypothetical  activities  and 
interactions  of  a person  in  that  role  pointed  directly  to  the  criteria  against  which  the  entire  program  will  be 
evaluated,  ensuring  that  the  overall  effort's  success  will  be  measurable  and  more  easily  repeatable. 
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Abstract:  Collaborative  technology  projects  require 
new  types  of  relationships  and  distinctive 
management  strategies.  This  paper  outlines  three 
critical  elements  in  creating  and  continuing  such 
partnerships:  common  vision,  a reward  system,  and 
consensus. 


Technology  ventures  often  demand  collaborative  efforts  which  bring 
together  personnel,  financial  resources,  needs  and  solutions  from  diverse 
entities.  Older  models  of  cooperation  worked  within  top-down  power 
structures;  new  structures  of  collaboration  are  complex  and  require  long- 
term thinking  and  conflict  management.  Furthermore,  sustaining 
collaboration  is  critical  because  the  payoffs  of  such  efforts  are  often  long- 
term. We  identify  here  three  elements  critical  to  building  and  sustaining 
collaboration:  common  vision,  a reward  system,  and  consensus 
operations. 

Common  Vision 

Diverse  groups  who  collaborate  in  technology  ventures  always 
face  semantic  difficulties,  terms  are  defined  differently  and  goals  are 
stated  in  language  only  the  originators  understand.  As  a result,  common 
definitions  of  terms  and  in-depth  discussion  about  problems,  solutions, 
and  common  issues  are  essential  (Winer  1994).  In  this  initial 
establishment  of  common  understanding,  group  members  must  adopt  a 
posture  of  tentative  knowing,  a non-judgmental  stance  which  reserves 
value  judgments  and  allows  for  continuing  conversation.  Although 
individuals  may  be  experts  in  their  domains,  they  need  to  drop  their 
"know  it  all"  perspective  when  they  work  collaboratively.  Rarely  does 
any  entity  get  all  it  wants,  but  group  members  must  be  honest  about  what 


they  are  willing  to  put  into  a collaborative  effort  as  well  as  what  they 
wish  to  get  out  of  the  collaboration. 

A Reward  System 

Different  entities  have  different  currencies  and  rewards. 
Rewards  must  come  to  group  members  in  the  currency  valuable  to  them 
in  their  system,  be  it  promotion,  tenure,  credit  hours,  test  scores,  or 
bonuses.  Furthermore,  different  stakeholders  define  accountability 
differently,  as  staying  within  budget  or  positive  publicity  (Report  1997). 
As  specific  rewards  are  discussed,  entities  must  spell  out  to  whom  and 
how  they  are  accountable  and  how  that  will  affect  their  participation. 

Consensus 

In  diverse  collaborative  groups,  absolute  agreement  is  rare. 
Thus  it  is  essential  to  establish  that  the  group  will  operate  by  consensus 
and  to  establish  a process  for  achieving  consensus  and  resolving 
conflicts.  Even  when  members  do  not  agree  on  an  outcome,  if  they  have 
subscribed  to  a specified  process  which  has  been  followed  consistently, 
they  will  be  able  to  accept  the  result  (Winer  1994).  Once  established, 
conflict  resolution  needs  to  be  called  into  play  quickly  when  problems 
arise.  Difficulties  which  drag  on  are  much  harder  to  deal  with. 

Clearly  collaborative  ventures  in  technology  are  neither  simple  nor  easy. 
However,  the  power  of  successful  collaboration  is  synergistic. 
Collaborative  groups  which  are  able  to  sustain  their  efforts  can  achieve  a 
whole  which  is  indeed  greater  than  the  sum  of  the  parts. 
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Abstract:  This  paper  presents  a system  called  Cyberscience'  dedicated  to  distance  education. 

The  learning  interface  of  Cyberscience  includes  a component  consisting  in  activating  an 
interactive  multimedia  simulation  in  which  the  student  can  carry  out  direct  manipulation 
tasks  making  the  simulation  a virtual  laboratory  (vitual  lab  in  short).  A virtual  lab  for  genetic 
study  is  currently  realized.  As  an  intelligent  distance  learning  system.  Cyberscience  includes 
a number  of  intelligent  learning  agents  that  operates  in  the  space  of  the  virtual  lab  in  order  to 
help  students  to  achieve  their  goals.  This  paper  presents  an  overview  of  those  agents. 

Introduction 

There  is  already  a lot  that  has  been  said  about  software  agents,  especially  intelligent  agents’ 
technology.  While  this  is  a really  a current  trend  in  software  engineering.  The  concept  of  intelligent  agents  is 
at  the  same  time  simple  in  principle  and  complex  in  classification  (Jennings  and  Wooldridge,  1998;  Nwana, 
1999)  While  intelligent  agents  systems  are  widely  being  developed,  most  of  the  frameworks  available  simplify 
the  architectural  and  technical  aspects  and  provide  relatively  no  support  to  implement  agent  behavior.  The 
goal  of  this  research  is  to  provide  people  with  a strong  basic  architecture  and  shell  to  handle  the  complex  task 
of  implementing  intelligent  behavior  in  software  agents.  The  tools  provided  must  have  a fairly  generic  aspect 
so  they  can  be  applied  to  a wide  variety  of  problems  and  systems  while  still  being  able  to  adapt  sufficiently  to  a 
problem  to  provide  relevant  intelligent  features.  It  should  be  possible  to  achieve  such  capability  by  building  a 
set  of  general-purpose  agents  that  would  be  easily  customizable  and  reusable. 

A set  of  multiple  agents 

The  first  part  of  building  such  a set  of  agents  is  to  determine  a set  of  feature  that  are  desirable  and 
that  are  sufficiently  high  level  to  be  reusable.  The  current  proposal  includes  a planning  agent,  a decision 
making  agent,  a data  mining  agent,  a rule-based  reasoning  agent,  a data  extracting  agent  and  a case  based 
reasoning  agent.  These  agents  cooperate  in  the  context  of  CyberScience  in  order  to:  create  an  action  plan  for 
the  current  instructional  objective  associated  with  the  selected  course  or  theme,  according  to  the  student  model, 
select  the  relevant  resource  the  student  will  use  to  achieve  the  goal,  update  the  student  state  of  knowledge,  help 
the  student  during  the  learning  process  and  so  on.  In  the  following,  we  describe  each  of  the  agents. 

Data  Mining  Agent.  The  Data  miner  has  a high  probability  of  being  highly  useful  to  many  different 
applications.  Basically,  this  agent’s  purpose  is  to  retrieve  useful  information  from  databases  and  electronic 
documents  both  on  a local  machine,  a network  or  even  the  Internet.  Of  course  it  can  mean  only  search  for 
certain  keywords  but  it  is  also  possible  to  implement  more  complex  and  advanced  intelligent  search 
algorithms.  It  remains  to  be  decided  how  it  is  most  convenient  to  inteiface  with  such  an  agent.  It  is  also  to  be 
considered  if  such  an  agent  should  be  mobile  or  not  or  even  if  it  should  decide  whether  it  should  move  itself  to 
a different  host  system  to  be  able  to  accomplish  it’s  job  more  efficiently. 

Rule-Based  Reasoning  Agent.  The  rule-based  reasoning  agent  acts  mostly  as  an  expert  system.  Expert 
systems  have  proven  over  time  to  be  efficient  to  a large  number  of  tasks.  The  goal  of  this  agent  is  to  derive 
conclusions  from  a knowledge  base  consisting  of  facts  and  knowledge.  It  needs  an  engine  and  a set  of 
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deductive  rules.  It  needs  an  interface  to  specify  the  content  of  the  knowledge  base.  An  interface  also  needs  to 
be  defined  to  query  the  truth  of  undefined  facts.  The  engine  would  be  most  likely  based  on  the  Rete  algorithm 
that  is  already  well  known  and  extensively  used  and  tested. 


Case-Based  Reasoning  Agent.  The  case-based  reasoning  agent  on  the  other  hand  compares  the  different  cases 
and  makes  generalizations  out  of  them.  When  the  case-based  reasoning  agent  is  invoked  on  a particular 
situation  it  tries  to  find  already  known  cases  that  are  similar  to  the  current  one. 

Planning  Agent.  The  planning  agent  is  software  capable  of  making  plans.  The  goal  is  to  come  up  with  a 
combination  of  actions  to  be  executed  in  the  future  in  order  to  achieve  a desired  behavior.  The  interface  to  the 
planner  agent  is  a bit  more  complicated  than  previously  discussed  agents,  but  this  is  expected  considering  the 
complexity  of  planning  problems.  First  it  needs  a way  to  specify  an  initial  state  and  a goal  specifying  the 
desired  task  to  be  achieved.  It  also  needs  a set  of  specifications  of  the  actions  feasible  by  the  system.  Finally  it 
needs  a planning  strategy,  a starting  plan  and  a set  of  bound  parameters  both  in  time  and  space.  The  agent’s 
engine  would  be  most  likely  to  base  on  SIMPLAN  because  of  its  capacity  to  operate  in  highly  variable 
environments  (Nkambou  and  Kabanza,  1999).  This  agent  will  generate  a plan  of  actions  repairing  the  plan 
given  in  input  that  best  satisfies  the  goal  from  the  initial  state,  given  the  restrictions  implied  by  the  bound 
parameters. 

Decision-Making  Agent.  Finally  the  decision-making  agent  and  the  knowledge  extraction  agents  are  making 
extensive  use  of  previously  discussed  agents  in  this  paper.  They  would  probably  encapsulate  specific  instances 
of  these.  The  decision-making  agent,  as  its  name  implies,  takes  decisions  about  the  different  tasks  to  be  done 
and  how  they  must  be  accomplished  effectively.  It  would  work  by  using  the  rule-based  agent  to  generate  a 
goal  as  of  what  to  do.  A planning  agent  starts  to  search  how  to  best  achieve  the  goal  by  calculating  a new  plan 
while  case  based  reasoning  agent  tries  to  find  a relevant  plan  that  was  previously  used  in  a similar  situation. 

Knowledge  Extraction  Agent.  The  knowledge  extraction  agent  finds  relevant  information  from  different 
digital  data  sources.  Like  the  decision-making  agent,  the  knowledge  extractor  uses  a combination  of  three 
internal  agents.  It  obviously  uses  a Data  Mining  agent  to  retrieve  pieces  of  documents  or  database  information 
related  to  a subject  requested  by  an  internal  rule-based  reasoning  agent.  But  a case-based  reasoning  agent  also 
tries  to  find  similar  cases  as  where  relevant  was  found  to  help  the  data  mining  agent.  Once  it  is  retrieve  the 
information  can  then  be  used  by  the  system. 

Conclusions 

The  development  of  the  set  of  intelligent  agents  involved  in  CyberScience  will  lead  to  a shell  to  easily 
implement  intelligent  learning  agents  systems.  Most  of  the  required  features  for  each  agent  have  already  well 
known  solutions.  There  are  also  numerous  frameworks  that  already  offer  facilities  to  create  agents  and  to  give 
them  mobility  abilities.  It  is  expected  that  a higher  level  of  intelligence  in  behavior  of  the  system  would  be 
achieved  by  close  collaboration  of  these  agents.  It  has  to  be  defined  how  it  should  be  organized  to  provide  a 
generic  functionality  that  would  be  reusable.  Also,  it  must  be  established  how  would  the  specification  for  a 
custom  system  based  on  these  agents  would  take  place.  With  this  shell  the  use  of  intelligent  learning  agents 
system  is  likely  to  widespread.  But  the  first  application  it  is  used  on,  as  a test  case,  is  the  Cyberscience  virtual 
laboratory.  While  it  could  merely  be  used  as  a replacement  for  current  expert  system  to  add  them  a higher  level 
of  intelligence,  adaptability  and  interaction  level,  it  is  also  expected  they  could  be  applied  to  new  breeds  of 
problems  directed  at  a less  specialized  category  of  users.  It  can  be  easily  assumed  how  this  system  could  be 
used  to  create  assistants  or  coaches  in  a context  of  an  Web-based  intelligent  tutoring  system. 
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Abstract 

The  Open  Learning  Agency  in  British  Columbia,  Canada,  is  involved  in  a multi-year 
project  aimed  at  producing  a number  of  courses  that  are  derived  from  structured 
SGML  documents.  Presently,  fourteen  courses  are  being  delivered  on-line  in  33 
School  Districts  throughout  British  Columbia.  This  paper  describes  the  use  of 
structured  documents  for  course  development  vvdth  a focus  on  the  benefits  and 
challenges  of  balancing  technological  and  instructional  design  issues. 


Background 

The  Open  School,  a division  of  the  Open  Learning  Agency,  is  involved  in  a multi-year  project  aimed  at 
producing  a number  of  courses  that  are  derived  from  structured  SGML  documents.  The  goal  is  to  offer 
innovative  and  current  educational  resources  for  students  and  teachers  that  are  directly  linked  to  the 
government’s  curriculum  and  provide  flexibility  through  delivery  of  alternative  formats  within  a distributed 
learning  environment  (e.g.,  for  use  in  a classroom  environment  and  also  within  an  independent  self-paced 
learning  environment).  Fourteen  Grade  1 1 and  12  courses  were  developed  within  a structured  docum.ent 
environment  using  SGML  (Standard  Generalized  Mark-Up  Language)  technology  to  meet  Open  School’s 
goal.  These  courses  are  presently  being  delivered  to  about  1200  students  registered  either  in  one  of  the  nine 
Distance  Education  Schools  or  within  one  of  the  regular  classroom-based  schools  in  British  Columbia. 

Open  School  has  traditionally  developed  large  print  -based  courses  with  some  complimentaiy  wrap  around 
media  such  as  text,  video,  or  audio.  Development  time  from  the  planning  stage  to  deliveiy  took,  on  average, 
two  years  to  complete.  Once  a course  was  delivered,  it  was  cumbersome  and  time  consuming  to  update  and 
maintain.  Courses  were  seen  as  being  out  of  date,  too  large,  too  much  work,  and  provided  primarily  a 
singular  pathway  for  students  to  learn.  Open  School  had  some  previous  experience  with  developing  courses 
for  on-line  use,  but  realized  the  need  to  change  their  course  development  processes  in  order  to  develop 
more  timely  and  flexible  content.  The  implementation  of  SGML  as  an  underlying  course  development  tool 
for  planning,  development,  production  and  delivery  was  identified  as  a means  to  an  end.  This  technology 
can  facilitate  the  following: 

• creation  of  courses  from  instructional  design  to  deliver, 

• creation  of  assessment  types  and  mechanisms, 

• identification,  association,  and  categorization  of  resources  within, 

• identifies  the  course  architecture, 

• creation  of  custom  views  of  materials  for  different  types  of  learners, 

• storage  and  management  of  parts  and  pieces  of  course  materials, 

• output  to  both  print  and  web,  as  well  as  to  other  media. 

SGML  has  roots  in  the  high  tech  and  publishing  industries  and  is  the  international  standard  for  representing 
electronic  text  in  a mark-up  language.  The  mark-up  provides  a means  for  interpreting  the  text.  SGML  is  not 
dependent  on  any  specific  hardware  or  operating  system  SGML  provides  a consistent  approach  to 
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identifying  the  components  of  a document  and  how  they  relate  to  each  other.  The  process  permits  explicit 
tagging  to  identify  the  components  of  a "course":  lessons,  learning  outcomes,  glossary  terms,  activities, 
online  resources,  etc.  There  are  an  infinite  number  of  document  structures  that  can  be  defined  depending  on 
the  specific  requirements.  SGML  is  a descriptive  mark-up  language  rather  than  a procedural  mark-up 
language.  A descriptive  mark-up  language  uses  codes,  called  elements,  to  name  and  categorize  parts  of  a 
document.  Procedural  mark-up  language,  like  that  of  HTML,  is  often  used  to  represent  text  in  a certain 
style  like  bold.  SGML  also  includes  a DTD:  Document  Type  Definitions.  The  DTD  defines  the  "allowed" 
elements  and  specifies  the  order  in  which  the  elements  may  or  may  not  appear.  The  authoring  process  must 
produce  documents  that  conform  to  the  specifications  of  the  DTD. 


The  Marriage  of  Instructional  Design  and  SGML:  The  Benefits 

Open  School,  as  part  of  the  Open  Learning  Agency,  has  a long  history  and  experience  with  implementing 
sound  instructional  design  practices  for  developing  learning  materials.  Fundamental  stages  of  instructional 
design  including  needs  assessment,  learner  analysis,  planning  (review),  development  (prototype,  formative 
evaluation),  production  (evaluation),  delivery,  and  summative  evaluation  - all  have  been  at  the  heart  of 
course  development  practices  for  over  twenty  years.  Various  instructional  technologies  have  been  used  over 
the  years  and  have  often  been  perceived  as  tools  whether  they  were  word  processing  software  programs, 
computer  conferencing  systems,  videos  audiotapes,  and  more  recently  the  World  Wide  Web/Intemet. 

However,  with  the  implementation  of  SGML,  instructional  design  methods  became  more  explicit  as  part  of 
the  structured  process  dictated  by  the  technology.  Open  School’s  SGML  based  courses  have  been 
developed  with  very  explicit  Document-Type-Definitions  that  lay  out  the  architecture  of  courses  from  the 
instructional  design  planning  stage  to  the  delivery  stage.  This  process  has  permitted  course  developers  to 
explicitly  examine  the  contents  of  their  courses  in  a more  detail  manner  that  was  never  before  possible.  A 
course  basically  “grows”  from  an  instructional  design  plan  (e.g.,  consisting  of  learning  profiles,  learning 
outcomes,  resources,  proposed  activities  and  assessment)  into  a fully  authored  final  product  (e.g., 
completed  modules  with  lessons  including  narrative,  instructions,  content,  activities,  and  assessment). 
Because  of  the  structured  information  approach  to  course  development,  “bits  and  pieces”  (e.g.,  granules, 
learning  objects)  of  a course  can  be  re-used  and  re-purposed  for  other  learning  situations  or  outputted  to 
different  formats  (e.g.,  print  or  the  Web).  Developing  14  courses  for  on-line  and  in  the  print  at  the  same 
time  would  not  have  been  possible  without  the  use  of  SGML  as  an  underlying  tool. 


The  Marriage  of  Instructional  Design  and  SGML:  The  Challenges 

Although  there  are  many  benefits  in  the  implementation  of  SGML  as  a course  development  framework, 
Open  School  has  also  faced  a number  of  challenges.  Here  is  a list  of  the  main  challenges: 

• Adopting  a new  technology  that  entailed  a whole  new  infrastructure  (e.g.,  with  new  hardware, 
software,  additional  staff,  extensive  training  for  internal  staff,  new  procedures  to  follow). 

• Reviewing  and  analyzing  existing  processes  and  products  in  order  to  come  up  with  consistent 
structures  and  language  for  different  course  architectures  (e.g.,  deciding  on  what  constitutes  a 
course,  a module,  a lesson,  an  activity,  etc). 

• Designing  content  to  be  used  in  different  learning  situations.  With  a database  as  a backbone, 
SGML  files  can  be  used  by  other  course  developers  to  re-use  existing  content.  There  are  issues 
around  designing  content  to  be  stand-alone  (e.g.,  learning  objects).  How  granular  can  content  be 
(e.g.,  a paragraph  or  a lesson)? 

• Ensuring  that  content  which  is  designed  in  “bits  and  pieces”  (e.g.,  learning  objects)  is  pulled 
together  into  a cohesive  manner  which  provides  an  effective  learning  environment  for  students. 


World  Wide  Web  Course  Delivery  in  Developing  Countries:  A South  African 

Perspective 

Shaun  Father  Geoff  J. Erwin 

School  of  Mathematics,  Information  Technology  & Computer  Science 
University  of  Durban  Westville,  Private  Bag  X54001 
Durban,  South  Africa 

spather@is.udw.ac.za  erwin@is.udw.ac.za 

Abstract:  There  is  increasing  international  usage  of  the  World  Wide  Web  for  course  delivery.  The 
current  South  African  education  system  is  still  structured  as  a legacy  of  the  apartheid  system.  The 
South  African  government  currently  promotes  the  uses  of  technology  to  enhance  access  to 
education  and  to  foster  resource-bas^  learning.  The  authors  report  findings  of  a survey  of  Web- 
based  course  delivery  at  South  African  tertiary  institutions,  including  problems  being  experienced 
by  academics  in  adopting  Web-based  techniques. 

Introduction 

The  Internet  and  the  World  Wide  Web  (WWW)  are  part  of  a shift  away  from  traditional  classroom  based  course 
delivery  methods  (See  for  example  Pilgrim  & Creek,  1998,  p.  189;  Kinzie  et.al.,  1996  p.59).  The  South  African 
situation  does  not  reflect  this  trend.  South  Africa  is  characterized  by  a transforming  education  system  that  aims  to 
address  imbalances  brought  about  by  the  apartheid  era.  Disenfranchised  sectors  of  the  South  African  population 
have  historically  had  limited  access  to  higher  education  (Christie,  1 986). 

Background 

The  South  African  Department  of  Education  promotes  the  use  of  technology  (including  the  World  Wide  Web)  to 
enhance  and  widen  access  to  higher  education  and  to  foster  resource-based  learning.  The  Department  of  Education  is 
actively  engaged  in  promoting  technology-enhanced  learning.  It  has  initiated  a five-year  strategic  plan,  through  the 
National  Centre  for  Educational  Technology,  to  build  an  infrastructure  to  support  technology-enhanced  learning 
(Department  of  Education,  South  Africa).  The  current  social  and  education  systems  are  still  heavily  influenced  by 
inheritances  from  the  apartheid  political  system  before  1 994.  For  example,  the  majority  of  South  Africa's  population 
(mainly  black)  do  not  have  access  to  the  Internet  (Webcheck  National  Survey,  1999).  Potential  university  students 
have  no  experience  in  the  use  of  information  and  communication  technologies  (ICTs).  Currently,  only  0.02%  of 
South  African  schools  have  an  Internet  connection  (IDRC,  2000).  Their  ability  to  use  Web-based  education  is 
limited.  The  lack  of  telecommunications  infrastructure,  combined  with  low  levels  of  technological  literacy  of  the 
population  is  a major  obstacle  to  deployment  of  Web-based  material.. 

The  Survey 

A survey  of  academics  from  fourteen  South  African  tertiary  institutions  was  conducted  between  June  and  September 
1999.  A non-experimental  survey  method,  using  a structured  questionnaire,  was  used  to  investigate  the  following: 
The  extent  of  usage  of  specific  WWW  features  for  course  delivery.  In  particular  the  authors  assessed  to  what  extent 
current  adopters  have  used  the  WWW  to  move  away  from  traditional  teaching  methods.  Specific  features  listed  in 
the  survey  were:  Use  of  Email,  posting  of  syllabi  (dynamic  & static),  detailed  course  information  updated  regularly, 
distribution  of  lecture  notes,  links  to  external  sites,  feedback  mechanisms  to  students,  online  chat-room,  streaming 
video,  multimedia  online  lectures,  animation,  submission  of  online  homework,  testing  and  examinations.  Problems 
that  exist  at  tertiary  institutions  that  are  impeding  the  implementation  of  WWW  technology.  Specifically  the  survey 
examined  areas  of  availability  of  human  and  physical  resources,  organizational  conditions,  and  the  attitude  of 
academics  to  the  use  of  WWW  as  compared  with  traditional  classroom  based  methods. 


Findings 
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1.  Current  and  intended  usage  of  the  WWW  by  South  African  academics: 

Figure  1 (see  http://isMdw.ac.za/staJf/research/web/figureLhtm)  indicates  that  current  levels  of  adoption  of  the 
thirteen  WWW  based  activities  listed  in  the  survey  fall  into  two  broad  categories.  The  first  category  reflects  a 
relatively  high  level  of  use.  The  seven  activities  in  this  category  (e.g.  Email,  syllabus  postings,  distribution  of 
lecture  notes)  are  activities  that  are  widely  used  and  are  those  that  support  traditional  styles  of  teaching.  They  do  not 
necessarily  change  the  actual  method  of  classroom-based  delivery.  A possible  reason  for  this  could  be  that  such 
activities  could  be  implemented  at  a low  cost  and  with  no  need  to  spend  a large  amount  of  time  doing  in-depth 
planning  and  design.  The  second  category  reflects  very  low  or  non-existent  levels  of  use  (six  activities).  This 
category  involves  activities  (e.g.  Chatrooms,  multimedia  online  lectures,  online  testing)  that  would  significantly 
move  away  from  traditional  teaching  methods.  These  are  activities  that  require  intensive  planning  and  design,  and 
are  also  dependent  on  a high  availability  of  a technological  infrastructure  supported  by  adequately  trained  staff. 
There  is  an  increased  level  of  usage.  Respondents  also  indicated  an  intention  to  increase  level  of  use  of  activities  that 
fall  into  the  second  category  listed  above.  This  is  a positive  indication,  since  the  activities  that  fall  into  this  category 
are  those  that  will  realise  a paradigm  shift  away  from  traditional  course  delivery  methods  to  a student-centred, 
constructivist  approach. 

2.  Comparison  of  level  of  South  African  tertiary  institutions  with  international  usage 

The  WWW  features  listed  in  the  survey  echoed  those  from  an  international  survey  that  was  conducted  by  an 
American  university  at  the  end  of  1998  (Peffers  & Bloom,  1998).  Figure  Two  (see 

http://is.udw.ac.za/stajf/research/web/figure2.htm)  provides  a comparison  between  the  South  African  tertiary 
educators  surveyed  and  those  of  the  international  survey.  The  similarities  in  the  trends  are  noted  in  the  paper. 
However,  it  must  be  noted  that  this  not  imply  that  the  broad  state  of  WWW  adoption  for  course  delivery  in  South 
Africa  is  at  the  same  levels  of  usage  as  our  international  counterparts. 

3.  Existing  difficulties  in  the  implementation  of  Web-based  course  delivery. 

Stajfing  problems:  The  results  of  the  survey  suggest  that  academic  staff  do  no  have  the  necessary  skills  to 

implement  Web-based  courses.  The  situation  is  further  compounded  by  the  absence  of  support  staff  needed  to  run 
Web-based  courses.  Organizational  conditions:  The  Survey  suggested  that  the  organizational  conditions  such  as 
time  and  flexibility  in  respect  of  work  commitments  do  not  allow  academics  (especially  if  they  are  novices  at 
implementation  of  Web  technologies)  to  experiment  with  WWW  technologies.  Financial  Restraints:  The  problem 
of  shrinking  financial  resources  is  affecting  service  delivery  at  all  levels  of  education  in  South  Africa.  The  survey 
indicates  that  current  budgets  do  not  allow  them  to  invest  in  Internet  technologies. 

Conclusion 

The  desire  for  on-line  teaching  at  South  African  tertiary  institutions  is  increasing.  The  survey  indicates  that 
academics  acknowledge  the  positive  role  of  the  WWW  in  course  delivery.  Key  recommendations  are:  Adopters  of 
Web  technology  in  the  classroom  need  to  be  encouraged  to  utilise  the  Web  for  more  than  merely  support  of 
traditional  lectures;  Management  at  the  faculty  and  institutional  level  should  be  encouraged  to  create  the  possibility 
for  and  invest  in  adequate  training  of  both  academic  and  support  staff  Heads  of  academic  units  support  the 
adoption  of  Web  technologies  by  providing  incentives  to  staff.  Tertiary  institutions  should  investigate  possible 
collaboration  with  external  partners  in  the  private  sector  as  a means  of  alleviating  high  costs.  Some  of  the  issues  that 
require  further  research  and  investigation  are:  international  experiences  and  collaboration  with  counterparts; 
Development  of  guidelines  for  designing  Web-based  courses;  the  long  term  effectiveness  of  Web-based  teaching  on 
students;  the  role  of  academics  in  developing  Web-based  courses;  and  user-  friendly  authoring  tools. 

References 

Christie,  P.  (1986).  The  Right  to  learn  - The  struggle  for  education  in  South  Africa.  Raven  Press/SACHED,  Johannesburg. 

Kinzie  et.al  Kinzie,  M.B.,  Larsen,  V.A.,  Burch,  J.B.,  Baker,  S.M.  (1996).  Frog  Dissection  via  the  World  Wide  Web. 
Educational  Technology  Research  and  Development,  44(2),  59-69. 

Peffers,  K.  and  Bloom,  S.  (1999).  Internet  Based  Innovations  for  Teaching  IS  Courses:  The  State  of  Adoption,  1998-2000.  Jitta 

1(1). 

Pilgrim  & Creek  Pilgrim,  C.J.,  and  Creek,  M.J.  (1998).  The  Swinburne  On-line  Educational  Project.  SIGSCE  Bidletin,  30(3), 
189-192. 

IDRC,  The  Teacher/Mail  & Guardian,  21  February  2000.  Most  without  classrooms  - Study  by  International  Development 
Research  Centre(IDRC). 

Webchek  (1999).  Survey  of  South  African  Internet  Users,  <http://www.webchek.co.za/reports.html> 


1711 


Online  Conferencing:  Variations  on  Structure  and  Participation 


Hsinyi  (Sindy)  Peng 
University  of  Missouri-Columbia 
152  Parkade  center 
Columbia,  MO  6521 1 
Sindy520@mizzou.edu 

Gail  E.  Fitzergald 
University  of  Missouri-Columbia 
1 52  Parkade  center 
Columbia,  MO  65211 
Fitzgerald@missouri.edu 

Louis  P.  Semrau 
Arkansas  State  University 
P.O.Box  1450 
State  University,  AR  72467 


Abstract:  This  paper  looked  at  the  variation  of  structure  and  participations  for  online  conferences. 
Four  different  structural  models  were  used  in  designing  the  conferences,  which  were  (1)  Open- 
ended  discussions  by  expert,  (2)  Scaffolded  discussion  with  facilitated  follow-up,  (3)  Scaffolded 
discussion  without  facilitated  follow-up,  and  (4)  Open-ended  discussion  without  facilitated  follow- 
up. Data  were  collected  on  frequency  of  messages,  word  count  of  messages,  and  key  word  analysis 
for  themes.  The  findings  will  provide  comparative  and  descriptive  data  related  to  conference 
structures,  participant  variables,  and  levels  of  participation  for  future  research  in  online 
conferencing. 


Use  of  Online  Conferencing  as  a Vehicle  for  Training  Special  Educators 

There  is  evidence  that  materials  designed  to  assist  teachers  in  working  with  students  with  behavioral  problems  are 
currently  in  high  demand.  The  design  and  development  of  the  Internet  as  a delivery  medium  for  such  continuous 
training  in  both  preservice  and  inservice  teacher  education  has  been  documented  in  special  education  (Smith, 
Martin,  & Lloyd,  1998).  A team  of  researchers,  funded  by  a U.S.  Department  of  Education  grant,  has  been  hosting  a 
series  of  online  conferences  for  preservice  and  inservice  educators  under  the  auspices  of  the  Virtual  Resource  Center 
in  Behavioral  Disorders  (Fitzgerald  & Semrau,  1998-2000).  These  virtual  conferences  involve  the  participation  of 
national  experts  in  the  behavioral  disorders  field  for  the  purpose  of  providing  in-depth  discussion  of  topics  and 
social  discourse  regarding  the  application  of  the  information. 

Design  and  Structure  of  the  Virtual  Conferences 

Four  different  structural  models  were  used  in  designing  the  online  conferences.  Each  conference  ran  for  two  weeks, 
but  the  degree  of  openness  and  structure,  involvement  of  the  online  expert,  and  the  use  of  discussion  facilitators 
differed  across  the  conferences. 

Conference  #1 : Open-ended  Discussion  by  Expert.  95  participants;  477  messages. 

One  expert  online  for  two- week  duration  with  all  participants  in  one  large  discussion  group;  no  required  pre- 
conference preparation  or  readings;  messages  not  organized  by  themes  or  available  via  archives. 

Conference  #2:  Scaffolded  Discussion  with  Facilitated  Follow-up.  105  participants;  283  messages. 

One  expert  online  for  first  week  with  all  participants  in  large  group;  participants  grouped  into  four  smaller 
discussion  groups  directed  by  project  facilitators  for  second  week;  required  writing  of  a case  scenario;  suggested 
readings  offered  via  web  site;  messages  organized  by  themes  and  available  on  web  archives. 
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Conference  #3:  Scaffolded  Discussion  without  Facilitated  Follow-up.  37  participants;  155  messages. 

One  expert  online  for  first  week  with  all  participants  in  large  group;  participants  continued  discussion  for  second 
week  without  expert  or  facilitators;  pre-conference  self-evaluation  encouraged;  suggested  readings  offered  via  web 
site;  messages  organized  by  themes  and  available  on  web  archives. 

Conference  #4:  Open-ended  Discussion  without  Facilitated  Follow-up. 

Three  national  facilitators  for  first  week  with  all  participants  in  one  large  discussion  group;  participants  continue  for 
second  week  without  national  or  project  facilitators;  no  required  pre-conference  contributions  required;  free  CD- 
ROM  provided  prior  to  conference  containing  suggested  software  and  resources  for  viewing;  messages  organized  by 
themes  and  available  on  web  archives;  greater  mix  of  field  teachers  participating  with  preservice  teachers. 


Preliminary  Findings  & Importance  of  the  Work 

The  findings  reveal  equivalent  levels  of  participation  regardless  of  prior  computer  experience,  teaching  experience, 
access  to  equipment,  typing  skills,  learning  styles,  writing  anxiety,  or  frequency  of  e-mail  and  Internet  use.  Writing 
anxiety  may  play  a role  in  length  of  message  but  not  frequency  of  responding.  Graduate  students  demonstrate 
significantly  higher  levels  of  participation  compared  to  undergraduate  students.  Less  structured  conferences  allow 
participants  to  initiate  more  of  their  own  topics  of  concern.  Instructors  have  an  important  role  in  providing  technical 
assistance.  Instructors  and  course  requirements  impact  participation  and  authenticity  of  involvement  of  preservice 
participants.  Readings  provide  common  ground  for  discussion.  Written  scenarios  can  be  requested  from  participants, 
but  they  emerge  naturally.  Structure  and  facilitation  provide  an  even  level  of  participation  and  ongoing  discussion. 
Messages  must  be  archived  by  threads  for  asynchronous  access. 

This  research  provides  a comparison  of  online  participation  during  conferences  that  differ  in  open-ended  vs, 
scaffolded  features  and  the  use  of  facilitation.  Online  conferencing  appears  to  be  a valid  and  valued  method  of 
providing  preservice  and  inservice  training,  but  the  variables  of  designing  and  running  virtual  conferences  are  more 
of  an  art  than  a science  at  this  stage  of  research.  These  studies  will  provide  comparative,  descriptive  data  related  to 
conference  structure,  participant  variables,  and  levels  of  participation  to  guide  future  research  in  this  mode  of 
instruction. 
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Introduction 

In  medical  education,  Problem-Based  Learning  (PBL)  is  becoming  appreciated  more  and  more.  The  design  and 
implementation  of  a problem-oriented  medical  curriculum,  however,  is  very  time-consuming  for  the  educational 
staff  Case-based,  Web-based  training  systems  can  help  the  educators  to  develop,  implement,  organise  and  reuse 
well-structured  multimedia  cases.  For  students,  such  a system  can  be  used  to  improve  and  test  their  problem-solving 
competence.  With  respect  to  heterogeneous  learning  environments,  the  system  should  be  flexible  and  adaptable. 
CAMPUS  tries  to  meet  these  requirements  by  covering  the  needs  of  different  user  groups  (e.g.,  students,  educators, 
physicians)  in  different  application  scenarios  (e.g.  self-study,  presentation,  learning  groups)  to  get  maximum  benefit 
of  integrated  medical  cases.  With  the  CAMPUS  player  component  (learner’s  front-end),  users  can  work  out  medical 
cases  in  a simulative  - and  therefore  realistic  and  interactive  - way  or  let  the  system  present  a case  in  different 
manners.  Users  can  get  help  via  expert  comments  or  context-sensitive  systematic  knowledge  which  is  available  in 
addition  to  the  case  data.  With  the  CAMPUS  authoring  component,  medical  experts  can  feed  the  system  with  cases 
from  any  medical  field.  CAMPUS,  part  of  a comprehensive  virtual  university  project,  is  going  to  be  integrated  in  a 
reformed  medical  curriculum  approach  at  the  University  of  Heidelberg. 


The  idea  of  CAMPUS  is  to  develop  a case-based,  Web-based  training  system  in  Medicine  including  a broad  variety 
of  usage  modes  with  the  aim  of  minimizing  acceptance  problems  and  maximizing  the  effectiveness  and  efficiency  of 
case-data  input,  which  is  time-consuming  and  still  one  of  the  central  problems  of  case-based  systems. 

To  achieve  such  a functionality,  CAMPUS  provides  the  following  features: 

a)  Integration  of  medical  cases  from  any  medical  field 

b)  Storage  of  case  data  in  a very  detailed  manner  in  a relational  database 

c)  Presentation/lreatment  of  cases  in  different  adaptable  ways 

The  degree  of  interactivity/realism  of  a case  presentation/treatment  is  optional.  The  most  interactive  form  is 
‘decision’,  where  the  student  has  to  treat  a case  in  a realistic,  simulative  way  leading  to  a maximum  of  active 
knowledge  processing.  The  user  has  to  ask  questions  of  anamnesis,  order  physical,  lab  or  technical  exams,  make 
diagnostic  decisions  and  propose  a therapy.  In  the  form  ‘compact’  the  student  only  has  to  make  diagnostic 
decisions  and  propose  the  therapy.  The  results  of  the  different  tests  are  shown  en  bloc.  If  a user  is  only 
interested  in  medical  case  data  he  can  choose  the  form  ‘total’,  where  the  whole  case  is  presented  at  once  with  no 
interactivity  at  all. 

With  these  different  forms  of  application  the  user  can  look  at  a case  the  way  he  likes  and  is  able  to.  For 
example,  if  a user  does  not  want  to  do  the  anamnesis  on  his  own  for  any  reason,  he  may  choose  the  form 
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‘compact’.  If,  on  the  other  hand,  he  wants  to  concentrate  his  learning  activities  on  the  physical  exam,  he  may 
choose  the  form  ‘decision’, 

d)  Expert  comments  on  demand 

It  is  very  important  to  explain  certain  things  about  a case,  e.g.,  why  a diagnosis  has  been  made  or  why  a 
particular  exam  has  been  done.  Hence,  for  every  particular  point  within  a case  the  CAMPUS  author  can  define 
an  expert  comment. 

e)  Questions  if  wanted 

Questions  are  - besides  other  things  - an  important  aspect  of  interactivity  and  active  knowledge  processing. 
Thus  the  case  author  can  define  questions  of  different  types  on  every  definable  point  within  a case,  e.g.,  after  or 
before  the  user  gets  the  answer  to  a certain  question  of  the  anamnesis  or  after  or  before  the  whole  physical 
exam.  The  user  can  choose  if  he  wants  to  see  only  case-dependent  questions,  case-dependent  and  case- 
independent  questions  or  no  questions  at  all, 

f)  Distinction  between  case  data  and  systematic  knowledge 

Every  user  has  a different  state  of  knowledge.  Therefore,  every  user  needs  help  at  different  points  when 
working  on  a case.  Hence,  CAMPUS  shows  context-sensitive  systematic  knowledge  only  on  demand.  The 
knowledge  can  be  internal  (integrated  by  CAMPUS  authors)  or  external  (using  external  knowledge  sources 
from  digital  libraries  over  the  internet). 

g)  Detailed  case  search 

Cases  are  searchable  by  defining  symptoms  or  diagnoses.  For  example,  one  could  look  for  cases  with  fever, 
shivering  and  chest  X-ray  abnormal.  Such  a specific  search  is  possible  because  of  the  detailed  database. 

h)  Execution  over  the  intemet/intranet  or  locally 

To  acquire  platform- independence,  permanent  actuality  and  world- wide  access  along  with  all  capabilities  of  a 
programming  language,  CAMPUS  is  implemented  in  Java  as  a Browser  applet.  For  users  at  home,  it  may  be 
better  to  have  a local  application  to  save  time  and  costs.  Therefore,  CAMPUS  is  also  executable  locally  using  a 
local  database. 


In  order  to  avoid  the  structure  and  vocabulary  of  cases  varying  from 
author  to  author,  a case  has  a well-defined  structure  and  has  to  use  a 
defined  vocabulary  for  anamnesis  questions,  exams,  diagnosis  (ICD-10) 
and  therapies.  Every  case  has  the  following  structure:  anamnesis, 
physical  exam,  suspected  diagnosis  followed  by  one  to  many  passes  of  a 
therapy  loop  with  examinations  (physical,  technical,  lab),  working 
diagnosis  and  therapy.  That  means  that,  unlike  many  other  case-based 
training  systems,  CAMPUS  doesn’t  stop  at  the  first  diagnosis  given  by 
the  user  but  considers  a case  in  a more  detailed  and  realistic  way. 

To  give  the  user  an  easy  to  understand  user  interface,  we  implemented  a 
so-called  situated  learning  environment,  where  the  ‘situation’  is  a 
physician’s  room. 


Conclusion 

CAMPUS  implements  a new  dimension  of  flexibility  of  a case-based,  medical  Web-based  training  system.  It  can  be 
used  by  different  user  groups  in  different  learning  environments.  Therefore,  the  effectiveness  and  efficiency  of  the 
time-consuming  case  data  input  has  been  improved.  CAMPUS  offers  the  chance  to  develop  a case-database  with 
well -structured  multimedia  cases  to  be  used  by  students  as  well  as  educational  staff  (especially  in  problem-oriented 
curricula)  who  can  use  and  - more  important  for  the  educators  - reuse  cases  in  several  scenarios. 

A first  prototype  is  available  and  is  going  to  be  integrated  in  the  reformed  medical  curriculum  at  Heidelberg  at  least. 
A comprehensive  evaluation  is  taking  place  in  June/July  2000.  An  international  version  of  CAMPUS  is  planned. 
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Evaluating  the  Effects  of  Web-based  Collaboration  in  a Distance  Education 

Program 


Educators  have  increasingly  recognized  the  important  role  collaboration  can  play  in  learning.  Asynchronous 
web  conferencing  has  been  identified  as  a unique  tool  in  promoting  such  collaboration,  particularly  when 
applied  in  distance  education  coursework  [Knapczyk,  Chung  & Baik  2000,  Moller  1998,  Duffy  et  al.  1998, 
Bonk  & Cunningham  1998,  Ernest  1995,  McMahon,  O’Neill  & Cunningham  1992].  In  the  past  three  years  we 
have  integrated  web  conferencing  into  our  existing  video-based  distance  education  program  at  Indiana 
University  to  promote  collaboration  and  ownership  among  learners  [Rodes  et  al.  2000,  Rodes,  Knapczyk  & 
Chapman  1999,  Chung,  Rodes  & Knapczyk  1998]. 

While  we  have  been  pleased  with  the  results  of  web  conferencing  in  our  program,  we  recognize  the  need  to 
evaluate  its  effects  in  verifiable  terms.  In  fact,  although  much  has  been  written  about  the  benefits  of 
asynchronous  web  conferencing,  few  studies  have  been  conducted  to  verify  these  findings.  The  education 
literature  typically  offers  descriptions  of  programs  and  effects,  with  little  controlled  study  or  verification. 
Furthermore,  little  has  been  written  on  the  possibilities  of  web  conferencing  when  integrated  with  other  distance 
education  technologies.  To  address  this  lack,  we  have  been  conducting  a study  in  Spring  2000  to  investigate  the 
effects  of  asynchronous  web-based  conferencing  on  both  cognitive  and  emotional  aspects  of  learning. 

The  Collaborative  Teacher  Education  Project  at  Indiana  University  offers  graduate-level  special  education 
coursework  to  cohort  groups  of  teachers  in  rural  Indiana  communities.  Classes  are  delivered  via  two-way  video 
and  are  supplemented  by  e-mail,  fax  machines,  and  web-based  conferencing  using  the  SiteScape  Forums  web 
conferencing  system  [Knapczyk  et  al.  2000,  Knapczyk,  Rodes  & Chung  1998].  In  the  Spring  2000  semester  we 
offered  a single  course  on  behavior  management  to  in-service  teachers  at  six  sites. 

For  the  current  study  we  divided  our  course  content  into  four  units.  The  independent  variable  of  our  study  was 
the  use  of  asynchronous  web-based  conferencing.  For  each  unit,  we  developed  two  parallel  sets  of  assignments 
and  activities:  one  set  was  to  be  completed  individually  with  collaboration  taking  place  at  the  learners’ 
discretion;  the  other  set  was  to  be  completed  collaboratively  in  web-based  learner  teams.  During  each  unit, 
learners  at  three  sites  completed  their  work  using  web-based  collaboration,  while  learners  at  the  other  three  sites 
completed  their  work  using  the  “traditional”  assignments. 

To  facilitate  direct  comparison,  during  every  unit  each  web-based  site  was  paired  with  a non- web-based  site  to 
meet  for  a video-based  course  once  a week.  In  this  way  we  ensured  that  the  sites  to  be  compared  received  the 
same  instruction.  After  each  unit  the  roles  assigned  to  the  groups  were  reversed.  For  example,  learners  in  Tell 
City  were  linked  each  week  with  learners  in  Spencer  in  a video  session  originating  from  the  lU-Bloomington 
campus  where  the  instructors  were  located.  For  the  first  unit  of  the  course,  the  Tell  City  group  completed 
assignments  in  web-based  teams,  while  the  Spencer  group  completed  assignments  individually  or  using 
informal  collaboration.  For  the  second  unit  of  the  course,  these  roles  were  switched,  and  so  on.  By  pairing  sites 
in  this  way,  by  switching  the  assigned  roles  at  the  end  of  each  unit,  and  by  repeating  this  full  cycle  two  times 
during  the  semester  we  aimed  to  minimize  the  effects  of  any  differences  among  learner  characteristics  and 
environments  across  the  different  sites. 
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Abstract:  To  study  the  effects  of  asynchronous  web-based  collaboration  in  an  existing  video- 
based  distance  education  program,  the  authors  structured  coursework  to  enable  evaluation  of 
differences  in  cognitive  and  emotional  aspects  of  learning  between  those  using  web-based 
collaboration  and  those  completing  assignments  through  traditional  independent  methods. 


The  dependent  variables  we  have  been  investigating  reflect  both  cognitive  and  emotional  aspects  of  learning. 
We  are  evaluating  the  following  three  elements  of  content  learning  [Bloom  1956]: 

• acquisition— through  product  evaluation  and  content-based  tests 

• application— through  product  evaluation  and  field  reports  from  learners 

• retention— through  follow-up  tests  and  surveys  on  content  issues 

In  the  emotional  domain,  we  are  using  both  immediate  and  follow-up  surveys  to  evaluate  learner  satisfaction, 
which  is  a particularly  important  factor  to  consider  in  working  with  adult  learners  [Galbraith  1998]. 

A specific  grading  rubric  was  used  to  ensure  uniformity  of  evaluation  and  to  be  sure  products  were  assessed  for 
issues  of  applicability  as  well  as  acquisition.  To  facilitate  clear  comparison,  all  surveys,  testing,  and  product 
evaluation  have  been  linked  to  the  four  individual  units  of  the  course.  In  this  way  we  can  compare  the  results  at 
each  study  site  not  only  to  the  linked  cohort  site  using  the  opposite  approach,  but  also  to  the  results  at  that  same 
site  during  other  units,  to  track  any  differences  as  learners  moved  from  web-based  to  non- web-based  activities. 

By  completing  this  study  we  hope  to  have  gathered  reliable  data  on  the  effects  and  the  effectiveness  of  web- 
based  collaboration  when  combined  with  other  distance  education  technologies.  We  anticipate  that  our  findings 
will  contribute  to  an  understanding  of  the  importance  of  collaborative  learning  and  web-based  conferencing  in 
distance  education  programs,  and  for  education  in  general. 
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Abstract:  Edubox®  is  a system,  built  at  the  Open  University  of  The  Netherlands,  that  supports 
educational  innovation  processes  in  higher  education.  The  system  allows  maximal  flexibility  in  that 
it  enables  educational  designers  to  model  an  instructional  system  completely  after  their  ideas.  In 
this  paper  the  phases  of  the  design,  development  and  implementation  of  an  instructional  system  are 
described  with  particular  attention  to  Edubox®. 

Introduction 

Edubox®  is  a system,  built  at  the  Open  University  of  The  Netherlands,  that  supports  educational  innovation 
processes  in  higher  education.  The  system  allows  maximal  flexibility  in  that  it  enables  educational  designers  to 
model  an  instructional  system  completely  after  their  ideas,  unhindered  by  implicit  didactic  beliefs  similar  systems 
often  harbor. 

An  instructional  system  is  here  defined  as  the  assembly  of  'objects'  that  collectively  assist  a student  in  acquiring  a 
set  of  learning  objectives.  At  a more  concrete  level,  an  educational  system  features  such  items  as  people  (e.g. 
teacher,  fellow  students,  experts),  tools  (e.g.  word  processor,  browser),  knowledge  sources  (e.g.  books,  videos,  cd- 
rom's),  and  activities  (e.g.  writing  a paper,  peer-assessing  a fellow  student). 

Design  Stage 

Edubox®  is  able  to  provide  the  eductional  designer  with  such  freedom  and  flexibility  because  it  makes  use  of  a 
didactic  metamodel  embedded  in  an  Educational  Markup  Language  (EML)  - a generic  XML-based  language  - as 
described  by  Koper  (1999). 

In  principle,  everything  the  learner  or  teacher  will  experience  in  Edubox®  has  to  be  modelled  by  the  designer.  In 
contrast  with  traditional  electronic  learning  platforms  it  does  not  make  any  (implicit)  assumptions  (Onstenk  & 
Meijer,1998;  Dabbagh,  Bannan-Ritland  & Silc,  1999)  on  preferred  didactic  scenarios,  thus  providing  the  designer 
with  a didactic  "clean  slate". 

When  working  with  Edubox®  the  educational  designer,  needs  to  focus  his  design  activities  on  two  main  questions 
right  from  the  start: 

• What  is  the  structure  of  the  instructional  system  needed  to  support  the  acquisition  of  certain  learning  objectives 
by  students? 

• What  will  be  the  delivery  medium  of  the  designed  instructional  system? 

Decisions  on  the  Instructional  System 

The  instructional  system  to  be  developed  may  be  modelled  at  several  levels:  the  micro-level  (e.g.module),  meso- 
level  (e.g.  course),  and  macro-level  (e.g.  curriculum).  The  designer  has  to  make  up  his  mind  about  the  main  level  for 
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his  design  activities.  Subsequently,  the  designer  has  to  opt  for  a pedagogic  approach,  taking  into  account  several 
ingredients  of  his  instructional  system: 

• educational  objectives  and  prerequisites,  (aimed  at  the  acquisition  of  kno\vledge,  skills,  attitudes,  competences, 
complex  cognitive  ski  Is)  and  target  group 

• relations  between  the  stated  educational  objectives,  prerequisites  and  the  structure  of  the  instructional  system 

• the  context  in  which  the  education  takes  place  (roles,  sources,  instruments,  on-the-job) 

• situational  experiences  and  domain  specific  information  in  education 

• individual  differences 

• co-operation  and  collaboration 

• communication 

• the  use  of  different  educational  materials 

• assessment 

• the  kind  of  activities  which  have  to  be  performed  to  assure  learning  processes 

• the  learning  process  and  transfer  processes  passes  off 

• stimulating  motivation 

• sequencing  actvities 

Descisions  on  the  Delivery  Medium 

The  educational  designer  has  to  make  a decision  on  how  the  educational  system  will  be  delivered  to  the  learners.  For 
example:  will  the  product  be  a book  or  will  it  be  delivered  on  the  Internet?  Depending  on  this  decision  and  the  level 
of  implementation  of  the  instructional  system,  the  designer  will  have  to  design  a style  sheet  for  the  lay-out  of  a book 
or  an  entire  graphical  user  interface  (G.U.I.). 

Development 

Developing  the  Instructional  System 

In  the  development  phase  the  whole  instructional  system  has  to  be  structured  using  the  Educational  Markup 
Language  (EML).  For  specific  pedagogical  orientations  (e.g.  problem-based  learning,  case-based  learning),  ready 
made  EML-templates  may  be  available.  EML  knows  the  following  main  elements  (Tab.  1): 


eiement 

description  i 

metadata 

all  general  data  concerning  an  educational  object 

roles 

the  different  roles  persons  can  have  within  an  instructional 
activity 

objectives 

(learning)  objectives  which  are  related  to  an  instructional  system 

prerequisites 

prerequisites  (e.g.  in  terms  of  knowledge,  personal 
charactaristics,  instruments  available  to  learner)  related  to  an 
instructional  system 

content 

all  componounds  (defined  within  the  elements  'activity'  and 
'environment' ) which  are  conditional  for  the  acquisition  of 
(learning)  objectives 

activity 

a stimulation  of  the  learner  to  perform  certain  behavior 

environment 

componounds  (e.g.  tests,  knowledge  sources,  instruments) 
which  are  conditional  for  the  acquisition  of  (learning)  objectives 

method 

structure,  didactics  and  processes  within  an  instructional  system 

Developing  the  delivery  medium 

In  the  development  phase  the  necessary  features  for  publication  to  the  medium  chosen  during  the  design  phase 
should  be  developed. 

Implementation 

Because  of  the  richness  of  EML  and  the  flexibility  of  Edubox®,  different  designers  may  implement  similar  ideas  in 
very  different  ways. 
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Abstract:  Throughout  history,  technological  advances  have  led  to  scientific  breakthroughs. 
Computer-mediated  learning  environments  have  the  potential  to  function  similarly:  as 
technological  tests  of  educational  theories.  Here,  this  unexplored  research  role,  for  one  such 
learning  environment,  is  illuminated.  The  Generative  Virtual  Classroom,  a web-delivered 
virtual  classroom  for  teacher  education,  is  demonstrated  and  preliminary  research  findings 
with  respect  to  theory  advancement  reported.  Some  emerging  implications  for  computer- 
mediated  learning  and  for  the  generation  of  educational  knowledge  are  discussed. 


Cultural  development  is  littered  with  striking  examples  of  technological  advances  that  have  led  to  scientific 
breakthroughs.  The  successful  building  of  European  cathedrals  provoked  the  development  of  a detailed 
engineering  science  (Bronowski  1973/1992).  In  our  own  time,  pioneering  brain-imaging  technologies  have 
brought  important  new  neuroscientific  insights  (see,  for  example.  Carter  1998  and  McCrone  1999).  Similarly, 
technological  tools  for  learning,  especially  those  designed  on  particular  educational  principles,  ought  to  expose 
the  worth  of  those  principles.  In  action,  such  technologies  may  well  constitute  a potent  and,  as  yet,  unexplored 
research  tool  by  means  of  which  educational  theorising  can  be  advanced.  In  this  work-in-progress  paper,  I 
explore  this  research  role  for  one  such  technological  tool  for  learning:  the  Generative  Virtual  Classroom. 


The  Generative  Virtual  Classroom 

This  paper  begins  by  demonstrating  the  Generative  Virtual  Classroom  itself,  so  as  to  elucidate  its  educational 
principles.  Emerging  from  a series  of  classroom  studies  of  learning  and  learning  to  teach  (in  particular,  Cosgrove 
& Schaverien  1996  and  Schaverien  & Cosgrove  1997),  this  web-delivered  learning  environment  attempts 
sophisticated  computer-mediated  teacher  education  in  elementary  technology-and-science.  Its  design  embodies  a 
generative  view  of  learning  in  which  learning  is  conceived  as  a generating  and  testing  cycle  (after  Minsky  1986, 
Wittrock  1974,  1994  and  Plotkin  1994,1997).  According  to  this  view,  learners  generate  ideas  that  they  then  test 
on  their  value,  keeping  those  that  survive  their  tests  (Schaverien  & Cosgrove  1999,  2000).  In  a nested  pair  of 
virtual  classrooms  (a  virtual  elementary  one  and  a virtual  tertiary  one).  Education  students  can  develop  their 
views  of  learning  by  observing  exemplary  learning  by  young  children  from  an  archive  of  pre-recorded  digitised 
video  excerpts.  They  can  formulate  their  thoughts  about  these  excerpts  and  communicate  with  others  about  them, 
considering  others’  views  of  them,  including,  in  particular,  a generative  view  of  learning.  As  an  environment  in 
which  learners  can  develop  a generative  view  of  learning,  in  ways  consistent  with  such  a theory,  the  Generative 
Virtual  Classroom  is  poised  to  be  a powerful  technological  test  of  this  generative  theory  itself 


Do  Learners  Learn  Generatively  in  the  Generative  Virtual  Classroom? 

Already,  pilot  research  conducted  in  the  Generative  Virtual  Classroom  affirms  the  fruitfulness  of  viewing 
learning  generatively.  For  example,  in  a nine-month  autobiographical  study  of  the  development  of  her  views  of 
learning  in  the  Generative  Virtual  Classroom  (Allard  1 998),  an  Honours  Education  student  was  able  to  establish 
her  conceptual  progression,  for  herself,  and  to  identify  what  she  still  needed  to  understand.  On  analysis,  this 
student’s  learning  was  well  explained  in  generative  terms;  and,  as  well,  on  the  basis  of  her  experience,  minor 
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alterations  were  able  to  be  made  to  the  learning  environment  itself.  Such  studies  suggest  the  value  of  proceeding 
to  larger  scale  tests  of  the  worth  of  this  theory.  Consequently,  a large-scale  collaborative  research  trial  of  the 
Classroom  is  now  in  its  beginning  stages.  In  this  trial,  a full  cohort  of  First  Year  Education  students  (n="-100)  has 
begun  to  work  in  the  Generative  Virtual  Classroom,  yielding  early  data  on  the  nature  of  the  learning  of  a diverse 
range  of  individual  students  over  a semester.  Students’  text  contributions  to  the  Classroom’s  community  database 
and  e-mail  discussion  group  will  be  collated,  described  and  analysed  for  evidence  of  generativity,  in  an  attempt  to 
assess  the  worth  of  such  a theory  in  making  sense  of  what  they  did. 


Implications  for  Computer-mediated  Learning  and  the  Generation  of  Educational 
Knowledge 

Clearly,  the  recent  proliferation  of  technologically  mediated  environments  in  which  learners  can  teach  themselves 
has  brought  new  opportunities  to  explore  the  nature  of  learning.  Even  more  recently,  such  writers,  inter  alia,  as 
Alexander  & McKenzie  1998  and  Hannafin,  Hannafin  & Oliver  1997  have  begun  to  insist  that  these  new  learning 
environments  be  explicitly  grounded  in  educational  principles.  From  these  positions,  it  is  a short  but  significant 
step  to  conceiving  of  computer-mediated  learning  environments  as  instruments  for  the  testing  of  educational 
theories,  an  idea  whose  time  has  come.  Once  crystallised  in  these  terms,  such  a research  program  has  the 
potential  to  redress  long-standing  dissatisfaction  with  the  apparent  lack  of  theoretical  progression  in  Education  in 
particular  (Thagard  1992)  and  in  the  social  sciences  in  general  (Ziman  1978/1991).  The  paper  closes 
speculatively  by  discussing  the  implications  of  such  a re-conceptualisation  for  computer-mediated  learning  and 
for  the  generation  of  educational  knowledge. 
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Abstract;  This  paper  draws  on  data  gathered  during  a year-long  ethnographic  study  of  graduate 
education  classes  in  a 2-way  audio  2-way  video  distance  education  classroom.  Cultural  analysis  of 
transcript  excerpts  indicating  student  reactions  to  their  experiences  in  the  technologically  mediated 
classroom  highlight  the  anxiety  and  confusion  they  experience  in  making  the  shift  from  the 
familiarity  of  traditional  classroom  culture  to  the  uncertainty  of  a new,  often  strange  environment. 
This  transition  phase  in  classroom  cultures,  what  Turner  (1969)  calls  the  liminal  phase,  is  a time  of 
high  anxiety  but  also  of  high  learning.  Our  data  suggests  that  instructors  who  take  on  the  role  of 
technology  guide  and  provide  students  the  opportunity  to  vent  anxiety  and  learn  about  the 
technology,  create  environments  that  significantly  increase  the  comfort  levels  of  students  and 
receive  much  higher  ratings  from  them. 


Introduction 

Distance  learning,  via  web  based  and  two-way  interactive  video  classes  has  changed  the  market  areas  of 
educational  institutions  from  how  far  faculty  are  willing  to  drive  to  how  far  the  Internet  or  phone  lines  will  reach. 
Given  this  new  global  market,  institutions  of  higher  education  are  rushing  to  fill  the  niche.  Students  are  recruited, 
new  learning  environments  are  created  and  faculty  are  suddenly  asked  to  teach  classes  in  front  of  computer  screens 
and  cameras.  Although  the  literature  encourages  the  use  of  instructional  design  principles  when  teaching  this  way 
(Moore  & Kearsley  1 996),  in  the  rush  to  stake  out  new  territory,  such  classes  are  often  thrown  together  with  little 
planning  and  less  understanding. 

According  to  Willis  (1993),  “research  suggests  that  distance  education  and  traditionally  delivered 
instruction  can  be  equally  effective  if  the  distance  educator  puts  adequate  preparation  into  understanding  the  needs 
of  the  student  and  adapting  the  instruction  accordingly”  (p.  22).  But  how  does  one  do  that?  How  can  students  and 
teachers  anticipate  their  needs  in  a substantially  new  learning  environment?  Are  these  new  learning  environments 
traditional  classrooms  with  a few  technological  add-ons  where  the  assumption  can  be  made  that  student’s  needs  are 
little  changed?  Or  are  they  complex  new  environments  that  require  descriptive  and  analytic  illumination  by 
researchers  such  that  student  needs  can  be  assessed  and  then  addressed?  Our  research  suggests  the  latter. 

It  is  our  contention,  that  the  situation  people  face  in  distant  classes  is,  from  a cultural  perspective,  radically 
different  from  the  traditional  classroom.  The  enculturation  process  that  occurs  during  years  of  schooling  solidifies  in 
the  learner  norms  for  the  role  of  the  teacher,  for  methods  of  classroom  communication,  and  for  student  interaction 
that  lasts  a lifetime  (Henry,  1955).  The  entire  dynamic  of  teaching  and  learning,  which  is  deeply  grounded  in 
traditional  classroom  culture,  is  changed  in  complex  and  nuanced  ways  when  the  class  becomes  technologically 
mediated.  The  study  on  which  this  paper  is  based  used  the  approach  of  ethnography  and  the  perspectives  of  cultural 
anthropology  (Geertz  1973)  and  social  interaction! sm  (Mead  1934)  to  uncover  these  changes  in  classroom  culture 
and  illuminate  changes  in  the  teaching/leaming  dynamic. 

In  this  brief  paper  we  can  do  little  more  than  present  fragments  of  interviews  to  show  how  descriptions  by 
students  of  their  distance  education  experiences  and  an  awareness  of  traditionally  based  student  expectations  can 
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inform  a deeper  understanding  of  the  needs  of  students.  This  understanding  then  becomes  the  basis  for  strategies 
that  instructors  can  use  to  help  learners  through  the  cultural  transformation  process  and  to  have  more  successful 
learning  experiences. 


The  Study 

The  study  is  based  on  data  gathered  for  an  ethnographic  case  study  of  graduate  education  classes  during  the 
inaugural  year  of  a 2-way  audio  2-way  video  interactive  distance  learning  classroom.  Courses  included  in  the  study 
varied  significantly  in  teaching  style,  course  objectives,  course  content  and  student  responses.  Traditional 
ethnographic  and  qualitative  research  methods  facilitated  data  gathering  and  analysis  (Agar  1996;  Maxwell  1996). 
Data  gathering  involved  more  than  400  hours  of  participant  observations  in  classrooms,  formal  interviews  and 
informal  conversations  with  students  and  faculty,  weekly  e-mail  correspondence,  open  ended  surveys,  focus  group 
interviews,  and  video  of  all  classes.  Thematic  coding,  frequency  counts,  and  frequent  debriefings  between 
ethnographers  L.  and  R.  Schmertzing  contributed  to  interpretation  and  ongoing  analysis  of  the  data  (Spradley  & 
Mann  1975). 


The  Data 

Excerpts  of  student  comments  from  e-mails  and  interview  transcripts  introduce  the  complex  culturally  based  issues 
that  students  face  in  the  interactive  televised  classroom.  All  of  the  following  passages  were  written  by  students  in 
response  to  questions  from  the  researchers  regarding  the  students  first  night  in  the  distance  classroom.  We  will 
present  the  excerpts  in  the  data  section  and  comment  on  them  in  our  discussion. 

When  1 saw  the  room  set  up  I was  in  shock,  actually.  It  was  my  first  interactive  distance 
learning  class  and  I walked  into  the  room  thinking  "why  did  I sign  up  for  this?"  At  the  sight  of  all 
these  different  video  screens  and  cameras  all  around  the  room,  I didn’t  think  that  the  grades  would 
be  the  same.  I thought  they  would  favor  the  class  that  had  the  most  physical  interaction  with  the 
professor.  I became  sort  of  jealous  of  the  other  campus  and  resented  the  fact  that  she  (the 
instructor)  was  there.  I felt  like  I was  at  the  main  campus  where  the  professor  should  have  been. 

If  the  other  class  was  at  a branch  campus,  they  should  have  primarily  been  in  the  distance  learning 
aspect  of  it.  My  GPA  is  very  important  to  me.  I feel  that  the  grade  I earn  will  be  affected  by  how 
much  time  1 will  have  to  spend  watching  her  on  a monitor.  There  were  so  many  distractions,  it 
was  hard  for  me  to  stay  focused.  Also,  there  was  so  much  zooming  in  and  out  with  the  video 
cameras,  I honestly  got  motion  sickness  and  walked  out  of  class  feeling  sick  to  my  stomach  with  a 
headache.  I did  not  like  it.  The  cameras  were  set  up  in  awkward  angles  and  you  couldn't  see  the 
professor’s  head  when  she  was  talking.  We  got  great  views  of  the  back  of  her  head. 

Many  students  made  similar  comments,  although  the  attitudes  of  others  were  more  like  the  following  who  noticed 
the  difference  but  struggled  to  adjust. 

I was  really  surprised  at  my  reaction.  I’ve  taken  a web-based  class  before  and  I have  seen 
the  equipment  for  this  classroom  before.  I didn’t  think  1 would  feel  any  different,  but  I did!  I was 
surprised  at  how  I looked  on  camera  and  feel  that  I will  have  to  work  on  this  so  that  it  doesn’t 
impact  my  participation.  Also,  I forgot  several  times,  where  to  “look”  so  I would  be  facing  the 
remote  group  when  I “spoke”.  I also  found  that  as  the  night  wore  on,  I kind  of  avoided  responding 
to  questions  ‘cause  I felt  like  I was  monopolizing  the  session.  I did  like  being  able  to  “face”  the 
other  group  and  do  not  feel  that  the  experience  effected  my  learning.  I found  myself  “concerned” 
about  the  remote  site  when  we  had  a few  technical  problems  and  am  concerned  about  the  learning 
experience  when  the  instructor  goes  to  the  other  class  and  I am  in  the  remote  site,  having  to  rely  on 
the  technology. 

For  some  students  the  two-way  audio/video  class  they  participated  in  was  an  entirely  positive  experience. 
“After  the  first  night  of  class  I am  totally  satisfied  with  everything.  I especially  enjoy  being  able  to  take  the  class  at 
this  campus  and  am  excited  to  have  the  opportunity  to  be  a part  of  this  class.” 

Other  students  cited  particular  elements  of  the  technologically  mediated  classroom  that  fed  their  anxieties 
and  complicated  their  ability  to  focus  in  the  classroom.  “I’m  just  much  more  aware  of  asking  questions  and  making 
contributions,  because  I have  to  do  this  mechanical  thing.”  The  necessity  to  physically  encounter  technology  in 
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order  to  participate  in  class  contused  students  and  contributed  to  a magnified  concern  for  the  value  of  their  own 
comments.  “You  don’t  have  the  ability  to  just  sit  there  and  say,  ‘Wait  a minute;’  you’re  on  camera  now,  you’re  on 
air-time,  and  so  if  you  have  what  you  think  may  be  a little  question,  you’re  wasting  class  time  [if  you  ask  it].” 


Discussion 

The  variation  in  student  responses  points  toward  the  diversity  and  complexity  in  their  perspectives. 
Students  from  each  class  all  have  different  ideas  about  their  experiences  in  their  new  classroom  environment.  They 
all,  however,  enter  the  classroom  with  an  understanding  of  the  way  a traditional  class  works  and  quickly  confront 
the  differences.  Students  expect  the  classroom  to  look  a certain  way;  they  expect  to  focus  on  a teacher  standing  at 
the  front  of  their  classroom;  they  expect  to  have  easy  means  to  speak  and  be  heard.  When  these  culturally  based 
expectations  are  not  met,  students  are  often  left  with  a sense  of  anxiety  and  feelings  of  uncertainty.  We  found  that 
complex  technologically  mediated  interactions  between  learners,  instructors,  and  content  forced  the  re-construction 
of  the  way  participants  had  previously  “done  class.”  Learners,  caught  between  the  old  way  and  the  new  way,  moved 
through  a liminal  state  (Turner  1969)  to  find  a balance  that  provided  enough  comfort  with  the  environment  so  that 
distractions  were  minimized  and  learning  maximized.  The  student  need  indicated  in  this  paper  is  the  need  to  be 
guided  through  this  liminal  state,  the  need  to  be  helped  through  the  adjustment  phase.  In  such  a context,  the  new 
role  for  the  instructor  is  as  a guide. 

Instructors  that  we  observed  to  be  successful  guides  used  various  strategies  to  speed  up  and  enhance 
students’  adaptation  to  the  new  classroom.  Students  found  balance  more  quickly  when  instructors  recognized, 
articulated,  and  showed  sympathy  for  the  “awkward,”  “uncomfortable,”  “surprising,”  “overwhelming”  state  of  the 
learner.  When  instructors  talked  to  their  students  in  an  “up  front,”  “understanding,”  “sensitive,”  (from  student 
transcripts)  manner  about  the  strangeness  of  having  class  in  the  new  environment,  students  were  more  responsive  to 
instructions  on  how  to  use  the  technology  to  accomplish  class  goals.  When  learners  were  allowed  opportunities  to 
express  their  feelings  about  the  environment  without  fear  of  repercussions,  they  appeared  to  overcome  the  added 
burden  of  “hiding  [their]  anxieties”  about  to  taking  a distance  class  and  became  more  “willing”  to  work  with  the 
new  technologically-mediated  environment.  The  instructor  who  provided  students  the  opportunity  to  “vent”  through 
both  class  discussions  and  online  threaded  discussions  achieved  the  highest  rating  for  classroom  interaction  of  all  the 
instructors  considered  in  this  study  and  did  so  in  the  least  amount  of  time. 

The  style,  manner,  and  attitude  an  instructor  established  the  first  night  of  class  was  also  a key  factor  in  how 
long  it  took  students  to  learn  the  technological  system  and  for  the  ease  and  speed  with  which  they  learned  to 
function  effectively.  First  class  meetings  rated  highly  by  students  were  those  that  were  characterized  as  “fun,” 
“calm,”  and  “slow  enough  to  understand.”  Instructors  who,  in  early  classes,  laid  ground  rules  for  communication, 
clearly  and  completely  explained  the  technology  of  the  classroom,  gave  students  an  opportunity  to  use  the  system  in 
a non-threatening  manner,  and  provided  a thorough  picture  of  what  would  be  expected  throughout  the  semester  built 
bridges  between  participants  and  the  technology.  Once  the  bridge  was  built  and  the  connection  made,  the 
environment  felt  “safer”  and  interaction  took  place  more  easily. 

These  are  a few  of  the  insights  and  strategies  that  have  emerged  from  our  analysis  so  far.  We  expect  that 
our  continued  analysis  will  produce  many  more. 
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Abstract 

Within  the  scope  of  the  development  of  a new  multimedia  education  concept  for  the  teaching 
profession  “Computer  Science”,  the  strengths  and  weaknesses  need  to  be  constituted  already 
during  the  development-phase  and  weaknesses  need  to  be  eliminated  if  necessary.  Therefore, 
the  concept  is  examined  within  the  formative  evaluation  on  its  tangibility  and  effectiveness. 
The  newly  developed  education  concept,  which  was  developed  at  the  Gerhard  Mercator 
University  Duisburg  under  usage  of  new  media,  aims  at  creating  new  possibilities  for 
differentiation  and  decentralized  learning  by  a multimedia  teaching  and  learning  concept.  The 
didactical  new  conception  of  the  course  is  supposed  to  take  the  students  different  learning 
prerequisites  into  account.  Thus,  procurement  methods  are  striven  for  that  enable  individual 
preparation,  consultation  and  practice  dependent  on  the  students  educational  background. 


Introduction 

The  teaching  profession  with  the  vocational  field  “Computer  Science”  at  the  Gerhard  Mercator  University 
Duisburg  can  be  attended  by  students  with  different  main  subjects.  Students  can  combine  the  specialized 
professional  field  “Computer  Science”  with  different  main  subjects,  c.g.  mechanical  engineering,  chemistry 
technique  or  textile  and  clothing  technique.  So  a teaching  concept  had  to  be  developed  due  to  the  different  bases 
in  the  information  technical  education  of  the  different  courses.  It  has  to  consider  the  heterogeneous 
foreknowledge  of  the  students  and  offer  according  intensive  procurement  methods. 

The  developed  concept  is  used  in  the  basic  course  “Fundamentals  of  Computer  Science”  as  first  trial  phase.  The 
course  covers  topics  such  as  the  Boolean  algebra  or  switching  algebra,  the  technical  implementation  of  logical 
operations,  which  are  based  on  the  Boolean  algebra  and  finally  the  fundamental  digital  circuits  and  systems  as 
well  as  simple  digital  computers,  which  are  based  on  these  operations. 

The  procurement  of  contents  and  the  conversion  of  the  quoted  objective  requires  the  completion  of  the  course 
forms  so  far  by  means  of  a suitable  concept  on  computer  base  and  the  transfer  by  means  of  multimedia 
technique.  The  interactive,  multimedia  teaching  and  learning  software  is  set  up  in  such  a way  that  the  different 
learning  methods  of  the  students  are  taken  into  account.  Therefore,  different  points  of  entry  are  available  for  the 
software. 

One  entry  point  runs  over  the  content  structure,  here  the  student  can  choose  his  learning  path  in  accordance  with 
the  content  of  the  lecture.  The  course  so  far  included  exercises  for  practicing  and  applying  the  taught  methods. 
This  exercise  part  is  also  integrated  in  a suitable  form  into  the  interactive  learning  concept.  The  second  entry 
point  clarifies  the  interaction  between  the  acquired  knowledge  and  the  latter  practice  on  the  basis  of  a descriptive 
task.  Thus,  a learning  goal  oriented  or  res.  a problem  based  learning  is  enabled.  The  student  receives  an  course  of 
action.  A detailed  description  of  the  concept  is  given  in  (Schwarz,  Hunger,  Werner,  1 999). 


Initial  Considerations  for  Evaluation 

Sources  of  information  that  are  available  for  a study  group  are  on  the  one  hand  conservative  sources  such  as 
books,  scripts  and  patterns,  the  instructor  and  laboratory/experiment  device.  Additionally,  fellow  students  also 
count  as  information  source,  since  notes,  sources  of  information  and  interpretations  can  be  exchanged  with  them. 
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New  media  can  take  the  place  of  an  information  source  (book)  and  the  place  of  an  experiment  device 
(simulations).  It  can  also  partly  take  over  the  role  of  a teacher,  by  answering  standard  questions  and  checking 
exercise  results,  as  far  as  the  exercises  are  machine  controllable.  Furthermore,  new  media  creates  more  extensive 
communication  possibilities  (Dweck,  1988). 

The  entire  focus  during  the  introduction  of  new  media  in  complex,  problem  based  learning  situations  is 
determined  and  already  distributed  on  different  learning  tasks.  The  attention  can  not  be  increased  by  the 
introduction  of  new  media!  In  the  case  of  new  channels  being  available,  the  individual  must  redistribute  its 
bandwidth  and  thus  its  attention  on  the  available  channels.  A rearrangement  takes  place  on  expense  of  other 
channels.  Refocusing  might  lead  to  a favoring  of  channels  due  to  improved  accessibility  of  information  through 
new  media,  which  enables  individuals  to  receive  information  more  easily  and  thus  the  attention  on  other 
channels  is  encouraged.  We  can  speak  of  a positive  modification  of  the  social  behavior  induced  by  introducing 
new  media,  if  both  possibilities  deal  with  the  channels  which  the  individual  use  to  interact  with  other  members 
of  the  group. 


Formative  Evaluation 

The  concept  is  evaluated  in  several  steps.  In  the  first  step,  the  teaching  and  learning  processes  are  evaluated  by 
observing  and  interviewing  the  participants.  The  experiences  of  students  from  different  subgroups  are  of 
importance  and  thus  raised  in  particular.  The  members  of  the  group  are  interviewed  by  using  a questionnaire. 
This  questionnaire  ranks  important  data  through  accordingly  formulated  questions  on  socio-metric  matters  for 
the  4 partitions  of  the  theory  model  (economy  of  attention,  social  behavior,  motivation  control,  accessibility  of 
information).  The  behavior  and  movements  of  the  user  is  recorded  at  the  same  time.  Thus,  it  is  to  be  determined, 
whether  the  mapped  mental  model  is  applicable  on  the  learning  group  and  whether  weaknesses  in  the  navigation 
and  comprehensibility  can  be  discovered.  The  results  flow  directly  into  the  development.  Furthermore,  an  expert 
workshop  is  set  up  in  order  to  determine  suitable  scenarios  of  different  emphasis.  The  expert  workshop  primarily 
consists  of  professors  of  the  respective  field. 

The  evaluation  leads  to  the  following  results: 

70  participants  were  interviewed,  among  them  where  25%  that  use  MODULO  regularly.  The  participants  hardly 
use  MODULO  for  stringent  reading,  but  rather  for  the  direct  search  of  information.  This  functions  well  to 
outstanding;  students  indicated  that  finding  information  under  usage  of  MODULO  is  substantially  better.  The 
links  between  exercise  and  lecture  were  considered  as  helpful.  The  complex  scenarios,  which  are  supposed  to 
clarify  the  application  of  knowledge  in  vocational  practice,  are  considered  less  helpful  for  understanding  the 
material.  Most  students  rather  find  the  scenarios  helpful  at  ranking  the  material  into  vocational  practice. 

In  a further  step,  a summative  evaluation  will  take  place  after  final  completion  in  order  to  elevate  the  acquired 
knowledge  of  numerous  years  and  using  it  for  advancements  of  the  concept. 


Summary  and  Prospect 

In  this  article  a new  concept  for  supporting  the  theory  of  the  teaching  profession  course  ’’Computer  Science”  and 
an  appending  formative  evaluation  was  presented.  The  concept  aims  at  media-technical  editing  of  the 
disposition,  the  development  of  a training  concept  and  the  development  and  implementation  of  a WEB-database 
with  dynamic  access.  This  concept  holds  the  advantage  of  being  able  to  go  into  different  needs,  previous 
knowledge  and  learning  methods  of  students,  due  its  modular  structure.  It  enables  a self-controlled  learning, 
promotes  media  competence  and  offers  demanding  possibilities  of  acting. 

The  first  evaluation  which  accompanied  the  development  of  the  system  had  positive  results  and  constructive 
criticism.  The  entire  system  is  to  be  completed  by  September  2000,  so  that  the  summative  evaluation  can  be 
started  in  the  winter  semester  2000/01. 
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NS  A PARTNERSHIP  WITH  CAL  STATE  L.A.:  COURSEWARE 
CONVERSION  AND  EVALUATION  PROJECT 


Abstract:  CSULA  was  charged  with  assisting  the  NS  A in  the  research,  development, 
documentation  and  the  necessary  expertise  to  convert  a traditional  face-to-face  platform 
course  to  a multimedia  web-based  course.  We  converted  a platform-based  course  taught  by  the 
NSA  called  "EEO  100:  Equal  Employment  Opportunity  Law  and  Diversity  Training  for 
Selection  Boards."  To  accomplish  the  task,  an  interdisciplinary  team  of  graduate  and 
undergraduate  students  from  Instructional  Technology,  Art  Education,  Computer  Science,  and 
Design  collaborated  on  producing  the  website. 


Constructivism 

Recent  studies  in  education  have  promoted  the  constructivist  approach  to  learning  “...where  students 
develop  their  knowledge  through  team  collaboration,  discuss  different  interpretations  of  a problem,  and 
negotiate  and  synthesize  ideas  drawing  from  various  disciplines”  (Boyer  & Semrau,  1995,  14).  Technology  can 
play  a significant  role  in  this  process  by  having  students  become  more  proactive  in  their  own  learning  and  more 
independent  in  their  search  for  knowledge.  Students  using  technology  have  access  to  ongoing  real  world 
problems,  on-line  dialog  with  scientists,  and  access  to  the  latest  technological  advances.  Ultimately,  students 
will  be  engaged  in  their  learning,  see  relevance  in  it,  and  take  on  an  ownership  in  it.  To  meet  the  needs  and 
interests  of  students,  education  must  emphasize  problem  solving  and  application  of  information,  instead  of 
memorization  of  facts.  Roberts  et  al  state,  “...students  need  to  be  able  to  find  and  use  relevant  information,  share 
and  discuss  data  and  ideas,  and  collaborate  on  problem  solving”  (1990,  1 16). 

Year  One  of  Project 

July  of  1998,  California  State  University,  Los  Angeles  (CSULA)  began  a 3-year  project  sponsored  by  the 
National  Security  Agency  (NSA)  entitled  “NSA  Partnership  with  Cal  State  L.A.:  Courseware  Conversion  and 
Evaluation  Project.” 

CSULA  was  charged  with  assisting  the  NSA  in  the  research,  development,  documentation  and  the 
necessary  expertise  to  convert  a traditional  face-to-face  course  to  a multimedia  web-based  course.  CSULA 
students  were  selected  to  work  on  the  project  with  a focus  on  collaborative  group  work  and  a constructivist 
approach  to  learning.  An  interdisciplinary  team  of  graduate  and  undergraduate  students  from  Instructional 
Technology,  Art  Education,  Computer  Science,  and  Design  were  brought  together  under  the  direction  of  two 
faculty  members. 

In  this  project  we  are  converting  to  the  web  a platform-based  course  taught  by  the  NSA  called  “EEO  100: 
Equal  Employment  Opportunity  Law  and  Diversity  Training  for  Selection  Boards.”  EEO  100  is  directed  to 
employees  who  serve  or  who  may  serve  on  boards,  panels,  or  committees  charged  with  the  responsibility  of 
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rating  workers  for  selection  purposes.  The  course  is  designed  to  provide  participants  with  an  understanding  of 
the  extent  to  which  their  rating  activities  are  subject  to  applicable  civil  rights  laws,  and  to  appreciate  the  value 
and  contributions  of  every  candidate.  Students  after  taking  this  course  will  be  able  to: 

1 . Understand  the  requirement  for  EEO  1 00 

2.  Understand  legal  rights  and  responsibilities  for  fair  and  equal  treatment  in  the  decision-making  process 

3.  Gain  a common  understanding  of  “diversity” 

4.  Recognize  “blind  spots”  and  conditioning  from  society,  and  how  these  factors  impact  decisions 

5.  Provide  a hands-on  opportunity  to  improve  the  decision-making  process 

6.  Appreciate  the  value  and  contributions  of  every  individual 

Training  was  provided  to  the  CSULA  student  team  on  the  basics  of  html,  Adobe  Photoshop,  Premiere,  and 
Pagemill  in  order  for  them  to  be  able  to  undertake  the  task  of  courseware  conversion  to  the  web.  They  were  also 
trained  on  the  use  of  WebCT  for  developing  the  website,  uploading  of  web  pages  to  website,  and  designing  of 
web  pages.  . WebCT  is  an  online  Internet  authoring  tool  for  web-based  courses.  The  students  were  guided 
regarding  the  overall  layout  and  screen  designs.  For  a foundation  in  working  collaboratively  in  building  a 
website  the  students  first  designed  a demonstration  site  on  streaming  media — RealAudio  and  RealVideo.  The 
demonstration  site  was  developed  for  sharing  our  research  and  for  communication  among  the  students  and 
project  faculty.  Focus  was  placed  on  developing  an  educational  website  demonstrating  various  applications  of 
streaming  video  and  audio  in  education  as  well  as  being  a source  of  research  data  and  instruction  for  creating 
streaming  video  and  audio  for  web-based  training. 

The  students  investigated  the  hardware  and  software  requirements  for  streaming  media,  selected  examples 
of  streaming  video,  and  html  coded  web  pages  documenting  their  gathered  data  about  streaming  media  and  its 
role  in  web-based  training.  The  WebCT  course  site  also  offered  additional  built-in  communication  features 
including  e-mail,  chat  rooms,  a bulletin  board,  and  white  board.  Students  used  the  bulletin  board  to  provide  help 
to  each  other  and  to  assess  each  other's  web  pages. 

For  the  foundational  first  part  of  our  collaborative  work,  the  students  researched  and  structured  their 
content  on  streaming  media  into  the  following  sub-areas: 

1 . Introduction  to  Streaming  Media 

2.  How  Does  It  Work— Streaming  Video? 

3.  How  Does  It  Work— Streaming  Audio? 

4.  Aesthetics  of  Streaming  Video 

5.  Aesthetics  of  Streaming  Audio 

6.  Educational  Applications  of  Streaming  Media 

7.  Our  Examples  of  Streaming  Video 

8.  Project  Report  1998-99 

9.  About  Us 

Each  student  was  charged  with  the  design,  structure,  research,  and  development  of  one  content  sub-area. 

Year  Two  of  Project 

During  the  fall,  1999  we  moved  onto  converting  the  EEO  100  course  content  into  a web-based  course. 
We  followed  the  pattern  established  in  the  prior  year  where  each  student  was  assigned  full  responsibility  for  the 
research,  design,  development,  and  coding  of  a topic.  The  EEO  100  content  was  subdivided  into  the  following 
main  topic  areas:  Current  requirements,  additional  requirements,  definitions  of  discrimination,  disparate  impact, 
harassment,  affirmative  action  review,  and  tools.  Each  student  studied  and  learned  the  EEO  1 00  course  content 
and  was  then  individually  tested  on  it.  In  the  process  of  understanding  the  content,  they  realized  that  it  would  be 
necessary  to  rewrite  and  revise  the  content,  as  well  as  integrate  multimedia  to  accommodate  a web-based 
learning  environment.  Next  they  html  coded  the  EEO  100  course  content  and  thus  begun  the  creation  of  the 
web-based  course.  One  of  the  students  developed  the  screen  design  incorporating  American  Disabilities 
Association  (ADA)  guidelines. 


They  developed  new  test  items  for  the  content.  They  researched  websites  and  selected  appropriate  ones 
related  to  the  content  and  created  external  links  to  these  websites.  We  designed  draft  layout  for  structuring  the 
menu  and  content.  And,  we  investigated,  selected,  and  adapted  a quiz  format  for  use  in  this  website  to  set  up 
pop  quizzes  that  were  integrated  within  content  areas. 

In  the  winter  of  2000,  the  students  analyzed  a video  that  was  a tape  of  an  original  face-to-face 
traditional  approach  to  teaching  the  class,  which  was  provided  by  the  NSA.  They  made  a list  of  the  video  clips 
matching  the  content  they  were  responsible  for.  Before  we  started  digitizing  the  content,  we  inspected  the 
length  and  filesize  of  each  video  clip  to  determine  how  to  reduce  the  size  of  very  large  clips.  We  came  up  with 
several  solutions  which  included  digitizing  only  the  audio  for  certain  clips  and  accompanying  the  audio  with 
still  diagrams  of  key  points  and  cutting  up  the  longer  clips  in  to  shorter  segments.  Next  the  students  captured 
the  video  using  Adobe  Premiere  and  the  Osprey  100  video  card  in  a Dell  Precision  401  computer.  They  edited 
down  the  clips  using  Adobe  Premiere.  The  audio  was  captured  and  edited  with  SoundForge.  Next,  the  clips 
were  streamed  with  RealProducer  and  html  coded  into  their  relevant  web  pages  of  content. 

To  polish  off  the  interface  design,  we  brought  on  an  art  student  who  was  charged  with  developing  an 
easy  to  use  menu  of  topics  and  navigational  system. 

In  the  spring  of  2000,  it  is  our  intent  to  synthesize  various  course  components  (content,  video,  test 
items)  into  a final  course  website  and  to  design  visual  learning  aids  (diagrams,  charts,  and  pictures)  and  tools 
(glossary,  law  books,  and  court  cases)  to  enhance  learning  of  the  course  content.  We  are  preparing  to  design  and 
incorporate  simulations  into  the  website.  NSA  has  been  encouraging  and  supportive  of  this  idea  as  an  extended 
project. 

Conclusion 

The  new  web-based  version  of  this  course  will  make  the  content  accessible  to  more  NSA  employees 
located  in  various  places  throughout  the  world.  Plus,  the  online  version  will  include  interactive  pop  quizzes 
with  immediate  scoring  and  feedback  as  well  as  streaming  media  lectures  to  complement  the  course  content, 
which  is  presented  as  text  on  the  web  pages. 

Through  this  project  the  students  became  constructively  involved  in  their  own  learning  and  have 
acquired  in-depth  experiences  in  collaborative  learning.  The  students  became  empowered  to  be  creators  of  their 
own  curricular  materials  and  web  pages  instead  of  being  passive  viewers  of  others’.  In  this  project  all  of 
Bloom’s  higher  level  taxonomies  are  implemented.  Students  are  analyzing  websites,  synthesizing  criteria  that 
they  have  researched,  applying  their  book  readings,  comparing  and  contrasting  their  criteria,  designing  and 
producing  their  own  web  pages,  and  more. 
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Abstract:  The  convergence  of  modem  data  acquisition  technologies  with  the  Web’s  interactivity, 
connectivity  and  multimedia  capabilities  presents  an  exceptional  opportunity  for  distance  education  in  the 
physical  sciences.  Web-mediated  access  and  control  of  laboratory  equipment  can  improve  utilization  of 
expensive  and  specialized  instruments,  facilitate  collaborative  data  sharing  and  analysis,  and  provide 
essential  practical  experience  in  physical  science  courses  delivered  at  a distance. 

This  paper  describes  a remotely  controlled  experiment  for  determining  the  rates  of  fast  chemical  reactions. 
The  experiment  is  not  a simulation;  it  involves  actual  equipment  controlled  in  real  time  from  remote  locations 
on  the  Web.  The  experiment  is  the  first  in  a series  designed  to  provide  a pedagogical ly  on-line  laboratory 
experience  for  Web-delivered  general  chemistry  courses.  Students  use  the  experiment’s  Web  interface  to 
collect  data,  to  obtain  interactive  technical  support  and  background  information,  and  to  display  and  analyze 
results.  Each  experiment  is  designed  to  encourage  sharing  of  data  and  collaboration  with  users  at  other 
institutions,  providing  students  with  a valuable  first  look  at  work  in  a distributed  laboratory  environment. 


Introduction 

Chemistry  is  an  experimental  subject.  Chemists  construct  knowledge  by  systematically  examining  quantitative  data 
for  patterns.  Patterns  suggest  hypotheses,  which  in  turn  form  the  basis  for  theories.  Theories  are  then  critically 
evaluated  in  the  light  of  new  experimental  data.  This  is  the  only  sanctioned  approach  for  progress  in  science. 
Pedagogy  that  is  not  founded  on  the  interplay  between  theory  and  experiment  poorly  serves  students  by  not 
involving  them  in  the  essential  process  of  science. 

The  experimental  nature  of  chemistry  presents  severe  challenges  for  teaching  the  subject  at  a distance.  Many  current 
online  chemistiy  courses  either  have  no  laboratory  experience  at  all  or  use  simple  kits  or  'kitchen  chemistry' 
experiments  that  can  be  performed  with  household  materials.  Obtaining  accurate,  quantitative  data  from  which 
relevant  and  interesting  conclusions  can  be  drawn  is  difficult  with  such  simple  equipment. 

Online  chemistry  courses  often  make  heavy  use  of  simulations.  While  simulations  can  be  quite  effective,  they  are 
theoretical  constructs  that  cannot  substitute  for  practical  experience.  For  example,  students  regarded  the  output  of 
simulations  in  a statistical  analysis  experiment  at  Oxford  University  as  "pretend"  data  (Cartwright,  1998).  They 
sometimes  failed  to  carry  over  the  lessons  learned  from  analyzing  simulated  data  to  data  collected  in  actual 
experiments.  Concerns  about  the  pedagogical  quality  of  the  simulations  were  addressed  by  the  development  of  a 
simple,  economical  "optical  rig"  (Cartwright,  1999),  which  allows  students  to  collect  and  analyze  actual  data  over 
the  Internet. 

This  project  addresses  the  problem  of  practical  experience  in  Web-delivered  courses  by  providing  students  with 
remote  access  and  control  of  real  equipment.  A remote-control  reaction  kinetics  experiment  suitable  for  chemistry 
students  at  the  undergraduate  or  advanced  secondary  level  is  described.  The  apparatus  monitors  the  course  of  a fast 
chemical  reaction  spectrophotometrically  using  a simple  continuous- flow  method.  Students  can  use  the  data  they 
collect  to  determine  the  rate  laws  for  the  reaction  under  a variety  of  conditions.  By  collaboratively  analyzing  the 
results  of  a large  group  of  experiments  performed  by  peers,  students  can  propose  mechanisms  for  the  reaction.  While 
the  experiment  is  conceptually  quite  simple,  it  involves  specialized  apparatus  that  is  unavailable  in  most 
undergraduate  laboratories.  Rapid  data  acquisition  can  allow  many  users  to  perform  experiments  in  real  time,  and 


nearly  spontaneously. 


Method 


The  experiment  is  an  adaptation  of  a classic  continuous  flow  method  for  studying  the  rates  of  fast  reactions  (Dalziel, 
1953;  Roughton  & Chance,  1963).  The  apparatus  can  be  used  to  determine  the  rate  of  any  sufficiently  fast  reaction 
that  produces  a colored  product,  although  practical  considerations  limit  the  choice  of  reaction(Shoemaker,  1974). 
Pilot  investigations  studied  the  formation  of  FeSCN^^  (an  orange  complex  that  absorbs  at  blue  light  at  455  nm)  and 
“Prussian  blue”,  an  intensely  colored  complex  used  in  blueprinting,  paints  and  inks  (Sharpe,  1976). 


A schematic  diagram  of  the  apparatus  is  shown  in  (Fig.  1). 

continuous  flow  reactor  and  sensor  array 


high  sensilivity  light  sensors  w/  0-10  V analog  output 

ii^i  JL  rSa  rl^  JL 


fiberoptic  canneUion  to  manochromalic  light  source 
2 mm  ID  digitally  ronlrolied 


, glass  capillary 
flow  tube 


solenoid  valve 


schematic  for  fast  reaction  kinetics  experiment 


reservoir  reservoir  gmj  monochromator 


Reactant  solutions  are  forced  from 
large  reservoirs  into  a T-shaped 
chamber,  where  they  mix  completely 
in  less  than  1 millisecond.  The 
reacting  mixture  then  travels  down  a 
long  capillary  tube.  The  product  of 
the  reaction  is  intensely  colored  and 
strongly  absorbs  a characteristic 
wavelength  of  light.  Product 
concentration  rises  as  the  mixture 
moves  along  the  tube.  Seven  light 
sensors  placed  at  fixed  positions 
along  the  tube  monitor  the  increase 
in  concentration.  Monochromatic 
light  is  passed  through  the  capillary 
at  each  position  using  a fiber-optic 
cable.  High  sensitivity  photocells 
placed  on  the  opposite  side  of  the 
tube  record  the  intensity  of 
transmitted  light  at  each  point;  the 
intensity  drop  caused  by  absorption 
of  the  light  by  the  complex  is  simply 
related  to  the  concentration.  A flow 
rate  sensor  allows  the  position  of 
each  of  the  spectrophotometers  to  be 
associated  with  a reaction  time. 


Data  from  the  sensors  is  acquired  by  “sanzone”,  a 500-MHz  PC  equipped  with  a National  Instruments  data 
acquisition  card.  A LabView  program  collects,  labels,  and  writes  the  absorbance,  temperature,  and  flow  rate  data  to 
an  "outgoing"  queue.  The  program  then  checks  for  requests  in  an  "incoming"  queue.  If  one  exists,  a new  run  is 
started.  Otherwise,  the  system  is  flushed  and  the  valves  are  closed  to  avoid  unnecessary  waste  of  reagent  solutions. 


A Webcam  collects  pictures  of  the  apparatus  during  the  run.  The  camera  view  includes  the  reservoir  manometer  and 
a digital  thermometer,  so  it  is  used  for  data  collection  as  well  as  for  helping  students  monitor  the  progress  of  the 
reaction.  The  Webcam  is  attached  to  a second  PC  to  avoid  contention. 


A Unix  Web  server  (Marie)  mounts  Sanzone’s  hard  drive  across  the  local  network.  Perl  scripts  running  on  Marie 
fetch  data  from  the  outgoing  queue,  interpret  it,  and  prepare  the  results  for  display. 

Students  perform  the  experiments  via  a "smart"  instrument  panel  served  from  Marie,  displayed  in  a standard 
browser.  The  instrument  panel  recreates  the  actual  look  and  feel  of  the  remote  experiment.  Operators  immediately 
see  the  results  of  their  actions  via  the  Webcam,  as  well  as  on  a rich  data  display.  The  panel's  panic  button  launches  a 
conferencing  connection  with  the  system’s  caretaker  via  Netmeeting  during  working  hours  and  by  email  at  other 
times.  The  panel  monitors  and  interprets  student  activity  to  diagnose  and  correct  conceptual  problems  and  to  prevent 
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actions  that  could  damage  or  tie  up  the  apparatus  for  long  periods  of  time. 

The  instrument  panel  front  end  is  implemented  in  JavaScript  and  Macromedia  Flash.  When  students  submit  a run 
from  the  instrument  panel,  backend  Perl  scripts  label  and  process  the  request  and  write  it  to  an  "incoming”  queue  on 
Sanzone  and  the  expected  time  of  completion  for  the  job  is  displayed  on  the  panel.  Jobs  complete  within  a few 
seconds  when  the  incoming  queue  is  empty.  Queues  are  intelligently  managed  by  consecutively  scheduling  jobs 
with  similar  operating  parameters  and  by  assigning  lower  priority  to  non-local  IP  addresses  (to  prevent  ‘walk-in’ 
users  from  monopolizing  the  experiment). 

Future  versions  of  the  experiment  will  allow  students  to  manipulate  additional  parameters,  such  as  reagent 
concentrations,  pH,  temperature,  ionic  strength,  and  flow  rate.  Data  from  different  groups  can  then  be  pooled  and 
analyzed.  Students  will  then  use  these  results  to  evaluate  proposed  mechanisms  for  the  reaction.  Further 
developments  will  be  outlined  on  the  project  home  page  at  http://antoine.frostburg.edu/chem/senese/inetchemset. 
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Abstract:  Distance-Learning  based  on  WWW  has  become  a trend  for  the  development  of 
education.  But  the  distance-learning  site  based  on  WWW  is  static,  the  designers  of  the 
courses  don’t  know  whether  the  design  of  courses  is  rational  or  consistent  with  the  teaching 
law.  While  there  is  a lot  of  students’  information  accumulated  in  the  web  site,  and  the 
information  is  useful  for  our  course  designers.  In  this  study,  we  present  a distance-learning 
model  based  on  Web  Mining,  which  can  take  advantage  of  those  students’  information 
accumulated  in  the  web  site. 


1. introduction 

Distance-learning  based  on  WWW  has  become  a trend  for  the  development  of  education.  The  available 
distance-learning  model  based  on  WWW  is  discussed  in  (Xuejun  Li  1999).  But  recently  the  distance-learning  site 
based  on  WWW  is  static,  and  the  designers  of  the  courses  don’t  know  whether  the  design  of  courses  is  rational 
and  consistent  with  the  teaching  law.  While  there  is  a lot  of  students’  information  accumulated  in  the  web  sites, 
such  as  students’  access  patterns  and  registration  and  communication  data  information,  and  these  information  is 
useful  for  our  course  designers  and  teachers. 

In  this  study,  we  present  a distance-learning  model  based  on  Web  Mining,  which  can  take  full 
advantage  of  those  students’  information  accumulated  in  the  web  sites. 


2. A  Distance-Learning  Based  on  Web  Mining 

The  model  integrates  the  Web  Mining  technology  into  the  available  model  base  on  WWW.  It’s 
architecture  is  shown  (Fig.l).  In  this  model,  we  add  the  Web  Mining  module  into  the  available  model.  The  Web 
Mining  model  is  located  in  the  server-side  and  performs  in  the  server-side. 

According  to  our  special  system  for  distance  learning,  the  Web  Mining  process  as  shown  in  (Fig.4). 
The  Web  Mining  module  has  three  phases(Robert  cooley  et  al.  1998):  Preprocessing(Yilin  yang  et  al.  1999), 
mining  process  and  pattern  analysis. 

3. ConcIusion 

our  model  has  following  points:  The  course  designers  can  reconstruct  a Web  sites  in  order  to  better 
serve  the  needs  of  students  of  a site  according  to  these  information  such  as  the  students’  access  patterns  and 
page  access  frequency  statistics;  it  can  provide  individualised  page  or  different  course  content  for  individual 
student;  and  it  can  provide  teachers  the  students’  feedback,  and  teachers  can  redesign  their  teaching  plans,too. 


Client-side 


Server-side 

Figure  1.  A Distance-Learning  Model  Based  on  Web  Mining 
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Introduction 

The  aim  of  this  paper  is  to  illustrate  and  demonstrate  the  value  of  a user-centered  method  for  CALL 
systems  development.  We  accept  the  argument  of  Hubbard  (1996),  Levy  (1997)  and  other  the  deficiencies  of 
current  CALL  system  can  be  attributed  to  the  lack  of  a systematic  methodological  theory  guiding  their  design, 
Accordingly,  a CALL  design  method  has  been  developed  which  draws  on  recent  developments  in  the  field  of 
human-computer  interaction  regarding  scenario-based  design  and  prototyping  in  relate  to  the  user-centered 
design  of  computer-based  learning  material. 


Scenario-Based  Prototyping 

The  main  features  of  the  method  include  the  use  of  scenarios  to  identify  user  requirements  (Carroll, 
1995),  the  development  of  prototypes  embodying  key  design  issues,  and  a series  of  formative  workshops  to 
evaluate  the  prototypes.  The  overall  results  confirm  the  general  validity  of  the  user-centered,  scenario-based 
methodological  approach. 

There  were  three  cycles  of  prototyping  process  in  this  research.  Each  cycle  was  based  on  a set  of  User 
Satisfaction  Scenarios.  The  scenarios  were  used  to  build  prototypes  in  two  different  ways:  user  dissatisfaction 
scenarios  were  used  to  define  existing  problems  that  have  been  found  from  the  current  CALL  products;  and 
user  satisfaction  scenarios  were  used  to  clarify  possible  problems  that  could  be  found  in  the  CALL  product  to 
be  built.  A template  for  scenarios  is  proposed  for  the  educational  environment.  The  template  include  the  user 
environment  (i.e.,  age,  school,  class,  user's  conception  toward  CALL  system,  motivation  of  using  CALL 
software,  software  type),  learning  environment  (i.e.,  classroom,  self-studying,  group- studying),  and  narrative 
user  interaction  scenario  to  define  features  user  satisfied/dissatisfied.  Overall,  this  paper  is  to  suggest  the 
method  of  use  of  scenarios  explicitly  for  designing  CALL  systems,  not  to  show  the  results  of  prototypes  which 
have  been  constructed  and  evaluated 

Developing  prototypes  is  an  integral  part  of  iterative  user-centered  design  process  because  it  enables 
designers  to  try  out  their  ideas  with  real  users  and  to  gather  feedback.  There  were  three  major  cycles  of 
prototyping  process  with  different  prototypes  embodied  different  design  issues,  which  is  based  on  user- 
satisfaction  scenarios.  The  main  design  issue  of  the  first  prototype  systems  was  to  find  how  user-control  could 
affect  to  achieve  high  quality  CALL  systems  in  terms  of  both  educational  and  technical  value.  Two 
comparative  systems  were  built  for  this  initial  stage:  one  was  a high  user-control  (learner-based)  system,  and 
the  other  was  a low  user-control  (teacher-oriented)  system.  According  to  the  feedback  from  a series  of 
formative  user  workshops,  it  has  been  found  that  the  effectiveness  of  user-control  were  dependent  on  learners' 
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experience  in  CALL  systems  and  their  learning  environments.  Based  on  the  feedback  from  the  first 
workshops,  the  user  satisfaction  scenarios  were  revised  to  find  the  design  issues  for  the  second  prototype. 

The  second  prototype  system  was  a hyperlink  system  between  three  different  systems:  Tutorial  system 
for  listening  skills  with  low  user^control,  Main  Lesson  system  for  practicing  listening  skills  with  high  user- 
control  for  experienced  users,  and  a sub-system  called  Speaking  Dictionary  to  provide  learners  linguistic  help. 
The  user  response  about  the  hyperlink  system  was  that  it  would  be  better  to  have  a different  user-control  by 
learning  stages  concerning  user  experiences.  Users  also  showed  positive  opinions  about  the  user  satisfaction 
scenarios  because  they  agreed  that  scenarios  make  it  easy  to  visualize  the  real-world  environment  and  to 
comprehend  users'  perspective  toward  CALL  systems.  On  the  other  hand,  users  responded  that  learners  need 
more  support  to  control  their  learning  process  more  effectively. 

Therefore,  the  third  prototype  constructed  was  focused  to  investigate  the  effectiveness  of  Teaching 
Agent  to  support  learners'  learning  process  with  respects  of  HCI  aspects  and  educational  achievement.  At  this 
stage,  we  argue  that  TA  should  have  domain  knowledge  (i.e.  English),  Training  knowledge  (i.e.  listening 
skills),  and  Specific  pupil  knowledge.  Here,  the  specific  pupil  knowledge  is  the  most  significant  to  advise 
learners  what  to  do  next.  While  the  knowledge  of  domain  and  training  are  static  and  independent  on  student  or 
teacher,  the  knowledge  of  specific  pupil  depends  on  followings:  how  well  is  the  learner  doing?;  what  listening 
skills  have  been  used?;  what  personality  does  the  learner  have?;  and  what  is  the  learning  goals?  The  TA 
system  called  Magic  Mirror  has  been  developed  to  apply  those  aspects  based  on  the  revised  user  satisfaction 
scenarios,  in  which  learner  personality  and  learning  goal  of  specific  pupil  knowledge  were  not  considered.  The 
result  from  the  third  workshop  with  teachers  was  positive  towards  the  concept  of  TA  in  general,  but  some 
teachers  questioned  how  TA  could  act  as  a substitute  for  a teacher.  On  the  other  hand,  learners  showed  more 
positive  attitude  towards  the  TA  because  they  compared  the  TA  with  Help  functions  in  other  CALL  system, 
not  with  their  teachers.  The  conclusion  is  that  TA  could  make  CALL  systems  more  effective  if  all  the  aspects 
of  the  specific  pupil  knowledge  are  regarded  on  the  development. 


Conclusion 

The  essential  elements  of  this  user-centered  method  for  designing  CALL  systems  are  scenarios, 
prototypes,  and  formative  workshops.  Prototyping  helps  designers  to  make  user-oriented  decisions  by  eliciting 
information  from  the  formative  evaluations  of  users,  and  scenarios  are  useful  to  articulating  new  design 
concepts  like  TA.  In  HCI  practice,  scenarios  of  use  are  increasingly  being  used  to  help  set  overall  design 
objectives  for  guiding  various  aspects  of  the  design  process.  Carroll  (1995)  argues  that  if  we  believe  the 
scenario  approach  to  be  a valuable  enhancement  of  current  development  practice,  we  must  ask  how  the  use  of 
scenarios  in  system  development  can  be  evoked,  supported,  and  brought  to  bear  on  various  activities  within  the 
system  development  cycle.  Overall,  this  user-centered  method  is  very  effective  regarding  the  following  aspects. 
Firstly,  Scenarios  are  used  as  a bridge  between  designers  and  users  to  define  learners'  or  teachers'  needs  and 
learning  environment  in  order  to  achieve  learning  goals.  Secondly,  prototypes  are  used  to  apply  design  issues 
and  to  demonstrate  the  expected  learning  environment  concerning  pedagogical  value.  Finally,  user  workshops 
are  held  to  investigate  user  satisfaction  to  evaluate  design  issues  and  principles  in  terms  of  both  technical  and 
educational  value. 
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Abstract:  Design  of  electronic  books  should  take  into  account  the  diversity  of  learning  styles, 
intelligences  and  preferences  of  each  user.  This  project  main  aim  is  to  provide  high  level  of 
electronic  book  interaction  and  flexibility  by  matching  different  categories  of  electronic  pages 
(graphic  page,  talking  page,  hypermedia  page  and  web  page)  to  the  different  learning  styles.  Each 
page  exhibits  programs  and  activities  that  attempt  to  cater  for  each  learning  style.  The  same  content 
in  each  page  will  be  presented  in  four  different  presentation  modes.  In  addition  to  the  technical 
system  development,  a conceptual  model  of  children's  electronic  textbook  will  also  be  proposed. 
This  model  is  based  on  Howard  Gardner's  theory  of  Multiple  Intelligences  and  Philip  Barker's 
generalization  of  electronic  books  model. 


Introduction  and  definition  of  electronic  book 

Educational  environment  stresses  the  importance  of  reading  materials:  textbooks,  reference  books,  encyclopaedias, 
magazines  and  newspapers  as  media  of  knowledge  carriers.  Hence  converting  printed  publications  especially 
textbook  into  interactive  electronic  form  should  prove  to  be  extremely  useful  and  helpful.  There  are  various 
definitions  of  electronic  book  (e-book).  Bonime  et  al.  (1998)  described  any  kind  of  information  ranging  from  a CD- 
ROM  title  to  an  online  interactive  database  including  collection  of  web  pages  as  electronic  book.  Barker  (1998) 
defined  e-book  as  essentially  a computer-based  information  storage  that  embeds  a book  metaphor.  More  precisely 
(Barker,  1999),  e-book  is  usually  considered  to  be  composed  of  a collection  of  reactive  pages  of  electronic 
information  that  are  organised  in  a thematic  way  and  that  exhibit  many  of  the  characteristic  features  and  properties 
of  a conventional  book.  Barker  categorises  e-book  into  ten  types  depending  on  the  types  of  information  embedded: 
textbooks,  picture  books,  talking  books,  moving  picture  books,  multimedia  books,  polymedia  books,  hypermedia 
books,  intelligent  books,  telemedia  books  and  cyberbooks.  While  Barker  categorises  ten  types  of  e-book,  Landoni  et 
al.  (1993)  classified  e-book  according  to  three  different  criteria:  books  which  are  portable  (e.g.  personal  digital 
assistant),  books  which  preserve  the  logical  structure  (i.e.  chapters,  sections  and  subsections)  and  lastly,  books  that 
support  both  the  physical  and  logical  aspects  of  a paper  book.  In  context  of  this  project,  definition  of  an  e-book  will 
be  defined  when  the  study  is  completed. 


Research  Question 

Children  learn  in  different  ways.  There  are  many  theories  on  how  people  learn  and  one  example  is  the  Multiple 
Intelligences  (MI)  theory  proposed  by  Howard  Gardner  (1993).  This  theory  states  that  there  exist  seven  intelligences 
(thus  seven  learning  styles):  verbal/linguistic,  logical-mathematical,  visual/spatial,  bodily-kinesthetic,  musical, 
interpersonal  and  intrapersonal  (to  date,  two  more  intelligences  have  been  added).  Based  on  this  theory,  Armstrong 
(1994)  wrote  a book  on  how  to  apply  this  theory  into  the  classroom  environment.  He  identifies  that  each  child  could 
learn  in  any  one  of  these  ways  or  through  a combination  of  several  ways.  With  these  different  learning  styles,  design 
of  e-books  should  take  into  account  these  diversity  and  attitudes  of  each  user  particularly  young  children.  Thus,  one 
question  can  be  extracted:  how  to  give  young  children  better  flexibility  so  as  to  cater  their  differences? 


Proposed  Solution:  Giving  E-Book  Flexibility 


There  are  many  methods  to  give  flexibility  in  e-books.  One  example  is  by  providing  tools  or  facilities,  which  can  be 
used  by  users  while  consulting  the  e-book.  Landoni  et.  al  (1993)  in  her  project  provided  users  with  facilities  such  as 
personalisation  (highlighting,  bookmarking,  note  taking),  searching  through  TOC  table  of  content  and  index,  and 
printing.  Other  examples  include  providing  system  adaptivity,  supporting  for  group-based  learning,  considering  the 
likelihood  that  teachers  will  use  the  program  in  class  and  including  different  levels  of  contents  in  the  books. 

It  is  the  intention  of  this  on  going  project  to  suggest  that  another  method  to  achieve  flexibility  is  through  flexible 
presentation  modes  that  include  programs  and  activities  which  match  each  learning  style.  The  following  gives  few 
examples  on  how  to  match  MI  theory  to  the  existing  e-book  model: 

(a)  Verbal/linguistic  - Is  matched  with  e-book  concept  of  hypermedia  page  through  programs  such  as  letting  users 
create  essays,  read  aloud  and  include  storytelling. 

(b)  Visual/spatial  - Is  matched  with  e-book  concept  of  graphic  page  through  programs  such  as  letting  users  draw 
and  paint  and  allowing  users  to  read  and  see  information  as  graphics,  maps,  charts  and  diagrams. 

(c)  Musical  - Is  matched  with  e-book  concept  of  talking  page  through  programs  such  as  combining  stories  with 
songs  and  letting  users  sing  along. 

(d)  Interpersonal  - Is  matched  with  e-book  concept  of  web  page  by  including  programs,  which  need  two  or  more 
users  at  the  same  time  and  group  project  or  discussion. 

Proposed  electronic  book  model 

Page  = Contents  + [Object  A + Object  B + Object  C + Object  D] 

Object  A = Graphic  page  + {Program  G1  | Program  G2  | ..  .|  Program  Gn} 

Object  B = Talking  page  + (Program  T1  | Program  T2  | . . .|  Program  Tn} 

Object  C = Hypermedia  page  + (Program  HI  | Program  H2  | . . .|  Program  Hn} 

Object  D = Web  page  + {Program  W1  | Program  W2  | . . .|  Program  Wn} 


Conclusion 

In  conclusion,  our  e-book  shall  exhibit  four  different  concepts  and  include  a number  of  presentation  modes  and 
styles,  in  order  to  provide  readers  with  a high  level  of  flexibility.  Users  can  choose  between  the  four  different 
presentation  modes  when  he/she  opens  any  page.  Upon  completion,  this  project  will  be  used  to  prove  these 
assumptions: 

HI  :lt  is  possible  to  address  diverse  learning  styles  by  providing  users  with  flexible  presentation  modes  of  each  page. 
H2:These  flexible  presentation  modes  are  probably  more  appropriate  to  use  and  as  such  give  e-book  high  level  of 
flexibility. 

H3:It  is  then  possible  to  suggest  that  due  to  this  high  level  of  e-book  flexibility,  users  are  more  likely  to  prefer  this 
type  of  e-book. 

H4:If  H1,H2  and  H3  are  proven,  then  we  could  postulate  that  an  additional  desirable  feature  for  e-book  would  be  to 
present  contents  by  mixing  many  presentation  modes. 
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Abstract:  The  Internet  is  making  inroads  into  almost  all  quarters  of  our  society.  Education 
is  no  exception  to  that,  nor  should  it.  Exciting  new  ways  of  building  content,  of  delivering 
it,  of  interacting  with  our  students,  indeed  of  managing  our  educational  ’industry’  can  be 
seen  to  evolve.  Even  if  one  may  not  wholeheartedly  support  all  these  innovations, 
particularly  for  all  those  with  an  inquisitive  mind  it  is  inspiring  to  see  education  being 
rethought  in  so  many  and  fundamental  ways.  Or  is  it? 

The  Internet  revolution  clearly  is  a technological  revolution.  It  pushes  developments  in  various  fields,  education 
being  no  exception  to  that.  However,  are  the  new  technologies  used  to  innovate  education,  that  is,  to  bring  about 
a genuine  transformation;  or  do  they  just  substitute  older  technologies  for  new  ones? 

An  educational  model  that  is  widely  implemented  these  days  - although  it  doesn’t  alway  go  by  this  name  - is  that 
of  the  extended  classroom.  As  the  name  suggests,  an  extended  classroom  takes  the  ordinary  classroom  as  its 
starting  point  and  extends  it,  mainly  spatially.  So  the  dominant  didactic  model  here  still  is  a teacher  who 
addresses  a group  of  students;  students  and  teacher  are  simultaneously  present;  the  teacher  takes  the  lead; 
students  may  ask  questions  but  those  only  serve  to  clarify  the  exposition  of  the  teacher;  students  seldom  interfere 
with  the  choice  of  content,  which  is  the  teacher’s  exclusive  realm;  students  hardly  ever  have  a say  in  the  didactics 
used,  this  also  is  the  teacher's  prerogative.  In  the  extended  classroom,  teacher  and  students  may  be  in  different 
locations,  the  distance  being  bridged  by  television,  videoconferencing,  audiographics  and  similar  technologies. 
Put  extremely,  with  the  extended  classroom  one  aims  to  mimick  a regular  classroom  to  the  largest  possible 
extent.  Hence  the  call  for  ever  larger  bandwith,  for  systems  that  allow  for  'natural’  interactions  between  the 
teacher  and  the  remote  classroom,  etc. 

However  cleverly  designed  our  communication  systems,  however  large  our  bandwidth,  extended  classrooms  will 
always  be  second  best,  a substitution  for  the  real  thing,  only  useful  as  a means  to  reach  out  to  the  remote  student, 
perhaps  to  serve  larger  groups  of  students  and  achieve  some  economy  of  scale.  Lecture  notes  that  are  being  put 
on  the  web  as  a service  to  the  students,  or  e-mail  and  newsgroups  that  deliver  additional  support  only  serve  to 
underline  the  main  conclusion:  in  the  extended  classroom  technology  is  used  to  substitute  or  augment  existing 
teaching  modalities. 

Obviously,  there  is  nothing  inherently  wrong  with  this.  Technology  is  a means  to  an  end  and  should  be  used 
accordingly,  that  is,  how  we  see  fit.  However,  if  it  is  our  lack  of  imagination  or  if  it  is  the  rigidity  of  our 
institutions  that  prevent  us  from  using  technology  in  more  creative  ways,  then  we  should  not  rest  content.  We 
should  reexamine  our  educational  practices  and  ask  ourselves  to  what  extent  they  were  limited  by  existing 
technologies  and  to  what  extent  the  new  technologies  empower  us  to  innovate.  And  indeed,  it  is  our  claim  that 
we  only  stand  at  the  beginnings  of  truly  innovative  educational  practices  and  genuinely  new  didactic  principles. 

Distributed  learning  is  a term  that  is  often  used  in  this  context.  In  distributed  learning  systems,  there  are  no 
classrooms  anymore,  no  teachers  offering  students  prepackaged  chunks  of  education  carefully  aranged  to 
conform  to  the  needs  and  capacities  of  the  average  student.  Students  now  occupy  center  stage,  assembling  and 
arranging  content  that  fits  their  specific  needs;  they  take  it  in  when  and  where  it  suits  them  best,  perhaps  in  a 
didactic  style  that  optimally  matches  their  learning  style.  The  role  of  the  teacher  now  is  not  communicating 
knowledge,  but  to  make  it  available,  to  empower  students  via  intakes  and  assessments  and  help  them  assimilate 
knowledge,  to  offer  custom  support.  In  distributed  learning,  technological  innovation  is  not  the  push  behind 
educational  innovation,  as  is  the  case  in  the  extended  classroom.  Rather,  the  converse  holds  true,  educational 
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innovation  pushing  technology.  And  indeed,  distributed  learning  requires  specific  technologies  that  extend 
beyond  the  mere  facilitation  of  communication.  Technologies  are  needed  for  creating  learner  centered  content, 
for  the  delivery  of  such  content,  for  flexible  intake  and  assessment  procedures,  for  portfolio  management,  etc.  In 
our  presentation  we  will  discuss  these  technologies  at  some  length. 


O 
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This  paper  describes  a study  aimed  at  investigating  the  impact  of  a technology-enabled  training  program  for  Project 
Executives  (PE's)  in  the  IBM  Corporation.  The  program  is  focused  on  a mentoring  approach  to  learning  whereby 
PE’s  considered  to  be  experts  in  specific  skills  coach  less  experienced  colleagues  in  the  development  of  critical 
skills.  Since  job  demands  prevent  ongoing  face-to-face  mentoring  relationships,  technology  which  enables 
collaboration  at  a distance  is  implemented  to  support  the  mentoring  relationship.  Of  interest  in  this  study  is  the 
effectiveness  of  instructional  design  approach,  the  impact  of  the  technology-enabled  learning  environment  on 
sustaining  mentoring  relationships,  and  the  nature  of  online  discussions  between  experts  and  students. 

IBM  PE's  are  responsible  for  managing  large  information  technology  contracts.  These  professionals  deliver  on 
commitments  to  customers,  ensure  customer  satisfaction,  secure  business  opportunities,  and  mentor  other  project 
executives.  Their  role  of  mentor  is  critical  to  the  professional  development  of  less  experienced  colleagues.  While 
traditional  face-to-face  mentoring  is  desired  and  encouraged,  job  demands  often  prevent  mentors  and  students  from 
sustaining  the  type  of  relationships  needed  for  the  transfer  of  critical  skills. 

In  the  current  study,  a combination  of  face-to-face  instruction  and  a technology  enabled  learning  environment  is 
used  to  develop  and  sustain  mentoring  relationships  among  expert  PE’s  and  less  experienced  colleagues.  The  face- 
to-face  instruction  includes  a two  day  workshop  consisting  of  five  sessions  led  by  the  expert  PE’s.  The  purpose  of 
the  workshop  sessions  is  to  1)  initiate  a mentoring  relationship  among  expert  PE’s  and  a group  of  colleagues  and  2) 
to  begin  the  mentoring  process  which  includes  the  transfer  of  a specific  set  of  problem-solving  skills. 

At  the  conclusion  of  the  workshop  participants  are  introduced  to  e-mentor,  a Lotus  Notes  discussion  database 
which  can  be  accessed  using  a Lotus  Notes  Client  or  a Web  browser.  The  discussion  database  is  used  to  provide  an 
online  learning  environment  in  which  the  mentor-protege  relationship  is  preserved.  Mentors  post  mini-lessons  on 
topics  relevant  to  the  development  of  complex  problem-solving  skills  needed  in  customer  situations.  Participants 
respond  to  the  lessons  in  discussion  format,  post  questions  and  collaborate  with  one  another  in  solving  customer 
problems.  Mentors  are  available  to  assist  participants  with  appropriate  strategies  for  addressing  customer  problems. 

The  instructional  design  approach  used  in  the  workshop  and  online  is  based  on  cognitive  apprenticeship,  an 
instructional  approach  developed  by  Collins,  Brown  and  Newman  (1989).  Cognitive  apprenticeship  focuses  on  the 
development  of  higher  level  thinking  skills  such  as  problem-solving.  Because  of  its  focus  on  the  teaching  of 
cognitive  and  metacognitive  knowledge,  cognitive  apprenticeship  may  be  a more  appropriate  instructional  design 
model  for  online  mentoring  than  more  traditional  instructional  designs  such  as  a lecture  approach.  Whereas  text  and 
lecture-based  instructional  design  models  focus  on  teaching  concepts,  facts,  and  procedures  in  non-situated 
environments,  cognitive  apprenticeship  focuses  on  teaching  the  processes  and  strategies  used  in  expertise  and  how 
this  knowledge  is  used  to  solve  real-world  problems.  Snyder  (2000)  demonstrated  the  effectiveness  of  cognitive 
apprenticeship  as  an  instructional  design  approach  for  teaching  technical  skills  in  an  online  environment. 

Collins,  Brown,  and  Newman  (1989)  and  Collins  (1991)  identify  four  aspects  of  cognitive  apprenticeship 
learning:  content,  instructional  methods,  sequencing  of  instruction,  and  sociology.  Content  refers  to  the 
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different  types  of  knowledge  required  for  expertise  and  includes  domain  knowledge  and  strategic  knowledge. 

Domain  knowledge  consists  of  concepts,  facts,  and  procedures;  strategic  knowledge  refers  to 

knowledge  which  underlies  an  expert’s  ability  to  make  use  of  concepts,  facts,  and  procedures  to  solve  problems. 

Instructional  methods  are  the  learning  activities  used  during  instruction  to  help  students  construct,  use,  manage, 
and  acquire  new  knowledge  (Collins,  Brown,  & Newman,  1989).  Seven  instructional  methods  are  recommended. 
These  methods  include  teachers  "modeling"  skills,  "coaching"  learners  as  they  attempt  to  mimic  expert  skills, 
providing  "scaffolding"  in  the  form  of  support  for  learners,  and  gradually  "fading"  support,  as  learners  become  more 
proficient.  Learners  are  also  encouraged  to  "articulate"  their  knowledge,  "reflect"  on  their  problem-solving 
processes,  and  "explore"  new  approaches  to  problem  solve  on  their  own. 

Sequencing  involves  the  staging  of  learning  whereby  tasks  are  presented  in  increasing  complexity  and  diversity 
so  that  students  develop  a broad  understanding  of  the  domain  of  expertise.  Sociology  deals  with  the  authenticity  of 
the  learning  environment.  Technological,  social,  time,  and  motivational  characteristics  of  real-world  situations  are 
designed  into  the  learning  environment  so  that  students  will  learn  when,  where,  and  how  the  knowledge  applies  to 
other  situations  (Collins,  Brown,  & Newman,  1989). 

The  PE  workshop  in  this  study  and  the  e-mentor  database  are  designed  to  incorporate  the  four  aspects  of 
cognitive  apprenticeship  learning.  In  the  workshop,  learning  takes  place  in  the  context  of  solving  a variety  of  real- 
world  complex  customer  problems.  Mentors  begin  sessions  with  an  introduction  a complex  customer  problem. 

Facts  and  concepts  about  the  customer  environment  are  presented  (domain  knowledge)  and  followed  by  modeling  of 
the  strategies  and  processes  used  to  solve  the  problem.  During  this  activity,  mentors  use  think-aloud  protocols  to 
model  the  strategic  thinking  required  to  solve  the  problem.  Next,  tips  and  techniques  (scaffolds)  for  remembering 
the  processes  and  strategies  are  provided  to  students.  Students  are  then  assigned  to  small  teams  of  six  and  work 
together  to  develop  a solution  to  an  existing  customer  problem.  The  problems  used  in  this  exercise  are  real-world 
and  are  brought  to  the  workshop  by  students.  Students  are  required  to  articulate  the  processes  and  strategies  used  to 
address  the  customer  problem  and  then  briefly  explore  alternative  approaches  to  problem-solving.  At  the  conclusion 
of  this  exercise,  mentors  model  their  thinking  with  respect  to  solving  the  team  problems.  When  the  two-day 
workshop  ends,  the  mentor-  protege  relationship  is  maintained  using  the  Lotus  Notes  discussion  database.  Workshop 
participants  use  the  database  to  request  help  from  colleagues  and  mentors  regarding  problems  experienced  in  the 
workplace.  Mentors  use  of  the  cognitive  apprenticeship  method  when  responding  in  the  online  environment. 

The  results  of  this  study  to  date  have  been  successful  in  that  this  combination  of  face-to-face  and  online 
mentoring  has  enabled  a community  of  IT  executives  who  are  geographically  dispersed  to  form  working 
relationships  and  share  knowledge  on  a regular  basis.  This  model  has  also  provided  participants  with  an  environment 
for  continuous  learning  on  a just-in-time  basis,  convenient  to  their  demanding  work  schedules.  The  instructional 
design  approach  has  been  notably  successful  in  providing  an  effective  way  for  PE’s  to  learn  new  skills.  Participants 
in  the  workshop  quickly  adopted  the  habit  of  thinking  about  problem-solving  from  a strategic  perspective  (that  is, 
identifying  the  processes  and  strategies  needed  to  solve  customer  problems  and  using  reflection  and  exploration 
techniques  before  identifying  a solution).  This  learning  approach  has  also  provided  a common  ground  for  PE’s  to 
work  together  to  solve  customer  problems.  Results  of  a survey  administered  to  participants  of  the  workshop 
indicated  high  ratings  with  respect  to  the  methods  fostering  working  relationships  among  PE’s,  the  value  of  the 
content  of  the  workshop,  its  structure  (instructional  design  approach)  and  the  value  of  the  e-mentor  database  in 
providing  a continuous  learning  environment  which  facilitates  collaboration. 
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NJ:  Erlbaum  Associates. 

Snyder,  K.,  (2000),  Asynchronous  Learning  Networks  and  Cognitive  Apprenticeship.  A Model  for  Teaching 
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Abstract:  This  work- in-progress  paper  describes  a pedagogical  approach  to  designing  an  online 
collaborative  learning  environment  that  scaffolds  learners  in  becoming  better  collaborative  learners. 
This  environment  is  based  on  Collaborative  Problem  Solving  (CPS)  instructional  theory.  The  paper 
will  overview  the  need  for  the  environment,  and  the  design  principles  used  to  design  and  develop  it. 


Need 


As  online  education  becomes  more  prevalent  as  a system  for  distance  education,  a need  arises  for  an  online 
structure  that  supports  instructors  and  learners  engaging  in  small  group  collaborative  problem  solving  or  project 
work  at  a distance  from  one  another.  Particularly,  there  is  an  increasing  need  for  online  learning  environments  that 
support  effective,  efficient  collaborative  processes,  such  as  those  used  by  expert  group  problem  solvers. 

When  online  environments  impose  restrictive  structures  or  do  not  account  fully  for  these  processes,  they 
may  hinder  the  flow  of  learning,  and  curtail  effective,  efficient  problem  solving.  Most  online  environments  that 
attempt  to  use  collaboration  as  a learning  strategy  incorporate  a smorgasbord  of  communications  technologies  that 
are  not  structured  according  to  research  into  the  most  effective  ways  to  scaffold  effective  collaborative  interactions. 
Furthermore,  the  applications  included  in  these  systems  (such  as  email,  threaded  discussions,  and  file  storage)  are 
generally  lower-level  communication  tools  as  opposed  to  tools  that  could  support  robust  collaboration.  Because 
they  do  not  use  a guiding  theory  that  is  pedagogically  based,  the  resulting  interaction  often  do  not  adequately 
address  the  social,  intellectual,  and  logistical  complexity  of  effective  collaborative  learning.  This  is  partially  due  to 
the  fact  that  until  recently,  no  comprehesive  theory  capable  of  support  effective  online  collaboration  existed. 

Specifically,  the  weakness  of  these  approaches  is  their  primary  reliance  on  a document-based  paradigm 
(such  as  Lotus  QuickPlace  or  Hyperwave),  conferencing-based  paradigm  (such  as  Microsoft  NetMeeting  or  Lotus 
Sametime),  or  room-based  paradigm  (such  as  BioMOO  or  Diversity  University)  (Miao,  Fleschutz,  & Zentel  1999). 
For  the  design  to  fully  support  a cohesive  collaborative  learning  approach,  the  strengths  of  each  of  these  approaches 
must  be  synthesized  into  a context-based  system  that  allows  for  a more  flexible  combination  of  people,  places,  tools, 
documents,  and  interactions  than  these  other  types  of  systems  afford,  thus  providing  robust  support  for  the 
communication,  collaboration,  and  coordination  necessary  to  learn  effectively  as  a small  group. 


Proposed  Solution 

We  propose  to  design,  develop,  implement,  and  evaluate  an  online  collaborative  learning  environment  that 
both  supports  pedagogically  sound  instruction  and  provides  appropriate  scaffolding  for  moving  learners  from  novice 
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to  expert  collaborative  learners.  It  will  be  based  on  Collaborative  Problem  Solving  (CPS),  an  instructional  theory 
designed  for  effective,  efficient  small-group  problem  solving  (Nelson  1999).  This  theory  outlines  a detailed 
collaborative  learning  process  derived  from  a series  of  carefully  documented  case  studies  of  expert  learning  groups 
integrated  with  findings  from  current  research  in  areas  such  as  small  group  dynamics,  cooperative  learning,  and 
problem-based  learning.  The  goal  of  this  theory  is  to  provide  step-by-step  scaffolding  to  both  instructors  and 
learners  that  will  help  learners  make  the  difficult  transition  from  novice  to  expert  collaborative  learners. 

Unlike  existing  tools  that  merely  provide  a collaboration  platform,  this  online  system  will  be  designed  to 
serve  as  a robust  teaching  and  learning  environment  that  will  provide  guidance  for  both  instructors  (in  designing 
their  instruction  and  in  appropriately  monitoring,  supporting,  and  assessing  the  learners’  progress)  and  learners  (in 
developing  effective  collaborative  skills  while  learning  at  a distance).  This  learning  environment  will  be  capable  of 
scaling  to  any  number  of  small  groups  and  will  support  instruction  in  a variety  of  disciplines  desiring  to  engage 
learners  in  project-  or  problem-based  learning  around  complex,  real-world  problems.  Its  structure  will  afford 
support  for  every  phase  of  effective  small  group  collaboration,  from  building  readiness  to  providing  closure,  by 
providing  all  the  collaborative  learning  spaces,  tools,  and  resources  necessary  to  do  so,  such  as  “just-in-time 
instruction”  for  learners  as  they  progress  through  the  given  project  or  problem  scenario. 


Project  Goals 

The  goal  is  to  create  an  online  learning  environment  that  has  the  following  features  and  characteristics: 

• Teaches  and  supports  the  nine  phases,  not  just  isolated  activities,  of  Nelson’s  effective  collaborative 
problem  solving  process,  which  includes:  building  readiness,  forming  and  norming  the  groups,  determining 
a preliminary  problem  definition,  defining  and  assigning  roles,  engaging  in  an  iterative  collaborative 
problem-solving  process,  finalizing  the  solution  or  project,  synthesizing  and  reflecting,  assessing  products 
and  processes,  and  providing  closure  to  the  learning  experience, 

• Employs  proven  pedagogical  theory  and  arrays  tools  around  this  theoretical  structure, 

• Responds  to  the  needs  of  the  users  by  being  flexible,  customizable,  and  reconfigurable, 

• Runs  entirely  on  open  standards-based  Internet  protocols, 

• Enables  distributed  groups  to  learn  and  work  together  at  least  as  effectively  as  co-located  groups, 

• Provides  guidance  for  both  instructors  and  learners  in  effective  collaborative  learning,  and 

• Archives  the  work  of  small  groups  for  both  the  group’s  reference  and  for  research  analysis. 


Project  Design 

Our  design  approach  is  to  translate  each  element  of  CPS  theory  into  a corresponding  aspect  or  structure  of 
the  environment,  such  that  it  both  implicitly  and  explicitly  supports  effective,  efficient  collaborative  problem  solving 
philosophy  and  methods.  From  this  blueprint,  we  will  work  with  a software  designer  to  create  a data  model.  From 
the  data  model,  the  software  designers  will  create  a database-driven,  internet-based  collaborative  environment  that 
will  scale  to  any  number  of  small  teams.  Simultaneously,  we  will  begin  a paper  prototyping  phase  to  determine  the 
initial  visual  design,  working  with  a target  user  pool  of  instructors  and  students  at  the  two  distance  education 
entities.  We  will  then  employ  a rapid  prototyping  method  to  improve  our  visual  design  and  address  the  usability 
issues  of  instructors  and  learners  testing  alpha  versions  of  the  system.  Once  we  have  a beta  quality  environment,  we 
will  conduct  pilot  studies  with  instructors  and  learners  in  diverse  content  areas  and  geographical  locations.  Finally, 
we  will  evaluate  case  studies  of  the  system  fully  implemented  in  a variety  of  learning,  work,  and  research  settings 
and  use  the  data  to  refine  a final  version  that  we  will  release  for  use  on  the  local  servers  of  any  interested  partners. 
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Abstract:  This  paper  describes  the  implementation  of  a pilot  project  involving  the  formation 
of  networked  technology  learning  communities  designed  to  nurture  the  growth  and  sharing  of 
expertise  in  meaningful  uses  and  integration  of  technologies  into  teaching  and  learning  among 
college  faculty  members,  student  teacher  candidates  and  classroom  teachers.  The  paper 
focuses  on  issues  related  to  the  establishment  of  technology  learning  communities  and 
includes  brief  studies  of  selected  members  of  the  learning  communities. 


Introduction 

Recent  studies  (Trotter,  1997;  U.S.  Department  of  Education,  1998)  indicate  that  a large  majority  of 
classroom  teachers  in  the  U.  S.  continue  to  feel  inadequately  prepared  to  integrate  technology  into  classroom 
instruction.  This  work-in-progress  paper  describes  the  implementation  of  a pilot  project  that  focuses  on  ways  to 
address  this  problem.  The  project  involves  the  formation  of  networked  technology  learning  communities 
designed  to  nurture  the  growth  and  sharing  of  technological  expertise  among  college  faculty  members,  teacher 
candidates  and  classroom  teachers  in  the  integration  of  technologies  into  teaching.  The  project  studies  potential 
for  such  communities  to  enlarge  the  number  of  technologically  proficient  current  and  future  teachers  by 
increasing  the  number  of  college  faculty  members  who  model  the  use  of  technologies  in  their  teaching,  the 
number  of  teacher  candidates  exposed  to  technology-proficient  college  and  classroom  teachers,  and  the  pool  of 
technologically  proficient  classroom  mentor  teachers.  Ultimately,  the  goal  for  this  project  is  to  support  a cycle 
of  increased  meaningful  integration  of  technology  into  teaching. 


Technology  Learning  Communities 

Technology  learning  communities  provide  “a  culture  of  adult  learning  and  mutual  support” 
(McKenzie,  1999,  p.  71)  that  enable  individuals  with  different  skill  levels  to  assist  each  other  in  the  learning  of 
ways  to  integrate  technologies  meaningfully  into  teaching.  Technology  learning  by  members  occurs  in  both 
informal  and  formal  settings,  with  support  and  expertise  provided  when  requested  by  members  of  the 
communities.  These  communities  sustain  the  different  learning  styles  of  the  members.  The  strength  of  a 
learning  community  lies  in  the  emphasis  on  members  teaching  each  other  what  needs  to  be  learned  and  at  a 
pace  that  meets  individual  needs  and  preferences  (McKenzie,  1999).  Emphasis  in  the  pilot  learning 
communities  is  on  supporting  projects  that  have  a direct  impact  on  teaching  and  learning. 

Inherent  in  the  formation  of  learning  communities  with  membership  from  diverse  groups,  i.e.  college 
faculty,  college  students,  and  classroom  teachers,  are  many  challenges.  If  a learning  community  is  to  be 
successful,  membership  must  be  “flat  and  non-hierarchical”  (McKenzie,  1999)  in  the  sense  that  all  members  are 
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equal  partners.  Ideally,  the  learning  communities  should  have  an  impact  on  education  that  radiates  beyond  the 
communities.  The  focus  of  each  member  of  a community  must  be  on  \vays  to  use  technology  to  improve 
teaching,  not  simply  on  the  development  of  individual  technological  expertise.  Members  of  a learning 
community  must  accept  the  responsibility  to  take  the  acquired  skills  and  knowledge  back  to  their  circle  of 
professionals.  Each  member  must  be  willing  to  build  on  acquired  skills  and  knowledge,  even  after  the  pilot 
project  ends. 

Each  ten  member  learning  community  in  this  project  is  composed  of  college  faculty  members,  teacher 
candidates,  and  preK-5  teachers,  with  varying  degrees  of  technological  expertise.  Criteria  for  membership  were 
established  prior  to  the  formation  of  the  learning  communities.  Student  members  were  selected  based  on 
meeting  those  criteria  and  documented  faculty  support  for  their  membership.  Faculty  members  and  K-5  teachers 
meeting  the  criteria  and  having  a range  of  technological  expertise  were  identified  and  invited  by  project  leaders 
to  participate  in  the  project.  The  members  of  the  current  learning  communities  appear  to  have  successfully 
established  a non-hierarchical  relationship  partly  due  to  the  selection  process.  Each  member  of  the  learning 
communities  is  equipped  with  a laptop  computer  and  Internet  access  to  facilitate  collaboration. 

In  the  initial  meeting  of  the  learning  communities,  members  were  surveyed  on  their  levels  of 
technology  expertise  and  discussed  their  learning  expectations.  This  information  assisted  the  coordinator  of  the 
learning  communities  in  planning  group  learning  sessions.  In  addition  to  these  face-to-face  meetings,  members 
of  the  communities  interact  using  email  and  a discussion  list.  The  discussion  list,  TechTalk,  provides  a 
mechanism  for  members  to  share  and  request  resources,  and  to  obtain  fast  response  to  technology- related 
inquiries.  With  support  from  learning  community  members,  the  college  faculty  members  are  increasing 
technology  use  in  the  courses  they  teach.  The  teacher  candidates  have  the  opportunity  to  assist  in  the 
implementation  of  technology-based  lessons,  to  design  lessons  and  materials  to  be  used  during  student  teaching 
with  preK-12  students,  and  to  work  with  the  college  faculty  members  as  they  integrate  technology  into  courses. 
The  classroom  teachers  are  sharing  and  expanding  their  expertise  in  integrating  technology  into  the  preK-5 
curriculum. 

Members  of  the  learning  communities  have  formed  partnerships  as  reflected  in  interviews  done  with 
one  student  and  one  faculty  member  in  the  communities.  Faculty  Member  A is  a beginning  technology  user, 
though  she  is  knowledgeable  about  the  Internet.  As  a college  history  professor,  she  wants  to  explore  the  use  of 
the  Internet  not  as  an  electronic  library  reflected  in  the  web  sites  that  she  includes  in  her  syllabus  bibliography, 
but  as  a vehicle  to  encourage  her  education  students  to  think  critically  about  events  in  history.  She  finds  the 
learning  communities  helpful  in  learning  about  social  studies  issues  important  to  elementary  teachers  but 
receives  the  most  help  from  one-on-one  tutoring  from  individual  members.  She  is  especially  appreciative  of  the 
help  from  Student  B,  an  above  average  technology  user,  who  acquired  many  of  her  skills  at  the  college.  In 
addition  to  tutoring  Faculty  Member  A,  Student  B is  helping  two  classroom  teachers  in  her  learning  community 
to  teach  their  students  HyperStudio.  She  finds  that  the  learning  communities  aid  her  goal  of  being  a teacher 
because  through  this  collaboration  she  is  spending  time  in  a classroom  setting  working  with  children. 

Evaluation  of  the  progress  being  made  by  members  of  the  learning  communities  in  meeting  the  initial 
goals  of  this  pilot  project  is  encouraging.  This  project  is  providing  the  groundwork  for  modification  and 
expansion  of  future  technology  learning  communities. 
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Four  Challenges  for  TeleTeachers  in  Rural  Schools 

Ken  Stevens,  Memorial  University  of  Newfoundland,  Canada 

The  introduction  of  teleleaming  in  schools  in  Canada,  as  in  other  developed  countries, 
has  been  particularly  noticeable  in  rural  areas  and  has  been  influenced  by  declining 
school  enrolments.  There  are  four  challenges  common  to  the  technological  enhancement 
of  rural  schools:  the  building  of  appropriate  infrastructures,  the  location  of  appropriate 
technologies,  the  development  of  pedagogy  for  teleteaching  and  the  need  for  on-going 
professional  development. 

Challenge  One:  Building  Appropriate  Structures  (Digital  Intranets) 

By  constructing  web-sites  both  teachers  and  students  in  a rural  school  can  enable  people 
located  almost  anywhere  to  connect  with  them  and  learn  about  their  programs  and 
activities.  In  Canada  regional  and  national  educational  networks  link  classes  in  schools 
on  dispersed  sites.  Stem-Net  in  Newfoundland  and  Labrador  in  eastern  Canada  is  the 
provincial  arm  of  the  federal  SchoolNet  System.  These  federated  electronic  structures 
enable  teachers  to  extend  the  curriculum  available  to  learners  in  dispersed  and,  often, 
very  isolated  sites.  Schools  in  rural  Canada  are  developing  open  and  flexible  academic 
and  administrative  structures  as  they  electronically  link  with  one  another.  These  new 
structures  exist  within  the  traditional  and,  by  comparison,  closed  school  systems  in  each 
school  district.  In  the  open  educational  structure  of  teleteaching  within  digital  intranets, 
constructed  to  link  schools  for  the  delivery  of  specific  courses  for  some  students, 
participating  institutions  academically  and  administratively  interface  for  that  part  of  the 
school  day  during  which  classes  are  being  taught.  This  is  a very  different  educational 
structure  from  the  traditional  and,  by  comparison,  closed  educational  environment  of  the 
autonomous  school  with  its  own  teachers  and  its  own  students.  In  Canadian  networks 
such  as  Stem-net,  each  site  becomes  an  inter-dependent  part  of  a virtual  school,  without 
which  it  could  not  provide  all  the  courses  that  local  people  are  increasingly  accessing. 

Challenge  Two:  Finding  Appropriate  Technology 

As  schools  become  teaching  and  learning  sites  within  digital  Intranets,  there  is  increasing 
demand  for  adequate  technological  support  and  co-ordination.  Few  teachers  have  the 
skills  to  provide  adequate  technological  support  for  the  increasing  numbers  of  computers 
and  computer  users  in  their  schools.  The  result  is  that  the  emerging  teaching  and  learning 
infrastructure  in  many  places  is  often  very  fragile.  An  essential  aspect  of  the  development 
of  open  electronic  classes  is  the  coordination  of  both  hardware  and  software  between 
schools.  Without  coordinated  technology,  schools  cannot  fully  participate  in  electronic 
networks.  However,  the  purchase  of  appropriate  hardware  and  software  is  a matter  of 
confusion  for  many  principals,  teachers  and  School  Boards  who  seek  support  and  advice. 
Many  rural  schools  with  open  electronic  classes  realize  that  the  successful  administration 
of  a network  requires  local  technical  support.  Unless  adequate  technical  support  systems 
can  be  established,  electronic  networked  classes  could  be  curtailed  by  teachers  who  could 
argue,  with  justification,  that  there  is  insufficient  back-up  to  justify  their  investment  in 
teleleaming.  The  need  to  have  people  in  schools,  or  available  to  schools,  to  maintain 
information  technology  hardware  and  software  is,  for  many  teachers,  a matter  of  growing 
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importance.  In  geographically  isolated  communities  the  assurance  of  adequate  technical 
backup  is,  however,  often  particularly  difficult  to  provide  for  teachers  and  principals.  In 
schools  in  Newfoundland  and  Labrador,  the  majority  of  which  are  designated  rural, 
telelearning  often  depends  on  the  goodwill  of  one  or  two  teachers  with  enough 
knowledge  to  keep  computers  going  when  they  mal- function. 

Challenge  Three:  Pedagogy  for  TeleTeaching 

In  the  changing  technological  environment,  teachers  often  have  a choice  of  teaching  face 
to  face  and,  for  part  of  a school  day,  providing  or  receiving  instruction  for  their  students 
at  a distance.  This  can  be  done  in  synchronous  or  asynchronous  mode,  with  a range  of 
technologies,  and  with  a range  of  learners.  The  use  of  information  technologies  in  schools 
has  been  vigorously  promoted  by  the  federal  agency.  Industry  Canada  (Information 
Highway  Advisory  Council,  1995;  1997).  Students  often  have  more  independence  in 
managing  their  learning  in  open  electronic  classes  but  most  have  to  be  assisted  by 
teachers  in  the  setting  of  goals,  the  meeting  of  deadlines  and  in  evaluating  their  progress 
(Stevens,  1996).  Teachers  are  effective  in  open  electronic  classes  if  they  can  be  flexible 
in  ways  that  they  allow  students  to  participate  in  on-line  lessons.  Strategies  and  protocols 
for  on-line  teaching  have  to  be  developed  between  participating  schools  if  all  students  are 
to  be  able  to  fully  participate.  The  introduction  of  a rural  school  to  an  open  electronic 
network  considerably  improves  its  resource  base  for  both  teachers  and  learners,  but  does 
not  solve  all  of  its  problems.  It  is  often  difficult  to  coordinate  the  timetables  of  networked 
schools  and  a considerable  measure  of  inter-institutional  and  intra-institutional 
cooperation  is  required  to  develop  detailed  and  effective  plans  for  collaboration.  The 
need  for  rural  networked  schools  to  have  a close  relationship  with  the  suppliers  of 
technology  has  become  increasingly  apparent  to  schools,  school  boards  and  to  technology 
retailers.  Rural  schools  with  teleieaming  infrastructures  require  expert  advice  and  support 
from  technology  suppliers  at  the  network  rather  than  at  the  individual  school  level.  There 
are  several  immediate  pedagogical  challenges  to  be  considered  for  effective  teaching  in  a 
digital  Intranet: 

1 . Teachers  have  to  learn  to  teach  from  one  site  to  another.  This  is  fundamental  to  the 
success  of  teleteaching.  Teaching  face  to  face  and  on-line  are  different  skills. 

2.  Teachers  have  to  learn  to  teach  collaboratively  with  colleagues  from  multiple  sites. 

3.  Teachers  have  to  judge  when  it  is  appropriate  to  teach  on-line  and  when  it  is 
appropriate  to  teach  students  in  traditional  face-to-face  ways.  These  judgements  have 
to  be  defended  on  the  basis  of  sound  pedagogy. 

Challenge  Four:  Professional  Development  for  TeleTeaching 

A difficulty  many  teachers  face  in  Canadian  schools  is  simply  finding  the  time  in  their 
busy  days  to  learn  about  information  and  communication  technologies.  It  is  impossible 
for  some  teachers  to  do  this,  let  alone  reflect  on  how  new  technologies  can  be  integrated 
into  their  classrooms  and  their  professional  lives.  There  is  not  only  a widespread  need  for 
professional  development  for  teleteaching;  there  is  a need  for  it  to  be  provided  on  a 
continuous  basis.  A second  challenge  facing  many  teachers  is  finding  effective  ways  of 
integrating  information  and  communication  technologies,  including  the  Internet,  into 
teaching  and  learning.  It  is  not  difficult  to  add  information  and  communication 
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technologies  to  any  classroom  but  it  requires  considerable  planning  to  integrate 
technological  developments  so  that  both  teaching  and  learning  are  enhanced.  A third 
challenge  for  the  professional  development  of  teachers  is  to  be  able  to  justify,  on  the  basis 
of  professional  judgements,  why  the  Internet  and  other  technologies  should  be  used  in 
teaching  and  learning  at  all. 

Conclusion 

The  introduction  of  inter-school  electronic  networks  has  added  a new  dimension  to  rural 
education  in  Canada  and  in  other  developed  countries.  In  Canada  and  in  many  other 
places,  teachers  and  researchers  are  seeking  appropriate  ways  of  using  existing  as  well  as 
emerging  technologies  to  enhance  teaching  and  learning.  The  teachers  and  researchers 
who  are  collaborating  in  the  development  of  new  electronic  structures  for  delivering 
education  to  dispersed,  rural  sites  in  Atlantic  Canada  are  very  conscious  of  being 
pioneers.  Both  realize  there  is  a lot  to  learn  to  about  both  teaching  and  learning  in  this 
new  environment. 

References 

Information  Highway  Advisory  Council  (1995)  The  Challenge  of  the  Information 
Highway,  Ottawa,  Industry  Canada 

Information  Highway  Advisory  Council  (1997)  Preparing  Canada  for  a Digital  World, 
Ottawa,  Industry  Canada 

Stevens,  K.J  (1996)  Teleleaming  and  New  Zealand  Schools  - Some  Implications  of  the 
Convergence  of  Information  and  Communication  Technologies  for  the 
Management  of  Education,  New  Zealand  Association  for  Research  in  Education, 
Nelson 


1751 


Group  interaction  as  a predictor  of  learning  effectiveness  in  a computer 
supported  collaborative  problem  solving 


Neli  Stoyanova 

Faculty  of  Educational  Science  and  Technology,  University  of  Twente 
P.O.Box  217 

AE  7500  Enschede,  The  Netherlands. 
stoyanovn@edte.utwente.nl 

Abstract:  The  paper  presents  an  experimental  validation  of  learning  effectiveness  of  three  different 
types  of  group  interaction  for  computer  mediated  collaborative  learning  and  problem  solving.  The 
assumption  underlying  this  research  is  that  the  form  in  which  knowledge  is  shared  in  collaborative 
learning  is  a substantial  for  cognitive  construction  and  reconstruction.  The  experiment  revealed  that 
the  learning  effectiveness  is  influenced  significantly  by  the  mode  of  group  interaction  that  depends 
on  the  extend  to  which  students  share  their  learning  not  only  as  results  but  also  as  a process  of 
knowledge  acquisition  and  creation  by  a direct  interaction. 

Theoretical  background  and  research  question 

Rapidly  changing  technologies  generate  new  requirements  for  creating  computer  environments  that  support  learning 
in  the  broad  context  of  global  networking  communities.  Different  theoretical  perspectives  emphasise  collaboration 
as  a successful  and  powerful  activity  for  learning  and  problem  solving. 

Collaboration  creates  an  extension  of  the  internal  cognition  of  the  personality  in  outside  world  (Perkins,  1993). 
According  Salomon  (1993)  the  individuals’  and  the  shared  group  cognitions  interact  with  one  another  in  a spiral-like 
fashion  whereby  individual  inputs,  through  students’  collaborative  activities,  affect  the  nature  of  the  joint,  shared 
system,  which  in  turn  affects  their  cognitions.  It  is  emphasised  that  reconstruction  of  individual  cognition  requires  a 
profound  and  mutual  understanding  of  collaborators’  perspectives  and  shared  interpretations  of  the  problem.  Only 
the  knowledge  that  is  meaningful  for  individuals  could  be  internalised. 

The  assumption  underlying  this  research  is  that  the  form  in  which  knowledge  is  shared  in  collaborative  learning  is  a 
substantial  for  cognitive  construction  and  reconstruction.  The  main  research  question  is:  Is  tke  mode  of  group 
interaction  a predictor  of  learning  effectiveness  in  a computer  based  collaborative  problem  solving  design? 


Collaborative  scenarios 

Three  scenarios  of  problem  solving  group  interaction  have  been  designed,  based  on  the  assumption  that  knowledge 

exists  in  three  forms:  as  individual  mental  constructs  and  precepts,  as  knowledge  in  action,  and  as  knowledge 

embodied  in  artefacts  (Hedlund  & Nonaka,  1994) 

1.  Distributed  interaction^  Group  members  work  autonomously  and  produce  intermediate  artefacts  embodying 
their  knowledge  and  visions,  which  are  passed  to  the  other  members.  This  circuit  is  repeated  until  all  group 
members  reach  a common  vision  of  the  problem.  The  process  of  knowledge  acquisition,  creation  and 
internalisation  is  individual. 

2.  Moderated  interaction^  Interaction  process  is  facilitated  by  a group  moderator  (the  role  is  taken  by  one  of  the 
group  members)  adjusting  individually  produced  artefacts  until  a common  group  vision  is  reached.  The 
representations  of  individual  cognitive  structures  are  not  directly  accessible  but  group  members  are  involved  in 
the  process  of  negotiation  of  meanings  and  ideas  that  take  place  between  them  and  the  group  moderator. 

3.  Shared  interaction.  Group  members  interact  directly  by  synchronous  activity  and  common  efforts  to  solve  the 
problem  as  a group.  They  share  their  knowledge  in  action.  Knowledge  is  communicated  in  the  process  of  its 
acquisition  and  creation  and  is  not  mediated  by  intermediate  artefacts.  Collaborative  actions  of  students  are  the 
individual  inputs  toward  shared  cognition. 

Experimental  validation 
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The  learning  effectiveness  of  the  three  modes  of  group  interaction  was  tested  experimentally.  The  experimental 
design  was  a random  assignment  pre-test  post  test  control  group.  The  independent  variable  was  the  mode  of  group 
interaction  with  three  levels  - ‘‘Distributed”  “Moderated”  and  “Shared”. 

The  dependent  variable  was  learning  effectiveness  operationalised  as  individual  learning  achievements  in  terms  of: 
fluency  (number  of  concepts  representing  individual  understanding  of  the  problem),  flexibility  (variety  of  concepts 
and  ideas),  knowledge  acquisition  (number  of  new  concepts  acquired),  individual-to-group  transfer  (individuals’ 
inputs  in  the  group  solution),  group-to- individual  transfer  (concepts  derived  from  the  group  solution), 
reconfiguration  (changes  in  the  concepts’  structure),  retention  (contra  indicated  by  the  reduced  concepts),  and 
creativity  (new  concepts  and  ideas  generated  individually  after  the  collaboration). 

Twenty-six  students  (6  groups)  of  University  ofTwente,  Faculty  of  Educational  Science  and  Technology,  enrolled 
in  “Linear  and  Hypermedia”  course,  were  selected  as  experimental  subjects  and  were  randomly  assigned  to  the  three 
collaborative  scenarios.  The  group  task  was  formulated  as  an  open-ended  problem.  Concept  mapping  technique  was 
chosen  for  representation  and  communication  of  knowledge  because  of  its  effectiveness  as  a tool  for  extemalisation 
and  representation  of  cognition.  (Stoyanov  & Kommers,  1999).  Data  was  analysed  applying  one-way  ANOVA 
procedure  by  SPSS  9.0  statistical  package. 

Results  and  Discussion 

The  results  are  significantly  predictive  for  the  learning  effectiveness  of  the  three  group  interaction  scenarios.  The 
mode  of  group  interaction  influences  significantly  the  knowledge  fluency  (Sig.  = .047),  concepts  acquisition  (Sig.  = 
.045),  and  group-to-individual  transfer  (Sig.  = .004).  On  these  criteria  students  in  Shared  groups  scored  significantly 
higher  than  students  in  Moderated  and  Distributed  groups. 

All  three  scenarios  enabled  students  to  present  and  incorporate  their  knowledge  in  the  group  process.  No  significant 
difference  was  found  on  individual-to-group  transfer. 

Shared  mode  was  highly  beneficial  for  acquiring  knowledge  both  from  each  other  and  from  taking  new  perspectives 
in  exploring  the  problem  space.  A significant  part  of  the  knowledge  that  was  elaborated  in  group  session  were 
incorporated  as  a meaningful  part  of  students’  cognitive  structures  and  transferred  to  individual  cognitions. 

Students  working  in  Moderated  mode  incorporated  less  new  concepts  in  their  personal  cognitive  structure  than 
students  in  Shared  mode.  Only  those  aspects  of  the  group  solution  that  were  developed  by  the  personal  participation 
of  a particular  student  were  internalised  in  his/her  cognition. 

Regardless  that  students  in  Distributed  mode  had  a broad  access  to  all  other  students’  knowledge  representation  they 
were  not  able  to  gain  enough  from  the  externalised  experience  of  the  others. 

Both  the  Shared  and  Moderated  mode  proved  their  potential  for  reconstruction  of  the  individual  cognition 
represented  mainly  by  reshaping  of  map  spatial  configuration.  Significant  interdependence  between  common  group 
problem  solution  and  the  individual  outputs  was  found.  Students  working  within  Distributed  mode  resist  on  their 
prior  conceptual  structure. 

No  significant  difference  was  found  on  criteria  of  knowledge  flexibility.  The  three  types  of  groups  were  able  to 
explore  the  problem  space  from  different  perspectives  and  within  different  conceptual  paradigms. 

Surprisingly,  the  individual  creativity  was  not  influenced  by  the  mode  of  group  interaction. 

Nevertheless  no  significant  difference  was  found  (Sig.  = .345),  it  should  be  stated  that  students  working  in  Shared 
mode  showed  a lower  level  of  retention  of  their  initial  concepts  than  the  students  in  the  other  two  modes. 

In  summary,  the  experiment  revealed  that  the  learning  effectiveness  is  influenced  significantly  by  the  mode  of  group 
interaction.  In  general.  Shared  interaction  scenario  proved  to  be  the  most  effective  in  collaborative  learning  and 
problem  solving.  These  leads  to  the  conclusion  that  the  learning  effectiveness  depends  on  the  extend  to  which 
students  share  their  learning  not  only  as  results  but  also  as  a process  of  knowledge  acquisition  and  creation  by  a 
direct  interaction. 
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Abstract:  The  use  of  databases  involves  assigning  resources  to  users  with  changing  access  rights. 
How  will  the  information  be  presented  to  the  right  users?  How  can  resources  be  reused  for  different 
audiences  and  how  should  access  rights  be  provided.  Who  has  the  right  to  change,  who  can  read  it 
and  who  is  owner  of  the  objects  in  a resource  base? 


Introduction 

The  faculty  of  Educational  Science  and  Technology  started  a faculty  wide  implementation  of  Teleleaming  with  the 
TeleTop  project  in  1997.  The  aim  of  the  project  was  to  support  teachers  and  students  with  a web  based  database 
driven  environment.  Two  years  later  the  entire  university,  including  ten  faculties,  is  ready  to  the  implement  the 
Teletop  system.  How  reuse  of  material  should  be  provided  is  a point  of  research  because  content  management  can 
make  the  use  of  objects  more  efficient. 

The  problem  of  assigning  resources  to  people 

Scaling  up  a course  management  system  means  that  more  information  will  be  stored.  Reusing  includes  retrieving 
stored  information.  Finding  relevant  material  and  assigning  this  material  to  the  right  users  is  a problem  because  of 
the  large  amount  of  objects  and  the  large  scale  of  users.  A second  point  of  attention  is  the  use  of  access  rights. 
Access  rights  are  used  to  give  users  rights  to  read  or  change  objects.  Using  databases  includes  controlled  access  to 
different  objects  for  different  users. 

Objects: 

Every  object  has  users  that  can  read  or  use  the  object,  users  who  are  able  to  change  or  create  the  object  and  users 
that  can  actual  delete  an  object.  These  rights  are  set  in  the  access  control  list  and  every  database  uses  some  sort  of 
access  control  list  to  control  access  objects  in  a database.  These  objects  can  consist  of  (elements  of)  learning 
material,  course  descriptions,  assignments,  multimedia  fragments,  HTML  pages  or  any  other  type  of  digital  content. 
Controlling  access  to  information  is  needed  so  that  copyrights  or  confidential  material  can  be  stored  in  a protected 
environment;  i.e.  an  environment  that  is  not  accessible  for  unauthorized  users. 

Users: 

Users  are  mostly  unaware  of  these  access  control  lists,  because  software  developers  make  these  environments  as 
easy  as  possible  by  not  showing  objects  that  users  are  not  authorized  to  see.  The  problem  in  most  cases  is  that 
objects  may  be  accessible  for  one  user,  but  not  for  another.  Resources  available  for  teachers  may  not  be  accessible 
for  students.  Assigning  students  to  these  resources  may  result  in  irrelevant  links  for  the  students.  This  makes  reuse 
for  different  users  and  audiences  a difficult  issue.  This  problem  has  not  earlier  been  identified  because  most  web- 
based  environments  are  still  pilots  or  prototypes.  A faculty  wide  implementation  includes  a large  number  of  users 
who  can  create  objects  and  even  more  users  that  should  be  able  to  use  these  objects.  Dealing  with  a large  scale  of 
objects  and  really  reusing  these  in  database  driven  courses  is  a new  development. 


Solution  for  large  numbers  of  objects 

Maintaining  access  control  for  every  single  object  in  a very  large  resource  base  is  not  manageable.  Managing  many 
(more  than  10.000)  users  over  many  (more  than  10.000)  objects  is  a realistic  example  in  our  situation  at  the 
University  of  Twente.  Combining  a number  of  related  objects  in  a course  is  a solution  to  make  control  access  rights 
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manageable.  In  this  way  it  is  possible  to  assign  a set  of  objects  to  several  predefined  user  groups  comprised  of 
teachers  or  students.  One  group  can  be  assigned  the  role  of  author  and  other  groups  as  readers.  Copying  objects  to 
new  courses  can  involve  reuse  of  objects.  This  method  has  been  sufficient  until  now,  because  in  most  cases 
instructors  wanted  to  reuse  only  the  material  created  by  themselves.  Reuse  as  described  gives  the  possibility  to  make 
changes  to  objects  for  the  new  audience. 

Making  objects  more  flexible 

Considering  the  results  of  reuse  we  recently  developed  a new  model  for  access  rights.  This  model  is  based  on  roles 
that  are  assigned  to  user  groups.  A user  group  is  a list  of  people  who  can  access  the  same  functionality  in  a database. 
So  the  access  control  of  every  object  in  a course  depends  on  the  role  a user  has.  Three  roles  where  identified: 
Teacher,  student  and  administrator.  Using  roles  makes  reuse  possible  by  employing  an  abstract  access  level  that  can 
be  used  over  courses  and  provides  a consistent  control  of  rights.  Objects  can  be  used  in  every  course  without 
changing  access  control  lists  of  involved  courses.  The  development  of  a resource  browser  is  in  progress.  In  the  first 
place  reuse  of  objects  within  a course  is  being  researched,  in  the  second  place  exchange  of  objects  between  courses 
will  be  researched. 

Challenge,  future  plans 

In  the  next  years  reuse  of  objects  at  the  university  level  should  be  supported.  Objects  should  be  reused  within 
courses,  between  courses  and  faculties.  Even  different  universities  should  be  able  to  reuse  objects  in  courses. 
Therefore,  interfaces  should  be  developed  that  support  users  selecting  the  right  resources.  Also,  access  right  models 
have  to  be  developed.  Another  objective  of  our  research  should  be  directed  at  the  ownership  of  objects;  i.e, 
copyrights.  Ownership  can  be  seen  as  part  of  the  access  rights  involved  with  objects.  Is  a user  who  creates  an  object 
the  owner  and  is  this  user  the  only  one  who  may  reuse  or  add  or  edit  this  object?  Who  owns  such  copyrights?  All 
these  points  need  further  research.  In  this  respect  it  is  important  to  note  that  a number  of  organizations  involving 
standards  for  learning  technology  and  related  issues  are  aware  of  these  problems  and  are  trying  to  find  solutions,  but 
until  now  there  is  not  a standard  way  to  handle  large  scale  resource  bases.  Examples  of  these  organizations  are  the 
Instructional  Management  Systems  project  (IMS)\  the  Alliance  of  Remote  Instructional  Authoring  and  Distribution 
Networks  for  Europe  (ARIADNE)^  the  IEEE  1484P  Learning  Technology  Systems  Architecture  (LTSA)^, 
Advanced  Distributed  Learning  (ADL)*^  and  the  Aviation  Industry  CBT  Committee  (AICC)^. 


’ IMS:  http://www.imsproject.org 
^ ARIADNE:  http://ariadne.unil.ch 
^ LTSA:  http://www.manta.ieee.org/pl484 
^ ADL:  http://www.adlnet.org 
^ AICC:  http.7/www. aicc.org 
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Abstract:  As  the  Internet  brings  new  means  of  presenting  information  to  learners,  the  need  for 
meaningful  assessment  increases.  We  describe  the  design  and  development  of  a computer  tool  to 
facilitate  a particular  form  of  assessment  that  can  be  meaningful  in  online  learning  contexts.  The 
tool  combines  a web  browser  with  an  annotation  facility,  a database,  and  special  means  for 
visualizing  assessment  data  captured  with  the  tool. 

Introduction: 

As  educators  explore  the  many  possibilities  for  delivering  instruction  online  via  the  Internet,  they  tend  to  focus 
primarily  upon  delivering  presentations  of  information.  The  forms  of  presentation  include  web  pages  and  web  sites, 
text,  images,  audio  clips,  video  clips,  queries  to  databases,  and  Java  applets  that  run  simulations.  In  addition  to  the 
information  itself,  the  Internet  also  enables  the  online  gathering  and  experience  of  groups  of  people  who  might  (or 
might  not)  be  physically  distant  from  one  another.  The  online  communities  of  learners  can  be  formed  such  that  the 
members  are  all  at  a very  similar  level  of  expertise  in  the  subject  they  are  studying.  What  is  largely  missing  from 
these  innovations,  however,  is  a form  of  educational  assessment  that  is  well  adapted  to  the  new  patterns  of  learning. 

Assessment  is  sometimes  regarded  as  the  unpleasant  part  of  education,  in  part  because  traditional  testing  is 
sometimes  distasteful  to  students  and  because  assessment  is  typically  considered  less  important  to  the  presentation 
of  information.  However,  studies  of  assessment  in  support  of  learning  (rather  than  in  support  of  administration  or  of 
college  admission,  for  example)  have  lead  to  recommendations  for  alternative  assessment  methods  such  as 
performance  assessment,  authentic  assessment,  portfolio-based  assessment,  and  continuous  assessment  (see 
Tanimoto,  1998).  These  methods  offer  the  possibility  to  make  assessment  a more  "natural”  part  of  the  learning 
process,  and  to  permit  learning  to  occur  with  less  stress  and  in  ways  more  in  line  with  workplace  goals.  Yet  these 
methods  are  often  difficult  to  implement  because  they  may  require  more  teacher  training,  more  teachers’  time  spent 
on  assessment,  and  because  their  results  may  seem  to  be  more  subjective  than  the  results  of  standardized  tests.  We 
have  developed  a software  tool  for  assessment  to  help  us  explore  one  approach  to  assessing  online  learning.  The 
tool,  called  INFACT,  permits  the  assessment  to  be  performed  unobtrusively  in  a manner  reminiscent  of 
psychologists'  observations  from  behind  the  glass  wall. 

Pedagogical  Approach 

Our  approach  to  assessment  takes  place  within  a particular  pedagogical  methodology.  We  have  applied  the 
methodology  in  two  subject  areas:  high-school  physics  and  undergraduate  statistics.  Students  in  the  class  are 
divided  into  groups  of  4-6  students  each.  An  instructional  unit  lasts  one  week.  On  Monday,  a problem  that  requires 
conceptual  thinking  is  posed  by  the  teacher.  Each  student  in  the  class  is  told  to  make  her/his  own  interpretation  of 
the  problem  and  suggestion  for  solving  it  and  to  write  and  post  a newsgroup  entry  using  a specially  modified  version 
of  the  HyperNews  software  from  the  University  of  Illinois.  The  posting  is  due  by  12  noon  on  Tuesday.  This  initial 
post  is  invisible  to  other  students  until  a virtual  curtain  is  lifted  shortly  after  noon  on  Tuesday,  at  which  time  it 
becomes  visible  to  the  other  members  of  the  student’s  group.  The  students  are  then  required  to  respond  to  each 
other’s  postings  in  an  online  discussion,  and  they  are  told  to  come  up  with  a group  solution  by  Friday.  Over  the 
course  of  the  week,  their  postings  accumulate,  creating  an  archive  which  feeds  our  assessment  tool,  INFACT. 
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INFACT  stands  for  "Integrated  Networked  Facet-Based  Assessment  Capture  Tool."  It  is  used  by  teachers  to  view 
and  annotate  student  newsgroup  postings  in  such  a way  as  to  build  up  a database  of  assessment  information  that  can 
be  used  to  track  student  and  group  progress  in  understanding  the  concepts  of  the  instructional  unit.  INFACT  makes 
use  of  catalogs  of  standard  misconceptions  that  students  commonly  have  about  a subject.  Each  misconception  is 
called  a facet,  and  facets  are  grouped  into  clusters  such  that  all  facets  in  a cluster  represent  misconceptions  about  the 
same  concept.  A facet  catalog  must  be  compiled  in  advance  of  using  INFACT  for  assessment.  However,  it  is 
possible  to  refine  a facet  catalog  in  the  course  of  assessment.  INFACT  makes  the  assumption  that  facets  can  be 
linearly  ordered  by  their  "level  of  expertise"  (see  Minstrell  1992  and  Hunt  and  Minstrel 1 1994  for  details). 

The  Design  of  INFACT 

INFACT  comprises  the  following  components:  an  integrated  web  browser,  a database  (currently  an  MSQL 
database),  dialogs  for  creating  and  editing  assessment  items,  and  a visualization  component.  The  browser  component 
permits  the  teacher  to  load  and  view  HyperNews  postings  in  essentially  the  same  form  that  the  students  see  when 
they  read  and  post.  This  component  also  supports  creating  selections  of  text  within  the  document  being  viewed. 
The  INFACT  software  determines  (insofar  as  possible  given  the  limitations  of  available  browser  software)  the  start 
and  end  positions  within  the  document  where  the  selection  occurs.  Selections  of  this  form  can  then  be  annotated. 

Annotations  of  two  types  are  permitted:  facet  annotations  and  free  annotations.  A facet  annotation  represents  an 
atomic  assessment  item,  and  it  consists  of  the  text  selection,  a reference  to  the  document  in  which  the  text  occurs, 
identification  of  the  student  who  authored  the  text,  identification  of  the  assessor,  reference  a particular  facet  for 
which  this  text  provides  evidence,  a certainty  value  given  by  the  assessor,  the  date  of  authoring  and  the  date  of 
assessment.  A free  annotation  is  simply  a comment  associated  with  the  selection.  The  assessor  can  check  a box 
associated  with  each  annotation  to  make  it  visible  to  the  student.  The  database  is  used  to  keep  track  of  the 
annotations  and  to  make  assessment  items  retrievable  by  student,  by  facet  cluster,  by  date,  etc.  The  database  also 
holds  the  facet  catalog.  The  visualization  portion  on  INFACT  permits  the  user  (usually  the  teacher)  to  call  up 
assessment  data  by  student,  by  group,  by  date,  or  by  facet  cluster.  The  user  can  also  view  graphs  that  help  to  reveal 
trends  in  the  data.  Two  types  of  graphs  supported  by  INFACT  are  (a)  histograms  of  facet  frequencies,  and  (b)  facet 
transition  diagrams.  The  latter  show  how  one  or  more  students'  misconceptions  about  a concept  change  over  time. 

As  a teacher  reads  student  postings  with  INFACT  and  makes  annotations,  s/he  can  use  INFACT  to  send  feedback  to 
the  students.  INFACT  supports  two  means  of  feedback:  direct  email  and  student-visible  annotations.  Direct  email 
permits  the  teacher  to  send  a message  to  a student  in  the  context  of  an  assessment  item.  This  can  be  used  to  get 
clarification  from  the  student  about  something  not  clear  from  the  posting.  It  can  also  be  used  to  commend  the 
student  or  to  make  a suggestion.  However,  its  main  value  over  email  from  ordinary  mail  programs  is  the  automatic 
association  created  between  the  message  and  the  assessment  context  in  which  it  originates.  INFACT  makes  it 
possible  for  the  teacher  to  include  a multiple-choice  question  in  the  message  that  when  answered  online,  provides 
additional  evidence  in  favor  of  one  facet  or  another  within  the  facet  cluster  associated  with  the  assessment  item.  For 
a screen  shot  of  INFACT,  see  http://www.cs.washington.edu/research/metip/crlt/infact-l.gif 


Hunt,  E.,  and  Minstrell,  J.  1994.  A cognitive  approach  to  the  teaching  of  physics.  In  McGilly,  K.  (ed.).  Classroom  Lessons. 
Cambridge,  MA:  MIT  Press,  pp.51-74. 

Minstrell,  J.  1992.  Facets  of  students'  knowledge  and  relevant  instruction.  In  Duit,  R.,  Goldberg,  F.,  and  Niedderer,  H.  (eds.). 
Research  in  Physics  Learning:  Theoretical  Issues  and  Empirical  Studies.  Kiel,  Germany:  Kiel  Univ.,  Insti.  for  Science  Educ. 

Scardamalia,  M;  and  Bereiter,  C.  1996.  Student  communities  for  the  advancement  of  knowledge.  CACM,  39,  No.-^4,  pp.  36-37. 

Tanimoto,  S.~L.  1998.  Towards  an  ontology  for  alternative  assessment  in  education.  Technical  Report  UW-CSE-98-09-04,  Dept 
of  Computer  Science  and  Engineering,  Univ.  of  Wash.,  Seattle,  September  1998. 

Acknowledgements:  Thanks  to  J.  Phillips  and  J.  Baer  for  contributions  to  the  visualization  code  and  to  J. 
Ammerlahn  for  the  supporting  Perl  scripts.  Support  from  NSF  Grant  CDA-9616532  is  gratefully  acknowledged. 


References 


BFST  COPY  AVAIl^RIF 


Evaluation  of  an  ITS  for  the  Passive  Voice  of  the  English  Language 
Using  the  CIAO!  Framework 


Maria  Virvou,  Victoria  Tsiriga,  Dimitris  Maras 
Department  of  Informatics, 

University  of  Piraeus, 

Piraeus  18534,  Greece 

E-mail:  mvirvou  @unipi.gr,  vtsir@unipi.gr,  eagle@ath, fothnet.gr 


Introduction 

The  primary  aim  of  educational  software  is  to  help  students  learn.  Therefore,  evaluation  of  this  kind  of 
software  is  crucial.  Its  aim  is  to  test  the  functionality  and  usability  of  the  design  and  to  identify  and  rectify 
any  problems  (Dix  et  al.  1993).  In  particular,  formative  evaluation  is  one  of  the  most  critical  steps  in  the 
development  of  learning  materials  because  it  helps  the  designer  improve  the  cost-effectiveness  of  the 
software  and  this  increases  the  likelihood  that  the  final  product  will  achieve  its  stated  goals  (Chou  1 999).  In 
this  paper,  we  present  and  discuss  the  evaluation  of  an  intelligent  tutoring  system  for  the  passive  voice  of 
the  English  language  using  the  CIAO!  framework  (Jones  et  al.  1999). 

The  Passive  Voice  Tutor 

The  tutoring  system  that  we  developed  and  evaluated  is  called  Passive  Voice  Tutor  (Virvou  & Maras 
1999a;  1999b).  The  Passive  Voice  Tutor  complies  with  the  architecture  of  most  ITSs,  which  consists  of  the 
domain  knowledge,  the  student  modeller,  the  advice  generator  and  the  user  interface  (Wenger  1 987). 

In  particular,  the  domain  knowledge  of  the  Passive  Voice  Tutor  comprises  knowledge  about  how  to 
convert  a sentence  from  active  to  passive  voice  and  vice  versa.  It  also  provides  the  knowledge  needed  for 
the  dynamic  construction  of  new  exercise  sentences.  Finally,  it  comprises  knowledge  about  the  semantic 
relation  between  the  words  used  for  the  construction  of  new  sentences. 

The  student  modeller  checks  the  student’s  answer  and  in  case  of  error,  it  performs  error  diagnosis.  It 
contains  a bug  library  of  the  most  common  student  errors  and  generic  mal-rules  that  students  often  apply 
when  they  have  misconceptions.  Another  responsibility  of  the  student  modeller  is  to  keep  a history'  record 
concerning  each  student’s  progress  in  learning.  This  record  is  used  in  cases  were  ambiguity  arises,  so  that 
the  system  attributes  a particular  error  to  a category  of  errors  that  the  student  seems  to  be  prone  to. 

The  advice  generator  is  responsible  for  responding  in  the  most  appropriate  way  to  a student’s  error,  by 
informing  the  user  about  the  cause  of  the  error  and  by  showing  to  him/her  the  relevant  part  of  the  theory. 
Finally,  the  user  interface  of  the  Passive  Voice  Tutor,  is  a multimedia  user  interface,  which  involves 
animations,  sounds  and  a limited  form  of  natural  language  so  that  it  can  attract  the  student’s  interest. 

The  Evaluation 

The  CIAO!  framework  outlines  three  dimensions  to  evaluate:  (i)  context;  (ii)  interactions;  and  (iii) 
attitudes  and  outcomes.  One  important  aspect  of  context  is  the  reason  why  CAL  is  adopted  in  the  first  place, 
i.e.  the  underlying  rationale  for  its  development  and  use;  different  rationales  require  different  evaluation 
approaches.  According  to  the  framework,  the  reason  for  looking  at  students’  interactions  with  the  software 
is  in  order  to  understand  more  about  their  learning  processes;  such  interactions  can  provide  protocol  data 
for  later  analysis.  Finally,  at  the  “outcomes”  stage,  information  from  a variety  of  sources  needs  to  be  used 
including,  pre  and  post-achievement  tests,  interviews  and  questionnaires  with  students  and  tutors.  The 
overall  emphasis  of  this  framework  is  on  educational  issues. 

There  are  many  other  evaluation  techniques,  such  as  the  JIGSAW  model  (Squire  & Preece  1996)  or  the 
‘set  of  learning  with  software  heuristics’  (Squires  & Preece  1999),  which  also  address  the  problem  of 
integrating  both  usability  and  learning  issues  in  the  evaluation  of  educational  software.  In  the  JIGSAW 
model,  the  evaluation  is  performed  in  three  levels.  In  Level  1 , the  subtasks  of  the  learning  and  operational 
tasks  are  considered  independently  of  each  other.  As  we  move  from  Level  1 to  Level  2,  integration  within 
the  learning  and  the  operational  tasks  is  considered.  At  Level  3 integration  between  the  learning  and 
operational  tasks  is  considered.  On  the  other  hand,  the  set  of  ‘learning  with  software’  heuristics,  are  an 
adaptation  of  the  “usability  heuristics”  presented  in  Nielsen  (1994),  so  as  to  relate  them  to  socio- 
constructivist criteria  for  learning.  These  heuristics  include  the  need  to  consider  appropriate  levels  of 
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learner  control,  the  need  for  the  prevention  of  peripheral  cognitive  errors  and  the  need  for  strategies  for  the 
cognitive  error  recognition,  diagnosis  and  recovery  cycle.  The  Cl  AO!  framework,  seemed  to  be  a more 
suitable  method,  in  the  sense  that  it  also  evaluated  the  students’  use  of  the  resource  package. 

For  the  evaluation  along  the  dimensions  of  the  CIAO!  framework,  presented  above,  we  have  done  the 
following: 

i)  Context;  The  underlying  rationale  for  the  development  and  use  of  the  Passive  Voice  Tutor  is  to  help 
students  during  the  process  of  learning.  Hence,  one  of  the  primary  aims  of  the  system  is  to  address 
individual  learners’  misconceptions  and  help  them  correct  their  errors.  Evaluation  in  terms  of  the 
context  has  been  addressed  by  questionnaires  to  human  tutors  before  their  students  had  ever  used  the 
system.  This  was  done  so  that  the  domain  knowledge  and  student  modeller  could  be  evaluated.  In 
particular,  tutors  were  given  a set  of  erroneous  answers  to  questions  relating  to  the  domain,  which  were 
generated  using  the  system’s  domain  knowledge  and  bug  library.  For  each  question,  each  tutor  had  to 
provide  an  explanation  as  to  what  s/he  thought  the  underlying  misconception  of  the  error  was.  Each 
human  tutor’s  explanation  was  compared  to  the  Passive  Voice  Tutor’s  explanation.  A satisfactory 
degree  of  compatibility  was  found  (45%).  However,  the  comparisons  also  showed  that  the  explanations 
could  be  improved  at  a later  version  of  the  Passive  Voice  Tutor. 

ii)  Interactions:  On  a second  phase,  tutors  were  asked  to  evaluate  the  system,  after  their  students  had 
interacted  with  the  Passive  Voice  Tutor.  Protocol  data  was  collected  through  the  individual  long  term 
user  model  of  each  student.  This  kind  of  protocol  data  was  first  automatically  analysed  by  the  student 
modeller  and  resulted  in  information  that  concerned  the  student’s  individual  way  of  learning.  Then, 
human  tutors  were  also  given  the  same  set  of  students’  protocols  and  were  asked  to  analyse  them. 
Finally  the  tutors’  analysis  was  compared  to  the  system’s  analysis.  To  a large  extent,  human  tutors  had 
similar  beliefs  to  the  system’s  beliefs  about  the  students  who  took  part  in  this  evaluation  stage. 

iii)  Attitudes  and  Outcomes;  In  the  case  of  the  Passive  Voice  Tutor,  at  the  outcomes  stage,  questionnaires 
were  given  to  students  and  tutors.  Students’  questions  included  the  following:  “Did  you  consider  the 
system  responses  to  your  errors  helpful  for  understanding  your  mistakes?”  Tutors  were  also  asked  to 
evaluate  the  outcome  of  the  use  of  the  system,  after  their  students  had  interacted  with  the  Passive  Voice 
Tutor.  Therefore,  the  teachers  were  asked  to  provide  information  about  whether  the  software  helped 
them  to  understand  better  the  difficulties  that  their  students  had  in  learning  the  passive  voice  of  the 
English  language.  Tutors  also  seemed  quite  satisfied  by  the  system’s  student  modelling  component. 
Finally,  the  progress  statistics  of  each  student  that  had  been  recorded  in  his/her  long  term  student  model 
was  also  used  at  this  stage  of  the  evaluation.  The  analysis  of  this  kind  of  data  showed  that  the  majority 
of  students  (57%)  made  a progress  in  their  learning,  whereas  only  43%  did  not  seem  to  have  any 
progress  in  their  learning  of  the  passive  voice.  This  is  an  encouraging  result  in  terms  of  the  educational 
effectiveness  of  the  system. 

Conclusions 

The  CIAO!  framework  for  evaluating  educational  software  has  provided  insight  for  several  aspects  of 
the  software  that  needed  evaluation  at  several  stages;  e.g.  before  any  student  had  interacted  with  the 
software  and  after.  In  this  way,  both  learning  effects  and  usability  issues  could  be  evaluated.  The  results  of 
the  evaluation  showed  that  the  Passive  Voice  Tutor  was  quite  satisfactory  but  there  is  scope  for 
improvement  in  terms  of  the  explanations  provided  by  the  system. 
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Introduction 

Web-based  education  has  the  obvious  benefit  of  allowing  platform  independent  access  to  easily  updated 
teaching  material.  Recently,  a large  number  of  educational  applications  have  been  delivered  through  the 
World  Wide  Web.  Unfortunately,  most  of  these  systems  are  static,  they  do  not  maintain  a student  model 
and  thus  they  are  unable  to  provide  individualised  tutoring.  Nonetheless,  the  future  of  web-based  education 
seems  promising  since  researchers  in  the  field  of  Intelligent  Tutoring  Systems  have  made  quite  successful 
attempts  to  either  move  existing  ITSs  to  the  WWW  or  build  from  scratch  web-based  ITSs  (Eliot  et  al., 
1997;  Stem  et  al.,  1997).  However,  the  development  of  an  ITS  requires  the  involvement  of  a large  number 
of  people,  including  experts  of  the  specific  domain,  instructors  and  programmers.  A way  to  overcome 
these  problems  may  be  the  development  of  authoring  tools,  which  will  help  construct  cost-effective  and 
reusable  ITSs  in  various  domains.  One  such  authoring  tool  for  ITSs  is  WEAR  (WEb-based  authoring  tool 
for  Algebra  Related  domains).  WEAR  incorporates  knowledge  about  the  construction  of  exercises  and  a 
mechanism  for  student  error  diagnosis  that  is  applicable  to  many  domains  that  can  be  “described”  by 
algebraic  equations  (Virvou  & Moundridou,  1999). 

WEAR  uses  a talking  head  in  its  interface  with  the  students.  Animated  characters  have  often  been  used 
in  the  interfaces  of  systems  (Rist  et  al.,  1997;  Stone  8l  Lester,  1996).  Such  an  interface  makes  a system 
more  appealing  and  attractive  to  the  user  and  if  talking  about  an  educational  application  it  may  also 
promote  the  learning  objectives.  Walker  et  al.  (1994)  investigated  subjects’  responses  to  a synthesised 
talking  head  displayed  on  a computer  screen  in  the  context  of  a questionnaire  study.  Their  findings 
showed  that  compared  to  subjects  who  answered  questions  presented  via  text  display  on  a screen,  subjects 
who  answered  the  same  questions  spoken  by  a talking  head  spent  more  time,  made  fewer  mistakes,  and 
wrote  more  comments.  In  this  paper  we  describe  how  the  talking  head  is  used  in  the  whole  educational 
application. 

Description  of  the  System  and  its  Interface 

WEAR  aims  to  be  useful  to  teachers  and  students  of  domains  that  make  use  of  algebraic  equations. 
Such  domains  could  be  chemistry,  economics,  physics  etc.  In  particular  the  tool  accepts  input  from  a 
human  instructor  about  a specific  equation-related  domain  (e.g.  physics).  This  input  consists  of  knowledge 
about  variables,  units  of  measure,  formulae  and  their  relation.  When  the  human  instructor  wishes  to  create 
exercises  s/he  is  guided  by  the  system  through  a step  by  step  procedure.  At  each  step  of  this  procedure  the 
instructor  specifies  values  for  some  parameters  needed  to  construct  an  exercise.  Such  parameters  could  be 
for  example  what  is  given  and  what  is  asked  in  the  exercise  to  be  constructed.  After  the  completion  of  this 
procedure  the  tool  constructs  the  full  problem  text  and  provides  consistency  checks  that  help  the  instructor 
verify  its  completeness  and  correctness. 

WEAR  assigns  to  each  student  a level  of  knowledge  according  to  his/her  past  performance  in  solving 
problems  with  the  tool.  The  tool  suggests  each  student  to  try  the  problems  corresponding  to  his/her  level  of 
knowledge.  When  a student  attempts  to  solve  an  exercise  the  system  provides  an  environment  where  the 
student  gives  the  solution  step  by  step.  The  system  compares  the  student’s  solution  to  its  own.  The 
system’s  solution  is  generated  by  the  domain  knowledge  about  algebraic  equations  and  about  the  specific 
domain  in  which  the  exercise  belongs  (e.g.  economics).  While  the  student  is  in  the  process  of  solving  the 
exercise  the  system  monitors  his/her  actions.  If  the  student  makes  a mistake,  the  diagnostic  component  of 
the  system  will  attempt  to  diagnose  the  cause  of  it. 

When  interacting  with  the  students  WEAR  responds  through  a talking  head,  representing  in  some 
cases  the  instructor  and  in  some  others  a co-student.  The  talking  head  component  of  the  system  uses 
speech  synthesis  to  automatically  produce  speech  output  from  text  using  MBROLA,  a freely  available 
speech  synthesiser  (http://tcts.fpms.ac.be/synthesis/mbrola.html).  The  talking  head  renders  the  interface 
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quite  attractive  to  students  through  the  sound  of  speech.  Moreover,  since  WEAR  is  an  authoring  tool  for 
ITSs  there  are  a lot  messages  that  are  dynamically  formed  during  the  execution  of  the  application. 
Therefore,  the  authoring  tool  could  not  use  pre-stored  material  for  the  speech  feature  but  rather  a speech 
synthesiser. 

In  the  case  when  the  talking  head  represents  the  instructor  it  guides  the  student  to  the  environment, 
recommends  what  problem  to  solve  next  and  reads  the  problem  text  from  the  database  of  the  authoring 
tool.  Since  WEAR  is  an  authoring  tool,  new  exercises  are  continuously  added  to  its  database.  When  the 
student  begins  to  solve  the  problem,  s/he  may  choose  to  solve  it  either  with  a simulated  “co-student”  or 
with  the  “instructor”.  These  two  different  choices  are  only  on  the  level  of  the  user  interface.  This  means 
that  they  use  the  same  underlying  reasoning  abilities  from  the  diagnostic  component  of  WEAR. 

If  the  student  selects  the  mode  of  the  “co-student”  then  the  talking  head  provides  very  friendly 
messages  as  a peer  to  the  student.  This  simulated  student  is  responsible  for  providing  positive  feedback 
when  the  student’s  actions  are  correct  and  for  pointing  out  the  student’s  underlying  misconception  in  case 
of  an  erroneous  action.  The  information  concerning  which  actions  are  considered  correct  or  not  and  also 
the  messages  that  the  simulated  student  should  say  are  provided  by  the  diagnostic  component  of  WEAR. 
The  talking  head  as  a simulated  student  is  aimed  at  increasing  the  student’s  attention  and  possibly 
collaboration  attitude  (although  in  a limited  form).  Indeed  as  VanLehn  et  al.  (1994)  have  pointed  out  peer 
learning  even  in  a setting  where  the  other  peer  is  a simulated  student  may  increase  students’  collaboration 
skills. 

If  the  student  selects  the  mode  of  the  “instructor”  then  the  talking  head  provides  messages  similar  to 
the  “co-student”  but  they  are  more  formal  and  the  diagnosis  of  misconceptions  goes  one  step  further  to 
resolve  ambiguities  using  the  long  term  student  model.  Indeed,  there  may  be  cases  where  a student’s 
erroneous  action  may  be  attributed  to  more  than  one  misconception.  In  such  cases,  the  diagnostic 
component  first  consults  the  long  term  student  model  and  then  through  the  talking  face  asks  the  student  a 
question  to  determine  his/her  underlying  misconception  and  resolve  the  ambiguity.  The  benefit  of  directly 
asking  the  student  is  twofold:  firstly,  the  system  may  find  out  the  real  reason  for  the  erroneous  action  and 
provide  appropriate  feedback  and  secondly,  the  student  through  explaining  why  s/he  acted  in  that  way 
gains  more  knowledge  and  understanding.  It  is  a common  finding  in  many  researches  that  explaining 
things  either  to  oneself  or  to  another  student  helps  one’s  understanding  (Webb,  1989;  Pressley  et  al., 
1992). 

Conclusions 

In  this  paper  we  described  the  work  in  progress  of  the  development  of  a web-based  authoring  tool  for 
Intelligent  Tutoring  Systems.  The  tool,  called  WEAR,  aims  to  be  used  by  teachers  and  students  working  in 
domains  that  make  use  of  algebraic  equations.  In  this  paper  we  focussed  on  the  interface  of  the  ITSs  that 
are  generated  by  WEAR:  animated  talking  heads  are  used  to  simulate  both  the  instructor  and  a co-leamer 
of  the  student  solving  a problem.  In  that  way,  instruction  and  feedback  are  more  “human-like”  resulting  in 
friendlier  and  more  appealing  ITSs.  Furthermore,  the  students  interacting  with  WEAR,  may  gain  better 
understanding  of  the  domain  being  taught  and  also  improve  their  ability  to  collaborate  with  others; 
benefits  that  arise  from  the  peer  learning  situation  in  which  the  students  are  involved. 
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Introduction 

Intelligent  Tutoring  Systems  have  traditionally  been  very  good  at  providing  dynamic  aspects  to 
the  educational  system’s  reasoning.  This  is  usually  due  to  the  student  modelling  component  that  attempts 
to  follow  the  student’s  way  of  learning  taking  into  account  possible  misconceptions  that  the  student  may 
have.  Other  important  dynamic  features  of  ITSs  are  due  to  the  advice  generator  that  models  the  tutoring 
strategy  by  which  the  student  is  going  to  be  taught  the  domain. 

However,  ITSs  have  often  been  criticised  that  they  represent  immensely  complex  approaches  to 
building  learning  environments  (Boyle,  1997).  A solution  to  this  problem  may  be  authoring  tools  that 
provide  the  facility  of  building  multiple  ITSs.  In  particular,  as  Murray  (1999)  points  out  ITS  authoring 
tools  may  give  the  instructional  designer  a combination  of  facilities  to  produce  visually  appealing, 
interactive  screens  and  a deep  representation  of  content  and  pedagogy. 

This  paper  describes  an  authoring  tool  for  ITSs  that  incorporates  a domain-independent  reasoning 
mechanism.  The  reasoning  mechanism  is  used  for  the  student  modelling  and  the  tutoring  strategy  of  the 
resulting  ITS. 


Related  Work 

The  domain-independent  reasoning  mechanism  of  the  authoring  tool  described  in  this  paper,  is 
based  on  an  adaptation  of  a cognitive  theory  called  Human  Plausible  Reasoning  (Collins  & Michalski, 
1989).  Human  Plausible  Reasoning  theory  (henceforth  referred  to  as  HPR)  is  a domain-independent  theory 
originally  based  on  a corpus  of  people’s  answers  to  everyday  questions.  HPR  formalises  the  plausible 
inferences  based  on  similarities,  dissimilarities,  generalisations  and  specialisations  that  people  often  use  to 
make  plausible  guesses  about  matters  that  they  only  know  partially.  These  inferences  may  lead  to  either 
correct  or  incorrect  guesses;  in  any  case  these  guesses  are  plausible. 

This  theory  can  be  used  as  a domain-independent  generator  of  plausible  human  errors  or  as  a way 
of  reasoning  for  open  questions.  Indeed  in  previous  research  we  adapted  and  used  HPR  in  the  user 
modelling  component  of  an  intelligent  help  system  for  UNIX  users  (Virvou,  1998;  Virvou  & DuBoulay, 
1999)  and  in  an  intelligent  learning  environment  for  novice  users  of  a GUI  (Virvou  & Stavrianou,  1999; 
Virvou  & Kabassi,  2000).  In  the  present  research  we  adapted  and  used  HPR  for  the  student  modelling  of 
multiple  domains,  such  as  geography,  biology  etc.  and  as  a tutoring  strategy  for  teaching  students  how  to 
deal  plausibly  with  open  questions.  In  addition  we  have  provided  the  facility  to  human  instructors  to 
construct  the  knowledge  - base  of  the  domain  to  be  taught  to  students. 


Operation  of  the  Authoring  Tool 

The  initial  input  to  the  authoring  tool  is  a description  of  factual  knowledge  of  a specific  domain 
given  by  a human  teacher.  Then  the  tool  constructs  a knowledge  base  concerning  the  specific  domain  in 
the  form  of  hierarchies,  which  is  compatible  with  HPR.  The  human  teacher  may  also  construct  tests  that 
consist  of  questions  concerning  the  factual  knowledge  of  the  domain. 

The  resulting  ITS  may  be  used  by  students  who  can  be  shown  facts  from  the  knowledge  base.  In 
addition  students  may  test  their  knowledge  by  answering  the  questions  that  the  human  teacher  has  given. 
The  student  modelling  component  examines  the  correctness  of  each  student’s  knowledge  and  the 
soundness  of  the  reasoning  s/he  has  used.  Information  about  each  student  is  kept  in  a long  term  student 
model.  This  kind  of  information  may  be  used  to  measure  the  student’s  progress  and  record  persistent 
misconceptions  and  weaknesses  of  the  particular  student. 
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The  tutoring  component  of  the  resulting  ITS  poses  questions  to  the  student  who  is  asked  to  give 
an  answer  and  an  explanation  for  this  answer.  For  example,  in  the  domain  of  geography  the  tutor  may 
pose  the  question:  “Does  Italy  produce  lemons?”  Then  the  student  may  answer  either  “Yes”  or  “No”  and 
give  an  explanation  for  his/her  answer.  An  explanation  could  be  “Yes,  because  I know  it  does”.  If  the 
student  does  not  possess  the  factual  knowledge  needed  for  this  answer  s/he  may  say:  “My  guess  is  yes.  I 
know  Spain  produces  lemons;  Spain  is  similar  to  Italy  in  terms  of  agricultural  products.  Therefore,  Italy 
probably  produces  lemons”.  In  this  case,  both  the  answer  and  the  reasoning  are  examined  by  the  ITS. 
There  may  be  cases  where  the  student’s  answer  may  be  incorrect  but  the  reasoning  reveals  that  the  student 
has  a good  knowledge  of  geography  and  uses  it  in  a plausible  way. 

In  particular,  there  may  be  several  cases  of  combinations  in  terms  of  the  correctness  of  the 
guessed  answer  and  the  correctness  of  the  reasoning.  One  case  is  both  the  guessed  answer  and  the 
reasoning  used  to  be  correct.  Another  case  is  the  guessed  answer  to  be  wrong  but  the  reasoning  used  to  be 
correct.  In  this  case  the  tutor  informs  the  student  about  the  mistake  but  admits  that  the  reasoning  that  the 
student  used  was  plausible.  A third  case  is  the  guessed  answer  to  be  correct  but  the  reasoning  used  to  be 
incorrect.  In  this  case  the  tutor  informs  the  student  that  the  answer  was  correct  by  chance.  There  are  also 
cases  where  the  reasoning  used  may  be  correct  but  the  factual  knowledge  of  relevant  topics  may  be 
incorrect.  For  example,  if  the  student  was  asked  whether  Italy  produced  coffee  and  answered  “My  guess  is 
yes  because  Spain  produces  coffee  and  Spain  is  similar  to  Italy  in  agricultural  products”.  In  this  example 
Spain  is  similar  to  Italy  but  it  does  not  produce  coffee. 

The  learning  environments  that  result  from  this  authoring  tool  are  more  open  than  tutoring 
systems  which  do  not  have  any  reasoning  abilities  about  correct  and  incorrect  knowledge.  For  example, 
Ohlsson  (1993)  suggests  the  analysis  of  epistemic  activities  (arguing,  describing,  explaining,  predicting, 
etc.)  to  be  more  relevant  for  higher  order  learning  of  declarative  knowledge  than  the  study  of  goal-oriented 
action.  In  addition,  Andriessen  & Sandberg  (1999)  point  out  that  the  process  of  becoming  an  expert  in  a 
certain  domain  should  no  longer  be  solely  viewed  as  the  acquisition  of  a representation  of  correct 
knowledge;  the  knowledge  to  be  acquired  should  flexibly  manage  open  problems.  In  this  sense  the 
authoring  tool  provides  support  in  the  student’s  learning  of  factual  knowledge  and  the  reasoning  that  may 
manage  this  knowledge. 


Conclusion 

In  this  paper  we  described  the  work  in  progress  of  an  ITS  authoring  tool  that  incorporates  a 
domain-independent  reasoning  mechanism.  The  reasoning  mechanism  is  based  on  HPR,  which  is  a 
cognitive  theory  about  how  people  make  plausible  guesses.  In  this  way,  the  resulting  ITSs  may  provide 
quite  open  learning  environments  for  the  students  to  practice  their  factual  knowledge  on  a certain  topic, 
factual  knowledge  on  possibly  relevant  topics  and  the  reasoning  they  use  to  deal  with  open  problems  by 
combining  factual  knowledge  from  relevant  topics. 

The  authoring  tool  is  in  the  process  of  being  evaluated  in  the  domain  of  geography  and  biology. 
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Abstract:  This  paper  reports  on  the  results  of  the  project  "Open  and  Distance  Learning 
Network  in  Information  Technology  and  Microelectronics  - EDIT"  mn  by  University 
"Politehnica"  of  Bucharest,  Technical  University  of  Cluj,  Technical  University  "Gh.  Asachi" 
of  Iasi,  University  of  Craiova,  Romania  and  IBM  Global  Services,  Education&Training,  La 
Hulpe,  Belgium.  The  paper  focuses  on  the  description  of  the  main  objectives,  technical 
infrastmcture,  course  materials,  impact/target  groups,  type  of  learning  environment  and 
further  developments.  A distributed  web-based  learning  environment  has  been  developed 
within  the  EDIT  project. 


The  main  objective  of  EDIT  Project  was  to  develop  the  first  Romanian  academic  open  and  distance 
education  network  aimed  at  training  the  students  and  academics  from  the  four  partners  universities.  In  the 
long  run  the  network  will  extend  to  link  the  main  technical  universities  in  Romania,  including  researchers  and 
experts  from  other  institutions.  The  project  is  financial  supported  by  the  Romanian  National  Higher  Education 
Funding  Council  for  a period  of  three  years. 

The  project's  specific  objectives  are:  1)  to  setup  and  to  develop  the  technical  infrastructure;  2)  to 
design  and  to  implement  some  good  examples  of  education/training  course  materials  in  the  field  of  IT  and 
Microelectronics;  3)  to  deliver  the  designed  courses  to  our  students/customers;  4)  to  find  the  best  suited 
learning  environments  for  a given  topic/target  group;  5)  to  support  itself  through  contracts  with  different 
customers  at  the  end  of  the  three  years  period. 


The  Technical  Infrastructure 

Because  of  the  unique  distance  learning  environment  of  the  Web  and  the  availability  in  all  Romanian 
universities  the  EDIT  Project  focuses  on  web-based  learning  environments. 

The  Design  and  Implementation  of  Course  Materials 

In  1998,  EDIT  Distance  Learning  Network  starts  to  deliver  his  first  course  on  Modeling  and 
Simulation  of  Semiconductor  Processes  and  Devices  (MSSPD)  for  the  MSc  students  from  Department  of 
Electronics  and  Telecommunications  (Drondoe  et  al.  1998)  using  an  in-house  designed  web-based  learning 
environment:  EDIT  Learning  Environment  - ELE.  Presently,  we  offer  four  courses  for  Department  of 


Objectives 


Electronics  and  Telecommunications:  Electron,  Devices  and  Circuits,  MSSPD,  Optimization  Methodologies 
and  Particle  Simulation  in  Nanostructures  (Drondoe  et  al.  1999). 


Impact/Target  Groups 

The  courses  are  open  for  undergraduate  and  graduate  students,  academics  and  researchers  that  can 
choose  between  different  level  of  complexity  depending  on  their  previous  knowledge  and  personal  goals. 

The  Type  of  Learning  Environment 

Depending  on  the  topic  and/or  target  group  the  chosen  learning  environment  used  for  implementation 
of  the  course  material  was  either  ELE  and/or  two  well-known  distance  learning  commercial  products  (i.e. 
LOTUS  Learning  Space  and  IBM  Distance  Learning  Space/DLS). 

The  main  features  of  ELE  are  (see  Zolti  et  al.  1999):  www  dedicated  arhitecture,  distributed  way  of 
working  both  in  design  and  exploitation,  high  degree  of  reusability,  integrated  communication  system  (e-mail 
and  virtual  classrooms),  web-based  publishing  system,  web-based  voting  system  and  secure  web  access  to  the 
database. 


Sustainability 

The  sustainability  of  the  project  is  envisaged  through  contracts  with  online  continuing  education 
courses  on  demand  dedicated  to  enhancement  of  adults'  knowledge/skills  and/or  to  professional  reorientation 
of  the  unemployed  people.  A special  attention  is  also  given  to  distance  training  of  the  work-force  in  the 
corporate  sector. 

Conclusions  and  Further  Development 

A distributed  web-based  learning  network  has  been  presented.  At  the  basis  of  this  implementation  lies 
WWW  dedicated  architectures.  This  has  allowed  us  to  reach  a high  level  of  reusability  and  also  cost- 
effectiveness.  Currently,  there  is  much  work  in  progress  to  implement  more  courses  within  EDIT  Distance 
Learning  Network  and  to  further  develop  the  ELE  by  establishing  some  efficient  authentication  methods  that 
would  make  awarding  certificates  more  legitimate. 
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This  short  paper  describes  dissertation  work  in  progress  investigating  the  cognitive 
processes  and  social  practices  involved  in  the  creation  of  computer  based  learning 
environments  (CBLEs).  Unfortunately,  knowledge  regarding  the  actual  design  process  is 
very  limited.  Formal  instructional  design  prescriptions  (Dick  & Carey  1985;  Gagne 
Briggs  & Wager  1988)  and  design  principles  based  on  cognitive  psychology  (Duffy  & 
Cunningham  1996;  Wasson,  1996)  do  exist  but  it  is  not  clear  how  these  prescriptions  and 
design  principles  are  used  in  ‘real  life’.  Research  into  the  cognitive  processes  involved  in 
instructional  design  tasks  is  limited.  Studies  have  examined  such  things  as  the  extent  and 
nature  of  alternative  designs  and  stopping  rules  (Kerr,  1983),  expert-novice  differences  in 
problem  understanding  and  problem  solution  (Rowland,  1 992)  and,  expert-novice 
differences  in  declarative  and  procedural  knowledge  and  representations  of  design  (Perez 
& Emery,  1995).  There  are  very  few  studies  of  the  design  process  involved  in  the 
creation  of  CBLEs.  The  lack  of  empirical  work  regarding  the  design  processes  and 
practices  involved  in  the  creation  of  CBLEs  is  surprising  given  the  important 
contributions  this  type  of  knowledge  could  bring  to  future  design  activities.  Sbadies  of  the 
process  could  indicate  where  typical  difficulties  exist  and  suggest  strategies  to  overcome 
them.  Additionally,  it  would  be  interesting  to  observe  the  effect  of  the  application  of  one 
of  design  principles  on  the  actual  design  process. 

Theoretical  Perspective 

The  theoretical  perspective  adopted  in  this  study  combines  traditional  human  problem- 
solving (Newell  & Simon,  1972)  perspective  with  an  activity  theory  (Engstrom,  1989) 
one.  Problem-solving  research  focuses  on  the  internal  mental  process  involved  in 
problem-solving  and  has  been  used  to  investigate  instructional  design  problem-solving 
process  but  it  has  not  been  used  rigorously  to  study  designers  of  computer  material.  On 
the  other  hand,  activity  theory  investigates  how  social  systems  influence  human  practice. 
Activity  theory  has  not  been  used  to  study  the  design  processes  involved  in  the  creation 
of  computer  based  material  but  has  been  used  to  study  other  computer  design  activities. 
Research  that  combines  human-problem  solving  and  activity  perspectives  provides  a 
theoretical  perspective  that  recognizes  the  cognitive  and  social  dimensions  of  design 
activities  (Bracewell  & Witte  1997;  Cobb,  1994;  Portes  1996). 


Questions  and  Method 

(1)  What  problem-solving  processes  are  involved  in  developing  computer-based  learning 
environments  (human-problem  solving  perspective) 

(2)  What  social  practices  are  involved  in  the  creation  of  developing  computer-based 
learning  environments  (activity  theory  perspective) 

A case  study  of  a design  team  at  a local  multimedia  development  firm  was  conducted 
over  several  weeks.  Participants  included  one  subject  matter  expert,  an  editors  and  a 
graphic  artists.  Data  consists  of  participant  observation  field  notes  collected  over  several 
weeks,  interviews  and  think-aloud  protocols. 

Results 

Preliminary  results  indicate  that  the  problem-solving  activity  of  team  members  is  goal 
directed  and  involves  establishing  numerous  local  plans  and  frequent  evaluations.  It  is 
clear  that  a concurrent  analysis  that  combines  problem-solving  and  activity  theory  is  a 
very  promising  approach.  It  provides  data  that  allows  researchers  to  analyse  and  interpret 
the  relationship  between  internal  mental  processes  and  social  context. 
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Abstract:  This  paper  describes  a case  study  on  the  use  of  community-based  low 

powered  FM  radio  that  has  been  undertaken  in  Uganda  as  part  of  the  work  of  the 
Commonwealth  of  Learning.  It  describes  a media  solution  that  employs  low-powered 
FM  radio  as  an  effective  method  of  delivering  education  and  information  to  a rural 
community.  It  also  describes  the  inhibitors  to  proliferation  of  community  radio  stations 
and  reflects  on  the  future  directions  for  the  use  of  radio  as  a delivery  system  for 
education. 


Introduction 

available  instruments  and  channels  of  information,  communications,  and  social  action  could  be  used 
to  help  convey  essential  knowledge  and  inform  and  educate  people  on  social  issues.  In  addition  to  the 
traditional  means,  libraries,  television,  radio  and  other  media  can  be  mobilized  to  realise  their  potential 
towards  meeting  basic  education  need  of  all”.  Final  Report  World  Conference  for  All:  Meeting  Basic 
Learning  Needs,  Jomtien,  Thailand,  1990. 

Why  Community  Radio? 

Community  radio  is  an  immensely  powerful  technology  for  the  delivery  of  education  with  enormous 
potential  reach  globally.  Creating  opportunities  for  communities  to  utilise  this  delivery  system  will  enable 
disadvantaged  groups  to  engage  in  a development  agenda,  sensitive  to  their  needs  and  aspirations.  In  order 
to  serve  the  underprivileged  and  rural  poor,  mass  media  such  as  radio  must  create  conditions  and 
mechanisms  that  provide  people  with  genuine  access  to  useful  information.  Such  mechanisms  will  offer 
ways  in  which  people  can  express  their  sentiments,  opinions,  views,  dreams  and  aspirations,  their  fears  and 
insecurities,  their  strengths  and  capabilities,  and,  of  course,  their  ideas  for  development. 

High  illiteracy  rates  and  low  levels  of  schooling  among  disadvantaged  groups,  especially  women,  in  many 
developing  countries  continues  to  limit  their  ability  to  lift  themselves  out  of  poverty.  The  existing 
educational  system  has  shown  itself  to  be  unable  to  respond  to  the  massive  demand  for  increased  education. 
This  is  especially  true  in  many  poverty-stricken  countries  with  respect  to  meeting  the  massive  education 
needs  of  the  rural  poor.  Consequently,  disadvantaged  groups  continue  to  be  denied  access  to  information, 
knowledge,  skills  and  technology  transfer. 

The  answer  is  to  deploy  Distance  Education  techniques  and  delivery  systems  such  as  radio  and  television 
based  at  the  community  level  to  address  directly  local  issues  and  needs. 

In  order  to  empower  disadvantaged  groups  as  equal  partners  in  development,  the 
limitations  of  formal  and  non-formal  education  are  now  being  challenged.  Urgent  and 
new  ways  to  achieve  mass  education,  that  can  be  both  efficient  and  effective,  are  being 
sought.  In  this  context,  radio,  an  effective  telecommunications  medium,  was  proposed  at 
the  Jomtien,  Thailand  UNESCO  Education  of  All  conference  in  1990  as  one  massive 
solution.  Radio  can  cut  across  geographic  and  cultural  boundaries.  Given  its  availability. 
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accessibility,  cost-effectiveness  and  power,  radio  represents  a practical  and  creative 
medium  for  facilitating  mass  education  in  a rural  setting.  However  ten  years  on  since 
Jomtien,  radio  still  continues  to  be  an  underutilized  technology  in  education.  This  is 
especially  surprising  because  from  a learner’s  point  of  view,  radio  is  user  friendly, 
accessible  and  a well-established  medium.  From  an  educational  provider’s  point  of  view 
it  is  easy  to  set  up,  produce  and  broadcast  programmes.  After  almost  hundred  years  of 
broadcasting  history  most  nations  of  the  world  have  more  than  a respectable  level  of 
engineering  skills  and  broadcasting  talent  to  apply  the  technology  in  education.  In  the  last 
ten  years,  radio  has  been  greatly  enhanced  by  the  emergence  of  new  technologies,  which 
have  opened  up  new  opportunities  for  a variety  of  forms  of  delivery  and  access  for  both 
broadcaster  and  listener.  For  example,  portable,  low  cost  FM  transmitting  stations  have 
been  developed  and  digital  radio  systems  that  transmit  via  satellite  and/or  cellular  are 
being  implemented  in  many  parts  of  the  globe.  Internet  streaming  audio  software 
technology  has  emerged  recently  to  allow  a global  audience  to  listen  the  news  from  a 
distant  country.  And  windup  and  solar  radios  have  been  developed  thus  freeing  radios 
from  expensive  power  sources.  COL  is  addressing  the  issue  of  education  for  all  with 
creative  media  solutions  that  fall  under  the  banner  of  an  initiative  called  The 
Commonwealth  of  Learning  Media  Empowerment  (COLME). 

COLME  - A Template  for  a Media  Projects 

COLME  is  building  creative  media  models  for  delivery  of  information  and  training  in  the 
area  of  non-formal  and  formal  education. 


COLME  strives  to: 

• Provide  new  skills  in  the  use  of  technology  for  the  disadvantaged 

• Provide  media  models  that  will  stress  local  participation  and  transfer  of 
knowledge  and  skills 

• Provide  opportunities  for  disadvantaged  groups  to  participate  and  benefit  from 
new  technology  and  media  based  initiatives 

• Create  a capacity  for  dialogue  among  government  sectors,  institutions  and 
different  interest  groups 

• Create  a research  base  and  body  of  knowledge  that  can  be  utilised  by 
Commonwealth  governments,  organisation  and  communities  as  models  for  media 
and  technology  based  initiatives. 

Media  projects  in  radio  stations  and  projection,  video  and  audio  production,  and 
computing  solutions  have  all  been  part  of  the  work  of  COLME  in  the  past  several  years. 
The  focus  here  is  a portable  FM  radio  station  that  has  been  used  in  several  developing 
countries  of  the  Commonwealth  such  as  Uganda,  Namibia  and  South  Africa. 


Overview  of  the  Station 


Radio  is  a very  powerful  technology  that  can  allow  large  sectors  of  the  population  to  be 
reached  with  information,  quickly  and  economically.  Due  to  national  broadcast 
regulations  in  many  countries,  community  radio  stations  have  not  developed.  Also  the 
cost  of  transmitters,  infrastructures,  and  equipment,  has  placed  most  potential  community 
broadcasters  at  a disadvantage,  especially  those  in  the  remote  rural  areas.  There  is  a 
distinct  information  gap  to  the  rural  comers  of  some  countries  resulting  from  the  lack  of 
service  by  national  broadcasters  who  in  some  cases  have  weak  or  non-existent  signal 
coverage.  Under  COLME,  portable  FM  radio  systems  have  been  tested  and  implemented 
as  part  of  media  project  work  over  the  past  three  years.  The  station  configuration  that 
was  first  developed  has,  with  input  and  data  gathered  from  COLME  initiatives  in  the 
field,  have  aided  the  manufacturer  in  altering  the  station  to  address  the  each  community’s 
need.  The  station  configurations  range  in  price  from  three  for  five  thousand  dollars  US 
including  all  elements,  antenna,  transmitter,  console,  mixer,  microphones  and  CD  and 
tape  decks  (Figure  1).  The  stations  can  mn  from  12  V DC  or  120/240  AC. 


Figure  I — Pictured  is  a station  in  its  watertight  carrying  case  (on  the  consul  starting  on  the  left  top  the 
gooseneck  microphone,  below  is  the  mixer,  top  right  are  two  tape  decks,  below  are  two  CD  decks).  The 
transmitter  and  power  supply,  not  pictured,  are  housed  under  the  consul.  The  consul  is  removed  from  the 
carrying  case  when  in  operation  (see  Figure  4 for  operational  mode). 


Keys  Elements  to  Success 

There  are  number  technological  factors  that  are  important  in  the  initial  needs  analysis 
before  a station  can  be  considered.  First  the  physical  landscape  must  be  conducive  to  an 
FM  signal  to  reach  the  intended  target  audience  especially  if  rebroadcast  of  the  origin 
station  signal  is  not  possible  due  to  cost  or  licensing  regulations.  If  the  landscape  is 
mountainous  then  there  will  be  difficulty  in  the  signal  reaching  a large  radius  of  users. 
Secondly,  the  station  target  audience  must  have  radios  or  access  to  radios.  Thirdly,  there 
must  a situation  where  there  is  a steady  flow  of  content  and  a regular  broadcast  schedule. 
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Fourthly,  the  station  must  be  targeted  to  the  local  users  so  that  they  can  directly  relate  to 
the  content,  language,  and  situations  discussed. 


In  the  feasibility  stage  before  station  implementation  certain  conditions  must  exist  to  improve  the  element 
of  sustainability.  In-country  stakeholders  are  identified  for  each  of  the  stations.  Their  role  is  to  insure 
infrastructure  is  in  place  for  FM  radio  and  that  all  licensing  and  issues  pertaining  to  community 
broadcasting  are  dealt  with. 

Another  important  factor  is  that  the  broadcasts  are  in  languages  that  are  used  daily  in  the  local  community 
level.  The  national  or  regional  stations  do  not  have  the  capacity  to  aim  linguistically  or  at  the  level  of 
information  detail  for  rural  community  issues.  Community-based  stations  can  be  effective  if  well  managed 
in  providing  information  and  training  directly  to  the  community.  In  the  case  of  the  COLME  installed 
community  station  in  Uganda,  it  was  imperative  that  the  station  be  able,  by  law,  to  rebroadcast  Radio 
Uganda  in  the  event  of  important  political  announcements.  Therefore,  among  the  technological  upgrades 
in  the  design  of  the  station,  in  addition  to  the  interface  for  telephone  calls,  extra  microphone  inputs  for 
group  discussions,  and  a more  powerful  transmitter,  a facility  for  radio  rebroadcast  of  the  national 
government  station,  (in  Uganda,  Radio  Uganda)  and  international  broadcasters  (such  as  the  BBC)  was 
implemented. 

The  overriding  factor  to  the  success  of  these  stations  has  been  the  proper  community 
access  and  ownership,  which  was  paramount  in  the  initial  project  design.  If  the  station  is 
or  becomes  an  integral  part  of  the  voice  of  the  community  and  local  interest  groups  have 
an  equal  say  in  the  information  that  is  disseminated  via  the  station,  then  there  is  a lesser 
risk  of  failure  in  the  long-term  sustainability  of  the  station.  This  can  be  achieved  with 
good  station  management  that  works  with  community  leaders  and  committees  consisting 
of  both  political  and  community  leaders. 

The  local  stakeholders,  with  the  aid  of  COLME,  will  provide  on  going  evaluation  of  the 
stations  via  listener  surveys  and  media  expert  evaluation.  Workshops  will  be  given  in 
production  and  survey  techniques  that  will  aid  broadcasters  with  improving  programming 
to  suit  the  needs  of  the  community.  Local  broadcasters  will  be  tapped  to  train  in 
advanced  broadcasting  techniques  and  programme  development  that  will  improve 
community  radio  personnel. 

A Solar  Station  on  the  Move  - A COLME  Case  Study 

Apac,  Uganda  is  located  in  the  northern  region  of  Uganda.  This  COLME  project  was  a cooperative  effort 
with  the  Minister  of  State  and  Tourism,  The  Right  Hon.  Akaki,  to  work  with  community  leaders  to 
implement  an  FM  radio  station  in  the  Apac  region.  The  COLME  feasibility  study  revealed  several 
limitations  with  the  electrical  infrastructure,  which  was  not  reliable.  This  was  a result  of  load  sharing 
throughout  the  country  (Apac  would  not  receive  power  for  several  days).  The  power  was  also  not  usable 
for  electronic  equipment  due  to  the  dramatic  power  fluctuations.  Therefore,  it  was  decided  that  in  order  to 
maintain  a reliable  broadcasting  schedule  and  develop  the  station  as  a center  point  to  community  activities 
by  different  groups.  Radio  Apac  would  be  operated  entirely  by  solar  power.  This  would  free  the  project 
from  the  constraints  of  electrical  situation  and  the  tariffs  associated  with  it.  A configuration  was 
determined,  in  consultation  with  a solar  distributor  in  Kampala,  to  allow  the  station  to  stay  operational 
during  the  eighteen-hour  broadcast  day.  Eight  solar  panels  and  seven  deep  cycle  batteries  were  installed  at 
the  station,  which  now  provide  lighting  and  all  the  station  power  requirements  for  daily  broadcasting  (see 
Figures  2 to  4).  The  lifespan  of  solar  installations  is  over  a decade  with  low  maintenance  costs. 

A committee  was  organized  and  a station  manager  appointed.  This  person  works  directly  with  the 
community  to  develop  programming  and  allow  the  development  of  community  involvement.  The  station 
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has  a rebroadcast  facility  incorporated  for  programmes  from  the  national  broadcaster  as  well  as  the  BBC 
and  WorldSpace.  A further  radius  of  listeners  will  be  taken  in  Spring  2000  with  a retransmission  unit  that 
will  allow  the  signal  to  take  in  other  populated  areas.  It  is  anticipated  that  the  total  potential  listening 
audience  will  be  over  one  million.  A VHF/UHF  radio  system  will  also  be  implemented  to  act  as  a 
community  telephone  that  will  interface  into  the  FM  radio  system.  This  will  allow  the  broadcasters  to  out 
to  the  community  and  speak  directly  to  groups  live  on  air  thus  eliminating  the  issue  of  travel  to  the  station 
by  people  such  as  farmers  and  medical  officers.  Links  have  been  made  to  educational  programmers  in 
South  Africa  to  supply  educational  programmes  to  the  station.  The  station  has  been  used  by  the 
government  for  health  related  issues  and  announcements.  There  is  also  an  element  of  income  generating  by 
the  station  with  the  interest  of  local  businesses  in  advertising  their  services. 


Figure  2 - Receiving  the  solar  panels  at  the  station  site 


Figure  3 - Installing  the  panels 


Figure  4 - Radio  Apac,  92.9  FM  on  air,  powered  by  the  sun 


Conclusion 

Radio  is  an  effective  system  for  delivery  of  education  to  large  numbers  of  people.  It  facilitates  information 
exchange  at  the  community  level,  acting  as  a “community  telephone”  and  can  be  effective  in  literacy  and 
formal/non  formal  education.  Analogue  systems  for  radio  will  be  supplanted  by  digital  broadcasting  in  the 
coming  decade,  however  digital  radio  will  pose  issues  including  cost  of  radio  receivers  and  renewal  of 
broadcasting  infrastructure.  Analogue  radio  systems,  such  as  the  portable  solution  that  COL  and  others 
have  utilised  in  community  FM  radio  initiatives,  can  be  effective  in  delivering  education  to  the  masses 
without  the  high  infrastructure  costs  associated  with  radio  broadcasting.  With  community  broadcasting  not 
only  can  broadcasters  focus  on  addressing  local  needs  through  their  own  produced  programming,  but  also 
have  the  choice  among  a tremendous  variety  of  quality  educational  content  that  is  available  via  rebroadcast 
from  national  and  international  sources  whether  it  is  delivered  via  satellite  or  via  the  Internet. 
Rebroadcasting  also  should  be  balanced  with  the  needs  of  the  local  community  and  the  provision  of 
appropriate  and  relevant  programming  content. 

There  is  a marriage  between  the  digital  and  the  FM  analogue  systems  that  is  taking  place.  The  convergence 
also  includes  Internet  streamed  audio  based  broadcasters  that  can  effectively  be  employed  by  the 
community  FM  station  in  a rebroadcast  mode.  Will  we  be  able  to  say  in  ten  years  that  radio's  potential  for 
educational  delivery  to  millions  of  disadvantaged  groups  has  finally  been  realised?  With  the  many  varied 
formulas  for  convergence  of  digital  and  analogue  technology  and  the  vast  selection  of  content  and  tools  to 
create  original  culturally  sensitive  material  for  education  at  the  community  level,  we  state  clearly  - yes.  But 
will  the  bodies  that  regulate  frequencies  for  community  radio  initiatives  reform  regulations  to  reflect  the 
current  technological  developments  and  pressing  need  for  mass  media  to  meet  the  goal  for  education  for  all 
in  the  next  ten  years?  We  can  only  hope.  The  past  ten  years  and  the  failure  of  Jomtien  is  a heavy  burden  to 
bear.  The  next  ten  should  see  the  harnessing  of  radio,  analogue,  and  more  so  digital,  as  the  powerhouse  for 
delivery  of  education.  Governments  should  be  prepared  to  adjust  broadcasting  regulations  to  adhere  to 
technological  developments  and  realities,  and  also  consider  community  based  mass  media  delivery  as  an 
effective  solution  for  improving  a nation's  human  resource  development  towards  the  goal  of  education  for 
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ABSTRACT  The  authors  will  describe  the  programs  used  in  the  preparation  of  faculty  who  are  participating 
in  an  Web-based  Integrated  Curriculum  in  Engineering  (ICE).  The  programs  are  a blend  of  efforts  between 
faculty  and  the  Educational  Technology  Team. 


The  ICE  program  is  a comprehensive  effort  between  engineering  faculty,  humanities  faculty,  physics  and 
mathematics  faculty.  The  ICE  program  includes  specifically  designed  sections  of  the  foundation  courses 
required  of  all  engineering  students:  the  calculus  sequence,  the  physics  sequence,  the  humanities  sequence,  social 
science,  and  introductory  engineering  courses.  What  differentiates  the  ICE  program  from  traditional  engineering 
curricula  is  that  all  courses  incorporate  active  and  collaborative  learning  and  a reliance  on  computer  and  web 
technology.  In  addition,  the  ICE  program  promotes  team  learning,  team  design  projects,  and  a well-documented 
series  of  assessment  practices. 

Background  of  ICE: 

The  motivation  behind  this  program  includes: 

■ high  attrition  rates  in  calculus,  physics,  and  chemistry; 

■ perceptions  of  the  faculty  that  these  students  do  not  read  and  write  well; 

■ a lack  of  comprehension  of  the  building  nature  of  the  foundation  courses  and  their  relationship  to  the 
engineering  curriculum; 

■ the  lack  of  continuous  use  of  common  computer  technology; 

■ failure  of  students  to  adjust  to  the  university  environment, 

■ and  the  isolation  that  many  students  feel  in  such  programs  of  study. 

The  Integrated  Curriculum  for  Engineers  was  introduced  in  1997  to  a volunteer  group  of  65  entering  freshmen 
selected  from  a pool  of  130  initial  volunteers.  In  succeeding  years,  similar  sized  classes  have  volunteered  to 
enter  this  program. 

This  paper  describes  the  programs  specifically  designed  by  the  Educational  Technology  Team  and  the  programs 
put  together  to  teach  faculty  how  to  use  active  and  cooperative  learning  in  the  various  courses  which  are  included 
in  ICE. 


Description  of  ICE  methodology 


Embry-Riddle  Aeronautical  University 
Daytona  Beach,  Florida  32114-3900 


Computer  and  web  technology  are  crucial  to  the  implementation  of  the  ICE  program.  Students  in  this 
program  must  have  laptops  for  use  in  each  course.  Classrooms  have  been  wired  so  students  have  access  to 
the  campus  network  and  access  to  the  Internet  Common  software  in  use  includes  Maple,  Microsoft  Office, 
Bentley  Microstation,  and  Internet  access.  All  courses  are  web-based  so  that  students  have  access  to  each 
course  in  the  program,  to  reference  materials,  tutorials,  reviews  for  exams,  and  to  each  faculty  member  and  to 
their  peers.  Forums,  emails,  chat  rooms  and  links  to  other  sites  are  all  available  to  every  student.  Students 
can  plug  their  laptops  into  any  campus  port  as  well  as  their  dorm  room.  The  authors  will  discuss  examples  of 
daily  classroom  usage  of  a web-course  and  we  will  comment  on  the  active  and  collaborative  nature  of  the 
courses  by  showing  how  these  are  incorporated. 

Another  crucial  component  of  the  ICE  program  is  the  use  of  cooperative  and  collaborative  learning  in  each 
course.  Thus,  in  addition  to  restructuring  the  content,  all  courses  have  been  redesigned  for  an  active  and 
collaborative  learning  environment.  Each  course  assigns  team  exercises,  daily  or  weekly,  which  are  to  be  done 
both  in  and  outside  of  the  classroom.  In  any  of  these  courses,  it  is  common  to  ask  a team  to  present  their  work  to 
their  peers  at  the  end  of  a daily  class.  In  addition,  the  mathematics  faculty  designs  external  team  problems  that 
are  reported  in  class  about  a week  later.  For  each  such  team  problem  it  is  required  that  the  problem  be  written 
using  a word  processor  and  the  report  is  graded  for  composition  and  exposition  as  well  as  the  mathematics. 

Class  presentations  are  oral  and  each  team  receives  a grade  based  upon  accuracy,  style  and  clarity  of  exposition. 

Students  in  the  program  are  assigned  to  a team  of  four.  A team  is  selected  based  on  the  class  rank  condition.  A 
team  stays  together  through  all  the  courses  during  that  semester.  The  engineering  faculty  devote  the  first  week  of 
the  introductory  engineering  course  to  team  building  skills  and  continue  to  monitor  the  effectiveness  of  the  teams 
throughout  the  semester.  This  engineering  course  also  serves  as  a student  success  course  based  on  the  University 
101  model  and  include  topics  such  as  time  management,  study  skills  as  well  as  adjustment  and  orientation  to  the 
university. 


Programs  of  the  Educational  Technology  Team 

Most  of  the  faculty  who  teach  at  Embry-Riddle  use  word  processing,  spreadsheets,  and  email.  A few  of  the  ICE 
faculty  had  some  acquaintance  with  use  of  the  web,  but  none  had  ever  put  a course  up  on  the  web  and  many  were 
apprehensive  about  trying  A template  was  designed  specifically  for  the  web-based  portion  of  the  ICE  program, 
and  faculty  members  were  provided  with  training  on  how  to  develop  and  post  materials  in  the  template. 

Because  the  ICE  faculty  were  technically  proficient  in  specialized  areas  and  working  with  a web  template 
specific  to  their  program,  a customized  training  program  was  developed  for  them  by  the  university  Educational 
Technology  Team.  A description  of  the  ICE  web  template  and  training  program  and  ongoing  support  for  the 
ICE  faculty  will  be  presented  in  this  talk 

Programs  used  to  acquaint  faculty  with  Active  and  Cooperative  Learning. 

When  faculty  think  of  teaching,  they  think  of  lecturing  - the  me-  centered  model  of  education.  Active  learning  is 
participatory  - an  idea  or  theme  is  selected,  the  students  think  about  it,  promote  their  own  understanding,  and 
share  it  with  the  class.  Collaborative  learning  is  group  based  - in  our  case,  team-based.  It  requires  the  team  to 
work  together  in  and  out  of  class  on  assignments. 


Assessment 

Assessment  of  these  training  programs  include  informal  and  formal  components.  In  this  presentation,  the  authors 
will  comment  on  each,  including  what  worked  and  what  did  not.  In  addition,  we  will  comment  upon  some  of  the 
measures  used  by  faculty  in  the  ICE  program  itself  These  include  attitudinal  surveys  about  use  of  teams,  use  of 
active  learning,  use  of  computer  technology,  use  of  computer  software,  use  of  web-based  courses,  adjustments  to 
the  university,  and  a variety  of  other  topics.  Assessment  of  use  of  web-based  materials  is  done  by  formal 
assessment  at  the  end  of  the  term.  All  aspects  of  assessment  will  be  shared  with  the  audience. 
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Abstract:  The  FutureBoard  system  has  been  developed  to  support  the  collaborative  learning  process,  especially 
design  activities  in  Project  Based  Learning. 

Two  kinds  of  board  system  have  been  created:  one  is  a Personal  Board  for  supporting  the  personal  reflective 
thinking  process  based  on  a wireless  note  PC,  and  the  other  is  a Collaboration  Board  for  supporting  collaborative 
learning  and  the  creative  process  based  on  a 50  inch  monitor  with  pen  input  support. 


Project  Based  Learning  and  Design  Activities 

In  Japan,  the  national  curriculum  standards  will  be  changed  in  2002,  and  Project-based  learning  known  as  “The 
period  of  integrated  study”  will  be  introduced  to  elementary  and  secondary  schools.  During  the  process  of  project- 
based  learning,  design  activities  such  as  making  books  and  posters  are  critical  because  they  give  students  the  chance 
to  reflect  on  their  ideas.  Computers  have  been  used  for  the  design  activities  in  project-based  learning,  however  they 
are  limited  to  student’s  individual  expression  when  using  graphic  software  and  DTP  software. 

We  believe  computers  can  render  the  process  more  dynamic  when  designing  activities  such  as  in  collaborative 
design  for  the  social  construction  of  ideas,  by  many  professional  designers  do. 

Design  Process  of  Professional  Designers 

We  have  analyzed  the  design  process  of  professional  interface  designers  (a  university  professor  and  two 
professional  designers  in  a private  corporation)  in  order  to  clarify  points  to  support  the  collaborative  design  process. 

1 . Using  Tracing  Paper 

All  used  tracing  paper  in  the  collaborative  design  process.  One  designer  drew  an  idea  on  paper,  while  the  other  put 
tracing  paper  on  it,  revised  and  added  new  ideas  to  previous  ideas. 

2.  Fast  and  Interactive  Process 

This  process  was  fast-paced  and  interactive.  They  sketched  ideas  in  a few  seconds  and  talked  like  a “This  is  better,” 
pointing  the  revised  idea. 

3.  Using  Personal  Spaces 

Sometimes,  Personal  spaces  were  used  such  as  on  a small  memo  or  a comer  of  the  tracing  paper  to  dot  down  their 
ideas. 

Outline  of  the  FutureBoard  System 

Following  this  analysis,  we  made  design  policies  for  the  new  system  that  supports  the  collaborative  design  process. 
We  named  it  “FutureBoard”. 

1 . Tracing  Paper  Function 

FutureBoard  contains  8 layers  of  tracing  paper  wherein  the  level  of  transparency  can  be  modified.  The  lowest  layer 
is  called  the  “Master  Layer”,  a special  layer  for  guiding  design  activities  such  as  Maps,  Axes,  Tables,  etc. 
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2.  Pen  Input  and  Drawing  Tool 

A user  can  employ  the  pen  input  (or  his/her  finger)  similar  to  ordinary  paper  resulting  in  the  interaction  being  much 
quicker.  Moreover,  the  software  becomes  not  just  a paint  tool  but  a drawing  tool,  so  that  each  object  drawn  can  be 
moved  or  deleted. 

3.  Personal  Board  and  Collaboration  Board 

Two  kinds  of  board  system  have  been  developed:  one  is  a Personal  Board  for  supporting  the  personal  reflective 
thinking  process  based  on  a wireless  note  PC,  and  the  other  is  a Collaboration  Board  for  supporting  collaborative 
learning  and  the  creative  process  based  on  a 50  inch  monitor  with  pen  input  support. 

Personal  Board  and  Collaboration  Board 

The  hardware  of  the  Collaboration  Board  system  (CB)  consists  of  a PC  and  a 50  inch  projection  monitor  with  pen 
input  support  and  a wireless  LAN  station. 

The  software  of  CB  is  a simple  drawing  programme  with  7 half  transparent  layers.  The  user  can  control  the 
visible/invisible  aspects  of  the  specific  layers  as  well  as  control  the  5 levels  of  transparency.  All  operations  are 
assigned  to  the  icons,  thereby  eliminating  the  menu  bar. 

The  hardware  of  the  Personal  Board  (PB)  system  consists  of  a pen-based  PC  and  a wireless  LAN  card. 

The  software  of  PB  is  a subset  of  the  Collaboration  Board  that  has  two  layers  and  is  optimized  for  SVGA  screen. 
(The  CB  is  of  XGA  size) 

Initially,  students  will  use  the  PB,  and  then  send  the  work  to  the  CB’s  layer.  With  the  CB,  they  can  lay  ideas  on  the 
other  half  of  the  transparent  layers.  And  CB’s  data  can  be  downloaded  to  a PB  using  the  wireless  LAN  system. 


FIGURE  1:  Jr.HS  students  using  Personal  Board  and  Collaboration  Board 

Evaluation 

The  FutureBoard  system  was  evaluated  from  November  to  December  of  1999  at  Utase  Junior  High  School  in  Chiba 
Prefecture,  JAPAN.  The  system  was  used  in  project-based  learning  about  “Makuhari  Bay  Town”  where  the  students 
live.  To  meet  the  goals  of  the  curriculum,  Small  groups  were  formed.  The  students  explored  and  researched  many 
aspects  of  their  town,  such  as  the  traffic  system  and  road  safety,  the  soundscape  of  the  town.  They  reported  the  result 
of  their  research  on  the  Personal  Board  and  transferred  the  data  to  the  Collaboration  Board.  Using  Collaboration 
board.  They  discovered  new  ideas  from  the  overlapped  information  and  made  final  presentations  to  an  audience 
composed  of  classmates,  teachers,  municipal  officials  and  parents. 

We  are  now  evaluating  how  these  tools  supported  the  collaborative  process  by  protocol  analysis  and  interviewing. 
The  main  points  of  the  evaluation  center  around  how  8 half-transparent  layers  and  drawing  tools  work  as  a cognitive 
resource  in  collaborative  process. 
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Abstract:  In  this  paper  we  describe  how  elementary  school  students  used  physical 
manipulatives  in  conjunction  with  the  digital  interface  of  educational  software  for  geometry. 
The  blending  of  physical  manipulatives  and  digital  interface  may  help  them  to  overcome  the 
limits  of  the  representation  and  interaction  modalities  of  the  digital  interface. 


Introduction 

Our  study  used  SuperTangrams  (Sedighian  & Westrom  1997),  a program  developed  by  the  EGEMS  group  at  the 
University  of  British  Columbia.  We  introduced  manipulatives  that  students  could  use  while  playing  with  the 
software.  The  idea  was  to  use  manipulatives  in  conjunction  with  the  software  in  order  to  overcome  the 
representational  problems  of  the  interface.  Manipulatives  allow  the  students  to  experience  what  they  are  doing  on 
the  screen  in  the  real  world. 

SuperTangrams.  SuperTangrams  is  a computer-based  learning  environment  for  two-dimensional 
transformational  geometry.  Learning  of  this  mathematical  domain  is  situated  in  the  context  of  the  traditional 
tangram  puzzles  game  activity.  The  player  is  challenged  to  put  together  several  geometric  shapes  to  fill  a given 
outline.  To  move  the  shapes  on  the  screen  the  student  selects  and  applies  to  them  one  of  three  transformations: 
translation,  rotation,  and  reflection. 

Subjects  and  Settings.  The  study  involved  approximately  180  fifth  grade  students  at  a College  Station,  Texas, 
elementary  school.  The  task  of  each  student  was  to  play  with  SuperTangrams  to  solve  the  tangram  puzzles.  The 
student  was  free  to  use  the  tools  that  we  placed  on  the  side  of  his  computer.  Data  collection  techniques  included 
class  observations,  video-recordings,  scratch  paper  used,  pre-tests  and  post-tests,  questionnaires  and  log  files  of 
computer  sessions. 


Use  of  Physical  Manipulatives 

First  set  of  manipulatives 

During  a first  study  session  (two  weeks),  we  provided  the  students  with  the  following  manipulatives: 

1 ) a set  of  tangram  plastic  shapes,  2)  a reflection  mirror,  3)  a colored  grid  board  with  a rotating  stick. 

The  board  had  two  different  drawings  on  it:  a line  for  reflection  and  a circle  divided  in  16  sectors  of  equal  size. 
This  is  consistent  with  the  software  that  allows  the  minimal  angle  to  be  1/1 of  a circle.  A stick  is  connected  to 
a hub  at  the  center  of  the  circle  and  can  rotate  360°.  The  idea  was  that  the  students  would  place  a shape  on  the 
circle  and  then  rotate  it  by  pushing  against  it  with  the  stick.  We  intended  for  them  to  use  the  reflection  mirror 
with  the  line  drawn  on  the  grid  board  to  find  the  reflection  line. 

Rotation.  One  common  use  of  the  shapes  was  to  put  them  on  the  screen  and  animate  them.  A video  recording 
shows  a girl  using  them  for  rotation.  She  started  superimposing  one  plastic  shape  on  the  digital  shape  on  the 
screen,  and  then  she  moved  the  shape  along  the  arc  of  rotation  trying  to  move  and  rotate  it  at  the  same  time.  In 
this  operation  she  paid  attention  to  the  position  of  the  shape  on  the  screen,  but  not  to  the  orientation,  or  angle  of 
rotation.  The  students  who  used  the  grid  positioned  the  plastic  shape  on  the  circular  drawing,  trying  to  position 
the  shape  in  the  same  orientation  as  the  one  on  the  screen.  Then  they  would  push  the  shape  with  the  stick  until 
the  shape  could  reach  the  orientation  of  the  target.  They  would  count  the  number  of  sectors  the  stick  moved  and 
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move  the  digital  handle  accordingly.  Some  students  felt  comfortable  with  the  board,  while  some  others  preferred 
to  bring  the  shapes  directly  to  the  screen. 

Reflection.  Students  did  not  use  the  mirror  on  the  grid  board  with  the  plastic  shapes  as  we  had  expected.  Instead, 
they  placed  the  mirror  directly  on  the  screen  and  looked  for  the  reflected  image  of  the  digital  shape.  The  first 
operation  involved  was  to  position  the  mirror  until  they  could  see  the  reflected  image  superimposed  on  the  target. 
Once  the  position  was  found,  they  had  to  play  with  the  digital  handles  to  position  the  reflection  line  along  the 
position  found  for  the  mirror. 


New  Manipulatives 

The  use  the  students  made  of  the  manipulatives  surprised  us,  since  they  preferred  to  use  them  on  the  screen, 
merging  the  physical  operations  with  the  digital  ones.  Thus,  for  the  second  session,  with  a second  group  of 
students,  we  modified  the  manipulatives  to  make  it  easier  for  the  students  to  use  them  on  the  screen.  We 
eliminated  the  grid  board  and  built  a rotation  tool  consisting  of  two  popsicle  sticks  joined  together  at  one  end  in 
such  a way  that  they  could  rotate,  and  placed  some  sticky  paste  on  the  other  ends.  The  popsicle  stick  had  more 
success  than  the  grid.  The  students  would  stick  the  chosen  shape  at  the  end  of  one  stick,  and  position  the  two 
overlying  sticks  directly  on  the  digital  radius  of  rotation,  with  the  plastic  shape  matching  the  digital  shape.  Then 
they  would  rotate  the  stick  holding  the  shape  and  keep  the  empty  stick  in  the  initial  position.  When  the  plastic 
shape  reached  the  target,  the  opening  of  the  two  popsicle  sticks  represented  the  sought  after  angle.  In  this  case 
they  could  hold  the  right  "angle"  in  their  hand.  Finally,  they  would  recreate  the  digital  angle  playing  with 
handles,  holding  the  physical  one  as  a model. 


Student  reactions 

The  second  set  of  tools  had  more  success  than  the  first.  It  was  more  natural  for  the  children  to  bring  the  tools  to 
the  screen,  and  they  could  physically  experiment  with  the  transformations.  In  the  questionnaires  we  asked 
children  if  it  was  good  to  have  manipulatives,  and  if  they  helped  them  learn.  Here  are  some  of  the  answers: 

“They  helped  because  they  were  3D;^*  **they  helped  you  to  learn  the  movement;**  **all  tools  because  they 
helped  you  think  ahead;**  **you  need  some  reference  to  look  at;**  **because  it  is  important  to  have  stuff  you 
can  work  with  in  your  hand;  **  “it  helps  understand.  ** 


Conclusions  And  Future  Work 

Software  running  on  a computer  is  something  abstract:  when  the  student  drags  a handle  with  the  mouse 
something  happens  on  the  screen.  In  educational  software  for  geometry  it  is  important  that  the  student  learns  to 
relate  what  happens  on  the  screen  to  the  real  word.  Our  preliminary  findings  indicate  that  physical  manipulatives 
can  be  a valuable  addition  to  educational  software.  We  noticed  how  the  use  of  manipulatives  helped  students 
overcome  limits  of  the  representation  and  of  the  types  of  interaction  allowed  by  the  interface.  For  example, 
additional  information  on  the  screen  whose  purpose  was  to  help  the  students,  such  as  additional  lines,  actually 
confused  them,  causing  them  to  miss  important  concepts.  Using  a physical  representation  of  those  concepts,  such 
as  an  angle  between  two  popsicle  sticks  joined  together,  helped  them  to  focus  on  the  main  aspects  of  the 
transformation  without  being  lost  in  representation  details.  During  our  study  we  collected  log  files  and  data 
relative  to  pre-test  and  post-test.  The  statistical  analysis  of  this  data  will  determine  the  effect  on  learning  of  the 
different  tools. 


References 

Sedighian  K.  & Westrom  M.  (1997).  Direct  Object  Manipulation  vs.  Direct  Concept  Manipulation:  Effect  of  Interface  Style 
on  Reflection  and  Domain  Learning.  HCT97  Conference  on  People  and  Computers  XU,  Bristol,  UK.  337-357. 

Acknowledgements 

We  thank  M.  KJave  and  EGEMS  for  their  support,  and  C.  Melick  and  L.  Stewart  for  their  help  with  the  experiment. 


1783 


Public  broadcasting  archives  and  documents  used  for  educational  purposes 


Pierre  C.  Belanger 

Societe  Radio-Canada 
1400,  boul.  Rene-Levesque  Est,  8®  etage 
Montreal  (Quebec),  H2L  2M2 
pierre  c belanger@radio-canada.ca 

Carole  Poirier 

Societe  Radio-Canada 
1400,  boul.  Rene-Levesque  Est,  8®  etage 
Montreal  (Quebec),  H2L  2M2 
carole  poirier@radio-canada.ca 


Contemporary  education  needs  to  be  connected  to  the  very  world  it  purports  to  study.  In  this  context,  media  archives  and 
documents  are  value-added  material  that  can  support  both  traditional  pedagogical  practices  and  media  literacy  related  activities. 
Public  broadcasting  archives  and  related  documents  can  play  an  edifying  role  in  offering  various  sources  through  which  to  grasp 
a host  of  current  and  historical  issues.  Recognizing  the  educational  value  of  its  media  assets,  Radio-Canada  is  working  with 
schools,  colleges,  and  universities  in  exploring  a variety  of  pedagogical  scenarios  in  which  media  content  can  complement 
traditional  study  material.  This  presentation  will  illustrate  how  Canada’s  public  broadcaster  can,  by  means  of  forging  partnerships 
with  various  organizations  (multimedia,  cultural,  or  educational),  and  repurposing  of  media  archives,  play  an  active  role  in  the 
development  of  educational  Web  content  as  part  of  a constructionist  approach  to  learning. 
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Learning  Code 


Suzanne  Rhodes,  Human  Code,  USA 


Who  We  Are 

The  Learning  CodeTM  development  team  extends  beyond  the  walls  of  our  studios  to  include  the  clients  we 
work  with  and  the  learners  we  work  for  as  well  as  a passionate  and  compassionate  team  of  education 
specialists  which  includes  the  following  professionals: 

Educators 

Interaction  Specialists 

Producers 

Artists 

Technologists 

What  We  Do 

At  Human  Code,  our  mission  is  to  humanize  technology  to  fundamentally  transform  the  way  people  learn, 
work  and  play.  Human  Code’s  Learning  Technology  Group  meets  this  challenge  by  designing  spectacular 
learning  environments. 

Our  approach  affords  the  Learning  Technology  Group  with  the  discipline  to  manage  the  complexity  of 
tasks  that  characterize  each  design  stage,  as  well  as  the  flexibility  to  provide  appropriate  consideration  to 
the  six  interrelated  components  that  define  each  of  our  spectacular  learning  environments.  These 
components  include: 

Setting 

Community 

Content 

Pedagogy 

Tools  and  Technologies 
Assessment 

How  We  Do  It 

Derived  from  years  of  best  practices  and  research  in  the  field  of  learning  technologies,  our  Learning 
CodeTM  approach  to  educational  environment  design  is  an  iterative  process  that  motivates  our  team  of 
specialists  through  the  following  comprehensive  design  stages: 

Envision 

Design 

Produce 

Implement 

Integrated,  On-going  Reflection 

Who  Benefits? 

Targeted  learners 

Learning  Code  TM  engages  the  learner  in  an  experience  that  is  personal,  relevant  and  meaningful.  Our 
environments  engender  a love  of  learning,  content  mastery,  higher-order  thinking,  and  inspire  life-long 
learning. 

Clients  and  Educational  Partners 
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Learning  Code’s  collaborative  nature  transforms  clients  into  development  associates.  Skillfully  managed 
projects,  delivered  on-time  and  on-budget  meet  the  needs,  goals,  and  expectations  of  our  client  partners. 

Human  Code 

Each  project  affords  our  company  teams  with  heightened  knowledge,  skills,  and  abilities  that  inform  future 
designs  and  techniques.  The  result  is  continually  evolving  processes,  products,  and  services. 

About  Human  Code 

Human  Code  is  a leading  developer  of  interactive  experiences  for  e-commerce,  games,  learning  systems 
and  marketing  communications.  The  company’s  approach  combines  creativity,  consulting  and  technology 
to  create  user-centered  solutions  that  change  the  way  people  learn,  work  and  play.  Human  Code's  creativity 
and  client-service  focus  are  maintained  through  the  company’s  Studio  Network,  which  includes  Austin 
Studios,  Presage  Studios  in  San  Rafael,  CA,  Human  Code  Japan  in  Tokyo,  and  Human  Code  Shanghai. 

The  company  was  founded  by  industrial  designers  in  Austin,  Texas  in  1 993,  and  received  venture 
investments  from  Austin  Ventures  and  Applied  Technology  in  1995  and  1999.  Additional  information  on 
Human  Code  can  be  found  at  www.humancode.com. 

Human  Code’s  Learning  Technologies  Group  is  emerging  as  a market  leader  in  creating  spectacular 
learning  environments.  From  a statewide  initiative  in  Idaho  to  programs  that  provide  entry-level  technology 
workers  with  training  in  hard  and  soft  skills,  Human  Code  is  becoming  known  as  a true  total  solutions 
provider  for  educational  technologies. 


Teaching  and  learning  in  a virtual  campus: 

The  model  of  the  Universitat  Oberta  de  Catalunya 


Albert  Sangra 
e-Education  Laboratory 
Universitat  Oberta  de  Catalunya 
asangra@campus.uoc.es 

Lourdes  Guardia 

Head  of  Instructional  Design  Unit 
Universitat  Oberta  de  Catalunya 
Iguardi  a@campus.  uoc.es 


By  combining  the  latest  technologies  with  both  a solid  pedagogical  foundation  and  continuous  evaluation,  the 
Universitat  Oberta  de  Catalunya  has  built  a Virtual  Campus:  an  electronic  communications  environment  that 
provides  the  means  to  effectively  support  the  tutoring  of  students,  and  the  provision  of  university  resources  and 
social  activities. 

It  has  presently  a total  of  13.000  registered  students.  Eight  main  degree  programs  are  offered,  as  well  as  Master 
programs,  postgraduate  courses  and  a wide  offer  of  continuous  education  programs. 

The  organization  is  also  working  on  the  Metacampus,  which  mission  is  to  demonstrate  the  global,  virtual  co- 
operation of  universities;  a concept  known  as  the  flexible  university  which  is  achieved  by  interconnecting  the 
partners  virtual  services  and  infrastructures. 

It  will  be  able  to  capture,  register,  invoice,  teach,  assess  and  accredit  courses  and  provide  all  the  administrative 
services  in  a global  scale,  using  the  Internet  as  the  only  medium  to  support  all  these  educational  processes. 
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Deploying  Web-Based  Assessments 

Eric  Shepherd 


Learn  the  principles  of  creating  and  administering  Web-based  tests  and  assessments  using  Question  Mark''^^ 
Perceptions^  software.  Find  out  how  to  use  Perception’s  authoring  wizard  to  create  questions  in  a variety  of 
formats:  multiple  choice,  multiple  response,  hot  spot,  text,  numeric,  selection,  and  matrix  questions  as  well 
as  explanation  screens  that  can  include  text,  graphics,  and  multimedia.  Participants  will  also  learn  how  to 
create  multiple  question  banks  from  which  to  assemble  tests,  surveys,  and  questionnaires.  Other  topics 
include  how  to  set  up  interactive  feedback,  create  adaptive  tests,  analyze  test  results,  and  create  reports.  . 
Example  questions  will  be  presented  from  various  applications  including  competency  testing,  employee 
recruitment,  customer  satisfaction  questionnaires,  study  aids,  diagnostic  tests,  skills  assessments,  product 
knowledge  exams,  course  evaluations,  and  certifications.  The  presentation  will  cover  such  issues  as  test 
security,  linking  with  learning  management  systems,  and  the  use  of  Perception  Secure  Browser  for  high- 
stakes  tests. 
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The  LeamingStation.com 

The  LeamingStation.com  is  dedicated  to  providing  a realistic  and  cost-effective  way  to  manage 
technology  and  make  it  available  to  the  teachers,  students,  and  administrators  who  will  transform 
the  educational  process  to  meet  the  expectations  of  the  information  economy. 

The  LeamingStation.com  is  the  leading  and  largest  Application  Service  Provider  (ASP)  and 
portal  in  the  educational  market,  providing  online  access  to  educational  content,  tools,  and 
applications,  on  a subscription  basis.  With  The  LeamingStation.com,  a managed  and  maintained 
library  of  applications  is  made  available  to  PCs,  Macintosh  systems,  and  thin-client  devices 
across  the  school’s  network  and  over  the  Internet.  The  LeamingStation.com  Chief  of  Innovation 
and  Co-Founder  Jim  Pennington  states,  ''Schools  are  extremely  excited  to  hear  that  they  can  get 
out  of  the  software  management  business,  and  focus  on  using  software  rather  than  delivering  it, " 
says  Pennington,  "and  the  fact  that  they  can  make  good  use  of  their  installed  base  of  cross- 
platform  computers  has  a significant  impact  on  their  ability  to  make  technology  accessible  to 
more  students  and  teachers. " 

The  Education  Market  Opportunity 

Technology  is  now  accepted  as  an  integral  part  of  education  in  Canada.  As  awareness  of  the 
importance  of  preparing  students  for  the  information  age  grows,  school  boards  are  turning  an 
increasing  amount  of  attention  to  this  process.  Educational  spending  now  accounts  for  8%  of 
Canada’s  budget,  and  technology  is  fast  becoming  a priority,  including  significant  investment  in 
hardware,  software,  and  cabling.  In  the  1997  - 1998  school  year,  $230  million  was  invested  in 
bringing  computer  technology  into  Canadian  classrooms. 

Despite  a high  level  of  spending,  most  schools  are  equipped  with  a collection  of  computers 
ranging  from  Apple  II  to  x386  to  Pentium-based  computers.  This  presents  an  implementation 
nightmare.  It  is  currently  impossible  for  schools  to  incorporate  new  software  applications  into 
their  curriculum  without  consistently  upgrading  their  hardware,  a situation  which  is  neither 
desirable  nor  cost-efficient.  Regular  overhauls  of  computer  equipment  is  financially  impossible 
for  schools  with  limited  budgets.  This  issue  is  probably  the  single  largest  baiTier  to  institutional 
technology  integration. 

The  Leamingstation.com  presents  a solution  to  this  problem.  It  has  the  potential  to  completely 
reorganize  instmctional  computing  as  we  know  it.  It  would  allow  schools  to  mn  brand  new 
applications,  as  well  as  provide  web  access  and  e-mail,  while  removing  the  financial  burden  of 
having  to  constantly  upgrade  hardware.  It  would  also  allow  schools  to  use  equipment  donated 
through  programs  such  as  the  Computers  for  Schools  Program,  which  provides  schools  with 
recycled  business  and  government  computers. 

What  is  thin-client  computing? 

Citrix  Systems,  the  world  leader  in  server-based  computing,  reports  over  2,000,000  ports  in  place 
supporting  over  10,000,000  users  today.  The  idea  behind  the  thin  client  is  simple:  concentrate 
computing  power,  storage,  applications,  and  data  on  "servers"  (powerful  computers)  and  provide 
users  with  a simple  "client"  computer  that  is  easy  to  install  and  maintenance-free.  Users  connect 
to  The  LeamingStation.com  servers  over  the  Internet  to  process  applications,  access  files,  print, 
and  perform  all  other 

services  available  to  ordinary  computers.  Since  the  applications  and  data  are  kept  on  a server, 
only  keystrokes,  mouse  clicks  and  screen  updates  travel  the  network,  so  just  a fraction  of  the 
normal  bandwidth  is  required  to  run  applications.  Thin-client  delivery  has  the  flexibility  to 
support  any  device — from  fiill-featured  computers  like  PCs,  Macintosh  systems,  and  UNIX 
workstations,  to  Windows-based  terminals  and  hand-held  information  appliances. 
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How  do  schools  benefit  from  using  an  Application  Service  Provider  (ASP)? 


• Reliability  and  Ease  of  Use 

ASPs  allows  teachers  and  students  to  spend  more  time  working  on  technology  projects  such  as 
accessing  information  on  the  Internet  and  using  educational  software — rather  than  spending  class 
time  fixing  and  upgrading  PCs  or  adjusting  software  programs  to  suit  each  individual 
workstation.  Users  simply  log  on  and  access  their  applications  and  data. 

• Simplified,  Centralized  Management 

The  LeamingStation.com’ s servers  are  managed  by  skilled  network  administrators, 
removing  this  responsibility  from  educators.  Updates  and  additions  are  made  only  once 
at  the  server  and  are  immediately  available  to  all  users. 

• Lower  Cost  of  Technology 

Zona  Research,  a technology  research  firm,  estimates  that  organizations  will  save  54-57%  of  their 
systems  administration  costs  over  a period  of  5 years  with  a thin-client  computing  approach. 
Using  the  services  of  an  ASP  further  reduces  costs  by  eliminating  software  upgrades  and 
inefficient  purchasing  volumes. 

• Security 

Vital  data  and  applications  may  be  kept  on  The  LeamingStation.com  servers  creating  a 
higher  level  of  security.  Sensitive  applications  and  data  are  made  available  to  authorized 
users  only  via  password  protection.  Data  is  effectively  secured  and  backed-up  in  one  central 
location  either  on  a local  customer  server  or  at  The  LeamingStation.com. 

• Universal  Access 

Since  applications  and  data  are  stored  on  the  server,  access  is  available  to  students  and 
faculty  from  any  machine  at  any  location  and  at  any  time  via  an  Internet  connection  and 
the  desktop  appearance  is  always  consistent. 

Strategic  Alliances 

The  LeamingStation.com  has  created  a network  of  strategic  partners  supporting  thin-client 
delivery  of  applications  to  the  education  market.  The  LeamingStation.com  is  continually  looking 
for  new  strategic  partners  to  enhance  it’s  offering  to  education  customers. 

Software  Publishers  and  Online  Content — The  LeamingStation.com  has  strategic 
agreements  to  offer  products  from: 

• Microsoft 

• The  Learning  Company 

• Sunburst  Communications 

• Saratoga  Group 

• Infonautics 

• Schepp-Tumer  Productions 

• CBT  Training 

• Lotus 

• Learning  Outfitters 

• Steck  Vaughn 

• MediaSeek 

• Webivore 

• Fog  Cutter 

• SkillsBank 

• And  many  others 

Technology  Partners — ^The  LeamingStation.com  has  strategic  agreements  with: 
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• Citrix 

• Sun  Microsystems 

• Empower  Corporation 

• MCNC 

o Wyse  Technology 

• Computer  Network  Power 

• Boundless  Technologies 

• Boca  Research 

• OZ  New  Media 

Server  Access  Accounts 

The  LeamingStation.com  customers  use  Server  Access  Accounts  (SAA)  that  allow  access  to  the 
LeamingStation.com  server  farm  via  the  Internet.  LFsers  have  access  to  a basic  package  of 
applications  and  services  that  include: 

• Reference.Center,  a customized  version  of  Infonautic’s  Electronic  Library 

• Adventure  Online,  real  world-based  core  learning  materials 

• Star  Office  Suite,  an  online  suite  of  productivity  tools 

• Facilitator,  conferencing  tools  for  collaborative  learning 

• Internet  filtering  and  virus  protection  utilities 

Additional  products  and  services  can  be  added  as  desired  including: 

• Microsoft  Office  or  Works 

• SkillsBank  or  Cornerstone  basic  skills  software 

• Classroom  Planner,  a comprehensive  teacher  productivity  tool 

• Webivore,  a directory  of  educationally  reviewed  web  content 

• Explorasource,  a curriculum  correlation  tool  for  teachers. 

• Many,  many  more 

Schools  require  a Local  Area  Network  (LAN)  to  provide  users  access  to  The 
LeamingStation.com.  A T-1  line,  or  a fast  broadband  connection  like  high  speed  cable 
modems,  is  recommended  for  school  site  access,  while  an  individual  account  can  gain 
access  over  a modem.  SAA’s  are  for  concurrent  usage,  so  schools  only  need  to  purchase  as  many 
accounts  as  they  will  use  at  one  time.  Access  is  24  hours  a day,  seven  days  a week,  allowing 
teachers,  students  and  parents  to  gain  access  from  home  and  after  hours  as  well. 

Total  Cost  of  Ownership 

Typically,  educational  technology  purchase  decisions  are  made  on  the  basis  of  acquisition  cost 
and  compatibility.  As  technology  usage  becomes  more  pervasive,  educators  will  need  to  pay 
more  attention  to  the  Total  Cost  of  Ownership  (TCO)  which  includes  support  and  maintenance 
costs,  reliability,  and  lifecycle  replacement  costs.  And  in  these  areas,  thin  clients  continue  to  out 
perform  new  PCs  in  cost  effectiveness  by  a wide  margin. 

Zona  Research,  a technology  research  firm,  estimates  that  organizations  will  save  54-57%  of  their 
systems  administration  costs  over  a period  of  5 years  with  a thin-client  computing  approach.  And 
this  cost  savings  is  in  addition  to  the  lower  cost  thin-client  devices  and  ability  to  leverage  your 
current  computing  infrastructure — hardware,  applications,  networks  and  training — to  extend  the 
reach  of  your  thin-client  network. 

A study  by  the  Gartner  Group  released  in  June  1999  estimated  that  thin-client  applications  require 
80  percent  less  management  than  traditional  PC  deployment  (Computer  Reseller  News  Online, 
6/1/99). 


Here  are  some  details  of  the  Gartner  Group  Study  for  annual  PC  cost  of  ownership: 

• Actual  cost  of  the  PC  Three-year  depreciation  $677 

• Non-Labor  Expenses  Software,  network  connectivity,  etc.  $ 1 677 

• Technical  Support  Service,  Maintenance,  Help  desk  $1089 

• Network  Capital  Cabling,  Servers,  etc.  $585 

• Network  Support  Management,  virus  control,  etc.  $544 

• End  User  Admin  Backups,  upgrades,  productivity  loss  $981  from  downtime,  etc. 

Total  Cost  of  Ownership  $6507 

Thin-Client  Computing  Cost  Savings 

While  schools  typically  cannot  provide  the  type  of  support  that  is  described  above,  the 
information  serves  to  illustrate  the  cost  savings  that  thin  client  computing  provides.  In  fact,  the 
lack  of  resources  available  to  support  PCs  makes  an  even  stronger  case  for  thin-client  devices  in 
education.  The  cost  savings  of  thin-client  computing  in  education  are  just  starting  to  be 
quantified: 

• Thin  client  devices  can  have  an  effective  life  of  5 - 10  years 

• Your  current  PC’s  can  be  easily  utilized  as  Thin  client  devices 

• Software  is  a fixed  annual  expense  without  hidden  version  upgrade  expenses 

• Network  support  can  be  provided  centrally  on  servers  instead  of  desktops 

• End  users  have  a common,  easy-to-leam  desktop  environment 

• Downtime  loss  from  viruses,  hardware  failures  and  vandalism  are  all  but  eliminated 

• Universal  access  enhances  productivity  and  leveirlges  the  technology  to  more  users 
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Campus  Notebook  Computer  Programmes 


Craig  Blurton,  University  of  Hong  Kong,  China 


Notebook  computer  programmes  are  proliferating  as  many  universities  move  from  “corporate”  ownership 
of  computing  resources  to  individual  ownership.  Benefits  of  such  programmes  have  been  widely  touted  by 
universities  and  commercial  notebook  computer  vendors  including  enhancement  of  universities  marketing 
position,  decreased  IT  expenses  because  of  cost-sharing  between  institutions  and  students,  extended 
learning  opportunities,  and  enhanced  IT  skills. 

But  how  each  individual  programme  is  implemented  is  highly  contextualized  on  local  goals,  needs,  and 
circumstances.  This  special  interest  group  meeting  will  give  representatives  from  tertiary  institutions  an 
opportunity  to  interact,  share  success  stories,  identify  common  concerns,  and  perhaps  develop  collaborative 
efforts  around  the  issues  of  "assured  access"  and  "mobile  computing." 

Issues  that  may  be  discussed  include: 

* Planning  and  implementing  such  programmes. 

* Multiple  or  single  vendor  solutions 

* Cost-effectiveness 

* Bridging  the  gap  between  academia  and  business 

* Managing  vendor  relationships 

* Equity  of  implementation 

* Staff  support  and  development 

* Barriers  to  success 

* Programme  evaluation 


ERIC 


Providing  Quality  Distance  Education  in  an  Outcomes-Oriented  Program 


Dr.  Ginny  Engman 
Fort  Lewis  College 
52  Hesperus  Hall 
Durango,  Colorado  81301.. 
engman.v@FORTLEWI  S .EDU 


Abstract:  Providing  on-line  programs  for  pre-service  teachers  has  presented  many  issues. 

Assessing  the  effectiveness  of  one’s  teaching  in  the  classroom  is  difficult  under  the  best  of 
circumstances;  doing  so  through  distance  education  is  even  more  challenging.  The  program  at  Fort 
Lewis  College  has  addressed  these  issues  and,  while  resolving  many  of  them,  has  developed  some 
concerns  that  require  further  study. 


Quality  Distance  Education  in  an  Outcomes-Oriented  Program 

The  development  of  an  on-line  program  in  elementary  and  early  childhood  licensure  has  raised  many  issues.  It  has 
also  caused  some  dissension  among  the  faculty.  The  program  consists  of  all  courses  in  the  traditional  program  being 
offered  through  Top  Class  and  Web  CT.  Currently  two  faculty  (of  the  10  full-time  faculty)  have  been  responsible 
for  the  majority  of  the  program  delivery.  There  has  been  an  increase  in  the  number  of  students  who  have  opted  for 
this  avenue  since  its  inception  three  years  ago.  Students  are  admitted  to  the  program  in  the  same  way  that  students 
are  admitted  to  the  on-campus  program:  minimum  gpa  of  2.5,  proof  of  having  worked  with  children,  pass  a state 
basic  skills  test,  pass  and  oral  exam,  and  adequate  ACT/SAT  score. 


Causes  For  Concern 

There  are  three  major  causes  for  concern.  First,  after  an  initial  orientation  meeting  on  campus,  the  faculty  have  little 
or  no  face-to-face  contact  with  the  students.  They  must  rely  on  the  written  evaluations  and  comments  of  the  public 
school  teachers  with  whom  the  students  are  placed.  Historically,  these  assessments  have  been  somewhat  skewed 
toward  the  positive.  In  fact,  the  teachers  are  willing  to  be  more  objective  if  they  do  not  have  to  put  their  observation 
in  writing.  However,  for  legal  purposes  we  need  to  keep  a “paper  trail”.  The  problem  arises  when  we  face  a 
disconnect  between  what  the  teachers  are  writing  and  the  actual  effectiveness  of  the  student  (teacher  candidate).  It 
comes  to  a head  at  the  student  teaching  experience — after  the  student  has  finished  all  course  work  and  pre-student 
teaching  practica.  The  dilemma  we  face  is  that  the  students  are  out  of  our  service  are  (hence,  the  term  distance 
education)  so  that  it  is  not  possible  for  our  faculty  to  observe  and  evaluate  them  on-site,  when  predicted  teacher  How 
do  we  balance  this  with  our  commitment  to  preparing  qualified,  caring  and  competent  teachers? 

Second,  besides  the  teacher  education  courses,  there  is  a need  for  courses  outside  the  department  to  be  delivered  on- 
line. There  are  auxiliary  courses  in  Exercise  Science,  Art  and  Music  that  our  students  need  as  part  of  their  licensure 
program.  Additionally,  if  the  are  seeking  a degree  in  Arts  and  Sciences,  they  will  need  to  take  classes  in  they  major. 
There  has  been  much  resistance  on  the  part  of  faculty  to  develop  an  electronic  delivery  of  their  courses.  How  can 
we  break  through  the  barrier  that  keeps  faculty  from  taking  a risk  in  the  area  of  distance  education? 

Third,  and  lastly,  until  recently,  the  upper  level  administration  has  been  reluctant  (not  necessarily  resistant)  to  invest 
significantly  in  distance  education.  There  is  a skepticism  about  the  ability  to  have  high  quality  and  require  high 
standards  in  an  electronically  delivered  program.  How  can  we  educate  decision-makers  so  that  we  can  get  their  buy- 
in  for  distance  education? 
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How  to  Prepare  Teachers  for  Integrating  Multimedia  and  Technology 

into  K-12  Classrooms 


Barbara  G.  Foster,  Ed.D.  - Spalding  University:  Louisville,  KY  USA  bfoster@spalding.cdii 


Abstract:  To  meet  the  needs  of  diverse  learners,  teachers  must  be  able  to  integrate  multimedia 
and  technology  into  K-12  classrooms.  Participants  will  walk  away  from  this  session  with 
practical  assignments  and  constructivist  activities  to  enhance  professional  productivity  and 
support  instruction,  successful  assessment  strategies  that  promote  positive  attitudes  for 
facilitating  the  lifelong  learning  of  various  technologies  by  teachers  and  students,  and  samples 
of  student  work.  Topics  will  include:  knowledge  of  basic  multimedia  terminology, 
applications,  and  technologies;  development  of  basic  skills  in  operating  multimedia  hardware 
to  support  instruction  and  learning;  development  of  basic  skills  in  using  and  integrating  the 
Internet  and  multimedia  applications  into  the  curriculum  to  meet  the  needs  of  learners  at 
different  age  levels  and  in  different  subject  areas;  experience  in  making  presentations  using 
multimedia  technologies;  experience  in  designing  and  planning  instruction  that  integrates 
multimedia  technologies  into  the  curriculum;  and  awareness  of  the  copyright  laws  as  they 
relate  to  educational  technology. 


Educational  Technology  and  Context:  an  exploration  of  values,  roles, 
and  concerns  from  Lecturers,  tutors  and  instructors. 


Alison  Hudson 

Learning  and  Teaching  Institute, 
Sheffield  Hallam  University 
Adsetts  Centre,  Howard  Street 
Sheffield,  UK,  SI  IWB 
a.r.hudson@shu.ac.uk 

John  Steel 

Learning  and  Teaching  Institute 
Sheffield  Hallam  University 
Adsetts  Centre,  Howard  Stree 
Sheffield,UK,  SI  IWB 
i.a.steel@shu.ac.uk 


Introduction 

Educational  and  instructional  technologies  are  increasingly  becoming  part  of  learning  and 
teaching  in  higher  education  institutions.  There  is  no  doubt  therefore  that  the  adoption  of 
such  technologies  in  this  context  impacts  upon  many  areas  of  university  life.  It  could  be 
argued  however,  that  in  addition  to  significant  benefits  to  the  institution  and  its  students, 
the  increasing  use  of  educational  technologies  raise  a number  problematic  issues  and 
areas  of  uncertainty. 

This  special  interest  discussion  therefore  seeks  to  get  to  grips  with  a number  of  key  issues 
around  the  role  of  educational  technology  in  the  learning  and  teaching  equation  from  the 
perspective  of  the  lecturer/tutor/instructor.  It  is  considered  therefore  that  by  generating 
discussion  around  a number  of  important  areas  of  interest  to  educational  practitioners, 
some  improved  awareness  of  obstacles  as  well  as  examples  of  best  practice  can  be  made. 
From  this  starting  point,  greater  understanding  and  progression  may  be  encouraged  or 
developed. 
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Applications  and  categorization  of  software-based  scaffolding 


Koos  Winnips,  University  of  Twente,  Netherlands;  Catherine  McLoughlin,  University  of  New  England, 

Australia 


The  increasing  popularity  of  computer-based  learning  environments,  combined  with  a need  for  time- 
flexible  and  self-reliant  learning,  has  caused  an  increase  in  the  demand  for  scaffolding  embedded  in 
instructional  software  environments  (see  for  example  Acovelli  & Gamble,  1997;  Guzdial  & Kehoe,  1998; 
Tabak  & Reiser,  1997;  Rada  & Yazdani,  1998).  The  aim  of  software-based  scaffolding  is  to  provide  some 
of  the  same  kinds  of  support  a teacher  could  provide  in  a classroom  setting,  but  now  in  a computer-based 
learning  environment.  In  this  setting  it  is  assumed  that  face-to-face  contact  between  student  and  teacher  is 
reduced  or  impossible.  Software  scaffolding  can  be  defined  by  the  following  three  characteristics  (Guzdial, 
1995,  Zhao,  1997,  Winnips,  1998): 

- Modeling:  the  desired  behavior  is  modeled  by  providing  a kind  of  structure,  communicating  what  is 
desired,  or  presentation  of  an  expert  model. 

- Support  is  given  to  the  learner  so  that  the  learner  can  perform  a task  independently. 

- Fading  takes  place  so  that  students  become  self-reliant. 

While  many  applications  and  examples  of  software  scaffolding  are  now  known,  a categorization  of  these 
examples  has  not  emerged  yet.  Applications  of  scaffolding  can  be  categorized  according  to  who  regulates 
the  scaffolding  (teacher,  peer-student,  computer,  self),  technology  used,  pedagogy  used,  or  according  to  its 
intended  learning  outcome.  An  example  of  a categorization,  based  on  types  of  support  given  is  found 
below: 

- Providing  examples:  ideally  these  examples  should  not  only  focus  on  products,  but  also  on  a process. 
Providing  examples  links  to  the  concept  of  modeling  as  Bandura  (1965)  described  it,  where  a teacher  would 
serve  as  a role  model. 

- Helping  students,  by  giving  away  (physical)  parts  of  the  solution.  This  can  serve  to  help  students  with  a 
solution,  or  the  help  may  just  serve  to  support  students  on  very  time-consuming  tasks  that  are  not 
absolutely  necessary  for  learning  (Acovelli  & Gamble,  1 997). 

- Providing  a model  for  design,  or  a structure  to  design  in.  For  example  providing  design  guidelines  (Collis 
& Winnips,  1998). 

- Cueing^inting:  helping  students  with  a solution  by  providing  a hint  or  cue  to  a possible  path  of  the 
solution  (Acovelli  & Gamble,  1 997). 

- Coaching  comments  (Jonassen,  1998).  These  comments  are  intended  for  motivation,  providing  feedback 
and  advice  on  performance,  and  provoking  reflection. 

- Asking  questions,  pointing  out  weaknesses,  asking  for  a motivation,  in  order  to  enhance  reflection. 

- Metacognitive  support:  stating  why  the  above  types  of  support  are  given,  in  order  to  model  the  type  of 
metacognition  that  experts  would  use. 

- Providing  a timeline,  with  fixed  dates  and  goals  built  in.  This  structure  could  be  present,  to  help  students 
appearring  to  be  very  goal  directed  to  build  in  multiple  evaluation  moments  into  the  actual  experience  of 
studying. 

Aim  of  this  SIG  discussion  is  to  reach  consensus  about  a number  of  categories  for  scaffolding  and  to 
construct  guidelines  about  the  relationship  of  learning  outcomes  to  particular  forms  of  scaffolding.  This  in 
order  to  provide  structure  for  future  research  and  to  be  able  to  better  link  previous  experiences  in  software 
scaffolding  to  future  experiments. 
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WORKSHOP  OBJECTIVES 

By  the  completion  of  the  workshop  participants  should  be  able  to; 

1.  understand  the  processes,  phases  and  activities  involved  in  planning  online  courses  and 
learning  resources 

2.  develop  a persuasive  p.roposal  for  project  funding  including  concept,  rationale,  team 
formation,  learner  profile,  timeline  and  budget 

3.  locate  potential  sources  of  project  funding 

4.  understand  the  pedagogic  principles  of  designing  online  courses  and  learning  resources 

5.  identify  project  learning  and  teaching  strategies 

6.  develop  the  content  for  online  courses  and  learning  resources 

7.  understand  the  legal  issues  involved  with  the  design  of  online  courses  and  learning 


This  tutorial  is  for  faculty  and  staff  developers  who  want  to  fund,  design,  develop,  implement 
and  evaluate  online  courses  and  multiple  media  learning  resources.  The  tutorial  is  set  at  an 
introductory  and  moderate  level. 


resources 
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ABSTRACT 


In  this  tutorial  we  discuss  and  commence  the  crucial  planning  tasks  required  to 

successfully  fund,  design,  develop,  implement  and  evaluate  online  courses  and  learning 

resources.  The  tutorial  will  be  of  most  benefit  to  participants  who  come  with  a specific 
project  idea  they  want  to  develop  further. 

We  discuss  a learner-centred  project  framework  that  consists  of  proposal,  media 
production  and  educational  design  processes,  phases  and  activities.  You  will  use  the 

framework  to  start  developing  a persuasive  proposal  including  the  project  concept, 

rationale,  objectives,  team  formation  and  learner  profile.  You  then  use  the  framework  to 
develop  sound  project  timelines  and  a justifiable  budget.  Potential  sources  of  project 
funding  are  identified. 

Design  principles  for  online  courses  and  learning  resources  are  discussed  based  on  the 
instructional  design  process.  Bigg's  concept  of  'alignment',  and  Laurillard's 
'conversational  model'  of  the  learning  and  teaching  process.  You  will  commence  identifying 
your  project's  learning  and  teaching  strategies.  We  discuss  methods  of  developing  the 
content  for  your  project  and  the  relevant  legal  issues. 


TUTORIAL  OUTLINE 

1.  Introduction  - participant's  projects 

2.  Learner-centered  project  proposal,  media  production  & educational  design  processes, 
phases  and  activities 

3.  Activity  ^1:  develop  project  concept,  rationale,  objectives,  team  and  learner  prof  He 

4.  Timelines  for  online  educational  projects 

5.  Activity  #2:  Develop  project  timeline 

6.  Budgeting  for  online  educational  projects 

7.  Activity  #3'  Develop  project  budget 

8.  Funding  sources 

9.  Design  principles  for  online  courses  and  learning  resources 

10.  Activity  #4.‘  Identify  pro Ject  learning  and  teaching  strategies 

11.  Developing  educational  content 


12. 


Relevant  legal  issues 


13.  Plenary  discussion 

14.  Conclusion  and  tutorial  evaluation 


PRESENTER'S  QUALIFICATIONS  & EXPERIENCE: 

Andrew  Litchfield  lectures  in  Learning  and  Teaching  at  the  Centre  for  Professional 
development,  Macquarie  University.  He  is  responsible  for  supporting  Macquarie  University 
staff  in  course  design,  the  use  of  educational  technology,  and  the  planning,  design  and 
management  of  teaching  innovations  and  learning  resources.  His  academic  experience  also 
includes  lectureships  in  Media  Communications  and  in  Educational  Technology. 

Andrew's  media  company.  Positive  Image,  has  produced  numerous  large-scale  educational 
media  projects.  He  has  extensive  project  management  and  media  design  experience  and  has 
produced  four  award-winning  projects  funded  by  the  Australian  Government  as  'Projects  of 
National  Significance'. 
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Research  & evaluation  of  online  systems  for  teaching  & learning 

Anne  A'Herran,  James  Cook  University,  Australia 

Piecemeal  adoption  of  methods  of  online  educational  delivery  can  duplicate  effort  and  waste  an  institution’s 
resources.  Early  enthusiasm  for  online  delivery  of  courses  soon  fades  without  strategic  initiatives  on  the  institution’s 
part.  In  August  1999  a Web  Educational  Development  Advisor  was  appointed  at  JCU  to  research  and  evaluate 
systems  for  online  teaching  and  learning,  with  a view  to  integrating  the  selected  system  university  wide.  That 
research  and  evaluation  is  the  subject  of  this  paper. 

The  relationship  between  learners'  personality  types  to  their  performance  in  computer- 
mediated  distance  education 

Justin  Ahn,  Fairfield  University,  USA;  MiLee  Ahn,  Hanyang  University,  South  Korea 

Many  distance  education  programs  tend  to  be  focused  on  programs  rather  than  on  individuals  and  their 
characteristics  with  the  goal  of  improving  the  learners’  performance.  More  research  into  the  effects  of  distance 
education  should  be  oriented  toward  the  individual  learners.  Little  is  known  about  the  personal  factors  that  promote 
or  inhibit  success  in  distance  education  environments.  This  study  examined  the  relationship  between  learners’ 
personality  types  and  their  performance  in  computer-mediated  distance  education.  This  study  found  that  Perceiving 
types  posted  fewer  but  longer  messages,  Judging  types  posted  more  but  shorter  messages,  Feeling  types  perceived 
that  they  had  gained  knowledge  more  than  other  types,  and  Sensing  types  were  less  satisfied  with  the  CMC 
experience.  There  were  no  significant  relationships  between  personality  types  to  grades,  level  of  small  group 
collaboration,  and  level  of  leadership  influence.  Students  indicated  that  the  leadership  changed  from  topic  to  topic 
and  learning  in  CMC  can  be  achieved  without  a leader.  They  used  others'  frequency  and  quality  of  postings,  writing 
skills,  and  feedback  to  postings  as  a factor  in  determining  the  leadership. 

Tutoring  Skills  for  Instructors  in  Distance  Delivery 

Mohamed  Ally,  Athabasca  University,  Canada 

The  growth  in  use  of  the  Internet  and  other  telecommunications  technologies  in  education  is  changing  the  role  of 
the  instructor  from  that  of  a provider  of  information  to  that  of  a tutor.  This  new  role  of  the  instructor  requires 
different  skills  to  work  with  students  in  a virtual  and  distributed  environment.  This  session  will  present  the  tutoring 
skills  that  are  required  by  tutors  to  function  in  a distance  delivery  mode. 

Development  of  an  Introductory  Financial  Accounting  Text  in  Print-based  and  Electronic 
Multimedia  Environments 

David  Annand,  School  of  Business,  Athabasca  University,  Canada 

Athabasca  University  is  Canada's  largest  distance-based,  open  university.  Students  presently  registering  in  the 
University's  inuoductory  financial  accounting  course  are  provided  with  a paper-based  textbook,  solutions  manual, 
study  guide,  and  assignment  manual.  These  materials  are  all  developed  and  printed  in-house.  The  800-page  textbook 
was  recently  converted  to  on-screen  presentation  format.  The  amount  of  material  was  significantly  reduced  by 
moving  discussion  cases  to  an  on-line  instructors’  manual.  The  material  was  edited  for  on-screen  use  by  altering 
backgrounds,  screen  breaks,  and  page  size.  The  files  were  then  converted  into  Authorware  Professional®. 
Information  presentation  was  streamlined  and  interactive  elements  were  incorporated.  Feedback  for  selected  text 
problems  was  inserted  at  appropriate  points.  Audio/visual  segments,  two  computer  simulations,  and  navigation  and 
index  systems  were  developed.  The  final  product  will  be  packaged  on  a CD-ROM,  included  with  each  print-based 
course  package  sent  to  students,  and  incorporated  into  the  University’s  Virtual  Teaching  and  Learning  (ViTAL) 
electronic  environment. 

The  Biotechnology  Project:  A Case  Study  on  Integrating  the  Use  of  The  Internet  for 
Research  and  On-Line  Communication  in  a Brazilian  School 

Cristiana  Assumpgao,  Columbia  University,  USA 

This  project  was  developed  in  a model  private  school  in  Brazil,  Colegio  Bandeirantes,  having  the  reputation  for  its 
high  academic  level  and  vision  for  the  future.  Using  an  advanced  topic  in  Biology,  students  were  challenged  to  go 
beyond  normal  academic  activities,  acquiring  skills  to  integrate  technology  and  become  lifelong  learners.  They  have 
had  the  opportunity  to  interact  with  researchers  in  the  field  of  Biotechnology,  as  well  as  with  peers  in  New  York. 
Students  have  learned  to  use  the  Internet  as  a research  tool  as  well  as  a place  for  collaboration  and  content  building. 
This  is  a study  of  the  role  of  technology  in  empowering  students  to  become  active  learners  and  researchers,  as  well 
as  adaptation  to  these  new  methods  of  communication.  Interesting  patterns  have  emerged  on  how  students 
communicate  online  versus  face  to  face.  The  study  continues  on  how  to  create  pedagogically  sound  best  practices 
for  technology  integration. 
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utilization  of  BBS  for  ESL  class 

Hideyuki  Baba,  Keio  Gijuku  Girls  High  School,  Japan 

We  introduced  the  usage  of  a BBS  system  into  ESL  learning  to  enhance  English  reading  and  writing  competence  of 
our  high  school  students.  The  BBS  system  can  enable  our  students  to  learn  English  in  a totally  relaxed  and 
comfortable  way.  They  are  expected  to  find  an  English  file  by  using  a Web  browzer  and  write  down  their  reaction 
on  a page  designed  in  the  BBS.  Students  are  also  expected  to  name  the  URL  on  the  top  of  the  reaction  writing  so 
that  we  can  easily  trace  back  what  kind  of  file  our  students  have  read  and  what  kind  of  viewpoint  they  have  taken. 
The  BBS  system  itself  is  not  so  complicated.  The  program  was  originally  created  by  a Japanese  college  student  and 
made  to  adapt  our  learning  goals  later.  The  good  points  of  this  system  is  (1)  it  is  quite  easy  to  grasp  the  several 
stages  of  developing  process  of  our  studets'  English  learning  only  by  giving  a look  at  the  writing  page,  (2)  students 
can  read  and  write  English  in  a non-stressed  way,  (3)  the  English  the  sudents  are  likely  to  write  is  not  so  intimate  or 
personal,  which  is  one  of  the  disadvantages  of  "pen-pal,  or  mail-pal"  type  of  writing,  (4)  anyone  concerned  can  write 
comments  to  the  writings  so  that  a kind  of  dialogue  is  possible  among  the  people  inside  the  system. 

The  eMINTS  Project:  Enhancing  Missouri's  Instructional  Networked  Teaching  Strategies  - 
• Promising  Developments  and  Projected  Outcomes 

Adam  Bickford,  OSEDA/University  of  Missouri  Extension,  USA;  Bill  Elder,  OSEDA/University  of  Missouri  Extension, 
USA;  Barbara  Hammer,  OSEDA/University  of  Missouri  Extension,  USA;  Patrick  McGinty,  OSEDA/University  of 
Missouri  Extension,  USA;  Priscilla  McKinley,  OSEDA/University  of  Missouri  Extension,  USA;  Susan  Mitchell, 
OSEDA/University  of  Missouri  Extension,  USA 

The  eMINTS  Project  is  an  ongoing  cooperative  initiative  between  the  Missouri  Department  of  Elementary  and 
Secondary  Education  (MoDESE),  the  Missouri  Research  and  Education  Network  (MOREnet)  and  98  elementary 
classrooms  in  44  school  districts.  eMINTS  classrooms  are  provided  with  an  extensive  technological  infrastructure: 
high-speed  internet  connectivity,  enough  student  computers  to  provide  one  computer  for  every  two  students,  a high 
capacity  teacher  workstation  with  video  conferencing  capabilities,  and  a computer  projector  and  a Smart  Board.  In 
addition  to  the  computing  infrastructure,  participating  teachers  receive  extensive  training  in  constructivist,  inquiry- 
based  teaching  methods  which  are  supported  by  the  University  of  Missouri  College  of  Education,  which  provides 
full-time  assistance  in  locating  and  using  on-line  instructional  resources.  Results  from  the  first  wave  of  this  project, 
the  MINTS  Project,  have  demonstrated  that  well  trained  teachers  working  in  technologically  rich  classrooms,  can 
focus  student  interest  and  improve  student  scores  on  measures  of  academic  achievement. 

LEZI:  a Tool  for  Easy  Development  of  Interactive  Video  for  Education 

Mario  A.  Bochicchio,  University  of  Lecce,  Italy;  Roberto  Paiano,  University  of  Lecce,  Italy;  Paolo  Paolini,  Politecnico 
di  Milano,  Italy;  Eiisabetta  Andreassi,  University  of  Lecce,  Italy;  Tiziana  Montanaro,  University  of  Lecce,  Italy 

The  design  and  implementation  of  complex  multimedia  application  is  an  extensive  effort,  requiring  time,  technical 
skills  and  sizable  budgets.  In  few  cases,  therefore,  in  educational  environment  ifs  possible  to  realize  complete 
hypermedia  applications.  It's  true,  nevertheless,  that  within  several  academic  institutions  there  is  the  capability  of 
developing  high  quality  educational  content,  traditionally  delivered  as  paper,  while  it  could  be  more  efficiently 
delivered  in  multimedia  format.  Our  Laboratory  is  devoted  to  development  of  Multimedia  applications,  some  of 
them  concerning  educational  subjects  in  collaboration  with  several  professors  from  the  Humanities.  Many  of  them 
asked  for  developing  their  own  "cultural"  application,  but  may  be  they  don't  have  the  budget,  human  resources  or 
technical  skills  to  carry  on  the  job.  With  this  requirements  in  mind  we  conceived  LEZI,  a tool  that  requires  a 
minimal  computer  expertise  in  order  to  develop  in  a quick  manner  educational  multimedia  application  based  on 
interactive  video  and  electronic  documents. 

New  Views  on  Community  and  Collaboration:  Building  and  Sharing  Dynamic  Worlds  in 
Cyberspace 

Paula  Bonta,  LCSI,  Canada;  Richard  Borovoy,  MIT  Media  Lab,  USA;  Susan  Einhorn,  Consultant,  Canada;  Brian 
Silverman,  MIT  Media  Lab,  USA 

Abstract:  The  Internet  has  the  potential  to  provide  an  unprecedented  dynamic,  interactive  learning  environment  - 
one  in  which  students  from  around  the  globe  can  collaborate  in  the  construction  of  exciting  new  worlds.  Despite 
the  Internet's  capabilities,  there  is  a lack  of  applications  geared  towards  fulfilling  this  need.  This  poster  session  will 
present  a prototype,  the  Kids  Internet  Construction  Kit  (KICK),  that  allows  K-12  students  to  create  and  share 
dynamic  worlds  on  the  Internet. 
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Reeves  Ten-Dimensional  Model:  Application  for  the  Design  and  Development  of  Web- 
Based  Learning  Scenario  on  Learning  Theories 

Caroline  Brassard,  Universite  du  Quebec  a Chicoutimi,  Canada;  Jacqueline  Bourdeau,  Universite  du  Quebec  a 
Chicoutimi,  Canada;  Pauline  Minier,  Universite  du  Quebec  a Chicoutimi,  Canada 

This  poster  presentation  covers  the  design  and  development  of  a media-based  learning  scenario  and  Web 
environment  with  a view  to  implementing  a pedagogical  design  that  integrates  new  technologies.  The  scenario 
responds  to  concerns  regarding  pedagogical  innovation  in  "Teachers  Training".  It  tries  to  facilitate  the  development 
of  the  skills  required  within  the  larger  framework  of  courses  where  theoretical  content  is  high  and  cooperative 
learning  methods  must  be  supported.  The  focus  was  therefore  placed  on  the  ability  to  "make  links"  and  on  the 
socio-constructivist  approach.  The  results  showed  the  importance  of  design,  the  need  for  cognitive  strategies,  the 
inherent  pedagogical  value  of  cooperative  learning  and  developing  the  ability  to  "make  links".  The  creation  of  links 
can  go  beyond  the  context  to  encompass  any  training  session  that  contains  a wide  range  of  theories  and  a broad 
body  of  knowledge.  It  appears  that  the  need  for  a design  that  produces  the  most  appropriate  overall  educational 
formula  may  lead  to  proven  pedagogical  effectiveness. 

The  Creative  Networkithe  development  of  a collaborative  learning  community  for  mature 
professionals 

Helen  Brown,  University  of  Central  England,  Birmingham  Institute  of  Art  and  Design,  England 

The  hypothesis  underlying  the  formation  of  the  Creative  Network  is  that  a radical  approach  to  Continuing 
Professional  Development  in  the  U.K.  is  needed  to  prepare  those  employed  in  the  Creative  Industries  for: 

1.  the  portfolio  career 

2.  short  term  contracts 

3.  rapidly  changing  technologies/ materials 

4.  structural  changes  in  the  companies,  flatter  hierarchies,  featuring  multi-skilling  and  cross  functional  teams 

5.  e-business  (e-commerce  and  e-volution) 

The  Creative  Network  web  site  is  the  gateway  to  a learning  community.  There  is  no  standard  course  or  assessment. 
Instead  the  Creative  Network  tutors  work  with  each  individual  learner  or  company  to  analyse  their  development 
needs  and  structure  a training  plan.  The  Creative  Network  aims  to  facilitate  deep  learning  based  upon  a spirit  of 
facilitation  which  helps  learners  to  contextualise  new  learning  and  construct  personal  relevance  and  meaning. 

Breathing  New  Life  Into  The  Classroom 

Lawrence  Bundy,  Nebraska  Department  of  Education,  USA 

Two  Nebraska  projects,  funded  by  grants  from  the  United  States  Department  of  Education,  will  work  together  to 
prepare  teachers  to  use  technolog)^  in  the  classroom.  The  Connections  Project,  a five-year  Education  Technology 
Innovation  Challenge  grant,  works  with  in-service  teachers.  Introductory,  intensive  five-day  summer  workshops 
focus  on  classroom  uses  of  technology,  brain-compatible  teaching  strategies,  life  skills,  multiple  learning  styles,  and 
integrated  curriculum  models.  On-going  staff  development  follows  the  workshops  at  local  sites  throughout  the 
academic  year.  The  teachers  create  curriculum  units  incorporating  the  strategies  they  have  learned  and  post  them  on 
the  project  Web  site — http://ois.unomaha.edu/connections.  They  also  create  CD-ROMs  of  the  workshop 
presentations,  which  will  be  distributed  to  over  6,000  teachers.  The  Catalyst  Project,  a three-year  Preparing 
Tomorrow’s  Teachers  to  use  Technology  (PTTI)  grant,  involves  seventeen  higher  education  institutions  that  are 
engaged  in  active  systemic  changes  to  prepare  pre-service  teachers  to  use  educational  technology.  The  two  projects 
will  work  together  to  identify  and  develop  a cadre  of  K-12  teachers  who  are  leaders  in  the  use  of  educational 
technology.  This  cadre  will  act  as  resources  in  the  higher  education  classroom  and  as  cooperating  teachers  for 
teacher  education  programs. 

Improving  Collaboration  in  Face-to-Face  Groups  using  Technology 

Robert  W.  Cavenagh,  Jr.,  Dickinson  College,  USA 

Abstract:  Face-to-face  collaborative  work  is  an  essential  tenet  of  much  contemporary  teaching,  especially  within 
constructivist  models.  The  practical  use  of  technology  by  small  intact  groups  has  received  relatively  little  research 
attention.  Observation  of  learners  in  existing  settings  suggests  that  commonplace  computing  facilities  may  limit 
effective  collaborative  work  by  more  than  two  individuals.  The  design  of  facilities  and  strategies  specifically  for 
collaboration  has  been  demonstrated  to  improve  the  effectiveness  of  such  work.  This  PosterDemo  reviews 
applicable  research,  presents  design  and  cost  criteria  for  and  photographic  examples  of  purpose-built  facilities,  and 
delineates  pedagogic  strategies  found  useful,  including  both  assignment  and  assessment  strategies.  It  summarizes 
findings  and  future  topics  to  explore.  It  also  simulates  the  look  and  feel  of  one  such  collaborative  facility. 
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Educational  Applications  of  Hypervideo 

Teresa  Chambel,  University  of  Lisbon,  Portuga 

Video  clips  can  greatly  enhance  the  authenticity  of  a computer  based  learning  environment.  Something 
constructivists  have  been  strongly  arguing  for.  Broadcast  television  does  not  afford  composition  or  the  time  to 
reflect.  But,  television  and  video,  when  properly  constructed,  can  be  a powerful  tool  for  reflection.  If  the  user  can 
select  what  is  to  be  seen  and  control  the  pace  of  the  material,  and  it  is  easy  to  go  back  and  forth,  to  stop,  to  make 
annotations,  to  compare  and  to  relate  to  other  materials.  Effective  reflection  requires  some  structure  and 
organization,  the  main  issues  in  hypermedia.  Hypervideo  refers  to  the  integration  of  video  in  truly  hypermedia 
documents,  taking  into  account  its  spatial  and  temporal  dimensions,  and  defining  the  semantic  and  mechanisms  to 
link  and  navigate  video  and  other  media.  In  this  presentation,  different  mechanisms  for  the  integration  and 
navigation  of  video  in  educational  applications  are  demonstrated.  These  were  developed  using  HTIMEL,  an 
extension  to  HTML  and  existing  Web  tools,  being  developed  and  used  in  the  Unibase  project  on  distance  learning. 

Collaboration  between  T&T3  Project  Team  and  ETU/CeLTS  at  HKIEd  in  Promoting 
Teaching  Skills  and  IT  for  HE 

Thomas  YH  Chan,  Lingnan  University,  Hong  Kong,  China;  Julianne  WY  Wong,  Hong  Kong  Institute  of  Education, 

Hong  Kong,  China;  Elson  Szeto,  Hong  Kong  Institute  of  Education,  Hong  Kong,  China 

We  are  presenting  the  fruit  of  our  collaboration  in  the  context  of  the  T&T  Project  — a publicly-funded  inter- 
institutional  collaborative  initiative  for  all  academic  and  professional  staff  in  Hong  Kong’s  HE  institutions.  Project 
mission  is  to  pull  together  human  and  instructional  resources  from  all  institutions  in  a collaborative  effort  in 
promoting,  fostering  and  advancing  quality  teaching  and  learning.  We  are  going  to  share  with  conference 
participants  the  following: 

• Designing  and  conducting  sessions  on  IT  in  teaching  and  learning  for  academic  and  professional  staff 
from  all  tertiary  institutions  in  Hong  Kong,  in  particular  on: 

• formatting  and  layout  of  text 

• digital  graphics 

• perfecting  presentation  skills 

• other  digital  media  for  enhancing  teaching  and  learning 

• Designing  and  producing  for  colleagues’  use  self-instructional  multimedia  packages  for  enabling  IT  self- 
sufficiency  (in  CD-ROM  format) 

o Initiating  audio  and  video  streaming  for  the  “T&T  Project  Website”  and  related  on-line  multimedia 
packages 

Teaching  Symbol  Recognition  with  Interactive  Java  GUI 

Li  Chao,  University  of  Houston-Victoria,  USA 

Abstract:  The  ease  of  learning  Java,  its  reliability,  and  its  portability  make  this  programming  language  more  and 
more  popular  among  students  and  faculty.  It  has  gained  much  attention  through  its  power  of  multithreading, 
networking,  and  computer  graphics.  As  an  assistant  in  teaching  computer  vision  such  as  symbol  recognition,  java 
has  the  flexibility  needed  to  develop  classroom  demonstrations  which  are  accessible  through  Internet.  The  objective 
of  this  paper  is  to  present  an  interactive  java  based  GUI  to  assist  the  teaching  of  symbol  recognition. 

Kabisa:  A computer-based  training  program  for  diagnostic  reasoning  in  tropical 
medicine 

Geraldine  Clarebout,  Univerisity  of  Leuven,  Belgium;  Jan  Elen,  University  of  Leuven,  Belgium;  Joost  Lowyck, 
Univerisity  of  Leuven,  Belgium;  Jef  Van  den  Ende,  Institute  of  Tropical  Medicine,  Belgium;  Stefano  Lagang  Italy 

A shift  took  place  in  educational  goals  from  knowing  everything  in  a certain  domain,  to  knowing  how  to  deal  with 
complex  problems.  The  reasoning  process  has  become  more  important  than  the  amount  of  information  to 
memorize.  In  medical  education  the  same  evolution  took  place.  A computer-based  training  program  that  is 
developed  to  guide  and  help  to  develop  diagnostic  reasoning  skills  in  tropical  medicine  is  KABISA.  In  this  poster, 
the  program  will  be  presented  and  all  its  functionalities,  as  well  as  a user-test  that  was  performed.  This  test  was  done 
to  identify  whether  KABISA  really  contributes  to  the  development  of  reasoning  skills  in  tropical  medicine.  The 
main  aim  of  this  test  is  to  determine  what  has  to  be  changed  to  the  program  to  provide  students  with  an  efficient 
learning  environment  to  learn  diagnostic  reasoning  and  to  optimize  the  program.  Use  was  made  of  a thinking  aloud 
method  and  log  file  analysis. 
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WEB-LIT:  Web-Based  Support  for  Learning  Theory  and  Instructional  Design 

Ken  Clinkenbeard,  University  of  Bergen,  Norway;  J.  Michael  Spector,  University  of  Bergen,  Norway 

The  primary  objectives  of  our  project  include  creating  an  open  and  flexible  web  site  to  host  materials  on  learning 
theory  and  instructional  design.  We  will  find  and  evaluate  existing  web-based  materials  on  these  topics  which  reflect 
a wide  variety  of  perspectives,  applications  and  lessons  learned  and  organize  the  site  so  that  materials  can  be  posted 
and  easily  accessed.  Interactivity  also  plays  a key  role.  Opportunities  are  provided  for  users  to  comment  on  existing 
materials  and/or  submit  new  materials.  Our  goal  is  not  to  create  an  online  course  in  the  area  of  instructional  design 
of  learning  theory,  but  to  augment  the  design  of  these  courses  by  offering  support  resources.  A secondary  goal  of 
this  site  is  to  create  a web-based  set  of  resources  that  can  provide  a source  of  evaluation  data  on  the  use  of  web- 
based  materials  in  graduate  education. 

Realism  and  Credibility  in  a Simulation-Based  Virtual  Physics  Laboratory  (VPLab):  An 
Empirical  Study 

Marc  Couture,  Tele-universite,  Canada;  Alexandre  Francis,  Universite  de  Montreal,  Canada 

Realism  and  effectiveness  of  computer  simulation-based  learning  or  training  environments  have  been  examined  in 
several  studies.  It  was  shown  that  under  certain  conditions,  simulations  can  be  as  efficient  as  real  experiments,  and 
that  increased  realism  may  result  in  gains  in  ’practical  appreciation'.  However,  few  have  investigated  the  relationship 
between  realism  and  credibility,  or  between  credibility  and  effectiveness.  The  VPLab  is  a simulation-based  learning 
environment  featuring  many  characteristics  and  constraints  normally  associated  with  real  experiments.  These 
include  uncertainty  in  measurement,  random  fluctuation  of  parameters,  and  limitations  in  user  control  over  the 
simulation.  This  approach  distinguishes  the  VPLab  from  most  existing  simulation-based  laboratories.  We  will 
present  first  results  of  an  experimentation  with  first-year  university  science  students,  in  which  we  sought  to  identify 
the  factors,  most  notably  those  associated  with  our  approach  of  realism,  that  may  enhance  the  credibility  of  such  an 
environment  and/or  its  perceived  relevance  as  a tool  for  learning  laboratory  skills  as  well  as  science  concepts. 

A Web-based  Tutorial  on  Optical  Communications 

Ignacio  de  Miguel,  University  of  Valladolid,  Spain;  Carlos  Jesus  Fuertes,  University  of  Valladolid,  Spain;  Abel  Prieto, 
University  of  Valladolid,  Spain;  Patricia  Fernandez,  University  of  Valladolid,  Spain;  Miguel  Lopez,  University  of 
Valladolid,  Spain;  Fernando  Gonzalez,  University  of  Valladolid,  Spain;  Juan  Carlos  Aguado,  University  of  Valladolid, 
Spain;  Ruben  M.  Lorenzo,  University  of  Valladolid,  Spain;  Evaristo  J.  Abril,  University  of  Valladolid,  Spain 

We  are  developing  a tutorial  on  optical  communications  in  order  to  increase  the  interest  of  the  students  in  this  area 
and  to  clear  up  some  concepts.  The  tutorial  is  divided  into  four  didactical  units:  Propagation  of  Signals; 
Components  and  Devices;  Optical  Communication  Systems;  and  Optical  Communication  Networks.  Within  each 
unit,  basic  information  is  provided  as  well  as  simulators  and  animations.  The  students  can  also  perform  self- 
evaluations  to  check  their  progress.  The  self-evaluations  are  randomly  generated  from  the  contents  of  a database 
according  to  the  students’  preferences  (topics,  difficulty  and  kind  of  items:  questions  and/or  short  problems).  The 
tutorial  is  being  developed  at  http://pesquera.tel.uva.es/tutorial.  The  utilization  of  web  technologies  allows  the 
students  to  access  the  tutorial  from  their  homes,  and  we  hope  it  will  facilitate  the  collaboration  with  other  groups 
working  in  these  topics.  This  work  is  partially  supported  by  Consejeria  de  Educacion  y Cultura  (Junta  de  Castilla  y 
Leon). 

Posting  Visual  Cues  on  Threaded-Message  Boards  in  Science  Curricula:  Enhancing 
Transfer  of  Theory  Knowledge  to  Clinical  Practice 

Gregory  A.  DeBourgh,  University  of  San  Francisco,  USA 

Results  of  a pilot  study  (16  baccalaureate  nursing  students)  suggest  posting  thematic,  integrated  photographs, 
graphic,  and  text  data  on  asynchronous,  threaded-message  boards  facilitates  transfer  of  theory  knowledge  to  clinical 
practice  and  supports  the  acquisition  of  skills  in  clinical  reasoning  (pattern  recognition,  critical  analysis,  problem 
solving,  and  identification  of  therapeutic  interventions).  Visual  cues  provide  powerful  representations  of  reality  that 
enable  students  to  recognize  manifestations  and  patterns  of  clinical  pathologies.  Text-based  learning  prompts 
stimulate  reflective  thinking  and  collaborative  problem  solving.  Used  synergistically,  they  create  clinical  context, 
authentic  complexity,  and  scaffold  learning.  Provided  both  words  and  pictures  during  instruction,  learners  encode 
information  in  such  a manner  that  recall  is  facilitated  and  transfer  of  knowledge  enhanced.  Students  report  on-line 
interaction  results  in  the  positive  experience  of  belonging  to  a professional  community  during  learning,  affords 
support  and  opportunities  for  collaboration,  and  increases  access  to  the  instructor  to  clarify  ’’accuracy  of  learning" 
and  provide  feedback.  Using  visually-enriched  on-line  conferencing  is  an  enjoyable,  convenient,  and  satisfying  way 
to  amplify  and  enhance  learning,  and  "brings  the  course  content  alive". 
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The  «BacVert»  or  «Blue  Box»  - A Knowledge  Recycling  Bin  for  Students 

Celine  Desjardins,  Centre  de  recherche  LICEF,  Canada;  Efoe  Wallace,  Centre  de  recherche  LICEF,  Canada;  Claude 
Ricciardi-Rigault,  Centre  de  recherche  LICEF,  Canada 

While  it  has  become  common  to  provide  students  with  a list  of  documents  to  peruse  on  the  Internet,  information 
gathering,  net*surfing  and  annotation  all  have  an  exploratory,  individual  and  work-in-progress  quality.  Sharing 
insights  gained  through  these  activities,  especially  in  distance  education,  is  complicated  and  requires  highly 
structured  settings.  The  INF9002  «BacVerD>  web-based  dispositif  at  Tele-universite  supports  such  sharing.  The 
natural  language  parsing  performed  by  Nomino  provides  flexible  automatic  structuration,  both  of  the  core  Internet 
material  proposed  by  the  teacher  to  which  the  students  gradually  add  their  findings,  and  of  the  comments  about 
these  documents  produced  by  the  students.  By  allowing  a group  of  students  to  constitute  an  evolutive  information 
base,  to  comment  on  it  and  to  compare  notes,  the  INF9002  «BacVerD>  constitutes  a step  towards  tackling  the 
complexity  and  heterogeneity  of  the  information  environment,  while  supporting  exchanges  among  peers  about  the 
material  at  hand. 

Building  Global  Information  Communities:  the  University  of  Iowa  Center  for  Electronic 
Resources  in  African  Studies  (CERAS) 

Barbara  Dewey,  University  of  Iowa  Libraries,  USA 

This  poster  session  provides  a case  study  of  the  issues  and  lessons  learned  regarding  development  of  the  University 
of  Iowa’s  Center  for  Electronic  Resources  in  African  Studies  (CERAS)  (http://sdrc.lib.uiowa.edu/ceras/).  The 
Center  is  a “virtual”  space  for  creating,  disseminating,  and  accessing  scholarly  electronic  resources  in  text, 
multimedia,  and  interactive  format,  and  a “virtual”  place  for  further  development  of  electronic  resources  (textual 
and  multimedia-based)  pertaining  to  Africa  created  by  scholars  from  the  United  States  as  well  as  from  Africa  and 
other  parts  of  the  world  (http://sdrc.lib.uiowa.edu/ceras/).  CERAS  is  part  of  the  University  Libraries’  Scholarly 
Digital  Resources  Center  supporting  students  and  faculty  engaged  in  African  Studies  research  and  teaching  at  the 
University  of  Iowa,  and  scholars  nationally  and  internationally.  CERAS  is  also  a venue  for  U.S.  and  African  scholars 
for  creating  and  disseminating  African  Studies  scholarship  and  information  resources  in  electronic  format.  Several 
CERAS  initiatives  have  focused  on  image  creation,  particularly  in  the  field  of  African  art,  a strength  at  the 
University  of  Iowa.  CERAS  will  serve  as  a venue  for  digitizing  projects  related  to  Africana  scholarship  (i.e.  museum 
collections,  selected  non-copyrighted  collections,  archival  materials,  and  documents).  Other  goals  include  bringing 
high  quality  African  Studies  scholarly  electronic  collections  and  web  sites  currendy  in  existence  into  a user-friendly 
digital  library  web  page;  providing  a venue  for  “publishing”  original  materials,  including  pre-prints,  (articles,  books, 
conference  proceedings)  as  well  as  creating  significant  thematic  textual  and  multimedia  collections  of  materials  in 
electronic  format;  and  developing  an  electronic  international  community  of  Africana  scholars  where  dialogue  about 
issues,  research,  and  projects  can  take  place.  This  latter  feature  is  intended  to  provide  a broader  opportunity  for 
Africana  scholars  to  post  drafts  of  research  papers  and  projects  for  critique  and  discussion  by  other  Africanists  from 
around  the  world.  The  University  of  Iowa  Libraries  is  seeking  collaborative  projects  and  partnerships  with 
institutions  and  Africana  scholars  from  around  the  world. 

Introducing  a Distance  Learning  version  of  a Postgraduate  Program  on  Networking  in 
Argentina 

Dean  Javier  Diaz,  University  of  La  Plata,  Argentine;  Maria  Alejandra  Osorio,  University  of  La  Plata,  Argentine;  Ana 
Paola  Amadeo,  University  of  La  Plata,  Argentine 

The  great  popularity  of  the  Internet  and  the  widespread  use  of  the  World  Wide  Web  are  promoting  a new 
methodology  for  distance  learning.  The  main  advantage  of  the  Internet  based  courses  is  possibility  to  promote  a bi- 
directional communication  between  the  students  and  the  professor  and  to  highly,  motivate  the  student  due  to  an 
adequate  feedback.  This  poster  describe  the  introduction  of  Internet  tools  (such  as  www,  e-mail,  IRC,  FTP,  mailing 
list)  in  the  distance  learning  postgraduate  program  of  the  ‘Magister  de  Redes’  (http:/ /www.linti.unlp.edu.ar/ master) 
of  the  UNLP  . The  ongoing  experience  was  successful  due  to  certain  facts  that  are  stated  in  the  poster  which  also 
mentions  some  drawbacks  and  the  strategy  for  overcoming  them.  The  distance-learning  version  of  the  program 
started  in  1998  as  way  to  reduce  the  obstacles  for  he  students  (professionals  from  the  engineering  and  the  computer 
science  field  in  the  networking  area),  mainly  the  traveling  expenses  and  the  time  it  takes  due  the  large  geography  of 
our  country  and  the  long  distances. 

The  Bones  Of  The  Skull:  Creating  Anatomical  Models  With  Quicktime  Vr 

Marilyn  Dispensa,  The  University  of  Iowa,  USA;  Jim  Duncan,  The  University  of  Iowa,  USA;  Jerry  Moon,  The 
University  of  Iowa,  USA 

This  electronic  demonstration  will  provide  an  overview  of  QuickTime  Virtual  Reality  (QTVR)  and  its  utility  in  the 
development  of  virtual  anatomical  models  which  can  be  used  alone  or  integrated  into  computer-based-learning 
(CBL)  software.  The  Bones  of  the  Skull:  A 3-D  Learning  Tool  is  a CD-ROM-based  module  designed  to  help 
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college-level  anatomy  students  learn  the  bones  and  important  landmarks  of  the  skull.  The  software  has  two  main 
features:  1.  A coUecdon  of  Quicktime  VR  movies  that  allow  the  user  to  rotate  and  view  the  skull  and  its  bones  in 
three  dimensions.  2.  An  interactive  ’’textbook”  that  contains  high  quality  2-D  images,  descriptive  text  and  many 
embedded  activities  to  encourage  interaction  with  the  content.  This  presentation  will  demonstrate  the  product  and 
provide  an  overview  of  QuickTime  VR,  including  technical  requirements  and  hardware/ software  costs.  The  Bones 
of  the  Skull  won  the  1999  Sandoz/Slice  of  Life  Student  Software  Award. 

The  impact  of  establishing  a virtual  university:  a case  study  at  MUST 

Nomusa  DIodIo,  National  University  of  Science  and  Technology,  Zimbabwe 

The  purpose  of  establishing  any  university  is  to  cater  for  the  educational  needs  of  the  surrounding  community.  In 
developing  countries  universities  fail  to  cope  with  large  numbers  of  applicants.  The  reason  is  that  the  number  of 
universities  in  such  countries  is  limited  due  partly  to  historical  circumstances  and  partly  due  to  financial  constraints. 
The  exercise  of  building  brick  and  mortar  universities  and  acquiring  skilled  staff  to  run  them  is  expensive. 

A Distributed  Maritime  Simulation  Training  Environment  based  on  HLA 

Mirko  Dobermann,  Computer  Graphics  Center  ZGDV,  Germany;  Harro  Kucharzewski,  MarineSoft  GmbH,  Germany 

A new  approach  for  advanced  training  systems  is  the  integration  of  computer  simulation  into  such  applications, 
allowing  students  to  experiment  in  complex  scenarios.  The  presentation  introduces  a distributed  simulation-based 
training  scenario  , generated  in  a distributed  environment,  which  that  offers  extended  functionality  for  interaction 
and  demonstration  within  a maritime  training  scenarioapplication.  The  main  objectives  of  the  solution  are  threefold: 
to  set  up  and  establish  a distributed  simulation-based  training  environment,  to  provide  real-time  interaction 
functionality  during  run-time,  as  well  asand  to  integrate  already  available  simulators  that  are  currently  on  the  market. 
The  infrastructure  for  this  simulation  is  based  on  HLA/RTI,  while  which  already  offers  functionality  to  couple 
embedded  applications  with  networked  simulation  applications.  The  Computer  Graphics  Center  in  Rostock 
(ZGDV)  currently  focuses  on  maritime  scenarios  related  to  competency  training  for  crews  aboard  and  ashore.  The 
development  of  the  simulation  environment  is  part  of  a research  project  in  collaboration  with  regional  enterprises  in 
the  Maritime  sector. 

Human  Anatomy  Predissection  Lecture-On-Demand  at  The  National  University  of 
Singapore 

Gilles  Doiron,  National  University  of  Singapore,  Singapore 

The  NUS  Integrated  Virtual  Learning  Environment  (TVLE)  was  developed  so  that  staff  and  students  could  use  this 
information  technology  infrastructure  to  communicate,  exchange  documents  and  information,  discuss,  chat,  and 
access  custom  learning  materials  and  course  related  web  sites.  IVLE  also  enabled  the  university 'to  consider  new 
pedagogical  approaches,  which  would  utilise  its  campus-wide  broadband  ficcess  to  meet  specific  teaching  and 
learning  needs.  A ”lecture-on-demand”  (LoD)  delivery  was  seen  as  a viable  tool  that  would  allow  students  to  take 
more  responsibility  for  their  learning  and  enable  them  to  have  greater  control  over  their  time  schedule.  With  the 
collaboration  of  the  Faculty  of  Medicine,  the  Center  for  Instructional  Technology  and  the  Centre  for  Development 
of  Teaching  & Learning,  a prototype  anatomy  pre-dissection  LoD,  "The  Abdominal  Wall  & Inguinal  Canal",  was 
produced.  This  paper  examines  the  design  and  development  issues  addressed  in  building  the  prototype.  Field  test 
data  on  the  technical  reliability,  and  ratings,  and  comments  from  student  feedback  are  presented. 

ExploraGraph  and  CINEMA 

Aude  Dufresne,  Univ.  of  Montreal,  Canada;  Claire  Isabelle,  Univ.  of  Moncton,  Canada;  Roger  Nkambou,  Univ.of 
Montreal,  Canada;  Yan  Laporte,  Univ.  of  Montreal,  Canada;  Frank  Ferront,  Univ.  of  Montreal,  Canada 

The  ExploraGraph  interface  was  designed  to  facilitate  interaction  in  the  context  of  distant  learning.  It  was 
developed  as  an  alternative  to  simple  web  interaction,  in  order  to  increase  flexibility,  visibility,  and  structure  in  the 
learning  environment.  It  may  be  used  as  a front  end  to  existing  courses  on  the  web  and  to  support  learner  in  them. 
The  ExploraGraph  Navigator  makes  it  possible  to  navigate  through  conceptual  graphs  with  automatic  arrangement 
of  elements,  zoom  and  "fish  eye"  effects.  Each  node  of  the  graph  may  have  a description  attached  to  it  and  may 
give  direct  access  to  an  application,  a document  or  an  Internet  site.  Graphic  structures  may  thus  be  used  to 
represent  the  organization  of  tools,  activities,  concepts  or  documents.  The  Navigator  offers  each  user  a tool  to 
specify  his  goals  and  the  system  can  support  him,  using  multiple  modalities:  Hypertext,  graphical  cues,  Ms  Agents 
avatars,  voice,  visual  demonstrations  and  force  feedback  guiding. 
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Future  Special  Education  Teachers'  Abilities  to  Integrate  New  Technology  Into  Teaching 
Reading  Comprehension 

Catherine  Dumoulin,  Universite  du  Quebec  a Chicoutimi,  Canada;  Jacqueline  Bourdeau,  Universite  du  Quebec  a 
Chicoutimi,  Canada 

This  study  questions  future  teachers’  abilities  to  integrate  new  technology  into  their  teaching.  A pedagogical  scenario 
based  on  a cognitive  approach  was  proposed  to  graduating  students  of  the  bachelor  of  Special  Education  program 
in  order  to  teach  Grade  3 students  with  reading  comprehension  difficulties.  This  pedagogical  scenario  integrated 
new  technology  via  the  "Village  Prologue",  a computerized  environment.  Future  teachers  were  found  to  organize 
pedagogical  activities,  create  learning  situations  suitable  for  teaching  reading  comprehension,  help  motivate  children 
to  read,  and  finally,  guide  children  in  their  learning  processes  within  the  computerized  environment.  Furthermore, 
the  pedagogical  scenario  helped  in  motivating  future  teachers  to  include  new  technology  in  their  future  teaching. 
The  computerized  environment  also  permitted  the  future  teachers  to  adapt  their  teaching  to  the  specific  needs  of 
children  with  reading  comprehension  difficulties. 

School  Festival  on  the  Internet  - Project-based  and  cooperative  learning  - 

Hironori  Egi,  Keio  University,  JAPAN;  Yoshitomo  Tsutsui,  Kamogata  High  school,  JAPAN;  Yuuki  Nishimura,  Keio 
University,  JAPAN;  Keiichirou  Ishibashi,  Keio  University,  JAPAN 

We  report  a case  of  project-based  and  cooperative  learning  activities.  School  Festival  on  the  Internet,  Okayama 
1999  (SF99)  in  Japan.  Three  public  high  schools  in  Okayama,  Keio  University  and  Kurashiki  University  of  Science 
and  the  Arts  participated  in  SF99.  The  goal  of  SF99  was  to  put  the  school  festivals  held  at  each  of  the  high  school 
on  the  web  so  that  the  students  efforts  could  be  widely  appreciated.  School  festivals  are  considered  as  an 
educational  event  in  Japan,  and  we  implemented  SF99  as  a project-based  and  cooperative  activity.  The  festival 
provides  students  with  opportunities  to  present  achievements  of  their  research  or  their  other  interests.  Video 
streams  and  online  communication  through  a Bulletin  Board  System  (BBS)  were  the  main  applications  used  to 
achieve  the  end.  SF99  is  one  of  the  few  examples  were  extra-curricular  activities  such  as  school  festivals  were 
digitalized.  Information-based  learning  and  extra-curricular  activity  have  similar  goals.  We  also  analyzed  the 
achievements  of  SF99  and  discussed  the  educational  effect  of  information-based  learning  using  the  Internet. 

The  ParlEuNet-project:  Problems  with  the  validation  of  socio-constructivist  design 
principles  in  ecological  settings 

Jan  Elen,  University  of  Leuven,  Belgium;  Geraldine  Clarebout,  University  of  Leuven,  Belgium 

Instructional  design  aims  at  generating  indications  about  optimal  relationships  between  learner-related  and 
instruction-related  variables  in  view  of  the  attainment  of  instructional  and/ or  learning  goals.  In  this  poster,  a study 
is  presented  in  which  it  was  started  from  socio-constructivist  view  on  learning  and  instruction  to  identify  a number 
of  design  principles.  These  principles  were  used  to  design  and  develop  a concrete  and  innovative  rich  technological 
leaning  environment,  the  ParlEuNet  learning  environment.  The  main  features  of  the  this  environment  can  be 
described  as  follows: 

• A variety  of  technologies 

• A problem-based  learning  environment  where  use  is  made  of  ill-structured  tasks 

• International  collaboration 

It  was  hypothesized  that  such  an  environment  would  have  an  influence  on  certain  learning  environment.  In  this 
poster  the  environment  and  the  results  of  the  research  will  be  presented,  which  indicate  that  there  is  an  influence  of 
the  learning  environment,  however  in  a different  direction  than  was  expected. 

Examining  Medical  Students’  Attitudes  and  Learning  Experiences  in  BioWorld 

Sonia  Faremo,  McGill  University,  Canada;  Susanne  Lajoie,  McGill  University,  Canada;  David  Fleiszer,  Faculty  of 
Medicine,  McGill  University,  Canada 

This  poster  presents  BioWorld  (Lajoie  et  al.,  1998),  a computer-based  learning  environment  (CBLE)  that  engages 
learners  in  realistic  problem  solving  as  they  attempt  to  diagnose  medical  cases.  Its  design  is  consistent  with  both 
problem-based  learning  (PBL)  and  cognitive  principles  of  instruction.  Solving  a BioWorld  case  involves  interpreting 
case  history  information,  ordering  diagnostic  tests,  developing  diagnostic  hypotheses,  and  reflecting  on  one’s  own 
performance.  As  part  of  a larger  study,  the  data  for  this  poster  consisted  of  measures  of  the  attitudes  and  learning 
experiences  of  medical  students  using  BioWorld.  The  experimental  procedure  involved  having  medical  students 
individually  diagnose  a set  of  BioWorld  cases  and  complete  a questionnaire  concerning  their  experiences.  Students 
rated  several  aspects  of  the  system  (level  of  interest,  difficulty,  utility  of  learning  activities,  etc.).  The  results  have 
implications  for  the  design  of  CBLEs  for  complex  domains.  They  also  suggest  that  BioWorld  is  an  effective  learning 
environment. 
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CyberARTS:  A Integrated  Arts  Curriculum 

Tito  Faria,  Don  Mills  Collegiate  Insititute,  Canada;  Sholom  Eisenstat,  Don  Mills  Collegiate  Institute,  Canada 

The  CyberARTS  program  is  offered  at  a number  of  school  sites  in  Toronto.  CyberARTS  uses  a constructivist 
methodology  to  deliver  visual  integrated  arts,  media  communication,  computer  technology  and  now  Gr.  9 
Geography  credits.  Students  have  constant  access  to  labs  of  Macs  and  PCs  and  a range  of  peripherals.  Teams  of 
teachers  work  coUaboratively  creating  and  delivering  the  curriculum.  Senior  students  complete  the  program  with  a 
co-op  experience.  Their  placements  include:  Alias  Wavefront,  Web  Feat,  Apple,  Microsoft,  the  Design  Exchange 
and  Nelvana.  Our  partners  include:  Waterloo  University,  Apple,  Kodak,  Rogers  Cable  and  Softimage.  Our  senior 
class  was  involved  in  designing  and  producing  the  first  edition  of  “RE:  DESIGN”,  a web  “zine”  about  design  for 
teachers  and  students  by  students,  www.dxnet.net.  Currently  a number  of  senior  co-op  students  have  placements  at 
the  Design  Exchange  where  they  are  working  on  the  website  for  Toronto’s  Olympic  bid.  Our  program  has  won  an 
Apple  School  of  Distinction  Award  and  some  of  its  teachers  have  won  a 1999  Prime  Minister’s  Award  for 
Achievement.  We  will  focus  on: 

• problem  oriented  curriculum  projects  and  units 

• subject  integration  and  authentic  learning  assessment  techniques 

• real-world  connections  and  business  partners 

Managing  Electronic  Resources  - Auto-registration  for  Distance  Learning 

Richard  Fasse,  Rochester  Institute  of  Technology,  US;  Damon  Betlow,  Rochester  Institute  of  Technology,  US;  Randy 
Overbeck,  Rochester  Institute  of  Technology,  US 

Electronic  resources  used  in  distance  learning  are  often  restricted  to  registered  students  through  login  ids  and 
passwords.  As  electronic  resources  proliferate,  so  do  the  problems  with  managing  these  accounts  for  both  students 
and  administrators.  One  option  is  a form  of  auto-registration  with  “course  keys”  that  allows  users  to  select  their 
own  login  id  and  password,  but  restricts  registration  to  only  those  who  received  the  “course  key”  in  a handout  or 
email. 

Interactive  Training  Materials  for  Early  Childhood  Educators:  Cases,  Tools,  and 
Reflection 

Gail  Fitzgerald,  University  of  Missouri-Columbia,  USA;  Louis  Semrau,  Arkansas  State  University,  USA 

Materials  to  assist  early  childhood  educators  in  working  with  young  children  with  behavioral  problems  are  currendy 
in  high  demand.  Multimedia  programs  offer  an  effective  method  for  providing  instruction  based  on  authentic  cases 
for  developing  necessary  knowledge  and  skills.  In  the  multimedia  program,  "Trisha,"  users  can  seek  professional 
knowledge  from  an  information  database,  watch  Trisha  in  multiple  settings,  interview  her  teachers  and  other  care 
providers,  read  case  records,  listen  to  expert  commentators  discuss  theoretical  approaches  and  care 
recommendations,  and  carry  out  a series  of  real-world  activities  related  to  Trisha's  needs.  Following  problem- 
solving activities,  users  engage  in  a series  of  reflective  prompts  to  revtiew  their  decisions  based  on  best  practices.  The 
program  contains  electronic  performance  support  tools  that  can  be  used  within  the  program  as  well  as  in  actual  job 
situations.  "Trisha"  is  designed  for  implementation  with  workers  in  childcare  settings  and  preservice  courses. 
Additional  information  is  available  at  http://www.coe.missouri.edu/~vrcbd/. 

Integrating  computer  ethics  into  the  computer  science  and  computer  engineering 
curricula 

John  Fodor,  Educational  Media  Resources,  Inc.,  USA 

This  demonstration  will  show  how  to  integrate  computer  ethics  into  the  computer  science  and  computer 
engineering  curricula  by  using  the  interactive  CD-ROM  Understanding  Computer  Ethics.  (UCE  was  made  possible 
by  grants  from  the  National  Science  Foundation  and  has  won  AXIEM,  Communicator  and  International  CINDY 
Awards.)  We  will  discuss  such  topics  as  computer  privacy,  computer  security,  ownership  of  intellectual  property, 
software  piracy,  hacking  and  professional  responsibility.  We  will  examine  ways  of  developing  skills,  sensitivities,  and 
understandings  to  "human  value"  issues  and  problems  raised  by  computing  and  information  technologies,  including 
ways  to  elicit  more  reflective  performance  understandings.  We  will  analyze  four  ways  specific  ways  of  integrating 
computer  ethics  into  the  computer  science  and  computer  engineering  curricula: 

1)  as  a stand-alone  class, 

2)  incorporating  computer  ethics  into  existing  cs  & ce  classes, 

3)  as  part  of  a capstone  senior  project,  and 

4)  as  an  independent  study  class. 
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Development  and  Implementation  of  Computerized  Instructional  Materials;  The  Mismatch 

Between  Formal  and  Informal  Technology  Support  Structures  in  a Medical  College 

Cynthia  Frank,  The  University  of  Arizona,  U.S.A. 

This  study  focused  on  the  development  and  implementation  of  instructional  computer  materials  in  the  everyday 
teaching  environment  of  a southwestern  U.S.  medical  college.  What  happened  between  the  time  an  instructor 
voluntarily  decided  to  implement  computet  technologies  in  a course  and  the  time  a student  could  actually  use  the 
technologies?  The  existence  of  formal  technology  support  structures  led  to  the  assumption  that  instructors  would 
utilize  professional  help.  This  turned  out  not  to  be  the  case.  The  majority  of  instructors  used  informal  support 
structures  to  develop  and  implement  their  materials.  Much  of  the  work  was  done  by  students  who  had  no  training 
in  instructional  design,  multimedia  programming,  or  graphic  design.  This  preliminary  finding  indicates  a 
fundamental  mismatch  between  the  type  of  support  professors  selected  and  the  type  of  support  offered  by  the 
institution. 

Putting  Curriculunn  in  Control:  Interactive  Courseware  Development  with  the  TechDisc 

Instructional  Design  Template 

Traig  Friedrich,  Sencore,  Inc.,  United  States;  Bill  Wann,  Sencore,  Inc.,  United  States;  Karen  Korth,  University  of 

South  Dakota,  United  States;  Mike  Hoadley,  University  of  South  Dakota,  United  States;  Dan  Eastmond,  University  of 

South  Dakota,  United  States 

The  Technology  for  Training  and  Development  division  at  the  University  of  South  Dakota  has  joined  forces  with 
Sencore,  Inc.,  a multimedia  application  provider,  in  the  development  of  an  innovative  multimedia  authoring  tool. 
Designed  specifically  to  apply  recognized  instructional  design  techniques  to  "drag-and-drop”  multimedia 
development,  this  tool  puts  interactive  courseware  authoring  into  the  hands  of  the  classroom  educator.  This 
approach  to  multimedia  application  development  benefits  everyone  involved  in  K-12  education  by  enhancing  the 
classroom  teacher's  control  of  course  curriculum  and  specific  course  content.  During  this  demonstration  of  the 
TechDisc  Instructional  Design  Template,  participants  are  welcome  to  engage  in  open  discussions  of  the  tool,  trends 
in  multimedia  project  authoring,  and  other  issues  surrounding  interactive  courseware  development. 

Using  Virtual  Reality  to  Assist  Air  Travelers  With  Disabilities 

Clark  Germann,  Metropolitan  State  College  of  Denver,  USA;  Jane  Broida,  Metropolitan  State  College  of  Denver,  USA 

Getting  to  the  airport,  checking  our  luggage,  going  through  security,  boarding  the  plane— these  are  all  tasks  most  of 
us  take  for  granted  each  time  we  fly.  But  to  persons  with  physical  disabilities  who  use  a wheel  chair,  the  above  tasks 
can  be  difficult  and  intimidating.  No  two  airports  are  exactiy  the  same,  and  procedures  for  air  travelers  with 
disabilities  differ  with  each  air  carrier.  To  help  ease  the  anxiety,  faculty  and  students  at  Metropolitan  State  College  of 
Denver  have  used  computer  technology  to  duplicate  one  major  airport  into  virtual  reality.  Now,  future  air  travelers 
can  preview  the  experience  first  on  the  computer  prior  to  going  to  the  airport  in  person.  With  .135,000  people  in 
wheelchairs  in  Colorado  alone,  provision  of  resources  for  air  travel  is  especially  pertinent.  Survey  research  and 
anecdotal  evidence  has  shown  that  these  virtual  representations  can  be  helpful  in  reducing  anxiety  in  persons  with 
disabilities  by  providing  a preview  of  a location  prior  to  visiting  it  in  person. 

Gender  and  Race:  The  Accuracy  of  Internet  Clip  Art 

Carol  Gilley,  University  of  Arkansas,  USA;  Elaine  Terrell,  University  of  Arkansas,  USA 

Abstract:  Computers  are  being  used  to  create  various  print  media  as  well  as  multimedia.  Some  questions  are  being 
raised  in  developing  these  media  materials.  Is  there  a supply  of  free  clip  art  on  the  Internet  that  represents  the  real 
world?  Do  viewers  see  themselves  in  clip  art  used  in  print  and  computer-based  multimedia?  This  study  explores  the 
issues  of  equity  and  stereotyping  with  regard  to  the  images  in  free  Internet  clip  art  by  investigating  the  following  two 
hypothesis.  1)  There  is  no  significant  difference  between  the  proportion  of  men,  women,  boys,  and  girls  represented 
in  free  Internet  clip  art  and  the  proportion  of  men,  women,  boys,  and  girls  in  the  general  population.  2)  There  is  no 
significant  difference  between  the  proportion  of  White,  Blacks,  and  other  minorities  represented  in  free  Internet  clip 
art  and  the  proportion  of  White,  Blacks,  and  other  minorities  in  the  general  population. 

How  To  Be  Innovative  Designing  Educational  And  Interactive  Environments  For  Children 

Gloria  Elena  Gomez  Escobar,  University  of  Los  Andes,  Colombia 

LUDOMATICA  PROJECT  looks  to  effectively  attend  children's  educational  needs.  It  seeks  to  help  them  to 
creatively  think  and  act,  validating  their  rights  to  high-quality  education  and  to  actively  participate  as  social  change 
agents.  This  validation  is  accomplished  through  innovations  in  formal,  non-formal  and  informal  education,  using 
Information  and  Telecommunication  Technologies  within  a non-conventional  pedagogical.  We  want  to  develop  the 
intellectual  potential  and  cognitive  abilities  of  chUdren  who  live  in  high-risk  situations,  taking  advantage  of  their 
adverse  learning  to  stimulate  creative  potentials,  communicative  resources  and  have  them  discover  their  own 
adventurous  spirit  and  creative  capabilities.  The  interactive  game  THE  FANTASTIC  CITY  is  one  of  the 
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environments  created  for  these  purposes.  This  is  a digital  multimedia  microworld  that  modvates  and  develops 
children's  fantasy  and  creadvity  by  means  of  enigmas  and  pu2zles,  seeking  to  generate  cognidve  and  affective 
imbalances.  This  interacdve  environment  leads  children  to  acdon.  It  promotes  curiosity  and  experiendal  learning;  it 
allows  speculadng,  discovering,  learning  from  mistakes;  and  it  sdmulates  the  desire  to  learn. 

Teaching  Dynamics  of  Dairy  Herd  Health  and  Management 

Claudia  Haferkamp-Wise,  Cornell  University,  USA;  Heather  G.  Allore,  Cornell  University,  USA;  Yrjo  T,  Grohn,  Cornell 
University,  USA;  Lorin  D.  Warnick,  Cornell  University,  USA 

Cornell  University  offers  "Dynamics  of  Dairy  Herd  Health  and  Management" 
(http://courseinfo.cit.cornell.edu/courses/vetmed745)  - a newly  designed  web-based  distribudon  course  to  its  on- 
campus  veterinary  students  and  to  students  at  9 off-campus  test  sites  within  the  United  States  and  abroad.  This 
course  uses  a computer-based  instrucdonal  delivery  system  (DairySim)  to  teach  basic  concepts  of  epidemiology  and 
economics  of  dairy  catde  disorders,  experimental  design,  and  stadsdcs.  The  courseware  provides  two  simuladon 
models  - SIMHEALTH  and  SIMMAST  - with  an  interacdve  user-friendly  interface  and  output  analysis  - thus 
making  research  tools  accessible  to  a broader  audience.  DairySim  and  its  accompanying  user  guides  are  available  on- 
line, Lectures  can  be  attended  in  person  on-campus  or  viewed  via  the  internet.  Students  can  print  handouts  for  use 
in  computer  laboratory  sessions,  take  on-line  quizzes  and  download  homework  assignments.  Completed 
assignments  are  handed  in  by  uploading  the  documents.  Instructor-student  and  student-student  communicadon  for 
off-campus  students  takes  place  via  email  and  an  on-line  discussion  board. 

A Comprehensive  Model  for  Improving  Technology  in  Teacher  Education 

Robert  (Bob)  Hannafin,  College  of  William  and  Mary,  US;  Robert  (Bob)  Hanny,  College  of  William  and  Mary,  US, 

This  paper  idendfies  the  professional  development  model  developed  at  the  College  of  William  and  Mary  to  close  the 
gaps  between  its  strategic  vision  for  the  teacher  educadon  program  and  its  capacity.  The  College's  vision  is  to 
produce  teachers  who  seamlessly  integrate  technology  by  designing  and  assessing  authendc  student-centered 
learning  acdvides.  Three  areas  were  identified  as  requiring  parallel  effort:  faculty  development,  institudonal 
development,  and  clinical  faculty/PDS  development.  Goals  were  established  that  addressed  capacity  shortfall  and 
advance  the  program  to  the  point  where  significant  innovadon  could  occur.  Both  our  students  and  faculty  needed 
to  master  both  basic  technology  skills  and  the  subder  art  of  pracdcing  creadve  integration  strategies.  It  also  required 
placing  our  students  with  teachers  who  are  competent  technology  users  and  encouraging  in  technologically-adequate 
classrooms.  Insdtudonal  changes  were  necessary  on  two  fronts:  upgrading  the  technology  available  in  School 
classroom.s,  and  insdtudng  incendves  to  endce  faculty  to  use  and  integrate  technology. 

Web-Based  Tools  For  Courses  On  Transport  Phenomena 

Bernardo  Hernandez-Morales,  Universidad  Nacional  Autonoma  de  Mexico,  Mexico;  Rafael  Fernandez-Flores, 
Universidad  Nacional  Autonoma  de  Mexico,  Mexico;  Jorge  Tellez-Martinez,  Universidad  Nacional  Autonoma  de 
Mexico,  Mexico 

Even  though  muldmedia  applicadons  have  been  extensively  used  in  many  courses,  the  field  of  transport  phenom- 
ena has  received  comparadvely  less  attendon.  Thus,  we  are  developing  interacdve  tools,  to  be  accessed  through 
Internet,  to  solve  problems  observed  when  tradidonal  teaching  techniques  are  used.  The  material  consists  of 
modules  to  solve  typical  problems  in  transport  phenomena.  Each  module  is  a Webpage  that  includes  the  theo- 
retical background  (with  hyperlinks  to  all  course  material)  and  an  interacdve  worksheet  to  solve  a particular 
problem.  The  worksheets  are  built  with  Java  applets  QDK  1.1.  compadble)  and  may  may  be  used  in  any  of  three 
modes:  example  mode  (fixed  set  of  values),  training  mode  (variable  set  of  values),  and  exploradon  mode  (range  of 
values  of  a key  independent  variable).  Thus,  unlike  problems  found  in  tradidonal  textbooks,  the  student  may 
explore  a range  of  values  for  key  independent  variables  to  establish  its  influence  on  the  dependent  variables. 

Shaknoma:  A collaborative  tool  for  shared  knowledge  management 

Oriel  Herrera,  Pontificia  Universidad  Catolica  de  Chile,  Chile;  Sergio  Ochoa,  Pontificia  Universidad  Catolica  de  Chile, 
Chile;  Vidal  Rodriguez,  Pontificia  Universidad  Catolica  de  Chile,  Chile;  David  Fuller,  Pontificia  Universidad  Catolica 
de  Chile,  Chile 

Nowadays,  the  amount  of  scienrific  knowledge  produced  is  overwhelming  and  it  is  clear  that  it  will  grow  even  more. 
Shaknoma  (Shared  Knowledge  Manager)  is  a tool  useful  to  make  available  and  share  the  people's  individual 
knowledge.  The  activities  carried  out  on  this  tool  respond  to  a model  that  ensure  the  input,  validation,  use  and 
evolution  of  the  knowledge.  The  process  is  implemented  through  four  major  components:  the  shared  repository, 
the  management  model  (see  Herrera  et  al.  2000),  the  representation  language  of  the  knowledge,  and  the 
collaborative  environment.  The  knowledge  management  model  provides  decision-making  mechanisms,  which 
mainly  includes  discussion  and  voting  activities.  As  a way  of  structuring  and  graphic  representation,  Shaknoma  uses 
a language  based  on  conceptual  maps  (Novak  1984).  The  information  repository  does  not  store  only  knowledge,  but 
also  the  information  of  its  use  and  evolution. 
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Teachers  and  Computer  Technology  Training 

David  Hofmeister,  Central  Missouri  State  University,  USA;  Gerry  Peterson,  Central  Missouri  State  University,  USA 

The  poster  session  provides  an  overview  of  the  processes  and  materials  used  to  advance  the  integration  of 
computer  technology  into  student  learning  in  K-12  schools. 

Results  and  conclusions  from  a Learning  Software  Design  Competition 

Brad  Hokanson,  University  Of  Minnesota,  USA;  Simon  Hooper,  University  Of  Minnesota,  USA;  Paul  Bernhardt, 
University  Of  Minneosta,  USA 

The  University  of  Minnesota’s  Design  Institute  is  sponsoring  a competition  to  recognize  the  design  of  innovative 
educational  software.  The  competition  was  open  to  professional  and  not-for-profit  educational  software  designers 
in  such  areas  as  higher  education,  K-12  education  and  commercial  training.  Prizes  will  be  given  in  each  category, 
and  entries  for  the  competitions  were  due  February  15,  2000.  Winners  of  the  competition  were  announced  April  10, 
2000.  One  of  the  competition’s  goals  is  to  establish  a bank  of  exemplary  educational  software  that  can  be  examined 
by  designers  and  students  of  educational  software  design.  To  facilitate  this  goal,  winning  designs  (and  others 
selected  for  inclusion)  will  be  compiled  and  made  available  to  the  public.  We  will  present  the  results  of  the 
competition  in  a workshop  format  at  ED-MEDIA  2000.  The  intended  audience  is  educators  and  practitioners  in 
the  field  of  educational  technology. 

Constructive  Hypertext  for  Learning 

Hsi-chi  Huang,  The  Ohio  State  University,  USA 

This  study  focuses  on  teachers  as  designers  using  constructive  hypertext  and  teachers'  perspectives  on  the  use  of  the 
World  Wide  Web  in  educational  contexts  and  its  implications.  Constructive  hypertext  encourages  the  participants 
not  only  to  be  users  of  hypertext,  but  also  to  be  designers  of  hypertext.  The  study  describes  the  participants’ 
experiences  as  designers  and  their  thoughts  on  designing  hypertext  for  learning.  In  designing  Web-based  hypertext 
learning  environments,  educators  can  make  use  of  the  Web  from  three  approaches:  the  Web  as  a resource  for 
information,  the  Web  as  a medium  of  expression-representation,  and  the  Web  as  a space  for  dialogue.  The 
distribution  perspective  takes  advantage  of  the  Web  as  an  information  network.  The  expression-representation 
perspective  uses  the  Web  as  a space  for  publishing,  and  expressing  ideas  and  creativity.  The  Web  as  a space  for 
dialogue  has  the  capacity  to  encourage  and  embrace  different  voices.  To  participate  in  the  dialogue  is  to  create 
voices.  The  dialogue  space  provides  possibilities  for  students  to  converse  and  exchange  learning  experiences. 

Internet  Infrastructure  with  ATM  for  Small  Countries;  Case  Studies 

Necdet  ICIL,  Eastern  Mediterranean  University,  TURKIYE 

Rapid  development  on  communication  technologies,  become  easy  to  support  QoS  (Quality  of  Services)  application 
on  Internet.  As  a case  study.  North  Cyprus  which  is  a small  country  with  5 small  cities,  develop  a project  to  connect 
all  cities  with  ATM  BackBone.  The  purposes  of  the  project  is  to  connect  all  the  local  government  offices  to  the 
cities  node  and  to  use  all  multimedia  and  information  facilities  on  the  BackBone.  All  governmental  documentation 
will  be  send  electronically  between  the  offices.  Virtual  meeting  will  be  possible  by  using  video  conferencing 
facilities.  Also  all  schools  will  connect  to  the  Ministry  of  Education  to  use  educational  facilities  which  are  prepared 
by  the  trainers. 

NACSIS-ILL  Distance  Training  System  in  IDLE  Distance  Learning  Project 

Tomo'o  Inoue,  National  Institute  of  Informatics,  Japan;  Hamki  Ueno,  National  Institute  of  Informatics,  Japan 

We  have  started  the  distance  learning  project  named  IDLE  (Integrated  Distance  Learning  Environment)  in  1998. 
The  primary  motivation  of  beginning  the  project  is  to  meet  the  social  needs.  There  are  not  many  practical  and 
sustainable  DL  systems  in  Japan,  while  on  the  other  hand  DL  is  already  recognized  as  a natural  form  of  getting 
education  in  the  US.  Current  core  product  of  IDLE  is  NACSIS-ILL  Distance  Training  System,  a WWW  based 
system  for  training  NACSIS-ILL  operators.  NACSIS-ILL  is  a system  that  our  institute  offers  to  nearly  700 
university  libraries  to  support  exchange  of  information  for  the  inter-library  loan  service.  Our  institute  also  offers 
NACSIS-ILL  Training  Courses  to  train  the  staff  in  the  inter-library  loan  sections  of  the  participating  libraries.  We 
have  put  emphasis  on  its  practicality  in  the  project  and  applied  DL  to  the  training  course.  The  system  will  be  in 
actual  use  this  year. 
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Linking  Active  Learning  to  Web-based  Instruction:  Students  Teaching  Students  through 
Multimedia  Productions 

James  S.  Javenkoski,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at  Urbana-Champaign, 
USA;  Elizabeth  F.  Reutter,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at  Urbana- 
Champaign,  USA;  James  E.  Painter,  Department  of  Food  Science  and  Human  Nutrition,  University  of  Illinois  at 
Urbana-Champaign,  USA 

Instructors  (academic  teachers  and  corporate  trainers)  recognize  the  value  of  video  as  an  educational  medium. 
Video  provides  learners  (students  and  trainees)  with  opportunities  to  observe  phenomenological  examples  that 
reinforce  the  concepts  that  are  verbally  described  during  classroom  and  laboratory  sessions  and  on-the-job-training. 
Until  recently,  educational  video  production  was  a multi-step,  multi-component  process  requiring  at  the  minimum  a 
camera  or  VCR,  a computer  (with  a high  speed  processor,  very  large  hard  drive,  and  video  editing  and  processing 
software  installed),  and  a high  speed  serial  I/O  connection  to  transfer  the  video  data  between  hardware  devices. 
The  new  Apple©  iMac^^  DV  desktop  computer  and  iMovie^^  software  offer  a simple,  accessible,  and  affordable 
solution  to  instructors  who  wish  to  create  continuous  media  (audio  and  video)  for  use  in  their  courses.  Our 
demonstration  will  showcase  the  QuickTime  streaming  videos  produced  in  UIUC’s  FSHN  199:  Business  Etiquette 
and  Protocol  and  FSHN  355:  Fine  Dining  Management  courses. 

Using  Tailored,  Interactive  Soap  Operas  for  Breast  Cancer  Education  of  High-Risk 
Hispanic  Women 

Maria  Jibaja,  Baylor  College  of  Medicine,  USA;  Paul  Kingery,  Hamilton  Fish  Institute,  USA;  Nancy  Neff,  Baylor 
College  of  Medicine,  USA;  Quentin  Smith,  Baylor  College  of  Medicine,  USA;  Jennifer  Bowman,  USA;  J.  David 
Holcomb,  USA 

While  Hispanic  women  have  lower  rates  of  breast  cancer  than  other  groups,  among  women  from  all  ethnic  groups, 
they  are  the  least  likely  to  undergo  screening  exams.  This  study  evaluated  a culturally  sensitive  and  linguistically 
appropriate  tailored  computer-based  educational  program  on  breast  cancer  early  detection  aimed  at  high-risk 
Hispanic  women.  Spanish-speaking  Hispanic  women  from  an  inner-city  community  health  clinic  were  recruited  and 
randomly  assigned  to  either  a computer  intervention  with  an  interactive  soap-opera  format  (n=118)  or  a 
comparison  group  (n=60).  True-false  pre-  and  post- tests  were  used  to  identify  any  change  in  breast  cancer  related 
knowledge/beliefs.  Both  younger  (18-40  y,o,)  and  older  (41-65  y.o.)  women  in  the  intervention  group  increased 
significantly  their  knowledge/beliefs  as  compared  to  the  younger  and  older  women  in  the  comparison  group 
(p<.05).  Computer-based  tailored  and  interactive  soap  operas  that  are  linguistically  and  culturally  appropriate  are 
effective  in  increasing  breast  cancer  screening  knowledge/beliefs  in  high-risk  Spanish-speaking  Hispanic  women, 

webStract  - a Distributed  Tool  for  Collaborative,  Project-driven  Learning  on  the  Internet 

Werner  B.  Joerg,  NetEssence  / U of  Alberta,  USA 

webStract  is  a combined  software  tool  that  supports  the  construction  and  delivery  of  Internet  based  courseware.  It 
embodies  a project-driven  paradigm  - a problem  based  learning  approach  guided  by  the  needs  of  student  projects.  It 
allows  content  providers  to  make  effective  use  of  the  vast  knowledge  source  of  the  World  Wide  Web  and  to  present 
the  students  with  qualified  material  in  a structured  manner.  It  takes  advantage  also  of  the  interaction  capabilities  of 
the  Internet,  to  enable  focused  synchronous  and  asynchronous  communication  among  distributed  teams  and  team 
members.  Following  a brief  introduction  of  the  motivation  and  the  learning  paradigm,  our  presentation  centers  on 
the  three  principal  areas  of  service  covered  by  webStract:  knowledge  qualification  and  structuring  tools  for  the 
content  provider,  collaboration  and  management  tools  for  the  teams,  and  knowledge  management  tools  for  the 
individual  student.  webStract  is  currently  being  used  in  experimental  courses.  We'll  report  feedback  from  those 
experiments. 

Digital  Learning  System:  Web's  Cool 

Myunghee  Kang,  Ewha  Womans  University,  Korea 

Web's  Cool  (can  also  be  pronounced  Web  School)  program  with  SRL(Self-Rcgulated  learning)  principles, 
PBL(Problem  Based  Learning)  approaches  and  Keller’s  ARCS  model  for  motivation  has  been  designed  by  the 
author  and  developed  by  Samsung  Electronics  Company  in  Korea.  Now,  all  the  contents  and  study  modules  are 
completed  in  CD-ROM  and  the  total  service  including  Q&A  and  real  tutorial  help  is  now  available  on  the  UNITEL 
web  service. 
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Learning  Amidst  A Sea  Of  Information  In  The  New  Millennium 

Dan  Kauwell,  University  of  Illinois,  Urbana-Champaign,  USA;  Jim  Levin,  University  of  Illinois,  Urbana-Champaign, 

USA;  Daniel  Schiff,  University  of  Illinois,  Urbana-Champaign,  USA;  Young-Jin  Lee,  University  of  Illinois,  Urbana- 
Champaign,  USA 

Visualization  can  improve  our  ability  to  access  information  and  construct  knowledge.  Information  visualization  is 
particularly  well  suited  for  the  location  and  analysis  of  information  found  on  the  Internet  and,  for  the  construction 
of  knowledge  from  that  information.  Our  research  has  led  us  to  develop  VisIT,  a tool  for  the  visualization  of 
Internet  based  information  and  the  analysis  of  that  information.  Instead  of  lengthy  lists  of  search  results,  the  user  is 
presented  with  a graphical,  spatial  representation  of  the  search  space.  Now  the  user  can  ’’see”  the  hits  returned  by 
the  search  engine  as  well  as  other  pages  from  the  same  site.  When  any  of  the  pages  are  clicked,  the  appropriate  page 
is  displayed  in  the  browser  window.  Furthermore,  VisIT  facilitates  the  knowledge  construction  process  by  allowing 
its  graphical  displays  to  be  edited,  saved  and  re-opened  later. 

An@tomedia:  A New  Approach  TO  Medical  Education  Developments  In  Anatomy 

David  M.  Kennedy,  Monash  University,  Australia;  Norm  Eizenberg,  The  University  of  Melbourne,  Australia;  Chris 
Briggs,  The  University  of  Melbourne,  Australia;  Ivica  Grkovic,  The  University  of  Melbourne,  Australia;  Priscilla  Barker, 
The  University  of  Melbourne,  Australia 

Evaluations  have  indicated  that  students  find  An@tomedia  very  engaging  (Kennedy,  Eizenberg,  & Kennedy,  In  this 
volume).  An@tomedia  supports  students  actively  in  their  learning  tasks,  fostering  a deeper  understanding  of  human 
structure,  with  a stronger  basis  for  clinical  diagnosis  and  procedures.  The  new  medical  course  has  focused  less  on 
traditional  teaching,  instead  emphasising  on  focussed  dissection,  problem-based  and  self-directed  learning. 
An@tomedia  provides  multiple  perspectives  of  the  human  body.  Each  module  is  independent  and  of  equal 
importance.  The  modules  are: 

1.  Back, 

2.  Abdomen, 

3.  Thorax, 

4.  Neck, 

5.  Upper  Limb, 

6.  Lower  Limb, 

7.  Head,  and 

8.  Pelvis. 

The  first  3 of  the  8 modules,  the  Back,  Abdomen,  and  Thorax  have  been  completed,  each  organised  into  4 major 
perspectives. 

1 . ’Dissection'  includes  practical  (including  emergency)  procedures. 

2.  ’Imaging’  incorporates  sectional  and  endoscopic  anatomy. 

3.  ’Regions'  incorporates  surface  arid  functional  anatomy. 

4.  ’Systems'  incorporates  conceptual  and  clinical  anatomy. 

References 
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Offering  Online  Degree  Programs:  A Case  Study  Issues,  Challenges,  Successes,  and 
Lessons  Learned 

Gerard  Kickul,  University  of  St.  Francis,  USA;  Laurel  Jeris,  University  of  St.  Francis,  USA;  Michael  LaRocco, 

University  of  St.  Francis,  USA 

Offering  degree  programs  online  offers  an  attractive  option  in  the  higher  education  environment.  As  schools 
confront  this  opportunity,  familiar  issues  of  quality,  access,  participation,  retention,  and  assessment  take  on  new 
meaning.  Online  delivery  systems  parallel  traditional  classrooms,  including  discussion  and  assignment  areas,  chat 
rooms,  and  online  assessments.  After  experiencing  high  student  dropout  rates  the  University  focused  on  several 
solutions  to  reduce  its  student  dropout  rate.  These  included  online  support  systems  with  a centralized  approach  to 
addressing  faculty  and  student  technical  concerns,  complaints,  and  requests  for  assistance.  A new  assessment 
procedure  consists  of  a two-phase  process,  a pre-course  assessment  and  a more  traditional  assessment  occurring 
during  the  final  three  weeks  of  the  course.  Finally,  feedback  from  faculty  on  the  various  instructional  design 
challenges  including  learning  curve,  time  commitment,  recommended  class  size,  ethical  issues,  adaptability  of 
subject  matter  content,  and  practical  suggestions  for  reconceptualizing  syllabi  and  assignments  for  online  learning 
will  be  included. 
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Developing  a Distance  Education  Infrastructure 

Gerard  Kickul,  University  of  St.  Francis,  USA;  Mark  Snodgrass,  University  of  St.  Francis,  USA 

Development  of  distance  learning  infrastructure  is  a multifaceted  project,  from  the  desktop  to  the  far  reaches  of  the 
Internet.  Attention  must  be  focused  on  the  outgoing  data  and  information,  as  well  as,  incoming  data  and 
information.  There  are  four  basic  areas  of  concern  when  designing  an  infrastructure  to  facilitate  Internet  distance 
education  solutions:  hardware,  software,  support,  and  security.  Providing  the  proper  hardware  to  developers  and 
support  personnel  is  crucial  in  creating  a productive  and  efficient  work  environment.  Reliable  and  efficient  Internet 
access  is  essential,  with  24-hour  availability  and  minimal  down  time.  Software  components  for  development  and 
support  are  key  ingredients  to  success.  From  a student  and  faculty  standpoint,  the  course  software  must  be  user 
friendly  with  a high  degree  of  stability.  Support  must  be  considered  from  both  a human  and  a technological 
standpoint.  Essential  to  any  online  endeavor  is  secure  access  and  distance  learning  demands  the  highest  levels  of 
security. 

Content  Management  for  Web  Based  Learning 

Thomas  Kleinberger,  TECMATH  AG,  Germany;  Paul  Muller,  University  of  Kaiserslautern,  Germany 

With  the  help  of  the  newest  information  and  communication  technology  all  the  needs  of  modern  educational 
systems  can  be  met  for  the  first  time  in  one  tool:  the  computer.  Computer  based  solutions  provide  the  technological 
support  necessary  for  the  whole  teaching  and  learning  process.  The  greatest  advantage  of  this  tool  is  the  possible 
combination  of  all  its  usable  instruments,  their  flexibility  and  configurability.  With  new  technologies  and  new  media 
types  it  will  be  possible  to  support  all  kinds  of  training,  self-study  and  continuing  education.  It  is  clear  that  these 
new  methods  and  technologies  require  new  methods  and  solutions  for  the  tasks  content  creation,  delivery  and 
archiving.  Especially  the  use  of  new  media  sets  high  requirements  on  these  tasks.  This  article  describes  how  content 
management  systems,  a technology  already  used  in  other  kinds  of  business,  can  support  these  tasks  in  web  based 
teaching  and  learning.  A definition  of  content  management  is  made  and  refined  for  the  area  of  web  based  learning. 
The  main  building  blocks  and  features  of  content  management  systems  are  described  and  the  improvements  for 
web  based  learning  are  highlighted. 

Computer-Mediated  Communication  for  Distance  Education:  Developing  and  Teaching  a 
Second  Language  Course  in  Academic  Reading 

Esther  Klein-Wohl,  Open  University  of  Israel,  Israel 

The  Open  University  of  Israel  is  searching  for  ways  to  improve  its  distance  education  methods.  Taking  advantage  of 
the  flexibility  of  computer-mediated  communication  (CMC),  we  will  describe  the  application  of  CMC  to  academic 
reading.  We  chose  e-mail  support,  which  allows  students  to  work  on  texts,  and  to  receive  feedback  and  help  when 
needed.  Reading  - a mode  of  language  use,  and  distance  education  - a mode  of  language  instruction,  feature  striking 
parallels:  the  reader  (in  “real  hfe’")  and  the  learner  (when  learning  to  read)  each  functions  in  isolation  when 
interacting  with  a text.  Discourse  is  enacted  at  a distance,  a disassociated  first  person  (the  author  or  the  instructor)  is 
actively  present,  and  no  reciprocity  is  manifest,  within  the  interactive  context.  Therefore,  reading  instruction  and 
distance  education  seem  very  well-suited.  Because  CMC  is  particularly  learner-centered,  it  seemed  appropriate  to 
adopt  this  more  motivating  medium  of  instruction  to  our  traditional  reading  program. 

The  Major  Promise  of  Distance  Education  Is  On  Campus 

W.  R.  (Bill)  Klemm,  Texas  A&M  University,  USA 

Distance  Education  (DE)  is  reforming  on-campus  instruction.  DE's  impact  extends  beyond  the  mere  use  of 
technology  to  put  more  razzle-dazzle  into  classroom  presentations  and  beyond  the  advantages  of  assisted  learning 
via  computer  simulations,  drills,  and  animations.  DE  should  force  teachers  to  re-examine  teaching  philosophies, 
style,  and  tactics.  Distance  Education  shows  that  learning  need  not  depend  on  classroom  lecturing.  Indeed,  we 
should  now  re-consider  the  whole  purpose  of  the  classroom  environment.  DE  prompts  us  to  re-examine 
accrediting  criteria  of  "contact  hours"  and  "credit  hours."  Teachers,  now  freed  from  the  lecturing  straight- jacket, 
can  explore  creative  ways  to  show  students  how  to  find,  comprehend,  manage,  integrate,  and  apply  information. 
DE  reveals  the  value  and  need  in  teaching  for  more  interpersonal  communication,  between  teachers  and  students 
and  among  students.  I see  a revival  of  interest  among  teachers  in  group  work.  Specific  strategies  for  this  new  kind 
of  classroom  teaching  are  illustrated  in  the  poster. 

Interactive  Ophthalmic  Pathology  Tutor 

Gordon  K.  Klintworth,  Duke  University  Medical  Center,  USA;  Anthony  N.  Benson,  Duke  University  Medical  Center, 
USA;  Ann  L.  Bushyhead,  Duke  University  Medical  Center,  USA 

A CD-ROM  interactive,  instructional  course  on  ophthalmic  pathology  and  relevant  anatomy  with  hundreds  of  high 
quality  images  has  been  developed  for  users  with  different  knowledge  backgrounds. 
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Cognitive  Idea  Processor  — Modified  Mandal-Art 

Kagemasa  Kozuki,  Konami  Co,  Japan;  Atsushi  Tsubokura,  Osaka  Electro  Communiation  Univ.,  Japan;  Shogo 
Harima,  Osaka  Electro  Communiation  Univ.,  Japan;  Noboru  Ashida,  Osaka  Electro  Communiation  Univ.,  Japan; 
Katsuhide  Tsushima,  Osaka  Electro  Communiation  Univ.,  Japan 

The  new  idea  processor  called  modified  Mandal-Art  which  can  assist  a user  in  his  associative  thinking  between 
given  item  have  been  developed  in  which  a user  can  communicate  with  the  system  using  eye  movement , finger- 
hand  motion  and  voice.  A user  of  this  idea  processor  is  forced  to  associate  by  seeing  only  nine  items  on  the  matrix 
called  Mandala  on  CRT  at  a time.  We  have  measured  this  information  retrieve  behavior  from  item  pool  of  our 
testee  putting  on  eye  mark  recorder  and  data  gloves.  Human  associative  thinking  in  his  problem  solving  is 
accelerated  by  using  this  cognitive  idea  processor.  This  type  of  idea  processor  may  brings  us  innovation  of 
educational  style  on  the  computer  by  using  the  student  model  which  take  account  the  real  time  associative  behavior 
of  a learner. 

Dilemma  of  Inquiry  and  Reflection  Through  and  With  Technology:  New  Directions  for 
Research  in  Post-Secondary  Teaching 

Olga  Kritskaya,  Michigan  State  University,  USA;  Tony  Clay.  Michigan  State  University,  USA 

The  study  explored  the  issues  of  pedagogy  for  teaching  reflective  inquiry  in  the  undergraduate  program  in 
Educational  Psychology.  The  analysis  focused  on  the  role  of  the  learners'  imagination  and  the  use  of  instructional 
hyper-media  texts  in  construction  of  meanings  associated  with  the  course  content.  Particular  attention  was  paid  to 
the  development  of  student  design  ideas  that  they  use  for  the  arrangement  of  images,  which  reflect,  as  a special 
form  of  narrative,  on  their  understanding  of  the  disciplinary  concepts,  personalities  and  life  experiences.  The  data 
illuminate  the  instructional  conditions  when  the  student's  act  of  arranging  information  becomes  an  act  of  insight. 
The  computer-based  student  projects  (Teacher's  Portfolio,  Multi-Media  Gallery),  along  with  the  instructional  hyper- 
media texts,  reveal  a critical  instructional  shift  in  the  pedagogy  of  teaching  reflective  thinking-from  structure  to 
process,  from  enforcing  students'  competence  to  engaging  them  into  performance- that  fosters  a deeper 
understanding  of  the  dialectics  of  socio-cultural  processes. 

Assessing  and  Training  Admunct  Faculty  with  Technology:  Enhancing  their  Classroom 
Learning 

Karen  Krupar,  Metropolitan  State  College  of  Denver,  USA 

Increasing  numbers  of  adjunct  faulty  at  all  major  institutions  of  higher  education  has  made  it  imperative  that 
institutions  examine  the  technology  competencies  of  this  large  contingent  of  faculty  who  now  instruct  40%  of  the 
courses  on  the  undergraduate  level.  Very  little  effort  is  being  made  to  provide  development  or  training  for  these 
faculty  in  technology  applications  that  would  keep  their  students  current  in  the  rapidly  altering  global  worla  of  the 
21st  centuty.  Metropolitan  State  College  of  Denver  reviewed  their  650  adjunct  faulty  and  found  many  interesting 
factors  that  will  be  reported  during  this  poster  session.  It  is  evident  that  adjunct  faculty  need’ assistance  in  both 
learning  specific  technology  programs  and  in  applying  the  technology  into  their  coursework. 

Design  an  Enhanced  Virtual  Experiment  Environment  Using  Science  Process  Skills  on 
WWW 

Li-Ping  Kuo,  Chung  Yuan  Christian  University,  Taiwan,  R.O.C;  Da-Xian  Dong,  Chung  Yuan  Christian  University, 
Taiwan,  R.O.C;  Chang-Kai  Hsu,  Chung  Yuan  Christian  University,  Taiwan,  R.O.C;  Jia-Sheng  Heh,  Chung  Yuan 
Christian  University,  Taiwan,  R.O.C 

This  paper  proposes  a methodology  of  applying  a proper  distance  learning  system  following  Seven  Steps  of 
Problem  Solving  and  Science  Process  Skills.  To  complete  this  process  of  distance  learning,  this  paper  also  suggests 
some  tools  for  student  undergoing  all  the  process  needed  in  the  problem  solving.  Virtual  Experiment  Environment 
and  Experiment  Record  are  presented  for  some  new  notions  in  this  paper.  In  Virtual  Experiment  Environment, 
Visual  Lab  is  a new  idea  to  operate  experiment  in  Internet  and  show  the  reality  of  experiment.  Experiment  Record 
solves  the  difficult  of  implementing  sheet  and  chart  in  HTML  document.  An  environment  for  teacher  design 
Science  Process  Skills  is  also  presented  in  this  paper. 

Analysis  of  messages  on  the  Only  One  Earth  Club;  TV-based  collaborative  learning  site 
on  the  Internet 

Haruo  Kurokami,  Kanazawa  University,  Japan;  Tatsuya  Horita,  Toyama  University,  Japan;  Yuhei  Yamauchi,  Ibaraki 
University,  Japan 

Messages  sent  to  our  BBS  for  collaborative  learning  can  be  classified  to  10  categories.  Each  of  these  categories  has 
some  types  of  interactions  among  participants  and  site-staffs.  The  types  of  interactions  are  promotion  of  in-depth 
learning,  discussion,  and  question  Sc  answers. 
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Aged  and  Disability  Care  training:  A CD-ROM  based  project 

Mark  Laidler,  RMIT  UNIVERSITY,  Australia 

This  demonstration  has  been  developed  to  highlight  one  solution  for  delivering  high  quality  work-based  training  in 
a multimedia  environment. 

Civilization  in  the  21st  Century 

David  F.  Lancy,  Utah  State  University,  USA;  David  DeBry,  Utah  State  University,  USA;  Megan  Andrew-Hobbs,  Utah 

State  University,  USA 

We  will  report  on  the  evolution  of  an  on-line  course.  The  Civilization/Humanities  course  had  its  origins  in  the 
reform  of  the  university's  General  Education  curriculum  in  1994-95.  It  was  one  of  several  classes  created  to  replace 
existing  requirements.  The  reform  effort  was  designed  to  create  interdisciplinary  classes  that  would  put  the 
emphasis  on  universal  aspects  of  inquiry  rather  than  the  narrow  focus  of  t}pical  introductory  classes.  Other 
expected  features  of  these  new  courses  were  an  emphasis  on  writing  and  the  integration  of  technology.  The  initial 
Civilization /Humanities  class  met  these  criteria  and  was,  by  several  measures,  quite  successful.  In  developing  his 
version  of  the  class.  Dr.  Lancy  digitized  his  very  large  slide  library  and  students  were  thus  able  to  more  readily 
access  the  collection  for  study  and  review.  Another  milestone  occurred  in  1998  when  the  library  adopted  an 
Electronic  Reserve  policy.  Forced  to  incorporate  this  system  into  his  class,  Lancy  went  further  and  adopted  many  of 
the  built-in  features  of  ERes”  such  as  the  "Bulletin  Board."  Civilization/  Humanities  was  moving  towards  the  new 
millennium  as  it  were.  It  became  clear  in  the  second  year  of  implementing  the  new  Gen  Ed  program  (referred  to  as 
University  Studies)  that  not  enough  faculty  were  signing  on  to  teach  the  new  courses.  A bottleneck  emerged  which 
provided  further  incentives  to  adapt  the  few  classes  that  had  been  developed  to  meet  the  new  criteria  for  delivery  to 
a larger  audience.  The  most  recent  stage  in  this  evolutionary  process  we  will  report  on  is  the  transformation  of  the 
Civilization  course  from  a (primarily)  classroom  to  (primarily)  on-line  delivery.  In  concluding  we  will  generalize 
about  this  evolutionary  process  from  cases  gathered  across  the  curriculum  on  our  campus. 

Wired  for  Learning 

Donna  Landin,  West  Virginia  Deaprtment  of  Education,  United  States;  Roberta  Taylor,  IBM,  United  States;  Lynn 

Blaney,  Wheeling  Park  High  School,  United  States;  Susan  Alkire,  Romney  Junior  High  School,  United  States 

The  Reinventing  Education  project  was  established  under  a $2  million  dollar  grant  from  IBM  to  the  West  Virginia 
Department  of  Education.  Its  purpose  is  to  define  and  validate  criteria  for  creating  instructional  plans  that  use  the 
power  of  the  Internet  to  address  the  West  Virginia  Instructional  Goals  and  Objectives  and  improve  student 
achievement  and  learning.  A Criteria  for  Excellence  was  created,  then  employed  by  a group  of  pilot  teachers  to 
develop  lesson  plans  that  would  be  peer  reviewed,  validated  by  field  testing,  observed  during  classroom 
implementation,  and  repeatedly  revised.  The  resulting  lesson  plans  have  resulted  in  significant  learning  improvement 
and  have  been  placed  in  the  Best  Practices  database  and  shared  with  all  teachers  in  West  Virginia.  Instructional 
plans  have  been  developed  for  K-12  classrooms  in  the  areas  of  math,  language  arts,  social  studies  and  science  by 
teams  of  teachers,  pre-service  teachers  and  college  instructors.  On  line  professional  development  facilitates 
implementation  of  the  instructional-collaborative  environment. 

Redesigning  An  Individualized  Paper-Based  Course 

Stephane  Lavoie,  SOFAD,  Canada;  Jo-Ann  Stanton,  SOFAD,  Canada 

Web-based  courses  that  are  an  adaptation  of  a paper-based  version  rarely  take  full  advantage  of  the  Internet  and 
related  technologies.  Taking  this  into  account,  we  developed  a successful  interactive  web  version  of  a French 
grammar  course.  This  poster  session  will  show  how  "Du  fran^ais  sans  fautes"  was  reengineered  to  avoid  the  pitfalls 
of  shovelware  using  an  Oracle  database  created  with  Visual  Basic  applications.  We  will  offer  an  insight  into  the  most 
challenging  aspect  of  the  process:  the  development  of  interactive  exercices.  We  will  focus  on  how  we  worked 
around  technical  constraints  in  order  to  maintain  our  pedagogical  goals.  More  than  1500  students  have  enrolled  in 
the  course  since  it  went  online  in  September  1998.  You  may  access  "Du  fran^ais  sans  fautes"  at 
http://www.dfsf.com. 

Team-paced  versus  Self-paced:  The  Effects  of  Educational  Game  Design  on 

Collaboration,  Learning  and  Attitude  towards  Information  Technology 

Edith  Law,  University  of  British  Columbia,  Canada;  Maria  Klawe,  University  of  British  Columbia,  Canada;  Cristina 

Conati,  University  of  British  Columbia,  Canada;  John  Meech,  National  Research  Council,  Canada 

Avalanche  is  a multi-player  game  where  players  must  cooperate  to  achieve  a common  goal.  Using  Avalanche,  a pilot 
study  was  conducted  to  investigate  (a)  the  effects  of  cooperative  (team-paced)  and  independent  (self-paced)  learning 
on  communication  patterns,  performance,  learning  and  attitude  of  the  players,  and  (b)  whether  there  exists  any 
gender  differences  in  how  boys  and  girls  interact  with  the  computer  and  their  teammates  in  a cooperative  gaming 
environment.  16  elementary  school  children,  in  same-gender  groups  of  four,  played  Avalanche  for  two-hour 
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sessions.  Results  surest  that  although  team-paced  learning  foster  a positive,  cooperative  team  dynamics,  self-paced 
learning  has  a more  significant  effect  on  learning  itself.  In  addition,  boys  and  girls  react  to  the  two  versions  of  the 
game  differently.  In  particular,  the  self-paced  female  group  demonstrates  more  cooperative  and  help  seeking/ giving 
behavior  than  the  team-paced  female  group,  while  the  reverse  was  observed  among  the  boys. 

Assured  Quality  Strategies  in  Web-based  Accredited  Programs 

Judy  Lee,  University  of  Central  Florida,  USA 

Distance  learning  poses  a challenge  to  voluntary  accreditation.  Quality  assurance  in  emerging  technology  mediated 
distance  learning  is  a central  issue  for  accredited  programs.  The  Council  for  Higher  Education  Accreditation  report 
(CHEA,  1998)  defines  "quality  assurance"  as  the  means  by  which  the  institutions  or  providers  set  their  accredited 
programs  goals  and  measure  results  against  those  goals  in  distance  learning.  Regional  and  specialized  accrediting 
organizations  are  currently  engaged  in  assuring  quality  in  distance  learning  as  part  of  their  ongoing  review  of 
institutions  and  programs.  "Distance  learning  is  seen  by  many  as  transformative  vehicle  for  increasing  the  pace  of 
change  and  reform  in  higher  education.  For  these  and  other  reasons,  analysis  of  quality  assurance  is  an  essential 
topic  for  national,  state  and  institutional  policy  development  (CHEA,  1998)."  Areas  of  interest  regarding  Quality 
Assurance  Strategies  include  course  and  program  concerns  and  needs,  faculty  and  student  concerns  and  needs, 
technical  needs,  and  administrative  support. 

Reference 

Council  for  Higher  Education  Accreditation  (1998,  April)  . Assuring  quality  in  distance  learning  . Washington,  DC: 
The  Institute  for  Higher  Education  Policy 

Training  Students  to  Be  Self-Learners  in  Educational  Hypermedia  and  Technology 
Courses 

Amy  S.  C.  Leh,  California  State  University  San  Bernardino,  USA; 

Modern  technology  is  changing  our  education  paradigm.  Due  to  technology  advancement,  instructors  can  no  longer 
play  the  role  of  "information-giver",  rather  have  to  become  a facilitator.  Facing  tremendous  amount  of  technology 
information,  students  in  Educational  Technology  need  to  feel  comfortable  of  learning  technologies  by  themselves. 
The  presenter  employed  a variety  of  instructional  strategies  in  her  graduate  courses  to  train  her  students  to  be 
technology  self-learners.  In  the  courses,  her  students  searched  for  Internet  tools  useful  for  instruction,  selected 
computer  applications  that  they  would  like  to  study,  learned  the  selected  tools  in  a team,  and  trained  other  class 
members  the  use  of  the  tools  they  learned.  In  this  poster  session,  the  author  will  present  teaching  strategies, 
instructional  activities,  and  course  assignments  that  are  related  to  the  training.  She  will  also  report  students'  feedback 
on  this  kind  of  training. 

Effective  Use  of  Client-Server  Software  and  Teaching  Strategies  in  Online  Courses 

Amy  S.  C.  Leh,  California  State  University  San  Bernardino,  USA;  Laura  Howzell  Young,  California  State  University 
San  Bernardino,  USA 

In  the  field  of  education,  many  professors  are  incorporating  online  teaching  into  traditional  courses.  What  is  the 
impact?  The  authors  received  a university  internal  grant  to  convert  two  education  courses — an  instructional 
technology  course  and  a course  in  adolescent  development — to  include  online  components.  They  report  the 
benefits  and  challenges  of  using  WebCT  (course  tools)  with  a variety  of  teaching  methodologies.  The  poster  session 
reflects  results  of  a pre-  and  post-  survey  used  in  both  courses  to  determine  teacher  attitudes  toward  online 
coursework.  Students  in  both  courses  appreciated  the  flexible  schedule  of  online  work  and  time  saved  by  not  having 
to  commute.  The  majority  of  teachers  in  both  classes  had  not  previously  taken  online  courses;  many  liked  the 
threaded  group  discussions  and  online  testing,  but  a few  who  did  not  believe  at  the  outset  that  on-line  teaching  is 
beneficial  continued  to  reject  it.  For  most  of  the  participants,  technology  improved  learning  and  achievement. 

Stepnet:  quality  newspapers  and  web-based  learning  in  the  Netherlands 

Jan  Lepeltak,  Meulenhoff  Educatief,  The  Netheriands 

Stepnet  is  a web-based  newspaper-project  in  the  Netherlands.  Due  to  some  educational  reforms  students  in  the 
upper  grades  of  the  Dutch  secondary  schools  are  to  study  partly  individually,  which  implies  that  teachers  are  obliged 
to  provide  them  with  proper  assignments.  The  results  of  these  are  part  of  the  students'  final  examination.  The  use 
of  ICT  should  also  be  part  of  their  activities  and  a relation  with  daily  (world-)  news-events  is  strongly 
recommended.  In  order  to  facilitate  the  above  aproach  for  teachers  and  students,  PCM,  a Dutch  publishing 
company  consisting  of  newspaper  and  educational  publishing  houses,  developed  a web-site.  This  site  contains  a D- 
base  filled  with  assignments  related  to  the  contents  of  two  daily  national  quality  newspapers.  Every  three  weeks 
during  the  annual  schoolterm  a special  magazine  is  published.  Schools  will  pay  a fee  and  will  receive  a licence  in 
return,  which  entitles  all  participating  students  and  teachers  to  receive  a copy  of  this  magazine  called  the  ‘*@krant”, 
as  well  as  access  to  the  assignment  D-base  and  the  @krant  D-base  containing  the  newspaper  articles.  In  addition, 
teachers  will  receive  a half  a day  on-the-job  training  about  working  with  Stepnet. 
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Custom-Authored  Hypermedia  Exercises  for  Use  in  College  Classrooms 

Amy  Lobben,  Central  Michigan  University,  USA 

Continuously  improving  technology  now  allows  educators  new  opportunities  to  move  beyond  traditional  lecture 
style  methods  to  create  custom-authored  multimedia,  hypermedia,  and  animation  presentations  for  use  in  classroom 
demonstrations  or  as  computer-run  student  exercises.  The  development  of  new  software  programs  and  faster,  more 
powerful  computers,  is  beginning  to  change  the  creation  process  of  dynamic  presentations  (multimedia, 
h5^ermedia,  and  animation).  The  once  time  consuming,  programming  intensive,  and  complicated  methods  of 
authoring  such  presentations  have  become  manageable  tasks  through  the  use  of  improving  computer  programs, 
many  of  which  are  easy  to  learn  and  use,  A series  of  computer-administered  lessons  have  been  created  using 
Macromedia  Director  and  Freehand,  Each  interactive  lesson  is  designed  to  explain  a specific  topic  and  includes 
animation,  graphics,  photographs,  quizzes,  while  maintaining  stringent  design  guidelines.  Applications  such  as  these 
could  have  profound  effects  in  large  college  classrooms,  where  traditional  lecture-style  approaches  limit  the 
individual,  active  learning  by  students. 

Groupware  Tools:  Web  File  System  Allows  Students  to  Access  Information  Anywhere, 
Anytime 

Tyrone  Lobo,  SiteScape,  Inc.,  Canada 

This  poster/demo  is  based  on  SiteScape's  Web  File  System  and  will  explore  how  the  Web  File  System  allows  users 
to  securely  store  and  manage  large  volumes  of  information  entirely  over  the  Web,  giving  students  the  freedom  to 
access  information  from  any  industry-standard  web  browser, 

A Web-based  Hypermedia  Pedagogical  Course 

Alla  N.  Makarova,  St-Petersburg  State  Pedagogical  Univeristy,  Russia 

The  success  of  any  hypermedia  distance-learning  course  strongly  depends  on  its  psychological  and  pedagogical 
aspects.  A study  was  made  of  the  practical  realization  of  some  common  pedagogical /psychological  criteria  in  a 
Web-based  hypermedia  course.  As  a result  of  this  study  a template  was  developed  which  includes  these  criteria.  This 
template  may  be  included  as  a building  block  for  many  different  hypermedia  courses  devoted  to  a particular  subject. 
Some  open  problems  with  the  template  are  discussed  and  analyzed. 

Info  Pursuits 

Kerrie  Manning,  Leichhardt  High  School,  Australia;  Rosemary  Ward,  UTS,  Australia;  Tony  Ward,  Award  Consulting 
Enterprises,  Australia 

Info  Pursuits  is  an  interactive  CD-ROM  game  designed  to  teach  and  practice  Library  Information  Skills,  Students 
will  answer  Trivial  Pursuits  style  questions  and  will  be  offered  Information/ Research  Skills  hints  and  suggestions  as 
part  of  the  feedback.  These  will  include  such  things  as  techniques  for  finding  resources,  using  encyclopaedias, 
dictionaries,  non-print  resources,  Internet,  statistical  resources,  telephone  books,  indexes,  directories,  interviews  etc. 
Positive  experience  in  using  these  resources  may  stimulate  students  to  use  them  again  in  a different  research  task. 
The  same  skills  are  practiced  many  times  with  different  resources  and  in  different  contexts,  to  facilitate  transfer.  The 
questions  will  be  arranged  in  teaching  area  groups,  to  be  used  by  teachers  in  ordinary  lessons  within  the  standard 
curriculum,  or  as  extension  work  within  standard  curriculum  areas.  Many  questions  will  practise  general  skills,  but 
each  curriculum  area  will  provide  experience  with  subject-specific  resources. 

Content  Area  Integration:  A Step  Toward  Emiratisation 

Reo  H.  McBride,  Dubai  Women's  College,  United  Arab  Emirates;  Michael  Ford,  Dubai  Women's  College,  United  Arab 
Emirates;  David  Thomson,  Dubai  Women's  College,  United  Arab  Emirates;  Raymond  Yarsley,  Dubai  Women's 
College,  United  Arab  Emirates;  Laila  Hawker,  Dubai  Women's  College,  United  Arab  Emirates 

The  Higher  Colleges  of  Technology  (HCT)  is  dedicated  to  the  delivery  of  technical  and  professional  programs  of 
the  highest  quality  to  the  citizens  of  the  United  Arab  Emirates  (U.A.E.).  This  is  the  goal  of  “Emiratisation”:  an 
affirmative  action  policy  of  employing  nationals  in  larger  numbers  in  the  workforce,  which  is  currently  dominated 
by  ex-patriot  workers.  The  authors  of  this  article  state  that  the  most  effective  way  to  better  prepare  our  students  in 
fulfilling  their  roles  in  the  task  of  emiratisation,  is  to  integrate  the  skills  taught  in  English,  Math  and  Basic 
Computing  in  the  classroom,  and  NOT  teach  them  as  completely  separate  entities.  In  essence,  integrating  the  skills 
taught  in  English,  Math  and  Basic  Computing  develops  in  students  greater  linguistic  abilities,  technical  skills, 
intellectual  capacities,  and  leadership  potential.  This  study  describes  efforts  made  toward  accomplishing  such  a 
daunting  yet  absolutely  necessary  task  for  college  students  in  the  U.A.E. 

High  Tech  Classrooms-Going  The  Distance  with  Distance  Education 

Patricia  McNames,  Indiana  University  Southeast,  USA 

The  rapidly  evolving  paradigm  of  interactive  distance  education  is  changing  the  learning  environment  on  and  off 
campuses  and  schools  nationwide.  Educators  are  suddenly  surrounded  by  not  only  the  opportunities  of  distance 
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education,  but  also  by  its  demands.  Thus,  when  educators  begin  using  these  boundary-spanning  technologies  to 
enhance  the  learning  process,  it  does  dramatically  require  teachers  and  students  to  change  what  they  do.  The 
purpose  of  this  paper  and  poster/demo  is  to  serve  as  a resource  for  educators  who  may  be  considering 
incorporating  distance  education  approaches  into  their  courses.  In  cyberspace,  time  and  place  are  fourth 
dimensional.  The  four  dimensions  that  will  be  discussed  in  this  presentation  are  1)  defining  the  appropriate  distance 
education  system,  2)  designing  the  interactive  course,  3)  developing  a virtual  learning  community,  and  4)  dealing 
with  technical  difficulties. 

An  on-line  introduction  to  quantitative  methods 

Moira  McPherson,  Lakehead  University,  Canada;  William  Montelpare,  Lakehead  University,  Canada 

This  poster/ demo  describes  the  development,  implementation,  and  evaluation  of  an  “on-line”  introduction  to  an 
undergraduate  course  in  quantitative  methods.  The  virtual  curriculum  modules  developed  for  the  course  provided  a 
forum  in  which  information,  knowledge,  expertise,  and  questions  could  be  explored  and  updated  throughout  the 
course.  Working  asynchronously,  the  user  completed  six  assignments.  Questions  which  arose  during  the  user’s 
session  were  passed  to  “experts”  that  provided  feedback,  electronically.  The  “question  - feedback”  loop  was  a 
dynamic  component  of  the  organizational  structure  and  used  different  modes  of  communication,  including  e-mail, 
electronic  conference  rooms,  bulletin  boards,  and  on-line  help  pages.  The  assignments  were  submitted  electronically 
to  teaching  assistants  who  evaluated  the  submissions  and  provided  feedback  directly  to  the  users.  The  inclusion  of 
direct  feedback  communication  between  experts  and  users  reduced  the  impersonal  characteristics  that  might  arise 
within  an  asynchronous  learning-environment. 

Principles  Of  Cognitive  Evaluation  For  An  Educational  CD-ROM  For  History 

Christina  Metaxaki-Kossionides,  University  of  Thrace,  Greece;  Elevtheria  Gonida,  University  of  Thrace,  Greece; 
Stavroula  Lialiou,  University  of  Athens,  Greece;  Georgios  Kouroupetroglou,  University  of  Athens,  Greece 

The  evaluation  of  CD-ROMs  and  multimedia  products  used  in  educational  environments  ,is  a major  topic.  One  of 
the  erasing  problems  is  the  formation  of  a set  of  pedagogical  principles  or  concepts  and  their  technological 
implementation.  We  present  the  formation  of  a set  of  cognitive  principles.  Those  are  included  in  the  set  of 
evaluation  , concerning  a CD-ROM  developed  by  us.  The  topic  was  a histor)^  lesson  about  the  Homer  period.  We 
have  selected  two  types  of  principles  which  , together  with  the  ones  for  software  development,  should  form  the  set 
for  the  formative  evaluation.  The  one  sub  - set  included  the  pedagogical  requirements  for  the  lesson  of  history.  The 
other  sub  - set  included  the  principles  delivered  by  the  cognitive  scientists.  These  last  ones  were  developed  for  the 
specific  topic.  They  mainly  refer  to  the  nature  and  processes  of  learning  and  instruction.  The  set  of  the  cognitive 
requirements  were  added  explicitly  as  fields  for  a database  for  software  evaluation. 

Authoring  Multimedia,  Designing  Animations  for  Physics  Education 

Donald  J.  Metz,  University  of  Winnipeg,  Canada 

Computer  animations  used  in  high  school  physics  instruction  are  demonstrated.  The  programs  are  written  using 
Asymetrix  Multimedia  Toolbook  and  are  designed  to  address  the  specific  needs  of  the  students  and  their  learning 
environment. 

Dynamic  Communication  Layer  Between  Virtual  Laboratory  and  Intelligent  Agents 

Kaufmann  Meudja,  UniversitT  de  Sherbrooke,  Canada;  Roger  Nkambou,  UniversitT  de  Sherbrooke,  Canada 

In  the  last  few  Years,  virtual  laboratory  has  become  an  important  part  in  online  courses.  Some  of  those  virtual  labs 
involve  intelligent  behaviour.  However,  this  intelligent  behaviour  is  usually  hard-coded  in  the  virtual  lab  and 
therefore,  limit  the  possibility  of  using  reusable  intelligent  agents  that  are  completely  separate  from  the  virtual  lab 
and  which  can  control  most  users  actions  in  the  virtual  laboratory  environment.  We  propose  a way  to  separate 
intelligent  behaviour  from  the  virtual  lab  by  the  implementation  of  a communication  layer  between  the  virtual 
laboratory  and  intelligent  learning  agents. 

Design,  Delivery,  and  Evaluation  of  Online  Courses:  A Primer 

Jerold  Miller,  United  States  International  University,  USA 

Online  learning  environments  can  be  highly  interactive  and  student-centered.  Incorporating  the  contributions  and 
experiences  of  each  member  of  a learning  community  into  the  coursework  encourages  students  to  contribute  more 
of  themselves  to  the  course  and  allows  them  to  learn  more  from  their  teachers  and  peers.  Creation  of  functional 
online  learning  environments  is  contingent  upon  the  effective  design,  delivery,  and  evaluation  of  online  courses. 
This  session  will  address  the  design  of  online  courses  and  adaptation  of  traditional  courses  to  the  online  delivery 
mode.  Participants  will  understand  appropriate  online  delivery  methods,  explore  teacher-centered  vs.  student- 
centered  learning,  understand  effective  strategies  for  communication  online,  know  how  to  adapt  content  and 
curriculum  for  online  teaching,  discover  new  assessment  strategies  for  distance  learning,  and  learn  how  to  create  an 
online  community  of  learners. 
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Discovering  xy Algebra:  Intelligent  Interactive  Internet  Instruction 

John  Miller,  The  City  College  of  CUNY,  USA 

Abstract:  Passive  activities  such  as  watching  presentations,  listening  to  explanations  of  general  principles  and 
watching  experts  solve  sample  problems  are  helpful,  but  peripheral,  to  the  mathemadcal  learning  process.  For 
students  the  indispensable  step  is  solving  problems  for  themselves.  Yet  most  commercial  mathematics  software  still 
concentrates  on  presentations  and  sample  problems,  while  sending  students  off  line  to  do  practice  problems  on 
paper  without  interactive  support.  Answers  are  either  multiple  choice  or  limited  to  a single  simplified  final  step. 
Early  Internet  courses  are  even  less  interactive.  In  contrast,  students  using  xyAlgebra  can  enter  each  step  of  each 
problem  solution.  They  enjoy  intelligent  support  at  every  step  as  xyAlgebra’s  suggested  solution  strategy  changes  in 
response  to  their  steps  in  simplifying  expressions,  solving  equations  and  even  in  setting  up  and  solving  verbal 
problems.  The  next  version  of  xyAlgebra  will  support  instruction  over  the  Internet,  yet  the  entire  package  can  be 
downloaded  without  cost  at  mathO.sci.ccny.cuny.edu/ xyalgebra. 

Problem-based  Learning  and  Flash  4.0:  An  Experiment  in  Science  Education 

Leslie  Miller,  Rice  University,  USA;  Janice  Mayes,  Rice  University,  USA;  Donna  Smith,  Rice  University,  USA 

How  does  one  design  learning  resources  that  will  reach  large  audiences,  capture  the  imagination,  and  compete  with 
the  game  environments  that  are  popular  among  of  middle  school  students?  The  Reconstructors  is  a new  episodic 
adventure  series,  designed  for  the  Web,  with  a substantive  educational  message.  See  http:/ / reconstructors.rice.edu 
where  a student  enters  a futuristic  world  in  which  he  or  she  assumes  the  role  of  a "reconstructor."  In  the  interactive 
mystery  format,  a student  learns  science  and  history  relevant  to  the  discovery  and  use  of  opiates.  Over  the  course  of 
four  episodes  or  missions,  students  "solve  the  problem."  The  concepts  of  neurotransmission,  the  neurobiology 
underlying  drug  addiction,  drug  tolerance,  and  analgesia,  as  well  as  the  history  of  opium  use  are  presented. 

On-Line  Education  Using  Video  Broadcasting  Delivered  from  Perth  Campus,  Algonquin 

College 

Maike  Luiken  Miller,  Algonquin  College,  Canada;  Dave  Osborne,  Algonquin  College,  Canada;  Ian  McCormick, 

Algonquin  College,  Canada 

As  part  of  the  LCN  (Lanark  Communications  Network)  TAP  project,  Algonquin  College  at  Perth  Campus  engaged 
in  a video  broadcast/conferencing  pilot  to  deliver  a course,  "The  History  and  Philosophy  of  Architectural 
Conservation"  (ARC9001),  simultaneously  in  2 modes:  lectures  with  electronic  slides  and  traditional  black  board 
work  and,  in  parallel,  video  broadcasts  (IN@SEC  VP  Broadcaster)  with  e-mail  feedback.  This  pilot  demonstrated 
successfully  that  on-line  co-delivery  allows  to  teach  students  simultaneously  at  different  locations  employing  video, 
audio  and  text  via  a network  using  relatively  inexpensive  broadcasting/conferencing  software  with  minimal  set-up 
requirements  at  the  clients'  site.  Course  delivery  improved.  Students  enjoyed  the  choice  of  delivery'  modes  to  suit 
their  learning  styles.  Professor  appreciated  more  time  for  interacting  with  the  students.  Re-broadcasting  at  a later 
time  possible.  The  delivery  model  - independent  of  the  specific  broadcasting  tool  - provides  a means  to  reach 
remote  connected  students  everywhere  - be  it  rural  or  elsewhere  around  the  world. 

TLC  : Teachers’  And  Learners’  Collaborascope- A Platform  For  Analysing  And 

Evaluating  Online  Education 

Michelle  Montgomery  Masters,  University  of  Glasgow,  UK;  Stewart  Macneill,  University  of  Glasgow,  UK;  Prof 

Malcolm  Atkinson,  University  of  Glasgow,  UK 

Millions  is  being  spent  on  installing  networks,  computers  and  assembling  distance  learning  material  and  ICT 
(information  and  communications  technology)  software.  Yet  the  network  technology  is  not  being  used  to  assemble 
databases  which  researchers,  managers  and  politicians  can  "data  mine"  to  guide  their  investment.  In  the  TLC  project 
we  are  pioneering  techniques  for  assembling  and  querying  educational  data.  Our  goal  is  to  initiate  research  into  the 
best  methods  of  rapidly  assessing  the  effectiveness  of  resources  deployed  and  then  interpreting  the  data  to  improve 
our  educational  performance.  We  are  not  trying  to  reinvent  the  wheel  and  produce  yet  another  online  learning 
environment  like  WebCT  or  Virtual-U.  Instead  we  are  building  tools  which  will  work  in  collaboration  with  these 
existing  learning  environments  and  provide  teachers  and  students  with  valuable  information  about  the  usage  of  their 
learning  environment.  The  TLC  project  aims  to  utilise  the  emerging  standards  such  as  those  from  the  IMS  project. 

The  Third  Dimension  For  World  Of  Knowledge:  An  Acceptable  Way  To  Get  In 

Mikhail  Morozov,  Mari  State  Thechnical  University,  Russia;  Aleksander  Markov,  Mari  State  Thechnical  University, 

Russia 

Educational  CD-ROMs  should  be  as  interesting  and  attractive  for  children  as  computer  games.  The  approach 
offered  by  the  authors  allows  to  reduce  the  existing  gap  between  multimedia  for  education  and  for  entertainment.  It 
is  achieved  due  to  the  unusual  design  of  3D  interface  and  by  using  a special  technique  of  development.  As  a general 
metaphor  for  organisation  of  multimedia  environment  the  metaphor  of  theatre  with  all  appropriate  attributes  was 
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chosen:  the  stage  with  visual  educational  content,  side-  scenes,  a curtain,  etc.  For  presentation  of  educational 
material  in  addition  to  traditional  multimedia  components,  the  new  integrated  forms  are  used.  Because  of 
complexity  and  richness  of  information  of  the  CD-ROM  model,  its  realization  is  based  on  graph  of  program 
transitions  and  application  of  script  paradigm,  which  is  accepted  for  the  description  of  structure  of  the  educational 
and  navigating  information.  These  solutions  can  be  recommended  for  creation  of  a new  generation  of  ’'three- 
dimensional"  educational  multimedia  CD-ROMs. 

Using  a Web  Site  to  Increase  Student  Motivation  to  Write  and  Enhance  the 
Reading/Writing  Relationship 

Susan  Nelson,  Elm  Street  Elementary  School,  USA;  Harrison  Yang,  State  University  of  New  York  at  Oswego,  USA 

This  action  research  project  is  an  example  of  how  Internet  technology  is  being  used  in  an  elementary  classroom  as  a 
means  to  increase  student  achievement.  The  project  explores  the  relationship  between  writing  and  reading  and  its 
affects  on  early  literacy  development.  It  also  explores  the  concept  of  motivating  students  to  write  by  publishing 
student  work  for  real  life  audiences.  The  project  uses  the  current  technology  of  an  Internet  web  site  as  a source  for 
publishing  student  writing.  It  explores  the  effects  that  this  type  of  publication  has  on  student  attitude,  motivation  to 
write,  and  literacy  development. 

TELSI,  Web-Based  Software  For  Collaborative  Learning 

Graciela  Nielsen,  Norwegian  State  Center  for  Adult  Education,  Norway;  Boo  Never,  Majornas  Vuxengymnasium, 
Sweden;  Aase  Steinmetz,  Undervisningsministerium,  Denmark 

The  TELSI  software  has  been  developed  with  the  support  of  the  European  Union  under  the  SIMULAB-project. 
SIMULAB  is  a concept  for  Web-based  collaborative  tasks.  TELSI  has  been  created  to  cater  for  the  needs  of  this 
type  of  activity,  and  is  a flexible  and  user-friendly  environment. 

Development  of  Speech  Input  System  for  Web-based  Courseware 

Ryuichi  Nisimura,  Nara  Institute  of  Science  and  Technology , Japan;  Shoji  Kajita,  Nagoya  University,  Japan;  Kazuya 
Takeda,  Nagoya  University,  Japan;  Fumitada  Itakura,  Nagoya  University,  Japan 

We  proposes  WebSPEAC(Web  SPEech  Acquisition  for  Courseware)  System  that  provides  a speech-input  function 
for  Web-based  online  education  system.  WebSPEAC  has  the  following  three  features;  (1)  easy  to  maintain  the 
system  by  the  use  of  a simple  speech  input  program  in  Web  browsers,  while  the  other  speech  processing  are 
performed  on  the  Web  server,  (2)  a natural  interactivity  between  Web  server  and  users  by  using  the  server  push 
techonology,  (3)  a simple  installation  and  no  needs  of  external  programs  on  client  machines.  To  show  the 
effectiveness,  an  Web-based  speech  analysis  system,  a speech  recognition  system  and  a student  identification  system 
are  developed  as  the  examples  of  WebSPEAC  System.  By  using  the  speech  analysis  system,  it  is  clarified  that 
WebSPEAC  System  can  significantly  reduce  task  steps,  error  steps  and  elasped  time  for  speech-input  task, 
compared  with  the  conventional  method  (file  upload). 

The  Instructional  Portal  Project 

Kevin  Oliver,  Virginia  Tech,  USA 

The  Instructional  Portal  is  a clearinghouse  of  information,  resources,  tools,  and  services  at  Virginia  Tech,  designed 
to  facilitate  practical  instructional  design  among  faculty.  Faculty  can  increasingly  adopt  and  easily  utilize  educational 
technologies  in  their  courses.  While  some  faculty  use  technology  to  modify  and  improve  teaching  and  learning 
practices,  too  many  rely  on  tools  to  simply  make  their  existing  teaching  practices  more  efficient.  The  portal  is 
designed  to  help  faculty  integrate  technology  more  effectively  by  applying  three  instructional  design  components. 
The  analysis  component  outlines  strategies  and  instruments  to  account  for  environmental  and  learner 
characteristics,  and  provides  information  about  structuring  course  content.  The  strategy  component  describes 
instructional  methods  and  models  that  utilize  technology  effectively,  and  provides  teaching  modules  covering 
popular  web-based  instruction  and  interface  design  issues.  The  evaluation  component  describes  evaluation  methods 
and  models  and  provides  tools  to  help  faculty  revise  and  improve  their  technology  innovations.  Visit  the  portal  at: 
http://www.edtech.vt.edu/edtech/portal/index.html 

Integrating  and  Adapting  Multimedia  Resources 

Mary  Panko,  UNITEC  Institute  of  Technology,  New  Zealand 

In  UNITEC  students  enrolled  on  an  Automotive  Trades  course  have  access  to  a series  of  32  CDX  disks,  entitled 
The  Automotive  Trainee  Series  that  provides  them  with  a learning  resource  in  addition  to  their  basic  theoretical  and 
practical  teaching.  This  study  attempted  to: 

• discover  what  beneficial  learning  effects  the  students  perceived  as  being  currently  provided  by  the 
multimedia  component; 

• examine  the  extent  to  which  the  tutors  integrated  and  adapted  the  multimedia  material  as  part  of  their 
teaching;  and 
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• compare  patterns  of  current  disk  utilization  with  those  of  the  previous  year. 

Results  showed  that  although  both  staff  and  students  thought  the  multimedia  material  valuable,  its  overall  utilization 
by  students  had  diminished  over  time.  The  reason  for  this  appears  to  be  the  lack  of  integration  of  the  resource  by 
tutors  into  their  teaching.  Furthermore,  tutors  had  generally  not  adapted  the  material  for  their  own  needs  although 
where  workbooks  had  been  provided,  this  seemed  to  increase  the  students'  satisfaction. 

DISCETECH-BIMBOTECH:  An  Experimental  Project  For  Introducing  Innovative 
Technology  Within  The  Italian  School  System 

Paolo  Paolini,  Politecnico  di  Milano,  Italy;  Pierluigi  Della  Vigna,  Politecnico  di  Milano,  Italy;  Francesca  Collina, 
Politecnico  di  Milano,  Italy;  Paolo  Magatti,  Politecnico  di  Milano,  Italy;  Silvia  Moiola,  Politecnico  di  Milano,  Italy 

Since  1996  Politecnico  di  Milano  has  launched  a program  for  experimenting,  within  the  Italian  school  system,  the 
use  of  advanced  technology  for  teaching  and  learning.  Two  projects  (in  Como  and  Lecce)  have  involved  more  than 
700  teachers  and  more  than  8.000  pupils,  being  the  largest  of  this  kind  in  Italy,  both  in  High  Schools  (Discetech 
project)  and  grammar  schools  (Bimbotech).  The  project  emphasizes  the  role  of  the  teachers,  who  are  trained  on 
multimedia  and  Internet  technology  first.  Secondly  they  are  “guided”  and  assisted  in  shaping  up  specific  activities 
with  usage  of  either  multimedia  CD-ROM’s  or  Internet  Sites.  A large  collection  of  CD-ROM’s  has  been  created  and 
several  hundreds  educational  Websites,  on  different  subjects,  have  been  classified.  The  project  has  been  very 
successful  with  the  teachers  (many  of  them  have  never  used  advanced  technology  before)  and  the  pupils,  the 
learning  and  the  interest  of  which  has  been  evaluated  as  sharply  improved. 

Learning  Experiences  in  New  Learning  Environments 

Heli  Pekkala,  University  of  Jyvaskyla,  Finland 

One  of  the  outcomes  of  a departmental  action  research  project  at  our  language  centre  is  a modernised  Learning 
Centre,  where  students  and  staff  of  the  university  have  access  to  learning  materials  in  over  30  languages.  The 
philosophy  and  pedagogical  framework  underlying  the  operations  of  the  centre  is  to  support  and  encourage 
students  in  developing  their  learning  skills  and  self-directiveness  for  continuous  language  learning  and  for  making 
effective  use  of  new  learning  environments  . This  poster  presents  the  Learning  Centre  (interactive  studios,  writing 
studio,  multimedia  studio  etc.)  and  reports  student  experiences  of  learning  a new  language  independently.  One  part 
of  this  learner  training  course  Learning  to  Learn  Languages  was  a self-access  project  during  which  students  were 
requested  to  reflect  upon  the  kinds  of  qualities  and  procedures  that  self-directed  language  learning  requires  from  a 
learner.  They  also  evaluated  what  areas  of  language  were  possible  to  access  without  external  structuring  and 
teaching. 

Microsoft  Office  as  a Learning  and  Teaching  Tool 

Craig  Place,  Acadia  University,  Canada;  Michael  Shaw,  Acadia  University,  Canada 

Many  useful  features  in  popular  software  applications  often  go  unnoticed  by  many  users.  Often  thought  of  as  a 
business  application,  Microsoft  Office  is  one  such  suite  of  applications  that  is  often  overlooked  as  a tool  for  use  in 
the  teaching  and  learning  environment  at  most  grades,  and  at  the  university  and  college  level.  With  just  Word,  Excel 
and  PowerPoint,  MS  Office  can  be  used  as  a tool  to  develop  the  basic  understanding  and  expertise  for  a number  of 
other  applications  including  photo-editors,  drawing  and  animation  programs,  multimedia  authoring  and  web- 
development  tools.  Used  as  a stepping  stone  to  more  advanced  applications  in  this  way,  the  interface  of  each  of 
these  3 applications  can  be  simplified  to  be  useful  at  a number  of  different  grade  levels,  or  levels  of  expertise.  Using 
a combination  of  examples  and  hands-on  exercises  the  workshop  will  illustrate  to  the  participants  how  MS  Office 
can  be  used  to  easily  and  effectively  create  multimedia  hyper-documents,  dynamic  learning-objects,  student  review 
exercises  and  interactive  non-linear  presentations.  Specific  examples  will  be  the  use  of  PowerPoint  as  a concept- 
mapping tool  and  as  a multimedia  authoring  and  animation  tool.  Using  Excel  to  assist  in  language  learning  and 
create  dynamic  graphs  for  interactive  what-if  scenarios,  and  using  Word  to  develop  hypermedia  templates  and 
collaborative  writing  documents.  Several  examples  of  the  usefulness  and  creation  of  macros  will  also  be  shown. 

Students'  expectations  and  perceptions  of  efficacy  for  self-regulated  learning  in 
mediated  learning  environments:  Partial  results  of  an  evaluation 

Laura  Helena  Porras,  Universidad  de  las  Americas,  Puebla,  Mexico 

This  paper  discusses  partial  results  of  an  evaluation  applied  to  an  educational  teleconferencing  system  in  a Mexican 
higher-education  institution.  Courses  offered  through  the  system  are  based  on  a pedagogical  model,  which 
emphasizes  the  active  role  of  the  learner  as  responsible  of  her  own  learning.  Therefore,  perceptions  of  the  self,  and 
self- regulatory  processes  become  key  components  in  this  learning  environment.  Conclusions  here  presented 
correspond  only  to:  (i)  students’  expectancies,  (ii)  perceptions  of  efficacy  for  self- regulated  learning,  and  perceptions 
of  the  course,  and  of  the  teacher  and  learner’s  work.  Results  indicate  students’  expectancies  focus  on  "learning  more 
than  in  face-to  face  situations”,  followed  by  "using  technology",  and  "having  freedom  and  initiative".  Perceptions  of 
efficacy  for  self-regulated  learning  were  high  overall.  Nevertheless,  analyses  of  individual  items  identify  specific  areas 
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of  improvement.  Qualitative  data  on  perceptions  of  the  course,  and  teacher’s  and  learner’s  work  suggest  that  these 
courses  are  more  demanding  than  the  face-to-face  ones  they  are  taking.  Students’  responses  include  attitudinal  and 
cognitive  adjustments  needed  for  these  learning  environments.  These  results  have  led  to  improvements,  which  are 
presently  being  applied. 

An  Online  Model  For  Precalculus 

Hari  Pulapaka,  Valdosta  State  University,  USA;  Denise  Reid,  Valdosta  State  University,  USA 

Some  of  the  greatest  challenges  to  designing  and  delivering  a completely  online  mathematics  course  are  effective 
and  efficient  uses  of  the  leading  technologies  that  are  available  for  this  purpose.  The  authors  have  just  completed 
designing  the  first  version  of  a precalculus  course  that  is  being  delivered  through  a combination  of  synchronous  and 
asynchronous  environments.  Precalculus  was  chosen  over  other  mathematics  courses  for  several  reasons.  The 
course  has  been  planned  as  a model  online  course  for  any  mathematics  course  (especially  at  the  early  levels).  Future 
versions  of  this  course  and  related  courses  will  undoubtedly  incorporate  additional  technologies  and  components  in 
an  effort  to  improve  student  learning,  interactivity,  and  end-user  feasibility.  Future  plans  for  the  course  include 
development  and  refeinement  of  the  unfinished  sections  of  the  course  and  tighter  integration  with  graphing 
calculator  and/or  mathematical  software. 

IMAGUS:  An  Environment  To  Design  Video  Applications  For  Digital  Libraries 

Andre  Luis  Alice  Raabe,  Pontificia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil:  Lucia  Maria  Martins  Giraffa, 
Pontificia  Universidade  Catolica  do  Rio  Grande  do  Sul,  Brasil 

This  poster  presents  the  description  of  the  IMAGUS  environment.  IMAGUS  has  tools  to  built  video  applications 
supported  by  Internet.  This  environment  has  being  developed  in  order  to  contribute  for  the  construction  of  the 
PUCRS  digital  library  collection. 

Art,  Poetry,  and  Instructional  Media 

Thomas  Reinartz,  Rosemount  High  School,  USA;  Brad  Hokanson,  University  of  Minnesota,  USA 

In  many  high  school  English  courses,  students  are  trained  to  seek  out  proper  form  and  work  toward  clarity  in  their 
writing  assignments.  They  are  taught  to  write  using  complete  sentences  and  complete  thoughts,  fix  their 
punctuation  and  grammar,  and  strive  for  polish  in  all  written  work.  That  is  what  English  is  all  about.  On  one  level, 
we  can  be  pleased  with  their  concern.  However,  this  quest  for  excellence  and  perfection  seems  to  interfere  with  the 
poetry  writing  process.  Many  successful  poems  lack  complete  sentences,  defy  traditional  uses  of  punctuation,  and 
contain  word  combinations  that  require  readers  to  make  inferential  connections,  often  sacrificing  the  writing  clarity 
that  EngUsh  teachers  work  so  hard  teach.  We  have  taught  them  to  be  concise,  clear,  and  correct  in  all  writing 
endeavors,  which  in  most  cases,  is  sound  advice.  Writing  poetry,  however,  demands  skills  that  require  us  to  play 
with  words  and  dwell  in  areas  where  word  juxtapositions  and  combinations  are  limited  only  by  one’s  imagination. 
The  many  choices  in  this  "zone”  of  possibility  are  frighteningly  endless,  so  it  is  no  wonder  students  resist  entering. 
They  may  get  lost.  To  help  foster  an  awareness  about  word  choice  and  imagery,  we  developed  an  interactive 
program  to  generate  "found"  poetry  with  student  input  and  used  it  in  a twelve  week  American  literature  class  taught 
by  T.  J.  Reinartz,  one  of  the  authors.  The  program  evolved  from  discussion  between  the  authors  regarding  the  use 
of  computers  to  inspire  poetry  writing.  A multi  modal  qualitative  research  study  was  undertaken  involving 
classroom  observation,  video  taping,  survey  questions  and  analysis  of  poetry  segments.  The  findings  of  that  research 
and  analysis  will  be  presented,  with  broader  observations  of  the  use  of  computers  ’in  the  service’  of  poetry,  and  their 
appropriate  use  in  the  high  school  classroom. 

Students  as  Producers:  Changing  the  Way  we  Teach  and  Learn 

Gail  Ring,  University  of  Florida,  USA;  Sebastian  Foti,  University  of  Florida,  USA;  Melissa  McCallister,  University  of 
Florida,  USA;  Tamara  Pearson,  University  of  Florida,  USA;  Ebraheem  Alkazeemi,  University  of  Florida,  USA 

Educational  technology  courses  at  the  University  of  Florida  are  modeling  the  concept  of  students  as  producers. 
Through  student-based  projects  such  as  an  online  support  center,  educational  software,  student  portfolios,  and 
documentary-type  CD-roms  our  students  are  producing  educational  products.  These  ^real’  activities  are  improving 
our  courses  as  students  learn  by  doing,  thereby  creating  a constructivist  learning  environment.  Relevant,  contextual 
projects  such  as  these  provide  students  with  an  increased  sense  of  empowerment  and  ownership.  In  our 
presentation  we  will  share  how  the  traditional  roles  of  instructor  and  learner  may  no  longer  appropriate. 

The  GPB  project;  A successful  PHD  seminar  using  Videoconference  in  real  time  and  a 
list  to  communicatebetween  the  classes 

Claire  Roberge,  University  of  Quebec  in  Montreal,  Canada 

This  winter  2000,  a seminar  was  offered  at  the  PHD  level  in  the  departement  of  Communication  at  the  University 
of  Quebec  in  Montreal.Three  teachers  participated,  one  from  Montreal,  one  from  Paris  and  one  from  Grenoble. 
Seven  sessions  were  offered  in  videoconference  to  all  students  enrolled.  A website  and  a list  were  added  tools 
offered  to  the  students  to  communicate  between  the  sessions.  The  sessions  in  videoconference  were  in  real  time. 
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Teachers  and  students  made  presentations  and  periods  of  questions  were  planned  at  the  end  of  each  of  those 
presentations. The  experience  which  will  be  renewed  next  winter  was  a success.  Technology  was  used  wisely,  the 
teachers  made  sure  the  environment  available  enhance  the  learning  process  of  their  students.The  differences  in 
cultural  habitus  and  in  the  uses  of  technology  were  taken  in  consideration.An  evaluation  was  made  taking  seriously 
all  factors. 

Multimedia  Training  , Virtual  Instrumentation  and  Remote  Laboratory  a New  Approach  to 
the  Electronic  Courseware 

Nicoletta  Sala,  Academy  of  arch.  University  of  Italian  Switzerland,  Switzerland 

This  work  describes  a project  developed  at  the  Department  of  Electronic  Politecnico  of  Torino  in  the  field  of 
multimedia  technologies  in  educational  process.  The  goal  was  to  allow  the  students  to  carry  out  a pre-training 
activity  outside  the  laboratory  and  possibly  at  home;  each  student  could  thus  individually  adequate  the  learning  rate 
to  his  own  capabilities.  After  this  pre-training  phase,  students  who  enter  the  laboratory  require  reduced  assistance 
and  less  time  to  complete  the  training  activity.  This  approach  can  reduce  the  qualified  assistance  that  is  not  easily 
found.  For  this  reason  several  hypermedia  mod-ules  have  been  developed  (using  Multimedia  Toolbook)  to  assist  the 
students  to  acquire  the  fundamentals  of  the  basic  electronic  instrumentation.  The  subjects  developed  are: 

• fundamentals  of  the  analog  and  digital  oscilloscope,  which  includes  practical  exercises; 

• fundamentals  of  the  analog  and  digital  voltmeters  and  their  operating  theory; 

• the  IEEE488  standard  interface  for  programmable  instrumentation; 

• the  logic  analyzer; 

• the  spectrum  analyzer. 

Virtual  instruments  are  implemented  in  order  to  allow  simple  simulations  of  the  real  instruments  during  the  self- 
training phase  A cUent-server  system  has  been  designed  in  order  to  allow  the  students  to  operate  on  a remote 
laboratory  for  experimental  training.  The  idea  is  to  design  a laboratory  that  is  simultaneously  and  remotely  accessible 
to  several  students,  who  concurrently  share  the  same  instrumentation,  but  not  necessarily  the  same  experiment.  The 
instrumentation  and  the  other  hardware  resources  are  accessible  in  a sort  of  time  sharing  process,  which  is  managed 
by  a server  and  transparent  to  the  user.  The  virtual  laboratory  architecture  is  composed  of  two  kinds  of  subsystems: 
the  measurement  Ser\'^er  and  the  measurement  clients. 

Strategies  for  Teaching  and  Learning:  Lessons  Gained  from  an  Ethnography  of  Graduate 
Classes  in  an  Interactive  Distance  Learning  Studio 

Lorraine  C.  Schmertzing,  University  of  West  Florida,  USA;  Richard  W.  Schmertzing,  Valdosta  State  University,  USA 

This  poster  session  is  based  on  data  collected  during  a one-year  ethnographic  case  study  of  graduate  education 
classes  during  the  inaugural  year  of  a 2- way  audio/2-way  video  interactive  distance  learning  classroom  at  a regional 
university  in  the  southern  United  States.  Cultural  anthropology  and  symbolic  interactionism  acted  as  guide  to 
uncover  and  describe  the  processes  through  which  adult  learners  transformed  traditional  classroom  culture  as  they 
implicitly  redefined  that  culture  during  their  initial  exposure  to  a specific  distance  learning  environment.  This  poster 
session  addresses  the  often  un-addressed  complexities  inherent  in  re-defining  ones  learning  environment  so  that 
effective  learning  can  occur  and  offers  strategies  for  teaching  and  learning  based  on  these  complexities.  Strategies 
include  ideas  for  the  first  night  of  class,  cultural  awareness  activities,  feedback  forums,  and  rules  of  engagement. 

Use  of  Telecommunications  Technology  in  Radiation  Accident  Simulation 

Domenic  Screnci,  Boston  University  School  of  Medicine,  USA;  Kirsten  Levy,  Boston  University  School  of  Medicine, 
USA;  Erwin  Hirsch,  Boston  University  School  of  Medicine,  USA;  Richard  Aghababian,  University  of  Massachusetts 
Medical  School,  USA;  Tracey  Russo,  Boston  University  School  of  Medicine,  USA 

Radiation  accidents  are  rare  but  increasing  in  frequency.  Partners  in  a regional  project  are  providing  training  on 
radiation  accident  preparedness  in  Eastern  Europe  and  the  New  Independent  States.  Recently,  the  partners  devised 
a radiation  accident  exercise  to  simulate  radiation  exposure  and  test  inter-regional  cooperation  among  eight  nations. 
A variety  of  multimedia,  communication  and  distance  learning  technologies  was  used  to  execute  the  exercise.  This 
poster  describes: 

• Objectives 

• Design 

• Procedures 

• Communication  Plan 

• Educational  Technologies 

The  exercise  demonstrates  a live  response  to  a simulated  radiation  accident;  the  ability  to  exchange  information 
among  peers,  request  resources  and  notify  appropriate  agencies;  and  the  use  of  communication  technologies  to 
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debrief  the  experience.  Also  demonstrated  is  the  use  of  distance  learning  technology  to  cultivate  regional 
harmonization  of  response  in  a field  in  which  preparedness  is  essential  but  practical  exercises  lacking. 

Teaching  data  analysis  using  thin  client  technology  for  remote  access 

Robert  Seidman,  San  Diego  State  University,  USA;  K.  Michael  Peddecord,  San  Diego  State  University,  USA 

This  poster  describes  implementation  and  outcomes  of  using  thin  client  technology  to  provide  remote  access  in  a 
data  analysis  course.  AJl  software  applications  run  on  the  course  server,  so  no  downloading  of  data  or  software  is 
required.  Results  from  statistical  analyses  are  transmitted  to  students’  client  machines.  Students  were  able  to  launch 
applications  and  complete  assignments  from  any  location  that  had  Internet  access.  In  addition  to  7/24  access, 
students  were  able  to  use  software  they  did  not  own  and  large  data  files  they  did  not  have  on  their  own  computers. 
While  they  generally  expressed  satisfaction  with  this  remote  access  capability,  drawbacks  included  initial 
programming  time  and  cost  of  the  server  software,  lack  of  familiarity  with  operating  in  a networked  environment, 
frustration  with  server-side  disconnects  or  ISP  interruptions,  and  occasional  inability  to  print  at  home  because  the 
server  software  does  not  include  all  old  printer  drivers. 

Multimedia  Guide  to  Fractal  Geometry 

Vladimir  Shlyk,  Belarusian  State  Pedagogical  University,  Belarus 

Fractal  geometry  has  changed  our  view  of  natural  processes  and  geometric  forms.  The  Guide  is  a software  kit 
supporting  a university  course  on  this  science.  It  contains  an  introductory  electronic  textbook,  annotated  list  of 
publications,  favorite  Internet  links  and  provides  facilities  to  run  programs.  One  program  of  the  project  visualizes 
the  mechanism  of  birth  of  classical  and  IFS  fractals.  A student  can  grow  fractals  by  himself  and  explore  their 
features  by  trying  different  generators  and  mappings.  Another  program  demonstrates  connection  between  the 
Mandelbrot  and  Julia  sets.  It  throws  a bridge  from  fractal  geometry  to  nonlinear  dynamics  and  chaos  theory  by 
showing  different  variants  of  behaviour  inherent  in  underlying  quadratic  process.  Having  advantages  over  analogous 
programs,  it  is  already  being  used  at  several  universities.  To  create  the  most  complete  educational  software  on 
fractal  geometry  we  would  be  pleased  to  collaborate  with  other  specialists  and  enthusiasts. 

Project-Based  Learning  + Multimedia:  Adding  Value  to  Students'  Education 

Michael  Simkins,  Silicon  Valley  Network,  USA;  William  R.  Penuel,  SRI  International,  USA 

’’Does  technology  really  make  a difference  in  student  learning?”  Definitely,  according  to  the  Challenge  2000 
Multimedia  Project,  one  of  the  original  Technology  Innovation  Challenge  Grants  funded  by  the  U.S.  Department  of 
Education.  The  Project  has  demonstrated  significant  educational  results  when  technology  investment  includes 
teacher  training,  technical  support  staffing,  and  curriculum  materials  development.  In  a five-year  evaluation  of  the 
$6.2  million  project  in  California’s  Silicon  Valley,  SRI  evaluators  found  that  technology-using  students  surpassed 
their  non-technology-using  peers  in  developing  some  of  the  most  critical  skills  for  the  new  economy  of  the  21st 
century.  These  skill  advantages  included  better  communication,  teamwork  and  problem-solving.  Researchers  also 
found  important  positive  changes  in  teachers  classroom  practices  that  support  a constructivist  approach  to  learning. 
In  addition,  these  same  students  equaled  their  non-technology-using  peers  in  learning  the  basic  skills  measured  in 
standardized  tests.  In  short,  students  in  technology-using  classes  not  only  ’’got  the  basics,”  they  got  more. 

The  Search  for  the  Skunk  Ape:  An  Information  Literacy  WebQuest 

Charlotte  Slater,  The  Walden  Institute,  USA;  Pamela  Sawallis,  Florida  Gulf  Coast  University,  USA 

A web-based  instructional  resource  for  distance  and  on-campus  learners  uses  elements  of  goal  based  scenarios 
(Schank,  1994)  and  WebQuests  (Dodge,  1997)  to  provide  a ”fun,”  story-based  environment  for  learning 
information  literacy  skills.  This  project  represents  a partnership  between  instructional  technology  and  library 
services  at  a state  university. 

Online  Learning  in  a 3-D  Robotics  Workshop:  The  NASA  ROVer  Ranch 

Stephanie  Smith,  NASA/LinCom,  USA 

Tlie  ROVer  Ranch  is  an  online  environment  based  on  NASA's  mission  as  it  relates  to  robotics  engineering. 
Students  are  situated  where  they  learn  about  robots  and  then  build  and  run  simulated  robots  in  a 3-D  virtual  world. 
The  robotics  workshop  has  all  the  components  to  build  a ’’soft”  version  of  a robot  and  the  tools  needed  to  instruct 
the  robot  to  complete  its  mission.  Student  engineers  or  teams  select  a mission,  review  the  mission  goals  then  design 
a robot  with  the  appropriate  attributes  (power  source,  navigation  planning,  tools,  etc.)  based  on  the  mission 
objectives.  Once  the  robot  is  constructed,  students  plan  a navigation  path  and  then  place  the  robot  in  the  simulation 
to  execute  the  mission.  This  simplified  design  and  programming  exercise  is  an  interactive  method  to  learn  science 
and  math  skills  both  individually  and  in  a team  setting  as  students  share  their  experience  in  journals  and  team 
message  boards. 
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OATS:  Operational  Amplifier  Tutorial  Simulation;  a dynamic  presentation 

Sarah  Sniderman,  Concordia  University,  Canada;  Gary  Boyd,  Concordia  University,  Canada;  Geza  Joos,  Concordia 

University,  Canada 

Abstract:  Operational  Amplifiers  are  among  the  most  important  components  used  in  analog  electronic  devices. 
They  are  complex  and  initially  counter-intuitive  in  behaviour.  Electrical  engineering  and  electrotechnology  students 
frequently  need  special  tutorial  help  to  understand  them  and  configure  them  to  work  properly  in  various  circuits. 
The  main  advantage  of  simulation  programs  for  higher  education  in  technical  fields  is  that,  when  carefully  designed, 
they  can  promote  higher  level  instructional  goals  of  prime  m eta-cognitive  value,  especially  those  involving  the 
autonomous  active  construction  and  exploration  of  knowledge  at  a pace  appropriate  for  the  learners.  Note, 
however,  that  such  free  exploration  of  knowledge  at  a responsive  pace  is  not  a quality  inherent  in  multimedia 
instruction.  High-level  interactivity  involves  more  than  guided  clicking  through  branched  instruction.  A truer 
understanding  requires  a learning  environment  which  accommodates  the  testing,  falsifying,  validating,  and 
elaboration  of  understandings. 

The  Munchhausen-Trick:  Learning  Internet  by  Internet 

Ralph  Sontag,  TU  Chemnitz,  Germany;  Dietrich  Thie,  TU  Chemnitz,  Germany 

The  Chemnitz  University  of  Technology  is  the  first  one  in  Germany,  which  is  able  to  offer  an  internet-based  study 
about  information  and  communication  technology.  The  participants  of  the  courses  are  able  to  collect  fundamental 
and  applicable  knowledge  about  internet-related  technologies  using  it  - a problem  comparable  to  the  baron  of 
Munchhausen,  which  drags  himself  at  his  tuft  out  of  a marsh.  After  two  introducing  lessons  the  communication 
between  learners  and  teachers  is  stricly  internet-based.  Stepwise  the  students  are  qualified  to  understand  and 
maintain  complex  situations  in  management  of  ip-networks  and  servers.  The  knowledge  can  be  proofed  in  a remote 
managed,  but  actually  existing  laboratory.  Moreover  we  had  to  consider  the  very  heterogenous  working  times  of  our 
participants.  We  prefer  asynchronous  communication  with  short  turn-around-times  in  conjunction  with  online-tools 
like  "virtual  seminars",  tests  or  experiments.  The  consequently  platform-independent  project  uses  open  protocols 
and  standardized  document  formats  like  HTML  or  XML. 

In  defence  of  ‘shovelware’:  A powerful  tool  in  staff  development  for  online  teaching  and 

learning 

Heather  Sparrow,  Edith  Cowen  University,  Australia;  Jan  Herrington,  Edith  Cowen  University,  Australia;  Tony 

Herrington,  Edith  Cowen  University,  Australia 

A significant  challenge  for  universities  creating  effective  online  learning  environments  is  educating  and  empowering 
teaching  staff.  Those  who  are  not  confident  using  technolog}'  are  frequently  unconvinced  about  the  benefits  and  are 
concerned  that  it  will  force  unwanted  changes  on  their  current  practices.  This  poster  outlines  a project  which 
introduced  a large  number  of  variously  experienced  academic  staff  to  online  teaching,  whilst  maintaining  or 
enhancing  effective  learning.  The  project  included  the  simple  conversion  of  print-based  external  units  to  an  online 
mode,  as  one  of  several  strategies  of  development.  Whilst  this  process  is  often  referred  to  derogatively  as 
shovelware,  some  of  the  positive  and  immediate  outcomes  achieved  will  be  described,  and  challenges  will  be  made 
to  extremely  negative  attitudes  towards  such  conversions. 

An  Evaluation  of  Video  Compression  Schemes  for  Teleteaching  Applications 

Ognjen  Stanovcic,  Open  Systems  AG  Zurich,  Switzerland;  Thomas  Walter,  Swiss  Federal  Institute  of  Technology 

Zurich,  Switzerland;  Bernhard  Plattner,  Swiss  Federal  Institute  of  Technology  Zurich,  Switzerland 

Synchronous,  interactive  teleteaching  is  demanding  with  respect  to  required  network  bandwidth  and  short  end-to- 
end  delays.  An  uncompressed  digital  video,  e.g.,  requires  a data  rate  of  261  MBit/s.  Video  compression  is  a method 
to  reduce  required  network  bandwidth.  Efficient  video  compression  is  almost  lossless  with  respect  to  perceived 
video  quality  at  the  receiving  site,  but  it  is  time  intensive.  In  support  of  interactivity,  however,  response  times  must 
be  short  to  allow  participants  to  audio- visually  communicate  like  being  in  the  same  room.  This  implies  that  the  end- 
to-end  communication  delay  should  be  less  than  150  ms.  This  requirement  puts  a hard  constraint  on  video 
compression  since  compression  time  is  only  one  factor  that  adds  to  the  overall  end-to-end  delay.  We  present  a 
framework  for  the  evaluation  of  video  compression  schemes.  We  define  evaluation  criteria  such  as  compression 
delay  and  compression  efficiency.  The  video  compression  schemes  evaluated  are  Motion  JPEG,  MPEG,  H.261  and 
H.263. 

Multibook:  making  web  based  learning  resources  personal 

Achim  Steinacker,  TU  Darmstadt,  Germany;  Cornelia  Seeberg,  GMD  IPSi,  Germany;  Ralf  Steinmetz,  GMD IPSI, 

Germany 

Considering  the  amount  of  knowledge  accumulated  on  the  Web,  the  discussion  of  an  effective  use  of  the  web 
resources  becomes  urgent.  The  main  question  is  how  to  find  the  relevant  information  . In  our  web  based  teaching 


o 

ERIC 


1332 


system  Multibook  we  use  modularized  learning  content  stored  in  a knowledge  base  and  describe  these  modules  such 
that  the  combination  of  modules  can  be  matched  to  the  users  preferences,  represented  in  a user  profile.  In  this 
demo  we  will  show  our  knowledge  base  containing  both  the  actual  multimedia  learning  resource  and  the  metadata 
in  form  of  a terminological  ontology,  attributes  for  the  resources  and  relations  between  the  resources.  Then  we 
sketch  the  user  profile  and  finally  we  show  how  these  components  are  matched  together 

Math  and  Science  Curriculum  Revision:  A Collaborative  Approach  to  Improving 
Preservice  Teachers'  Use  of  Technology  Knowledge  and  Instructional  Skills. 

David  Stokes,  Westminster  College,  USA;  Carolyn  Jenkins,  Westminster  College,  USA;  Lorel  Huhnke,  Westminster 
College,  USA;  Gothard  Grey,  Westminster  College,  USA;  Cheryl  Manning,  Bryant  Intermediate  School,  USA 

Reports  on  a Preparing  Tomorrow’s  Teachers  to  use  Technology  Capacity  Grant  project,  a consortium  comprising 
Bryant  Intermediate  School,  and  Westminster  College,  both  of  Salt  Lake  City,  Utah.  The  consortium  participants 
engaged  in  collaborative  curriculum  development  to  provide  preservice  teachers  greater  exposure  to  the 
instructional  uses  of  technology  in  education.  Skills  in  the  application  of  communication,  presentations,  and  CLE 
technologies  were  explored.  Teaching  units  integrating  sound  curriculum  with  technologies  including:  video- 
microscopes, CBL  probes,  graphing  calculators.  Excell,  PowerPoint,  various  software  programs,  and  the  Internet 
were  developed.  Outcomes  suggest  improved  teaching  with  technology  skills,  better  real-world  collaborative  inquiry, 
greater  focus  upon  multiple  learning  styles;  and  gains  in  extending  the  power  of  scientific  inquiry  into  mathematical 
and  scientific  reasoning. 

Integrating  pedagogy,  content  and  technology  into  your  curriculum 

David  Stokes,  Westminster  College,  USA;  Wanda  Carrasquillo-Gomez,  Westminster  College,  USA 

Perhaps  there  are  fewer  more  daunting  tasks  to  teacher  educators  than  to  find  the  time,  means,  and  information  to 
begin  to  more  fully  integrate  technology  as  an  instructional  and  learning  tool  into  their  courses.  Over  the  past  year 
the  authors  conducted  a thorough  curricular  revision  of  technology  courses  offered  in  the  Master  of  Education 
program  at  Westminster  College  in  Salt  Lake  City.  Courses  were  redesigned  focusing  curriculum  upon  technology  as 
a communication  rather  than  a presentation  medium.  Outcomes  surest  this  approach  is  beneficial  to  generating 
increased  understanding  of  the  role  technology  can  play  in  teaching  and  learning,  and  in  strengthening  key  linkages 
between  constructivist  theory  and  teaching  practice. 

SMILE  Maker  • An  Intelligent  Learning  Environment  for  Problem  Solving 

Svetoslav  Stoyanov,  University  of  Twente,  The  Netherlands 

This  poster  introduces  SMILE  Maker  - a web-based  tool  supporting  ill-structured  problem  solving  activities  of 
students  involved  in  problem-based  learning.  SMILE  Maker  stands  for  Solution,  Mapping,  Intelligent  Learning 
Environment.  It  is  both  a problem  solving  and  a learning  tool.  As  a problem  solving  tool  it  proposes  a method  for 
creative  problem  solving  based  on  the  synergy  of  some  mapping  approaches  like  concept  mapping,  mind  mapping, 
cognitive  mapping,  flowscapping,  and  process  mapping,  and  a systematic  creative  problem  solving  methodology. 
The  method  facilitates  elicitation  of  tacit  knowledge,  an  effective  organization  of  the  explicit  knowledge,  and 
avoidance  of  some  negative  individual  problem  solving  syndromes  like  analysis-paralysis,  functional  fixedness, 
premature  judgment,  and  etc.  As  a learning  method  it  provides  an  individualized  learning  environment  for  studying 
and  applying  this  method.  It  adapts  learning  to  four  personal  constructs  such  as  learning  styles,  problem  solving 
styles,  learning  locus  of  control  and  prior  knowledge. 

An  Adaptive  Web-based  Course  in  Financial  Engineering  with  Dynamic  Assessment 

Alberto  Suarez,  Universidad  Autonoma  de  Madrid,  Spain;  Estrella  Pulido,  Universidad  Autonoma  de  Madrid,  Spain; 
Rosa  Carro,  Universidad  Autonoma  de  Madrid,  Spain 

The  increasing  relevance  of  interdisciplinary  skills  calls  for  the  design  and  implementation  of  novel  strategies  to 
address  the  problem  of  teaching  a corpus  of  knowledge  to  a heterogeneous  audience.  We  have  chosen  a problem  in 
financial  engineering,  the  valuation  of  European  options,  whose  mastery  requires  the  interplay  of  mathematical, 
computational  and  financial  tools.  The  target  audience  in  financial  engineering  courses  is  generally  composed 
students  of  either  financial  background  or  from  a scientific,  non-financial  area  (mathematics,  computer  science, 
physics).  The  course  is  designed  within  the  framework  of  the  TANGOW  system 
(http://www.ii.uam.es/esp/investigacion/tangow/present.html),  which  provides  the  necessary  mechanisms  to 
construct  a web-based  course  whose  contents  are  dynamically  generated  according  to  the  student's  profile  and 
actions.  The  system  includes  the  possibility  of  dynamic  assessment,  which  involves  testing  not  only  the  student’s 
ability  to  understand  static  knowledge,  but  also  his/her  ability  to  actively  design  procedures  (programming  analytical 
formulas,  realization  of  simulations)  to  achieve  the  prescribed  goal. 
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Building  Learner  and  Teacher  Autonomy  for  New  Learning  Environments 

Leena  Subra,  University  of  Jyvaskyla,  Finland;  Liisa  Kallio,  University  of  Jyvaskyla,  Finland;  Ulla  Lautiainen, 

University  of  Jyvaskyla,  Finland 

The  poster  reports  on  the  experiences  of  our  departmental  action  research  project  aiming  to  explore  the  use  of  ICT- 
enhanced  methods  in  promoting  both  learner  and  teacher  autonomy/self-directedness  in  discipline-based  language 
learning  and  instruction.  The  action  research  approach  involves  continuous  staff  development,  needs  identification 
and  aims-setting,  teacher  and  learner  self-assessment,  learner  training,  and  development  of  a variety  of  learning  tasks 
suitable  for  self-directed  learning  in  both  authentic  and  virtual  learning  environments.  It  also  involves  an  attempt  of 
creating  a set  of  ICT-enhanced  course  formats  which  - without  significant,  time-consuming  and  technically 
demanding  modifications  - could  be  used  in  teaching  and  learning  languages.  The  poster  describes  pilot  courses 
which  use  different  approaches,  materials  and  programmes,  and  are  aimed  at  different  kinds  of  learners.  They  cover 
a variety  of  possibilities  and  combinations  but  remain  within  the  same  pedagogical  framework,  since  learner  and 
teacher  autonomy  are  the  basic  guidelines  in  all  the  pilot  courses. 

Development  of  a Computer  Aided  Cooperative  Classroom  Environment 

Akira  Suganuma,  Kyushu  University,  Japan;  Ryunosuke  Fujimoto,  Kyushu  University,  Japan;  Yutaka  Tsutsumi, 

Kumamoto  Gakuen  University,  Japan 

In  the  educational  domain,  a popularization  of  computers  and  the  Internet  enable  us  to  hold  lectures  using  Web 
contents  as  a teaching  material.  We  propose  CACCE,  Computer  Aided  Cooperative  Classroom  Environment, 
which  is  an  WWW-based  supporting  system  for  a classroom  teaching.  CACCE  consists  of  a server-software  and  a 
client-software,  that  is  to  say,  a teacher's  browser  and  a student's  browser.  These  browsers  are  connected  together 
by  a socket  connection,  sends  and  receives  some  messages  such  as  a URL  and  the  CACCE's  command.  There  are 
three  main  features  of  CACCE.  First,  CACCE  automatically  refreshes  student's  browsers.  Secondly,  CACCE 
displays  a cooperated  mark  and  text  on  these  browsers.  Finally,  student's  browsers  automatically  report  an  URL  of 
page  shown  by  them  to  teacher's  browser.  This  information  is  used  as  one  of  the  guidelines  for  teaching.  It  is 
concluded  that  CACCE  is  not  only  a tool  that  broadcasts  the  document  on  a teacher's  browser  to  students,  but  also 
a cooperative  environment. 

Using  Knowledge  Models  in  Intelligent  Tutoring  Systems 

Stephen  G.  Taylor,  Champlain  Regional  College,  Canada 

The  Virtual  Tutee  system  will  be  introduced  as  a protot)rpe  of  an  intelligent  tutoring  system.  The  user  of  this  system 
adopts  the  role  of  the  tutor  and  studies  photosynthesis  by  tutoring  a virtual  tutee  residing  within  the  software.  The 
system  was  produced  as  a simulation  based  on  a model  of  the  tutoring  learning  situation  called  the  Knowledge 
Modelling  Approach,  which  will  also  be  introduced.  Fifty  college  students  participated  in  a test  of  the  system. 

An  Evaluation  of  Interactive  Studio  Physics  I 

Holly  Traver,  Rensselaer  Polytechnic  Institute,  USA;  Michael  Kalsher,  Rensselaer  Polytechnic  Institute,  USA;  Karen 

Cummings,  Rensselaer  Polytechnic  Institute,  USA;  Keith  Hmieleski,  Rensselaer  Polytechnic  Institute,  USA 

Rensselaer  Polytechnic  Institute  has  implemented  an  innovative  physics  course  ("Studio  Physics")  that  promotes 
collaborative  interaction.  Students  work  in  groups  of  2-4  on  computers  connected  to  the  local  network  and 
Internet.  Class  time  is  spent  on  short  lecture  and  gtoup  exercises.  This  research  study  evaluated  Studio  Physics  in 
terms  of  grades  and  alumni  responses.  An  examination  of  course  grades  from  students  who  took  the  first  courses  of 
Studio  Physics  yielded  no  significant  difference  between  traditional  lecture  and  studio  courses.  However,  as  the 
studio  course  has  evolved,  learning  has  improved.  Alumni  were  contacted  and  asked  for  their  opinions  on  all  types 
of  courses,  including  studio.  Alumni  indicated  working  coUaboratively  with  others  helped  them  to  improve 
communication,  interpersonal,  and  computer  skills,  as  well  as  the  abilities  to  work  with  others,  learn  new  material, 
and  perform  at  their  current  job.  This  finding  is  important  for  work  organizations,  since  much  of  work  today  is 
group-based. 

Grasp  of  Few  body  element's  correlative  operation  using  Multi  points  measure  data- 

Compair  with  2 kind  selection- 

Atsushi  Tsubokura,  Osaka  Electro-Communiation  Univ.,  Japan;  Kiyoshi  Masui,  Osaka  Electro-Communication  Univ., 

Japan;  Noboru  Ashida,  Osaka  Electro-Communication  Univ.,  Japan;  Kagemasa  Kozuki,  Konami  Co.,  Japan; 

Katsuhide  Tsushima,  Osaka  Electro-Communication  Univ.,  Japan 

Expert  Operator  has  any  skill  for  machine  operation. But  novice  has  a little  skill.The  novice  do  and  error  operate 
when  try  need  to  new  operation  skill. Our  target  is  to  trace  the  Operator's  skill  learning  process  and  the  task 
identificate  for  the  skill  learned.The  newer  operation  skill  based  another  already  learned  operation  skill.These 
Operator's  skil  was  changing  during  Try  Operating.The  changed  operation  can  show  operation  timming  and  used 
skill  for  operation.  We  used  few  body's  measure  data  and  identificate  his  operation  using  symbolic  way.Measure 
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device  is  Eye  Mark  Recorder, 3D  position  or  degress  sencer(for  Arm  Modon),Data  Grove.This  paper  propose  the 
symbolic  way  for  Multi  Points  Measure  data's  correladon  and  single  measure  data's  characterisdc,  and  evaluate  the 
measure  data  using  these  way.This  paper  is  introducdon  of  these  reserch. 

Offline  projects  for  cross-cultural  competence  improvement 

Victoria  Tuziukova,  Rostov  State  Pedagogical  University,  Russia;  Irina  Rozina,  Rostov  State  Pedagogical  University, 
Russia;  Cyndy  Woods,  Arizona  state  University,  USA 

Russian  and  U.S  faculty  members  from  the  Department  of  Foreign  Languages,  Rostov  State  Pedagogical  University 
- RSPU,  Russia  and  Arizona  State  University,  Phoenix,  USA,  developed  off-line  wridng  projects  designed  to 
increase  language  proficiency,  cross-cultural  awareness,  computer  and  networking  skills.  Targedng  EFL  students 
(English  as  a Foreign  Language),  the  project  has  demonstrated  that  students  not  only  increase  English  language 
proficiency,  but  students  learn  more  about  the  way  of  life,  basic  values,  and  cultural  background  of  the  people, 
whose  language  they  study. 

Analyzing  Artists'  Interaction  with  an  Artificial  Reality 

Elpida  Tzafestas,  National  Technical  University  of  Athens,  Greece 

Our  long  term  goal  is  to  understand  how  humans  build  models  of  biological  phenomena.  To  this  end,  we  are 
studying  the  relation  between  an  artificial  reality  and  the  model  of  it  that  an  observer  deduces.  The  study  is  currently 
based  on  the  observation  of  how  Digital  Art  students  interact  with  "PainterAnts",  an  educational  simulation 
software  that  is  used  for  laboratory  work  on  intelligent  and  complex  systems.  The  software  is  a special-purpose  ant 
population  demo  that  tries  to  convey  and  teach  regulation  principles  using  graphical  means.  Artists  have  shown  a 
high  motivation  to  experiment  with  the  system,  partly  because  they  tend  to  regard  it  as  a simple  abstract  art  tool. 
They  manage  to  create  a functional  model  of  complexity,  without  really  understanding  the  underlying  models.  Few 
experiments  with  users  of  different  background,  have  shown  that  humans,  independently  of  background 
knowledge,  tend  to  find  complex  top-down  explanations  to  apparently  complex  phenomena. 

Supporting  Virtual  Classrooms  through  Extranet  technology:  the  Eurydices  system 

Kostas  Bovilas,  University  of  Patras,  Greece;  Aspasia  Kanta,University  of  Patras,  Greece;  Nikos  Piskopos,  University 
of  Patras,  Greece;  Bill  Vassiliadis,  University  of  Patras,  Greece;  Athanassios  Tsakalidis,  University  of  Patras,  Greece; 
John  Tsaknakis,  University  of  Patras,  Greece;  Evangelos  Sakkopoulos,  University  of  Patras,  Greece;  Christos 
Makris,  University  of  Patras,  Greece 

This  work  describes  how  collaborative,  extranet-based  services  can  be  used  to  teach  virtual  classes  to 
geographically-dispersed  academic  participants.  Eurydices  is  an  extranet-based  training  application,  whose  aim  is  the 
efficient  management  of  the  great  variety  of  users  and  courses  provided  by  the  Greek  Universities  Network 
(GUNet). 

A Web-based  educational  tool  for  solving  equations 

Maria  Virvou,  University  of  Piraeus,  Greece;  Maria  Moundridou,  University  of  Piraeus,  Greece;  Michael  Loizos, 
University  of  Piraeus,  Greece;  Athanasios  Papachristou,  University  of  Piraeus,  Greece;  Nikolaos  Polyzotis,  University 
of  Piraeus,  Greece 

This  paper  describes  a web-based  intelligent  authoring  tool  for  constructing  problems  in  the  area  of  equation 
solving.  The  main  objective  of  this  tool  is  to  be  useful  to  teachers  and  students  of  high  school  level  in  the  area  of 
algebraic  equations.  The  teacher  using  this  tool  is  able  to  provide  the  information  needed  to  construct  an  exercise 
through  a user-friendly  interface  on  the  web.  Every  exercise  that  has  been  constructed  is  immediately  available  to 
students.  The  tool  also  monitors  students  closely  while  they  are  solving  the  exercises  through  its  student  modelling 
component.  The  aspects  we  consider  important  in  this  tool  are: 

• The  authoring  capabilities  it  embodies.  Teachers  can  continuously  add  new  exercises  and  be  assisted  at 
their  construction.  Therefore  the  tool  can  be  very  useful  for  the  construction  of  student  assignments 
and/or  exam  tests. 

• It  is  a Web-based  tool.  Therefore  it  can  be  used  as  a support  tool  for  distance  learning.  It  can  also  be  used 
in  classrooms  and/or  home  as  homework,  exam  tests,  student  practice  or  tutorial. 

• The  tool  has  ITS  features  such  as  the  diagnostic  capability.  In  this  sense  the  tool  can  help  the  teacher  to 
assess  his/her  students'  progress  and  level  of  knowledge.  It  can  also  be  very  helpful  at  marking  the 
assignments  and/or  exam  tests.  In  addition  it  can  be  very  useful  to  students  in  two  ways.  First  for  the  self- 
assessment  of  their  skills  and  second  for  the  practice  they  can  have  with  the  provision  of  individualised 
feedback. 
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Constructing  a Learning  Environment  for  Knowledge  Advancement 

Feng-Kwei  Wang,  University  of  Missouri  - Columbia,  USA 

The  purpose  of  this  research  is  to  introduce  the  use  of  technology  to  construct  a learning  environment  for 
knowledge  advancement  through  three  levels  of  knowledge  - know-what,  know-how,  and  know-why,  Know-what 
is  concept-based  domain  content  which  is  prepackaged  and  generalized.  Know-how  translates  know-what  into 
action-oriented  skills.  Know-why  permits  one  to  renew  know-what  and  know-how  and  thus  to  solve  larger  and 
more  complex  problems.  Different  levels  of  knowledge  require  different  learning  methods  and  entail  different 
learning  activities.  The  most  efficient  approach  to  acquiring  know-what  is  through  comprehension  and  feedback. 
The  process-based  know-how  can  be  best  achieved  through  practices  in  simulation  and  case  studies.  Know-why  can 
only  be  built  over  time  through  personal  encounters  with  many  different  problems  and  solutions  in  real-world 
projects.  This  poster  session  will  depict  the  theoretical  grounds  of  this  knowledge  advancement  framework  and 
show  how  technology  is  employed  to  construct  a learning  environment  based  on  this  framework. 

The  Use  of  Cooperative  and  Collaborative  learning  in  a Web-based  Integrated  Curriculum 

for  the  First  Two  Years  of  an  Engineering  Program  of  Study 

John  Watret,  Embry-Riddle  Aeronautical  University,  USA;  Charles  Martin,  Embry-Riddle  Aeronautical  University, 

USA 

The  authors  will  describe  and  present  the  results  of  an  innovative  Integrated  Curriculum  in  Engineering  (ICE) 
designed  for  the  first  two  years  of  an  engineering  program.  Entering  freshmen  engineering  student  volunteers  are 
placed  into  specifically  designed  sections  of  the  foundation  courses  required  of  all  engineering  students:  the  calculus 
sequence,  the  physics  sequence,  the  humanities  sequence,  social  science,  and  introductory  engineering  courses. 
What  differentiates  the  ICE  program  from  traditional  engineering  curricula  is  that  all  courses  incorporate 
cooperative  and  collaborative  learning  and  a reliance  in  computer  and  web  technology.  In  addition,  the  ICE 
program  promotes  team  learning,  team  design  projects,  and  a well-documented  series  of  assessment  practices. 
Although  our  program  was  written  for  an  engineering  curriculum,  the  underl^ang  philosophy  is  applicable  to  a wide 
spectrum  of  programs  of  study  as  well  as  the  general  education  component  of  many  degrees. 

Distance  Learning  For  Government  Agencies  / Online  Academies 

Joseph  Wiiczak,  American  West  Enterprise,  USA 

Government  agencies  are  turning  to  online  presentation  formats  for  computer  based  instruction  related  to  distance 
learning,  CBT  lends  itself  well  to  innovative  visual  curriculum  and  realtime  information  sharing.  The  educational  / 
training  implications  that  combine  computer  technology,  the  internet,  email,  and  telecommunications  are 
enormously  untapped,  CD  Rom,  online  student  / instructor  interaction  through  chat  rooms,  or  both  are  utilized. 
Material  is  presented  in  text  supported  with  voice  overs,  and  graphics.  While  writing  screen  text,  voice  over  scripts 
should  be  written  simultaneously,  and  graphics  last.  Lectures  can  be  conducted  online  supported  by  video 
telecommunications,  A final  online  exam  is  provided  at  the  completion  of  each  course  with  follow  up  sessions 
scheduled  depending  on  course  content.  Presentations  are  suitable  for  newly  enrolled  student  course  material  as  well 
as  recertification  or  ongoing  certification  requirements  of  the  particular  agency. 

The  Management  of  the  Telecommunications  Function:  The  Impact  on  Organizational 

Support,  Planning  and  Training  Quality 

John  Willems,  Eastern  Illinois  University,  USA;  Karen  Ketler,  Eastern  Illinois  University,  USA 

This  article  reports  the  results  of  a survey  of  marketing  managers  in  order  to  obtain  information  from  the  end  user 
about  the  impact  of  the  management  structure  of  the  telecommunications  function  on  1)  the  importance  of  the 
function  within  the  organization,  2)  the  formalization  of  planning  practices  and  3)  the  quality  of  training  on  thirty 
issues.  While  the  information  systems  manager  was  the  most  common  form  of  management  of  the 
telecommunication  function,  the  organizations  with  a distinct  telecommunications  manager  reported  greater 
satisfaction  with  top  management  support  and  a more  formalized  telecommunications  planning  and  implementation 
process  than  the  other  organizations.  The  respondents  from  the  organizations  with  a telecommunications  manager 
also  indicated  better  telecommunications  training  (although  still  somewhat  unsatisfactory)  than  the  other 
respondents.  This  was  especially  true  in  the  managerial  issues  with  a long  term  focus  such  as  the  use  of 
telecommunications  to  gain  a competitive  advantage,  and  managing  innovation  and  technology. 

Online  Learning  Communities:  Vehicles  for  Collaboration  in 

Stephanie  Woolley,  University  of  Colorado  at  Denver,  United  States;  Stacey  Ludwig-Hardman,  University  of  Colorado 

at  Denver,  United  States 

Learning  communities  are  environments  that  encourage  mutual  exchange  between  community  members  to  support 
their  individual  and  collective  learning.  Learning  communities  are  founded  on  the  social  negotiation  of  meaning. 
Collaboration  is  the  key  tenet  of  constructivism  and  small-group  theory  (Springer  Springer,  Stanne,  & Donovan, 
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1999;  Grabinger,  1995;  Duffy  & Jonassen,  1992)  that  explains  the  social  component  of  learning  and  demonstrates 
that  “conceptual  growth  comes  from  sharing  perspectives  and  modifying  our  internal  representations  in  response  to 
that  sharing”  (Grabinger,  1995,  p.  669).  Online  learning  communities  provide  the  means  for  achieving  “shared 
creation”  and  “shared  understanding”  (Shrage,  1991,  p.  40).  They  are  an  excellent  tool  for  integrating  collaboration 
in  online  learning  environments.  The  authors’  poster  demonstration  defines  collaboration  and  learning  communities 
and  addresses  their  theoretical  foundations.  The  authors  will  discuss  the  benefits  of  collaboration  and  learning 
communities  in  online  learning  environments  and  provide  a case  study  of  a new  online  learning  community  at 
Western  Governors  University. 

The  Construction  of  World-Wide-Web  Resource  for  Chinese  Medicine  and  Acupuncture 

Fan  Wu,  China  Medical  College,  Taiwan;  Shao-Fu  Huang,  China  Medical  College,  Taiwan 

As  the  information  technology  makes  great  strides  in  recent  years,  Internet  has  irresistibly  reached  into  every  aspect 
of  our  daily  life.  With  its  unique  capability  of  well-integrating  text,  image  and  video,  World-Wide-Web  has  been 
pervasive  in  the  dissemination  of  medical  discovery,  the  promotion  of  medical  education  and  the  application  of 
clinical  medicine.  The  issue  of  capitalizing  on  the  state-of-the-art  web  technology  to  build  Chinese  medicine  and 
acupuncture  multi-media  web  resource  is  well  addressed  in  this  thesis.  It  is  organized  around: 

1.  The  largest  Chinese  medicine  news  web  site  domestically. 

2.  The  Chinese-medicine-centric  high  resolution  multi-media  Video  on  Demand  in  the  field  of  Chinese 
medicine. 

3.  The  electronic  journal  on  Chinese  medicine  in  the  field  of  Chinese  medicine. 

4.  The  virtual  classroom  with  complete  features  on  Chinese  Medicine  and  group-learning  systems. 

In  addition,  all  the  relevant  web  sites  on  Chinese  medicine  will  be  listed  and  linked  for  the  benefit  of  the  public. 
Integrated  with  Chinese  medicine  database,  this  set  of  medical  resource  helps  to  present  teachers  and  researchers 
with  an  informative  environment  surrounded  by  Chinese  medicine  data.  And  it  gives  students  the  chance  of 
individual  learning  and  the  space  for  the  development  for  voluntary  acquisitiveness  for  knowledge  as  well  as  paves 
the  way  for  academic  exchanges  and  pervasion  of  the  information  on  Chinese  medicine. 

Automatic  Generation  of  the  Optimal  Tutorial-Plan  in  Adaptive  Educational  Hypermedia 
System 

Haiyan  Xu,  University  of  Science  & Technology  of  China,  P.R.China;  XueHai  Zhou,  University  of  Science  & 
Technology  of  China,  P.R.China;  Zhenxi  Zhao,  University  of  Science  & Technology  of  China,  P.R.China 

Adaptive  hypermedia  system  provides  a good  framework  for  web-based  education.  An  ideal  adaptive  educational 
hypermedia  system  adapts  to  different  learners  during  the  whole  period  of  the  study.  Aiming  at  the  automatic 
generation  of  the  adaptive  tutorial-plan,  we  simply  introduce  the  architecture  of  the  ICDAEHS;  an  adaptive 
educational  hypermedia  system  based  on  structural  computing,  propose  the  concept  of  knowledge  structure  graph 
represented  by  AND/OR  graph  and  then  discusses  the  algorithms  used  to  obtain  optimal  tutorial  plan.  Combined 
with  the  pedagogical  strategies  specified  by  the  instructors,  KDAEHS  generates  the  optimal  tutorial-plan  which 
adapts  to  different  learners  based  on  the  knowledge  structure  graph  and  the  learners’  knowledge  level.  Every 
tutorial-plan  is  well  suit  to  the  individual  learner  and  can  help  the  learner  to  fulfill  the  study  goal  efficiently. 

Virtual  Reality  in  astronomy  teaching 

Yoav  Yair,  GET /The  Open  University,  Israel;  Rachel  Mintz,  CET/Tel-Aviv  University,  Israel;  Shai  Litvak,  CET/Tel- 
Aviv  University,  Israel 

A new  3D  model  of  the  solar  system  with  virtual  reality  (VR)  features.  It  is  based  on  powerful  scientific  visualization 
techniques  and  can  be  used  as  an  effective  aide  in  astronomy  teaching  is  presented.  The  model  allows  for  a powerful 
learning  experience,  and  facilitates  the  mental  construction  of  three-dimensional  space,  where  objects  are  varied  and 
different,  but  share  common  features  and  obey  the  same  physical  principles.  The  learner  “enters”  a virtual  model  of 
the  physical  world,  journeys  through  it,  zooms  in  or  out  as  he  wishes,  changes  his  view  point  and  perspective,  as  the 
virtual  world  continues  to  “behave”  and  operate  in  its  usual  manner.  The  new  view  helps  to  overcome  the  inherent 
geocentric  view  and  ensures  the  transition  to  a scientific,  heliocentric  view  of  the  solar  system. 

Uncovering  cognitive  processes  in  discourse  synthesis  using  hypermedia 

Shu  Ching  Yang,  National  Sun  Yat-Sen  University,  Taiwan  ROC 

An  exploratory  study  was  undertaken  to  uncover  the  cognitive  process  of  discourse  synthesis  within  a hypermedia 
learning  environment.  The  focus  was  on  learners'  discourse  synthesis  behavior  and  the  problem  solving  strategies 
they  employed  using  the  Perseus  database.  The  data  for  the  study  was  transcribed  from  tape  recordings  of  the 
'thinking  aloud'  protocols  as  the  learners  attempted  to  work  on  their  tasks.  Ail  protocols  were  transcribed  verbatim 
and  categorized  according  to  each  strategies  content.  A functional  taxonomy  were  developed  to  characterize  the 
cognitive  operations  and  thinking  processes  of  the  learners'  interactions  with  Perseus.  The  proposed  taxonomy 
characterizes  their  cognitive  engagement  within  hypermedia  from  a broader  context.  It  is  a multifaceted  construct 
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which  entails  thick  cognition  with  many  highly  correlated  factors.  The  study  is  concluded  with  practical  and 
theoretical  concerns  emerging  out  of  this  focus  on  discourse  synthesis  within  hypermedia. 

Integrating  Technology  Into  the  Curriculum 

Rosanne  Yost,  University  of  South  Dakota,  USA;  Ray  Thompson,  University  of  South  Dakota,  USA 

This  session  describes  a teacher  education  course  designed  to  help  K-12  educators  integrate  technology  into  their 
curriculums.  The  Technology  for  Training  and  Development  (TTD)  Division  at  the  University  of  South  Dakota  has 
developed  the  course  by  using  the  same  curriculum  planning  process  that  the  course  is  designed  to  teach.  The 
overall  goal  of  TTD  782:  Technology  Integration,  is  to  help  teachers  develop  an  understanding  of  the  potential  for 
technology  to  transform  school  curriculum  so  that  it  is  student  centered  and  future  orientated.  As  the  professors 
have  modeled  the  curriculum  planning  process  they  teach  to  their  students  they  have  themselves  become  better  at 
using  technology.  The  benefits  to  both  teacher  educators  and  classroom  teachers  have  included  classrooms  where 
active  learning  occurs  and  where  students  gain  some  control  over  there  own  learning.  A variety  of  learning  styles 
and  needs  are  now  being  met  as  teachers  and  students  learn  "how  to  learn"  together. 

Using  Web  Resources  to  Enhance  the  Interdisciplinary  Nature  of  Freshman  Preceptorial 

Course 

Kesheng  Yu,  Union  College,  USA 

The  purpose  of  this  poster  is  to  demonstrate  the  design  and  development  of  the  web-based  set  of  materials 
designed  to  help  instructors  become  familiar  with  topics  or  readings  in  which  the  preceptors'  group  has  little 
expertise.  The  focus  of  Freshman  Preceptorial  is  on  cultural  difference  and  on  contemporary  issues  seen  in 
historical  or  philosophical  perspective.  Three  Freshman  Preceptorial  projects  were  funded  in  1998  and  1999.  While 
two  of  these  projects  are  still  in  the  progress,  the  Resources  for  Teaching  the  Bible  are  finished.  This  website  is 
designed  for  faculty  teaching  and  students  taking  Freshman  Preceptorial,  though  others  are  certainly  welcome  to  use 
it.  V^ile  the  demonstration  of  the  design  and  development  of  the  web  resources,  the  effect  of  web  resources  on 
Freshman  Preceptorial  course  and  student  attitude  toward  the  use  of  the  web  resources  would  be  discussed  as  well. 
The  URL  for  the  web  site:  http://www.union.edu/RESOURCES/curriculum/fpbible/ 

Development  of  an  Virtual  University  Online  Course  Using  an  Interprofessional  Approach 

to  Teach  Qualitative  Research 

Cheryl  Zaccagnini,  Shippensburg  University,  US;  Denise  Anderson,  Shippensburg  University,  US;  Kent  Chrisman, 

Shippensburg  University,  US 

Using  an  interprofessional  approach,  a qualitative  research  course  was  designed  and  taught  as  a virtual  university 
course.  Professors  from  the  disciplines  of  early  childhood,  special  education,  and  social  work  developed  a qualitative 
research  online  course  to  utilize  virtual  university  resources  to  reach  graduate  and  undergraduate  students  at  distant 
education  sites.  It  was  developed  to  implement  the  course  as  an  interactive  research  course  with  web-based 
programming  and  video  conferencing  sites  to  enhance  the  quality  of  the  course  and  to  provide  sufficient  web-based 
content  but  most  importantly  to  create  an  atmosphere  of  communication  and  connectedness  with  the  professors 
and  students  at  other  sites.  Syllabi  was  developed  using  a team  teaching  approach  with  access  to  the  professors  via 
video  conferencing,  e-mail,  and  web-based  instruction.  Formative  and  summative  evaluation  results  indicate  that 
teaching  qualitative  research  through  virtual  university  resources  can  be  an  effective  alternative. 

Going  the  DistanceiOffering  Design  Curriculum  in  the  University  of  Utah’s  Distance 

Learning  MFA  in  Directing/Theatre  Education 

David  Zemmels,  University  of  Utah,  USA 

In  the  fall  of  1999,  the  University  of  Utah’s  Department  of  Theatre,  in  collaboration  with  Sundance 
Institute/Sundance  Theatre  Lab  admitted  the  first  cohort  to  its  pilot  distance  learning  MFA  in  Directing/Theatre 
Education.  The  intent  of  this  session  is  to  provide  an  overview  of  the  pilot  program — its  underpinnings,  its 
infrastructure — and  examine  one  solution  to  offering  a theatrical  design  course  within  the  parameters  set  by  the 
program.  Demonstration  and  examples  of  assignments  and  deliver}^  methods  will  help  conference  participants 
envision  ways  to  use  distance  learning  technologies  as  part  of  program  development  at  their  home  sites. 

Merging  Fine  and  Performing  Art  with  Digital  Technology:An  Exploration  of  the 

University  of  Utah’s  Arts  Technology  Certificate  Program 

David  Zemmels,  University  of  Utah,  USA 

Successfully  integrating  traditional  forms  of  artistic  expression  with  digital  technology  is  an  ongoing  challenge  in  the 
fine  and  performing  arts.  In  the  21st  Century,  digital  art  forms  will  likely  become  even  more  important.  Computer 
technology  has  become  a pervasive  and  necessary  aspect  of  the  arts,  whether  you  are  a music  therapist,  art  historian, 
actor,  musician,  designer,  photographer,  or  painter.  This  past  fall,  the  University  of  Utah’s  College  of  Fine  Arts 
began  the  Arts  Technology  Certificate  Program,  an  interdisciplinary  pilot  program  offering  classes  in  digital 
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technolog}^  as  a tool  for  artistic  expression  across  all  art  forms.  The  intent  of  this  session  is  to  provide  an  overview 
of  the  pilot  program  and  the  methods  and  strategies  used  to  implement  it,  and  to  showcase  representative  examples 
of  student  work.  Demonstration  participants  will  also  explore  ways  to  integrate  digital  technology  curriculum  as  part 
of  program  development  at  their  home  sites. 

Comprehensive  Examinations  via  e-mail 

David  Zimmerman,  James  Madison  University,  USA 

This  paper  will  examine  the  electronic  administration  of  the  written  comprehensive  examination  for  the  Masters 
Degree  in  School  Library  Media  Services  from  James  Madison  University.  Prior  to  the  fall  semester,  1998,  all 
students  traveled  to  the  main  campus  in  Harrisonburg,  Virginia  to  take  the  three  hour  exam.  Beginning  in 
November,  1998,  four  students  were  sent  the  exam  via  e-mail  and  returned  their  responses  via  the  same  method. 
The  following  paper  will  include  a comparison  of  the  two  methods  of  administration,  the  results  of  a seven-item 
questionnaire  and  a review  of  the  James  Madison  University  Honor  System. 
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